mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-13 23:11:40 +00:00
9f97111edd
* feat(fireworks_ai): sync chat completions endpoint with full API surface Add 23 missing request parameters to get_supported_openai_params(): seed, top_logprobs, min_p, typical_p, repetition_penalty, mirostat_target, mirostat_lr, logit_bias, echo, echo_last, ignore_eos, prompt_cache_key, prompt_cache_isolation_key, raw_output, perf_metrics_in_response, return_token_ids, safe_tokenization, service_tier, metadata, speculation, prediction, stream_options, sampling_mask. Also add reasoning_history gated on supports_reasoning. Fix prompt_truncate_length to prompt_truncate_len to match the actual API parameter name. The old name was never in DEFAULT_CHAT_COMPLETION_PARAM_VALUES, so it always went to extra_body and was rejected by Fireworks; it never actually worked. Normalize reasoning_effort boolean values to strings: True becomes "medium", False becomes "none". The Fireworks OpenAPI schema documents these as accepted types, but the server rejects non-string values with HTTP 400 in practice. Integers pass through as-is since the server is expected to validate them. Auto-inject stream_options.include_usage=true when stream=true and the user has not explicitly set stream_options. Without this, Fireworks returns null usage in all streaming chunks, which is inconsistent with the non-streaming behavior where usage is always present. If the user explicitly sets include_usage=false, it is preserved. Capture Fireworks-specific response fields in transform_response(): perf_metrics, prompt_token_ids, raw_output, and token_ids are now extracted from the response and stored in response._hidden_params (fireworks_perf_metrics, fireworks_prompt_token_ids, fireworks_raw_outputs, fireworks_token_ids) so they are accessible to logging, the proxy, and downstream consumers when the corresponding request parameters are enabled. Remove deprecated document inlining logic. Document inlining was deprecated on 2025-06-30 (https://docs.fireworks.ai/updates/changelog#-document-inlining-deprecation). This removes _add_transform_inline_image_block(), the file-to-image_url migration in _transform_messages_helper(), and the disable_add_transform_inline_image_block lookup. Current models that support image input do so natively as VLMs. cache_control, provider_specific_fields, and thinking_blocks stripping is retained. Update get_provider_info() to look up supports_vision and supports_pdf_input from the model cost map instead of hardcoding both to True (which was based on the now-deprecated document inlining). supports_prompt_caching remains True. API docs: https://docs.fireworks.ai/api-reference/post-chatcompletions Reasoning guide: https://docs.fireworks.ai/guides/reasoning Prompt caching: https://docs.fireworks.ai/guides/prompt-caching * fix fireworks chat api surface gaps * Scope Fireworks thinking param to reasoning models * style: fix black formatting * fix(test): update minimax-m3 expected_vision to True * test: cover non-dict content branch in transform_messages_helper * fix(fireworks_ai): remove metadata from supported params to prevent internal metadata disclosure * test(fireworks_ai): replace stale document-inlining capability test The CircleCI-only litellm_utils_tests suite still asserted the old behavior where document inlining made every Fireworks model report supports_pdf_input and supports_vision as True. That premise was removed in this change, so the test now reflects cost-map-driven capabilities: unmapped models no longer advertise vision/PDF support while mapped VLMs like minimax-m3 still do. * test(fireworks_ai): add end-to-end regression for native OpenAI params The existing coverage for the newly supported OpenAI-native params asserted list membership in get_supported_openai_params or called map_openai_params with a hand-built dict, both of which bypass the get_optional_params gate (DEFAULT_CHAT_COMPLETION_PARAM_VALUES). That gate is what previously raised UnsupportedParamsError for seed, top_logprobs, logit_bias, prompt_cache_key, service_tier and prediction when drop_params=False. Assert the full path so a revert of the supported-params additions fails the test instead of passing a shallow membership check. * test(fireworks_ai): fix test isolation in vision/inlining tests Use monkeypatch in test_fireworks_ai_vision_capability_from_cost_map so the LITELLM_LOCAL_MODEL_COST_MAP env var and litellm.model_cost are restored after the test instead of leaking global state into the rest of the process. Switch the document-inlining integration tests off deepseek-v3p1, whose supports_vision is null in the cost map, onto minimax-m3 which is explicitly supports_vision:true. The pass-through assertions no longer depend on a model incidentally not being marked non-vision. * fix(fireworks_ai): gate image rejection on exact vision capability The image_url rejection read supports_vision via _get_model_cost_capability, which falls back to hyphen-boundary substring matching when no exact cost-map entry exists. A custom or fine-tuned model id that merely contains a known non-vision model's short name (e.g. an id ending in -glm-5p2) inherited that entry's supports_vision:false and hard-failed valid image_url blocks on a vision-capable deployment. Split the exact candidate-key lookup into _get_model_cost_capability_exact and use it for the hard rejection so a fuzzy match can never block images; the substring fallback stays a soft signal for capability reporting. Also rewrites the fallback as a comprehension + max instead of an accumulating loop. * feat(fireworks_ai): surface response fields on streaming responses The Fireworks-specific response fields (perf_metrics, prompt_token_ids, per-choice raw_output and token_ids) were only captured into _hidden_params in transform_response, which runs for non-streaming completions; streaming chat went through the default OpenAI chunk handler and dropped them. Add a FireworksAIChatCompletionStreamingHandler that the provider now returns from get_model_response_iterator. It reuses one extraction helper with transform_response and attaches the fields to each streamed chunk's provider_specific_fields, which is the channel litellm preserves when it rebuilds streamed chunks (per-chunk _hidden_params is not carried through). Per-choice token_ids/raw_output ride the content chunks; response-level perf_metrics/prompt_token_ids ride the final usage chunk. Covered by an end-to-end streaming test through litellm.completion(stream=True). --------- Co-authored-by: Ahmad Shahzad <ahmad@shahzad.dev> Co-authored-by: Graham Neubig <398875+neubig@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| fixtures | ||
| realtime | ||
| reasoning_effort_grid | ||
| test-skill | ||
| test_llm_response_utils | ||
| test_skills_data | ||
| base_audio_transcription_unit_tests.py | ||
| base_embedding_unit_tests.py | ||
| base_llm_unit_tests.py | ||
| base_rerank_unit_tests.py | ||
| conftest.py | ||
| dog.wav | ||
| duck.png | ||
| gettysburg.wav | ||
| guinea.png | ||
| log.xt | ||
| Readme.md | ||
| test_a2a.py | ||
| test_anthropic_completion.py | ||
| test_aws_base_llm.py | ||
| test_azure_agents.py | ||
| test_azure_ai.py | ||
| test_azure_o_series.py | ||
| test_azure_openai.py | ||
| test_bedrock_agentcore.py | ||
| test_bedrock_agents.py | ||
| test_bedrock_anthropic_regression.py | ||
| test_bedrock_common_utils.py | ||
| test_bedrock_completion.py | ||
| test_bedrock_dynamic_auth_params_unit_tests.py | ||
| test_bedrock_embedding.py | ||
| test_bedrock_govcloud.py | ||
| test_bedrock_gpt_oss.py | ||
| test_bedrock_invoke_tests.py | ||
| test_bedrock_llama.py | ||
| test_bedrock_mantle.py | ||
| test_bedrock_moonshot.py | ||
| test_bedrock_nova_embedding.py | ||
| test_bedrock_nova_json.py | ||
| test_cloudflare.py | ||
| test_cohere.py | ||
| test_containers_api.py | ||
| test_convert_dict_to_image.py | ||
| test_crusoe.py | ||
| test_databricks.py | ||
| test_deepgram.py | ||
| test_deepseek_completion.py | ||
| test_elevenlabs.py | ||
| test_evals_api.py | ||
| test_fireworks_ai_translation.py | ||
| test_gemini.py | ||
| test_gemini_image_usage.py | ||
| test_gigachat.py | ||
| test_gpt4o_audio.py | ||
| test_groq.py | ||
| test_hosted_vllm_embedding_e2e.py | ||
| test_huggingface_chat_completion.py | ||
| test_hyperbolic.py | ||
| test_infinity.py | ||
| test_jina_ai.py | ||
| test_lambda_ai.py | ||
| test_langgraph.py | ||
| test_litellm_proxy_provider.py | ||
| test_minimax_tts.py | ||
| test_mistral_api.py | ||
| test_model_cost_map_resilience.py | ||
| test_morph.py | ||
| test_nvidia_nim.py | ||
| test_openai.py | ||
| test_openai_o1.py | ||
| test_openai_record_replay_proxy.py | ||
| test_openrouter.py | ||
| test_optional_params.py | ||
| test_perplexity_reasoning.py | ||
| test_prompt_caching.py | ||
| test_prompt_factory.py | ||
| test_replicate.py | ||
| test_rerank.py | ||
| test_router_llm_translation_tests.py | ||
| test_sambanova_chat_transformation.py | ||
| test_skills_api.py | ||
| test_skills_e2e.py | ||
| test_snowflake.py | ||
| test_text_completion.py | ||
| test_text_completion_unit_tests.py | ||
| test_together_ai.py | ||
| test_triton.py | ||
| test_unit_test_bedrock_invoke.py | ||
| test_v0.py | ||
| test_vcr_classification.py | ||
| test_vcr_conftest_common_banner.py | ||
| test_vcr_filters.py | ||
| test_vcr_redis_persister.py | ||
| test_voyage_ai.py | ||
| test_watsonx.py | ||
| test_xai.py | ||
Unit tests for individual LLM providers.
Name of the test file is the name of the LLM provider - e.g. test_openai.py is for OpenAI.
Redis-backed VCR cache
Every test in this directory is auto-decorated with @pytest.mark.vcr (via
conftest.py). The first time a test runs we hit the live provider and
record the HTTP exchange into Redis under
litellm:vcr:cassette:<test_id>. Every subsequent run within 24h replays
from Redis without touching the network. The 24h TTL means each new day's
first run records again, so upstream API drift surfaces within a day.
The persister, header scrubbing, and 2xx-only filtering are defined in
tests/_vcr_redis_persister.py. Files that already use respx (which
patches the same httpx transport vcrpy does) are excluded from the
auto-marker — see _RESPX_CONFLICTING_FILES in conftest.py.
The same VCR cache is used by other test directories that exercise live
provider APIs. The reusable conftest plumbing lives in
tests/_vcr_conftest_common.py and is wired into:
tests/llm_translation/tests/llm_responses_api_testing/tests/audio_tests/tests/batches_tests/tests/guardrails_tests/tests/image_gen_tests/tests/litellm_utils_tests/tests/local_testing/(coverslocal_testing_part1,local_testing_part2,litellm_router_testing,litellm_assistants_api_testing,langfuse_logging_unit_tests)tests/logging_callback_tests/tests/pass_through_unit_tests/tests/router_unit_tests/tests/unified_google_tests/
Test directories that run LiteLLM proxy in Docker (e.g. build_and_test,
proxy_logging_guardrails_model_info_tests, proxy_store_model_in_db_tests)
are intentionally not included: VCR.py patches the in-process httpx
transport, so it cannot intercept the LLM calls that originate inside the
Docker container.
Required environment
CASSETTE_REDIS_URL — separate Redis instance from the application
Redis (REDIS_URL/REDIS_HOST) so test cassettes are not flushed by
proxy tests. Provider credentials (ANTHROPIC_API_KEY, OPENAI_API_KEY,
AWS_*, etc.) are needed only on cache-miss (the daily re-record), not
on replay.
Flushing the cache
When you want the next run to re-record immediately instead of waiting for the 24h TTL:
make test-llm-translation-flush-vcr-cache
Disabling VCR
Skip the cache entirely (every call goes live, no recording):
LITELLM_VCR_DISABLE=1 uv run pytest tests/llm_translation/test_<file>.py