mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-08 03:08:45 +00:00
9f97111edd
* feat(fireworks_ai): sync chat completions endpoint with full API surface Add 23 missing request parameters to get_supported_openai_params(): seed, top_logprobs, min_p, typical_p, repetition_penalty, mirostat_target, mirostat_lr, logit_bias, echo, echo_last, ignore_eos, prompt_cache_key, prompt_cache_isolation_key, raw_output, perf_metrics_in_response, return_token_ids, safe_tokenization, service_tier, metadata, speculation, prediction, stream_options, sampling_mask. Also add reasoning_history gated on supports_reasoning. Fix prompt_truncate_length to prompt_truncate_len to match the actual API parameter name. The old name was never in DEFAULT_CHAT_COMPLETION_PARAM_VALUES, so it always went to extra_body and was rejected by Fireworks; it never actually worked. Normalize reasoning_effort boolean values to strings: True becomes "medium", False becomes "none". The Fireworks OpenAPI schema documents these as accepted types, but the server rejects non-string values with HTTP 400 in practice. Integers pass through as-is since the server is expected to validate them. Auto-inject stream_options.include_usage=true when stream=true and the user has not explicitly set stream_options. Without this, Fireworks returns null usage in all streaming chunks, which is inconsistent with the non-streaming behavior where usage is always present. If the user explicitly sets include_usage=false, it is preserved. Capture Fireworks-specific response fields in transform_response(): perf_metrics, prompt_token_ids, raw_output, and token_ids are now extracted from the response and stored in response._hidden_params (fireworks_perf_metrics, fireworks_prompt_token_ids, fireworks_raw_outputs, fireworks_token_ids) so they are accessible to logging, the proxy, and downstream consumers when the corresponding request parameters are enabled. Remove deprecated document inlining logic. Document inlining was deprecated on 2025-06-30 (https://docs.fireworks.ai/updates/changelog#-document-inlining-deprecation). This removes _add_transform_inline_image_block(), the file-to-image_url migration in _transform_messages_helper(), and the disable_add_transform_inline_image_block lookup. Current models that support image input do so natively as VLMs. cache_control, provider_specific_fields, and thinking_blocks stripping is retained. Update get_provider_info() to look up supports_vision and supports_pdf_input from the model cost map instead of hardcoding both to True (which was based on the now-deprecated document inlining). supports_prompt_caching remains True. API docs: https://docs.fireworks.ai/api-reference/post-chatcompletions Reasoning guide: https://docs.fireworks.ai/guides/reasoning Prompt caching: https://docs.fireworks.ai/guides/prompt-caching * fix fireworks chat api surface gaps * Scope Fireworks thinking param to reasoning models * style: fix black formatting * fix(test): update minimax-m3 expected_vision to True * test: cover non-dict content branch in transform_messages_helper * fix(fireworks_ai): remove metadata from supported params to prevent internal metadata disclosure * test(fireworks_ai): replace stale document-inlining capability test The CircleCI-only litellm_utils_tests suite still asserted the old behavior where document inlining made every Fireworks model report supports_pdf_input and supports_vision as True. That premise was removed in this change, so the test now reflects cost-map-driven capabilities: unmapped models no longer advertise vision/PDF support while mapped VLMs like minimax-m3 still do. * test(fireworks_ai): add end-to-end regression for native OpenAI params The existing coverage for the newly supported OpenAI-native params asserted list membership in get_supported_openai_params or called map_openai_params with a hand-built dict, both of which bypass the get_optional_params gate (DEFAULT_CHAT_COMPLETION_PARAM_VALUES). That gate is what previously raised UnsupportedParamsError for seed, top_logprobs, logit_bias, prompt_cache_key, service_tier and prediction when drop_params=False. Assert the full path so a revert of the supported-params additions fails the test instead of passing a shallow membership check. * test(fireworks_ai): fix test isolation in vision/inlining tests Use monkeypatch in test_fireworks_ai_vision_capability_from_cost_map so the LITELLM_LOCAL_MODEL_COST_MAP env var and litellm.model_cost are restored after the test instead of leaking global state into the rest of the process. Switch the document-inlining integration tests off deepseek-v3p1, whose supports_vision is null in the cost map, onto minimax-m3 which is explicitly supports_vision:true. The pass-through assertions no longer depend on a model incidentally not being marked non-vision. * fix(fireworks_ai): gate image rejection on exact vision capability The image_url rejection read supports_vision via _get_model_cost_capability, which falls back to hyphen-boundary substring matching when no exact cost-map entry exists. A custom or fine-tuned model id that merely contains a known non-vision model's short name (e.g. an id ending in -glm-5p2) inherited that entry's supports_vision:false and hard-failed valid image_url blocks on a vision-capable deployment. Split the exact candidate-key lookup into _get_model_cost_capability_exact and use it for the hard rejection so a fuzzy match can never block images; the substring fallback stays a soft signal for capability reporting. Also rewrites the fallback as a comprehension + max instead of an accumulating loop. * feat(fireworks_ai): surface response fields on streaming responses The Fireworks-specific response fields (perf_metrics, prompt_token_ids, per-choice raw_output and token_ids) were only captured into _hidden_params in transform_response, which runs for non-streaming completions; streaming chat went through the default OpenAI chunk handler and dropped them. Add a FireworksAIChatCompletionStreamingHandler that the provider now returns from get_model_response_iterator. It reuses one extraction helper with transform_response and attaches the fields to each streamed chunk's provider_specific_fields, which is the channel litellm preserves when it rebuilds streamed chunks (per-chunk _hidden_params is not carried through). Per-choice token_ids/raw_output ride the content chunks; response-level perf_metrics/prompt_token_ids ride the final usage chunk. Covered by an end-to-end streaming test through litellm.completion(stream=True). --------- Co-authored-by: Ahmad Shahzad <ahmad@shahzad.dev> Co-authored-by: Graham Neubig <398875+neubig@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| agent_tests | ||
| audio_tests | ||
| basic_proxy_startup_tests | ||
| batches_tests | ||
| benchmarks | ||
| code_coverage_tests | ||
| documentation_tests | ||
| enterprise | ||
| guardrails_tests | ||
| image_gen_tests | ||
| integration | ||
| litellm | ||
| litellm-proxy-extras | ||
| litellm_core_utils | ||
| litellm_utils_tests | ||
| llm_responses_api_testing | ||
| llm_translation | ||
| load_tests | ||
| local_testing | ||
| logging_callback_tests | ||
| mcp_tests | ||
| multi_instance_e2e_tests | ||
| ocr_tests | ||
| old_proxy_tests/tests | ||
| openai_endpoints_tests | ||
| otel_tests | ||
| pass_through_tests | ||
| pass_through_unit_tests | ||
| proxy_admin_ui_tests | ||
| proxy_behavior | ||
| proxy_e2e_anthropic_messages_tests | ||
| proxy_migration_tests | ||
| proxy_security_tests | ||
| proxy_unit_tests | ||
| router_unit_tests | ||
| scim_tests | ||
| search_tests | ||
| spend_tracking_tests | ||
| store_model_in_db_tests | ||
| test_litellm | ||
| unified_google_tests | ||
| vector_store_tests | ||
| windows_tests | ||
| __init__.py | ||
| _fake_openai_endpoint_server.py | ||
| _flush_vcr_cache.py | ||
| _live_test_helpers.py | ||
| _openai_record_replay_proxy.py | ||
| _vcr_conftest_common.py | ||
| _vcr_redis_persister.py | ||
| eval_swe_bench.py | ||
| fake_openai_endpoint.py | ||
| gettysburg.wav | ||
| large_text.py | ||
| openai_batch_completions.jsonl | ||
| README.MD | ||
| test_anthropic_compaction_usage.py | ||
| test_budget_management.py | ||
| test_callbacks_on_proxy.py | ||
| test_config.py | ||
| test_debug_warning.py | ||
| test_default_encoding_non_root.py | ||
| test_end_users.py | ||
| test_entrypoint.py | ||
| test_fallbacks.py | ||
| test_gpt5_azure_temperature_support.py | ||
| test_health.py | ||
| test_keys.py | ||
| test_litellm_proxy_responses_config.py | ||
| test_logging.conf | ||
| test_models.py | ||
| test_new_vector_store_endpoints.py | ||
| test_openai_endpoints.py | ||
| test_organizations.py | ||
| test_otel_thread_leak.py | ||
| test_passthrough_endpoints.py | ||
| test_presidio_latency.py | ||
| test_proxy_server_non_root.py | ||
| test_ratelimit.py | ||
| test_resource_cleanup.py | ||
| test_service_logger_otel.py | ||
| test_spend_logs.py | ||
| test_team.py | ||
| test_team_logging.py | ||
| test_team_members.py | ||
| test_users.py | ||
In total litellm runs 1000+ tests
[02/20/2025] Update:
To make it easier to contribute and map what behavior is tested,
we've started mapping the litellm directory in tests/test_litellm
This folder can only run mock tests.