mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-29 06:33:28 +00:00
e0d903b971
* fix(proxy): drop router_settings_override before routing and apply it on every branch route_request only popped router_settings_override inside the branch that runs after the api_key/api_base, comma-separated batch and user_config branches. Requests taking one of those earlier branches forwarded the field to the provider, which Anthropic rejects with a 400, and silently lost the key/team num_retries, fallbacks and timeout. The override is now popped and merged into the request before any branch is chosen, keeping the existing precedence where request values win and None values are skipped Fixes #43582 * fix(proxy): apply key/team fallbacks when the request carries api_key or api_base The previous commit skipped fallbacks, context_window_fallbacks and content_policy_fallbacks from router_settings_override whenever the request had api_key or api_base. That broke the scenario from #43582, where a trusted pre-call hook sets api_key and the key/team router_settings still need to apply. All per-request router settings are merged again on every branch, with None values skipped and request values still winning The tests now check that fallbacks run on the api_key and api_base paths and that the provider never receives router_settings_override * fix(proxy): answer 400 for a non-object router_settings_override and treat null as absent * fix(proxy): drop a malformed router_settings_override under drop_params * fix(proxy): route key and team router settings like any request, honor user_config and global drop_params * fix(proxy): keep Router-only override settings out of direct provider calls Calls that route_request sends straight to litellm (evals, endpoints that need no model, pass_through_all_models, the video fallback and the no-router paths) now leave out context_window_fallbacks, content_policy_fallbacks and model_group_retry_policy merged from key or team router settings. Only the Router reads them, and providers reject them as unknown parameters. * test(proxy): drop a comment from the router settings override test * fix(proxy): keep key router settings on the model read-through retry route_request now merges router_settings_override once, before the first attempt. The retry after a model read-through used to pop the merged enable_tag_filtering again and recompute an empty Router-only set, so a model created on another instance was served without the key's tag filtering * test(proxy): check the read-through retry keeps key tag filtering through a real router The regression test now routes through a real Router with two tagged deployments from the database instead of replacing Router.acompletion. Before the fix the retry picks the lower-order untagged-for-request deployment, after it the tagged one * fix(proxy): keep key Router-format fallbacks out of direct provider calls Key and team fallbacks use the Router's model-group mapping. litellm.acompletion reads each mapping as a fallback config, so a direct call retried the same model with the mapping merged into the provider body. The override's fallbacks now stay out of direct calls like the other Router-only settings, while num_retries and timeout still pass --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| _support | ||
| base_sdk_tests | ||
| benchmarks | ||
| code_coverage_tests | ||
| documentation_tests | ||
| e2e | ||
| e2e_harness | ||
| fixtures | ||
| harness_e2e | ||
| integration | ||
| rust-python-harness | ||
| test_litellm_rust | ||
| typing | ||
| unit | ||
| windows_tests | ||
| __init__.py | ||
| _fake_openai_endpoint_server.py | ||
| _flush_vcr_cache.py | ||
| _live_test_helpers.py | ||
| _master_key.py | ||
| _openai_record_replay_proxy.py | ||
| _process_helpers.py | ||
| _vcr_conftest_common.py | ||
| _vcr_redis_persister.py | ||
| _wait_helpers.py | ||
| _ws_vcr.py | ||
| AGENTS.md | ||
| capturing_transport.py | ||
| eval_swe_bench.py | ||
| fake_openai_endpoint.py | ||
| gettysburg.wav | ||
| large_text.py | ||
| openai_batch_completions.jsonl | ||
| pyrightconfig.json | ||
| README.MD | ||
| test_logging.conf | ||
| test_rust_python_harness.py | ||
| VCR_TESTING.md | ||
| white_100x100.png | ||
In total litellm runs 1000+ tests
[02/20/2025] Update:
To make it easier to contribute and map what behavior is tested,
we've started mapping the litellm directory in tests/unit
This folder can only run mock tests.