mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-07 08:26:10 +00:00
59da6e75a5
* fix(router): resolve fallbacks against the tier a pre-routing hook selected A complexity or auto router picks a tier behind the router group name, but fallback lookup kept using kwargs["model"], which is still the router name. The tier's configured chain never ran, so a provider failure on its first hop went straight back to the client with "No fallback model group found for original model_group=smart-router". The hook assigns the selected model to a local only, and fallback resolution runs on an outer kwargs dict that **kwargs already copied, so writing it there is not visible. Record the selection in the metadata bucket instead, which is a nested dict shared by reference across those copies and is how the router already carries values back up, then key fallback lookup off it when present. Applies to the generic, context-window, content-policy and weighted-failover lookups. Reporting keeps using the router name, since that is what the caller asked for. Fixes #38832 * fix(router): annotate the recorded-selection helper with a read-only mapping record_pre_routing_selection only reads the request kwargs, writing into the nested metadata bucket it finds there, so Mapping states what it actually needs and clears the LIT001 mutable-annotation budget without a suppression. * test(router): assert the no-kwargs path leaks nothing The tolerated-None case called the helper without checking anything, which the test-quality gate counts as a test with no assertion. Assert that a fresh mapping still reads back empty, so the case proves the call is a no-op rather than only that it does not raise. * fix(router): stop declaring loop-assigned locals Final in the selection helpers Both helpers annotated a loop-assigned local as Final, which reassigns a Final on every iteration and cost three basedpyright errors. Read the buckets through a generator instead, so the write path iterates a for-target and the read path resolves in one shot with next(), which also matches the functional style the type-discipline rules ask for. * style(router): apply ruff format to the selection helpers * fix(router): derive the pre-routing tier fresh on every fallback hop The metadata buckets also carry whatever the caller sent, so an inbound pre_routing_selected_model let a client pick which fallback chain its request fell into. A fallback hop also inherited the previous hop's tier, so the second hop keyed its own failure off the tier that already failed and never ran its own chain. Clear the key at the top of async_function_with_fallbacks. Every hop re-enters there, so only the hook that routed that hop can set it. * fix(router): drop the cast at the fallback-hop clear call site * feat(router): fall back on anthropic safeguard refusals on /v1/messages --------- Co-authored-by: Priyansh Nandwana <nandwana.priyansh103@gmail.com> |
||
|---|---|---|
| .. | ||
| conftest.py | ||
| create_mock_standard_logging_payload.py | ||
| gettysburg.wav | ||
| README.md | ||
| test_completion_no_copy.py | ||
| test_default_deployment_copy.py | ||
| test_get_model_list_alias_optimization.py | ||
| test_pre_call_checks_optimization.py | ||
| test_prompt_management_check.py | ||
| test_router_acancel_batch.py | ||
| test_router_adding_deployments.py | ||
| test_router_anthropic_messages_fallback.py | ||
| test_router_aresponses_streaming_fallback.py | ||
| test_router_batch_utils.py | ||
| test_router_cooldown_per_deployment.py | ||
| test_router_cooldown_utils.py | ||
| test_router_embedding_headers.py | ||
| test_router_embedding_integration.py | ||
| test_router_endpoints.py | ||
| test_router_handle_error.py | ||
| test_router_helper_utils.py | ||
| test_router_index_management.py | ||
| test_router_prompt_caching.py | ||
Router component unit tests.
Please name all files with the word 'router' in them.
This is used to ensure all functions in the router are tested.