litellm/tests/unit/router_utils
Kerry Lu e2625d0509
fix(router): re-check and reserve per-model rate limits on fallback targets (#50994)
* fix(router): re-check and reserve per-model rate limits on fallback targets

A provider error fallback handed the caller the fallback model group without
its per-model TPM/RPM limits ever being checked. The router gets a
fallback_rate_limit_check hook next to fallback_budget_check, the proxy wires
it to the v3 limiter, which reserves the target model's per-model descriptors
before each fallback hop and skips targets that are over their limit.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(router): dedupe a repeated fallback target before the reserving rate limit check

Both walks now consult the attempted set before the rate limit check, so a
chain like ["B", "B"] reserves B once. The deprecated settings parity test
also learns the new setting.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(router): a bidirectional fallback chain never re-reserves the requested model group

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(router): match the surfaced failure in the fallback reservation tests

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(rate-limit): keep the admission stash untouched while building fallback descriptors

Building the fallback target's descriptors could rewrite the request stash's
PTU ceiling, which post-call settlement reads for the model the caller named.
The fallback check now hides the stash while it builds, like the frame check.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-11-10 16:36:10 -08:00
..
pre_call_checks fix(router): let an explicit routing tag outrank the encrypted content pin (#50975) 2026-11-10 11:18:30 -08:00
__init__.py test: add __init__.py to intermediate tests/unit packages 2026-09-20 12:55:27 +00:00
test_access_windows.py refactor(router): route the router family through RoutingContext and close the outside surface (#49118) 2026-10-30 19:18:54 -07:00
test_add_retry_fallback_headers.py refactor(router): route the router family through RoutingContext and close the outside surface (#49118) 2026-10-30 19:18:54 -07:00
test_client_initalization_utils.py refactor(router): delete the python router core, the rust core is the only core (#50178) 2026-11-05 10:40:46 -08:00
test_cooldown_cache.py feat(redis): serve the proxy's common Redis call with Rust stores (#50170) 2026-11-06 06:01:11 +00:00
test_cooldown_handlers.py feat(redis): serve the proxy's common Redis call with Rust stores (#50170) 2026-11-06 06:01:11 +00:00
test_deployment_pricing.py refactor(router): route the router family through RoutingContext and close the outside surface (#49118) 2026-10-30 19:18:54 -07:00
test_embedding_batch_size_check.py feat(embeddings): enforce per-request input batch limit (#44574) 2026-10-20 11:35:18 +00:00
test_endpoint_eligibility.py feat(router): opt in to non-chat deployment filtering (#46257) 2026-10-29 08:33:36 -07:00
test_fallback_event_handlers.py fix(router): re-check and reserve per-model rate limits on fallback targets (#50994) 2026-11-10 16:36:10 -08:00
test_get_retry_from_policy.py refactor(router): route the router family through RoutingContext and close the outside surface (#49118) 2026-10-30 19:18:54 -07:00
test_handle_error.py fix(router): restore four base behaviors the RoutingContext refactor changed (#49395) 2026-10-31 22:58:34 +00:00
test_health_check_allowed_fails_integration.py feat(redis): serve the proxy's common Redis call with Rust stores (#50170) 2026-11-06 06:01:11 +00:00
test_health_state_cache.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_kubernetes_pod_discovery.py refactor(router): delete the python router core, the rust core is the only core (#50178) 2026-11-05 10:40:46 -08:00
test_pattern_match_deployments.py refactor(router): route the router family through RoutingContext and close the outside surface (#49118) 2026-10-30 19:18:54 -07:00
test_prompt_caching_cache.py fix(anthropic)!: carry prompt_cache_breakpoint to cache_control on chat completions to Claude (#49783) 2026-11-03 12:56:22 +00:00
test_ptu_shares.py refactor(router): route the router family through RoutingContext and close the outside surface (#49118) 2026-10-30 19:18:54 -07:00
test_reasoning_effort_capability.py refactor(router): route the router family through RoutingContext and close the outside surface (#49118) 2026-10-30 19:18:54 -07:00
test_request_priority.py fix(router)!: schedule default_priority requests once and reject a malformed priority (#47498) 2026-10-20 08:00:57 +00:00
test_router_health_check_routing.py refactor(router): delete the python router core, the rust core is the only core (#50178) 2026-11-05 10:40:46 -08:00
test_router_interactions_endpoints.py refactor(router): delete the python router core, the rust core is the only core (#50178) 2026-11-05 10:40:46 -08:00
test_router_utils_common_utils.py fix(router): drop supports_web_search false deployments for Anthropic web_search_* and Gemini googleSearch tools (#50275) 2026-11-06 01:58:17 +00:00
test_routing_read_batch.py feat(redis): serve the proxy's common Redis call with Rust stores (#50170) 2026-11-06 06:01:11 +00:00