litellm/tests/unit/test_router
Kerry Lu e2625d0509
fix(router): re-check and reserve per-model rate limits on fallback targets (#50994)
* fix(router): re-check and reserve per-model rate limits on fallback targets

A provider error fallback handed the caller the fallback model group without
its per-model TPM/RPM limits ever being checked. The router gets a
fallback_rate_limit_check hook next to fallback_budget_check, the proxy wires
it to the v3 limiter, which reserves the target model's per-model descriptors
before each fallback hop and skips targets that are over their limit.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(router): dedupe a repeated fallback target before the reserving rate limit check

Both walks now consult the attempted set before the rate limit check, so a
chain like ["B", "B"] reserves B once. The deprecated settings parity test
also learns the new setting.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(router): a bidirectional fallback chain never re-reserves the requested model group

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(router): match the surfaced failure in the fallback reservation tests

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(rate-limit): keep the admission stash untouched while building fallback descriptors

Building the fallback target's descriptors could rewrite the request stash's
PTU ceiling, which post-call settlement reads for the model the caller named.
The fallback check now hides the stash while it builds, like the frame check.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-11-10 16:36:10 -08:00
..
core fix(router): re-check and reserve per-model rate limits on fallback targets (#50994) 2026-11-10 16:36:10 -08:00
__init__.py test: add __init__.py to every tests/unit directory phase 16 touches 2026-09-20 11:52:38 +00:00
test_enforce_model_rate_limits.py feat(router): daily_reset_time picks when deployment rpd and tpd daily limits start over (#48957) 2026-10-28 20:20:01 -07:00
test_io_token_rate_limits.py fix(router): a cancelled or retried attempt releases its io-token reservation and least-busy count (#50486) 2026-11-07 16:54:26 -08:00
test_router.py refactor(model-cost): move the cost map inputs, fetch and register_model overlay into Rust (#50501) 2026-11-10 02:37:35 +00:00
test_router_callback_hook_sequence.py test: replace 61 live logging and otel tests with offline unit and integration coverage (#45362) 2026-10-08 15:55:57 -07:00
test_router_endpoints.py refactor(router): delete the python router core, the rust core is the only core (#50178) 2026-11-05 10:40:46 -08:00
test_router_entry_points.py feat(proxy): mixed Codex model picker and LiteLLM routing on native codex routes (#49408) 2026-11-06 11:14:47 -08:00
test_router_facade.py refactor(router): delete the python router core, the rust core is the only core (#50178) 2026-11-05 10:40:46 -08:00
test_router_fallbacks.py refactor(router): delete the python router core, the rust core is the only core (#50178) 2026-11-05 10:40:46 -08:00
test_router_fuzz_regressions_rust_only.py refactor(router): delete the python router core, the rust core is the only core (#50178) 2026-11-05 10:40:46 -08:00
test_router_get_deployments.py refactor(router): delete the python router core, the rust core is the only core (#50178) 2026-11-05 10:40:46 -08:00
test_router_helper_utils.py fix(router): count a Responses WebSocket turn's tokens before the next turn is admitted (#50730) 2026-11-10 04:13:51 +00:00
test_router_host.py refactor(router): move router callers onto the entry points and invert the proxy imports (#48847) 2026-10-29 17:26:33 +00:00
test_router_max_parallel_requests.py refactor(router): delete the python router core, the rust core is the only core (#50178) 2026-11-05 10:40:46 -08:00
test_router_provider_endpoints.py feat(redis): serve the proxy's common Redis call with Rust stores (#50170) 2026-11-06 06:01:11 +00:00
test_router_retries.py refactor(router): delete the python router core, the rust core is the only core (#50178) 2026-11-05 10:40:46 -08:00
test_router_surface_check.py refactor(router): fence the router surface and document the router-core boundary (#48839) 2026-10-29 00:01:52 +00:00
test_router_timeout.py refactor(router): delete the python router core, the rust core is the only core (#50178) 2026-11-05 10:40:46 -08:00
test_router_vector_store_endpoints.py test: migrate SDK and proxy-booting legacy CircleCI tests to their tiers (#45758) 2026-10-11 00:17:21 -07:00