mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-14 23:21:35 +00:00
53a2138d05
Adds live coverage for four routing-strategy registry cells (cost, tpm, simple-shuffle weight, latency) and for context-window fallback rerouting. Each routing test builds a model group whose deployments differ in only the dimension the strategy under test reads, drives the strategy per request through router_settings_override, and asserts which deployment served the call from the x-litellm-model-id header. The fallback test overflows a real gpt-4 deployment's context window and asserts the same prompt is served by the gpt-5.5 fallback least_busy.picks_lowest_traffic is deliberately left uncovered. Proving it needs the router's in-flight tally for a deployment to be observable at the moment of the pick, and the proxy exposes no such read: the tally is absent from the in-memory cache dump, prometheus is off, and spend rows only land once a call finishes. Every barrier available instead observes an earlier layer, which would make the assertion a probability bet rather than a proof |
||
|---|---|---|
| .. | ||
| complexity_router_client.py | ||
| conftest.py | ||
| reliability_support.py | ||
| test_complexity_router_e2e.py | ||
| test_reliability_cache_e2e.py | ||
| test_reliability_context_window_fallback_e2e.py | ||
| test_reliability_fallbacks_e2e.py | ||
| test_reliability_routing_e2e.py | ||
| test_reliability_timeouts_e2e.py | ||