yassin 75b290969b fix(router): enforce model tpm limits against shared redis usage across replicas ... The model tpm pre-call check read only the in-memory counter, so each proxy replica enforced the limit against its own traffic and the deployment admitted up to N times the configured tpm across N replicas. Read the shared Redis counter when the local counter is under the limit, keep the local counter authoritative when it is already at the limit, and fall back to local usage when Redis is unavailable Supersedes #40854, Fixes #40291 Co-authored-by: Jahanzeb-git <jahanzebahmed2002@gmail.com> Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>		2026-09-19 00:41:55 +00:00
..
pre_call_checks	fix(router): enforce model tpm limits against shared redis usage across replicas	2026-09-19 00:41:55 +00:00
router_callbacks	fix(router): count TPM/RPM usage before building rate-limit headers	2026-09-16 18:45:01 +00:00
add_retry_fallback_headers.py	fix(router): count TPM/RPM usage before building rate-limit headers	2026-09-16 18:45:01 +00:00
auto_router_model_naming.py	feat(router): apply entitlement limits to forecast classifiers	2026-09-15 16:17:00 -07:00
auto_router_tuning_baseline.py	feat(complexity_router): rebalance heuristic weights in the dashboard and grade custom dimensions by match count (#40205)	2026-09-08 13:28:57 -07:00
batch_utils.py	chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom	2026-08-05 02:37:24 -07:00
client_initalization_utils.py	feat(router): reject with 429 when a deployment's max_parallel_requests slots are all in use	2026-09-17 18:14:09 +00:00
clientside_credential_handler.py	feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011)	2026-08-04 12:54:39 -07:00
common_utils.py	chore: merge litellm_internal_staging into litellm_lit_7022_azure_ai_passthrough_config	2026-09-09 16:09:57 -07:00
cooldown_cache.py	fix(router): give cooldowns their own cache so siblings see a bench in ~1s (#40025)	2026-09-08 10:11:20 -07:00
cooldown_callbacks.py	feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011)	2026-08-04 12:54:39 -07:00
cooldown_handlers.py	fix(router): time fallback-hop 408s against now, not the previous hop's end_time	2026-09-15 21:11:40 +00:00
fallback_event_handlers.py	fix: re-check budget on router fallback targets	2026-09-16 19:06:38 +09:00
get_retry_from_policy.py	refactor(router): resolve retry policy by exception MRO and add DefaultRetries	2026-09-04 16:09:01 -07:00
handle_error.py	fix(router): name the all-deployments-in-cooldown error on 429 responses	2026-09-13 09:34:16 +00:00
health_state_cache.py	fix(router): treat a breaker-refused Redis read as a miss in the health state cache	2026-09-10 18:16:59 -07:00
pattern_match_deployments.py	fix(proxy): make the invalid-model 403 path cheap under a burst of rejections (#39892)	2026-09-05 11:53:04 -07:00
prompt_caching_cache.py	refactor(typing): replace Any with proven types in 89 more backend files	2026-09-02 23:08:42 +00:00
reasoning_effort_capability.py	fix(mistral): never round reasoning_effort none onto the strength ladder	2026-09-18 12:22:34 -07:00
response_headers.py	LiteLLM Minor Fixes & Improvements (11/26/2024) (#6913)	2024-11-28 00:01:38 +05:30
routing_groups.py	refactor(router): drop explanatory docstrings from routing group helpers	2026-09-16 01:59:17 +00:00
search_api_router.py	refactor(typing): replace Any with proven types in 65 backend files	2026-09-02 09:11:36 +00:00