mirror of
https://github.com/BerriAI/litellm.git
synced 2026-11-12 08:42:18 +00:00
7356ec9f94
* fix(router): let a retry leave a pinned deployment after a retryable failure The retry loop only recorded a failed deployment for non-retryable statuses, so encrypted_content_affinity re-pinned the failing origin on every 5xx, 429 or connection-error retry. Track every deployment the request failed on and hand that list to the callback filters, leaving selection's skip list alone. Fixes #47607 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(router): carry failed deployments through stream retries too retry_stream started each retry with no failed ids, so an affinity pin on a stream that failed early stayed on the deployment it failed on. Seed the list with the stream's deployment and add each retry that fails. Drop the 429 python case, whose RateLimitError mock cannot be deep-copied by the router. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * style(router): cargo fmt Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(router): leave out deployments that failed the request when affinity strips When the origin had failed this request and no boundary peer existed, the affinity check stripped the reasoning and returned every candidate, the failed origin included. Later retries then carried no marker and re-picked the failed deployment one time in three. Return the untried candidates instead. --------- Co-authored-by: Joshua Valluru <326636767+joshua-berri@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| pre_call_checks | ||
| __init__.py | ||
| test_access_windows.py | ||
| test_add_retry_fallback_headers.py | ||
| test_client_initalization_utils.py | ||
| test_cooldown_cache.py | ||
| test_cooldown_handlers.py | ||
| test_deployment_pricing.py | ||
| test_embedding_batch_size_check.py | ||
| test_endpoint_eligibility.py | ||
| test_fallback_event_handlers.py | ||
| test_get_retry_from_policy.py | ||
| test_handle_error.py | ||
| test_health_check_allowed_fails_integration.py | ||
| test_health_state_cache.py | ||
| test_kubernetes_pod_discovery.py | ||
| test_pattern_match_deployments.py | ||
| test_prompt_caching_cache.py | ||
| test_ptu_shares.py | ||
| test_reasoning_effort_capability.py | ||
| test_request_priority.py | ||
| test_router_health_check_routing.py | ||
| test_router_interactions_endpoints.py | ||
| test_router_utils_common_utils.py | ||
| test_routing_read_batch.py | ||