mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-07 02:59:05 +00:00
96f58fac53
* fix(router): don't cool down parent deployment on advisor sub-call failure Advisor orchestration issues a sub-call to a different provider/credentials than the selected deployment. When that sub-call fails (e.g. a 401 because no advisor API key is configured), the exception propagates up and the router's deployment_callback_on_failure attributes it to the healthy parent deployment's model_info.id, cooling it down and rejecting unrelated callers to the same model group. Tag advisor sub-call failures on the exception and skip cooldown for them in deployment_callback_on_failure. The exception is tagged rather than wrapped so its type is preserved and retry/fallback classification and the client-facing error are unchanged. Genuine executor/deployment failures are untagged and still cool down as before. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(router): tag advisor orchestration failures via provider-neutral util Address review on LIT-4565: move the cooldown-exemption marker into litellm/router_utils/cooldown_handlers.py so the router imports it at module top instead of an in-function anthropic import, and extend the exemption to AdvisorMaxIterationsError so a max-iterations orchestration failure no longer cools down the healthy executor deployment. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: shivam <shivam@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| pre_call_checks | ||
| router_callbacks | ||
| add_retry_fallback_headers.py | ||
| batch_utils.py | ||
| client_initalization_utils.py | ||
| clientside_credential_handler.py | ||
| common_utils.py | ||
| cooldown_cache.py | ||
| cooldown_callbacks.py | ||
| cooldown_handlers.py | ||
| fallback_event_handlers.py | ||
| get_retry_from_policy.py | ||
| handle_error.py | ||
| health_state_cache.py | ||
| pattern_match_deployments.py | ||
| prompt_caching_cache.py | ||
| response_headers.py | ||
| search_api_router.py | ||