mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-13 23:11:40 +00:00
fc5848174e
The read gate cannot cause a wrong pin. A deployment is only pinned when the cache already holds an entry for the prefix, and async_log_success_event writes entries against the deployment's real model rather than the group alias, so a model that will not cache a prefix never records one and there is nothing to pin it to That makes this gate purely a cheap short-circuit deciding whether the cache lookup is worth doing, so the threshold must be the lowest minimum in the group. Taking the highest skipped the lookup for a prefix a lower-minimum member had genuinely cached, losing a hit it earned, and protected against nothing. It also broke the Fable 5 direction this ticket is meant to fix: its real minimum is 512, so a group gate stuck at a higher value would skip the lookup for a prefix Fable 5 had actually cached |
||
|---|---|---|
| .. | ||
| pre_call_checks | ||
| test_add_retry_fallback_headers.py | ||
| test_cooldown_cache.py | ||
| test_fallback_event_handlers.py | ||
| test_health_check_allowed_fails_integration.py | ||
| test_health_state_cache.py | ||
| test_router_health_check_routing.py | ||
| test_router_interactions_endpoints.py | ||
| test_router_utils_common_utils.py | ||