Nate Armstrong ee03f153bd feat(gateway): share router cooldowns and follow proxy key invalidations through Redis (#47017) ... Gateway replicas now share state through the coordination Redis the Python proxy uses (general_settings.coordination_redis, a redis response cache, or REDIS_*): - A deployment one replica cools down is written under Python's deployment:<id>:cooldown key and applied by every other replica's background poll (one MGET a second), so the fleet stops sending to a failing deployment together. allowed_fails counts failures across replicas, as Python's v1 rule does; the v2 failure-rate rule stays per replica, as in Python. Deployments are named by model_info.id, or an id derived from the config that is the same on every replica. - The virtual key cache follows the proxy's litellm_proxy.auth_cache_invalidation channel, so a key blocked, deleted or edited through the admin API (or its team or user) is re-read on every replica at once instead of after up to 60 s. general_settings.user_api_key_cache_ttl sets the key TTL. Nothing is added to the path of a successful request. Without Redis, or while it is down, each replica keeps its own state as before. Redis Cluster and Sentinel are refused at startup for now.		2026-10-17 16:34:03 -07:00
..
adrs	docs(rust): add ADRs for the Rust core (#45205)	2026-10-08 10:21:56 -07:00
customer-proxy	feat(gateway): share router cooldowns and follow proxy key invalidations through Redis (#47017)	2026-10-17 16:34:03 -07:00