mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-13 23:11:40 +00:00
9ec900f964
Two defects combined to make a Redis outage take the proxy down rather than degrade it. First, connection kwargs were dropped whenever Redis was configured by url. _get_redis_url_kwargs built its allowlist from inspect.getfullargspec(redis.Redis.from_url); from_url is declared (cls, url, **kwargs), so the argspec carried no connection kwargs and the function returned ['cls', 'url', 'url']. socket_timeout went with the rest, and socket_connect_timeout falls back to it, so both ended up None and a Redis host that drops packets rather than refusing them blocked callers indefinitely. get_redis_connection_pool's url branch lost the same kwargs by a different route, rebuilding its pool kwargs from scratch. The allowlist now comes from the connection class redis-py actually forwards those kwargs to, walking the MRO because redis-py splits them between AbstractConnection and its subclasses. Deriving it from the client instead would admit client-only settings such as single_connection_client and the SSLConnection-only ssl_* family, which reach AbstractConnection and raise TypeError on first connect. Second, the circuit breaker could not trip even once calls failed fast. _redis_circuit_breaker_guard inferred success from the method returning, but async_get_cache, async_batch_get_cache, async_set_cache, async_set_cache_pipeline, async_set_cache_sadd and async_get_ttl catch their own connection errors and return a default so callers degrade. Each failed call therefore reset the failure streak and the breaker never opened, so an unreachable Redis stayed in the pool and every request kept paying a full socket timeout on it. Those methods now mark the failure and the guard records success only when nothing failed while the method ran. Lua script execution went through none of this, which mattered most because the rate limiter issues all of its Redis traffic that way, so the guard is now a small helper shared by both. The per-call marker is a ContextVar rather than a counter on the breaker. Breakers are shared by every concurrent caller, so a shared counter cannot tell "my call failed" from "some other in-flight call failed", and a success overlapping someone else's failure would be discarded until a Redis that was still answering got evicted from the pool anyway. Only connectivity failures feed the breaker. Command and data errors say nothing about whether Redis is reachable, and counting them would let a caller provoke evictions on demand (an INCR against a non-numeric value, say), dropping rate limiting to per-process counters that spreading traffic across replicas can outrun. |
||
|---|---|---|
| .. | ||
| test_azure_blob_cache.py | ||
| test_caching.py | ||
| test_caching_handler.py | ||
| test_check_and_fix_namespace_none_guard.py | ||
| test_disk_cache.py | ||
| test_dual_cache.py | ||
| test_embedding_router.py | ||
| test_gcs_cache.py | ||
| test_in_memory_cache.py | ||
| test_llm_caching_handler.py | ||
| test_llm_client_cache_e2e.py | ||
| test_qdrant_semantic_cache.py | ||
| test_redis_cache.py | ||
| test_redis_cluster_cache.py | ||
| test_redis_connection_pool.py | ||
| test_redis_semantic_cache.py | ||
| test_s3_cache.py | ||
| test_valkey_semantic_cache.py | ||