scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-08-28 05:25:59 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 9
litellm/tests/test_litellm/caching
History
Exact
michelligabriele 8dc0c97958
fix(caching): check REDIS_CLUSTER_NODES env var in Cache and Router class selection (#22790) 
When Redis Cluster is configured via the REDIS_CLUSTER_NODES environment
variable, Cache.__init__() and Router._create_redis_cache() ignored the
env var and always created RedisCache instead of RedisClusterCache. This
caused the v3 rate limiter's cluster detection (_is_redis_cluster()) to
return False, skipping hash-slot key grouping. The resulting CROSSLOT
errors were silently caught, falling back to per-instance in-memory
counting — breaking RPM/TPM enforcement across multiple proxy instances.

Add REDIS_CLUSTER_NODES env var detection to both Cache.__init__() and
Router._create_redis_cache(), matching the existing pattern in
_redis.py:215-220. When the env var is set and no explicit startup_nodes
parameter is provided, parse it and create RedisClusterCache.

Fixes #22748
Related to #20836
2026-03-06 17:31:30 -08:00
..
test_azure_blob_cache.py add azure blob cache support (#12587) 2025-07-15 11:47:38 -07:00
test_caching_handler.py Helicone base url support + fix for embedding cache hits on str input (#11211) 2025-05-28 22:02:55 -07:00
test_dual_cache.py fix(cache): prevent DualCache async batch check-then-act race (#20986) 2026-02-13 18:32:41 +05:30
test_gcs_cache.py Add GCS bucket caching support (#13122) 2025-08-04 16:09:33 -07:00
test_in_memory_cache.py [Performance] Reduce complexity of InMemoryCache.evict_cache from O(n*log(n)) to O(log(n)) (#15000) 2025-09-30 16:49:35 -07:00
test_llm_caching_handler.py fix: don't close HTTP/SDK clients on LLMClientCache eviction (#22925) 2026-03-05 12:00:38 -08:00
test_llm_client_cache_e2e.py fix: don't close HTTP/SDK clients on LLMClientCache eviction (#22925) 2026-03-05 12:00:38 -08:00
test_qdrant_semantic_cache.py feat(semantic-cache): support configurable vector dimensions for Qdrant (#21649) 2026-02-21 00:51:15 -08:00
test_redis_cache.py fix(proxy): add LPOP pipeline error checking and fix org spend ServiceType 2026-02-24 14:22:57 -08:00
test_redis_cluster_cache.py fix(caching): check REDIS_CLUSTER_NODES env var in Cache and Router class selection (#22790) 2026-03-06 17:31:30 -08:00
test_redis_connection_pool.py Revert "Add LLMClientCache regression tests for httpx client eviction safety" 2026-02-27 12:43:03 -08:00
test_redis_semantic_cache.py Litellm fix GitHub action testing (#11163) 2025-05-26 14:41:42 -07:00
test_s3_cache.py verify expires field prior to serving cache entry 2025-08-25 10:42:12 +02:00