mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-12 03:48:56 +00:00
6048c84b01
Rebase onto latest main and address review feedback from @lets-order-some-fries: - DualCache resolves kwargs per tier: the in-memory tier gets default_in_memory_ttl, the redis tier gets default_redis_ttl and falls back to the in-memory default when no Redis-specific TTL is set, so the two tiers expire together unless an operator asks for otherwise. An explicit ttl= still reaches both tiers unchanged. The previous version mutated one shared kwargs dict, so the in-memory TTL leaked into the redis tier and the redis tier could not be configured independently. - Those per-tier dict copies carry `# mutable-ok:` reasons so the type-discipline budget in scripts/type-discipline-budget.json stays within its ceiling. - UserApiKeyCache computes effective_redis_ttl = default_redis_ttl if set else default_in_memory_ttl in __init__ and passes that to super().__init__ and key_object_cache, so the auth-cache contract (Redis mirrors in-memory TTL when no explicit Redis TTL is configured) is preserved without special-casing DualCache. - _attach_redis_usage_cache no longer passes the global litellm.default_redis_ttl to user_api_key_cache.attach_redis_cache; the auth cache already mirrors its own in-memory TTL, and overriding it here made the effective TTL depend on YAML key order. - _init_cache reads default_redis_ttl without popping the caller's dict and filters a copy out before constructing Cache, so redis-py Redis() does not receive an unexpected keyword argument and DB reloads do not force a cache rebuild on every poll. - Add regression tests covering per-tier TTL writes and the auth-cache attach path. Refs #43187 Signed-off-by: apex-mochen <2756823972@qq.com> |
||
|---|---|---|
| .. | ||
| __init__.py | ||
| test_azure_blob_cache.py | ||
| test_cache_preset_key.py | ||
| test_caching.py | ||
| test_caching_handler.py | ||
| test_check_and_fix_namespace_none_guard.py | ||
| test_disk_cache.py | ||
| test_dual_cache.py | ||
| test_embedding_router.py | ||
| test_evicted_client_closer.py | ||
| test_gcs_cache.py | ||
| test_in_memory_cache.py | ||
| test_llm_caching_handler.py | ||
| test_llm_client_cache_e2e.py | ||
| test_qdrant_semantic_cache.py | ||
| test_redis_batch.py | ||
| test_redis_cache.py | ||
| test_redis_cluster_cache.py | ||
| test_redis_cluster_node_isolation.py | ||
| test_redis_connection_pool.py | ||
| test_redis_semantic_cache.py | ||
| test_request_redis_batch_post_call.py | ||
| test_request_redis_batch_pre_call.py | ||
| test_responses_stream_cache_keys.py | ||
| test_s3_cache.py | ||
| test_unit_test_caching.py | ||
| test_valkey_semantic_cache.py | ||