mirror of
https://github.com/BerriAI/litellm.git
synced 2026-11-14 09:03:16 +00:00
68fa7e5aaa
With enable_redis_auth_cache on, everything user_api_key_cache keeps (user-key objects under bare hashed tokens, team/user/alias/org/end-user/project and the other management objects, the global proxy spend scalar) now lives in litellm-redis-store's auth_cache module: both memory partitions plus Redis, behind litellm/redis/stores/auth_cache.py. redis-py never reads or writes the keyspace and UserApiKeyCache.redis_cache stays None; features that only borrow the connection (MCP refresh lock, user provider credential cache) get it from redis_connection. With the flag off the cache stays Python memory as before. Values cross pyo3 as plain data and are written as main's json.dumps (str() for the sync set_cache path), read back with json.loads then literal_eval, with main's TTL rules, pinned by matching inline cases in the Rust and Python tests. The identity and owner prefetches read on the request's native pre-call batch and arm the cooldown read right before, so both leave in one pipeline; write-backs and pre-call SET/DEL ride the same batch. The invalidation publisher and subscriber stay in Python and forward evictions to the native memory tier. The auth cache sends the commands UserApiKeyCache does. Prefetch reads are one MGET, set_many writes the key-object partition and then the rest as two pipelines, delete runs both partitions as concurrent multi-key DELs in chunks of 1000, and increment_if_present is a plain EVAL with the amount written the way redis-py encodes it. |
||
|---|---|---|
| .. | ||
| __init__.py | ||
| _event_loop_reuse.py | ||
| test_azure_blob_cache.py | ||
| test_cache_preset_key.py | ||
| test_caching.py | ||
| test_caching_handler.py | ||
| test_check_and_fix_namespace_none_guard.py | ||
| test_dual_cache.py | ||
| test_embedding_router.py | ||
| test_evicted_client_closer.py | ||
| test_gcs_cache.py | ||
| test_in_memory_cache.py | ||
| test_llm_caching_handler.py | ||
| test_llm_client_cache_e2e.py | ||
| test_qdrant_semantic_cache.py | ||
| test_redis_batch.py | ||
| test_redis_cache.py | ||
| test_redis_cluster_cache.py | ||
| test_redis_cluster_node_isolation.py | ||
| test_redis_connection_pool.py | ||
| test_redis_semantic_cache.py | ||
| test_request_redis_batch_post_call.py | ||
| test_request_redis_batch_pre_call.py | ||
| test_responses_stream_cache_keys.py | ||
| test_s3_cache.py | ||
| test_unit_test_caching.py | ||
| test_valkey_semantic_cache.py | ||