scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-11-13 08:52:02 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 174
litellm/tests/unit/caching
History
Exact
Emerson Gomes f8933ecae4
fix(redis): free each request's Redis batch scope once the request ends (#50751) 
* fix(spend): respect use_redis_transaction_buffer when selecting the SpendLogs ledger

* revert: keep the SpendLogs ledger independent of use_redis_transaction_buffer

Measured on a live proxy, ledger on and ledger off grow RSS at the same rate per request,
so gating the ledger on the Redis buffer moved deployments off it without fixing the growth.

* fix(redis): free each request's Redis batch scope once the request ends

The native request batch keeps a copy of the context it is built in. It was built inside the
request scope, so that context pointed back at the scope, and the cycle collector cannot see
through the native object. Every request's batches, spend counter batches and native Redis
state stayed alive, growing worker RSS about 40 KiB per request and slowing each request as
the heap grew. Build the native batch in the context the scope was opened from instead.
2026-11-10 16:14:12 +00:00
..
__init__.py
_event_loop_reuse.py fix(caching): key cached async clients by a per-loop token instead of id(loop) (#48877) 2026-10-29 13:14:48 -07:00
test_azure_blob_cache.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_cache_preset_key.py
test_caching.py refactor(typing): clear about 1,100 basedpyright Any errors across 303 files (#49986) 2026-11-06 13:31:15 -08:00
test_caching_handler.py fix(caching): don't cache or replay a reply that has no output (#50057) 2026-11-06 21:32:59 -08:00
test_check_and_fix_namespace_none_guard.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_dual_cache.py fix(redis): share one native circuit breaker between RedisCache and the Rust stores (#50473) 2026-11-07 17:50:06 +09:00
test_embedding_router.py refactor(router): move router callers onto the entry points and invert the proxy imports (#48847) 2026-10-29 17:26:33 +00:00
test_evicted_client_closer.py fix(caching): key cached async clients by a per-loop token instead of id(loop) (#48877) 2026-10-29 13:14:48 -07:00
test_gcs_cache.py test: migrate legacy unit dirs and unrun folders into tests/unit, tests/integration and tests/e2e (#45940) 2026-10-11 06:24:57 -07:00
test_in_memory_cache.py feat(router): add deployment rpd and tpd daily limits to enforce_model_rate_limits (#48875) 2026-10-28 15:52:44 -07:00
test_llm_caching_handler.py refactor(model-cost): move the cost map inputs, fetch and register_model overlay into Rust (#50501) 2026-11-10 02:37:35 +00:00
test_llm_client_cache_e2e.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_qdrant_semantic_cache.py refactor(router): move router callers onto the entry points and invert the proxy imports (#48847) 2026-10-29 17:26:33 +00:00
test_redis_batch.py fix(redis): free each request's Redis batch scope once the request ends (#50751) 2026-11-10 16:14:12 +00:00
test_redis_cache.py fix(redis): share one native circuit breaker between RedisCache and the Rust stores (#50473) 2026-11-07 17:50:06 +09:00
test_redis_cluster_cache.py refactor(router): delete the python router core, the rust core is the only core (#50178) 2026-11-05 10:40:46 -08:00
test_redis_cluster_node_isolation.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_redis_connection_pool.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_redis_semantic_cache.py refactor(typing): clear about 1,100 basedpyright Any errors across 303 files (#49986) 2026-11-06 13:31:15 -08:00
test_request_redis_batch_post_call.py fix(proxy): show committed spend in the remaining budget gauges (#49199) 2026-11-09 20:05:37 +00:00
test_request_redis_batch_pre_call.py feat(redis): serve the proxy's common Redis call with Rust stores (#50170) 2026-11-06 06:01:11 +00:00
test_responses_stream_cache_keys.py refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
test_s3_cache.py test: move routing, caching and callback tests out of local_testing (#45761) 2026-10-10 09:33:09 -07:00
test_unit_test_caching.py fix(caching): read the cache key's model group from the router's metadata for the call (#48917) 2026-10-29 05:35:23 -07:00
test_valkey_semantic_cache.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00