litellm/tests/unit/caching
paveln83 d089d524a3 fix(caching): key cached async clients by a per-loop token instead of id(loop) (#48877)
* fix(caching): key cached async clients by a per-loop token instead of id(loop)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(caching): drop deferred closes queued on a collected event loop

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(caching): pin that a close queued on a live loop waits for that loop

* test(caching): run loop-reuse tests until a loop address repeats

The collected-loop tests ran a fixed 50 loops and passed with the fix reverted whenever no loop reused an earlier address. They now stop at the first reuse and skip if none happens, through a shared helper.

* test(caching): cover the cache key fallback for loops without weak references

Also corrects the evicted client closer docstring: a dropped bucket's clients are left to the garbage collector.

* fix(http): free short-lived event loops whose handler finalizer close finished as the loop stopped

* fix(caching): share sync clients across event loops and catch a failed finalizer close

* fix(agentops): stop closing the shared sync http client after the auth call

---------

Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
(cherry picked from commit 4ce724e3df)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-11-05 01:39:40 +00:00
..
__init__.py ci: move caching, proxy-extras, gateway and enterprise tests into tests/unit and run them from litellm-tests (#42902) 2026-09-24 15:49:59 -07:00
_event_loop_reuse.py fix(caching): key cached async clients by a per-loop token instead of id(loop) (#48877) 2026-11-05 01:39:40 +00:00
test_azure_blob_cache.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_cache_preset_key.py ci: move caching, proxy-extras, gateway and enterprise tests into tests/unit and run them from litellm-tests (#42902) 2026-09-24 15:49:59 -07:00
test_caching.py feat(gemini): allow disabling thought summaries while keeping thinking on (#36460) 2026-10-16 09:01:33 +00:00
test_caching_handler.py fix: backport the third batch of rc/1.107.0 CI triage fixes from main (#47174) 2026-10-17 14:21:49 -07:00
test_check_and_fix_namespace_none_guard.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_disk_cache.py test: migrate legacy unit dirs and unrun folders into tests/unit, tests/integration and tests/e2e (#45940) 2026-10-11 06:24:57 -07:00
test_dual_cache.py fix(caching): make Redis Cluster commands wait for a pool slot instead of tripping the circuit breaker (#46488) 2026-10-14 15:27:00 -07:00
test_embedding_router.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_evicted_client_closer.py fix(caching): key cached async clients by a per-loop token instead of id(loop) (#48877) 2026-11-05 01:39:40 +00:00
test_gcs_cache.py test: migrate legacy unit dirs and unrun folders into tests/unit, tests/integration and tests/e2e (#45940) 2026-10-11 06:24:57 -07:00
test_in_memory_cache.py fix(router): keep the prompt caching affinity pin alive as long as the prefix's cache_control ttl (#46084) 2026-10-14 21:10:09 -07:00
test_llm_caching_handler.py fix(caching): key cached async clients by a per-loop token instead of id(loop) (#48877) 2026-11-05 01:39:40 +00:00
test_llm_client_cache_e2e.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_qdrant_semantic_cache.py refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
test_redis_batch.py fix(otel): nest cache spans under their operation and name service spans by purpose (#44150) 2026-10-03 09:20:27 -07:00
test_redis_cache.py test(caching): baseline breaker metrics from live state after gc (#46652) 2026-10-15 05:53:30 -07:00
test_redis_cluster_cache.py fix(caching): make Redis Cluster commands wait for a pool slot instead of tripping the circuit breaker (#46488) 2026-10-14 15:27:00 -07:00
test_redis_cluster_node_isolation.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_redis_connection_pool.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_redis_semantic_cache.py fix(caching): skip the cache past max_messages and keep tool_result text in semantic prompts (#43878) 2026-10-07 17:32:39 +00:00
test_request_redis_batch_post_call.py fix(proxy): backport auth cache freshness and spend updates 2026-11-02 13:58:11 -08:00
test_request_redis_batch_pre_call.py fix: cap cold MCP schema workers at usable CPUs and deflake wall-clock unit tests (#45791) 2026-10-15 00:47:59 -07:00
test_responses_stream_cache_keys.py refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
test_s3_cache.py test: move routing, caching and callback tests out of local_testing (#45761) 2026-10-10 09:33:09 -07:00
test_unit_test_caching.py refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
test_valkey_semantic_cache.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00