mirror of
https://github.com/BerriAI/litellm.git
synced 2026-11-16 09:22:22 +00:00
* fix(caching): key cached async clients by a per-loop token instead of id(loop)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(caching): drop deferred closes queued on a collected event loop
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(caching): pin that a close queued on a live loop waits for that loop
* test(caching): run loop-reuse tests until a loop address repeats
The collected-loop tests ran a fixed 50 loops and passed with the fix reverted whenever no loop reused an earlier address. They now stop at the first reuse and skip if none happens, through a shared helper.
* test(caching): cover the cache key fallback for loops without weak references
Also corrects the evicted client closer docstring: a dropped bucket's clients are left to the garbage collector.
* fix(http): free short-lived event loops whose handler finalizer close finished as the loop stopped
* fix(caching): share sync clients across event loops and catch a failed finalizer close
* fix(agentops): stop closing the shared sync http client after the auth call
---------
Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
(cherry picked from commit
|
||
|---|---|---|
| .. | ||
| __init__.py | ||
| _event_loop_reuse.py | ||
| test_azure_blob_cache.py | ||
| test_cache_preset_key.py | ||
| test_caching.py | ||
| test_caching_handler.py | ||
| test_check_and_fix_namespace_none_guard.py | ||
| test_disk_cache.py | ||
| test_dual_cache.py | ||
| test_embedding_router.py | ||
| test_evicted_client_closer.py | ||
| test_gcs_cache.py | ||
| test_in_memory_cache.py | ||
| test_llm_caching_handler.py | ||
| test_llm_client_cache_e2e.py | ||
| test_qdrant_semantic_cache.py | ||
| test_redis_batch.py | ||
| test_redis_cache.py | ||
| test_redis_cluster_cache.py | ||
| test_redis_cluster_node_isolation.py | ||
| test_redis_connection_pool.py | ||
| test_redis_semantic_cache.py | ||
| test_request_redis_batch_post_call.py | ||
| test_request_redis_batch_pre_call.py | ||
| test_responses_stream_cache_keys.py | ||
| test_s3_cache.py | ||
| test_unit_test_caching.py | ||
| test_valkey_semantic_cache.py | ||