scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-08 22:21:35 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 78
litellm/tests/test_litellm/llms/custom_httpx
History
Exact
Cursor Agent eb1a13f7a7 fix: don't close httpx clients on cache eviction (fixes 'client has been closed' errors) 
Root cause: LLMClientCache._remove_key() was calling aclose()/close() on
httpx clients when their cache entries expired (TTL) or were evicted (LRU).
However, callers of get_async_httpx_client() hold direct references to
these client objects (e.g. litellm.module_level_aclient, streaming handlers,
passthrough request handlers). Closing the underlying httpx client while
these references are still in use causes:

    RuntimeError: Cannot send a request, as the client has been closed.

This was introduced in commits 6931fea9 and bfeed0e5 (Feb 20, 2026) which
added close-on-eviction to prevent connection pool leaks. The fix removes
the close-on-eviction behavior: when a cache entry is evicted, only the
cache's reference is dropped. Python GC will finalize transport/connection
pool resources when all external references are released.

The bug manifests after ~1 hour of traffic (_DEFAULT_TTL_FOR_HTTPX_CLIENTS
= 3600s) when cache entries expire, or earlier under high concurrency if
the 200-entry LRU limit is reached. It affects all providers using the
shared httpx client cache (Vertex AI, Bedrock, etc.).

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
2026-02-26 17:59:20 +00:00
..
test_aiohttp_cleanup_closed.py fix(aiohttp): only set enable_cleanup_closed when required (#21897) 2026-02-23 21:06:29 -08:00
test_aiohttp_handler.py Fix linting 2025-09-07 18:04:00 -07:00
test_aiohttp_transport.py fix(aiohttp): prevent closing shared ClientSession in AiohttpTransport (#21117) 2026-02-16 18:28:28 +05:30
test_client_cache_eviction.py fix: don't close httpx clients on cache eviction (fixes 'client has been closed' errors) 2026-02-26 17:59:20 +00:00
test_gemini_session_leak.py fix: HTTP client memory leaks in Presidio, OpenAI, and Gemini (#19190) 2026-01-19 19:02:55 -08:00
test_http_handler.py Revert "fix(http_handler): bypass cache when shared_session is provided for aiohttp tracing (#20630)" 2026-02-21 12:27:46 -08:00
test_llm_http_handler.py add tests 2025-12-01 14:25:37 -05:00
test_mock_transport.py fix(ollama): thread api_base to get_model_info + graceful fallback (#21970) 2026-02-23 21:00:37 -08:00