`AsyncHTTPHandler` and `HTTPHandler` each closed their httpx client from
`__del__`. Closing an httpx client tears down the connection pool every
in-flight response is streaming through, and it permanently invalidates the
client for later requests. Neither is safe on garbage collection, because a
handler is routinely collected while its client is still in use:
- Streaming. Nothing in a response's reference graph points back at the
handler: httpx binds the response to the transport stream and the pool, never
to the wrapper that built them. A cached handler therefore becomes
collectable mid-stream when its one-hour TTL expires, and every body still
being read off that pool is cut. That is #24929; the window recurs hourly per
process, because the TTL clock starts at first client construction.
- Borrowed clients. `litellm/a2a_protocol/main.py` and
`litellm/proxy/pass_through_endpoints/pass_through_endpoints.py` keep only
`handler.client`. Both take the handler from `get_async_httpx_client`, so the
cache pins it for the same one-hour TTL and then lets it go on eviction, while
the borrowed client is still serving a longer-lived consumer:
`create_a2a_client` hands its client to the a2a SDK and documents it as
"create client once, reuse for multiple requests". The next request on that
client raises `RuntimeError: Cannot send a request, as the client has been
closed.`
- Someone else's client. `litellm/llms/azure/azure.py` wraps
`litellm.client_session` in a throwaway `HTTPHandler` for a single image
generation, and `HTTPHandler.close()` closes whatever client it was handed.
One such call left the user's shared session closed for the rest of the
process.
`LLMClientCache` already documents the invariant these finalizers broke:
evicted clients "may still be in use by in-flight requests", so the cache
deliberately does not close them and leaves them to normal garbage collection.
Closing on collection is precisely what that rules out.
Deleting both finalizers removes the cause instead of narrowing the window.
The async one was never dependable regardless: it needs a running event loop,
so collection outside one silently did nothing. Explicit
`close()`/`aclose()` is untouched, and `close_litellm_async_clients()` still
closes cached async clients at exit.
The cost is that an evicted client is no longer closed eagerly. On the default
aiohttp transport that shows up as five lines of asyncio ERROR output per
evicted handler ("Unclosed client session", "Unclosed connector", and their
context keys) with no change in descriptors or RSS, since aiohttp's own
connector finalizer closes the connections as it warns. On the httpcore
transport there is no log output, but idle keep-alive sockets from evicted
pools are held until a generation-2 collection reclaims them: over 5000
evictions that plateaus at 56 descriptors and ~1 MB of RSS, and with 20
concurrent requests per pool it peaks around 650 descriptors before each gen-2
pass returns it to baseline. Bounded by the GC cycle, not by the eviction
count.
Fixes#24929
3f00e6fa46