From eb03abd9e60e661269bdda88abb2186e8154da0d Mon Sep 17 00:00:00 2001 From: Ishaan Jaffer Date: Thu, 5 Mar 2026 11:57:23 -0800 Subject: [PATCH] =?UTF-8?q?docs(AGENTS.md):=20add=20rule=20=E2=80=94=20nev?= =?UTF-8?q?er=20close=20HTTP/SDK=20clients=20on=20cache=20eviction?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- AGENTS.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/AGENTS.md b/AGENTS.md index c99d7757582..1ad30d508db 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -209,6 +209,8 @@ When opening issues or pull requests, follow these templates: Using helpers like `supports_reasoning` (which read from `model_prices_and_context_window.json` / `get_model_info`) allows future model updates to "just work" without code changes. +9. **Never close HTTP/SDK clients on cache eviction**: Do not add `close()`, `aclose()`, or `create_task(close_fn())` inside `LLMClientCache._remove_key()` or any cache eviction path. Evicted clients may still be held by in-flight requests; closing them causes `RuntimeError: Cannot send a request, as the client has been closed.` in production after the cache TTL (1 hour) expires. Connection cleanup is handled at shutdown by `close_litellm_async_clients()`. See PR #22247 for the full incident history. + ## HELPFUL RESOURCES - Main documentation: https://docs.litellm.ai/