diff --git a/AGENTS.md b/AGENTS.md index c99d7757582..1ad30d508db 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -209,6 +209,8 @@ When opening issues or pull requests, follow these templates: Using helpers like `supports_reasoning` (which read from `model_prices_and_context_window.json` / `get_model_info`) allows future model updates to "just work" without code changes. +9. **Never close HTTP/SDK clients on cache eviction**: Do not add `close()`, `aclose()`, or `create_task(close_fn())` inside `LLMClientCache._remove_key()` or any cache eviction path. Evicted clients may still be held by in-flight requests; closing them causes `RuntimeError: Cannot send a request, as the client has been closed.` in production after the cache TTL (1 hour) expires. Connection cleanup is handled at shutdown by `close_litellm_async_clients()`. See PR #22247 for the full incident history. + ## HELPFUL RESOURCES - Main documentation: https://docs.litellm.ai/