scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-08-28 05:25:59 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 8
litellm/tests/test_litellm/llms
History
Exact
Cursor Agent eb1a13f7a7 fix: don't close httpx clients on cache eviction (fixes 'client has been closed' errors) 
Root cause: LLMClientCache._remove_key() was calling aclose()/close() on
httpx clients when their cache entries expired (TTL) or were evicted (LRU).
However, callers of get_async_httpx_client() hold direct references to
these client objects (e.g. litellm.module_level_aclient, streaming handlers,
passthrough request handlers). Closing the underlying httpx client while
these references are still in use causes:

    RuntimeError: Cannot send a request, as the client has been closed.

This was introduced in commits 6931fea9 and bfeed0e5 (Feb 20, 2026) which
added close-on-eviction to prevent connection pool leaks. The fix removes
the close-on-eviction behavior: when a cache entry is evicted, only the
cache's reference is dropped. Python GC will finalize transport/connection
pool resources when all external references are released.

The bug manifests after ~1 hour of traffic (_DEFAULT_TTL_FOR_HTTPX_CLIENTS
= 3600s) when cache entries expire, or earlier under high concurrency if
the 200-entry LRU limit is reached. It affects all providers using the
shared httpx client cache (Vertex AI, Bedrock, etc.).

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
2026-02-26 17:59:20 +00:00
..
amazon_nova/chat [New Model] Add Amazon Nova as first party provider for chat completions (#17351) 2025-12-06 14:43:55 -08:00
anthropic fix(adapter): populate cache_read_input_tokens from prompt_tokens_details for OpenAI/Azure (#22090) 2026-02-26 00:10:18 -08:00
azure fix llm tests 2026-02-17 20:13:23 +05:30
azure_ai fix: support Azure AD token auth for non-Claude azure_ai models (#20981) 2026-02-11 10:48:44 -08:00
baseten/chat add testing and remove redundant function 2025-08-21 11:25:26 -07:00
bedrock fix(bedrock): pass timeout param to bedrock rerank http client (#22021) 2026-02-24 09:32:11 -08:00
bytez/chat Integration: Bytez as a model provider (#12121) 2025-07-12 10:50:39 -07:00
chat Litellm fix GitHub action testing (#11163) 2025-05-26 14:41:42 -07:00
chatgpt fix(proxy): fix master key rotation Prisma validation errors (#21330) 2026-02-16 15:13:05 -08:00
cohere fix(unified_guardrail.py): support during_call event type for unified guardrails (#17514) 2025-12-04 22:06:13 -08:00
cometapi/chat feat: add CometAPI provider support with chat completions and streaming (#13458) 2025-08-11 18:06:37 -07:00
compactifai Fix CompactifAI provider tests and implementation 2025-09-15 22:03:42 +02:00
custom_httpx fix: don't close httpx clients on cache eviction (fixes 'client has been closed' errors) 2026-02-26 17:59:20 +00:00
dashscope fix: remove list-to-str transformation from dashscope 2026-02-19 07:30:13 +00:00
databricks Address Greptile review: fix SDK auth fallback and remove unused imports 2026-02-18 15:57:50 +09:00
datarobot Updated URL handling for DataRobot provider base 2025-08-21 19:42:46 -06:00
deepgram fix: return the diarized transcript when it's required in the request (#16133) 2025-11-02 09:45:18 -08:00
deepinfra Fix: tool content should be str 2026-01-07 11:01:31 +05:30
docker_model_runner fix docker model runner tests 2025-11-22 11:07:56 -08:00
featherless_ai/chat test: fixes 2025-05-31 12:42:56 -07:00
fireworks_ai Fix: Extra inputs are not permitted, field: 'messages[2].provider_specific_fields' 2026-02-03 16:23:18 +05:30
gemini Add get files API support and tests 2026-02-03 18:57:39 +05:30
github_copilot fix(github_copilot): preserve encrypted_content in reasoning items for multi-turn conversations (#17130) 2025-12-05 12:42:25 -08:00
heroku adds tool calling test 2025-08-06 14:23:55 -06:00
hosted_vllm Fix None (TypeError: 'NoneType' object is not a mapping) 2026-02-25 14:39:12 +05:30
huggingface fix: improve test isolation for parallel execution 2026-02-17 21:28:23 -03:00
jina_ai/embedding feat(JinaAI): support multimodal embedding models (#13181) 2025-08-05 19:21:56 -07:00
lemonade Adding unit tests and documentation 2025-09-30 12:12:24 -06:00
litellm_proxy/chat LiteLLM SDK <-> Proxy: support user param + Prisma - remove use_prisma_migrate flag - redundant as this is now default (#13555) 2025-08-12 22:03:39 -07:00
llamafile/chat test llamafile 2025-07-03 22:35:22 -07:00
lm_studio fix(lm_studio): resolve illegal Bearer header value issue 2025-09-12 22:41:30 +02:00
manus [Feat] New provider - Manus API on /responses, GET /responses (#18804) 2026-01-08 18:37:42 +05:30
meta_llama fix: add missing OpenAI chat completion params to OPENAI_CHAT_COMPLETION_PARAMS (#21360) 2026-02-16 20:31:21 -08:00
minimax Add tests for chat completion minmax 2025-12-23 17:10:46 +05:30
mistral Added thinking streaming support for mistral (#16434) 2025-11-10 18:41:45 -08:00
moonshot [Feat] New LLM API Integration - Add Moonshot API (Kimi) (#12551) (#12592) 2025-07-14 15:23:34 -07:00
nebius test: fixes 2025-05-31 12:42:56 -07:00
novita/chat Litellm fix GitHub action testing (#11163) 2025-05-26 14:41:42 -07:00
nscale/chat Litellm fix GitHub action testing (#11163) 2025-05-26 14:41:42 -07:00
oci/chat Fixes #20957 2026-02-11 11:20:18 +00:00
ollama fix(ollama): thread api_base to get_model_info + graceful fallback (#21970) 2026-02-23 21:00:37 -08:00
openai fix: add prompt_cache_key and prompt_cache_retention support for OpenAI 2026-02-24 16:42:54 -08:00
openai_like fix(proxy): fix master key rotation Prisma validation errors (#21330) 2026-02-16 15:13:05 -08:00
openrouter Add tests for openrouter 2026-01-14 10:17:01 +05:30
ovhcloud Add support of audio transcription for OVHcloud (#17305) 2025-12-01 18:26:39 -08:00
perplexity format 2026-02-23 18:05:07 +00:00
pg_vector/vector_stores [Bug Fix] QA - Use PG Vector Vector Store with LiteLLM (#12716) 2025-07-18 08:41:18 -07:00
publicai fix: add missing OpenAI chat completion params to OPENAI_CHAT_COMPLETION_PARAMS (#21360) 2026-02-16 20:31:21 -08:00
ragflow/chat fix failing vertex tests 2025-12-06 00:04:04 +05:30
recraft [Feat] Add Recraft API - Image Edits Support (#12874) 2025-07-22 15:03:08 -07:00
runwayml [Feat] RunwayML - Add support for /audio/speech eleven_multilingual_v2 endpoint (#16604) 2025-11-13 14:32:09 -08:00
s3_vectors [Feat] RAG API - Add s3_vectors as provider on /vector_store/search API + UI for creating + PDF support for /rag/ingest (#19895) 2026-01-27 16:30:59 -08:00
sagemaker fix(sagemaker): Support TEI raw array response format for embeddings (#20487) 2026-02-12 19:39:05 +05:30
sambanova Feat/sambanova embeddings (#13308) 2025-08-12 17:15:26 -07:00
sap Add header for SAP AI Core Tracking (#18714) 2026-01-06 23:59:02 +05:30
snowflake Snowflake provider support: added embeddings, PAT, account_id (#15727) 2025-11-17 20:27:46 -08:00
stability feat(stability): add Stability AI image generation support (#17894) 2025-12-14 08:29:45 +05:30
vercel_ai_gateway feat(vercel_ai_gateway): add embeddings support 2026-01-23 15:11:37 -03:00
vertex_ai Merge pull request #22170 from BerriAI/litellm_fix_video_veo_vertex 2026-02-26 18:55:25 +05:30
volcengine test: Fix additional broken tests 2026-02-15 13:08:41 -03:00
voyage/rerank [Fix] CI/CD – Clean Up Performance PR Changes & others (#17838) 2025-12-11 12:50:03 -08:00
wandb (feat): Add W&B Inference to LiteLLM 2025-09-11 00:07:30 +05:30
watsonx Merge branch 'main' into litellm_oss_staging_02_17_2026 2026-02-18 17:26:33 +05:30
xai Add xai websearch params support 2026-01-28 09:54:43 +05:30
zai feat(zai): Add GLM-4.7 model with reasoning support (#18476) 2026-01-04 00:44:19 +05:30
test_cache_control_and_reasoning.py Add Prompt caching and reasoning support for MiniMax, GLM, Xiaomi 2026-01-28 17:25:26 +05:30
test_lifecycle_fix.py fix: revert httpx client caching that caused closed client errors (#20025) 2026-02-03 16:15:04 -08:00
test_oom_fixes.py fix: HTTP client memory leaks in Presidio, OpenAI, and Gemini (#19190) 2026-01-19 19:02:55 -08:00