litellm/tests/unit/llms
Prince Jain ec48e2491c
fix(aiohttp): reset the upstream connection when a stream closes early, so aborted Bedrock Invoke streams stop (#50657)
* fix(bedrock): close httpx response stream when invoke stream is aborted

* test(bedrock): use Final state holders and fixed datetime in invoke abort tests

* fix(bedrock): call response.aiter_bytes eagerly and await aclose safely

* refactor(bedrock): share one stream close helper module across routes

Move SupportsAclose, aclose_if_supported, and aiter_bytes_then_close into
litellm/llms/custom_httpx/stream_close.py so the Bedrock invoke paths, the
generic HTTP handler, and the Anthropic messages iterator use one copy, and
drop the mock-tolerant close in production code

* fix(aiohttp): drop the provider connection when a stream closes before its body ends

Closing an unfinished aiohttp response on uvloop starts a graceful TLS shutdown that keeps reading the provider's body for up to 30 s. Aborting the transport first ends the upstream read at once.

* test(bedrock): cover each invoke stream layer closing its source

* test(aiohttp): pin the TLS test certificate dates and run it through uvloop.run

* test(bedrock): assert the converse stream closes its response when an exception event ends it

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-11-14 12:08:03 -05:00
..
a2a refactor(typing): clear about 1,100 basedpyright Any errors across 303 files (#49986) 2026-11-06 13:31:15 -08:00
aiml chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
aiohttp_openai fix(aiohttp_openai): forward service_tier in transform_response (#47263) 2026-10-28 13:14:44 -07:00
anthropic fix(anthropic): restore sync messages streams for OpenAI and Azure (#51305) 2026-11-13 21:34:21 -08:00
apiserpent test(unit): add package markers to migrated unit test directories 2026-09-20 11:53:14 +00:00
aws_polly fix(aws_polly): read the engine from the bare model name (#47963) 2026-10-23 18:10:51 -07:00
azure fix(azure): send no api-version on Foundry project /openai/v1 Responses bases (#51767) 2026-11-14 05:24:12 -05:00
azure_ai fix(azure_ai): send reasoning_effort to Foundry Kimi K3 and keep its reasoning_content (#51747) 2026-11-14 02:08:11 -05:00
base_llm fix(router): prevent Responses fallback from replaying delivered tool calls (#51683) 2026-11-13 11:43:03 -08:00
baseten fix(aiand)!: show ai&'s name and logo, and sync ai&'s model list (#49158) 2026-10-30 10:55:26 -07:00
bedrock fix(aiohttp): reset the upstream connection when a stream closes early, so aborted Bedrock Invoke streams stop (#50657) 2026-11-14 12:08:03 -05:00
bedrock_mantle fix(bedrock): forward Codex agent messages that carry encrypted content intact (#50457) 2026-11-06 17:47:43 -08:00
bigmodel feat(zai): support response_format and max_completion_tokens, add glm-5v-turbo pricing (#48335) 2026-10-29 20:30:50 -07:00
black_forest_labs refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
brave chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
bytez test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
cerebras test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
chat fix(router): read the first stream chunk before returning so a failure before any token runs through the retry and fallback budget (#46493) 2026-10-14 14:19:25 -07:00
chatgpt fix(chatgpt): forward text and max_output_tokens to the Codex backend (#38086) 2026-11-06 06:14:32 -08:00
clarifai refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
claude_code refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
clf_ai_gateway refactor(router): route the router family through RoutingContext and close the outside surface (#49118) 2026-10-30 19:18:54 -07:00
cloudflare fix(streaming): keep merged think tags balanced and split SSE lines only on CR/LF (#48755) 2026-10-28 17:00:29 -07:00
codestral fix(streaming): keep merged think tags balanced and split SSE lines only on CR/LF (#48755) 2026-10-28 17:00:29 -07:00
codex refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
cohere fix(cohere): honor COHERE_API_BASE for embeddings (#50249) 2026-11-14 04:30:11 -05:00
cometapi chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
compactifai refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
crusoe fix(crusoe): update base URL to current endpoint (#33121) 2026-10-15 19:10:58 -07:00
custom_httpx fix(aiohttp): reset the upstream connection when a stream closes early, so aborted Bedrock Invoke streams stop (#50657) 2026-11-14 12:08:03 -05:00
dashscope feat(videos): add first-class OpenRouter video request fields (#50319) 2026-11-06 16:50:48 -08:00
databricks chore(databricks): price claude-fable-5-2 from the proprietary pricing page (#51232) 2026-11-12 00:43:40 +00:00
dataforseo refactor(typing): clear about 1,100 basedpyright Any errors across 303 files (#49986) 2026-11-06 13:31:15 -08:00
datarobot test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
deepagents feat: add litellm.agent() to run claude code, codex, opencode and deep agents through the ai gateway (#43885) 2026-10-01 22:27:49 +00:00
deepgram fix(audio-transcription): keep transcripts out of transcription error messages (#50137) 2026-11-05 18:07:15 -08:00
deepinfra chore(cost-map): add Soniox v6 and date Soniox, Bedrock Claude and Azure Whisper retirements (#51620) 2026-11-13 12:37:12 -08:00
deepseek fix(proxy): count tokens with the thinking, effort, and tool_choice that inference sends (#51079) 2026-11-14 02:29:16 +00:00
docker_model_runner test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
duckduckgo chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
e2b refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
edenai feat(videos): add first-class OpenRouter video request fields (#50319) 2026-11-06 16:50:48 -08:00
elevenlabs fix(audio-transcription): keep transcripts out of transcription error messages (#50137) 2026-11-05 18:07:15 -08:00
empiriolabs feat(decisions): add EmpirioLabs AI as a decisions provider (#49977) 2026-11-04 10:41:16 -08:00
exa_ai fix(exa): return highlights, highlight scores, score and the response envelope from Exa search (#47423) 2026-10-19 15:45:56 -07:00
fal_ai feat(fal_ai): bill fal video clips by duration and resolution (#51679) 2026-11-13 12:55:55 -08:00
fastcrw fix(llms): sanitize trailing slashes in provider get_complete_url (#42887) 2026-10-15 02:59:46 -07:00
featherless_ai refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
firecrawl fix(llms): sanitize trailing slashes in provider get_complete_url (#42887) 2026-10-15 02:59:46 -07:00
fireworks_ai fix(fireworks_ai): read fractional sizes like 1p5b in the size bucket fallback (#51451) 2026-11-13 17:49:49 +00:00
friendliai fix(friendliai): drop litellm-internal message fields before sending (#45092) 2026-10-20 01:22:07 +00:00
gdc test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
gemini feat(gemini): support gemini-decision-1 on /v1/decisions and /v1/systemone (#51220) 2026-11-11 18:33:22 -08:00
gigachat refactor(typing): clear about 1,100 basedpyright Any errors across 303 files (#49986) 2026-11-06 13:31:15 -08:00
github_copilot feat(github_copilot): support GitHub App authentication with PEM upload (#51427) 2026-11-14 10:33:20 -05:00
google_pse chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
gradient_ai fix(llms): sanitize trailing slashes in provider get_complete_url (#42887) 2026-10-15 02:59:46 -07:00
groq fix(openai): stop stripping cache_control from the caller's own messages and tools (#35602) 2026-10-31 00:29:10 -07:00
heroku fix(llms): sanitize trailing slashes in provider get_complete_url (#42887) 2026-10-15 02:59:46 -07:00
hosted_vllm fix(hosted_vllm): omit empty tools list from chat requests (#50812) 2026-11-09 19:49:13 -08:00
huggingface fix(huggingface): honor HUGGINGFACE_API_BASE and env endpoints for chat (#51449) 2026-11-13 10:35:34 -08:00
hyperbolic refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
inception refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
infinity chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
jina_ai fix(rerank): preserve token usage in spend logs (#49308) 2026-10-30 13:46:27 -07:00
lambda_ai test: move the unit half of 126 mixed legacy files into tests/unit (#45090) 2026-10-07 14:07:43 -07:00
langflow fix(ci): stop stale CI reds, keep unit tests off the host env, retry CyberArk policy conflicts (#43294) 2026-09-26 09:25:13 -07:00
langgraph fix(streaming)!: fail a stream that ends without a finish reason (#47665) 2026-10-31 11:29:26 -07:00
laya feat: add Bespoke Nimble gateway and OSS classifier support (#44246) 2026-10-02 19:37:08 -07:00
lemonade chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
linkup fix(llms): sanitize trailing slashes in provider get_complete_url (#42887) 2026-10-15 02:59:46 -07:00
litellm_proxy refactor(typing): clear about 1,100 basedpyright Any errors across 303 files (#49986) 2026-11-06 13:31:15 -08:00
llamafile refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
lm_studio refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
manus test: migrate SDK and proxy-booting legacy CircleCI tests to their tiers (#45758) 2026-10-11 00:17:21 -07:00
meta fix(realtime)!: keep turn_detection.type when a transcript guardrail disables auto-response (#44939) 2026-10-16 15:24:40 +00:00
meta_llama refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
microsoft_365_copilot refactor(router): route the router family through RoutingContext and close the outside surface (#49118) 2026-10-30 19:18:54 -07:00
milvus test: migrate legacy unit dirs and unrun folders into tests/unit, tests/integration and tests/e2e (#45940) 2026-10-11 06:24:57 -07:00
minimax fix(minimax): forward reasoning_effort instead of dropping it (#49583) 2026-11-09 14:03:50 -08:00
mistral fix(responses): accept a string or tuple allowed_openai_params on Responses models instead of answering 500 (#51751) 2026-11-14 04:32:38 -05:00
modelscope fix(llms): sanitize trailing slashes in provider get_complete_url (#42887) 2026-10-15 02:59:46 -07:00
mongodb feat(bedrock): search knowledge bases by image and return image hits (#51063) 2026-11-12 00:19:34 +00:00
moonshot feat(cost-map): move -latest model aliases into the cost map (#50491) 2026-11-07 11:56:47 -08:00
morph feat(morph): support open models and cached input pricing (#48676) 2026-10-27 18:06:14 -07:00
nadir test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
nebius test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
neosantara refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
nimble test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807) 2026-10-07 05:02:28 +00:00
nlp_cloud refactor(typing): clear 1141 basedpyright Any errors across 182 files (#50711) 2026-11-11 22:11:47 +00:00
novita test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
nscale test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
nvidia_nim fix(embeddings): unwrap extra_body for providers with their own embedding handlers (#51335) 2026-11-12 15:41:50 -08:00
nvidia_riva fix(nvidia_riva): send the api key exactly once by calling riva over grpc directly (#50140) 2026-11-04 18:07:31 -08:00
oci fix(fips): hash MCP advisory-lock keys with SHA-256 in fips_mode and compute the OCI body digest as a security hash (#51165) 2026-11-11 16:22:14 -08:00
ocr test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
ollama fix(embeddings): unwrap extra_body for providers with their own embedding handlers (#51335) 2026-11-12 15:41:50 -08:00
oobabooga fix(streaming): keep merged think tags balanced and split SSE lines only on CR/LF (#48755) 2026-10-28 17:00:29 -07:00
openai feat(fal_ai): bill fal video clips by duration and resolution (#51679) 2026-11-13 12:55:55 -08:00
openai_like feat: update EmpirioLabs models in model_prices_and_context_window.json (#49980) 2026-11-04 20:11:52 +00:00
opencode fix(opencode): stream Go Messages-protocol models through chat completions (#51715) 2026-11-13 22:21:52 -05:00
openrouter fix: raise on failed OpenRouter non-streaming completions (#50880) 2026-11-14 03:02:00 +00:00
opensandbox chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
opensearch feat(opensearch): map vector store search filters to k-NN filters and add hybrid search (#51131) 2026-11-14 03:30:55 +00:00
ovhcloud fix(audio-transcription): keep transcripts out of transcription error messages (#50137) 2026-11-05 18:07:15 -08:00
parallel_ai refactor(rust_bridge): route every Rust-to-Python call through host modules (#50822) 2026-11-11 01:57:54 +00:00
parasail refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
pass_through fix(guardrails): exclude litellm_metadata from pass-through guardrail scan text (#48276) 2026-10-26 12:07:58 -07:00
perplexity fix(search): forward Perplexity search filters (#49344) 2026-10-30 17:08:32 -07:00
pg_vector fix(security): remove the publicly known master key from the repo (#44718) 2026-10-06 10:55:24 -07:00
pi feat(harness): add pi harness for litellm.agent() (#45467) 2026-10-20 09:01:40 +00:00
publicai test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
ragflow test: migrate legacy unit dirs and unrun folders into tests/unit, tests/integration and tests/e2e (#45940) 2026-10-11 06:24:57 -07:00
recraft refactor(typing): clear about 1,100 basedpyright Any errors across 303 files (#49986) 2026-11-06 13:31:15 -08:00
reducto test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
replicate feat(replicate): add video generation, image generation and text to speech through the predictions API (#51360) 2026-11-12 08:44:50 -08:00
runwayml feat(videos): add first-class OpenRouter video request fields (#50319) 2026-11-06 16:50:48 -08:00
s3_vectors refactor(router): move router callers onto the entry points and invert the proxy imports (#48847) 2026-10-29 17:26:33 +00:00
sagemaker fix(bedrock): send max_tokens and priority in Cohere embedding requests (#51435) 2026-11-12 23:44:08 +00:00
sail test(router_strategy): keep foreign asyncio task noise out of the budget limiter caplog asserts (#43375) 2026-10-29 06:22:06 -07:00
sambanova chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
sap refactor(typing): clear about 1,100 basedpyright Any errors across 303 files (#49986) 2026-11-06 13:31:15 -08:00
scaledown feat(scaledown): add native text provider support (#44168) 2026-10-10 14:01:17 -07:00
scaleway fix(audio-transcription): keep transcripts out of transcription error messages (#50137) 2026-11-05 18:07:15 -08:00
searchapi chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
searxng chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
serper chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
snowflake fix(anthropic)!: send /v1/messages PDF documents to OpenAI-shaped providers as file parts (#47166) 2026-10-29 08:06:12 -07:00
soniox chore(cost-map): add Soniox v6 and date Soniox, Bedrock Claude and Azure Whisper retirements (#51620) 2026-11-13 12:37:12 -08:00
stability refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
strands_decider feat(strands_decider): call a Strands Decider on an AgentCore runtime ARN (#45678) 2026-10-09 22:51:19 -07:00
tavily fix(tavily): return raw_content, score, favicon and the response envelope from Tavily search (#48684) 2026-10-27 18:52:06 -07:00
tencent fix(logging): warn when drop_params drops a requested reasoning level on the remaining paths (#48933) 2026-10-30 08:48:43 -07:00
tinyfish test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807) 2026-10-07 05:02:28 +00:00
together_ai fix(responses bridge): round-trip tool_search_call and tool_search_output items between turns (#46444) 2026-10-14 21:40:14 -07:00
tool_loop feat(harness): add Harness.TOOL_LOOP, a minimal in-process tool-calling loop (#44391) 2026-10-03 17:33:01 +00:00
topaz fix(llms): sanitize trailing slashes in provider get_complete_url (#42887) 2026-10-15 02:59:46 -07:00
triton fix(streaming): keep merged think tags balanced and split SSE lines only on CR/LF (#48755) 2026-10-28 17:00:29 -07:00
v0 test: move the unit half of 126 mixed legacy files into tests/unit (#45090) 2026-10-07 14:07:43 -07:00
valkey test: migrate legacy provider tests to tests/unit (wave 2, phase 13) 2026-09-20 14:18:49 +00:00
vercel_ai_gateway refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
vertex_ai fix(vertex_ai): reuse one Google token across RAG Engine searches and cold bursts (#51239) 2026-11-14 03:12:09 -05:00
vllm fix(vllm): list hosted_vllm wildcard models when the server has no api key (#40066) 2026-11-03 04:13:28 +00:00
volcengine feat(videos): add first-class OpenRouter video request fields (#50319) 2026-11-06 16:50:48 -08:00
voyage fix(llms): sanitize trailing slashes in provider get_complete_url (#42887) 2026-10-15 02:59:46 -07:00
wandb test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
watsonx fix(audio-transcription): keep transcripts out of transcription error messages (#50137) 2026-11-05 18:07:15 -08:00
webiq feat(search): add Microsoft Web IQ provider (#50883) 2026-11-10 22:43:36 +00:00
xai fix(xai): honor deployment image pixel pricing (#50490) 2026-11-07 02:17:16 +00:00
xinference test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807) 2026-10-07 05:02:28 +00:00
you_com test: migrate phase 14 wave 2 provider tests to tests/unit 2026-09-20 11:01:40 +00:00
zai feat(zai): support response_format and max_completion_tokens, add glm-5v-turbo pricing (#48335) 2026-10-29 20:30:50 -07:00
__init__.py test(unit): add package markers to migrated unit test directories 2026-09-20 11:53:14 +00:00
test_api_base_trailing_slash_routes.py fix(llms): join api_base and route with one shared helper (#47535) 2026-10-21 06:59:32 +00:00
test_cache_control_and_reasoning.py fix(minimax): forward reasoning_effort instead of dropping it (#49583) 2026-11-09 14:03:50 -08:00
test_custom_llm.py refactor(rust_bridge): route every Rust-to-Python call through host modules (#50822) 2026-11-11 01:57:54 +00:00
test_file_content_block.py fix(prompt_templates): keep moved private helpers importable from common_utils (#50721) 2026-11-09 20:52:52 +00:00
test_file_search_responses.py refactor(prompt_templates): move JSON schema helpers out of common_utils into json_schema_utils (#50241) 2026-11-05 17:25:53 -08:00
test_get_complete_url_trailing_slash.py feat(dashscope): add image edits and text-to-speech (#48998) 2026-10-29 17:48:38 -07:00
test_lifecycle_fix.py refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
test_oss_decision.py feat(decisions): add Laya and Bespoke providers to the Add Model form (#45260) 2026-10-14 08:11:11 -07:00
test_polling_url_origin_match.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_predibase_transformation.py fix(streaming): keep merged think tags balanced and split SSE lines only on CR/LF (#48755) 2026-10-28 17:00:29 -07:00
test_subscription_accounts.py feat(usage): show subscription-covered providers and account fees separately (#44772) 2026-10-30 03:58:36 -07:00