litellm/tests/unit/llms
tavren27 4e29aea11c
fix(responses): accept a string or tuple allowed_openai_params on Responses models instead of answering 500 (#51751)
* fix(responses): accept any sequence for allowed_openai_params and give UnsupportedParamsError a response

A deployment whose allowed_openai_params is a string (or an SDK caller passing a tuple) made every
Responses-mode request answer 500 "can only concatenate list". The Responses path now builds the
allowlist from any sequence, matching the chat path.

UnsupportedParamsError now runs BadRequestError's constructor, so it carries response, body and
request like every other 400 instead of raising AttributeError on .response.

* fix(params): read a bare-string allowed_openai_params as one param name on every path

A deployment or request with allowed_openai_params: "presence_penalty" now allowlists presence_penalty
on chat completions, embeddings, the Responses API, the chat-to-Responses bridge, the router's tier
and precall filters, and Mistral's prompt_cache_key check. Before, chat and Responses read the string
letter by letter, so it allowlisted nothing, and the router ignored it.

UnsupportedParamsError's args now hold its own message instead of the BadRequestError one.

* fix(params): read a dict or comma-separated allowed_openai_params the way chat and embeddings did

A dict allowlist is read for its keys and a string is split on commas, so configs that worked on
chat or embeddings before keep working, and the router reads them the same way. The fallback errors
header names an error by its class when its type is empty, so UnsupportedParamsError still shows up
there by name.

* fix(params): read every param name in a string allowed_openai_params

A string allowlist now yields every name in it, whether the names are split by commas or spaces or
written as a list inside the string, so embeddings configs like "dimensions encoding_format" keep
working. A request with no allowlist returns early instead of raising and catching a validation error.

* fix(fallbacks): keep the fallback errors header type for errors other than UnsupportedParamsError

The header named an error by its class whenever its type was empty, which changed "None" to the class name for
timeouts and other errors built without a provider body. Only UnsupportedParamsError needs its class name now that
it inherits an empty type, so every other error keeps the type it carries.

* perf(params): read a string allowed_openai_params with one regex scan

The string reader walked every character in Python and ran twice per request, so a 1 MB allowlist added about
300 ms of event-loop time. `\w+` matches exactly the alphanumeric-or-underscore runs the old loop kept, checked
over every Unicode code point, and runs in C.

* test(params): cover an int allowed_openai_params and an admin request for a model two teams share

An allowlist that is no string, mapping, or sequence now allowlists nothing instead of raising TypeError, and an admin request with no team for a model name two teams share answers the multiple-teams 400 with the model name

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-11-14 04:32:38 -05:00
..
a2a refactor(typing): clear about 1,100 basedpyright Any errors across 303 files (#49986) 2026-11-06 13:31:15 -08:00
aiml chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
aiohttp_openai fix(aiohttp_openai): forward service_tier in transform_response (#47263) 2026-10-28 13:14:44 -07:00
anthropic fix(anthropic): restore sync messages streams for OpenAI and Azure (#51305) 2026-11-13 21:34:21 -08:00
apiserpent test(unit): add package markers to migrated unit test directories 2026-09-20 11:53:14 +00:00
aws_polly fix(aws_polly): read the engine from the bare model name (#47963) 2026-10-23 18:10:51 -07:00
azure fix(azure): strip GPT-5 routing marker from Responses deployments (#51438) 2026-11-12 11:14:52 -08:00
azure_ai fix(azure_ai): send reasoning_effort to Foundry Kimi K3 and keep its reasoning_content (#51747) 2026-11-14 02:08:11 -05:00
base_llm fix(router): prevent Responses fallback from replaying delivered tool calls (#51683) 2026-11-13 11:43:03 -08:00
baseten fix(aiand)!: show ai&'s name and logo, and sync ai&'s model list (#49158) 2026-10-30 10:55:26 -07:00
bedrock fix(bedrock): send s3:// image, document and video URLs to Converse as S3 sources (#50577) 2026-11-13 21:33:21 -08:00
bedrock_mantle fix(bedrock): forward Codex agent messages that carry encrypted content intact (#50457) 2026-11-06 17:47:43 -08:00
bigmodel feat(zai): support response_format and max_completion_tokens, add glm-5v-turbo pricing (#48335) 2026-10-29 20:30:50 -07:00
black_forest_labs refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
brave chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
bytez test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
cerebras test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
chat fix(router): read the first stream chunk before returning so a failure before any token runs through the retry and fallback budget (#46493) 2026-10-14 14:19:25 -07:00
chatgpt fix(chatgpt): forward text and max_output_tokens to the Codex backend (#38086) 2026-11-06 06:14:32 -08:00
clarifai refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
claude_code refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
clf_ai_gateway refactor(router): route the router family through RoutingContext and close the outside surface (#49118) 2026-10-30 19:18:54 -07:00
cloudflare fix(streaming): keep merged think tags balanced and split SSE lines only on CR/LF (#48755) 2026-10-28 17:00:29 -07:00
codestral fix(streaming): keep merged think tags balanced and split SSE lines only on CR/LF (#48755) 2026-10-28 17:00:29 -07:00
codex refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
cohere fix(cohere): honor COHERE_API_BASE for embeddings (#50249) 2026-11-14 04:30:11 -05:00
cometapi chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
compactifai refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
crusoe fix(crusoe): update base URL to current endpoint (#33121) 2026-10-15 19:10:58 -07:00
custom_httpx feat(custom-logger-v2): add types, dispatcher and route callback call sites (step 0) (#51214) 2026-11-13 22:08:05 +00:00
dashscope feat(videos): add first-class OpenRouter video request fields (#50319) 2026-11-06 16:50:48 -08:00
databricks chore(databricks): price claude-fable-5-2 from the proprietary pricing page (#51232) 2026-11-12 00:43:40 +00:00
dataforseo refactor(typing): clear about 1,100 basedpyright Any errors across 303 files (#49986) 2026-11-06 13:31:15 -08:00
datarobot test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
deepagents feat: add litellm.agent() to run claude code, codex, opencode and deep agents through the ai gateway (#43885) 2026-10-01 22:27:49 +00:00
deepgram fix(audio-transcription): keep transcripts out of transcription error messages (#50137) 2026-11-05 18:07:15 -08:00
deepinfra chore(cost-map): add Soniox v6 and date Soniox, Bedrock Claude and Azure Whisper retirements (#51620) 2026-11-13 12:37:12 -08:00
deepseek fix(proxy): count tokens with the thinking, effort, and tool_choice that inference sends (#51079) 2026-11-14 02:29:16 +00:00
docker_model_runner test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
duckduckgo chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
e2b refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
edenai feat(videos): add first-class OpenRouter video request fields (#50319) 2026-11-06 16:50:48 -08:00
elevenlabs fix(audio-transcription): keep transcripts out of transcription error messages (#50137) 2026-11-05 18:07:15 -08:00
empiriolabs feat(decisions): add EmpirioLabs AI as a decisions provider (#49977) 2026-11-04 10:41:16 -08:00
exa_ai fix(exa): return highlights, highlight scores, score and the response envelope from Exa search (#47423) 2026-10-19 15:45:56 -07:00
fal_ai feat(fal_ai): bill fal video clips by duration and resolution (#51679) 2026-11-13 12:55:55 -08:00
fastcrw fix(llms): sanitize trailing slashes in provider get_complete_url (#42887) 2026-10-15 02:59:46 -07:00
featherless_ai refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
firecrawl fix(llms): sanitize trailing slashes in provider get_complete_url (#42887) 2026-10-15 02:59:46 -07:00
fireworks_ai fix(fireworks_ai): read fractional sizes like 1p5b in the size bucket fallback (#51451) 2026-11-13 17:49:49 +00:00
friendliai fix(friendliai): drop litellm-internal message fields before sending (#45092) 2026-10-20 01:22:07 +00:00
gdc test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
gemini feat(gemini): support gemini-decision-1 on /v1/decisions and /v1/systemone (#51220) 2026-11-11 18:33:22 -08:00
gigachat refactor(typing): clear about 1,100 basedpyright Any errors across 303 files (#49986) 2026-11-06 13:31:15 -08:00
github_copilot refactor(router): delete the python router core, the rust core is the only core (#50178) 2026-11-05 10:40:46 -08:00
google_pse chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
gradient_ai fix(llms): sanitize trailing slashes in provider get_complete_url (#42887) 2026-10-15 02:59:46 -07:00
groq fix(openai): stop stripping cache_control from the caller's own messages and tools (#35602) 2026-10-31 00:29:10 -07:00
heroku fix(llms): sanitize trailing slashes in provider get_complete_url (#42887) 2026-10-15 02:59:46 -07:00
hosted_vllm fix(hosted_vllm): omit empty tools list from chat requests (#50812) 2026-11-09 19:49:13 -08:00
huggingface fix(huggingface): honor HUGGINGFACE_API_BASE and env endpoints for chat (#51449) 2026-11-13 10:35:34 -08:00
hyperbolic refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
inception refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
infinity chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
jina_ai fix(rerank): preserve token usage in spend logs (#49308) 2026-10-30 13:46:27 -07:00
lambda_ai test: move the unit half of 126 mixed legacy files into tests/unit (#45090) 2026-10-07 14:07:43 -07:00
langflow fix(ci): stop stale CI reds, keep unit tests off the host env, retry CyberArk policy conflicts (#43294) 2026-09-26 09:25:13 -07:00
langgraph fix(streaming)!: fail a stream that ends without a finish reason (#47665) 2026-10-31 11:29:26 -07:00
laya feat: add Bespoke Nimble gateway and OSS classifier support (#44246) 2026-10-02 19:37:08 -07:00
lemonade chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
linkup fix(llms): sanitize trailing slashes in provider get_complete_url (#42887) 2026-10-15 02:59:46 -07:00
litellm_proxy refactor(typing): clear about 1,100 basedpyright Any errors across 303 files (#49986) 2026-11-06 13:31:15 -08:00
llamafile refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
lm_studio refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
manus test: migrate SDK and proxy-booting legacy CircleCI tests to their tiers (#45758) 2026-10-11 00:17:21 -07:00
meta fix(realtime)!: keep turn_detection.type when a transcript guardrail disables auto-response (#44939) 2026-10-16 15:24:40 +00:00
meta_llama refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
microsoft_365_copilot refactor(router): route the router family through RoutingContext and close the outside surface (#49118) 2026-10-30 19:18:54 -07:00
milvus test: migrate legacy unit dirs and unrun folders into tests/unit, tests/integration and tests/e2e (#45940) 2026-10-11 06:24:57 -07:00
minimax fix(minimax): forward reasoning_effort instead of dropping it (#49583) 2026-11-09 14:03:50 -08:00
mistral fix(responses): accept a string or tuple allowed_openai_params on Responses models instead of answering 500 (#51751) 2026-11-14 04:32:38 -05:00
modelscope fix(llms): sanitize trailing slashes in provider get_complete_url (#42887) 2026-10-15 02:59:46 -07:00
mongodb feat(bedrock): search knowledge bases by image and return image hits (#51063) 2026-11-12 00:19:34 +00:00
moonshot feat(cost-map): move -latest model aliases into the cost map (#50491) 2026-11-07 11:56:47 -08:00
morph feat(morph): support open models and cached input pricing (#48676) 2026-10-27 18:06:14 -07:00
nadir test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
nebius test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
neosantara refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
nimble test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807) 2026-10-07 05:02:28 +00:00
nlp_cloud refactor(typing): clear 1141 basedpyright Any errors across 182 files (#50711) 2026-11-11 22:11:47 +00:00
novita test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
nscale test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
nvidia_nim fix(embeddings): unwrap extra_body for providers with their own embedding handlers (#51335) 2026-11-12 15:41:50 -08:00
nvidia_riva fix(nvidia_riva): send the api key exactly once by calling riva over grpc directly (#50140) 2026-11-04 18:07:31 -08:00
oci fix(fips): hash MCP advisory-lock keys with SHA-256 in fips_mode and compute the OCI body digest as a security hash (#51165) 2026-11-11 16:22:14 -08:00
ocr test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
ollama fix(embeddings): unwrap extra_body for providers with their own embedding handlers (#51335) 2026-11-12 15:41:50 -08:00
oobabooga fix(streaming): keep merged think tags balanced and split SSE lines only on CR/LF (#48755) 2026-10-28 17:00:29 -07:00
openai feat(fal_ai): bill fal video clips by duration and resolution (#51679) 2026-11-13 12:55:55 -08:00
openai_like feat: update EmpirioLabs models in model_prices_and_context_window.json (#49980) 2026-11-04 20:11:52 +00:00
opencode fix(opencode): stream Go Messages-protocol models through chat completions (#51715) 2026-11-13 22:21:52 -05:00
openrouter fix: raise on failed OpenRouter non-streaming completions (#50880) 2026-11-14 03:02:00 +00:00
opensandbox chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
opensearch feat(opensearch): map vector store search filters to k-NN filters and add hybrid search (#51131) 2026-11-14 03:30:55 +00:00
ovhcloud fix(audio-transcription): keep transcripts out of transcription error messages (#50137) 2026-11-05 18:07:15 -08:00
parallel_ai refactor(rust_bridge): route every Rust-to-Python call through host modules (#50822) 2026-11-11 01:57:54 +00:00
parasail refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
pass_through fix(guardrails): exclude litellm_metadata from pass-through guardrail scan text (#48276) 2026-10-26 12:07:58 -07:00
perplexity fix(search): forward Perplexity search filters (#49344) 2026-10-30 17:08:32 -07:00
pg_vector fix(security): remove the publicly known master key from the repo (#44718) 2026-10-06 10:55:24 -07:00
pi feat(harness): add pi harness for litellm.agent() (#45467) 2026-10-20 09:01:40 +00:00
publicai test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
ragflow test: migrate legacy unit dirs and unrun folders into tests/unit, tests/integration and tests/e2e (#45940) 2026-10-11 06:24:57 -07:00
recraft refactor(typing): clear about 1,100 basedpyright Any errors across 303 files (#49986) 2026-11-06 13:31:15 -08:00
reducto test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
replicate feat(replicate): add video generation, image generation and text to speech through the predictions API (#51360) 2026-11-12 08:44:50 -08:00
runwayml feat(videos): add first-class OpenRouter video request fields (#50319) 2026-11-06 16:50:48 -08:00
s3_vectors refactor(router): move router callers onto the entry points and invert the proxy imports (#48847) 2026-10-29 17:26:33 +00:00
sagemaker fix(bedrock): send max_tokens and priority in Cohere embedding requests (#51435) 2026-11-12 23:44:08 +00:00
sail test(router_strategy): keep foreign asyncio task noise out of the budget limiter caplog asserts (#43375) 2026-10-29 06:22:06 -07:00
sambanova chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
sap refactor(typing): clear about 1,100 basedpyright Any errors across 303 files (#49986) 2026-11-06 13:31:15 -08:00
scaledown feat(scaledown): add native text provider support (#44168) 2026-10-10 14:01:17 -07:00
scaleway fix(audio-transcription): keep transcripts out of transcription error messages (#50137) 2026-11-05 18:07:15 -08:00
searchapi chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
searxng chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
serper chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
snowflake fix(anthropic)!: send /v1/messages PDF documents to OpenAI-shaped providers as file parts (#47166) 2026-10-29 08:06:12 -07:00
soniox chore(cost-map): add Soniox v6 and date Soniox, Bedrock Claude and Azure Whisper retirements (#51620) 2026-11-13 12:37:12 -08:00
stability refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
strands_decider feat(strands_decider): call a Strands Decider on an AgentCore runtime ARN (#45678) 2026-10-09 22:51:19 -07:00
tavily fix(tavily): return raw_content, score, favicon and the response envelope from Tavily search (#48684) 2026-10-27 18:52:06 -07:00
tencent fix(logging): warn when drop_params drops a requested reasoning level on the remaining paths (#48933) 2026-10-30 08:48:43 -07:00
tinyfish test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807) 2026-10-07 05:02:28 +00:00
together_ai fix(responses bridge): round-trip tool_search_call and tool_search_output items between turns (#46444) 2026-10-14 21:40:14 -07:00
tool_loop feat(harness): add Harness.TOOL_LOOP, a minimal in-process tool-calling loop (#44391) 2026-10-03 17:33:01 +00:00
topaz fix(llms): sanitize trailing slashes in provider get_complete_url (#42887) 2026-10-15 02:59:46 -07:00
triton fix(streaming): keep merged think tags balanced and split SSE lines only on CR/LF (#48755) 2026-10-28 17:00:29 -07:00
v0 test: move the unit half of 126 mixed legacy files into tests/unit (#45090) 2026-10-07 14:07:43 -07:00
valkey test: migrate legacy provider tests to tests/unit (wave 2, phase 13) 2026-09-20 14:18:49 +00:00
vercel_ai_gateway refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
vertex_ai fix(vertex_ai): reuse one Google token across RAG Engine searches and cold bursts (#51239) 2026-11-14 03:12:09 -05:00
vllm fix(vllm): list hosted_vllm wildcard models when the server has no api key (#40066) 2026-11-03 04:13:28 +00:00
volcengine feat(videos): add first-class OpenRouter video request fields (#50319) 2026-11-06 16:50:48 -08:00
voyage fix(llms): sanitize trailing slashes in provider get_complete_url (#42887) 2026-10-15 02:59:46 -07:00
wandb test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
watsonx fix(audio-transcription): keep transcripts out of transcription error messages (#50137) 2026-11-05 18:07:15 -08:00
webiq feat(search): add Microsoft Web IQ provider (#50883) 2026-11-10 22:43:36 +00:00
xai fix(xai): honor deployment image pixel pricing (#50490) 2026-11-07 02:17:16 +00:00
xinference test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807) 2026-10-07 05:02:28 +00:00
you_com test: migrate phase 14 wave 2 provider tests to tests/unit 2026-09-20 11:01:40 +00:00
zai feat(zai): support response_format and max_completion_tokens, add glm-5v-turbo pricing (#48335) 2026-10-29 20:30:50 -07:00
__init__.py test(unit): add package markers to migrated unit test directories 2026-09-20 11:53:14 +00:00
test_api_base_trailing_slash_routes.py fix(llms): join api_base and route with one shared helper (#47535) 2026-10-21 06:59:32 +00:00
test_cache_control_and_reasoning.py fix(minimax): forward reasoning_effort instead of dropping it (#49583) 2026-11-09 14:03:50 -08:00
test_custom_llm.py refactor(rust_bridge): route every Rust-to-Python call through host modules (#50822) 2026-11-11 01:57:54 +00:00
test_file_content_block.py fix(prompt_templates): keep moved private helpers importable from common_utils (#50721) 2026-11-09 20:52:52 +00:00
test_file_search_responses.py refactor(prompt_templates): move JSON schema helpers out of common_utils into json_schema_utils (#50241) 2026-11-05 17:25:53 -08:00
test_get_complete_url_trailing_slash.py feat(dashscope): add image edits and text-to-speech (#48998) 2026-10-29 17:48:38 -07:00
test_lifecycle_fix.py refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
test_oss_decision.py feat(decisions): add Laya and Bespoke providers to the Add Model form (#45260) 2026-10-14 08:11:11 -07:00
test_polling_url_origin_match.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_predibase_transformation.py fix(streaming): keep merged think tags balanced and split SSE lines only on CR/LF (#48755) 2026-10-28 17:00:29 -07:00
test_subscription_accounts.py feat(usage): show subscription-covered providers and account fees separately (#44772) 2026-10-30 03:58:36 -07:00