litellm/tests/unit/llms
ryan-crabbe-berri 05de300ed6
fix(anthropic): keep signed empty thinking blocks and honor thinking.block_binding (#46342)
* fix(anthropic): treat adaptive thinking as enabled when mapping response_format

is_thinking_enabled only recognised thinking.type=enabled, so a request that
paired thinking.type=adaptive with response_format took the synthetic JSON
tool path with a forced tool_choice, which Anthropic rejects alongside
thinking. The same check also gates the Bedrock Converse and Databricks
forced tool_choice downgrades, so they now see adaptive thinking too.

* fix(anthropic): keep signed thinking blocks with empty text on chat completions

Models that default to display: omitted return signed thinking blocks whose
text is empty, and Anthropic rejects the next turn when they are not replayed
unchanged. is_unsignable_thinking_block dropped them along with unsigned
blocks, and the inline content path required non-empty text, so a tool-use
turn on Opus 4.7+ or the 5 families lost its thinking blocks and 400ed on the
tool-result turn. Only a missing signature now drops a block, and the
mid-conversation system placement mirrors that rule. The /v1/messages strip
helper is left to #41362.

* fix(anthropic): skip the thinking-strip retry when block_binding asks for an error

The /v1/messages retry loop strips thinking blocks and resends on any
invalid-thinking 400, including the binding mismatch a caller opted into
with thinking.block_binding.prefix_mismatch_behavior: error. The retry
decision now sees the request body and surfaces that 400 instead of
overriding the caller's choice. #42888 covers the litellm_params side.

* fix(bedrock): carry thinking.block_binding through the Invoke thinking rebuilds

Bedrock Invoke rebuilt the thinking param from scratch when display: updates
was requested, on both the /v1/messages and chat completions paths, and the
legacy enabled-to-adaptive translation plus the clear_thinking injection did
the same, so thinking.block_binding never reached Bedrock. Every rebuild now
spreads the caller's thinking and only overrides the keys it owns, and
AnthropicThinkingParam types block_binding. The thinking-binding-controls
beta was already on the Bedrock allowlist.

* fix(anthropic): keep forcing the response_format tool under adaptive thinking

This reverts ad9e861ae7. Anthropic, Bedrock Invoke and Vertex all accept a
forced tool_choice next to thinking.type=adaptive and only reject it with
type=enabled, so counting adaptive as enabled fixed no live failure. It did
drop the forced JSON tool, so Claude answered response_format requests in
prose, and it downgraded a required tool_choice to auto on Bedrock Converse

* test(e2e): drop the adaptive thinking tool round trip case

With adaptive thinking the model often skips thinking on the weather prompt, so
turn 1 carries no signed empty block and the case fails before the replay it is
meant to cover. Anthropic also answers the replay with a 200 whether or not the
block is forwarded, so the case could not tell the fix from main. The factory
unit tests pin the outgoing shape instead

* fix(anthropic): only skip the thinking-strip retry for a binding mismatch 400

prefix_mismatch_behavior: error asks Anthropic to reject a block bound to
another conversation. It says nothing about unrelated bad blocks such as a
null signature, so those still get the strip-and-retry self-heal. Also trims
the empty-thinking docstrings to what the code guarantees and types the new
test parameters

* refactor(anthropic): narrow thinking fields with the mapping validator instead of typing.cast

typing.cast is banned by the strict ruff gate (TID251), so the three new casts pushed that rule one over its merge-base count. The existing pydantic mapping validator gives the same typed view without an unchecked cast, and it now has a generic name since it is no longer only used for Claude Code payloads

* refactor(anthropic): read thinking settings through the validated mapping so basedpyright stays at its merge-base count

The Bedrock clear_thinking rewrite still narrowed with a cast and an untyped dict fallback, which added two reportUnknownVariableType errors. It now uses the same pydantic mapping validator, made public for that reuse, and the retry hook validates the request body itself so the handler passes a typed dict instead of an untyped one

---------

Co-authored-by: yuneng-jiang <yuneng@berri.ai>
(cherry picked from commit f470e418ea)
2026-10-20 01:06:24 -07:00
..
a2a test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
aiml test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
anthropic fix(anthropic): keep signed empty thinking blocks and honor thinking.block_binding (#46342) 2026-10-20 01:06:24 -07:00
apiserpent test(unit): add package markers to migrated unit test directories 2026-09-20 11:53:14 +00:00
azure feat(sdk): add litellm.systemone and trim litellm.decisions to the OpenAI shape (#45846) 2026-10-11 00:50:46 +00:00
azure_ai test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
base_llm feat(decisions): backport /v1/systemone, OpenAI-format /v1/decisions and the openai Decisions provider to rc/1.105.0 (#44236, #45184, #45214) (#45189) 2026-10-07 22:06:51 -07:00
baseten test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
bedrock fix(anthropic): keep signed empty thinking blocks and honor thinking.block_binding (#46342) 2026-10-20 01:06:24 -07:00
bedrock_mantle test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
black_forest_labs test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
bytez test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
cerebras test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
chat fix(params): validate stream_chunk_size once, before any provider call (#43222) 2026-09-26 23:01:20 +00:00
chatgpt test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
claude_code feat: add litellm.agent() to run claude code, codex, opencode and deep agents through the ai gateway (#43885) 2026-10-01 22:27:49 +00:00
cloudflare test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
codex feat: add litellm.agent() to run claude code, codex, opencode and deep agents through the ai gateway (#43885) 2026-10-01 22:27:49 +00:00
cohere refactor(ocr): remove the Python OCR execution path and require the Rust route (#43081) 2026-09-24 18:18:50 -07:00
cometapi test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
compactifai test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
crusoe test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
custom_httpx fix(anthropic): keep signed empty thinking blocks and honor thinking.block_binding (#46342) 2026-10-20 01:06:24 -07:00
dashscope test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
databricks fix(streaming): keep the served service_tier on streamed chunks and spend rows (#42870) 2026-09-29 12:54:17 -07:00
datarobot test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
deepagents feat: add litellm.agent() to run claude code, codex, opencode and deep agents through the ai gateway (#43885) 2026-10-01 22:27:49 +00:00
deepgram test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
deepinfra test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
deepseek refactor(anthropic): rename experimental_pass_through to pass_through (#43329) 2026-09-26 13:00:50 -07:00
docker_model_runner test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
edenai test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
elevenlabs fix(params): keep _litellm_* kwargs out of provider request bodies by construction (#43221) 2026-09-26 15:20:09 -07:00
fal_ai test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
fastcrw test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
featherless_ai test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
fireworks_ai chore(lint): remove the LIT002 mutable-construction rule (#43971) 2026-10-01 12:24:02 -07:00
gdc test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
gemini test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
gigachat test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
github_copilot refactor(anthropic): rename experimental_pass_through to pass_through (#43329) 2026-09-26 13:00:50 -07:00
gradient_ai test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
groq test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
heroku test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
hosted_vllm fix(hosted_vllm): keep reasoning_content on replayed assistant messages (#43599) 2026-09-30 13:12:28 -07:00
huggingface test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
inception test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
jina_ai test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
langflow fix(ci): stop stale CI reds, keep unit tests off the host env, retry CyberArk policy conflicts (#43294) 2026-09-26 09:25:13 -07:00
lemonade test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
litellm_proxy fix(spend): attribute CLI session spend to the per-user cli-session alias instead of the hashed session token (#40541) 2026-09-24 18:21:47 -07:00
llamafile test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
lm_studio test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
manus test: migrate phase 9 legacy llm provider tests to tests/unit 2026-09-20 08:13:11 +00:00
meta test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
meta_llama test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
minimax test: migrate phase 9 legacy llm provider tests to tests/unit 2026-09-20 08:13:11 +00:00
mistral test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
modelscope test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
mongodb test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
moonshot chore(cost-map): remove models past their deprecation date (#42435) 2026-09-22 21:19:26 +00:00
nadir test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
nebius test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
neosantara test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
nimble test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
novita test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
nscale test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
nvidia_nim test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
nvidia_riva test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
oci fix(params): validate stream_chunk_size once, before any provider call (#43222) 2026-09-26 23:01:20 +00:00
ocr test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
ollama test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
oobabooga test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
openai refactor(decisions): dispatch /v1/decisions through provider configs and the shared HTTP handler (#45130) 2026-10-11 00:37:59 +00:00
openai_like feat(providers): add Cortecs as an OpenAI-compatible provider (#43872) 2026-09-30 19:49:24 -07:00
opencode feat: add litellm.agent() to run claude code, codex, opencode and deep agents through the ai gateway (#43885) 2026-10-01 22:27:49 +00:00
openrouter fix(openrouter): bill responses, decisions and pass-through from usage.cost and always allow reasoning params (#45640) [partial: decisions and pass-through billing only] 2026-10-11 00:42:50 +00:00
ovhcloud test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
parallel_ai test: add __init__.py to phase 12 unit test directories 2026-09-20 12:36:15 +00:00
parasail test: add __init__.py to phase 12 unit test directories 2026-09-20 12:36:15 +00:00
pass_through test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
perplexity test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
pg_vector test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
publicai test: add __init__.py to phase 12 unit test directories 2026-09-20 12:36:15 +00:00
ragflow test: add __init__.py to intermediate phase 12 unit test directories 2026-09-20 12:37:21 +00:00
recraft test: add __init__.py to intermediate phase 12 unit test directories 2026-09-20 12:37:21 +00:00
reducto test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
runwayml test: add __init__.py to phase 12 unit test directories 2026-09-20 12:36:15 +00:00
s3_vectors test: add __init__.py to intermediate phase 12 unit test directories 2026-09-20 12:37:21 +00:00
sagemaker fix(params): validate stream_chunk_size once, before any provider call (#43222) 2026-09-26 23:01:20 +00:00
sail test(e2e): align completion, SAIL, and spend-log fixtures with supported contracts (#43902) 2026-09-30 14:21:22 -07:00
sambanova test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
sap test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
scaleway test: add __init__.py to phase 12 unit test directories 2026-09-20 12:36:15 +00:00
snowflake test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
soniox test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
stability test: add __init__.py to intermediate phase 12 unit test directories 2026-09-20 12:37:21 +00:00
strands_decider feat(strands_decider): call a Strands Decider on an AgentCore runtime ARN (#45678) 2026-10-11 00:47:30 +00:00
tencent refactor(anthropic): rename experimental_pass_through to pass_through (#43329) 2026-09-26 13:00:50 -07:00
tinyfish test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
together_ai test: migrate legacy provider tests to tests/unit (wave 2, phase 13) 2026-09-20 14:18:49 +00:00
valkey test: migrate legacy provider tests to tests/unit (wave 2, phase 13) 2026-09-20 14:18:49 +00:00
vercel_ai_gateway test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
vertex_ai fix(transcription): honor base_url alias for Groq Whisper and report it as the api base (#43917) 2026-09-30 15:03:23 -07:00
volcengine test: stop CI tests from downloading tokenizer files and images (#43257) 2026-09-25 19:27:48 -07:00
voyage test: add __init__.py to every tests/unit directory this migration touches 2026-09-20 11:52:09 +00:00
wandb test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
watsonx test: add __init__.py to every tests/unit directory this migration touches 2026-09-20 11:52:09 +00:00
xai feat(xai): add native xAI batches and files support (#42812) 2026-09-25 15:35:20 -07:00
you_com test: migrate phase 14 wave 2 provider tests to tests/unit 2026-09-20 11:01:40 +00:00
zai test: add __init__.py to every tests/unit directory this migration touches 2026-09-20 11:52:09 +00:00
__init__.py test(unit): add package markers to migrated unit test directories 2026-09-20 11:53:14 +00:00
test_cache_control_and_reasoning.py test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
test_file_content_block.py test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
test_file_search_responses.py test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
test_lifecycle_fix.py test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
test_polling_url_origin_match.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_predibase_transformation.py test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00