`non_default_params.get("reasoning_effort")` returns `Any | None`,
but `REASONING_EFFORT_TO_OUTPUT_CONFIG_EFFORT.get()` expects `str`.
Mypy flagged this on the strict pass. Narrow with `isinstance` before
the lookup; non-strings fall through to the existing `BadRequestError`
below with a clean validation message, so behavior is unchanged.
Fixes a regression introduced by 1a10746e95 in this PR.
The chat completion path (`_apply_output_config`) and the /v1/messages
pass-through (`AnthropicMessagesConfig._translate_reasoning_effort_to_anthropic`)
both gate `max` / `xhigh` per model. The two sites had diverged from
near-identical copies into separately maintained blocks, creating a real
drift risk when a new model tier (e.g. Claude 4.8) lands -- a contributor
could update one site and miss the other.
Centralise the gating in `AnthropicConfig._validate_effort_for_model`,
which returns an error message string or `None`. Each call site keeps
its own provider-appropriate exception type (`BadRequestError` for the
chat path, `AnthropicError` for the /v1/messages pass-through) but the
gating decision now comes from one place. Net -11 LOC.
Adds a parametrised unit test exercising the helper directly across
4.5 / 4.6 / 4.7 model families and `max` / `xhigh` / lower-effort
inputs. Existing tests at both call sites continue to pass unchanged.
Addresses Greptile finding on PR #27074.
`azure_ai` is registered in `litellm.openai_compatible_providers`, so
`add_provider_specific_params_to_optional_params` (litellm/utils.py)
auto-stuffs any non-OpenAI kwarg (e.g. `output_config={"effort": "..."}`)
into `optional_params["extra_body"]`. `AzureAnthropicConfig.transform_request`
then strips `extra_body` entirely on the way out, silently dropping the
param — and `AnthropicConfig._apply_output_config` never sees it, so
`effort="invalid"` / `effort="xhigh"` on a non-supporting model
quietly reaches the model with default behavior instead of returning a
clean 400 (as the native `anthropic` provider does).
Promote the keys back to top-level `optional_params` (using `setdefault`
so explicit top-level values win) before delegating to the parent
`AnthropicConfig`. Apply in both `validate_environment` and
`transform_request` so flag detection (`is_mcp_server_used`, etc.) and
output-config validation both run.
Surfaced by the QA matrix expansion on PR #27074: 20 cells where Azure
returned 200 while `anthropic` returned 400 — all `output_config` mode
across haiku_4_5, sonnet_4_5, opus_4_5, sonnet_4_6, opus_4_6, opus_4_7
families with `effort` in {invalid, xhigh, max, low, medium, high}.
Tests:
* `test_output_config_promoted_from_extra_body`: valid effort reaches data
* `test_invalid_output_config_effort_raises_via_extra_body`: 400 on bad effort
* `test_unsupported_effort_xhigh_raises_via_extra_body`: 400 on xhigh-on-Sonnet-4.6
* `test_extra_body_promotion_does_not_clobber_top_level`: setdefault semantics
The Databricks path called `AnthropicConfig._map_reasoning_effort` for
Claude models but never validated the effort string nor set
`output_config.effort` for adaptive models (Claude 4.6/4.7). Since
`_map_reasoning_effort` returns `type=adaptive` for ANY non-None /
non-"none" string on adaptive models (including "disabled",
"invalid", ""), Databricks silently accepted garbage and emitted a
request without an `output_config.effort`, collapsing every adaptive
tier to identical behavior.
Match the Anthropic native, Bedrock Converse, Bedrock Invoke, and
/v1/messages paths: when the resolved `thinking` is non-None on a
4.6/4.7 model, look up the value in
`REASONING_EFFORT_TO_OUTPUT_CONFIG_EFFORT` and either raise a clean
`BadRequestError` or set `optional_params["output_config"]`.
- claude-sonnet-4-6 + reasoning_effort=max no longer 400s. Renamed
_is_opus_4_6_model to _is_claude_4_6_model at three sites and added
supports_max_reasoning_effort: true to 12 model entries in the JSON
cost map (10 sonnet 4.6 ids + OpenRouter opus 4.6/4.7).
- _map_reasoning_effort now raises BadRequestError(400) directly with
llm_provider, instead of letting Databricks (and similar callers)
surface its raw ValueError as a 500.
- output_config.effort on Opus 4.5 over Bedrock no longer 400s for
missing effort-2025-11-24 beta. Flipped JSON to "effort-2025-11-24"
for bedrock + bedrock_converse and added an auto-attach branch in
_process_tools_and_beta for non-adaptive Anthropic + output_config
on Converse.
- reasoning_effort=xhigh / =max on legacy budget-mode models
(Haiku 4.5, Sonnet 4.5, Opus 4.5) now map to thinking.budget_tokens
8192 / 16384 instead of returning 400. Added two constants in
litellm/constants.py.
Tests updated for all four flips. Validated end-to-end via 306-cell
live proxy matrix (6 model families x 3 routes x 17 effort cases),
all pass.
Replace hardcoded _EFFORT_SUPPORTING_MODEL_PATTERNS with a JSON-backed
check that uses supports_*_reasoning_effort flags from the model map.
Add supports_minimal_reasoning_effort: true to opus-4-5 and mythos-preview
entries (which previously only carried supports_reasoning) so the JSON
remains the single source of truth for effort capability.
The drop_params strip work added `AnthropicConfig._EFFORT_SUPPORTING_MODEL_PATTERNS`
as a private class-level lookup tuple. `BaseConfig.get_config()` only
filtered the `__`-prefixed names plus `_abc` / `_is_base_class`, so
`_EFFORT_SUPPORTING_MODEL_PATTERNS` would have leaked into the request
body the same way `REASONING_EFFORT_TO_OUTPUT_CONFIG_EFFORT` did before
the previous commit.
Generalize the existing `_abc` / `_is_base_class` carve-outs to skip
every `_`-prefixed name. `AmazonConverseConfig.get_config()` overrides
the base method, so apply the same change there.
Also unblocks future internal helpers from accidentally serialising into
the wire body.
When a proxy fronts Claude Code (which always sends `output_config.effort`)
at a pre-4.5 Anthropic model — haiku-3, sonnet-3.5, opus-3, sonnet-4 — the
forwarded knob causes a forced 400 the client can't fix. Gating a strip
behind the existing `drop_params` flag lets operators opt into silent
fixup once and stop worrying about per-model param hygiene.
Default (`drop_params=False`) still forwards and surfaces the provider's
error, preserving the strict, debuggable contract from #27074.
Per https://platform.claude.com/docs/en/build-with-claude/effort the
supporting set is Opus 4.5+, Sonnet 4.6+, and Mythos Preview; everything
else is dropped (with a verbose_logger warning so the strip is visible).
Recognition uses model-name patterns plus a fallback to any
`supports_*_reasoning_effort` flag in the model map for forward
compatibility with new entries.
https://claude.ai/code/session_01WjHq31rvXT6xYNdVmSJvRp
(cherry picked from commit 1233943e78)
The reasoning-effort mapping dict was a public class attribute on
AnthropicConfig, so BaseConfig.get_config returned it as a request
parameter and every Anthropic-backed call (Anthropic / Azure / Vertex /
Bedrock Invoke) hit a 400 'REASONING_EFFORT_TO_OUTPUT_CONFIG_EFFORT:
Extra inputs are not permitted' from the provider. Move the mapping
to a module-level constant.
_supports_effort_level only looked the model up under
custom_llm_provider='anthropic', so bedrock-prefixed model ids
(e.g. bedrock/invoke/us.anthropic.claude-opus-4-7) returned False
for both 'max' and 'xhigh' even when the underlying model entry has
the flag set. Strip known provider prefixes and retry the lookup
against litellm.model_cost directly so per-model gating works on
every route.
Mirror the per-model xhigh/max gate from
AnthropicConfig._apply_output_config in
AnthropicMessagesConfig._translate_reasoning_effort_to_anthropic so
the /v1/messages route also raises a clean 400 instead of forwarding
the unsupported tier.
Both the chat completion path (AnthropicConfig.map_openai_params) and the
Bedrock Converse path (_handle_reasoning_effort_parameter) used
REASONING_EFFORT_TO_OUTPUT_CONFIG_EFFORT.get(value, value) which falls
back to the raw input on unmapped keys. Combined with _map_reasoning_effort
returning type='adaptive' for any string on Claude 4.6/4.7, garbage values
(e.g. 'disabled') could leak into optional_params['output_config']['effort']
unvalidated if map_openai_params ran without the downstream transform_request
or _validate_anthropic_adaptive_effort check.
Mirror the /v1/messages pattern: use .get(value) (no fallback) and raise
BadRequestError immediately when the value is unmapped, co-locating
validation with the mapping for defense in depth.
Closes the remaining QA-sweep gap on PR #27074: Bedrock Invoke
/v1/messages was silently ignoring ``reasoning_effort`` because the
shared param filter only kept native Anthropic keys, so every effort
tier collapsed to the same behavior on the wire (27/231 cells failing
across opus-4-5 / opus-4-6 / sonnet-4-6).
Map ``reasoning_effort`` to native Anthropic ``thinking`` /
``output_config.effort`` at the ``AnthropicMessagesConfig`` layer so
all four /v1/messages routes (direct Anthropic, Azure AI, Vertex AI,
Bedrock Invoke) inherit the same translation:
- Add ``reasoning_effort`` to ``AnthropicMessagesRequestOptionalParams``
so the param filter in
``AnthropicMessagesRequestUtils.get_requested_anthropic_messages_optional_param``
no longer drops it before the transformation runs.
- Add ``_translate_reasoning_effort_to_anthropic`` and call it from
``transform_anthropic_messages_request``. Mirrors
``AnthropicConfig.map_openai_params`` on the chat completion path
(re-uses ``_map_reasoning_effort`` and
``REASONING_EFFORT_TO_OUTPUT_CONFIG_EFFORT``) so the two routes
cannot drift. Pops ``reasoning_effort`` so it never reaches the wire.
- Caller-supplied native ``thinking`` / ``output_config.effort`` always
win — same precedence as
``_translate_legacy_thinking_for_adaptive_model``.
- Garbage values (``""``, ``"disabled"``, ``"invalid"``) raise
``AnthropicError(status_code=400)`` instead of falling through and
surfacing as 500s from the provider.
- ``"none"`` clears thinking + output_config so callers can opt out
per request.
Also restores the non-adaptive-model test coverage on Bedrock Invoke
/v1/messages that the previous commit lost when
``test_bedrock_messages_strips_output_config`` was renamed to the
``forwards`` variant on Opus 4.7.
Adds a new test file
``test_reasoning_effort_translation.py`` covering the translation at
the shared config level (adaptive + non-adaptive models, none, garbage,
caller precedence) so all four /v1/messages routes are exercised by a
single suite.
Adds parametrized + behavioral tests on the Bedrock Invoke /v1/messages
suite covering: minimal/low/medium/high/xhigh/max mapping for adaptive
models, thinking-budget mapping for non-adaptive Opus 4.5, ``none``
clears both, garbage raises 400, explicit ``output_config`` wins.
Refs: https://github.com/BerriAI/litellm/pull/27074
- Remove 'none' from the Bedrock _validate_anthropic_adaptive_effort error
message; it was listed as a valid value but rejected by the membership
check, leaving users in a feedback loop if they tried 'none'.
- Hoist the duplicated reasoning_effort -> output_config.effort mapping
out of AnthropicConfig.map_openai_params and
AmazonConverseConfig._handle_reasoning_effort_parameter into a single
AnthropicConfig.REASONING_EFFORT_TO_OUTPUT_CONFIG_EFFORT class constant
so the two routes cannot drift.
The documentation CI test (tests/documentation_tests/test_env_keys.py)
asserts every os.getenv() key in the source has a matching entry in the
litellm-docs config_settings.md table. ANTHROPIC_MIN_THINKING_BUDGET_TOKENS
tracks Anthropic's published wire-protocol minimum (1024) — it's not a
user-tunable, so making it env-overridable was wrong anyway. Drop the
os.getenv() wrapper; the value is now a plain literal.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
Follow-up bugs surfaced by the QA sweep on PR #27039
(https://github.com/BerriAI/litellm/pull/27039#issuecomment-4363363610).
1. Stop stripping output_config.effort on Bedrock + Vertex adaptive routes.
- Vertex AI Claude 4.6/4.7 accepts output_config.effort on rawPredict
(verified end-to-end against us-east5 / global). The strip helper now
no-ops for effort.
- Bedrock Converse routes output_config into additionalModelRequestFields
for anthropic base models so the requested adaptive tier (low/medium/
high/xhigh/max) actually reaches the wire instead of all collapsing to
identical thinking.
- Bedrock Invoke chat transformation (AmazonAnthropicClaudeConfig) stops
popping output_config from the post-AnthropicConfig request body.
- Bedrock Invoke /v1/messages allowlist (BedrockInvokeAnthropicMessagesRequest)
now lists output_config so the runtime allowlist filter forwards it.
2. Validate effort across Bedrock Converse so 'disabled' / 'invalid' / '' /
unsupported tiers (xhigh/max on Sonnet 4.6 or budget-mode 4.5 models)
surface as a clean 400 BadRequestError instead of 500.
3. ValueError -> BadRequestError throughout (AnthropicConfig.map_openai_params,
_apply_output_config, AmazonConverseConfig._handle_reasoning_effort_parameter).
Empty-string effort is now rejected (was silently passing the
'if effort and ...' short-circuit).
4. Floor reasoning_effort='minimal' at the Anthropic provider minimum
(1024 budget_tokens) via new ANTHROPIC_MIN_THINKING_BUDGET_TOKENS so it's
a usable tier on direct Anthropic / Azure AI Anthropic / Vertex AI Anthropic /
Bedrock Invoke (all of which 400 below 1024).
5. model_prices: dedupe duplicate supports_max_reasoning_effort key on
claude-opus-4-7 / claude-opus-4-7-20260416.
Adds regression tests across all five affected paths; existing tests asserting
the silent-strip behavior were updated to reflect the new pass-through and
clean 400 surfaces.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
Anthropic's main API no longer resolves the non-canonical 'claude-4-sonnet-20250514'
alias for freshly issued keys, returning 404 not_found_error. PR #27031 already
swept three other live tests pinned to this alias to claude-haiku-4-5-20251001
but missed test_multiturn_tool_calls in the responses API suite, which is now
failing reliably on PR CI runs (e.g. PR #27074, job 1603363).
Bump the two model references in test_multiturn_tool_calls to the same
claude-haiku-4-5-20251001 snapshot used by PR #27031 -- it covers everything
this test exercises (tool calling, multi-turn) and isn't on a deprecation
schedule.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
Setting reasoning_effort="none" on Anthropic chat models (direct, Bedrock
Invoke, Bedrock Converse, Vertex AI Anthropic, Azure AI Anthropic) crashed
LiteLLM with:
litellm.APIConnectionError: 'NoneType' object has no attribute 'get'
Both the Anthropic chat transformation and Bedrock Converse called
``AnthropicConfig._map_reasoning_effort`` and assigned the ``None`` it returns
for ``"none"`` directly to ``optional_params["thinking"]``. Downstream
``is_thinking_enabled`` then did ``optional_params["thinking"].get("type")``
and crashed.
Pop ``thinking`` (and on Claude 4.6/4.7, ``output_config``) instead of
assigning ``None``, restoring the documented contract that
``reasoning_effort="none"`` means "do not enable thinking". This also
prevents downstream Anthropic 400s ("thinking: Input should be an object",
"output_config.effort: Input should be ...") if the bug were ever masked.
Verified end-to-end against the live Anthropic API and Bedrock Converse
on claude-opus-4-{5,6,7} and claude-sonnet-4-6, plus Bedrock Invoke for
Claude 4.5/4.6. Vertex AI Anthropic and Azure AI Anthropic inherit the
fixed ``map_openai_params`` from ``AnthropicConfig`` and need no further
changes.
Adds litellm.disable_vertex_batch_output_transformation (default False).
When True, afile_content returns raw Vertex predictions.jsonl untouched
so users that parse candidates/modelVersion directly are not broken.
The method was overwriting logging_obj.optional_params, logging_obj.model,
and logging_obj.start_time on the caller's Logging instance. When invoked
from llm_http_handler.py's generic framework path, the framework's own
logging_obj (which already went through pre_call) had its properties
clobbered, causing model and start_time to reflect the last batch line's
values rather than the original call context.
Fix: create a fresh local Logging instance for the per-line transformation
instead of mutating the incoming logging_obj. The caller's object is now
left entirely untouched regardless of whether a logging_obj was passed in
or not.
Regression tests added to verify model, start_time, and optional_params
are not mutated on the caller's logging_obj.
Co-authored-by: Sameer Kankute <Sameerlite@users.noreply.github.com>
- Remove unused _encode_gcp_label_value / _decode_gcp_label_value singular
helpers; only the _chunks variants are actually called.
- Use 'is not None' check for custom_id so empty-string custom_ids are
still labeled and round-trip through batch outputs.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
Newer Cloudflare Workers AI models (e.g. Nemotron) emit 'response_text'
instead of 'response' on streamed chunks. The non-streaming path was
already updated to fall back to 'response_text' (#26385), but the
streaming chunk parser still only read 'response', which caused
streaming requests against those models to silently produce empty
content.
Mirror the non-streaming fallback in CloudflareChatResponseIterator.chunk_parser
and add a streaming test for the response_text shape.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
base_anthropic_messages_test.test_anthropic_messages_with_thinking and
test_anthropic_streaming_with_thinking still pinned to
claude-4-sonnet-20250514 — the same legacy alias Anthropic no longer
recognizes under freshly issued keys. The other four tests in this base
class already use claude-sonnet-4-5-20250929; these two were missed.
Bump to claude-haiku-4-5-20251001 (supports_reasoning=true, no upcoming
deprecation). Subclasses including TestAnthropicPassthroughBasic
inherit these methods.
test_anthropic_messages_streaming_cost_injection hits the proxy's
/v1/messages route, which routes via the anthropic/* wildcard to
api.anthropic.com. The 404 surfaced in the test was Anthropic's own
not_found_error propagated back through the proxy (visible from the
x-litellm-model-id hash on the response — the proxy did route).
Same root cause as the prior commit: the legacy claude-4-sonnet-20250514
alias is no longer recognized by Anthropic's main API under the new key.
Swap to claude-haiku-4-5-20251001 — same routing path, canonical model.
Three live-API tests pinned to claude-4-sonnet-20250514, which is a
non-canonical alias of claude-sonnet-4-20250514. Anthropic's main API
no longer resolves the legacy form under freshly issued keys, so the
tests fail with not_found_error. The token counter test pinned to
claude-sonnet-4-20250514 itself (deprecation_date 2026-05-14, two weeks
out) was on borrowed time too.
Bump all four to claude-haiku-4-5-20251001 — capability superset for what
these tests exercise (streaming, parallel tool calling, extended thinking,
token counting), no upcoming deprecation, cheaper per-token.
Mirrors the membership rule on /key/update so that /key/generate and
/key/{key}/regenerate apply the same `_validate_caller_can_assign_key_org`
gate when the caller specifies an `organization_id`. Proxy admins bypass.
The check no-ops when `organization_id` is not being set.
