scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-08-28 05:25:59 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 36

feat: add support for claude code goal mode for bedrock opus output config (#28898)

Browse source 
* feat: support goal mode for claude on bedrock

* fix failing lint test

* addressing greptile comments

* fixing failed test

* address greptile: copy output_config and warn on dropped converse format

* fix(bedrock): skip redundant output_config normalization on Converse reasoning_effort path

When reasoning_effort is mapped via _handle_reasoning_effort_parameter, the
resulting output_config is already normalized via
normalize_bedrock_opus_output_config_effort. Mark it as normalized so
_prepare_request_params can skip the redundant call (and the associated
get_model_info lookup) on every request.

Co-authored-by: Yassin Kortam <yassin@berri.ai>

* test(reasoning-effort-grid): reflect Bedrock opus-4-6 xhigh→max clamping

* fix(bedrock): stop leaking output_config marker and message-content mutation

* fix(bedrock): guard effort key access in normalize_bedrock_opus_output_config_effort

Defensively check that 'effort' is a valid key in _BEDROCK_OUTPUT_CONFIG_EFFORT_ORDER
before indexing, to prevent a KeyError if the hardcoded guard tuple ever drifts from
the order dict's keys.

Co-authored-by: Yassin Kortam <yassin@berri.ai>

* fix(bedrock): drop dead second clause in effort normalization guard

The 'effort not in _BEDROCK_OUTPUT_CONFIG_EFFORT_ORDER' check is
unreachable once 'effort not in ("xhigh", "max")' has been ruled out,
since both literals are present in the order dict. Keep the literal
membership check and let the dict lookups below speak for themselves.

* fix(bedrock): clamp output_config.effort against ceiling for any known value

The early return when effort was not 'xhigh'/'max' meant a ceiling of
'low' or 'medium' would silently forward an out-of-range value. Gate on
the known effort ordering instead so the ceiling comparison runs for
every recognized effort.

* test(grid_spec): use _CAPS_OPUS_4_7 for non-Bedrock opus-4-6 entries

claude-opus-4-6 now declares supports_xhigh_reasoning_effort in the model
map, so production accepts xhigh on Azure AI and Vertex AI routes. Update
those grid_spec entries to match production capabilities so expected()
predicts 200 for xhigh instead of 400.

Co-authored-by: Yassin Kortam <yassin@berri.ai>

* test(grid_spec): revert xhigh caps for non-Bedrock opus-4-6

azure_ai/claude-opus-4-6 and vertex_ai/claude-opus-4-6 do not declare
supports_xhigh_reasoning_effort in model_prices_and_context_window.json.
Azure AI upstream rejects xhigh with HTTP 400 ("Supported levels: high,
low, max, medium"). Restore _CAPS_4_6 so the grid predicts 400 for
xhigh, matching production capabilities.

* fix: stop advertising xhigh effort on Opus 4.5/4.6

Only Opus 4.7 supports the xhigh reasoning effort level. Remove the
supports_xhigh_reasoning_effort flag from every Opus 4.5 and Opus 4.6
entry (direct Anthropic, Bedrock, and regional variants) in both model
catalog files.

On the direct Anthropic path there is no effort clamp, so flagging 4.5/4.6
as xhigh-capable caused litellm to forward xhigh to a model that rejects it
(and made get_model_info misreport the capability). xhigh now correctly
degrades to high / raises on those models.

Bedrock graceful degradation for Claude Code goal mode is unaffected: it
relies solely on the bedrock_output_config_effort_ceiling clamp (4.5->high,
4.6->max, 4.7->xhigh), which runs before validation, so xhigh requests to
older Bedrock Opus models are still silently lowered rather than rejected.

Update effort-gating tests to reflect that 4.5/4.6 no longer accept xhigh.

* fix: clamp xhigh effort on Bedrock Invoke /v1/messages instead of rejecting

Claude Code "goal mode" sends output_config.effort=xhigh over the Anthropic
/v1/messages API, which routes Bedrock models through
AmazonAnthropicClaudeMessagesConfig. That path validated effort against the
model's native capability and raised 400 for xhigh on Opus 4.6, while the
chat-completions paths (Converse + Invoke) already clamp xhigh to the model's
bedrock_output_config_effort_ceiling. That asymmetry broke goal mode on the
exact API surface Claude Code uses.

Apply the same ceiling clamp on the messages path before the shared effort
gate runs, so xhigh degrades to max on Opus 4.6 (and stays xhigh on 4.7).
Scoped to adaptive-thinking models and to models that declare a ceiling, so
Sonnet 4.6 (no ceiling) and Opus 4.5 (budget mode) are unaffected and still
reject xhigh.

* fix(bedrock): preserve user output_config when applying reasoning_effort

- Converse path: merge mapped effort into existing output_config via
  setdefault instead of overwriting it, matching the Anthropic Messages
  path. Prevents user-supplied output_config.format from being silently
  dropped when reasoning_effort is also provided.
- tests: clear _get_local_model_cost_map lru_cache in the autouse
  fixture alongside get_bedrock_response_stream_shape to avoid stale
  cache leakage between tests.

Co-authored-by: Yassin Kortam <yassin@berri.ai>

* fix(bedrock): pre-clamp reasoning_effort for chat invoke; correct test caps

- Add _clamp_adaptive_reasoning_effort_for_bedrock to AmazonAnthropicClaudeConfig
  so raw reasoning_effort=xhigh degrades to the model's bedrock effort ceiling
  before AnthropicConfig.map_openai_params converts it to output_config.
  Mirrors converse path (_handle_reasoning_effort_parameter) and messages path
  (_clamp_adaptive_reasoning_effort_for_bedrock) so the three Bedrock paths
  are consistent.

- grid_spec: restore caps=_CAPS_4_6 for Bedrock converse/invoke Opus 4.6 entries
  so the test reflects the model's actual JSON capabilities. Teach expected()
  to bypass the xhigh/max cap check when bedrock_effort_ceiling will clamp
  the wire effort, so the test still passes for Bedrock's graceful degradation
  contract without lying about native model caps.

Co-authored-by: Yassin Kortam <yassin@berri.ai>

---------

Co-authored-by: Dennis Henry <dennis.henry@okta.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Yassin Kortam <yassin@berri.ai>
This commit is contained in:
Mateo Wang 2026-05-28 09:14:57 -07:00 • committed by GitHub
parent 5699a06413
commit 95015de733
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
20 changed files with 1068 additions and 217 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

5
litellm/llms/anthropic/chat/transformation.py
View file

@ -1793,7 +1793,10 @@ class AnthropicConfig(AnthropicModelInfo, BaseConfig):
self._ensure_context_management_beta_header(
headers, optional_params["context_management"]
)
if optional_params.get("output_format") is not None:
output_config = optional_params.get("output_config")
if optional_params.get("output_format") is not None or (
isinstance(output_config, dict) and output_config.get("format") is not None
):
self._ensure_beta_header(
headers, ANTHROPIC_BETA_HEADER_VALUES.STRUCTURED_OUTPUT_2025_09_25.value
)

9
litellm/llms/anthropic/experimental_pass_through/messages/transformation.py
View file

@ -427,8 +427,13 @@ class AnthropicMessagesConfig(BaseAnthropicMessagesConfig):
ANTHROPIC_BETA_HEADER_VALUES.CONTEXT_MANAGEMENT_2025_06_27.value
)
# Check for structured outputs
if optional_params.get("output_format") is not None:
# Check for structured outputs. Anthropic's newer request shape nests
# the schema under output_config.format; the older top-level
# output_format remains supported for backwards compatibility.
output_config = optional_params.get("output_config")
if optional_params.get("output_format") is not None or (
isinstance(output_config, dict) and output_config.get("format") is not None
):
beta_values.add(
ANTHROPIC_BETA_HEADER_VALUES.STRUCTURED_OUTPUT_2025_09_25.value
)

60
litellm/llms/bedrock/chat/converse_transformation.py
View file

@ -78,6 +78,7 @@ from ..common_utils import (
get_anthropic_beta_from_headers,
get_bedrock_tool_name,
is_claude_4_5_on_bedrock,
normalize_bedrock_opus_output_config_effort,
)
# Computer use tool prefixes supported by Bedrock
@ -448,10 +449,20 @@ class AmazonConverseConfig(BaseConfig):
value=reasoning_effort,
llm_provider="bedrock_converse",
)
existing_output_config = optional_params.get("output_config")
if not isinstance(existing_output_config, dict):
existing_output_config = {}
existing_output_config.setdefault("effort", mapped_effort)
normalize_bedrock_opus_output_config_effort(
model=model,
output_config=existing_output_config,
)
mapped_effort = existing_output_config["effort"]
self._validate_anthropic_adaptive_effort(
model=model, effort=mapped_effort
)
optional_params["output_config"] = {"effort": mapped_effort}
optional_params["output_config"] = existing_output_config
optional_params["_output_config_normalized"] = True
@staticmethod
def _validate_anthropic_adaptive_effort(model: str, effort: str) -> None:
@ -1201,6 +1212,12 @@ class AmazonConverseConfig(BaseConfig):
self, optional_params: dict, model: str
) -> Tuple[dict, dict, dict, Optional[OutputConfigBlock]]:
"""Prepare and separate request parameters."""
# Consume the internal ``_output_config_normalized`` marker set by
# ``_handle_reasoning_effort_parameter`` so it does not linger on the
# caller's ``optional_params`` after the transformation returns.
anthropic_output_config_already_normalized = bool(
optional_params.pop("_output_config_normalized", False)
)
# Filter out exception objects before deepcopy to prevent deepcopy failures
# Exceptions should not be stored in optional_params (this is a defensive fix)
cleaned_params = filter_exceptions_from_params(optional_params)
@ -1219,8 +1236,17 @@ class AmazonConverseConfig(BaseConfig):
# Anthropic-only ``output_config`` (snake_case) — re-attached to
# ``additionalModelRequestFields`` for Anthropic models below. The
# Bedrock-native ``outputConfig`` (camelCase) is handled separately.
# structured-output ``format`` subfield is consumed into Bedrock's
# native ``outputConfig`` (camelCase), which is handled separately.
anthropic_output_config = inference_params.pop("output_config", None)
output_config_format = None
if isinstance(anthropic_output_config, dict):
anthropic_output_config = dict(anthropic_output_config)
candidate_output_config_format = anthropic_output_config.pop("format", None)
if isinstance(candidate_output_config_format, dict):
output_config_format = candidate_output_config_format
if not anthropic_output_config:
anthropic_output_config = None
# Extract requestMetadata before processing other parameters
request_metadata = inference_params.pop("requestMetadata", None)
@ -1230,6 +1256,30 @@ class AmazonConverseConfig(BaseConfig):
output_config: Optional[OutputConfigBlock] = inference_params.pop(
"outputConfig", None
)
base_model = BedrockModelInfo.get_base_model(model)
if (
output_config is None
and output_config_format is not None
and output_config_format.get("type") == "json_schema"
and base_model.startswith("anthropic")
and self._supports_native_structured_outputs(
model, self.custom_llm_provider
)
):
output_config = self._create_output_config_for_response_format(
json_schema=output_config_format.get("schema"),
name=output_config_format.get("name"),
description=output_config_format.get("description"),
)
elif output_config is None and output_config_format is not None:
litellm.verbose_logger.warning(
"Bedrock Converse: dropping `output_config.format` for model=%s — "
"model does not advertise `supports_native_structured_output` in "
"model_prices_and_context_window.json. The schema will not be "
"enforced; pass `response_format` to use the synthetic tool-call "
"fallback.",
model,
)
# keep supported params in 'inference_params', and set all model-specific params in 'additional_request_params'
additional_request_params = {
@ -1275,7 +1325,6 @@ class AmazonConverseConfig(BaseConfig):
if anthropic_output_config is not None and isinstance(
anthropic_output_config, dict
):
base_model = BedrockModelInfo.get_base_model(model)
if base_model.startswith("anthropic"):
if (
litellm.drop_params is True
@ -1286,6 +1335,11 @@ class AmazonConverseConfig(BaseConfig):
model,
)
else:
if not anthropic_output_config_already_normalized:
normalize_bedrock_opus_output_config_effort(
model=model,
output_config=anthropic_output_config,
)
effort = anthropic_output_config.get("effort")
if effort is not None:
self._validate_anthropic_adaptive_effort(

57
litellm/llms/bedrock/chat/invoke_transformations/anthropic_claude3_transformation.py
View file

@ -16,8 +16,11 @@ from litellm.llms.bedrock.chat.invoke_transformations.base_invoke_transformation
AmazonInvokeConfig,
)
from litellm.llms.bedrock.common_utils import (
convert_bedrock_invoke_output_format_to_inline_schema,
get_anthropic_beta_from_headers,
normalize_bedrock_opus_output_config_effort,
normalize_tool_input_schema_types_for_bedrock_invoke,
pop_bedrock_invoke_output_config_format,
remove_custom_field_from_tools,
)
from litellm.types.llms.anthropic import ANTHROPIC_TOOL_SEARCH_BETA_HEADER
@ -75,6 +78,17 @@ class AmazonAnthropicClaudeConfig(AmazonInvokeConfig, AnthropicConfig):
# Use a model name that forces tool-based approach
model = "claude-3-sonnet-20240229"
# Clamp ``reasoning_effort`` to the Bedrock effort ceiling before the
# parent mapping converts it to ``output_config.effort`` and the
# downstream effort gate runs. Mirrors the converse path's
# ``_handle_reasoning_effort_parameter`` and the messages path's
# ``_clamp_adaptive_reasoning_effort_for_bedrock`` so adaptive Claude
# requests degrade ``xhigh`` -> ``max`` rather than 400-ing on
# models like Opus 4.6 that don't natively advertise xhigh.
self._clamp_adaptive_reasoning_effort_for_bedrock(
model=original_model, params=non_default_params
)
optional_params = AnthropicConfig.map_openai_params(
self,
non_default_params,
@ -88,6 +102,27 @@ class AmazonAnthropicClaudeConfig(AmazonInvokeConfig, AnthropicConfig):
return optional_params
@staticmethod
def _clamp_adaptive_reasoning_effort_for_bedrock(model: str, params: dict) -> None:
"""Lower ``reasoning_effort`` to the Bedrock effort ceiling before mapping.
Bedrock's adaptive Claude models accept the OpenAI-style
``reasoning_effort`` tier, but the request validator can reject tiers
the model does not natively advertise (e.g. ``xhigh`` on Opus 4.6).
Clamp the raw tier to the model's
``bedrock_output_config_effort_ceiling`` so Claude Code "goal mode"
keeps working. Non-adaptive models and models without a ceiling are
left untouched.
"""
if not AnthropicConfig._is_adaptive_thinking_model(model):
return
effort = params.get("reasoning_effort")
if not isinstance(effort, str):
return
clamped = {"effort": effort}
normalize_bedrock_opus_output_config_effort(model=model, output_config=clamped)
params["reasoning_effort"] = clamped["effort"]
def transform_request(
self,
model: str,
@ -157,6 +192,13 @@ class AmazonAnthropicClaudeConfig(AmazonInvokeConfig, AnthropicConfig):
for k, v in optional_params.items()
if k not in self.aws_authentication_params
}
output_config = filtered_params.get("output_config")
if isinstance(output_config, dict):
filtered_params["output_config"] = dict(output_config)
normalize_bedrock_opus_output_config_effort(
model=model,
output_config=filtered_params["output_config"],
)
filtered_params = self._normalize_bedrock_tool_search_tools(filtered_params)
anthropic_request = AnthropicConfig.transform_request(
@ -170,7 +212,20 @@ class AmazonAnthropicClaudeConfig(AmazonInvokeConfig, AnthropicConfig):
anthropic_request.pop("model", None)
anthropic_request.pop("stream", None)
anthropic_request.pop("output_format", None)
output_format = anthropic_request.pop("output_format", None)
output_config_format = pop_bedrock_invoke_output_config_format(
anthropic_request
)
if output_format:
convert_bedrock_invoke_output_format_to_inline_schema(
output_format=output_format,
request_body=anthropic_request,
)
elif output_config_format:
convert_bedrock_invoke_output_format_to_inline_schema(
output_format=output_config_format,
request_body=anthropic_request,
)
if not (
_supports_factory(
model=model,

138
litellm/llms/bedrock/common_utils.py
View file

@ -34,6 +34,15 @@ class BedrockError(BaseLLMException):
# Lazy import cache to avoid circular imports and performance impact
_get_model_info = None
BedrockOutputConfigEffort = Literal["low", "medium", "high", "max", "xhigh"]
_BEDROCK_OUTPUT_CONFIG_EFFORT_ORDER: Dict[BedrockOutputConfigEffort, int] = {
"low": 0,
"medium": 1,
"high": 2,
"max": 3,
"xhigh": 4,
}
def get_cached_model_info():
"""
@ -51,6 +60,79 @@ def get_cached_model_info():
return _get_model_info
@functools.lru_cache(maxsize=1)
def _get_local_model_cost_map() -> Dict:
from litellm.litellm_core_utils.get_model_cost_map import GetModelCostMap
return GetModelCostMap.load_local_model_cost_map()
def pop_bedrock_invoke_output_config_format(request_body: Dict) -> Optional[Dict]:
"""
Remove and return Anthropic's nested ``output_config.format`` field.
Bedrock Invoke paths convert the schema to inline message text. Any remaining
``output_config`` keys, such as ``effort``, are left in place.
"""
output_config = request_body.get("output_config")
if not isinstance(output_config, dict):
return None
output_format = output_config.pop("format", None)
if not output_config:
request_body.pop("output_config", None)
if isinstance(output_format, dict):
return output_format
return None
def convert_bedrock_invoke_output_format_to_inline_schema(
output_format: Dict,
request_body: Dict,
) -> None:
"""
Embed an Anthropic structured-output schema into the last user message.
Bedrock Invoke does not support ``output_format`` directly, so the schema is
appended to the final user message for prompt-engineered structured output.
The caller's ``messages`` list, message dict, and content list are not
mutated; a fresh ``messages`` list with a copied final user message is
written back to ``request_body``.
"""
schema = output_format.get("schema")
if not schema:
return
messages = request_body.get("messages")
if not isinstance(messages, list) or not messages:
return
last_user_idx = None
for i in range(len(messages) - 1, -1, -1):
message = messages[i]
if isinstance(message, dict) and message.get("role") == "user":
last_user_idx = i
break
if last_user_idx is None:
return
original = messages[last_user_idx]
content = original.get("content", [])
schema_block = {"type": "text", "text": json.dumps(schema)}
if isinstance(content, str):
new_content = [{"type": "text", "text": content}, schema_block]
elif isinstance(content, list):
new_content = [*content, schema_block]
else:
return
new_messages = list(messages)
new_messages[last_user_idx] = {**original, "content": new_content}
request_body["messages"] = new_messages
def remove_custom_field_from_tools(request_body: dict) -> None:
"""
Remove ``custom`` field from each tool in the request body.
@ -603,6 +685,62 @@ def is_claude_4_5_on_bedrock(model: str) -> bool:
return any(pattern in model_lower for pattern in claude_4_5_patterns)
def normalize_bedrock_opus_output_config_effort(model: str, output_config: Any) -> None:
"""
Normalize Anthropic ``output_config.effort`` values for Bedrock Opus ids.
Bedrock's Claude Opus request validator can accept a narrower effort
vocabulary than Anthropic's compatibility surface. The Bedrock ceiling is
read from ``model_prices_and_context_window.json`` via
``bedrock_output_config_effort_ceiling``.
Mutates ``output_config`` in place so callers can accept Claude Code's
``xhigh`` input without forwarding a provider-invalid value.
"""
if not isinstance(output_config, dict):
return
effort = output_config.get("effort")
if effort not in _BEDROCK_OUTPUT_CONFIG_EFFORT_ORDER:
return
ceiling = _get_bedrock_output_config_effort_ceiling(model)
if ceiling is None:
return
if (
_BEDROCK_OUTPUT_CONFIG_EFFORT_ORDER[effort]
> _BEDROCK_OUTPUT_CONFIG_EFFORT_ORDER[ceiling]
):
output_config["effort"] = ceiling
def _get_bedrock_output_config_effort_ceiling(
model: str,
) -> Optional[BedrockOutputConfigEffort]:
try:
model_info = get_cached_model_info()(
model=model,
custom_llm_provider="bedrock",
)
except Exception:
return None
ceiling = model_info.get("bedrock_output_config_effort_ceiling")
if isinstance(ceiling, str) and ceiling in _BEDROCK_OUTPUT_CONFIG_EFFORT_ORDER:
return ceiling # type: ignore[return-value]
model_cost_key = model_info.get("key")
if not isinstance(model_cost_key, str):
return None
local_model_info = _get_local_model_cost_map().get(model_cost_key, {})
ceiling = local_model_info.get("bedrock_output_config_effort_ceiling")
if isinstance(ceiling, str) and ceiling in _BEDROCK_OUTPUT_CONFIG_EFFORT_ORDER:
return ceiling # type: ignore[return-value]
return None
# Import after standalone functions to avoid circular imports
from litellm.llms.bedrock.count_tokens.bedrock_token_counter import BedrockTokenCounter

308
litellm/llms/bedrock/messages/invoke_transformations/anthropic_claude3_transformation.py
View file

@ -32,10 +32,13 @@ from litellm.llms.bedrock.chat.invoke_transformations.base_invoke_transformation
AmazonInvokeConfig,
)
from litellm.llms.bedrock.common_utils import (
convert_bedrock_invoke_output_format_to_inline_schema,
ensure_bedrock_anthropic_messages_tool_names,
get_anthropic_beta_from_headers,
is_claude_4_5_on_bedrock,
normalize_bedrock_opus_output_config_effort,
normalize_tool_input_schema_types_for_bedrock_invoke,
pop_bedrock_invoke_output_config_format,
remove_custom_field_from_tools,
)
from litellm.types.llms.anthropic import ANTHROPIC_TOOL_SEARCH_BETA_HEADER
@ -450,145 +453,15 @@ class AmazonAnthropicClaudeMessagesConfig(
else:
anthropic_messages_request.pop("context_management", None)
def _convert_output_format_to_inline_schema(
self,
output_format: Dict,
anthropic_messages_request: Dict,
) -> None:
"""
Convert Anthropic output_format to inline schema in message content.
Bedrock Invoke doesn't support the output_format parameter, so we embed
the schema directly into the user message content as text instructions.
This approach adds the schema to the last user message, instructing the model
to respond in the specified JSON format.
Args:
output_format: The output_format dict with 'type' and 'schema'
anthropic_messages_request: The request dict to modify in-place
Ref: https://aws.amazon.com/blogs/machine-learning/structured-data-response-with-amazon-bedrock-prompt-engineering-and-tool-use/
"""
import json
# Extract schema from output_format
schema = output_format.get("schema")
if not schema:
return
# Get messages from the request
messages = anthropic_messages_request.get("messages", [])
if not messages:
return
# Find the last user message
last_user_message_idx = None
for idx in range(len(messages) - 1, -1, -1):
if messages[idx].get("role") == "user":
last_user_message_idx = idx
break
if last_user_message_idx is None:
return
last_user_message = messages[last_user_message_idx]
content = last_user_message.get("content", [])
# Ensure content is a list
if isinstance(content, str):
content = [{"type": "text", "text": content}]
last_user_message["content"] = content
# Add schema as text content to the message
schema_text = {"type": "text", "text": json.dumps(schema)}
content.append(schema_text)
def transform_anthropic_messages_request(
def _get_bedrock_invoke_anthropic_beta_headers(
self,
model: str,
messages: List[Dict],
anthropic_messages_optional_request_params: Dict,
litellm_params: GenericLiteLLMParams,
headers: dict,
) -> Dict:
anthropic_messages_request = AnthropicMessagesConfig.transform_anthropic_messages_request(
self=self,
model=model,
messages=messages,
anthropic_messages_optional_request_params=anthropic_messages_optional_request_params,
litellm_params=litellm_params,
headers=headers,
)
#########################################################
############## BEDROCK Invoke SPECIFIC TRANSFORMATION ###
#########################################################
# 1. anthropic_version is required for all claude models
if "anthropic_version" not in anthropic_messages_request:
anthropic_messages_request["anthropic_version"] = (
self.DEFAULT_BEDROCK_ANTHROPIC_API_VERSION
)
# 2. `stream` is not allowed in request body for bedrock invoke
if "stream" in anthropic_messages_request:
anthropic_messages_request.pop("stream", None)
# 3. `model` is not allowed in request body for bedrock invoke
if "model" in anthropic_messages_request:
anthropic_messages_request.pop("model", None)
injected_thinking_for_clear_thinking = (
self._ensure_thinking_for_clear_thinking_context_management(
anthropic_messages_request=anthropic_messages_request,
model=model,
)
)
# 4. Remove `ttl` field from cache_control in messages (Bedrock doesn't support it for older models)
self._remove_ttl_from_cache_control(
anthropic_messages_request=anthropic_messages_request, model=model
)
# 5. Convert `output_format` to inline schema (Bedrock invoke doesn't support output_format)
output_format = anthropic_messages_request.pop("output_format", None)
if output_format:
self._convert_output_format_to_inline_schema(
output_format=output_format,
anthropic_messages_request=anthropic_messages_request,
)
# 5a. Bedrock Invoke supports output_config (effort) for Claude 4.6+ models,
# but older models do not — strip it to avoid request rejection.
# Ref: https://github.com/BerriAI/litellm/issues/22797
if not (
_supports_factory(
model=model,
custom_llm_provider="bedrock",
key="supports_output_config",
)
or AnthropicConfig._model_supports_effort_param(model)
):
if anthropic_messages_request.pop("output_config", None) is not None:
verbose_logger.warning(
"Bedrock Invoke: stripping unsupported `output_config` for "
"model=%s — neither `supports_output_config` nor any "
"`supports_*_reasoning_effort` flag is set in "
"model_prices_and_context_window.json. Add the capability "
"flag to the model JSON entry if this model accepts "
"`output_config`.",
model,
)
# 5b. Remove `custom` field from tools (Bedrock doesn't support it)
# Claude Code sends `custom: {defer_loading: true}` on tool definitions,
# which causes Bedrock to reject the request with "Extra inputs are not permitted"
# Ref: https://github.com/BerriAI/litellm/issues/22847
remove_custom_field_from_tools(anthropic_messages_request)
normalize_tool_input_schema_types_for_bedrock_invoke(anthropic_messages_request)
ensure_bedrock_anthropic_messages_tool_names(anthropic_messages_request)
# 6. AUTO-INJECT beta headers based on features used
anthropic_messages_request: Dict,
injected_thinking_for_clear_thinking: bool,
) -> List[str]:
anthropic_model_info = AnthropicModelInfo()
tools = anthropic_messages_optional_request_params.get("tools")
messages_typed = cast(List[AllMessageValues], messages)
@ -651,6 +524,160 @@ class AmazonAnthropicClaudeMessagesConfig(
dropped_user_betas,
)
return filtered_betas
def _strip_unsupported_bedrock_invoke_fields(
self,
anthropic_messages_request: Dict,
) -> Dict:
allowed = self.BEDROCK_INVOKE_ALLOWED_TOP_LEVEL_FIELDS
stripped = sorted(k for k in anthropic_messages_request if k not in allowed)
if stripped:
verbose_logger.debug(
"Bedrock Invoke: stripping unsupported top-level request fields: %s",
stripped,
)
return {k: v for k, v in anthropic_messages_request.items() if k in allowed}
@staticmethod
def _clamp_adaptive_reasoning_effort_for_bedrock(
model: str, optional_params: Dict
) -> None:
"""Lower ``reasoning_effort`` to the Bedrock effort ceiling before validation.
The shared ``/v1/messages`` effort gate rejects tiers a model does not
natively support (e.g. ``xhigh`` on Opus 4.6). Bedrock's chat paths instead
clamp the tier to the model's ``bedrock_output_config_effort_ceiling`` so
Claude Code "goal mode" keeps working; mirror that here so the messages
path degrades ``xhigh`` -> ``max`` rather than 400-ing. Non-adaptive models
and models without a ceiling are left untouched.
"""
if not AnthropicModelInfo._is_adaptive_thinking_model(model):
return
effort = optional_params.get("reasoning_effort")
if not isinstance(effort, str):
return
clamped = {"effort": effort}
normalize_bedrock_opus_output_config_effort(model=model, output_config=clamped)
optional_params["reasoning_effort"] = clamped["effort"]
def transform_anthropic_messages_request(
self,
model: str,
messages: List[Dict],
anthropic_messages_optional_request_params: Dict,
litellm_params: GenericLiteLLMParams,
headers: dict,
) -> Dict:
self._clamp_adaptive_reasoning_effort_for_bedrock(
model=model,
optional_params=anthropic_messages_optional_request_params,
)
anthropic_messages_request = AnthropicMessagesConfig.transform_anthropic_messages_request(
self=self,
model=model,
messages=messages,
anthropic_messages_optional_request_params=anthropic_messages_optional_request_params,
litellm_params=litellm_params,
headers=headers,
)
#########################################################
############## BEDROCK Invoke SPECIFIC TRANSFORMATION ###
#########################################################
# 1. anthropic_version is required for all claude models
if "anthropic_version" not in anthropic_messages_request:
anthropic_messages_request["anthropic_version"] = (
self.DEFAULT_BEDROCK_ANTHROPIC_API_VERSION
)
# 2. `stream` is not allowed in request body for bedrock invoke
if "stream" in anthropic_messages_request:
anthropic_messages_request.pop("stream", None)
# 3. `model` is not allowed in request body for bedrock invoke
if "model" in anthropic_messages_request:
anthropic_messages_request.pop("model", None)
injected_thinking_for_clear_thinking = (
self._ensure_thinking_for_clear_thinking_context_management(
anthropic_messages_request=anthropic_messages_request,
model=model,
)
)
# 4. Remove `ttl` field from cache_control in messages (Bedrock doesn't support it for older models)
self._remove_ttl_from_cache_control(
anthropic_messages_request=anthropic_messages_request, model=model
)
# 5. Convert structured-output params to inline schema.
# Bedrock Invoke doesn't support top-level `output_format`; its
# accepted `output_config` subset is also narrower than Anthropic's, so
# consume the newer `output_config.format` shape here instead of
# forwarding it as an unknown nested key.
existing_output_config = anthropic_messages_request.get("output_config")
if isinstance(existing_output_config, dict):
anthropic_messages_request["output_config"] = dict(existing_output_config)
output_format = anthropic_messages_request.pop("output_format", None)
output_config_format = pop_bedrock_invoke_output_config_format(
anthropic_messages_request
)
if output_format:
convert_bedrock_invoke_output_format_to_inline_schema(
output_format=output_format,
request_body=anthropic_messages_request,
)
elif output_config_format:
convert_bedrock_invoke_output_format_to_inline_schema(
output_format=output_config_format,
request_body=anthropic_messages_request,
)
normalize_bedrock_opus_output_config_effort(
model=model,
output_config=anthropic_messages_request.get("output_config"),
)
# 5a. Bedrock Invoke supports output_config (effort) for Claude 4.6+ models,
# but older models do not — strip it to avoid request rejection.
# Ref: https://github.com/BerriAI/litellm/issues/22797
if not (
_supports_factory(
model=model,
custom_llm_provider="bedrock",
key="supports_output_config",
)
or AnthropicConfig._model_supports_effort_param(model)
):
if anthropic_messages_request.pop("output_config", None) is not None:
verbose_logger.warning(
"Bedrock Invoke: stripping unsupported `output_config` for "
"model=%s — neither `supports_output_config` nor any "
"`supports_*_reasoning_effort` flag is set in "
"model_prices_and_context_window.json. Add the capability "
"flag to the model JSON entry if this model accepts "
"`output_config`.",
model,
)
# 5b. Remove `custom` field from tools (Bedrock doesn't support it)
# Claude Code sends `custom: {defer_loading: true}` on tool definitions,
# which causes Bedrock to reject the request with "Extra inputs are not permitted"
# Ref: https://github.com/BerriAI/litellm/issues/22847
remove_custom_field_from_tools(anthropic_messages_request)
normalize_tool_input_schema_types_for_bedrock_invoke(anthropic_messages_request)
ensure_bedrock_anthropic_messages_tool_names(anthropic_messages_request)
# 6. AUTO-INJECT beta headers based on features used
filtered_betas = self._get_bedrock_invoke_anthropic_beta_headers(
model=model,
messages=messages,
anthropic_messages_optional_request_params=anthropic_messages_optional_request_params,
headers=headers,
anthropic_messages_request=anthropic_messages_request,
injected_thinking_for_clear_thinking=injected_thinking_for_clear_thinking,
)
if filtered_betas:
anthropic_messages_request["anthropic_beta"] = filtered_betas
@ -669,16 +696,9 @@ class AmazonAnthropicClaudeMessagesConfig(
# Catches Anthropic-only extensions (output_config, speed, mcp_servers, ...)
# and any future additions Claude Code may start sending. ``context_management``
# has already been pre-filtered to its Bedrock-supported subset above.
allowed = self.BEDROCK_INVOKE_ALLOWED_TOP_LEVEL_FIELDS
stripped = sorted(k for k in anthropic_messages_request if k not in allowed)
if stripped:
verbose_logger.debug(
"Bedrock Invoke: stripping unsupported top-level request fields: %s",
stripped,
)
anthropic_messages_request = {
k: v for k, v in anthropic_messages_request.items() if k in allowed
}
anthropic_messages_request = self._strip_unsupported_bedrock_invoke_fields(
anthropic_messages_request
)
return anthropic_messages_request

57
litellm/model_prices_and_context_window_backup.json
View file

@ -982,7 +982,9 @@
"supports_tool_choice": true,
"supports_vision": true,
"tool_use_system_prompt_tokens": 159,
"supports_native_structured_output": true
"supports_native_structured_output": true,
"supports_output_config": true,
"bedrock_output_config_effort_ceiling": "high"
},
"anthropic.claude-opus-4-6-v1": {
"cache_creation_input_token_cost": 6.25e-06,
@ -1013,7 +1015,8 @@
"supports_native_structured_output": true,
"supports_output_config": true,
"supports_max_reasoning_effort": true,
"supports_minimal_reasoning_effort": true
"supports_minimal_reasoning_effort": true,
"bedrock_output_config_effort_ceiling": "max"
},
"global.anthropic.claude-opus-4-6-v1": {
"cache_creation_input_token_cost": 6.25e-06,
@ -1044,7 +1047,8 @@
"supports_native_structured_output": true,
"supports_output_config": true,
"supports_max_reasoning_effort": true,
"supports_minimal_reasoning_effort": true
"supports_minimal_reasoning_effort": true,
"bedrock_output_config_effort_ceiling": "max"
},
"us.anthropic.claude-opus-4-6-v1": {
"cache_creation_input_token_cost": 6.875e-06,
@ -1075,7 +1079,8 @@
"supports_native_structured_output": true,
"supports_output_config": true,
"supports_max_reasoning_effort": true,
"supports_minimal_reasoning_effort": true
"supports_minimal_reasoning_effort": true,
"bedrock_output_config_effort_ceiling": "max"
},
"eu.anthropic.claude-opus-4-6-v1": {
"cache_creation_input_token_cost": 6.875e-06,
@ -1105,7 +1110,8 @@
"supports_native_structured_output": true,
"supports_output_config": true,
"supports_max_reasoning_effort": true,
"supports_minimal_reasoning_effort": true
"supports_minimal_reasoning_effort": true,
"bedrock_output_config_effort_ceiling": "max"
},
"au.anthropic.claude-opus-4-6-v1": {
"cache_creation_input_token_cost": 6.875e-06,
@ -1135,7 +1141,8 @@
"supports_native_structured_output": true,
"supports_output_config": true,
"supports_max_reasoning_effort": true,
"supports_minimal_reasoning_effort": true
"supports_minimal_reasoning_effort": true,
"bedrock_output_config_effort_ceiling": "max"
},
"anthropic.claude-opus-4-7": {
"cache_creation_input_token_cost": 6.25e-06,
@ -1166,7 +1173,9 @@
"tool_use_system_prompt_tokens": 346,
"supports_native_structured_output": true,
"supports_max_reasoning_effort": true,
"supports_minimal_reasoning_effort": true
"supports_minimal_reasoning_effort": true,
"supports_output_config": true,
"bedrock_output_config_effort_ceiling": "xhigh"
},
"anthropic.claude-mythos-preview": {
"input_cost_per_token": 0,
@ -1212,7 +1221,9 @@
"tool_use_system_prompt_tokens": 346,
"supports_native_structured_output": true,
"supports_max_reasoning_effort": true,
"supports_minimal_reasoning_effort": true
"supports_minimal_reasoning_effort": true,
"supports_output_config": true,
"bedrock_output_config_effort_ceiling": "xhigh"
},
"us.anthropic.claude-opus-4-7": {
"cache_creation_input_token_cost": 6.875e-06,
@ -1243,7 +1254,9 @@
"tool_use_system_prompt_tokens": 346,
"supports_native_structured_output": true,
"supports_max_reasoning_effort": true,
"supports_minimal_reasoning_effort": true
"supports_minimal_reasoning_effort": true,
"supports_output_config": true,
"bedrock_output_config_effort_ceiling": "xhigh"
},
"eu.anthropic.claude-opus-4-7": {
"cache_creation_input_token_cost": 6.875e-06,
@ -1273,7 +1286,9 @@
"tool_use_system_prompt_tokens": 346,
"supports_native_structured_output": true,
"supports_max_reasoning_effort": true,
"supports_minimal_reasoning_effort": true
"supports_minimal_reasoning_effort": true,
"supports_output_config": true,
"bedrock_output_config_effort_ceiling": "xhigh"
},
"au.anthropic.claude-opus-4-7": {
"cache_creation_input_token_cost": 6.875e-06,
@ -1303,7 +1318,9 @@
"tool_use_system_prompt_tokens": 346,
"supports_native_structured_output": true,
"supports_max_reasoning_effort": true,
"supports_minimal_reasoning_effort": true
"supports_minimal_reasoning_effort": true,
"supports_output_config": true,
"bedrock_output_config_effort_ceiling": "xhigh"
},
"anthropic.claude-sonnet-4-6": {
"cache_creation_input_token_cost": 3.75e-06,
@ -9804,7 +9821,8 @@
"supports_response_schema": true,
"supports_tool_choice": true,
"supports_vision": true,
"tool_use_system_prompt_tokens": 159
"tool_use_system_prompt_tokens": 159,
"supports_output_config": true
},
"claude-opus-4-5": {
"cache_creation_input_token_cost": 6.25e-06,
@ -9832,7 +9850,8 @@
"supports_response_schema": true,
"supports_tool_choice": true,
"supports_vision": true,
"tool_use_system_prompt_tokens": 159
"tool_use_system_prompt_tokens": 159,
"supports_output_config": true
},
"claude-opus-4-6": {
"cache_creation_input_token_cost": 6.25e-06,
@ -31808,7 +31827,9 @@
"supports_tool_choice": true,
"supports_vision": true,
"tool_use_system_prompt_tokens": 159,
"supports_native_structured_output": true
"supports_native_structured_output": true,
"supports_output_config": true,
"bedrock_output_config_effort_ceiling": "high"
},
"global.anthropic.claude-opus-4-5-20251101-v1:0": {
"cache_creation_input_token_cost": 6.25e-06,
@ -31837,7 +31858,9 @@
"supports_tool_choice": true,
"supports_vision": true,
"tool_use_system_prompt_tokens": 159,
"supports_native_structured_output": true
"supports_native_structured_output": true,
"supports_output_config": true,
"bedrock_output_config_effort_ceiling": "high"
},
"eu.anthropic.claude-opus-4-5-20251101-v1:0": {
"cache_creation_input_token_cost": 6.25e-06,
@ -31865,7 +31888,9 @@
"supports_tool_choice": true,
"supports_vision": true,
"tool_use_system_prompt_tokens": 159,
"supports_native_structured_output": true
"supports_native_structured_output": true,
"supports_output_config": true,
"bedrock_output_config_effort_ceiling": "high"
},
"us.anthropic.claude-sonnet-4-20250514-v1:0": {
"cache_creation_input_token_cost": 3.75e-06,

3
litellm/types/llms/anthropic.py
View file

@ -39,7 +39,8 @@ class AnthropicOutputSchema(TypedDict, total=False):
class AnthropicOutputConfig(TypedDict, total=False):
"""Configuration for controlling Claude's output behavior."""
effort: Literal["high", "medium", "low"]
effort: Literal["high", "medium", "low", "xhigh", "max"]
format: AnthropicOutputSchema
class AnthropicMessagesTool(TypedDict, total=False):

3
litellm/types/utils.py
View file

@ -148,6 +148,9 @@ class ProviderSpecificModelInfo(TypedDict, total=False):
supports_xhigh_reasoning_effort: Optional[bool]
supports_max_reasoning_effort: Optional[bool]
supports_output_config: Optional[bool]
bedrock_output_config_effort_ceiling: Optional[
Literal["low", "medium", "high", "max", "xhigh"]
]
class SearchContextCostPerQuery(TypedDict, total=False):

3
litellm/utils.py
View file

@ -6036,6 +6036,9 @@ def _get_model_info_helper( # noqa: PLR0915
supports_max_reasoning_effort=_model_info.get(
"supports_max_reasoning_effort", None
),
bedrock_output_config_effort_ceiling=_model_info.get(
"bedrock_output_config_effort_ceiling", None
),
supports_computer_use=_model_info.get("supports_computer_use", None),
search_context_cost_per_query=_model_info.get(
"search_context_cost_per_query", None

57
model_prices_and_context_window.json
View file

@ -982,7 +982,9 @@
"supports_tool_choice": true,
"supports_vision": true,
"tool_use_system_prompt_tokens": 159,
"supports_native_structured_output": true
"supports_native_structured_output": true,
"supports_output_config": true,
"bedrock_output_config_effort_ceiling": "high"
},
"anthropic.claude-opus-4-6-v1": {
"cache_creation_input_token_cost": 6.25e-06,
@ -1013,7 +1015,8 @@
"supports_native_structured_output": true,
"supports_output_config": true,
"supports_max_reasoning_effort": true,
"supports_minimal_reasoning_effort": true
"supports_minimal_reasoning_effort": true,
"bedrock_output_config_effort_ceiling": "max"
},
"global.anthropic.claude-opus-4-6-v1": {
"cache_creation_input_token_cost": 6.25e-06,
@ -1044,7 +1047,8 @@
"supports_native_structured_output": true,
"supports_output_config": true,
"supports_max_reasoning_effort": true,
"supports_minimal_reasoning_effort": true
"supports_minimal_reasoning_effort": true,
"bedrock_output_config_effort_ceiling": "max"
},
"us.anthropic.claude-opus-4-6-v1": {
"cache_creation_input_token_cost": 6.875e-06,
@ -1075,7 +1079,8 @@
"supports_native_structured_output": true,
"supports_output_config": true,
"supports_max_reasoning_effort": true,
"supports_minimal_reasoning_effort": true
"supports_minimal_reasoning_effort": true,
"bedrock_output_config_effort_ceiling": "max"
},
"eu.anthropic.claude-opus-4-6-v1": {
"cache_creation_input_token_cost": 6.875e-06,
@ -1105,7 +1110,8 @@
"supports_native_structured_output": true,
"supports_output_config": true,
"supports_max_reasoning_effort": true,
"supports_minimal_reasoning_effort": true
"supports_minimal_reasoning_effort": true,
"bedrock_output_config_effort_ceiling": "max"
},
"au.anthropic.claude-opus-4-6-v1": {
"cache_creation_input_token_cost": 6.875e-06,
@ -1135,7 +1141,8 @@
"supports_native_structured_output": true,
"supports_output_config": true,
"supports_max_reasoning_effort": true,
"supports_minimal_reasoning_effort": true
"supports_minimal_reasoning_effort": true,
"bedrock_output_config_effort_ceiling": "max"
},
"anthropic.claude-opus-4-7": {
"cache_creation_input_token_cost": 6.25e-06,
@ -1166,7 +1173,9 @@
"tool_use_system_prompt_tokens": 346,
"supports_native_structured_output": true,
"supports_max_reasoning_effort": true,
"supports_minimal_reasoning_effort": true
"supports_minimal_reasoning_effort": true,
"supports_output_config": true,
"bedrock_output_config_effort_ceiling": "xhigh"
},
"anthropic.claude-mythos-preview": {
"input_cost_per_token": 0,
@ -1212,7 +1221,9 @@
"tool_use_system_prompt_tokens": 346,
"supports_native_structured_output": true,
"supports_max_reasoning_effort": true,
"supports_minimal_reasoning_effort": true
"supports_minimal_reasoning_effort": true,
"supports_output_config": true,
"bedrock_output_config_effort_ceiling": "xhigh"
},
"us.anthropic.claude-opus-4-7": {
"cache_creation_input_token_cost": 6.875e-06,
@ -1243,7 +1254,9 @@
"tool_use_system_prompt_tokens": 346,
"supports_native_structured_output": true,
"supports_max_reasoning_effort": true,
"supports_minimal_reasoning_effort": true
"supports_minimal_reasoning_effort": true,
"supports_output_config": true,
"bedrock_output_config_effort_ceiling": "xhigh"
},
"eu.anthropic.claude-opus-4-7": {
"cache_creation_input_token_cost": 6.875e-06,
@ -1273,7 +1286,9 @@
"tool_use_system_prompt_tokens": 346,
"supports_native_structured_output": true,
"supports_max_reasoning_effort": true,
"supports_minimal_reasoning_effort": true
"supports_minimal_reasoning_effort": true,
"supports_output_config": true,
"bedrock_output_config_effort_ceiling": "xhigh"
},
"au.anthropic.claude-opus-4-7": {
"cache_creation_input_token_cost": 6.875e-06,
@ -1303,7 +1318,9 @@
"tool_use_system_prompt_tokens": 346,
"supports_native_structured_output": true,
"supports_max_reasoning_effort": true,
"supports_minimal_reasoning_effort": true
"supports_minimal_reasoning_effort": true,
"supports_output_config": true,
"bedrock_output_config_effort_ceiling": "xhigh"
},
"anthropic.claude-sonnet-4-6": {
"cache_creation_input_token_cost": 3.75e-06,
@ -9804,7 +9821,8 @@
"supports_response_schema": true,
"supports_tool_choice": true,
"supports_vision": true,
"tool_use_system_prompt_tokens": 159
"tool_use_system_prompt_tokens": 159,
"supports_output_config": true
},
"claude-opus-4-5": {
"cache_creation_input_token_cost": 6.25e-06,
@ -9832,7 +9850,8 @@
"supports_response_schema": true,
"supports_tool_choice": true,
"supports_vision": true,
"tool_use_system_prompt_tokens": 159
"tool_use_system_prompt_tokens": 159,
"supports_output_config": true
},
"claude-opus-4-6": {
"cache_creation_input_token_cost": 6.25e-06,
@ -31683,7 +31702,9 @@
"supports_tool_choice": true,
"supports_vision": true,
"tool_use_system_prompt_tokens": 159,
"supports_native_structured_output": true
"supports_native_structured_output": true,
"supports_output_config": true,
"bedrock_output_config_effort_ceiling": "high"
},
"global.anthropic.claude-opus-4-5-20251101-v1:0": {
"cache_creation_input_token_cost": 6.25e-06,
@ -31712,7 +31733,9 @@
"supports_tool_choice": true,
"supports_vision": true,
"tool_use_system_prompt_tokens": 159,
"supports_native_structured_output": true
"supports_native_structured_output": true,
"supports_output_config": true,
"bedrock_output_config_effort_ceiling": "high"
},
"eu.anthropic.claude-opus-4-5-20251101-v1:0": {
"cache_creation_input_token_cost": 6.25e-06,
@ -31740,7 +31763,9 @@
"supports_tool_choice": true,
"supports_vision": true,
"tool_use_system_prompt_tokens": 159,
"supports_native_structured_output": true
"supports_native_structured_output": true,
"supports_output_config": true,
"bedrock_output_config_effort_ceiling": "high"
},
"us.anthropic.claude-sonnet-4-20250514-v1:0": {
"cache_creation_input_token_cost": 3.75e-06,

35
tests/llm_translation/reasoning_effort_grid/grid_spec.py
View file

@ -22,6 +22,7 @@ class ModelEntry:
required_env: FrozenSet[str] = field(default_factory=frozenset)
caps: FrozenSet[str] = field(default_factory=frozenset)
fail_reason: Optional[str] = None
bedrock_effort_ceiling: Optional[str] = None
def params(self) -> Dict[str, str]:
return dict(self.extra_params)
@ -59,9 +60,31 @@ _ADAPTIVE_EFFORT_LABEL: Dict[str, str] = {
"max": "max",
}
_EFFORT_RANK: Dict[str, int] = {
"low": 0,
"medium": 1,
"high": 2,
"max": 3,
"xhigh": 4,
}
_BAD_REQUEST_EFFORTS: FrozenSet[str] = frozenset({"disabled", "invalid", ""})
def _bedrock_clamps_effort(model: "ModelEntry", effort: str) -> bool:
"""Whether Bedrock will clamp ``effort`` down to ``bedrock_effort_ceiling``.
Bedrock chat/messages paths clamp unsupported high tiers (e.g. ``xhigh``
on Opus 4.6) to the model's ceiling rather than rejecting them, so the
missing native capability is OK — the wire effort just degrades.
"""
if model.bedrock_effort_ceiling is None:
return False
if effort not in _EFFORT_RANK or model.bedrock_effort_ceiling not in _EFFORT_RANK:
return False
return _EFFORT_RANK[effort] > _EFFORT_RANK[model.bedrock_effort_ceiling]
def expected(model: ModelEntry, effort: str) -> CellExpectation:
if effort in ("__omit__", "none"):
if model.mode == "budget":
@ -73,14 +96,20 @@ def expected(model: ModelEntry, effort: str) -> CellExpectation:
if effort in ("xhigh", "max"):
cap = f"supports_{effort}_reasoning_effort"
if cap not in model.caps:
if cap not in model.caps and not _bedrock_clamps_effort(model, effort):
return CellExpectation(status=400, thinking_type=OMIT)
if model.mode == "adaptive":
wire_effort = _ADAPTIVE_EFFORT_LABEL[effort]
if model.bedrock_effort_ceiling is not None:
wire_rank = _EFFORT_RANK[wire_effort]
ceiling_rank = _EFFORT_RANK[model.bedrock_effort_ceiling]
if wire_rank > ceiling_rank:
wire_effort = model.bedrock_effort_ceiling
return CellExpectation(
status=200,
thinking_type="adaptive",
output_config_effort=_ADAPTIVE_EFFORT_LABEL[effort],
output_config_effort=wire_effort,
)
return CellExpectation(
@ -219,6 +248,7 @@ BEDROCK_CONVERSE_MODELS: Tuple[ModelEntry, ...] = (
extra_params=(("aws_region_name", "us-east-1"),),
required_env=_BEDROCK_REQ,
caps=_CAPS_4_6,
bedrock_effort_ceiling="max",
),
ModelEntry(
alias="bedrock-claude-sonnet-4-6",
@ -247,6 +277,7 @@ BEDROCK_INVOKE_CHAT_MODELS: Tuple[ModelEntry, ...] = (
extra_params=(("aws_region_name", "us-east-1"),),
required_env=_BEDROCK_REQ,
caps=_CAPS_4_6,
bedrock_effort_ceiling="max",
),
ModelEntry(
alias="bedrock-invoke-claude-sonnet-4-6",

59
tests/test_litellm/llms/anthropic/chat/test_anthropic_chat_transformation.py
View file

@ -1622,6 +1622,29 @@ def test_effort_output_config_preservation():
assert result["output_config"]["effort"] == "medium"
def test_output_config_format_preservation_and_beta_header():
"""Test that output_config.format is preserved and treated as structured output."""
config = AnthropicConfig()
output_format = {
"type": "json_schema",
"schema": {"type": "object", "properties": {"answer": {"type": "string"}}},
}
optional_params = {"output_config": {"format": output_format, "effort": "xhigh"}}
result = config.transform_request(
model="claude-opus-4-7",
messages=[{"role": "user", "content": "Test"}],
optional_params=optional_params,
litellm_params={},
headers={},
)
headers = config.update_headers_with_optional_anthropic_beta({}, optional_params)
assert result["output_config"]["format"] == output_format
assert result["output_config"]["effort"] == "xhigh"
assert "structured-outputs-2025-11-13" in headers["anthropic-beta"]
def test_effort_beta_header_injection():
"""Test that effort beta header is automatically added when output_config is detected."""
from litellm.llms.anthropic.common_utils import AnthropicModelInfo
@ -1648,7 +1671,7 @@ def test_effort_validation():
messages = [{"role": "user", "content": "Test"}]
# Valid values should work
# Valid values should work (xhigh is Opus 4.7+ only, not 4.5)
for effort in ["high", "medium", "low"]:
optional_params = {"output_config": {"effort": effort}}
result = config.transform_request(
@ -2513,14 +2536,14 @@ def test_reasoning_effort_accepts_dict_shape_for_adaptive_model(reasoning_effort
)
# thinking must be set (adaptive for 4.6+)
assert "thinking" in result, (
f"thinking missing for reasoning_effort={reasoning_effort_value!r}"
)
assert (
"thinking" in result
), f"thinking missing for reasoning_effort={reasoning_effort_value!r}"
assert result["thinking"]["type"] == "adaptive"
# output_config must carry the mapped effort
assert "output_config" in result, (
f"output_config missing for reasoning_effort={reasoning_effort_value!r}"
)
assert (
"output_config" in result
), f"output_config missing for reasoning_effort={reasoning_effort_value!r}"
assert result["output_config"]["effort"] == "low"
@ -2532,7 +2555,9 @@ def test_reasoning_effort_accepts_dict_shape_for_adaptive_model(reasoning_effort
{"effort": "low", "summary": "concise"},
],
)
def test_reasoning_effort_accepts_dict_shape_for_non_adaptive_model(reasoning_effort_value):
def test_reasoning_effort_accepts_dict_shape_for_non_adaptive_model(
reasoning_effort_value,
):
"""
Non-adaptive (pre-4.6) branch: dict-shape reasoning_effort must still map
to ``thinking.type='enabled'`` + ``budget_tokens``. ``output_config`` must
@ -2547,9 +2572,9 @@ def test_reasoning_effort_accepts_dict_shape_for_non_adaptive_model(reasoning_ef
drop_params=False,
)
assert "thinking" in result, (
f"thinking missing for reasoning_effort={reasoning_effort_value!r}"
)
assert (
"thinking" in result
), f"thinking missing for reasoning_effort={reasoning_effort_value!r}"
assert result["thinking"]["type"] == "enabled"
assert "budget_tokens" in result["thinking"]
assert result["thinking"]["budget_tokens"] > 0
@ -2582,12 +2607,12 @@ def test_reasoning_effort_unparseable_dict_is_dropped(bad_value):
model="claude-sonnet-4-6-20260219",
drop_params=False,
)
assert "thinking" not in result, (
f"thinking should not be set for bad value {bad_value!r}"
)
assert "output_config" not in result, (
f"output_config should not be set for bad value {bad_value!r}"
)
assert (
"thinking" not in result
), f"thinking should not be set for bad value {bad_value!r}"
assert (
"output_config" not in result
), f"output_config should not be set for bad value {bad_value!r}"
@pytest.mark.parametrize(

33
tests/test_litellm/llms/anthropic/experimental_pass_through/messages/test_anthropic_messages_structured_outputs.py
View file

@ -48,6 +48,39 @@ def test_output_format_supported_and_transforms_correctly():
assert "structured-outputs-2025-11-13" in headers["anthropic-beta"]
def test_output_config_format_supported_and_transforms_correctly():
"""Test that output_config.format is preserved and adds the structured-output beta."""
config = AnthropicMessagesConfig()
supported_params = config.get_supported_anthropic_messages_params("claude-opus-4-7")
assert "output_config" in supported_params
output_format = {
"type": "json_schema",
"schema": {"type": "object", "properties": {"result": {"type": "string"}}},
}
optional_params = {
"max_tokens": 1024,
"output_config": {"format": output_format, "effort": "xhigh"},
}
headers = {}
result = config.transform_anthropic_messages_request(
model="claude-opus-4-7",
messages=[{"role": "user", "content": "test"}],
anthropic_messages_optional_request_params=optional_params.copy(),
litellm_params={},
headers=headers,
)
headers = config._update_headers_with_anthropic_beta(headers, optional_params)
assert result["output_config"]["format"] == output_format
assert result["output_config"]["effort"] == "xhigh"
assert "anthropic-beta" in headers
assert "structured-outputs-2025-11-13" in headers["anthropic-beta"]
def test_output_format_works_with_bedrock_and_azure():
"""Test that output_format works with Bedrock and Azure Foundry models."""
config = AnthropicMessagesConfig()

54
tests/test_litellm/llms/anthropic/experimental_pass_through/messages/test_reasoning_effort_translation.py
View file

@ -6,6 +6,9 @@ from litellm.llms.anthropic.common_utils import AnthropicError
from litellm.llms.anthropic.experimental_pass_through.messages.transformation import (
AnthropicMessagesConfig,
)
from litellm.llms.bedrock.messages.invoke_transformations.anthropic_claude3_transformation import (
AmazonAnthropicClaudeMessagesConfig,
)
@pytest.mark.parametrize(
@ -102,7 +105,6 @@ def test_invalid_reasoning_effort_raises_400(bad_effort):
"model,bad_effort",
[
("claude-opus-4-6", "xhigh"),
("bedrock/invoke/us.anthropic.claude-opus-4-6-v1", "xhigh"),
("claude-sonnet-4-6", "xhigh"),
],
)
@ -123,6 +125,56 @@ def test_reasoning_effort_unsupported_tier_raises_400_messages(model, bad_effort
assert "not supported by this model" in str(exc_info.value)
@pytest.mark.parametrize(
"model,effort,expected_effort",
[
("invoke/us.anthropic.claude-opus-4-6-v1", "xhigh", "max"),
("invoke/us.anthropic.claude-opus-4-6-v1", "max", "max"),
("invoke/us.anthropic.claude-opus-4-6-v1", "high", "high"),
("invoke/us.anthropic.claude-opus-4-7", "xhigh", "xhigh"),
],
)
def test_bedrock_invoke_messages_clamps_effort_to_ceiling(
model, effort, expected_effort
):
"""Bedrock Invoke /v1/messages degrades effort to the model's ceiling.
Claude Code "goal mode" sends ``xhigh``; Opus 4.6 must clamp to ``max``
instead of raising, while Opus 4.7 (ceiling ``xhigh``) keeps ``xhigh``.
"""
config = AmazonAnthropicClaudeMessagesConfig()
optional_params = {"max_tokens": 1024, "reasoning_effort": effort}
result = config.transform_anthropic_messages_request(
model=model,
messages=[{"role": "user", "content": "Hello"}],
anthropic_messages_optional_request_params=optional_params,
litellm_params={},
headers={},
)
assert result["output_config"]["effort"] == expected_effort
assert result["thinking"]["type"] == "adaptive"
def test_bedrock_invoke_messages_rejects_xhigh_without_ceiling():
"""Sonnet 4.6 on Bedrock has no effort ceiling, so xhigh is still rejected."""
config = AmazonAnthropicClaudeMessagesConfig()
optional_params = {"max_tokens": 1024, "reasoning_effort": "xhigh"}
with pytest.raises(AnthropicError) as exc_info:
config.transform_anthropic_messages_request(
model="invoke/us.anthropic.claude-sonnet-4-6",
messages=[{"role": "user", "content": "Hello"}],
anthropic_messages_optional_request_params=optional_params,
litellm_params={},
headers={},
)
assert exc_info.value.status_code == 400
assert "not supported by this model" in str(exc_info.value)
@pytest.mark.parametrize(
"model",
[

55
tests/test_litellm/llms/bedrock/chat/invoke_transformations/test_bedrock_chat_invoke_transformations_anthropic_claude3_transformation.py
View file

@ -430,6 +430,61 @@ def test_output_config_forwarded_for_bedrock_chat_invoke_request():
assert result["max_tokens"] == 100
def test_output_config_format_converted_for_bedrock_chat_invoke_request():
"""Bedrock Invoke chat path consumes ``output_config.format`` before forwarding."""
config = AmazonAnthropicClaudeConfig()
schema = {
"type": "object",
"properties": {"answer": {"type": "string"}},
}
result = config.transform_request(
model="anthropic.claude-opus-4-7",
messages=[{"role": "user", "content": "test"}],
optional_params={
"max_tokens": 100,
"output_config": {
"effort": "xhigh",
"format": {"type": "json_schema", "schema": schema},
},
},
litellm_params={},
headers={},
)
assert result.get("output_config") == {"effort": "xhigh"}
last_content = result["messages"][0]["content"]
assert json.loads(last_content[-1]["text"]) == schema
@pytest.mark.parametrize(
"model,expected_effort",
[
("anthropic.claude-opus-4-5-20251101-v1:0", "high"),
("anthropic.claude-opus-4-6-v1", "max"),
("anthropic.claude-opus-4-7", "xhigh"),
],
)
def test_output_config_effort_normalized_for_bedrock_chat_invoke_request(
model, expected_effort
):
"""Bedrock Invoke chat path accepts ``xhigh`` and forwards the provider-safe effort."""
config = AmazonAnthropicClaudeConfig()
result = config.transform_request(
model=model,
messages=[{"role": "user", "content": "test"}],
optional_params={
"max_tokens": 100,
"output_config": {"effort": "xhigh"},
},
litellm_params={},
headers={},
)
assert result.get("output_config") == {"effort": expected_effort}
def test_bedrock_chat_invoke_checks_output_config_support_with_bedrock_provider():
config = AmazonAnthropicClaudeConfig()
messages = [{"role": "user", "content": "test"}]

127
tests/test_litellm/llms/bedrock/chat/test_converse_transformation.py
View file

@ -318,6 +318,7 @@ def test_reasoning_effort_none_omits_thinking_for_anthropic_converse(model):
("bedrock/converse/us.anthropic.claude-opus-4-7", "high", "high"),
("bedrock/converse/us.anthropic.claude-opus-4-7", "xhigh", "xhigh"),
("bedrock/converse/us.anthropic.claude-opus-4-7", "max", "max"),
("bedrock/converse/us.anthropic.claude-opus-4-6-v1", "xhigh", "max"),
("bedrock/converse/us.anthropic.claude-opus-4-6-v1", "max", "max"),
("bedrock/converse/us.anthropic.claude-sonnet-4-6", "high", "high"),
("bedrock/converse/us.anthropic.claude-sonnet-4-6", "minimal", "low"),
@ -369,6 +370,132 @@ def test_output_config_effort_forwarded_into_additional_request_fields(model):
assert additional.get("output_config") == {"effort": "high"}
def test_output_config_format_translated_to_native_output_config_converse():
"""``output_config.format`` becomes Bedrock ``outputConfig`` and is not forwarded raw."""
config = AmazonConverseConfig()
schema = {
"type": "object",
"properties": {"answer": {"type": "string"}},
}
result = config._transform_request(
model="bedrock/converse/us.anthropic.claude-opus-4-7",
messages=[{"role": "user", "content": "hi"}],
optional_params={
"maxTokens": 256,
"thinking": {"type": "adaptive"},
"output_config": {
"effort": "xhigh",
"format": {"type": "json_schema", "schema": schema},
},
},
litellm_params={},
headers={},
)
additional = result.get("additionalModelRequestFields", {})
assert additional.get("output_config") == {"effort": "xhigh"}
assert "format" not in additional["output_config"]
assert result["outputConfig"]["textFormat"]["type"] == "json_schema"
parsed_schema = json.loads(
result["outputConfig"]["textFormat"]["structure"]["jsonSchema"]["schema"]
)
assert parsed_schema == {**schema, "additionalProperties": False}
def test_output_config_format_dropped_on_unsupported_converse_model_warns(caplog):
"""When Converse model lacks native structured-output support, the silently
dropped ``output_config.format`` must surface as a warning so callers can
diagnose plain-text responses."""
from unittest.mock import patch
config = AmazonConverseConfig()
schema = {
"type": "object",
"properties": {"answer": {"type": "string"}},
}
with patch.object(
AmazonConverseConfig,
"_supports_native_structured_outputs",
return_value=False,
):
with caplog.at_level("WARNING"):
result = config._transform_request(
model="bedrock/converse/us.anthropic.claude-3-haiku-20240307-v1:0",
messages=[{"role": "user", "content": "hi"}],
optional_params={
"maxTokens": 256,
"output_config": {
"format": {"type": "json_schema", "schema": schema},
},
},
litellm_params={},
headers={},
)
assert "outputConfig" not in result
assert any(
"dropping `output_config.format`" in record.getMessage()
for record in caplog.records
)
def test_output_config_normalized_marker_does_not_leak_into_optional_params():
"""The internal ``_output_config_normalized`` marker set by
``_handle_reasoning_effort_parameter`` must be consumed during request
preparation so it does not linger on the caller's ``optional_params``."""
config = AmazonConverseConfig()
optional_params = config.map_openai_params(
non_default_params={"reasoning_effort": "xhigh"},
optional_params={},
model="bedrock/converse/us.anthropic.claude-opus-4-6-v1",
drop_params=False,
)
assert optional_params.get("_output_config_normalized") is True
config._transform_request(
model="bedrock/converse/us.anthropic.claude-opus-4-6-v1",
messages=[{"role": "user", "content": "hi"}],
optional_params=optional_params,
litellm_params={},
headers={},
)
assert "_output_config_normalized" not in optional_params
@pytest.mark.parametrize(
"model,expected_effort",
[
("bedrock/converse/us.anthropic.claude-opus-4-5-20251101-v1:0", "high"),
("bedrock/converse/us.anthropic.claude-opus-4-6-v1", "max"),
("bedrock/converse/us.anthropic.claude-opus-4-7", "xhigh"),
],
)
def test_output_config_effort_normalized_for_bedrock_converse_opus(
model, expected_effort
):
"""Bedrock Converse accepts ``xhigh`` and forwards the provider-safe effort."""
config = AmazonConverseConfig()
result = config._transform_request(
model=model,
messages=[{"role": "user", "content": "hi"}],
optional_params={
"maxTokens": 256,
"thinking": {"type": "adaptive"},
"output_config": {"effort": "xhigh"},
},
litellm_params={},
headers={},
)
additional = result.get("additionalModelRequestFields", {})
assert additional.get("output_config") == {"effort": expected_effort}
@pytest.mark.parametrize(
"effort",
["disabled", "invalid", ""],

169
tests/test_litellm/llms/bedrock/messages/invoke_transformations/test_anthropic_claude3_transformation.py
View file

@ -767,6 +767,163 @@ def test_bedrock_messages_forwards_output_config_with_output_format():
assert "output_format" not in result
def test_bedrock_messages_converts_output_config_format_to_inline_schema():
"""``output_config.format`` is consumed so Bedrock does not see an unknown nested key."""
from unittest.mock import patch
from litellm.types.router import GenericLiteLLMParams
cfg = AmazonAnthropicClaudeMessagesConfig()
messages = [{"role": "user", "content": [{"type": "text", "text": "Hello"}]}]
schema = {
"type": "object",
"properties": {"answer": {"type": "string"}},
}
optional_params = {
"max_tokens": 4096,
"output_config": {
"effort": "xhigh",
"format": {"type": "json_schema", "schema": schema},
},
}
with patch(
"litellm.llms.bedrock.messages.invoke_transformations.anthropic_claude3_transformation._supports_factory",
return_value=True,
):
result = cfg.transform_anthropic_messages_request(
model="anthropic.claude-opus-4-7",
messages=messages,
anthropic_messages_optional_request_params=optional_params,
litellm_params=GenericLiteLLMParams(),
headers={},
)
assert result.get("output_config") == {"effort": "xhigh"}
assert "output_format" not in result
last_content = result["messages"][0]["content"]
assert json.loads(last_content[-1]["text"]) == schema
@pytest.mark.parametrize(
"model,expected_effort",
[
("anthropic.claude-opus-4-5-20251101-v1:0", "high"),
("anthropic.claude-opus-4-6-v1", "max"),
("anthropic.claude-opus-4-7", "xhigh"),
],
)
def test_bedrock_messages_normalizes_output_config_effort_for_opus(
model, expected_effort
):
"""Bedrock /v1/messages accepts ``xhigh`` and forwards the provider-safe effort."""
from unittest.mock import patch
from litellm.types.router import GenericLiteLLMParams
cfg = AmazonAnthropicClaudeMessagesConfig()
with patch(
"litellm.llms.bedrock.messages.invoke_transformations.anthropic_claude3_transformation._supports_factory",
return_value=True,
):
result = cfg.transform_anthropic_messages_request(
model=model,
messages=[{"role": "user", "content": [{"type": "text", "text": "Hello"}]}],
anthropic_messages_optional_request_params={
"max_tokens": 4096,
"output_config": {"effort": "xhigh"},
},
litellm_params=GenericLiteLLMParams(),
headers={},
)
assert result.get("output_config") == {"effort": expected_effort}
def test_bedrock_messages_does_not_mutate_callers_messages_when_embedding_schema():
"""Inline-schema embedding must not mutate the caller's ``messages`` list,
message dicts, or content list."""
from unittest.mock import patch
from litellm.types.router import GenericLiteLLMParams
cfg = AmazonAnthropicClaudeMessagesConfig()
caller_content = [{"type": "text", "text": "Hello"}]
caller_message = {"role": "user", "content": caller_content}
caller_messages = [caller_message]
schema = {"type": "object", "properties": {"answer": {"type": "string"}}}
optional_params = {
"max_tokens": 4096,
"output_config": {
"effort": "xhigh",
"format": {"type": "json_schema", "schema": schema},
},
}
with patch(
"litellm.llms.bedrock.messages.invoke_transformations.anthropic_claude3_transformation._supports_factory",
return_value=True,
):
result = cfg.transform_anthropic_messages_request(
model="anthropic.claude-opus-4-7",
messages=caller_messages,
anthropic_messages_optional_request_params=optional_params,
litellm_params=GenericLiteLLMParams(),
headers={},
)
assert caller_messages == [
{"role": "user", "content": [{"type": "text", "text": "Hello"}]}
]
assert caller_message == {
"role": "user",
"content": [{"type": "text", "text": "Hello"}],
}
assert caller_content == [{"type": "text", "text": "Hello"}]
last_content = result["messages"][-1]["content"]
assert json.loads(last_content[-1]["text"]) == schema
def test_bedrock_messages_does_not_mutate_callers_output_config():
"""`pop_bedrock_invoke_output_config_format` / effort normalization must not
leak into the caller's ``optional_params`` dict."""
from unittest.mock import patch
from litellm.types.router import GenericLiteLLMParams
cfg = AmazonAnthropicClaudeMessagesConfig()
schema = {
"type": "object",
"properties": {"answer": {"type": "string"}},
}
caller_output_config = {
"effort": "xhigh",
"format": {"type": "json_schema", "schema": schema},
}
optional_params = {
"max_tokens": 4096,
"output_config": caller_output_config,
}
with patch(
"litellm.llms.bedrock.messages.invoke_transformations.anthropic_claude3_transformation._supports_factory",
return_value=True,
):
cfg.transform_anthropic_messages_request(
model="anthropic.claude-opus-4-5-20251101-v1:0",
messages=[{"role": "user", "content": [{"type": "text", "text": "Hello"}]}],
anthropic_messages_optional_request_params=optional_params,
litellm_params=GenericLiteLLMParams(),
headers={},
)
assert caller_output_config == {
"effort": "xhigh",
"format": {"type": "json_schema", "schema": schema},
}
def test_bedrock_messages_strips_output_config_with_output_format():
"""
When both output_config and output_format are present, output_format
@ -1071,9 +1228,7 @@ def test_bedrock_messages_preserves_compact_context_management_and_adds_beta():
messages = [{"role": "user", "content": [{"type": "text", "text": "Hi"}]}]
optional_params = {
"max_tokens": 4096,
"context_management": {
"edits": [{"type": "compact_20260112"}]
},
"context_management": {"edits": [{"type": "compact_20260112"}]},
}
result = cfg.transform_anthropic_messages_request(
@ -1084,9 +1239,7 @@ def test_bedrock_messages_preserves_compact_context_management_and_adds_beta():
headers={},
)
assert result.get("context_management") == {
"edits": [{"type": "compact_20260112"}]
}
assert result.get("context_management") == {"edits": [{"type": "compact_20260112"}]}
assert "compact-2026-01-12" in result.get("anthropic_beta", [])
assert result["max_tokens"] == 4096
@ -1118,9 +1271,7 @@ def test_bedrock_messages_filters_unsupported_context_management_edits():
headers={},
)
assert result.get("context_management") == {
"edits": [{"type": "compact_20260112"}]
}
assert result.get("context_management") == {"edits": [{"type": "compact_20260112"}]}
assert "compact-2026-01-12" in result.get("anthropic_beta", [])

47
tests/test_litellm/llms/bedrock/test_bedrock_common_utils.py
View file

@ -1,9 +1,7 @@
import json
import os
import sys
import pytest
from fastapi.testclient import TestClient
sys.path.insert(
0, os.path.abspath("../../../..")
@ -12,7 +10,6 @@ sys.path.insert(
from litellm.llms.bedrock.common_utils import BedrockModelInfo
# --------------------------------------------------------------------------- #
# get_bedrock_response_stream_shape lazy-load tests #
# --------------------------------------------------------------------------- #
@ -24,8 +21,10 @@ def _reset_bedrock_response_stream_shape_cache():
import litellm.llms.bedrock.common_utils as mod
mod.get_bedrock_response_stream_shape.cache_clear()
mod._get_local_model_cost_map.cache_clear()
yield
mod.get_bedrock_response_stream_shape.cache_clear()
mod._get_local_model_cost_map.cache_clear()
def test_bedrock_response_stream_shape_lazy_loads_once():
@ -222,3 +221,45 @@ def test_context_window_suffix_stripped_for_cost_lookup():
get_bedrock_base_model("anthropic.claude-3-5-sonnet-20241022-v2:0:51k")
== "anthropic.claude-3-5-sonnet-20241022-v2:0"
)
def test_output_config_effort_normalization_uses_model_info_ceiling(monkeypatch):
import litellm.llms.bedrock.common_utils as mod
calls = []
def fake_get_model_info(model, custom_llm_provider=None):
calls.append((model, custom_llm_provider))
return {"bedrock_output_config_effort_ceiling": "max"}
monkeypatch.setattr(mod, "_get_model_info", fake_get_model_info)
output_config = {"effort": "xhigh"}
mod.normalize_bedrock_opus_output_config_effort(
model="custom-bedrock-alias-without-opus-pattern",
output_config=output_config,
)
assert output_config == {"effort": "max"}
assert calls == [("custom-bedrock-alias-without-opus-pattern", "bedrock")]
@pytest.mark.parametrize(
"model,expected_ceiling",
[
("anthropic.claude-opus-4-5-20251101-v1:0", "high"),
("anthropic.claude-opus-4-6-v1", "max"),
("anthropic.claude-opus-4-7", "xhigh"),
("us.anthropic.claude-opus-4-5-20251101-v1:0", "high"),
("us.anthropic.claude-opus-4-6-v1", "max"),
("us.anthropic.claude-opus-4-7", "xhigh"),
],
)
def test_bundled_bedrock_opus_model_info_declares_output_config_effort_ceiling(
model, expected_ceiling
):
from litellm.litellm_core_utils.get_model_cost_map import GetModelCostMap
model_info = GetModelCostMap.load_local_model_cost_map()[model]
assert model_info["bedrock_output_config_effort_ceiling"] == expected_ceiling

6
tests/test_litellm/test_utils.py
View file

@ -859,7 +859,11 @@ def test_aaamodel_prices_and_context_window_json_is_valid():
"supports_adaptive_thinking": {"type": "boolean"},
"supports_service_tier": {"type": "boolean"},
"supports_preset": {"type": "boolean"},
"supports_output_config": {"type": "boolean"},
"supports_output_config": {"type": "boolean"},
"bedrock_output_config_effort_ceiling": {
"type": "string",
"enum": ["low", "medium", "high", "max", "xhigh"],
},
"tool_use_system_prompt_tokens": {"type": "number"},
"tpm": {"type": "number"},
"provider_specific_entry": {"type": "object"},