fix(bedrock): apply same adaptive-thinking carve-out in converse fallback

The Bedrock converse path has the exact same forced-tool_choice bug
that the anthropic transformation has — same root cause
(is_thinking_enabled doesn't recognise thinking.type=="adaptive"),
same symptom from Bedrock:

    Thinking may not be enabled when tool_choice forces tool use.

For Bedrock-native paths, the anthropic transformation fix alone is
not enough because AmazonConverseConfig._translate_response_format_param
sets a forced tool_choice independently when not is_thinking_enabled
(litellm/llms/bedrock/chat/converse_transformation.py:1042). On the
in-tree models this is masked by the response_format whitelist, but it
hits any deployment that pushes adaptive-thinking models off the
native outputConfig.textFormat path (e.g. yaml override of
supports_native_structured_output to coerce the fallback because the
target Bedrock account has not yet whitelisted Opus 4.7 for native
structured outputs — this is why the bug shows up in production).

Reuse the existing AnthropicConfig._is_adaptive_thinking_model() helper
that's already imported here.

New regression test:
test_bedrock_converse_adaptive_thinking_response_format_no_forced_tool_choice
pins the carve-out by exercising _translate_response_format_param
directly with a non-whitelisted model and the helper monkeypatched
to True.
This commit is contained in:
Kent 2026-05-17 08:38:43 +00:00
parent d979093f9a
commit b21ca29126
2 changed files with 82 additions and 0 deletions

View file

@ -1035,11 +1035,20 @@ class AmazonConverseConfig(BaseConfig):
optional_params=optional_params, tools=[_tool]
)
# Adaptive-thinking models (Claude 4.7+) need the same carve-out
# as anthropic/chat/transformation.py: ``is_thinking_enabled``
# does not match ``thinking.type == "adaptive"``, so without
# this check we set a forced ``tool_choice`` while
# ``thinking: adaptive`` is still on the wire body and Bedrock
# rejects with "Thinking may not be enabled when tool_choice
# forces tool use".
is_adaptive_thinking = AnthropicConfig._is_adaptive_thinking_model(model)
if (
litellm.utils.supports_tool_choice(
model=model, custom_llm_provider=self.custom_llm_provider
)
and not is_thinking_enabled
and not is_adaptive_thinking
):
optional_params["tool_choice"] = ToolChoiceValuesBlock(
tool=SpecificToolChoiceBlock(name=RESPONSE_FORMAT_TOOL_NAME)

View file

@ -539,3 +539,76 @@ class TestBedrockAnthropicCombinedRegressions:
for c in user_msg["content"]
)
assert has_cache_control
def test_bedrock_converse_adaptive_thinking_response_format_no_forced_tool_choice(
monkeypatch,
):
"""Bedrock converse fallback path: same fix as PR #28114's anthropic-side
fix needs to apply here too.
AmazonConverseConfig._add_response_format_to_optional_params (the
response_format → synthetic-tool fallback) used to force tool_choice
when not is_thinking_enabled. Adaptive-thinking models (Claude 4.7+)
use ``thinking.type == "adaptive"`` and so are not matched by
is_thinking_enabled, leading to a forced tool_choice while thinking
is on the wire body — Bedrock returns:
Thinking may not be enabled when tool_choice forces tool use.
Reproduces the same bug in this provider as the anthropic-side
fallback. See PR #28114 description for full background.
"""
import litellm
from litellm.llms.bedrock.chat.converse_transformation import AmazonConverseConfig
from litellm.llms.anthropic.chat.transformation import AnthropicConfig
# Force the helper True so the test does not depend on which models
# the model_cost map happens to flag adaptive.
monkeypatch.setattr(
AnthropicConfig,
"_is_adaptive_thinking_model",
staticmethod(lambda model: True),
)
# Make sure supports_tool_choice is True so the original guard doesn't
# short-circuit for the test model.
monkeypatch.setattr(
litellm.utils, "supports_tool_choice", lambda model, custom_llm_provider=None: True
)
config = AmazonConverseConfig()
response_format_value = {
"type": "json_schema",
"json_schema": {
"schema": {
"type": "object",
"properties": {"answer": {"type": "integer"}},
"required": ["answer"],
"additionalProperties": False,
},
"name": "out",
},
}
optional_params = config._translate_response_format_param(
value=response_format_value,
# Use a model name that is not on the native-structured-output list
# so we hit the synthetic-tool fallback path.
model="some-future-adaptive-model",
optional_params={},
non_default_params={"response_format": response_format_value},
is_thinking_enabled=False,
)
assert "tool_choice" not in optional_params, (
"Adaptive-thinking models must not have a forced tool_choice "
"synthesised by the Bedrock response_format fallback path; got: "
f"{optional_params.get('tool_choice')!r}"
)
# The synthetic tool should still be present so the model can still
# emit JSON via it when it chooses to. ``_translate_response_format_param``
# runs early in the chain and tools are still in OpenAI shape at this
# point (function/parameters); they get converted to Bedrock toolSpec
# later in transform_request.
tools = optional_params.get("tools") or []
assert any(
t.get("function", {}).get("name") == "json_tool_call" for t in tools
), f"expected synthetic json_tool_call tool, got tools={tools}"