scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-16 04:29:31 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 138

fix(anthropic): keep signed empty thinking blocks and honor thinking.block_binding (#46342)

Browse source 
* fix(anthropic): treat adaptive thinking as enabled when mapping response_format

is_thinking_enabled only recognised thinking.type=enabled, so a request that
paired thinking.type=adaptive with response_format took the synthetic JSON
tool path with a forced tool_choice, which Anthropic rejects alongside
thinking. The same check also gates the Bedrock Converse and Databricks
forced tool_choice downgrades, so they now see adaptive thinking too.

* fix(anthropic): keep signed thinking blocks with empty text on chat completions

Models that default to display: omitted return signed thinking blocks whose
text is empty, and Anthropic rejects the next turn when they are not replayed
unchanged. is_unsignable_thinking_block dropped them along with unsigned
blocks, and the inline content path required non-empty text, so a tool-use
turn on Opus 4.7+ or the 5 families lost its thinking blocks and 400ed on the
tool-result turn. Only a missing signature now drops a block, and the
mid-conversation system placement mirrors that rule. The /v1/messages strip
helper is left to #41362.

* fix(anthropic): skip the thinking-strip retry when block_binding asks for an error

The /v1/messages retry loop strips thinking blocks and resends on any
invalid-thinking 400, including the binding mismatch a caller opted into
with thinking.block_binding.prefix_mismatch_behavior: error. The retry
decision now sees the request body and surfaces that 400 instead of
overriding the caller's choice. #42888 covers the litellm_params side.

* fix(bedrock): carry thinking.block_binding through the Invoke thinking rebuilds

Bedrock Invoke rebuilt the thinking param from scratch when display: updates
was requested, on both the /v1/messages and chat completions paths, and the
legacy enabled-to-adaptive translation plus the clear_thinking injection did
the same, so thinking.block_binding never reached Bedrock. Every rebuild now
spreads the caller's thinking and only overrides the keys it owns, and
AnthropicThinkingParam types block_binding. The thinking-binding-controls
beta was already on the Bedrock allowlist.

* fix(anthropic): keep forcing the response_format tool under adaptive thinking

This reverts ad9e861ae7. Anthropic, Bedrock Invoke and Vertex all accept a
forced tool_choice next to thinking.type=adaptive and only reject it with
type=enabled, so counting adaptive as enabled fixed no live failure. It did
drop the forced JSON tool, so Claude answered response_format requests in
prose, and it downgraded a required tool_choice to auto on Bedrock Converse

* test(e2e): drop the adaptive thinking tool round trip case

With adaptive thinking the model often skips thinking on the weather prompt, so
turn 1 carries no signed empty block and the case fails before the replay it is
meant to cover. Anthropic also answers the replay with a 200 whether or not the
block is forwarded, so the case could not tell the fix from main. The factory
unit tests pin the outgoing shape instead

* fix(anthropic): only skip the thinking-strip retry for a binding mismatch 400

prefix_mismatch_behavior: error asks Anthropic to reject a block bound to
another conversation. It says nothing about unrelated bad blocks such as a
null signature, so those still get the strip-and-retry self-heal. Also trims
the empty-thinking docstrings to what the code guarantees and types the new
test parameters

* refactor(anthropic): narrow thinking fields with the mapping validator instead of typing.cast

typing.cast is banned by the strict ruff gate (TID251), so the three new casts pushed that rule one over its merge-base count. The existing pydantic mapping validator gives the same typed view without an unchecked cast, and it now has a generic name since it is no longer only used for Claude Code payloads

* refactor(anthropic): read thinking settings through the validated mapping so basedpyright stays at its merge-base count

The Bedrock clear_thinking rewrite still narrowed with a cast and an untyped dict fallback, which added two reportUnknownVariableType errors. It now uses the same pydantic mapping validator, made public for that reuse, and the retry hook validates the request body itself so the handler passes a typed dict instead of an untyped one

---------

Co-authored-by: yuneng-jiang <yuneng@berri.ai>
This commit is contained in:
ryan-crabbe-berri 2026-10-15 11:49:53 -07:00 • committed by GitHub
parent e344dbd8db
commit f470e418ea
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
15 changed files with 344 additions and 98 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

11
litellm/litellm_core_utils/prompt_templates/common_utils.py
View file

@ -2187,10 +2187,8 @@ def is_unsignable_thinking_block(block: object) -> bool:
Anthropic verifies the signature cryptographically, so a block with a null,
empty, or missing signature (e.g. from an open-source reasoning model) is
rejected with a 400, and so is a block whose signature or data carries
another provider's encrypted reasoning. It also rejects a `thinking` block
whose text is empty or whitespace-only ("each thinking block must contain
thinking"), regardless of signature, e.g. when a `thinking_blocks` history
item from a non-Anthropic reasoning provider is replayed through this path.
another provider's encrypted reasoning. A signed block is kept whatever its
text, since ``display: omitted`` returns signed blocks with empty ``thinking``.
`redacted_thinking` blocks carry no signature and are always kept.
"""
if is_encrypted_reasoning_block(block):
@ -2201,10 +2199,7 @@ def is_unsignable_thinking_block(block: object) -> bool:
if mapping.get("type") != "thinking":
return False
signature: Final = mapping.get("signature")
if not (isinstance(signature, str) and len(signature) > 0):
return True
thinking_text: Final = mapping.get("thinking")
return not (isinstance(thinking_text, str) and len(thinking_text.strip()) > 0)
return not (isinstance(signature, str) and len(signature) > 0)
def native_redacted_thinking_data(data: str) -> str:

7
litellm/litellm_core_utils/prompt_templates/factory.py
View file

@ -2897,13 +2897,8 @@ def anthropic_messages_pt(
if not isinstance(m, dict):
continue
# handle thinking blocks
thinking_block = cast(str, m.get("thinking", ""))
text_block = cast(str, m.get("text", ""))
if (
m.get("type", "") == "thinking"
and len(thinking_block) > 0
and not is_unsignable_thinking_block(m)
): # don't pass empty text blocks. anthropic api raises errors.
if m.get("type", "") == "thinking" and not is_unsignable_thinking_block(m):
anthropic_message: ChatCompletionThinkingBlock | AnthropicMessagesTextParam = cast(
ChatCompletionThinkingBlock, m
)

3
litellm/litellm_core_utils/prompt_templates/mid_conversation_system.py
View file

@ -272,8 +272,7 @@ def _assistant_part_renders(part: object) -> bool:
"""A text part always renders: the converter pads empty text with a placeholder."""
part_type: Final = message_field(part, "type")
if part_type == "thinking":
thinking: Final = message_field(part, "thinking")
return isinstance(thinking, str) and bool(thinking) and _thinking_block_renders(part)
return _thinking_block_renders(part)
return part_type in _RENDERED_ASSISTANT_PART_TYPES or (
isinstance(part_type, str) and part_type.endswith("_tool_result")
)

42
litellm/llms/anthropic/common_utils.py
View file

@ -109,7 +109,7 @@ _INFERENCE_PROFILE_MINOR_RE: Final = re.compile(r":\d+$")
_DATED_RELEASE_SUFFIX_RE: Final = re.compile(r"-\d{8}$")
_DOTTED_VERSION_RE: Final = re.compile(r"(\d)\.(\d)")
_CLAUDE_CODE_BILLING_HEADER_PREFIX: Final = "x-anthropic-billing-header:"
_CLAUDE_CODE_OBJECT_MAPPING_ADAPTER: Final = TypeAdapter(dict[object, object])
_OBJECT_MAPPING_ADAPTER: Final = TypeAdapter(dict[object, object])
_CLAUDE_CODE_OBJECT_LIST_ADAPTER: Final = TypeAdapter(list[object])
@ -164,9 +164,9 @@ def is_claude_code_user_agent(user_agent: str) -> bool:
return user_agent.startswith(_CLAUDE_CODE_USER_AGENT_PREFIXES)
def _validated_claude_code_mapping(value: object) -> dict[object, object] | None:
def validated_object_mapping(value: object) -> dict[object, object] | None:
try:
return _CLAUDE_CODE_OBJECT_MAPPING_ADAPTER.validate_python(value)
return _OBJECT_MAPPING_ADAPTER.validate_python(value)
except ValidationError:
return None
@ -203,7 +203,7 @@ def _claude_code_billing_texts(system: object) -> tuple[str, ...] | None:
blocks: Final = _validated_claude_code_list(system)
if blocks is None:
return None
block_mappings: Final = tuple(_validated_claude_code_mapping(block) for block in blocks)
block_mappings: Final = tuple(validated_object_mapping(block) for block in blocks)
if any(block is None for block in block_mappings):
return None
text_values: Final = tuple(
@ -236,7 +236,7 @@ def is_claude_code_one_shot_subagent_request(
tools: object,
user_agent: str | None,
) -> bool:
only_message: Final = _validated_claude_code_mapping(messages[0]) if len(messages) == 1 else None
only_message: Final = validated_object_mapping(messages[0]) if len(messages) == 1 else None
return (
user_agent is not None
and is_claude_code_user_agent(user_agent)
@ -935,21 +935,23 @@ class AnthropicModelInfo(BaseLLMModelInfo):
return
if AnthropicModelInfo._supports_legacy_thinking(model, custom_llm_provider):
return
thinking: Final = optional_params.get("thinking")
if not isinstance(thinking, dict) or thinking.get("type") != "enabled":
thinking: Final = validated_object_mapping(optional_params.get("thinking"))
if thinking is None or thinking.get("type") != "enabled":
return
budget_tokens: Final = thinking.get("budget_tokens") or 0
effort: Final = AnthropicModelInfo._legacy_budget_to_effort(
model=model,
budget_tokens=int(thinking.get("budget_tokens") or 0),
budget_tokens=int(budget_tokens) if isinstance(budget_tokens, (int, float, str)) else 0,
custom_llm_provider=custom_llm_provider,
)
existing_output_config: Final = optional_params.get("output_config")
display: Final = thinking.get("display")
if display in ("summarized", "omitted"):
optional_params["thinking"] = {"type": "adaptive", "display": display}
else:
optional_params["thinking"] = {"type": "adaptive"}
supported_display: Final = {"display": display} if display in ("summarized", "omitted") else {}
carried_thinking_keys: Final = {
key: value for key, value in thinking.items() if key not in ("type", "budget_tokens", "display")
}
optional_params["thinking"] = {**carried_thinking_keys, "type": "adaptive", **supported_display}
optional_params["output_config"] = {
"effort": effort,
**(existing_output_config if isinstance(existing_output_config, dict) else MappingProxyType({})),
@ -1617,6 +1619,20 @@ def is_anthropic_invalid_thinking_block_error(error_text: str) -> bool:
return "must contain thinking" in lower
def is_anthropic_thinking_binding_mismatch_error(error_text: str) -> bool:
"""messages.N.content.M: Invalid `signature` in `thinking` block. The block is bound to a different conversation."""
return "bound to a different conversation" in error_text.lower()
def anthropic_request_errors_on_thinking_binding_mismatch(request_body: object) -> bool:
"""``thinking.block_binding.prefix_mismatch_behavior: error`` is the caller asking
for the 400, so a retry that strips thinking must not override it."""
body: Final = validated_object_mapping(request_body)
thinking: Final = validated_object_mapping(body.get("thinking")) if body is not None else None
binding: Final = validated_object_mapping(thinking.get("block_binding")) if thinking is not None else None
return binding is not None and binding.get("prefix_mismatch_behavior") == "error"
def strip_thinking_blocks_from_anthropic_messages(messages: Sequence[object]) -> list[object]:
"""
Return a new message list with thinking / redacted_thinking content blocks removed
@ -1915,7 +1931,7 @@ def _replayed_server_tool_use(block: object) -> _ReplayedServerToolUse | None:
def tool_call_is_rebuilt_as_server_tool_use(
tool_call_id: object, provider_specific_fields: object, answered_tool_ids: Container[str] = frozenset()
) -> bool:
fields: Final = _validated_claude_code_mapping(provider_specific_fields)
fields: Final = validated_object_mapping(provider_specific_fields)
if not isinstance(tool_call_id, str) or tool_call_id in answered_tool_ids or fields is None:
return False
return (

18
litellm/llms/base_llm/anthropic_messages/transformation.py
View file

@ -1,6 +1,6 @@
from abc import ABC, abstractmethod
from collections.abc import AsyncIterator
from typing import TYPE_CHECKING, Any
from typing import TYPE_CHECKING, Any, Final
import httpx
@ -179,16 +179,28 @@ class BaseAnthropicMessagesConfig(ABC):
"""
return 2
def should_retry_anthropic_messages_on_http_error(self, e: httpx.HTTPStatusError, litellm_params: dict) -> bool:
def should_retry_anthropic_messages_on_http_error(
self, e: httpx.HTTPStatusError, litellm_params: dict, request_body: object = None
) -> bool:
"""
When True, async_anthropic_messages_handler will transform the request body
and issue one more attempt (bounded by max_retry_on_anthropic_messages_http_error).
A binding-mismatch 400 is not retried when the body's ``thinking.block_binding``
asked for it, since the retry would strip the very blocks the caller pinned.
"""
from litellm.llms.anthropic.common_utils import (
anthropic_request_errors_on_thinking_binding_mismatch,
is_anthropic_invalid_thinking_block_error,
is_anthropic_thinking_binding_mismatch_error,
)
return e.response.status_code == 400 and is_anthropic_invalid_thinking_block_error(e.response.text)
if e.response.status_code != 400:
return False
error_text: Final[str] = e.response.text
caller_asked_for_this_error: Final = anthropic_request_errors_on_thinking_binding_mismatch(
request_body
) and is_anthropic_thinking_binding_mismatch_error(error_text)
return not caller_asked_for_this_error and is_anthropic_invalid_thinking_block_error(error_text)
def transform_anthropic_messages_request_on_http_error(self, e: httpx.HTTPStatusError, request_data: dict) -> dict:
"""

3
litellm/llms/bedrock/chat/invoke_transformations/anthropic_claude3_transformation.py
View file

@ -122,8 +122,7 @@ class AmazonAnthropicClaudeConfig(AmazonInvokeConfig, AnthropicConfig):
and isinstance(translated_thinking, dict)
and translated_thinking.get("type") == "adaptive"
):
thinking_with_display: Final[AnthropicThinkingParam] = {"type": "adaptive", "display": "updates"}
mapped_params["thinking"] = thinking_with_display
mapped_params["thinking"] = {**translated_thinking, "display": "updates"}
return mapped_params

20
litellm/llms/bedrock/messages/invoke_transformations/anthropic_claude3_transformation.py
View file

@ -17,7 +17,7 @@ from litellm.llms.anthropic.chat.transformation import (
DROP_UNSUPPORTED_OUTPUT_CONFIG_WARNING,
AnthropicConfig,
)
from litellm.llms.anthropic.common_utils import AnthropicModelInfo
from litellm.llms.anthropic.common_utils import AnthropicModelInfo, validated_object_mapping
from litellm.llms.anthropic.pass_through.messages.transformation import (
AnthropicMessagesConfig,
)
@ -49,7 +49,6 @@ from litellm.types.llms.anthropic import (
ANTHROPIC_BETA_HEADER_VALUES,
ANTHROPIC_FINE_GRAINED_TOOL_STREAMING_BETA_HEADER,
ANTHROPIC_TOOL_SEARCH_BETA_HEADER,
AnthropicThinkingParam,
)
from litellm.types.llms.bedrock import BedrockInvokeAnthropicMessagesRequest
from litellm.types.llms.openai import AllMessageValues
@ -349,18 +348,10 @@ class AmazonAnthropicClaudeMessagesConfig(
if not isinstance(output_config, dict):
output_config = {}
output_config.setdefault("effort", self._effort_from_thinking_budget(budget_tokens))
thinking: Final = anthropic_messages_request.get("thinking")
display: Final = thinking.get("display") if isinstance(thinking, dict) else None
thinking: Final = validated_object_mapping(anthropic_messages_request.get("thinking"))
carried: Final = {key: value for key, value in (thinking or {}).items() if key not in ("type", "budget_tokens")}
anthropic_messages_request["output_config"] = output_config
if display is None:
adaptive_thinking: Final[AnthropicThinkingParam] = {"type": "adaptive"}
anthropic_messages_request["thinking"] = adaptive_thinking
else:
adaptive_thinking_with_display: Final[AnthropicThinkingParam] = {
"type": "adaptive",
"display": display,
}
anthropic_messages_request["thinking"] = adaptive_thinking_with_display
anthropic_messages_request["thinking"] = {**carried, "type": "adaptive"}
verbose_logger.debug(
"Bedrock clear_thinking_20251015: injected adaptive thinking with effort=%s for model=%s",
output_config.get("effort"),
@ -691,8 +682,7 @@ class AmazonAnthropicClaudeMessagesConfig(
and isinstance(translated_thinking, dict)
and translated_thinking.get("type") == "adaptive"
):
thinking_with_display: Final[AnthropicThinkingParam] = {"type": "adaptive", "display": "updates"}
anthropic_messages_request["thinking"] = thinking_with_display
anthropic_messages_request["thinking"] = {**translated_thinking, "display": "updates"}
self._normalize_system_role_messages(anthropic_messages_request, model=model)
#########################################################
############## BEDROCK Invoke SPECIFIC TRANSFORMATION ###

4
litellm/llms/custom_httpx/llm_http_handler.py
View file

@ -2112,7 +2112,7 @@ class BaseLLMHTTPHandler:
# str when the caller passes a pre-serialized (unsigned) body to avoid
# re-dumping; bytes when a provider signed the request (e.g. Bedrock).
signed_json_body: str | bytes | None,
request_body: dict,
request_body: dict[str, object],
stream: bool,
logging_obj: LiteLLMLoggingObj,
provider_config: BaseAnthropicMessagesConfig,
@ -2139,7 +2139,7 @@ class BaseLLMHTTPHandler:
except httpx.HTTPStatusError as e:
hit_max_attempt = attempt_idx + 1 == max_attempts
should_retry = provider_config.should_retry_anthropic_messages_on_http_error(
e=e, litellm_params=litellm_params_dict
e=e, litellm_params=litellm_params_dict, request_body=request_body
)
if should_retry and not hit_max_attempt:
if logging_obj.baseline_cache_context is not None:

5
litellm/types/llms/anthropic.py
View file

@ -761,10 +761,15 @@ ANTHROPIC_API_ONLY_HEADERS: Final = { # fails if calling anthropic on vertex ai
}
class AnthropicThinkingBlockBinding(TypedDict, total=False):
prefix_mismatch_behavior: ReadOnly[Literal["error", "drop_block"]]
class AnthropicThinkingParam(TypedDict, total=False):
type: ReadOnly[Literal["enabled", "adaptive", "disabled"]]
budget_tokens: int
display: ReadOnly[Literal["summarized", "omitted", "updates"]]
block_binding: ReadOnly[AnthropicThinkingBlockBinding]
class ANTHROPIC_HOSTED_TOOLS(str, Enum):

94
tests/unit/litellm_core_utils/prompt_templates/test_litellm_core_utils_prompt_templates_factory.py
View file

@ -36,7 +36,7 @@ from litellm.litellm_core_utils.prompt_templates.factory import (
sanitize_messages_for_tool_calling,
THOUGHT_SIGNATURE_SEPARATOR,
)
from litellm.types.llms.openai import ChatCompletionToolMessage
from litellm.types.llms.openai import ChatCompletionAssistantMessage, ChatCompletionToolMessage
from litellm.utils import (
_invalidate_model_cost_lowercase_map,
function_setup,
@ -4571,44 +4571,42 @@ def test_anthropic_messages_pt_drops_a_system_message_with_no_text():
assert [m["role"] for m in result] == ["user", "assistant"]
def test_anthropic_messages_pt_drops_empty_but_signed_thinking_block():
"""
Anthropic rejects a `thinking` block whose `thinking` text is empty, even
when it carries a valid-looking signature, with:
400 messages.N.content.M.thinking: each thinking block must contain thinking
This shape is reachable via cross-provider replay of a `thinking_blocks`
history item (see PR #36033), so `is_unsignable_thinking_block()` must
also check the thinking text, not just the signature.
"""
@pytest.mark.parametrize(
"assistant_message",
(
{
"role": "assistant",
"content": "4",
"thinking_blocks": [{"type": "thinking", "thinking": "", "signature": "sig_abc123_looks_valid"}],
},
{
"role": "assistant",
"content": [
{"type": "thinking", "thinking": "", "signature": "sig_abc123_looks_valid"},
{"type": "text", "text": "4"},
],
},
),
ids=("thinking_blocks", "inline_content"),
)
def test_anthropic_messages_pt_keeps_empty_but_signed_thinking_block(
assistant_message: ChatCompletionAssistantMessage,
) -> None:
"""``display: omitted`` returns signed thinking blocks with empty text, and they
must reach the next turn unchanged."""
from litellm.litellm_core_utils.prompt_templates.factory import (
anthropic_messages_pt,
)
messages = [
{"role": "user", "content": "What's 2+2?"},
{
"role": "assistant",
"content": "4",
"thinking_blocks": [
{
"type": "thinking",
"thinking": "",
"signature": "sig_abc123_looks_valid",
}
],
},
]
result = anthropic_messages_pt(
messages=messages,
model="claude-sonnet-4-5-20250929",
messages=[{"role": "user", "content": "What's 2+2?"}, assistant_message],
model="claude-opus-5-5",
llm_provider="anthropic",
)
assistant_msg = result[1]
assert isinstance(assistant_msg["content"], list)
content_types = [block.get("type") for block in assistant_msg["content"]]
assert "thinking" not in content_types, "empty-text thinking block must be dropped even though it has a signature"
assert assistant_msg["content"][0] == {"type": "thinking", "thinking": "", "signature": "sig_abc123_looks_valid"}
def test_anthropic_messages_pt_keeps_non_empty_signed_thinking_block():
@ -4722,29 +4720,31 @@ def test_anthropic_messages_pt_drops_unsigned_thinking_block():
assert "thinking" not in content_types, "unsigned thinking block must still be dropped"
def test_is_unsignable_thinking_block_treats_whitespace_only_as_empty():
def test_anthropic_messages_pt_drops_unsigned_empty_thinking_block():
"""
Edge case: a `thinking` field that is present but whitespace-only (e.g.
a single trailing newline forwarded from another provider's empty
reasoning summary) is functionally empty and Anthropic's API will still
reject it with "each thinking block must contain thinking". We treat it
the same as a fully empty string and drop the block.
The check lives in the shared `is_unsignable_thinking_block` helper, which
`_replayable_thinking_blocks` calls standalone, so the whitespace-aware
test has to hold there rather than only at the factory call site.
An empty block without a signature is the shape Claude Code replays after a
non-Anthropic turn, and Anthropic rejects it, so only the missing signature
decides whether a block is dropped.
"""
from litellm.litellm_core_utils.prompt_templates.common_utils import (
is_unsignable_thinking_block,
from litellm.litellm_core_utils.prompt_templates.factory import (
anthropic_messages_pt,
)
whitespace_only_block = {
"type": "thinking",
"thinking": " \n\t ",
"signature": "sig_abc123_looks_valid",
}
result = anthropic_messages_pt(
messages=[
{"role": "user", "content": "What's 2+2?"},
{
"role": "assistant",
"content": [{"type": "thinking", "thinking": "", "signature": ""}, {"type": "text", "text": "4"}],
},
],
model="claude-opus-5-5",
llm_provider="anthropic",
)
assert is_unsignable_thinking_block(whitespace_only_block) is True
assistant_msg = result[1]
assert isinstance(assistant_msg["content"], list)
assert [block.get("type") for block in assistant_msg["content"]] == ["text"]
_CONTENT_LESS_USER_MESSAGES: Final = ({"role": "user"}, {"role": "user", "content": None})

2
tests/unit/litellm_core_utils/prompt_templates/test_litellm_core_utils_prompt_templates_mid_conversation_system.py
View file

@ -394,6 +394,7 @@ def test_flagged_placement_keeps_a_system_whose_empty_assistant_follower_ends_th
},
{"role": "assistant", "content": None, "thinking_blocks": [{"type": "redacted_thinking", "data": "x"}]},
{"role": "assistant", "content": [{"type": "thinking", "thinking": "hm", "signature": "s"}]},
{"role": "assistant", "content": [{"type": "thinking", "thinking": "", "signature": "s"}]},
{"role": "assistant", "content": None, "function_call": {"name": "f", "arguments": "{}"}},
litellm.Message(
role="assistant",
@ -414,6 +415,7 @@ def test_flagged_placement_keeps_a_system_whose_empty_assistant_follower_ends_th
"signed-thinking-block",
"redacted-thinking-block",
"signed-thinking-part",
"signed-empty-thinking-part",
"function-call",
"pydantic-tool-calls",
"unanswered-tool-search-use-beside-a-client-call",

74
tests/unit/llms/anthropic/test_anthropic_common_utils.py
View file

@ -14,6 +14,7 @@ import json
import os
import sys
import threading
from collections.abc import Mapping
from pathlib import Path
from types import SimpleNamespace
from typing import Final
@ -1385,6 +1386,12 @@ class TestPassthroughAuthToken:
assert url == "https://custom.example.com/v1/messages"
_BOUND_THINKING_ERROR: Final = (
"messages.1.content.0: Invalid `signature` in `thinking` block. The block is bound to a different conversation."
)
_NULL_SIGNATURE_ERROR: Final = "messages.1.content.0.thinking.signature.str: Input should be a valid string"
class TestAnthropicThinkingSignatureSelfHeal:
"""Helpers for retrying after invalid thinking blocks in replayed history:
invalid encrypted signatures, and blocks with empty thinking text."""
@ -2081,6 +2088,49 @@ class TestAnthropicThinkingSignatureSelfHeal:
assert "thinking" not in data
assert data["messages"] == []
@pytest.mark.parametrize(
("thinking", "error_message", "expected"),
(
(
{"type": "adaptive", "block_binding": {"prefix_mismatch_behavior": "error"}},
_BOUND_THINKING_ERROR,
False,
),
(
{"type": "adaptive", "block_binding": {"prefix_mismatch_behavior": "error"}},
_NULL_SIGNATURE_ERROR,
True,
),
(
{"type": "adaptive", "block_binding": {"prefix_mismatch_behavior": "drop_block"}},
_BOUND_THINKING_ERROR,
True,
),
({"type": "adaptive"}, _BOUND_THINKING_ERROR, True),
),
ids=("error", "error_on_unrelated_bad_block", "drop_block", "unbound"),
)
def test_anthropic_messages_retry_honors_thinking_block_binding(
self, thinking: Mapping[str, object], error_message: str, expected: bool
) -> None:
from litellm.llms.anthropic.pass_through.messages.transformation import (
AnthropicMessagesConfig,
)
req: Final = httpx.Request("POST", "https://api.anthropic.com/v1/messages")
resp: Final = httpx.Response(
400,
request=req,
text=json.dumps({"type": "error", "error": {"type": "invalid_request_error", "message": error_message}}),
)
err: Final = httpx.HTTPStatusError("bad", request=req, response=resp)
should_retry = AnthropicMessagesConfig().should_retry_anthropic_messages_on_http_error(
err, {}, request_body={"model": "claude-opus-5-5", "thinking": thinking}
)
assert should_retry is expected
class TestClaudeOpus48AdaptiveThinking:
"""Opus 4.8 requires adaptive thinking (``thinking.type='adaptive'`` +
@ -4209,6 +4259,30 @@ def test_shared_legacy_thinking_translation_preserves_supported_display(
assert optional_params["thinking"] == expected_thinking
@pytest.mark.usefixtures("local_model_cost_map")
def test_shared_legacy_thinking_translation_carries_block_binding() -> None:
from litellm.llms.anthropic.common_utils import AnthropicModelInfo
optional_params: Final = {
"thinking": {
"type": "enabled",
"budget_tokens": 2048,
"block_binding": {"prefix_mismatch_behavior": "error"},
},
}
AnthropicModelInfo.translate_legacy_thinking_for_adaptive_model(
model="claude-opus-5",
optional_params=optional_params,
custom_llm_provider="azure_ai",
)
assert optional_params["thinking"] == {
"type": "adaptive",
"block_binding": {"prefix_mismatch_behavior": "error"},
}
@pytest.mark.usefixtures("local_model_cost_map", "local_beta_headers_config")
@pytest.mark.parametrize("action", (None, "tool_addition", "tool_removal"))
@pytest.mark.parametrize("explicit_beta", (False, True))

57
tests/unit/llms/bedrock/chat/invoke_transformations/test_bedrock_chat_invoke_transformations_anthropic_claude3_transformation.py
View file

@ -761,6 +761,30 @@ def test_bedrock_chat_invoke_tool_based_response_format_still_upgrades_legacy_th
assert result["output_config"] == {"effort": "high"}
def test_bedrock_chat_invoke_adaptive_thinking_keeps_the_forced_response_format_tool(local_model_cost_map):
"""Anthropic accepts a forced tool_choice next to adaptive thinking, and dropping
it lets the model answer in prose instead of the requested JSON."""
result = AmazonAnthropicClaudeConfig().map_openai_params(
non_default_params={
"response_format": {
"type": "json_schema",
"json_schema": {
"name": "test_schema",
"schema": {"type": "object", "properties": {"result": {"type": "string"}}},
},
},
"thinking": {"type": "adaptive"},
"max_tokens": 8192,
},
optional_params={},
model="us.anthropic.claude-opus-5",
drop_params=False,
)
assert result["tool_choice"] == {"type": "tool", "name": result["tools"][0]["name"]}
assert result["thinking"] == {"type": "adaptive"}
def test_bedrock_chat_invoke_response_format_stub_still_upgrades_legacy_thinking(local_model_cost_map):
"""Regression: the tool-based ``response_format`` path swaps in a Claude 3 stub
model before the shared Anthropic mapping, which hid the adaptive-only model
@ -975,6 +999,39 @@ def test_bedrock_chat_invoke_preserves_display_when_translating_legacy_thinking(
assert ANTHROPIC_THINKING_DISPLAY_UPDATES_BETA_HEADER in result.get("anthropic_beta", [])
BLOCK_BINDING: Final = {"prefix_mismatch_behavior": "error"}
@pytest.mark.parametrize(
"requested_thinking",
(
{"type": "adaptive", "display": "updates", "block_binding": BLOCK_BINDING},
{"type": "enabled", "budget_tokens": 2048, "display": "updates", "block_binding": BLOCK_BINDING},
),
ids=("adaptive", "legacy"),
)
def test_bedrock_chat_invoke_keeps_block_binding_with_display_updates(
local_model_cost_map, local_beta_headers_config, requested_thinking
) -> None:
config: Final = AmazonAnthropicClaudeConfig()
model: Final = "us.anthropic.claude-opus-5"
optional_params: Final = config.map_openai_params(
non_default_params={"max_tokens": 512, "thinking": requested_thinking},
optional_params={},
model=model,
drop_params=False,
)
result: Final = config.transform_request(
model=model,
messages=[{"role": "user", "content": "Reply with OK"}],
optional_params=optional_params,
litellm_params={},
headers={},
)
assert result.get("thinking") == {"type": "adaptive", "display": "updates", "block_binding": BLOCK_BINDING}
FINE_GRAINED_TOOL_STREAMING_BETA: Final = "fine-grained-tool-streaming-2025-05-14"
EAGER_TOOL_SCHEMA: Final = {"type": "object", "properties": {"path": {"type": "string"}}, "required": ["path"]}

37
tests/unit/llms/bedrock/messages/invoke_transformations/test_anthropic_claude3_transformation.py
View file

@ -3690,6 +3690,43 @@ def test_bedrock_clear_thinking_preserves_display_updates() -> None:
assert ANTHROPIC_THINKING_DISPLAY_UPDATES_BETA_HEADER in result.get("anthropic_beta", [])
BLOCK_BINDING: Final = {"prefix_mismatch_behavior": "error"}
@pytest.mark.usefixtures("local_model_cost_map", "local_beta_headers_config")
@pytest.mark.parametrize(
("model", "requested_thinking", "extra_params"),
(
("eu.anthropic.claude-opus-5", {"type": "adaptive", "display": "updates", "block_binding": BLOCK_BINDING}, {}),
(
"eu.anthropic.claude-opus-5",
{"type": "enabled", "budget_tokens": 24000, "display": "updates", "block_binding": BLOCK_BINDING},
{},
),
(
"us.anthropic.claude-opus-4-6",
{"type": "enabled", "budget_tokens": 2048, "display": "updates", "block_binding": BLOCK_BINDING},
{"context_management": {"edits": [{"type": "clear_thinking_20251015"}]}},
),
),
ids=("adaptive", "legacy", "legacy_clear_thinking"),
)
def test_bedrock_messages_keeps_block_binding_with_display_updates(
model: str, requested_thinking: dict[str, object], extra_params: dict[str, object]
) -> None:
from litellm.types.router import GenericLiteLLMParams
result: Final = AmazonAnthropicClaudeMessagesConfig().transform_anthropic_messages_request(
model=model,
messages=[{"role": "user", "content": "Reply with OK"}],
anthropic_messages_optional_request_params={"max_tokens": 512, "thinking": requested_thinking, **extra_params},
litellm_params=GenericLiteLLMParams(),
headers={},
)
assert result.get("thinking") == {"type": "adaptive", "display": "updates", "block_binding": BLOCK_BINDING}
@pytest.mark.usefixtures("local_model_cost_map", "local_beta_headers_config")
@pytest.mark.parametrize("action", (None, "tool_addition", "tool_removal"))
@pytest.mark.parametrize("explicit_beta", (False, True))

65
tests/unit/llms/custom_httpx/test_llm_http_handler.py
View file

@ -1892,6 +1892,71 @@ async def test_anthropic_post_retry_reserializes_mutated_body():
assert "MUTATED" in second_sent # ... the mutated body
_BOUND_THINKING_ERROR: Final = (
'{"type":"error","error":{"type":"invalid_request_error","message":"messages.1.content.0: '
'Invalid `signature` in `thinking` block. The block is bound to a different conversation."}}'
)
@pytest.mark.asyncio
@pytest.mark.parametrize(
("prefix_mismatch_behavior", "expected_posts"),
(("error", 1), ("drop_block", 2)),
)
async def test_anthropic_post_skips_the_thinking_strip_retry_when_block_binding_errors(
prefix_mismatch_behavior: str, expected_posts: int
) -> None:
"""A caller that pinned ``prefix_mismatch_behavior: error`` asked for the 400,
so the handler must surface it instead of stripping the bound blocks and retrying."""
from litellm.llms.anthropic.pass_through.messages.transformation import AnthropicMessagesConfig
handler = BaseLLMHTTPHandler()
thinking_block = {"type": "thinking", "thinking": "", "signature": "sig"}
request_body = {
"model": "claude-opus-5-5",
"messages": [
{"role": "user", "content": "hi"},
{"role": "assistant", "content": [thinking_block, {"type": "text", "text": "hello"}]},
],
"thinking": {"type": "adaptive", "block_binding": {"prefix_mismatch_behavior": prefix_mismatch_behavior}},
}
request = httpx.Request("POST", "https://api.anthropic.com/v1/messages")
http_client = Mock()
http_client.post = AsyncMock(
side_effect=[
httpx.Response(400, request=request, text=_BOUND_THINKING_ERROR),
httpx.Response(200, request=request, json={"type": "message", "content": []}),
]
)
logging_obj: Final = Mock(baseline_cache_context=None)
logging_obj.model_call_details = {}
async def _post() -> httpx.Response:
return await handler._async_post_anthropic_messages_with_http_error_retry(
async_httpx_client=http_client,
request_url="https://api.anthropic.com/v1/messages",
headers={},
signed_json_body=None,
request_body=request_body,
stream=False,
logging_obj=logging_obj,
provider_config=AnthropicMessagesConfig(),
litellm_params=GenericLiteLLMParams(),
api_key="k",
model="claude-opus-5-5",
)
if expected_posts == 1:
with pytest.raises(BaseLLMException) as raised:
await _post()
assert raised.value.status_code == 400
assert "bound to a different conversation" in raised.value.message
assert request_body["messages"][1]["content"][0] == thinking_block
else:
assert (await _post()).status_code == 200
assert http_client.post.await_count == expected_posts
def test_base_responses_config_sign_request_is_noop_by_default():
"""Default responses sign_request must be a no-op: unchanged headers, no signed body.