scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-11 22:51:28 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 124

Merge pull request #25867 from BerriAI/litellm_day_0_opus_4.7_support

Browse source 
Litellm day 0 opus 4.7 support
This commit is contained in:
ishaan-berri 2026-04-16 09:42:11 -07:00 • committed by Sameer Kankute
parent 7e4e4545c5
commit e8516483d7
No known key found for this signature in database
13 changed files with 3136 additions and 256 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

366
docs/my-website/blog/claude_opus_4_7/index.md Normal file
View file

@ -0,0 +1,366 @@
---
slug: claude_opus_4_7
title: "Day 0 Support: Claude Opus 4.7"
date: 2026-04-16T10:00:00
authors:
- sameer
- ishaan-alt
- krrish
description: "Day 0 support for Claude Opus 4.7 on LiteLLM AI Gateway - use across Anthropic, Azure, Vertex AI, and Bedrock."
tags: [anthropic, claude, opus 4.7]
hide_table_of_contents: false
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
LiteLLM now supports [Claude Opus 4.7](https://www.anthropic.com/news/claude-opus-4-7) on Day 0. Use it across Anthropic, Azure, Vertex AI, and Bedrock through the LiteLLM AI Gateway.
{/* truncate */}
## Docker Image
```bash
docker pull ghcr.io/berriai/litellm:litellm_stable_release_branch-v1.82.0-stable.opus-4-7
```
## Usage - Anthropic
<Tabs>
<TabItem value="proxy" label="LiteLLM Proxy">
**1. Setup config.yaml**
```yaml
model_list:
- model_name: claude-opus-4-7
litellm_params:
model: anthropic/claude-opus-4-7
api_key: os.environ/ANTHROPIC_API_KEY
```
**2. Start the proxy**
```bash
docker run -d \
-p 4000:4000 \
-e ANTHROPIC_API_KEY=$ANTHROPIC_API_KEY \
-v $(pwd)/config.yaml:/app/config.yaml \
ghcr.io/berriai/litellm:litellm_stable_release_branch-v1.82.0-stable.opus-4-7 \
--config /app/config.yaml
```
**3. Test it!**
```bash
curl --location 'http://0.0.0.0:4000/chat/completions' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer $LITELLM_KEY' \
--data '{
"model": "claude-opus-4-7",
"messages": [
{
"role": "user",
"content": "what llm are you"
}
]
}'
```
</TabItem>
</Tabs>
## Usage - Azure
<Tabs>
<TabItem value="proxy" label="LiteLLM Proxy">
**1. Setup config.yaml**
```yaml
model_list:
- model_name: claude-opus-4-7
litellm_params:
model: azure_ai/claude-opus-4-7
api_key: os.environ/AZURE_AI_API_KEY
api_base: os.environ/AZURE_AI_API_BASE # https://<resource>.services.ai.azure.com
```
**2. Start the proxy**
```bash
docker run -d \
-p 4000:4000 \
-e AZURE_AI_API_KEY=$AZURE_AI_API_KEY \
-e AZURE_AI_API_BASE=$AZURE_AI_API_BASE \
-v $(pwd)/config.yaml:/app/config.yaml \
ghcr.io/berriai/litellm:litellm_stable_release_branch-v1.82.0-stable.opus-4-7 \
--config /app/config.yaml
```
**3. Test it!**
```bash
curl --location 'http://0.0.0.0:4000/chat/completions' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer $LITELLM_KEY' \
--data '{
"model": "claude-opus-4-7",
"messages": [
{
"role": "user",
"content": "what llm are you"
}
]
}'
```
</TabItem>
</Tabs>
## Usage - Vertex AI
<Tabs>
<TabItem value="proxy" label="LiteLLM Proxy">
**1. Setup config.yaml**
```yaml
model_list:
- model_name: claude-opus-4-7
litellm_params:
model: vertex_ai/claude-opus-4-7
vertex_project: os.environ/VERTEX_PROJECT
vertex_location: us-east5
```
**2. Start the proxy**
```bash
docker run -d \
-p 4000:4000 \
-e VERTEX_PROJECT=$VERTEX_PROJECT \
-e GOOGLE_APPLICATION_CREDENTIALS=/app/credentials.json \
-v $(pwd)/config.yaml:/app/config.yaml \
-v $(pwd)/credentials.json:/app/credentials.json \
ghcr.io/berriai/litellm:litellm_stable_release_branch-v1.82.0-stable.opus-4-7 \
--config /app/config.yaml
```
**3. Test it!**
```bash
curl --location 'http://0.0.0.0:4000/chat/completions' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer $LITELLM_KEY' \
--data '{
"model": "claude-opus-4-7",
"messages": [
{
"role": "user",
"content": "what llm are you"
}
]
}'
```
</TabItem>
</Tabs>
## Usage - Bedrock
<Tabs>
<TabItem value="proxy" label="LiteLLM Proxy">
**1. Setup config.yaml**
```yaml
model_list:
- model_name: claude-opus-4-7
litellm_params:
model: bedrock/anthropic.claude-opus-4-7
aws_access_key_id: os.environ/AWS_ACCESS_KEY_ID
aws_secret_access_key: os.environ/AWS_SECRET_ACCESS_KEY
aws_region_name: us-east-1
```
**2. Start the proxy**
```bash
docker run -d \
-p 4000:4000 \
-e AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID \
-e AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY \
-v $(pwd)/config.yaml:/app/config.yaml \
ghcr.io/berriai/litellm:litellm_stable_release_branch-v1.82.0-stable.opus-4-7 \
--config /app/config.yaml
```
**3. Test it!**
```bash
curl --location 'http://0.0.0.0:4000/chat/completions' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer $LITELLM_KEY' \
--data '{
"model": "claude-opus-4-7",
"messages": [
{
"role": "user",
"content": "what llm are you"
}
]
}'
```
</TabItem>
</Tabs>
## Advanced Features
### Adaptive Thinking
:::note
When using `reasoning_effort` with Claude Opus 4.7, all values (`low`, `medium`, `high`, `xhigh`) are mapped to `thinking: {type: "adaptive"}`. To use explicit thinking budgets with `type: "enabled"`, pass the native `thinking` parameter directly.
:::
<Tabs>
<TabItem value="completions" label="/chat/completions">
LiteLLM supports adaptive thinking through the `reasoning_effort` parameter:
```bash
curl --location 'http://0.0.0.0:4000/chat/completions' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer $LITELLM_KEY' \
--data '{
"model": "claude-opus-4-7",
"messages": [
{
"role": "user",
"content": "Solve this complex problem: What is the optimal strategy for..."
}
],
"reasoning_effort": "high"
}'
```
</TabItem>
<TabItem value="messages" label="/v1/messages">
Use the `thinking` parameter with `type: "adaptive"` to enable adaptive thinking mode:
```bash
curl --location 'http://0.0.0.0:4000/v1/messages' \
--header 'x-api-key: sk-12345' \
--header 'content-type: application/json' \
--data '{
"model": "claude-opus-4-7",
"max_tokens": 16000,
"thinking": {
"type": "adaptive"
},
"messages": [
{
"role": "user",
"content": "Explain why the sum of two even numbers is always even."
}
]
}'
```
</TabItem>
</Tabs>
### Effort Levels
Claude Opus 4.7 supports four effort levels: `low`, `medium`, `high` (default), and `xhigh`. These give you finer-grained control over how much reasoning the model applies to a task. Pass the effort level via the `output_config` parameter.
`xhigh` is a new effort level introduced with Opus 4.7 that sits above `high`. The `max` effort level is Claude Opus 4.6 only and is not available on 4.7.
<Tabs>
<TabItem value="completions" label="/chat/completions">
```bash
curl --location 'http://0.0.0.0:4000/chat/completions' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer $LITELLM_KEY' \
--data '{
"model": "claude-opus-4-7",
"messages": [
{
"role": "user",
"content": "Explain quantum computing"
}
],
"output_config": {
"effort": "xhigh"
}
}'
```
**Using OpenAI SDK:**
```python
import openai
client = openai.OpenAI(
api_key="your-litellm-key",
base_url="http://0.0.0.0:4000"
)
response = client.chat.completions.create(
model="claude-opus-4-7",
messages=[{"role": "user", "content": "Explain quantum computing"}],
extra_body={"output_config": {"effort": "xhigh"}}
)
```
**Using LiteLLM SDK:**
```python
from litellm import completion
response = completion(
model="anthropic/claude-opus-4-7",
messages=[{"role": "user", "content": "Explain quantum computing"}],
output_config={"effort": "xhigh"},
)
```
You can combine `reasoning_effort` with `output_config` for even more fine-grained control over the model's behavior.
</TabItem>
<TabItem value="messages" label="/v1/messages">
```bash
curl --location 'http://0.0.0.0:4000/v1/messages' \
--header 'x-api-key: sk-12345' \
--header 'content-type: application/json' \
--data '{
"model": "claude-opus-4-7",
"max_tokens": 4096,
"messages": [
{
"role": "user",
"content": "Explain quantum computing"
}
],
"output_config": {
"effort": "xhigh"
}
}'
```
</TabItem>
</Tabs>
**Effort level guide:**
| Effort | When to use |
|--------|-------------|
| `low` | Short, fast responses — simple lookups, formatting, classification |
| `medium` | Balanced tradeoff for everyday Q&A and light reasoning |
| `high` (default) | Complex reasoning, code generation, analysis |
| `xhigh` | Hardest problems — multi-step math, deep research, agentic planning |

10
litellm/anthropic_beta_headers_config.json
View file

@ -67,13 +67,13 @@
"compact-2026-01-12": null,
"computer-use-2025-01-24": "computer-use-2025-01-24",
"computer-use-2025-11-24": "computer-use-2025-11-24",
"context-1m-2025-08-07": null,
"context-management-2025-06-27": "context-management-2025-06-27",
"context-1m-2025-08-07": "context-1m-2025-08-07",
"context-management-2025-06-27": null,
"effort-2025-11-24": null,
"fast-mode-2026-02-01": null,
"files-api-2025-04-14": null,
"fine-grained-tool-streaming-2025-05-14": null,
"interleaved-thinking-2025-05-14": "interleaved-thinking-2025-05-14",
"interleaved-thinking-2025-05-14": null,
"mcp-client-2025-11-20": null,
"mcp-client-2025-04-04": null,
"mcp-servers-2025-12-04": null,
@ -98,12 +98,12 @@
"computer-use-2025-01-24": "computer-use-2025-01-24",
"computer-use-2025-11-24": "computer-use-2025-11-24",
"context-1m-2025-08-07": "context-1m-2025-08-07",
"context-management-2025-06-27": "context-management-2025-06-27",
"context-management-2025-06-27": null,
"effort-2025-11-24": null,
"fast-mode-2026-02-01": null,
"files-api-2025-04-14": null,
"fine-grained-tool-streaming-2025-05-14": null,
"interleaved-thinking-2025-05-14": "interleaved-thinking-2025-05-14",
"interleaved-thinking-2025-05-14": null,
"mcp-client-2025-11-20": null,
"mcp-client-2025-04-04": null,
"mcp-servers-2025-12-04": null,

1
litellm/constants.py
View file

@ -1034,6 +1034,7 @@ BEDROCK_CONVERSE_MODELS = [
"openai.gpt-oss-120b-1:0",
"anthropic.claude-haiku-4-5-20251001-v1:0",
"anthropic.claude-sonnet-4-5-20250929-v1:0",
"anthropic.claude-opus-4-7",
"anthropic.claude-opus-4-6-v1:0",
"anthropic.claude-opus-4-6-v1",
"anthropic.claude-opus-4-1-20250805-v1:0",

92
litellm/llms/anthropic/chat/transformation.py
View file

@ -61,6 +61,7 @@ from litellm.types.utils import (
from litellm.utils import (
ModelResponse,
Usage,
_supports_factory,
add_dummy_tool,
any_assistant_message_has_thinking_blocks,
get_max_tokens,
@ -175,6 +176,30 @@ class AnthropicConfig(AnthropicModelInfo, BaseConfig):
"""Check if the model is Claude Opus 4.5."""
return "opus-4-6" in model.lower() or "opus_4_6" in model.lower()
@staticmethod
def _is_opus_4_7_model(model: str) -> bool:
"""Check if the model is specifically Claude Opus 4.7."""
model_lower = model.lower()
return any(
v in model_lower for v in ("opus-4-7", "opus_4_7", "opus-4.7", "opus_4.7")
)
@staticmethod
def _supports_effort_level(model: str, level: str) -> bool:
"""Check ``supports_{level}_reasoning_effort`` in the model map.
Mirrors the pattern used in ``openai/chat/gpt_5_transformation.py`` so
that adding support for a new effort level is a pure model-map change.
"""
try:
return _supports_factory(
model=model,
custom_llm_provider="anthropic",
key=f"supports_{level}_reasoning_effort",
)
except Exception:
return False
def get_supported_openai_params(self, model: str):
params = [
"stream",
@ -193,9 +218,14 @@ class AnthropicConfig(AnthropicModelInfo, BaseConfig):
"speed",
]
if "claude-3-7-sonnet" in model or supports_reasoning(
model=model,
custom_llm_provider=self.custom_llm_provider,
if (
"claude-3-7-sonnet" in model
or AnthropicConfig._is_claude_4_6_model(model)
or AnthropicConfig._is_claude_4_7_model(model)
or supports_reasoning(
model=model,
custom_llm_provider=self.custom_llm_provider,
)
):
params.append("thinking")
params.append("reasoning_effort")
@ -710,7 +740,9 @@ class AnthropicConfig(AnthropicModelInfo, BaseConfig):
) -> Optional[AnthropicThinkingParam]:
if reasoning_effort is None or reasoning_effort == "none":
return None
if AnthropicConfig._is_claude_opus_4_6(model):
if AnthropicConfig._is_claude_4_6_model(
model
) or AnthropicConfig._is_claude_4_7_model(model):
return AnthropicThinkingParam(
type="adaptive",
)
@ -881,6 +913,12 @@ class AnthropicConfig(AnthropicModelInfo, BaseConfig):
"opus-4-5",
"opus-4.6",
"opus-4-6",
"opus-4.7",
"opus-4-7",
"sonnet-4.6",
"sonnet-4-6",
"sonnet_4.6",
"sonnet_4_6",
}
):
_output_format = (
@ -918,6 +956,21 @@ class AnthropicConfig(AnthropicModelInfo, BaseConfig):
optional_params["thinking"] = AnthropicConfig._map_reasoning_effort(
reasoning_effort=value, model=model
)
# For Claude 4.6+ models, effort is controlled via output_config,
# not thinking budget_tokens. Map reasoning_effort to output_config.
if AnthropicConfig._is_claude_4_6_model(
model
) or AnthropicConfig._is_claude_4_7_model(model):
effort_map = {
"low": "low",
"minimal": "low",
"medium": "medium",
"high": "high",
"xhigh": "xhigh",
"max": "max",
}
mapped_effort = effort_map.get(value, value)
optional_params["output_config"] = {"effort": mapped_effort}
elif param == "web_search_options" and isinstance(value, dict):
hosted_web_search_tool = self.map_web_search_tool(
cast(OpenAIWebSearchOptions, value)
@ -1290,6 +1343,37 @@ class AnthropicConfig(AnthropicModelInfo, BaseConfig):
return data
def _apply_output_config(
self, data: dict, model: str, optional_params: dict
) -> None:
"""Validate and apply output_config to the request data."""
if "output_config" not in optional_params:
return
output_config = optional_params.get("output_config")
if not output_config or not isinstance(output_config, dict):
return
effort = output_config.get("effort")
valid_efforts = ["high", "medium", "low", "xhigh", "max"]
if effort and effort not in valid_efforts:
raise ValueError(
f"Invalid effort value: {effort}. Must be one of: "
f"'high', 'medium', 'low', 'xhigh', 'max'"
)
# ``max`` is Claude Opus 4.6 only (not Sonnet 4.6, not Opus 4.5/4.7).
# Keep this hardcoded so the error message is specific and stable.
if effort == "max" and not self._is_opus_4_6_model(model):
raise ValueError(
f"effort='max' is only supported by Claude Opus 4.6. "
f"Got model: {model}"
)
# ``xhigh`` is data-driven via ``supports_xhigh_reasoning_effort`` so
# enabling it for a new model is a pure model-map change.
if effort == "xhigh" and not self._supports_effort_level(model, "xhigh"):
raise ValueError(
f"effort='xhigh' is not supported by this model. Got model: {model}"
)
data["output_config"] = output_config
def _transform_response_for_json_mode(
self,
json_mode: Optional[bool],

47
litellm/llms/anthropic/common_utils.py
View file

@ -215,17 +215,62 @@ class AnthropicModelInfo(BaseLLMModelInfo):
return False
@staticmethod
def _is_claude_4_6_model(model: str) -> bool:
"""Check if the model is a Claude 4.6 model (Opus 4.6 or Sonnet 4.6)."""
model_lower = model.lower()
return any(
v in model_lower
for v in (
"opus-4-6",
"opus_4_6",
"opus-4.6",
"opus_4.6",
"sonnet-4-6",
"sonnet_4_6",
"sonnet-4.6",
"sonnet_4.6",
)
)
@staticmethod
def _is_claude_4_7_model(model: str) -> bool:
"""Check if the model is a Claude 4.7 model (Opus 4.7)."""
model_lower = model.lower()
return any(
v in model_lower
for v in (
"opus-4-7",
"opus_4_7",
"opus-4.7",
"opus_4.7",
)
)
@staticmethod
def _is_adaptive_thinking_model(model: str) -> bool:
"""Claude 4.6+ models use adaptive thinking with output_config effort."""
return AnthropicModelInfo._is_claude_4_6_model(
model
) or AnthropicModelInfo._is_claude_4_7_model(model)
def is_effort_used(
self, optional_params: Optional[dict], model: Optional[str] = None
) -> bool:
"""
Check if effort parameter is being used.
Returns True if effort-related parameters are present.
Returns True if effort-related parameters are present and
the model requires the effort beta header. Claude 4.6+ models
use output_config as a stable API feature — no beta header needed.
"""
if not optional_params:
return False
# Claude 4.6+ models use output_config as a stable API feature — no beta header needed
if model and self._is_adaptive_thinking_model(model):
return False
# Check if reasoning_effort is provided for Claude Opus 4.5
if model and ("opus-4-5" in model.lower() or "opus_4_5" in model.lower()):
reasoning_effort = optional_params.get("reasoning_effort")

35
litellm/llms/anthropic/experimental_pass_through/messages/transformation.py
View file

@ -133,6 +133,36 @@ class AnthropicMessagesConfig(BaseAnthropicMessagesConfig):
return headers, api_base
@staticmethod
def _translate_legacy_thinking_for_adaptive_model(
model: str, optional_params: Dict
) -> None:
"""Translate legacy ``thinking.type=enabled`` to adaptive for 4.6/4.7.
Caller-provided ``output_config.effort`` is never overridden.
"""
if not AnthropicModelInfo._is_adaptive_thinking_model(model):
return
thinking = optional_params.get("thinking")
if not isinstance(thinking, dict) or thinking.get("type") != "enabled":
return
budget = int(thinking.get("budget_tokens") or 0)
if budget >= 24000:
effort = "xhigh"
elif budget >= 10000:
effort = "high"
elif budget >= 5000:
effort = "medium"
else:
effort = "low"
optional_params["thinking"] = {"type": "adaptive"}
existing_output_config = optional_params.get("output_config")
if not isinstance(existing_output_config, dict):
existing_output_config = {}
existing_output_config.setdefault("effort", effort)
optional_params["output_config"] = existing_output_config
def transform_anthropic_messages_request(
self,
model: str,
@ -154,6 +184,11 @@ class AnthropicMessagesConfig(BaseAnthropicMessagesConfig):
status_code=400,
)
self._translate_legacy_thinking_for_adaptive_model(
model=model,
optional_params=anthropic_messages_optional_request_params,
)
# Filter out x-anthropic-billing-header from system messages
system_param = anthropic_messages_optional_request_params.get("system")
if system_param is not None:

17
litellm/llms/bedrock/chat/converse_transformation.py
View file

@ -1094,11 +1094,24 @@ class AmazonConverseConfig(BaseConfig):
# Add computer use tools and anthropic_beta if needed (only when computer use tools are present)
if computer_use_tools:
# Determine the correct computer-use beta header based on model
# "computer-use-2025-11-24" for Claude Opus 4.6, Claude Opus 4.5
# "computer-use-2025-11-24" for Claude Opus 4.7, Opus 4.6, and Opus 4.5
# "computer-use-2025-01-24" for Claude Sonnet 4.5, Haiku 4.5, Opus 4.1, Sonnet 4, Opus 4, and Sonnet 3.7
# "computer-use-2024-10-22" for older models
model_lower = model.lower()
if "opus-4.6" in model_lower or "opus_4.6" in model_lower or "opus-4-6" in model_lower or "opus_4_6" in model_lower:
if (
"opus-4.7" in model_lower
or "opus_4.7" in model_lower
or "opus-4-7" in model_lower
or "opus_4_7" in model_lower
or "opus-4.6" in model_lower
or "opus_4.6" in model_lower
or "opus-4-6" in model_lower
or "opus_4_6" in model_lower
or "sonnet-4.6" in model_lower
or "sonnet_4.6" in model_lower
or "sonnet-4-6" in model_lower
or "sonnet_4_6" in model_lower
):
computer_use_header = "computer-use-2025-11-24"
elif "opus-4.5" in model_lower or "opus_4.5" in model_lower or "opus-4-5" in model_lower or "opus_4_5" in model_lower:
computer_use_header = "computer-use-2025-11-24"

12
litellm/llms/bedrock/common_utils.py
View file

@ -465,6 +465,18 @@ def is_claude_4_5_on_bedrock(model: str) -> bool:
"opus_4.5",
"opus-4-5",
"opus_4_5",
"sonnet-4.6",
"sonnet_4.6",
"sonnet-4-6",
"sonnet_4_6",
"opus-4.6",
"opus_4.6",
"opus-4-6",
"opus_4_6",
"opus-4.7",
"opus_4.7",
"opus-4-7",
"opus_4_7",
]
return any(pattern in model_lower for pattern in claude_4_5_patterns)

98
litellm/llms/bedrock/messages/invoke_transformations/anthropic_claude3_transformation.py
View file

@ -12,6 +12,9 @@ from typing import (
import httpx
from litellm.anthropic_beta_headers_manager import filter_and_transform_beta_headers
from litellm.constants import BEDROCK_MIN_THINKING_BUDGET_TOKENS
from litellm.litellm_core_utils.litellm_logging import verbose_logger
from litellm.llms.anthropic.common_utils import AnthropicModelInfo
from litellm.llms.anthropic.experimental_pass_through.messages.transformation import (
AnthropicMessagesConfig,
@ -180,10 +183,86 @@ class AmazonAnthropicClaudeMessagesConfig(
"opus_4", # Opus 4
"sonnet-4",
"sonnet_4", # Sonnet 4
"sonnet-4.6",
"sonnet_4.6",
"sonnet-4-6",
"sonnet_4_6",
"opus-4.6",
"opus_4.6",
"opus-4-6",
"opus_4_6",
"opus-4.7",
"opus_4.7",
"opus-4-7",
"opus_4_7",
]
return any(pattern in model_lower for pattern in supported_patterns)
def _ensure_thinking_for_clear_thinking_context_management(
self,
anthropic_messages_request: Dict,
model: str,
) -> bool:
"""
Bedrock rejects ``clear_thinking_20251015`` context-management edits unless
extended thinking is ``enabled`` or ``adaptive``. Claude Code often sends
context management without a top-level ``thinking`` field.
When we detect that edit type on a model that supports extended thinking on
Bedrock, inject a minimal ``thinking`` config so the request succeeds.
Returns:
True if ``thinking`` was added or upgraded for this fix (caller may
need to add the interleaved-thinking beta header).
"""
cm = anthropic_messages_request.get("context_management")
if not isinstance(cm, dict):
return False
edits = cm.get("edits")
if not isinstance(edits, list):
return False
needs_thinking = any(
isinstance(e, dict) and e.get("type") == "clear_thinking_20251015"
for e in edits
)
if not needs_thinking:
return False
if not self._supports_extended_thinking_on_bedrock(model):
return False
thinking = anthropic_messages_request.get("thinking")
if isinstance(thinking, dict):
t = thinking.get("type")
if t in ("enabled", "adaptive"):
return False
# ``disabled`` or unknown — replace with enabled so clear_thinking is valid
verbose_logger.debug(
"Bedrock clear_thinking_20251015: replacing thinking=%s with minimal enabled thinking",
thinking,
)
max_tokens = anthropic_messages_request.get("max_tokens")
budget = BEDROCK_MIN_THINKING_BUDGET_TOKENS
if isinstance(max_tokens, int) and max_tokens <= budget:
verbose_logger.warning(
"Bedrock clear_thinking_20251015: max_tokens=%s is not greater than "
"minimum thinking budget (%s); cannot inject thinking safely",
max_tokens,
budget,
)
return False
anthropic_messages_request["thinking"] = {
"type": "enabled",
"budget_tokens": budget,
}
verbose_logger.debug(
"Bedrock clear_thinking_20251015: injected thinking with budget_tokens=%s",
budget,
)
return True
def _is_claude_opus_4_5(self, model: str) -> bool:
"""
Check if the model is Claude Opus 4.5.
@ -251,6 +330,15 @@ class AmazonAnthropicClaudeMessagesConfig(
"opus_4.6",
"opus-4-6",
"opus_4_6",
# sonnet 4.6
"sonnet-4.6",
"sonnet_4.6",
"sonnet-4-6",
"sonnet_4_6",
# NOTE: Opus 4.7 on Bedrock does not support server-side tool search
# as of launch (2026-04-16). Bedrock rejects the tool type with:
# "tool type 'tool_search_tool_..._20251119' is not supported for this model".
# Re-add the opus-4.7 patterns here once AWS announces support.
]
return any(pattern in model_lower for pattern in supported_patterns)
@ -376,6 +464,13 @@ class AmazonAnthropicClaudeMessagesConfig(
if "model" in anthropic_messages_request:
anthropic_messages_request.pop("model", None)
injected_thinking_for_clear_thinking = (
self._ensure_thinking_for_clear_thinking_context_management(
anthropic_messages_request=anthropic_messages_request,
model=model,
)
)
# 4. Remove `ttl` field from cache_control in messages (Bedrock doesn't support it for older models)
self._remove_ttl_from_cache_control(
anthropic_messages_request=anthropic_messages_request, model=model
@ -412,6 +507,9 @@ class AmazonAnthropicClaudeMessagesConfig(
)
beta_set.update(auto_betas)
if injected_thinking_for_clear_thinking:
beta_set.add("interleaved-thinking-2025-05-14")
self._get_tool_search_beta_header_for_bedrock(
model=model,
tool_search_used=tool_search_used,

1120
litellm/model_prices_and_context_window_backup.json
View file

File diff suppressed because it is too large Load diff

1120
model_prices_and_context_window.json
View file

File diff suppressed because it is too large Load diff

470
tests/test_litellm/llms/bedrock/messages/invoke_transformations/test_anthropic_claude3_transformation.py
View file

@ -11,6 +11,12 @@ import pytest
sys.path.insert(0, os.path.abspath("../../../../../.."))
from litellm.litellm_core_utils.litellm_logging import Logging as LiteLLMLoggingObj
from litellm.llms.bedrock.common_utils import (
ensure_bedrock_anthropic_messages_tool_names,
normalize_tool_input_schema_types_for_bedrock_invoke,
remove_custom_field_from_tools,
)
from litellm.constants import BEDROCK_MIN_THINKING_BUDGET_TOKENS
from litellm.llms.bedrock.messages.invoke_transformations.anthropic_claude3_transformation import (
AmazonAnthropicClaudeMessagesConfig,
AmazonAnthropicClaudeMessagesStreamDecoder,
@ -178,3 +184,467 @@ def test_remove_ttl_from_cache_control():
request5 = {}
cfg._remove_ttl_from_cache_control(request5)
assert request5 == {}
def test_remove_custom_field_from_tools():
"""
Ensure the `custom` field is stripped from every tool definition.
Claude Code v2.1.69+ sends `custom: {defer_loading: true}` on tool
objects. Bedrock does not accept this extra field and returns
"Extra inputs are not permitted".
Ref: https://github.com/BerriAI/litellm/issues/22847
"""
# Case 1: tool with `custom` field should have it removed
request = {
"tools": [
{
"name": "Read",
"description": "Read a file",
"input_schema": {"type": "object", "properties": {}},
"custom": {"defer_loading": True},
},
{
"name": "Write",
"description": "Write a file",
"input_schema": {"type": "object", "properties": {}},
},
]
}
remove_custom_field_from_tools(request)
for tool in request["tools"]:
assert "custom" not in tool, f"Tool {tool['name']} still has 'custom' field"
# Other fields should be preserved
assert request["tools"][0]["name"] == "Read"
assert request["tools"][1]["name"] == "Write"
# Case 2: request without tools key (should not raise error)
request2 = {"messages": [{"role": "user", "content": "hi"}]}
remove_custom_field_from_tools(request2)
assert "tools" not in request2
# Case 3: empty tools list (should not raise error)
request3 = {"tools": []}
remove_custom_field_from_tools(request3)
assert request3["tools"] == []
# Case 4: tools with None value (should not raise error)
request4 = {"tools": None}
remove_custom_field_from_tools(request4)
assert request4["tools"] is None
def test_normalize_tool_input_schema_types_for_bedrock_invoke():
"""
Claude Code sends ``input_schema.type: \"custom\"`` for custom tools.
Bedrock Invoke rejects this; it requires JSON Schema ``type: \"object\"``.
"""
request = {
"tools": [
{
"name": "Agent",
"type": "custom",
"description": "subagent",
"input_schema": {
"type": "custom",
"additionalProperties": False,
"properties": {
"nested": {"type": "custom", "properties": {"x": {"type": "string"}}}
},
"required": ["nested"],
},
},
{
"name": "Read",
"input_schema": {"type": "object", "properties": {}},
},
]
}
normalize_tool_input_schema_types_for_bedrock_invoke(request)
agent_tool = request["tools"][0]
assert agent_tool["type"] == "custom"
assert agent_tool["input_schema"]["type"] == "object"
assert agent_tool["input_schema"]["properties"]["nested"]["type"] == "object"
assert request["tools"][1]["input_schema"]["type"] == "object"
request2 = {"messages": []}
normalize_tool_input_schema_types_for_bedrock_invoke(request2)
assert request2 == {"messages": []}
def test_ensure_bedrock_anthropic_messages_tool_names():
request = {
"tools": [
{"input_schema": {"type": "object", "properties": {}}},
{"name": "", "input_schema": {"type": "object", "properties": {}}},
{"name": " ", "input_schema": {"type": "object", "properties": {}}},
{"name": "KeepMe", "input_schema": {"type": "object", "properties": {}}},
]
}
ensure_bedrock_anthropic_messages_tool_names(request)
assert request["tools"][0]["name"] == "litellm_unnamed_tool_0"
assert request["tools"][1]["name"] == "litellm_unnamed_tool_1"
assert request["tools"][2]["name"] == "litellm_unnamed_tool_2"
assert request["tools"][3]["name"] == "KeepMe"
def test_bedrock_invoke_messages_transform_adds_name_when_tool_missing_name():
"""Bedrock requires tools.0.custom.name when the payload is schema-only."""
from litellm.types.router import GenericLiteLLMParams
cfg = AmazonAnthropicClaudeMessagesConfig()
optional_params = {
"max_tokens": 128,
"tools": [
{
"input_schema": {
"type": "object",
"properties": {"questions": {"type": "array"}},
"required": ["questions"],
},
}
],
"stream": False,
}
result = cfg.transform_anthropic_messages_request(
model="anthropic.claude-3-haiku-20240307-v1:0",
messages=[{"role": "user", "content": "hi"}],
anthropic_messages_optional_request_params=copy.deepcopy(optional_params),
litellm_params=GenericLiteLLMParams(),
headers={},
)
assert result["tools"][0]["name"] == "litellm_unnamed_tool_0"
def test_bedrock_invoke_messages_injects_thinking_for_clear_thinking_context_management():
"""
Bedrock requires extended thinking when ``clear_thinking_20251015`` appears in
``context_management`` (Claude Code sends CM without ``thinking``).
"""
from litellm.types.router import GenericLiteLLMParams
cfg = AmazonAnthropicClaudeMessagesConfig()
optional_params = {
"max_tokens": 32000,
"stream": False,
"context_management": {
"edits": [{"type": "clear_thinking_20251015", "keep": "all"}]
},
}
result = cfg.transform_anthropic_messages_request(
model="global.anthropic.claude-sonnet-4-6-v1:0",
messages=[{"role": "user", "content": "hi"}],
anthropic_messages_optional_request_params=copy.deepcopy(optional_params),
litellm_params=GenericLiteLLMParams(),
headers={},
)
assert result["thinking"]["type"] == "enabled"
assert result["thinking"]["budget_tokens"] == BEDROCK_MIN_THINKING_BUDGET_TOKENS
betas = result.get("anthropic_beta") or []
assert "interleaved-thinking-2025-05-14" in betas
def test_bedrock_invoke_messages_skips_thinking_injection_when_already_enabled():
from litellm.types.router import GenericLiteLLMParams
cfg = AmazonAnthropicClaudeMessagesConfig()
optional_params = {
"max_tokens": 32000,
"stream": False,
"thinking": {"type": "enabled", "budget_tokens": 2048},
"context_management": {
"edits": [{"type": "clear_thinking_20251015", "keep": "all"}]
},
}
result = cfg.transform_anthropic_messages_request(
model="global.anthropic.claude-sonnet-4-6-v1:0",
messages=[{"role": "user", "content": "hi"}],
anthropic_messages_optional_request_params=copy.deepcopy(optional_params),
litellm_params=GenericLiteLLMParams(),
headers={},
)
# Claude 4.6/4.7 reject ``thinking.type=enabled``; legacy ``enabled`` is
# translated to ``adaptive`` (budget_tokens => output_config.effort) and the
# pre-4.6 ``interleaved-thinking-2025-05-14`` beta must not be attached.
assert result["thinking"]["type"] == "adaptive"
betas = result.get("anthropic_beta") or []
assert "interleaved-thinking-2025-05-14" not in betas
def test_bedrock_invoke_messages_transform_converts_custom_tool_schema_type_to_object():
"""
End-to-end: AmazonAnthropicClaudeMessagesConfig must emit Bedrock Invoke bodies
where every ``input_schema`` uses JSON Schema types (``object``), not Anthropic
``type: \"custom\"`` (root and nested).
"""
from litellm.types.router import GenericLiteLLMParams
cfg = AmazonAnthropicClaudeMessagesConfig()
tools = [
{
"name": "Agent",
"type": "custom",
"description": "Subagent",
"input_schema": {
"type": "custom",
"additionalProperties": False,
"properties": {
"prompt": {"type": "string"},
"nested": {
"type": "custom",
"properties": {"x": {"type": "string"}},
"required": ["x"],
},
},
"required": ["prompt"],
},
}
]
optional_params = {
"max_tokens": 256,
"tools": copy.deepcopy(tools),
"stream": False,
}
messages = [{"role": "user", "content": "hi"}]
result = cfg.transform_anthropic_messages_request(
model="anthropic.claude-3-haiku-20240307-v1:0",
messages=messages,
anthropic_messages_optional_request_params=optional_params,
litellm_params=GenericLiteLLMParams(),
headers={},
)
assert "tools" in result
schema = result["tools"][0]["input_schema"]
assert schema["type"] == "object"
assert schema["properties"]["nested"]["type"] == "object"
# Tool discriminator stays Anthropic-side; only input_schema is normalized
assert result["tools"][0]["type"] == "custom"
def test_remove_scope_from_cache_control():
"""Ensure scope field is removed from cache_control for Bedrock (not supported)."""
cfg = AmazonAnthropicClaudeMessagesConfig()
# Test case 1: System with cache_control containing scope
request = {
"system": [
{
"type": "text",
"text": "You are an AI assistant.",
"cache_control": {
"type": "ephemeral",
"scope": "global",
},
}
],
"messages": [
{
"role": "user",
"content": [
{
"type": "text",
"text": "Hello",
"cache_control": {
"type": "ephemeral",
"scope": "global",
},
}
],
}
],
}
cfg._remove_ttl_from_cache_control(request)
# Verify scope is removed from system
assert "scope" not in request["system"][0]["cache_control"]
assert request["system"][0]["cache_control"]["type"] == "ephemeral"
# Verify scope is removed from messages
assert "scope" not in request["messages"][0]["content"][0]["cache_control"]
assert request["messages"][0]["content"][0]["cache_control"]["type"] == "ephemeral"
def test_bedrock_messages_strips_output_config():
"""
Ensure output_config is stripped from the request before sending to
Bedrock Invoke, which doesn't support this Anthropic-specific parameter.
Regression test for: https://github.com/BerriAI/litellm/issues/22797
"""
from litellm.types.router import GenericLiteLLMParams
cfg = AmazonAnthropicClaudeMessagesConfig()
messages = [{"role": "user", "content": [{"type": "text", "text": "Hello"}]}]
optional_params = {
"max_tokens": 4096,
"output_config": {
"effort": "high",
},
}
result = cfg.transform_anthropic_messages_request(
model="anthropic.claude-3-haiku-20240307-v1:0",
messages=messages,
anthropic_messages_optional_request_params=optional_params,
litellm_params=GenericLiteLLMParams(),
headers={},
)
assert (
"output_config" not in result
), "output_config should be stripped — Bedrock Invoke rejects it"
# Other params should be preserved
assert result.get("max_tokens") == 4096
def test_bedrock_messages_strips_output_config_with_output_format():
"""
When both output_config and output_format are present, both should be
stripped (output_format is converted to inline schema, output_config
is simply dropped).
"""
from litellm.types.router import GenericLiteLLMParams
cfg = AmazonAnthropicClaudeMessagesConfig()
messages = [{"role": "user", "content": [{"type": "text", "text": "Hello"}]}]
optional_params = {
"max_tokens": 4096,
"output_config": {"effort": "low"},
"output_format": {
"type": "json_schema",
"schema": {
"type": "object",
"properties": {"answer": {"type": "string"}},
},
},
}
result = cfg.transform_anthropic_messages_request(
model="anthropic.claude-3-haiku-20240307-v1:0",
messages=messages,
anthropic_messages_optional_request_params=optional_params,
litellm_params=GenericLiteLLMParams(),
headers={},
)
assert "output_config" not in result
assert "output_format" not in result
@pytest.mark.asyncio
async def test_promote_message_stop_usage_preserves_message_delta_output_tokens():
"""
Bedrock unified /messages streaming can send full usage on message_delta and a
conflicting smaller usage on message_stop (e.g. output_tokens 9 vs 12).
_promote_message_stop_usage must not replace message_delta output_tokens.
"""
cfg = AmazonAnthropicClaudeMessagesConfig()
async def _stream(): # type: ignore[return-type]
yield {
"type": "message_delta",
"delta": {"stop_reason": "end_turn", "stop_sequence": None},
"usage": {
"input_tokens": 3,
"cache_creation_input_tokens": 10553,
"cache_read_input_tokens": 25490,
"output_tokens": 12,
},
}
yield {
"type": "message_stop",
"usage": {"input_tokens": 3, "output_tokens": 9},
}
merged: list[dict] = []
async for chunk in cfg._promote_message_stop_usage(_stream()):
if isinstance(chunk, dict):
merged.append(chunk)
assert len(merged) >= 1
delta_out = merged[0]
assert delta_out["type"] == "message_delta"
assert delta_out["usage"]["output_tokens"] == 12
assert delta_out["usage"]["cache_creation_input_tokens"] == 10553
assert delta_out["usage"]["cache_read_input_tokens"] == 25490
assert delta_out["usage"]["input_tokens"] == 3
@pytest.mark.asyncio
async def test_unified_bedrock_messages_sse_usage_and_cost_claude_sonnet_46():
"""
End-to-end for Bedrock Invoke Anthropic Messages (unified) streaming path:
dict chunks -> _promote_message_stop_usage -> bedrock_sse_wrapper SSE bytes ->
same logging reconstruction as Anthropic /messages. Ensures token counts and
completion_cost match model_prices for us.anthropic.claude-sonnet-4-6.
"""
from litellm import completion_cost
from litellm.proxy.pass_through_endpoints.llm_provider_handlers.anthropic_passthrough_logging_handler import (
AnthropicPassthroughLoggingHandler,
)
cfg = AmazonAnthropicClaudeMessagesConfig()
async def _stream(): # type: ignore[return-type]
yield {
"type": "message_delta",
"delta": {"stop_reason": "end_turn", "stop_sequence": None},
"usage": {
"input_tokens": 3,
"cache_creation_input_tokens": 10553,
"cache_read_input_tokens": 25490,
"output_tokens": 12,
},
}
yield {
"type": "message_stop",
"usage": {"input_tokens": 3, "output_tokens": 9},
}
logging_obj = LiteLLMLoggingObj(
model="bedrock/us.anthropic.claude-sonnet-4-6",
messages=[{"role": "user", "content": "Hello"}],
stream=True,
call_type="chat",
start_time=datetime.now(),
litellm_call_id="test_unified_bedrock_messages_sse_cost",
function_id="test_unified_bedrock_messages_sse_cost",
)
collected: list[bytes] = []
async for sse in cfg.bedrock_sse_wrapper(
completion_stream=_stream(),
litellm_logging_obj=logging_obj,
request_body={"model": "us.anthropic.claude-sonnet-4-6"},
):
collected.append(sse)
built = AnthropicPassthroughLoggingHandler._build_complete_streaming_response(
all_chunks=collected,
model="us.anthropic.claude-sonnet-4-6",
litellm_logging_obj=Mock(),
)
assert built.usage is not None
assert built.usage.completion_tokens == 12
assert built.usage.prompt_tokens == 36046
assert built.usage.total_tokens == 36058
assert built.usage.cache_creation_input_tokens == 10553
assert built.usage.cache_read_input_tokens == 25490
cost = completion_cost(
completion_response=built,
model="bedrock/us.anthropic.claude-sonnet-4-6",
custom_llm_provider="bedrock",
)
assert cost == pytest.approx(0.052150725, rel=0, abs=1e-9)

4
tests/test_litellm/test_utils.py
View file

@ -660,6 +660,10 @@ def test_aaamodel_prices_and_context_window_json_is_valid():
"supports_web_search": {"type": "boolean"},
"supports_url_context": {"type": "boolean"},
"supports_reasoning": {"type": "boolean"},
"supports_minimal_reasoning_effort": {"type": "boolean"},
"supports_none_reasoning_effort": {"type": "boolean"},
"supports_xhigh_reasoning_effort": {"type": "boolean"},
"supports_max_reasoning_effort": {"type": "boolean"},
"supports_service_tier": {"type": "boolean"},
"supports_preset": {"type": "boolean"},
"tool_use_system_prompt_tokens": {"type": "number"},