Merge pull request #20551 from BerriAI/litellm_opus_4.6_thinking

Cherry-picked: 6fccaa6584
This commit is contained in:
Sameer Kankute 2026-02-06 19:30:47 +05:30 • committed by Ishaan Jaffer
parent 2962afbff6
commit cfe7c0fd3a
28 changed files with 5627 additions and 308 deletions

View file

@ -0,0 +1,378 @@
---
slug: claude_opus_4_6
title: "Day 0 Support: Claude Opus 4.6"
date: 2026-02-05T10:00:00
authors:
- name: Sameer Kankute
title: SWE @ LiteLLM (LLM Translation)
url: https://www.linkedin.com/in/sameer-kankute/
image_url: https://pbs.twimg.com/profile_images/2001352686994907136/ONgNuSk5_400x400.jpg
- name: Ishaan Jaff
title: "CTO, LiteLLM"
url: https://www.linkedin.com/in/reffajnaahsi/
image_url: https://pbs.twimg.com/profile_images/1613813310264340481/lz54oEiB_400x400.jpg
- name: Krrish Dholakia
title: "CEO, LiteLLM"
url: https://www.linkedin.com/in/krish-d/
image_url: https://pbs.twimg.com/profile_images/1298587542745358340/DZv3Oj-h_400x400.jpg
description: "Day 0 support for Claude Opus 4.6 on LiteLLM AI Gateway - use across Anthropic, Azure, Vertex AI, and Bedrock."
tags: [anthropic, claude, opus 4.6]
hide_table_of_contents: false
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
LiteLLM now supports Claude Opus 4.6 on Day 0. Use it across Anthropic, Azure, Vertex AI, and Bedrock through the LiteLLM AI Gateway.
## Docker Image
```bash
docker pull ghcr.io/berriai/litellm:litellm_stable_release_branch-v1.80.0-stable.opus-4-6
```
## Usage - Anthropic
<Tabs>
<TabItem value="proxy" label="LiteLLM Proxy">
**1. Setup config.yaml**
```yaml
model_list:
- model_name: claude-opus-4-6
litellm_params:
model: anthropic/claude-opus-4-6
api_key: os.environ/ANTHROPIC_API_KEY
```
**2. Start the proxy**
```bash
docker run -d \
-p 4000:4000 \
-e ANTHROPIC_API_KEY=$ANTHROPIC_API_KEY \
-v $(pwd)/config.yaml:/app/config.yaml \
ghcr.io/berriai/litellm:litellm_stable_release_branch-v1.80.0-stable.opus-4-6 \
--config /app/config.yaml
```
**3. Test it!**
```bash
curl --location 'http://0.0.0.0:4000/chat/completions' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer $LITELLM_KEY' \
--data '{
"model": "claude-opus-4-6",
"messages": [
{
"role": "user",
"content": "what llm are you"
}
]
}'
```
</TabItem>
</Tabs>
## Usage - Azure
<Tabs>
<TabItem value="proxy" label="LiteLLM Proxy">
**1. Setup config.yaml**
```yaml
model_list:
- model_name: claude-opus-4-6
litellm_params:
model: azure_ai/claude-opus-4-6
api_key: os.environ/AZURE_AI_API_KEY
api_base: os.environ/AZURE_AI_API_BASE # https://<resource>.services.ai.azure.com
```
**2. Start the proxy**
```bash
docker run -d \
-p 4000:4000 \
-e AZURE_AI_API_KEY=$AZURE_AI_API_KEY \
-e AZURE_AI_API_BASE=$AZURE_AI_API_BASE \
-v $(pwd)/config.yaml:/app/config.yaml \
ghcr.io/berriai/litellm:litellm_stable_release_branch-v1.80.0-stable.opus-4-6 \
--config /app/config.yaml
```
**3. Test it!**
```bash
curl --location 'http://0.0.0.0:4000/chat/completions' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer $LITELLM_KEY' \
--data '{
"model": "claude-opus-4-6",
"messages": [
{
"role": "user",
"content": "what llm are you"
}
]
}'
```
</TabItem>
</Tabs>
## Usage - Vertex AI
<Tabs>
<TabItem value="proxy" label="LiteLLM Proxy">
**1. Setup config.yaml**
```yaml
model_list:
- model_name: claude-opus-4-6
litellm_params:
model: vertex_ai/claude-opus-4-6
vertex_project: os.environ/VERTEX_PROJECT
vertex_location: us-east5
```
**2. Start the proxy**
```bash
docker run -d \
-p 4000:4000 \
-e VERTEX_PROJECT=$VERTEX_PROJECT \
-e GOOGLE_APPLICATION_CREDENTIALS=/app/credentials.json \
-v $(pwd)/config.yaml:/app/config.yaml \
-v $(pwd)/credentials.json:/app/credentials.json \
ghcr.io/berriai/litellm:litellm_stable_release_branch-v1.80.0-stable.opus-4-6 \
--config /app/config.yaml
```
**3. Test it!**
```bash
curl --location 'http://0.0.0.0:4000/chat/completions' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer $LITELLM_KEY' \
--data '{
"model": "claude-opus-4-6",
"messages": [
{
"role": "user",
"content": "what llm are you"
}
]
}'
```
</TabItem>
</Tabs>
## Usage - Bedrock
<Tabs>
<TabItem value="proxy" label="LiteLLM Proxy">
**1. Setup config.yaml**
```yaml
model_list:
- model_name: claude-opus-4-6
litellm_params:
model: bedrock/anthropic.claude-opus-4-6-v1:0
aws_access_key_id: os.environ/AWS_ACCESS_KEY_ID
aws_secret_access_key: os.environ/AWS_SECRET_ACCESS_KEY
aws_region_name: us-east-1
```
**2. Start the proxy**
```bash
docker run -d \
-p 4000:4000 \
-e AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID \
-e AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY \
-v $(pwd)/config.yaml:/app/config.yaml \
ghcr.io/berriai/litellm:litellm_stable_release_branch-v1.80.0-stable.opus-4-6 \
--config /app/config.yaml
```
**3. Test it!**
```bash
curl --location 'http://0.0.0.0:4000/chat/completions' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer $LITELLM_KEY' \
--data '{
"model": "claude-opus-4-6",
"messages": [
{
"role": "user",
"content": "what llm are you"
}
]
}'
```
</TabItem>
</Tabs>
## Compaction
Litellm supports enabling compaction for the new claude-opus-4-6.
### Enabling Compaction
To enable compaction, add the `context_management` parameter with the `compact_20260112` edit type:
```bash
curl --location 'http://0.0.0.0:4000/chat/completions' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer $LITELLM_KEY' \
--data '{
"model": "claude-opus-4-6",
"messages": [
{
"role": "user",
"content": "What is the weather in San Francisco?"
}
],
"context_management": {
"edits": [
{
"type": "compact_20260112"
}
]
},
"max_tokens": 100
}'
```
All the parameters supported for context_management by anthropic are supported and can be directly added. Litellm automatically adds the `compact-2026-01-12` beta header in the request.
### Response with Compaction Block
The response will include the compaction summary in `provider_specific_fields.compaction_blocks`:
```json
{
"id": "chatcmpl-a6c105a3-4b25-419e-9551-c800633b6cb2",
"created": 1770357619,
"model": "claude-opus-4-6",
"object": "chat.completion",
"choices": [
{
"finish_reason": "length",
"index": 0,
"message": {
"content": "I don't have access to real-time data, so I can't provide the current weather in San Francisco. To get up-to-date weather information, I'd recommend checking:\n\n- **Weather websites** like weather.com, accuweather.com, or wunderground.com\n- **Search engines** – just Google \"San Francisco weather\"\n- **Weather apps** on your phone (e.g., Apple Weather, Google Weather)\n- **National",
"role": "assistant",
"provider_specific_fields": {
"compaction_blocks": [
{
"type": "compaction",
"content": "Summary of the conversation: The user requested help building a web scraper..."
}
]
}
}
}
],
"usage": {
"completion_tokens": 100,
"prompt_tokens": 86,
"total_tokens": 186
}
}
```
### Using Compaction Blocks in Follow-up Requests
To continue the conversation with compaction, include the compaction block in the assistant message's `provider_specific_fields`:
```bash
curl --location 'http://0.0.0.0:4000/chat/completions' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer $LITELLM_KEY' \
--data '{
"model": "claude-opus-4-6",
"messages": [
{
"role": "user",
"content": "How can I build a web scraper?"
},
{
"role": "assistant",
"content": [
{
"type": "text",
"text": "Certainly! To build a basic web scraper, you'll typically use a programming language like Python along with libraries such as `requests` (for fetching web pages) and `BeautifulSoup` (for parsing HTML). Here's a basic example:\n\n```python\nimport requests\nfrom bs4 import BeautifulSoup\n\nurl = 'https://example.com'\nresponse = requests.get(url)\nsoup = BeautifulSoup(response.text, 'html.parser')\n\n# Extract and print all text\ntext = soup.get_text()\nprint(text)\n```\n\nLet me know what you're interested in scraping or if you need help with a specific website!"
}
],
"provider_specific_fields": {
"compaction_blocks": [
{
"type": "compaction",
"content": "Summary of the conversation: The user asked how to build a web scraper, and the assistant gave an overview using Python with requests and BeautifulSoup."
}
]
}
},
{
"role": "user",
"content": "How do I use it to scrape product prices?"
}
],
"context_management": {
"edits": [
{
"type": "compact_20260112"
}
]
},
"max_tokens": 100
}'
```
### Streaming Support
Compaction blocks are also supported in streaming mode. You'll receive:
- `compaction_start` event when a compaction block begins
- `compaction_delta` events with the compaction content
- The accumulated `compaction_blocks` in `provider_specific_fields`
## Effort Levels
Four effort levels available: `low`, `medium`, `high` (default), and `max`. Pass directly via the `effort` parameter:
```bash
curl --location 'http://0.0.0.0:4000/chat/completions' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer $LITELLM_KEY' \
--data '{
"model": "claude-opus-4-6",
"messages": [
{
"role": "user",
"content": "Explain quantum computing"
}
],
"effort": "max"
}'
```
## 1M Token Context (Beta)
Opus 4.6 supports 1M token context. Premium pricing applies for prompts exceeding 200k tokens ($10/$37.50 per million input/output tokens). LiteLLM supports cost calculations for 1M token contexts.
## US-Only Inference
Available at 1.1× token pricing. LiteLLM supports this pricing model.

View file

@ -0,0 +1,129 @@
import Image from '@theme/IdealImage';
# Claude Code - Fixing Invalid Beta Header Errors
When using Claude Code with LiteLLM and non-Anthropic providers (Bedrock, Azure AI, Vertex AI), you may encounter "invalid beta header" errors. This guide explains how to fix these errors locally or contribute a fix to LiteLLM.
## What Are Beta Headers?
Anthropic uses beta headers to enable experimental features in Claude. When you use Claude Code, it may send beta headers like:
```
anthropic-beta: prompt-caching-scope-2026-01-05,advanced-tool-use-2025-11-20
```
However, not all providers support all Anthropic beta features. When an unsupported beta header is sent to a provider, you'll see an error.
## Common Error Message
```bash
Error: The model returned the following errors: invalid beta flag
```
## How LiteLLM Handles Beta Headers
LiteLLM automatically filters out unsupported beta headers using a configuration file:
```
litellm/litellm/anthropic_beta_headers_config.json
```
This JSON file lists which beta headers are **unsupported** for each provider. Headers not in the unsupported list are passed through to the provider.
## Quick Fix: Update Config Locally
If you encounter an invalid beta header error, you can fix it immediately by updating the config file locally.
### Step 1: Locate the Config File
Find the file in your LiteLLM installation:
```bash
# If installed via pip
cd $(python -c "import litellm; import os; print(os.path.dirname(litellm.__file__))")
# The config file is at:
# litellm/anthropic_beta_headers_config.json
```
### Step 2: Add the Unsupported Header
Open `anthropic_beta_headers_config.json` and add the problematic header to the appropriate provider's list:
```json title="anthropic_beta_headers_config.json"
{
"description": "Unsupported Anthropic beta headers for each provider. Headers listed here will be dropped. Headers not listed are passed through as-is.",
"anthropic": [],
"azure_ai": [],
"bedrock_converse": [
"prompt-caching-scope-2026-01-05",
"bash_20250124",
"bash_20241022",
"text_editor_20250124",
"text_editor_20241022",
"compact-2026-01-12",
"advanced-tool-use-2025-11-20",
"web-fetch-2025-09-10",
"code-execution-2025-08-25",
"skills-2025-10-02",
"files-api-2025-04-14"
],
"bedrock": [
"advanced-tool-use-2025-11-20",
"prompt-caching-scope-2026-01-05",
"structured-outputs-2025-11-13",
"web-fetch-2025-09-10",
"code-execution-2025-08-25",
"skills-2025-10-02",
"files-api-2025-04-14"
],
"vertex_ai": [
"prompt-caching-scope-2026-01-05"
]
}
```
### Step 3: Restart Your Application
After updating the config file, restart your LiteLLM proxy or application:
```bash
# If using LiteLLM proxy
litellm --config config.yaml
# If using Python SDK
# Just restart your Python application
```
The updated configuration will be loaded automatically.
## Contributing a Fix to LiteLLM
Help the community by contributing your fix! If your local changes work, please raise a PR with the addition of the header and we will merge it.
## How Beta Header Filtering Works
When you make a request through LiteLLM:
```mermaid
sequenceDiagram
participant CC as Claude Code
participant LP as LiteLLM
participant Config as Beta Headers Config
participant Provider as Provider (Bedrock/Azure/etc)
CC->>LP: Request with beta headers
Note over CC,LP: anthropic-beta: header1,header2,header3
LP->>Config: Load unsupported headers for provider
Config-->>LP: Returns unsupported list
Note over LP: Filter headers:<br/>- Remove unsupported<br/>- Keep supported
LP->>Provider: Request with filtered headers
Note over LP,Provider: anthropic-beta: header2<br/>(header1, header3 removed)
Provider-->>LP: Success response
LP-->>CC: Response
```

View file

@ -109,14 +109,28 @@ const sidebars = {
type: "category",
label: "AI Tools (OpenWebUI, Claude Code, etc.)",
items: [
"tutorials/claude_responses_api",
"tutorials/openweb_ui",
{
type: "category",
label: "Claude Code",
items: [
"tutorials/claude_responses_api",
"tutorials/claude_code_max_subscription",
"tutorials/claude_code_customer_tracking",
"tutorials/claude_code_websearch",
"tutorials/claude_mcp",
"tutorials/claude_non_anthropic_models",
"tutorials/claude_code_plugin_marketplace",
"tutorials/claude_code_beta_headers",
]
},
"tutorials/opencode_integration",
"tutorials/cost_tracking_coding",
"tutorials/cursor_integration",
"tutorials/github_copilot_integration",
"tutorials/litellm_gemini_cli",
"tutorials/litellm_qwen_code_cli",
"tutorials/openai_codex",
"tutorials/openweb_ui"
]
},

View file

@ -0,0 +1,30 @@
{
"description": "Unsupported Anthropic beta headers for each provider. Headers listed here will be dropped. Headers not listed are passed through as-is.",
"anthropic": [],
"azure_ai": [],
"bedrock_converse": [
"prompt-caching-scope-2026-01-05",
"bash_20250124",
"bash_20241022",
"text_editor_20250124",
"text_editor_20241022",
"compact-2026-01-12",
"advanced-tool-use-2025-11-20",
"web-fetch-2025-09-10",
"code-execution-2025-08-25",
"skills-2025-10-02",
"files-api-2025-04-14"
],
"bedrock": [
"advanced-tool-use-2025-11-20",
"prompt-caching-scope-2026-01-05",
"structured-outputs-2025-11-13",
"web-fetch-2025-09-10",
"code-execution-2025-08-25",
"skills-2025-10-02",
"files-api-2025-04-14"
],
"vertex_ai": [
"prompt-caching-scope-2026-01-05"
]
}

View file

@ -0,0 +1,221 @@
"""
Centralized manager for Anthropic beta headers across different providers.
This module provides utilities to:
1. Load beta header configuration from JSON (lists unsupported headers per provider)
2. Filter out unsupported beta headers
3. Handle provider-specific header name mappings (e.g., advanced-tool-use -> tool-search-tool)
Design:
- JSON config lists UNSUPPORTED headers for each provider
- Headers not in the unsupported list are passed through
- Header mappings allow renaming headers for specific providers
"""
import json
import os
from typing import Dict, List, Optional, Set
from litellm.litellm_core_utils.litellm_logging import verbose_logger
# Cache for the loaded configuration
_BETA_HEADERS_CONFIG: Optional[Dict] = None
def _load_beta_headers_config() -> Dict:
"""
Load the beta headers configuration from JSON file.
Uses caching to avoid repeated file reads.
Returns:
Dict containing the beta headers configuration
"""
global _BETA_HEADERS_CONFIG
if _BETA_HEADERS_CONFIG is not None:
return _BETA_HEADERS_CONFIG
config_path = os.path.join(
os.path.dirname(__file__),
"anthropic_beta_headers_config.json"
)
try:
with open(config_path, "r") as f:
_BETA_HEADERS_CONFIG = json.load(f)
verbose_logger.debug(f"Loaded beta headers config from {config_path}")
return _BETA_HEADERS_CONFIG
except Exception as e:
verbose_logger.error(f"Failed to load beta headers config: {e}")
# Return empty config as fallback
return {
"anthropic": [],
"azure_ai": [],
"bedrock": [],
"bedrock_converse": [],
"vertex_ai": []
}
def get_provider_name(provider: str) -> str:
"""
Resolve provider aliases to canonical provider names.
Args:
provider: Provider name (may be an alias)
Returns:
Canonical provider name
"""
config = _load_beta_headers_config()
aliases = config.get("provider_aliases", {})
return aliases.get(provider, provider)
def filter_and_transform_beta_headers(
beta_headers: List[str],
provider: str,
) -> List[str]:
"""
Filter beta headers based on provider's unsupported list.
This function:
1. Removes headers that are in the provider's unsupported list
2. Passes through all other headers as-is
Note: Header transformations/mappings (e.g., advanced-tool-use -> tool-search-tool)
are handled in each provider's transformation code, not here.
Args:
beta_headers: List of Anthropic beta header values
provider: Provider name (e.g., "anthropic", "bedrock", "vertex_ai")
Returns:
List of filtered beta headers for the provider
"""
if not beta_headers:
return []
config = _load_beta_headers_config()
provider = get_provider_name(provider)
# Get unsupported headers for this provider
unsupported_headers = set(config.get(provider, []))
filtered_headers: Set[str] = set()
for header in beta_headers:
header = header.strip()
# Skip if header is unsupported
if header in unsupported_headers:
verbose_logger.debug(
f"Dropping unsupported beta header '{header}' for provider '{provider}'"
)
continue
# Pass through as-is
filtered_headers.add(header)
return sorted(list(filtered_headers))
def is_beta_header_supported(
beta_header: str,
provider: str,
) -> bool:
"""
Check if a specific beta header is supported by a provider.
Args:
beta_header: The Anthropic beta header value
provider: Provider name
Returns:
True if the header is supported (not in unsupported list), False otherwise
"""
config = _load_beta_headers_config()
provider = get_provider_name(provider)
unsupported_headers = set(config.get(provider, []))
return beta_header not in unsupported_headers
def get_provider_beta_header(
anthropic_beta_header: str,
provider: str,
) -> Optional[str]:
"""
Check if a beta header is supported by a provider.
Note: This does NOT handle header transformations/mappings.
Those are handled in each provider's transformation code.
Args:
anthropic_beta_header: The Anthropic beta header value
provider: Provider name
Returns:
The original header if supported, or None if unsupported
"""
config = _load_beta_headers_config()
provider = get_provider_name(provider)
# Check if unsupported
unsupported_headers = set(config.get(provider, []))
if anthropic_beta_header in unsupported_headers:
return None
return anthropic_beta_header
def update_headers_with_filtered_beta(
headers: dict,
provider: str,
) -> dict:
"""
Update headers dict by filtering and transforming anthropic-beta header values.
Modifies the headers dict in place and returns it.
Args:
headers: Request headers dict (will be modified in place)
provider: Provider name
Returns:
Updated headers dict
"""
existing_beta = headers.get("anthropic-beta")
if not existing_beta:
return headers
# Parse existing beta headers
beta_values = [b.strip() for b in existing_beta.split(",") if b.strip()]
# Filter and transform based on provider
filtered_beta_values = filter_and_transform_beta_headers(
beta_headers=beta_values,
provider=provider,
)
# Update or remove the header
if filtered_beta_values:
headers["anthropic-beta"] = ",".join(filtered_beta_values)
else:
# Remove the header if no values remain
headers.pop("anthropic-beta", None)
return headers
def get_unsupported_headers(provider: str) -> List[str]:
"""
Get all beta headers that are unsupported by a provider.
Args:
provider: Provider name
Returns:
List of unsupported Anthropic beta header names
"""
config = _load_beta_headers_config()
provider = get_provider_name(provider)
return config.get(provider, [])

View file

@ -92,8 +92,8 @@ def map_finish_reason(
return "length"
elif finish_reason == "tool_use": # anthropic
return "tool_calls"
elif finish_reason == "content_filtered":
return "content_filter"
elif finish_reason == "compaction":
return "length"
return finish_reason

View file

@ -2143,6 +2143,16 @@ def anthropic_messages_pt( # noqa: PLR0915
while msg_i < len(messages) and messages[msg_i]["role"] == "assistant":
assistant_content_block: ChatCompletionAssistantMessage = messages[msg_i] # type: ignore
# Extract compaction_blocks from provider_specific_fields and add them first
_provider_specific_fields_raw = assistant_content_block.get(
"provider_specific_fields"
)
if isinstance(_provider_specific_fields_raw, dict):
_compaction_blocks = _provider_specific_fields_raw.get("compaction_blocks")
if _compaction_blocks and isinstance(_compaction_blocks, list):
# Add compaction blocks at the beginning of assistant content : https://platform.claude.com/docs/en/build-with-claude/compaction
assistant_content.extend(_compaction_blocks) # type: ignore
thinking_blocks = assistant_content_block.get("thinking_blocks", None)
if (
thinking_blocks is not None

View file

@ -511,6 +511,9 @@ class ModelResponseIterator:
# Accumulate web_search_tool_result blocks for multi-turn reconstruction
# See: https://github.com/BerriAI/litellm/issues/17737
self.web_search_results: List[Dict[str, Any]] = []
# Accumulate compaction blocks for multi-turn reconstruction
self.compaction_blocks: List[Dict[str, Any]] = []
def check_empty_tool_call_args(self) -> bool:
"""
@ -591,6 +594,12 @@ class ModelResponseIterator:
)
]
provider_specific_fields["thinking_blocks"] = thinking_blocks
elif "content" in content_block["delta"] and content_block["delta"].get("type") == "compaction_delta":
# Handle compaction delta
provider_specific_fields["compaction_delta"] = {
"type": "compaction_delta",
"content": content_block["delta"]["content"]
}
return text, tool_use, thinking_blocks, provider_specific_fields
@ -719,19 +728,54 @@ class ModelResponseIterator:
content_block_start=content_block_start,
provider_specific_fields=provider_specific_fields,
)
elif (
content_block_start["content_block"]["type"]
== "web_search_tool_result"
):
# Capture web_search_tool_result for multi-turn reconstruction
# The full content comes in content_block_start, not in deltas
# See: https://github.com/BerriAI/litellm/issues/17737
self.web_search_results.append(
elif content_block_start["content_block"]["type"] == "compaction":
# Handle compaction blocks
# The full content comes in content_block_start
self.compaction_blocks.append(
content_block_start["content_block"]
)
provider_specific_fields["web_search_results"] = (
self.web_search_results
provider_specific_fields["compaction_blocks"] = (
self.compaction_blocks
)
provider_specific_fields["compaction_start"] = {
"type": "compaction",
"content": content_block_start["content_block"].get("content", "")
}
elif content_block_start["content_block"]["type"].endswith("_tool_result"):
# Handle all tool result types (web_search, bash_code_execution, text_editor, etc.)
content_type = content_block_start["content_block"]["type"]
# Special handling for web_search_tool_result for backwards compatibility
if content_type == "web_search_tool_result":
# Capture web_search_tool_result for multi-turn reconstruction
# The full content comes in content_block_start, not in deltas
# See: https://github.com/BerriAI/litellm/issues/17737
self.web_search_results.append(
content_block_start["content_block"]
)
provider_specific_fields["web_search_results"] = (
self.web_search_results
)
elif content_type == "web_fetch_tool_result":
# Capture web_fetch_tool_result for multi-turn reconstruction
# The full content comes in content_block_start, not in deltas
# Fixes: https://github.com/BerriAI/litellm/issues/18137
self.web_search_results.append(
content_block_start["content_block"]
)
provider_specific_fields["web_search_results"] = (
self.web_search_results
)
elif content_type != "tool_search_tool_result":
# Handle other tool results (code execution, etc.)
# Skip tool_search_tool_result as it's internal metadata
if not hasattr(self, "tool_results"):
self.tool_results = []
self.tool_results.append(content_block_start["content_block"])
provider_specific_fields["tool_results"] = self.tool_results
elif type_chunk == "content_block_stop":
ContentBlockStop(**chunk) # type: ignore
# check if tool call content block - only for tool_use and server_tool_use blocks

View file

@ -169,9 +169,10 @@ class AnthropicConfig(AnthropicModelInfo, BaseConfig):
tool_call["caller"] = cast(Dict[str, Any], anthropic_tool_content["caller"]) # type: ignore[typeddict-item]
return tool_call
def _is_claude_opus_4_5(self, model: str) -> bool:
@staticmethod
def _is_claude_opus_4_6(model: str) -> bool:
"""Check if the model is Claude Opus 4.5."""
return "opus-4-5" in model.lower() or "opus_4_5" in model.lower()
return "opus-4-6" in model.lower() or "opus_4_6" in model.lower()
def get_supported_openai_params(self, model: str):
params = [
@ -587,32 +588,38 @@ class AnthropicConfig(AnthropicModelInfo, BaseConfig):
@staticmethod
def _map_reasoning_effort(
reasoning_effort: Optional[Union[REASONING_EFFORT, str]],
reasoning_effort: Optional[Union[REASONING_EFFORT, str]],
model: str,
) -> Optional[AnthropicThinkingParam]:
if reasoning_effort is None:
return None
elif reasoning_effort == "low":
if AnthropicConfig._is_claude_opus_4_6(model):
return AnthropicThinkingParam(
type="enabled",
budget_tokens=DEFAULT_REASONING_EFFORT_LOW_THINKING_BUDGET,
)
elif reasoning_effort == "medium":
return AnthropicThinkingParam(
type="enabled",
budget_tokens=DEFAULT_REASONING_EFFORT_MEDIUM_THINKING_BUDGET,
)
elif reasoning_effort == "high":
return AnthropicThinkingParam(
type="enabled",
budget_tokens=DEFAULT_REASONING_EFFORT_HIGH_THINKING_BUDGET,
)
elif reasoning_effort == "minimal":
return AnthropicThinkingParam(
type="enabled",
budget_tokens=DEFAULT_REASONING_EFFORT_MINIMAL_THINKING_BUDGET,
type="adaptive",
)
else:
raise ValueError(f"Unmapped reasoning effort: {reasoning_effort}")
if reasoning_effort is None:
return None
elif reasoning_effort == "low":
return AnthropicThinkingParam(
type="enabled",
budget_tokens=DEFAULT_REASONING_EFFORT_LOW_THINKING_BUDGET,
)
elif reasoning_effort == "medium":
return AnthropicThinkingParam(
type="enabled",
budget_tokens=DEFAULT_REASONING_EFFORT_MEDIUM_THINKING_BUDGET,
)
elif reasoning_effort == "high":
return AnthropicThinkingParam(
type="enabled",
budget_tokens=DEFAULT_REASONING_EFFORT_HIGH_THINKING_BUDGET,
)
elif reasoning_effort == "minimal":
return AnthropicThinkingParam(
type="enabled",
budget_tokens=DEFAULT_REASONING_EFFORT_MINIMAL_THINKING_BUDGET,
)
else:
raise ValueError(f"Unmapped reasoning effort: {reasoning_effort}")
def _extract_json_schema_from_response_format(
self, value: Optional[dict]
@ -784,13 +791,8 @@ class AnthropicConfig(AnthropicModelInfo, BaseConfig):
if param == "thinking":
optional_params["thinking"] = value
elif param == "reasoning_effort" and isinstance(value, str):
# For Claude Opus 4.5, map reasoning_effort to output_config
if self._is_claude_opus_4_5(model):
optional_params["output_config"] = {"effort": value}
# For other models, map to thinking parameter
optional_params["thinking"] = AnthropicConfig._map_reasoning_effort(
value
reasoning_effort=value, model=model
)
elif param == "web_search_options" and isinstance(value, dict):
hosted_web_search_tool = self.map_web_search_tool(
@ -801,6 +803,9 @@ class AnthropicConfig(AnthropicModelInfo, BaseConfig):
)
elif param == "extra_headers":
optional_params["extra_headers"] = value
elif param == "context_management" and isinstance(value, dict):
# Pass through Anthropic-specific context_management parameter
optional_params["context_management"] = value
## handle thinking tokens
self.update_optional_params_with_thinking_tokens(
@ -933,8 +938,8 @@ class AnthropicConfig(AnthropicModelInfo, BaseConfig):
)
return tools
def _ensure_context_management_beta_header(self, headers: dict) -> None:
beta_value = ANTHROPIC_BETA_HEADER_VALUES.CONTEXT_MANAGEMENT_2025_06_27.value
def _ensure_beta_header(self, headers: dict, beta_value: str) -> None:
"""Add a beta header value to the anthropic-beta header, avoiding duplicates."""
existing_beta = headers.get("anthropic-beta")
if existing_beta is None:
headers["anthropic-beta"] = beta_value
@ -943,6 +948,38 @@ class AnthropicConfig(AnthropicModelInfo, BaseConfig):
if beta_value not in existing_values:
headers["anthropic-beta"] = f"{existing_beta}, {beta_value}"
def _ensure_context_management_beta_header(
self, headers: dict, context_management: dict
) -> None:
"""
Add appropriate beta headers based on context_management edits.
- If any edit has type "compact_20260112", add compact-2026-01-12 header
- For all other edits, add context-management-2025-06-27 header
"""
edits = context_management.get("edits", [])
has_compact = False
has_other = False
for edit in edits:
edit_type = edit.get("type", "")
if edit_type == "compact_20260112":
has_compact = True
else:
has_other = True
# Add compact header if any compact edits exist
if has_compact:
self._ensure_beta_header(
headers, ANTHROPIC_BETA_HEADER_VALUES.COMPACT_2026_01_12.value
)
# Add context management header if any other edits exist
if has_other:
self._ensure_beta_header(
headers, ANTHROPIC_BETA_HEADER_VALUES.CONTEXT_MANAGEMENT_2025_06_27.value
)
def update_headers_with_optional_anthropic_beta(
self, headers: dict, optional_params: dict
) -> dict:
@ -969,7 +1006,9 @@ class AnthropicConfig(AnthropicModelInfo, BaseConfig):
ANTHROPIC_BETA_HEADER_VALUES.CONTEXT_MANAGEMENT_2025_06_27.value
)
if optional_params.get("context_management") is not None:
self._ensure_context_management_beta_header(headers)
self._ensure_context_management_beta_header(
headers, optional_params["context_management"]
)
if optional_params.get("output_format") is not None:
headers["anthropic-beta"] = (
ANTHROPIC_BETA_HEADER_VALUES.STRUCTURED_OUTPUT_2025_09_25.value
@ -1131,6 +1170,8 @@ class AnthropicConfig(AnthropicModelInfo, BaseConfig):
Optional[str],
List[ChatCompletionToolCallChunk],
Optional[List[Any]],
Optional[List[Any]],
Optional[List[Any]],
]:
text_content = ""
citations: Optional[List[Any]] = None
@ -1142,6 +1183,8 @@ class AnthropicConfig(AnthropicModelInfo, BaseConfig):
reasoning_content: Optional[str] = None
tool_calls: List[ChatCompletionToolCallChunk] = []
web_search_results: Optional[List[Any]] = None
tool_results: Optional[List[Any]] = None
compaction_blocks: Optional[List[Any]] = None
for idx, content in enumerate(completion_response["content"]):
if content["type"] == "text":
text_content += content["text"]
@ -1172,6 +1215,12 @@ class AnthropicConfig(AnthropicModelInfo, BaseConfig):
thinking_blocks.append(
cast(ChatCompletionRedactedThinkingBlock, content)
)
## COMPACTION
elif content["type"] == "compaction":
if compaction_blocks is None:
compaction_blocks = []
compaction_blocks.append(content)
## CITATIONS
if content.get("citations") is not None:
@ -1193,7 +1242,7 @@ class AnthropicConfig(AnthropicModelInfo, BaseConfig):
if thinking_content is not None:
reasoning_content += thinking_content
return text_content, citations, thinking_blocks, reasoning_content, tool_calls, web_search_results
return text_content, citations, thinking_blocks, reasoning_content, tool_calls, web_search_results, tool_results, compaction_blocks
def calculate_usage(
self,
@ -1210,6 +1259,10 @@ class AnthropicConfig(AnthropicModelInfo, BaseConfig):
cache_creation_token_details: Optional[CacheCreationTokenDetails] = None
web_search_requests: Optional[int] = None
tool_search_requests: Optional[int] = None
inference_geo: Optional[str] = None
if "inference_geo" in _usage and _usage["inference_geo"] is not None:
inference_geo = _usage["inference_geo"]
if (
"cache_creation_input_tokens" in _usage
and _usage["cache_creation_input_tokens"] is not None
@ -1293,6 +1346,7 @@ class AnthropicConfig(AnthropicModelInfo, BaseConfig):
if (web_search_requests is not None or tool_search_requests is not None)
else None
),
inference_geo=inference_geo,
)
return usage
@ -1335,6 +1389,8 @@ class AnthropicConfig(AnthropicModelInfo, BaseConfig):
reasoning_content,
tool_calls,
web_search_results,
tool_results,
compaction_blocks,
) = self.extract_response_content(completion_response=completion_response)
if (
@ -1360,6 +1416,8 @@ class AnthropicConfig(AnthropicModelInfo, BaseConfig):
provider_specific_fields["web_search_results"] = web_search_results
if container is not None:
provider_specific_fields["container"] = container
if compaction_blocks is not None:
provider_specific_fields["compaction_blocks"] = compaction_blocks
_message = litellm.Message(
tool_calls=tool_calls,
@ -1368,6 +1426,7 @@ class AnthropicConfig(AnthropicModelInfo, BaseConfig):
thinking_blocks=thinking_blocks,
reasoning_content=reasoning_content,
)
_message.provider_specific_fields = provider_specific_fields
## HANDLE JSON MODE - anthropic returns single function call
json_mode_message = self._transform_response_for_json_mode(
@ -1398,18 +1457,7 @@ class AnthropicConfig(AnthropicModelInfo, BaseConfig):
model_response.created = int(time.time())
model_response.model = completion_response["model"]
context_management_response = completion_response.get("context_management")
if context_management_response is not None:
_hidden_params["context_management"] = context_management_response
try:
model_response.__dict__["context_management"] = (
context_management_response
)
except Exception:
pass
model_response._hidden_params = _hidden_params
return model_response
def get_prefix_prompt(self, messages: List[AllMessageValues]) -> Optional[str]:

View file

@ -22,10 +22,17 @@ def cost_per_token(model: str, usage: "Usage") -> Tuple[float, float]:
Returns:
Tuple[float, float] - prompt_cost_in_usd, completion_cost_in_usd
"""
return generic_cost_per_token(
model=model, usage=usage, custom_llm_provider="anthropic"
# If usage has inference_geo, prepend it as prefix to model name
if hasattr(usage, "inference_geo") and usage.inference_geo and usage.inference_geo.lower() not in ["global", "not_available"]:
model_with_geo_prefix = f"{usage.inference_geo}/{model}"
else:
model_with_geo_prefix = model
prompt_cost, completion_cost = generic_cost_per_token(
model=model_with_geo_prefix, usage=usage, custom_llm_provider="anthropic"
)
return prompt_cost, completion_cost
def get_cost_for_anthropic_web_search(
model_info: Optional["ModelInfo"] = None,

View file

@ -2,7 +2,11 @@ from typing import Any, AsyncIterator, Dict, List, Optional, Tuple
import httpx
from litellm.litellm_core_utils.litellm_logging import Logging as LiteLLMLoggingObj, verbose_logger
from litellm.anthropic_beta_headers_manager import (
update_headers_with_filtered_beta,
)
from litellm.litellm_core_utils.litellm_logging import Logging as LiteLLMLoggingObj
from litellm.litellm_core_utils.litellm_logging import verbose_logger
from litellm.llms.base_llm.anthropic_messages.transformation import (
BaseAnthropicMessagesConfig,
)
@ -80,6 +84,11 @@ class AnthropicMessagesConfig(BaseAnthropicMessagesConfig):
context_management=optional_params.get("context_management"),
)
headers = update_headers_with_filtered_beta(
headers=headers,
provider="anthropic",
)
return headers, api_base
def transform_anthropic_messages_request(

View file

@ -3,6 +3,9 @@ Azure Anthropic transformation config - extends AnthropicConfig with Azure authe
"""
from typing import TYPE_CHECKING, Dict, List, Optional, Union
from litellm.anthropic_beta_headers_manager import (
update_headers_with_filtered_beta,
)
from litellm.llms.anthropic.chat.transformation import AnthropicConfig
from litellm.llms.azure.common_utils import BaseAzureLLM
from litellm.types.llms.openai import AllMessageValues
@ -87,6 +90,12 @@ class AzureAnthropicConfig(AnthropicConfig):
if "anthropic-version" not in headers:
headers["anthropic-version"] = "2023-06-01"
# Filter out unsupported beta headers for Azure AI
headers = update_headers_with_filtered_beta(
headers=headers,
provider="azure_ai",
)
return headers
def transform_request(

View file

@ -11,6 +11,9 @@ import httpx
import litellm
from litellm._logging import verbose_logger
from litellm.anthropic_beta_headers_manager import (
filter_and_transform_beta_headers,
)
from litellm.constants import RESPONSE_FORMAT_TOOL_NAME
from litellm.litellm_core_utils.core_helpers import (
filter_exceptions_from_params,
@ -60,6 +63,7 @@ from ..common_utils import (
BedrockModelInfo,
get_anthropic_beta_from_headers,
get_bedrock_tool_name,
is_claude_4_5_on_bedrock,
)
# Computer use tool prefixes supported by Bedrock
@ -70,6 +74,14 @@ BEDROCK_COMPUTER_USE_TOOLS = [
"text_editor_",
]
# Beta header patterns that are not supported by Bedrock Converse API
# These will be filtered out to prevent errors
UNSUPPORTED_BEDROCK_CONVERSE_BETA_PATTERNS = [
"advanced-tool-use", # Bedrock Converse doesn't support advanced-tool-use beta headers
"prompt-caching", # Prompt caching not supported in Converse API
"compact-2026-01-12", # The compact beta feature is not currently supported on the Converse and ConverseStream APIs
]
class AmazonConverseConfig(BaseConfig):
"""
@ -339,6 +351,56 @@ class AmazonConverseConfig(BaseConfig):
}
}
def _handle_reasoning_effort_parameter(
self, model: str, reasoning_effort: str, optional_params: dict
) -> None:
"""
Handle the reasoning_effort parameter based on the model type.
Different model families handle reasoning effort differently:
- GPT-OSS models: Keep reasoning_effort as-is (passed to additionalModelRequestFields)
- Nova Lite 2 models: Transform to reasoningConfig structure
- Other models (Anthropic, etc.): Convert to thinking parameter
Args:
model: The model identifier
reasoning_effort: The reasoning effort value
optional_params: Dictionary of optional parameters to update in-place
Examples:
>>> config = AmazonConverseConfig()
>>> params = {}
>>> config._handle_reasoning_effort_parameter("gpt-oss-model", "high", params)
>>> params
{'reasoning_effort': 'high'}
>>> params = {}
>>> config._handle_reasoning_effort_parameter("amazon.nova-2-lite-v1:0", "high", params)
>>> params
{'reasoningConfig': {'type': 'enabled', 'maxReasoningEffort': 'high'}}
>>> params = {}
>>> config._handle_reasoning_effort_parameter("anthropic.claude-3", "high", params)
>>> params
{'thinking': {'type': 'enabled', 'budget_tokens': 10000}}
"""
if "gpt-oss" in model:
# GPT-OSS models: keep reasoning_effort as-is
# It will be passed through to additionalModelRequestFields
optional_params["reasoning_effort"] = reasoning_effort
elif self._is_nova_lite_2_model(model):
# Nova Lite 2 models: transform to reasoningConfig
reasoning_config = self._transform_reasoning_effort_to_reasoning_config(
reasoning_effort
)
optional_params.update(reasoning_config)
else:
# Anthropic and other models: convert to thinking parameter
optional_params["thinking"] = AnthropicConfig._map_reasoning_effort(
reasoning_effort=reasoning_effort, model=model
)
def get_supported_openai_params(self, model: str) -> List[str]:
from litellm.utils import supports_function_calling
@ -967,7 +1029,28 @@ class AmazonConverseConfig(BaseConfig):
# Add computer use tools and anthropic_beta if needed (only when computer use tools are present)
if computer_use_tools:
anthropic_beta_list.append("computer-use-2024-10-22")
# Determine the correct computer-use beta header based on model
# "computer-use-2025-11-24" for Claude Opus 4.6, Claude Opus 4.5
# "computer-use-2025-01-24" for Claude Sonnet 4.5, Haiku 4.5, Opus 4.1, Sonnet 4, Opus 4, and Sonnet 3.7
# "computer-use-2024-10-22" for older models
model_lower = model.lower()
if "opus-4.6" in model_lower or "opus_4.6" in model_lower or "opus-4-6" in model_lower or "opus_4_6" in model_lower:
computer_use_header = "computer-use-2025-11-24"
elif "opus-4.5" in model_lower or "opus_4.5" in model_lower or "opus-4-5" in model_lower or "opus_4_5" in model_lower:
computer_use_header = "computer-use-2025-11-24"
elif any(pattern in model_lower for pattern in [
"sonnet-4.5", "sonnet_4.5", "sonnet-4-5", "sonnet_4_5",
"haiku-4.5", "haiku_4.5", "haiku-4-5", "haiku_4_5",
"opus-4.1", "opus_4.1", "opus-4-1", "opus_4_1",
"sonnet-4", "sonnet_4",
"opus-4", "opus_4",
"sonnet-3.7", "sonnet_3.7", "sonnet-3-7", "sonnet_3_7"
]):
computer_use_header = "computer-use-2025-01-24"
else:
computer_use_header = "computer-use-2024-10-22"
anthropic_beta_list.append(computer_use_header)
# Transform computer use tools to proper Bedrock format
transformed_computer_tools = self._transform_computer_use_tools(
computer_use_tools
@ -989,7 +1072,14 @@ class AmazonConverseConfig(BaseConfig):
if beta not in seen:
unique_betas.append(beta)
seen.add(beta)
additional_request_params["anthropic_beta"] = unique_betas
filtered_betas = filter_and_transform_beta_headers(
beta_headers=unique_betas,
provider="bedrock_converse",
)
if filtered_betas:
additional_request_params["anthropic_beta"] = filtered_betas
return bedrock_tools, anthropic_beta_list

View file

@ -12,6 +12,9 @@ from typing import (
import httpx
from litellm.anthropic_beta_headers_manager import (
filter_and_transform_beta_headers,
)
from litellm.llms.anthropic.common_utils import AnthropicModelInfo
from litellm.llms.anthropic.experimental_pass_through.messages.transformation import (
AnthropicMessagesConfig,
@ -50,6 +53,10 @@ class AmazonAnthropicClaudeMessagesConfig(
DEFAULT_BEDROCK_ANTHROPIC_API_VERSION = "bedrock-2023-05-31"
# Beta header patterns that are not supported by Bedrock Invoke API
# These will be filtered out to prevent 400 "invalid beta flag" errors
def __init__(self, **kwargs):
BaseAnthropicMessagesConfig.__init__(self, **kwargs)
AmazonInvokeConfig.__init__(self, **kwargs)
@ -129,6 +136,182 @@ class AmazonAnthropicClaudeMessagesConfig(
if isinstance(cache_control, dict) and "ttl" in cache_control:
cache_control.pop("ttl", None)
def _supports_extended_thinking_on_bedrock(self, model: str) -> bool:
"""
Check if the model supports extended thinking beta headers on Bedrock.
On 3rd-party platforms (e.g., Amazon Bedrock), extended thinking is only
supported on: Claude Opus 4.5, Claude Opus 4.1, Opus 4, or Sonnet 4.
Ref: https://docs.anthropic.com/en/docs/build-with-claude/extended-thinking
Args:
model: The model name
Returns:
True if the model supports extended thinking on Bedrock
"""
model_lower = model.lower()
# Supported models on Bedrock for extended thinking
supported_patterns = [
"opus-4.5", "opus_4.5", "opus-4-5", "opus_4_5", # Opus 4.5
"opus-4.1", "opus_4.1", "opus-4-1", "opus_4_1", # Opus 4.1
"opus-4", "opus_4", # Opus 4
"sonnet-4", "sonnet_4", # Sonnet 4
]
return any(pattern in model_lower for pattern in supported_patterns)
def _filter_unsupported_beta_headers_for_bedrock(
self, model: str, beta_set: set
) -> None:
"""
Remove beta headers that are not supported on Bedrock for the given model.
Extended thinking beta headers are only supported on specific Claude 4+ models.
Advanced tool use headers are not supported on Bedrock Invoke API.
This prevents 400 "invalid beta flag" errors on Bedrock.
Note: Bedrock Invoke API fails with a 400 error when unsupported beta headers
are sent, returning: {"message":"invalid beta flag"}
Args:
model: The model name
beta_set: The set of beta headers to filter in-place
"""
# 1. Handle header transformations BEFORE filtering
# (advanced-tool-use -> tool-search-tool)
# This must happen before filtering because advanced-tool-use is in the unsupported list
has_advanced_tool_use = "advanced-tool-use-2025-11-20" in beta_set
if has_advanced_tool_use and self._supports_tool_search_on_bedrock(model):
beta_set.discard("advanced-tool-use-2025-11-20")
beta_set.add("tool-search-tool-2025-10-19")
beta_set.add("tool-examples-2025-10-29")
# 2. Apply provider-level filtering using centralized JSON config
beta_list = list(beta_set)
filtered_list = filter_and_transform_beta_headers(
beta_headers=beta_list,
provider="bedrock",
)
# Update the set with filtered headers
beta_set.clear()
beta_set.update(filtered_list)
# 2.1. Handle model-specific exceptions: structured-outputs is only supported on Opus 4.6
# Re-add structured-outputs if it was in the original set and model is Opus 4.6
model_lower = model.lower()
is_opus_4_6 = any(pattern in model_lower for pattern in ["opus-4.6", "opus_4.6", "opus-4-6", "opus_4_6"])
if is_opus_4_6 and "structured-outputs-2025-11-13" in beta_list:
beta_set.add("structured-outputs-2025-11-13")
# 3. Filter out extended thinking headers for models that don't support them
extended_thinking_patterns = [
"extended-thinking",
"interleaved-thinking",
]
if not self._supports_extended_thinking_on_bedrock(model):
beta_headers_to_remove = set()
for beta in beta_set:
for pattern in extended_thinking_patterns:
if pattern in beta.lower():
beta_headers_to_remove.add(beta)
break
for beta in beta_headers_to_remove:
beta_set.discard(beta)
def _get_tool_search_beta_header_for_bedrock(
self,
model: str,
tool_search_used: bool,
programmatic_tool_calling_used: bool,
input_examples_used: bool,
beta_set: set,
) -> None:
"""
Adjust tool search beta header for Bedrock.
Bedrock requires a different beta header for tool search on Opus 4 models
when tool search is used without programmatic tool calling or input examples.
Note: On Amazon Bedrock, server-side tool search is only supported on Claude Opus 4
with the `tool-search-tool-2025-10-19` beta header.
Ref: https://platform.claude.com/docs/en/agents-and-tools/tool-use/tool-search-tool
Args:
model: The model name
tool_search_used: Whether tool search is used
programmatic_tool_calling_used: Whether programmatic tool calling is used
input_examples_used: Whether input examples are used
beta_set: The set of beta headers to modify in-place
"""
if tool_search_used and not (programmatic_tool_calling_used or input_examples_used):
beta_set.discard(ANTHROPIC_TOOL_SEARCH_BETA_HEADER)
if "opus-4" in model.lower() or "opus_4" in model.lower():
beta_set.add("tool-search-tool-2025-10-19")
def _convert_output_format_to_inline_schema(
self,
output_format: Dict,
anthropic_messages_request: Dict,
) -> None:
"""
Convert Anthropic output_format to inline schema in message content.
Bedrock Invoke doesn't support the output_format parameter, so we embed
the schema directly into the user message content as text instructions.
This approach adds the schema to the last user message, instructing the model
to respond in the specified JSON format.
Args:
output_format: The output_format dict with 'type' and 'schema'
anthropic_messages_request: The request dict to modify in-place
Ref: https://aws.amazon.com/blogs/machine-learning/structured-data-response-with-amazon-bedrock-prompt-engineering-and-tool-use/
"""
import json
# Extract schema from output_format
schema = output_format.get("schema")
if not schema:
return
# Get messages from the request
messages = anthropic_messages_request.get("messages", [])
if not messages:
return
# Find the last user message
last_user_message_idx = None
for idx in range(len(messages) - 1, -1, -1):
if messages[idx].get("role") == "user":
last_user_message_idx = idx
break
if last_user_message_idx is None:
return
last_user_message = messages[last_user_message_idx]
content = last_user_message.get("content", [])
# Ensure content is a list
if isinstance(content, str):
content = [{"type": "text", "text": content}]
last_user_message["content"] = content
# Add schema as text content to the message
schema_text = {
"type": "text",
"text": json.dumps(schema)
}
content.append(schema_text)
def transform_anthropic_messages_request(
self,
model: str,

View file

@ -298,7 +298,8 @@ class DatabricksConfig(DatabricksBase, OpenAILikeChatConfig, AnthropicConfig):
if "reasoning_effort" in non_default_params and "claude" in model:
optional_params["thinking"] = AnthropicConfig._map_reasoning_effort(
non_default_params.get("reasoning_effort")
reasoning_effort=non_default_params.get("reasoning_effort"),
model=model
)
optional_params.pop("reasoning_effort", None)
## handle thinking tokens

View file

@ -1,11 +1,19 @@
from typing import Any, Dict, List, Optional, Tuple
from litellm.anthropic_beta_headers_manager import (
update_headers_with_filtered_beta,
)
from litellm.llms.anthropic.common_utils import AnthropicModelInfo
from litellm.llms.anthropic.experimental_pass_through.messages.transformation import (
AnthropicMessagesConfig,
)
from litellm.types.llms.anthropic import (
ANTHROPIC_BETA_HEADER_VALUES,
ANTHROPIC_HOSTED_TOOLS,
)
from litellm.types.llms.anthropic_tool_search import get_tool_search_beta_header
from litellm.types.llms.vertex_ai import VertexPartnerProvider
from litellm.types.router import GenericLiteLLMParams
from litellm.types.llms.anthropic import ANTHROPIC_BETA_HEADER_VALUES, ANTHROPIC_HOSTED_TOOLS
from ....vertex_llm_base import VertexBase
@ -51,13 +59,35 @@ class VertexAIPartnerModelsAnthropicMessagesConfig(AnthropicMessagesConfig, Vert
headers["content-type"] = "application/json"
# Add web search beta header for Vertex AI only if not already set
if "anthropic-beta" not in headers:
tools = optional_params.get("tools", [])
for tool in tools:
if isinstance(tool, dict) and tool.get("type", "").startswith(ANTHROPIC_HOSTED_TOOLS.WEB_SEARCH.value):
headers["anthropic-beta"] = ANTHROPIC_BETA_HEADER_VALUES.WEB_SEARCH_2025_03_05.value
break
# Add beta headers for Vertex AI
tools = optional_params.get("tools", [])
beta_values: set[str] = set()
# Get existing beta headers if any
existing_beta = headers.get("anthropic-beta")
if existing_beta:
beta_values.update(b.strip() for b in existing_beta.split(","))
# Check for web search tool
for tool in tools:
if isinstance(tool, dict) and tool.get("type", "").startswith(ANTHROPIC_HOSTED_TOOLS.WEB_SEARCH.value):
beta_values.add(ANTHROPIC_BETA_HEADER_VALUES.WEB_SEARCH_2025_03_05.value)
break
# Check for tool search tools - Vertex AI uses different beta header
anthropic_model_info = AnthropicModelInfo()
if anthropic_model_info.is_tool_search_used(tools):
beta_values.add(get_tool_search_beta_header("vertex_ai"))
if beta_values:
headers["anthropic-beta"] = ",".join(beta_values)
# Filter out unsupported beta headers for Vertex AI
headers = update_headers_with_filtered_beta(
headers=headers,
provider="vertex_ai",
)
return headers, api_base

View file

@ -51,6 +51,40 @@ class VertexAIAnthropicConfig(AnthropicConfig):
def custom_llm_provider(self) -> Optional[str]:
return "vertex_ai"
def _add_context_management_beta_headers(
self, beta_set: set, context_management: dict
) -> None:
"""
Add context_management beta headers to the beta_set.
- If any edit has type "compact_20260112", add compact-2026-01-12 header
- For all other edits, add context-management-2025-06-27 header
Args:
beta_set: Set of beta headers to modify in-place
context_management: The context_management dict from optional_params
"""
from litellm.types.llms.anthropic import ANTHROPIC_BETA_HEADER_VALUES
edits = context_management.get("edits", [])
has_compact = False
has_other = False
for edit in edits:
edit_type = edit.get("type", "")
if edit_type == "compact_20260112":
has_compact = True
else:
has_other = True
# Add compact header if any compact edits exist
if has_compact:
beta_set.add(ANTHROPIC_BETA_HEADER_VALUES.COMPACT_2026_01_12.value)
# Add context management header if any other edits exist
if has_other:
beta_set.add(ANTHROPIC_BETA_HEADER_VALUES.CONTEXT_MANAGEMENT_2025_06_27.value)
def transform_request(
self,
model: str,
@ -83,6 +117,11 @@ class VertexAIAnthropicConfig(AnthropicConfig):
beta_set = set(auto_betas)
if tool_search_used:
beta_set.add("tool-search-tool-2025-10-19") # Vertex requires this header for tool search
# Add context_management beta headers (compact and/or context-management)
context_management = optional_params.get("context_management")
if context_management:
self._add_context_management_beta_headers(beta_set, context_management)
if beta_set:
data["anthropic_beta"] = list(beta_set)

File diff suppressed because it is too large Load diff

View file

@ -610,7 +610,7 @@ ANTHROPIC_API_ONLY_HEADERS = { # fails if calling anthropic on vertex ai / bedr
class AnthropicThinkingParam(TypedDict, total=False):
type: Literal["enabled"]
type: Literal["enabled", "adaptive"]
budget_tokens: int
@ -630,6 +630,7 @@ class ANTHROPIC_BETA_HEADER_VALUES(str, Enum):
WEB_FETCH_2025_09_10 = "web-fetch-2025-09-10"
WEB_SEARCH_2025_03_05 = "web-search-2025-03-05"
CONTEXT_MANAGEMENT_2025_06_27 = "context-management-2025-06-27"
COMPACT_2026_01_12 = "compact-2026-01-12"
STRUCTURED_OUTPUT_2025_09_25 = "structured-outputs-2025-11-13"
ADVANCED_TOOL_USE_2025_11_20 = "advanced-tool-use-2025-11-20"

View file

@ -0,0 +1,35 @@
"""
Tool Search Beta Header Configuration
Reference: https://platform.claude.com/docs/en/agents-and-tools/tool-use/tool-search-tool
"""
from typing import Dict
from litellm.types.utils import LlmProviders
# Tool search beta header values
TOOL_SEARCH_BETA_HEADER_ANTHROPIC = "advanced-tool-use-2025-11-20"
TOOL_SEARCH_BETA_HEADER_VERTEX = "tool-search-tool-2025-10-19"
TOOL_SEARCH_BETA_HEADER_BEDROCK = "tool-search-tool-2025-10-19"
# Mapping of custom_llm_provider -> tool search beta header
TOOL_SEARCH_BETA_HEADER_BY_PROVIDER: Dict[str, str] = {
LlmProviders.ANTHROPIC.value: TOOL_SEARCH_BETA_HEADER_ANTHROPIC,
LlmProviders.AZURE.value: TOOL_SEARCH_BETA_HEADER_ANTHROPIC,
LlmProviders.AZURE_AI.value: TOOL_SEARCH_BETA_HEADER_ANTHROPIC,
LlmProviders.VERTEX_AI.value: TOOL_SEARCH_BETA_HEADER_VERTEX,
LlmProviders.VERTEX_AI_BETA.value: TOOL_SEARCH_BETA_HEADER_VERTEX,
LlmProviders.BEDROCK.value: TOOL_SEARCH_BETA_HEADER_BEDROCK,
}
def get_tool_search_beta_header(custom_llm_provider: str) -> str:
"""
Get the tool search beta header for a given provider.
"""
return TOOL_SEARCH_BETA_HEADER_BY_PROVIDER.get(
custom_llm_provider,
TOOL_SEARCH_BETA_HEADER_ANTHROPIC
)

File diff suppressed because it is too large Load diff

View file

@ -185,7 +185,7 @@ def test_extract_response_content_with_citations():
},
}
_, citations, _, _, _, _ = config.extract_response_content(completion_response)
_, citations, _, _, _, _, _, _ = config.extract_response_content(completion_response)
assert citations == [
[
{
@ -342,7 +342,7 @@ def test_web_search_tool_result_extraction():
}
}
text, citations, thinking_blocks, reasoning_content, tool_calls, web_search_results = config.extract_response_content(
text, citations, thinking_blocks, reasoning_content, tool_calls, web_search_results, tool_results, compaction_blocks = config.extract_response_content(
completion_response
)
@ -474,7 +474,7 @@ def test_multiple_web_search_tool_results():
]
}
text, citations, thinking_blocks, reasoning_content, tool_calls, web_search_results = config.extract_response_content(
text, citations, thinking_blocks, reasoning_content, tool_calls, web_search_results, tool_results, compaction_blocks = config.extract_response_content(
completion_response
)
@ -697,59 +697,6 @@ def test_anthropic_chat_transform_request_includes_context_management():
assert result["context_management"] == _sample_context_management_payload()
def test_transform_parsed_response_includes_context_management_metadata():
import httpx
from litellm.types.utils import ModelResponse
config = AnthropicConfig()
context_management_payload = {
"applied_edits": [
{
"type": "clear_tool_uses_20250919",
"cleared_tool_uses": 2,
"cleared_input_tokens": 5000,
}
]
}
completion_response = {
"id": "msg_context_management_test",
"type": "message",
"role": "assistant",
"model": "claude-sonnet-4-20250514",
"content": [{"type": "text", "text": "Done."}],
"stop_reason": "end_turn",
"stop_sequence": None,
"usage": {
"input_tokens": 10,
"cache_creation_input_tokens": 0,
"cache_read_input_tokens": 0,
"output_tokens": 5,
},
"context_management": context_management_payload,
}
raw_response = httpx.Response(
status_code=200,
headers={},
)
model_response = ModelResponse()
result = config.transform_parsed_response(
completion_response=completion_response,
raw_response=raw_response,
model_response=model_response,
json_mode=False,
prefix_prompt=None,
)
assert result.__dict__.get("context_management") == context_management_payload
provider_fields = result.choices[0].message.provider_specific_fields
assert (
provider_fields
and provider_fields["context_management"] == context_management_payload
)
def test_anthropic_structured_output_beta_header():
from litellm.types.utils import CallTypes
from litellm.utils import return_raw_request
@ -923,7 +870,7 @@ def test_server_tool_use_in_response():
]
}
text, citations, thinking_blocks, reasoning_content, tool_calls, web_search_results = config.extract_response_content(
text, citations, thinking_blocks, reasoning_content, tool_calls, web_search_results, tool_results, compaction_blocks = config.extract_response_content(
completion_response
)
@ -1051,7 +998,7 @@ def test_tool_search_complete_response_parsing():
}
# Extract content
text, citations, thinking_blocks, reasoning_content, tool_calls, web_search_results = config.extract_response_content(
text, citations, thinking_blocks, reasoning_content, tool_calls, web_search_results, tool_results, compaction_blocks = config.extract_response_content(
completion_response
)
@ -1171,7 +1118,7 @@ def test_caller_field_in_response():
"usage": {"input_tokens": 100, "output_tokens": 50}
}
text, citations, thinking, reasoning, tool_calls, web_search_results = config.extract_response_content(completion_response)
text, citations, thinking, reasoning, tool_calls, web_search_results, tool_results, compaction_blocks = config.extract_response_content(completion_response)
assert len(tool_calls) == 1
assert tool_calls[0]["id"] == "toolu_123"
@ -1812,3 +1759,630 @@ def test_calculate_usage_completion_tokens_details_with_reasoning():
expected_text_tokens = 500 - usage.completion_tokens_details.reasoning_tokens
assert usage.completion_tokens_details.text_tokens == expected_text_tokens
assert usage.completion_tokens == 500
# ============ Reasoning Effort Tests ============
def test_reasoning_effort_maps_to_adaptive_thinking_for_opus_4_6():
"""
Test that reasoning_effort maps to adaptive thinking type for Claude Opus 4.6.
For Claude Opus 4.6, reasoning_effort should map to {"type": "adaptive"}
regardless of the effort level specified.
"""
config = AnthropicConfig()
# Test with different reasoning_effort values - all should map to adaptive
for effort in ["low", "medium", "high", "minimal"]:
non_default_params = {"reasoning_effort": effort}
optional_params = {}
result = config.map_openai_params(
non_default_params=non_default_params,
optional_params=optional_params,
model="claude-opus-4-6-20250514",
drop_params=False
)
# Should map to adaptive thinking type
assert "thinking" in result
assert result["thinking"]["type"] == "adaptive"
# Should not have budget_tokens for adaptive type
assert "budget_tokens" not in result["thinking"]
# reasoning_effort should not be in the result (it's transformed to thinking)
assert "reasoning_effort" not in result
def test_reasoning_effort_maps_to_budget_thinking_for_non_opus_4_6():
"""
Test that reasoning_effort maps to budget-based thinking config for non-Opus 4.6 models.
For models other than Claude Opus 4.6, reasoning_effort should map to
thinking config with budget_tokens based on the effort level.
"""
config = AnthropicConfig()
# Test with Claude Sonnet 4.5 (non-Opus 4.6 model)
test_cases = [
("low", 1024), # DEFAULT_REASONING_EFFORT_LOW_THINKING_BUDGET
("medium", 2048), # DEFAULT_REASONING_EFFORT_MEDIUM_THINKING_BUDGET
("high", 4096), # DEFAULT_REASONING_EFFORT_HIGH_THINKING_BUDGET
("minimal", 128), # DEFAULT_REASONING_EFFORT_MINIMAL_THINKING_BUDGET
]
for effort, expected_budget in test_cases:
non_default_params = {"reasoning_effort": effort}
optional_params = {}
result = config.map_openai_params(
non_default_params=non_default_params,
optional_params=optional_params,
model="claude-sonnet-4-5-20250929",
drop_params=False
)
# Should map to enabled thinking type with budget_tokens
assert "thinking" in result
assert result["thinking"]["type"] == "enabled"
assert result["thinking"]["budget_tokens"] == expected_budget
# reasoning_effort should not be in the result (it's transformed to thinking)
assert "reasoning_effort" not in result
def test_code_execution_tool_results_extraction():
"""
Test that code execution tool results (bash_code_execution_tool_result,
text_editor_code_execution_tool_result) are properly extracted and exposed
in provider_specific_fields.
Related to: https://github.com/BerriAI/litellm/issues/xxxxx
"""
import httpx
from litellm.types.utils import ModelResponse
config = AnthropicConfig()
# Mock Anthropic response with code execution tool results
mock_anthropic_response = {
"id": "msg_01XYZ",
"type": "message",
"role": "assistant",
"model": "claude-sonnet-4-5-20250929",
"content": [
{
"type": "text",
"text": "I'll calculate that for you."
},
{
"type": "server_tool_use",
"id": "srvtoolu_01ABC",
"name": "bash_code_execution",
"input": {
"command": "python3 << 'EOF'\nprint(2 + 2)\nEOF\n"
}
},
{
"type": "bash_code_execution_tool_result",
"tool_use_id": "srvtoolu_01ABC",
"content": {
"type": "bash_code_execution_result",
"stdout": "4\n",
"stderr": "",
"return_code": 0
}
},
{
"type": "server_tool_use",
"id": "srvtoolu_01DEF",
"name": "text_editor_code_execution",
"input": {
"command": "create",
"path": "test.txt",
"file_text": "Hello"
}
},
{
"type": "text_editor_code_execution_tool_result",
"tool_use_id": "srvtoolu_01DEF",
"content": {
"type": "text_editor_code_execution_result",
"is_file_update": False
}
},
{
"type": "text",
"text": "Done!"
}
],
"stop_reason": "stop",
"stop_sequence": None,
"usage": {
"input_tokens": 100,
"output_tokens": 50
}
}
# Create mock HTTP response
mock_raw_response = MagicMock(spec=httpx.Response)
mock_raw_response.json.return_value = mock_anthropic_response
mock_raw_response.status_code = 200
mock_raw_response.headers = {}
model_response = ModelResponse()
transformed_response = config.transform_parsed_response(
completion_response=mock_anthropic_response,
raw_response=mock_raw_response,
model_response=model_response,
json_mode=False,
prefix_prompt=None,
)
# Verify tool calls are present
assert transformed_response.choices[0].message.tool_calls is not None
assert len(transformed_response.choices[0].message.tool_calls) == 2
# Verify first tool call
assert transformed_response.choices[0].message.tool_calls[0].id == "srvtoolu_01ABC"
assert transformed_response.choices[0].message.tool_calls[0].function.name == "bash_code_execution"
# Verify second tool call
assert transformed_response.choices[0].message.tool_calls[1].id == "srvtoolu_01DEF"
assert transformed_response.choices[0].message.tool_calls[1].function.name == "text_editor_code_execution"
# Verify tool results are in provider_specific_fields
provider_fields = transformed_response.choices[0].message.provider_specific_fields
assert provider_fields is not None
assert "tool_results" in provider_fields
assert provider_fields["tool_results"] is not None
assert len(provider_fields["tool_results"]) == 2
# Verify bash_code_execution_tool_result
bash_result = provider_fields["tool_results"][0]
assert bash_result["type"] == "bash_code_execution_tool_result"
assert bash_result["tool_use_id"] == "srvtoolu_01ABC"
assert bash_result["content"]["stdout"] == "4\n"
assert bash_result["content"]["return_code"] == 0
# Verify text_editor_code_execution_tool_result
editor_result = provider_fields["tool_results"][1]
assert editor_result["type"] == "text_editor_code_execution_tool_result"
assert editor_result["tool_use_id"] == "srvtoolu_01DEF"
assert editor_result["content"]["is_file_update"] is False
# Verify text content is properly concatenated
assert "I'll calculate that for you." in transformed_response.choices[0].message.content
assert "Done!" in transformed_response.choices[0].message.content
def test_tool_search_tool_result_not_in_tool_results():
"""
Test that tool_search_tool_result is NOT included in tool_results
since it's internal metadata, not actual tool execution results.
"""
import httpx
from litellm.types.utils import ModelResponse
config = AnthropicConfig()
mock_anthropic_response = {
"id": "msg_01XYZ",
"type": "message",
"role": "assistant",
"model": "claude-sonnet-4-5-20250929",
"content": [
{
"type": "text",
"text": "Found tools."
},
{
"type": "tool_search_tool_result",
"tool_references": ["tool1", "tool2"]
}
],
"stop_reason": "stop",
"stop_sequence": None,
"usage": {
"input_tokens": 100,
"output_tokens": 50
}
}
mock_raw_response = MagicMock(spec=httpx.Response)
mock_raw_response.json.return_value = mock_anthropic_response
mock_raw_response.status_code = 200
mock_raw_response.headers = {}
model_response = ModelResponse()
transformed_response = config.transform_parsed_response(
completion_response=mock_anthropic_response,
raw_response=mock_raw_response,
model_response=model_response,
json_mode=False,
prefix_prompt=None,
)
# Verify tool_search_tool_result is NOT in tool_results
provider_fields = transformed_response.choices[0].message.provider_specific_fields
assert provider_fields.get("tool_results") is None
def test_web_search_tool_result_backwards_compatibility():
"""
Test that web_search_tool_result continues to be stored in web_search_results
for backwards compatibility, not in tool_results.
"""
import httpx
from litellm.types.utils import ModelResponse
config = AnthropicConfig()
mock_anthropic_response = {
"id": "msg_01XYZ",
"type": "message",
"role": "assistant",
"model": "claude-sonnet-4-5-20250929",
"content": [
{
"type": "text",
"text": "Here are the results."
},
{
"type": "web_search_tool_result",
"search_query": "test query",
"results": [{"title": "Result 1", "url": "https://example.com"}]
}
],
"stop_reason": "stop",
"stop_sequence": None,
"usage": {
"input_tokens": 100,
"output_tokens": 50
}
}
mock_raw_response = MagicMock(spec=httpx.Response)
mock_raw_response.json.return_value = mock_anthropic_response
mock_raw_response.status_code = 200
mock_raw_response.headers = {}
model_response = ModelResponse()
transformed_response = config.transform_parsed_response(
completion_response=mock_anthropic_response,
raw_response=mock_raw_response,
model_response=model_response,
json_mode=False,
prefix_prompt=None,
)
# Verify web_search_tool_result is in web_search_results (not tool_results)
provider_fields = transformed_response.choices[0].message.provider_specific_fields
assert "web_search_results" in provider_fields
assert provider_fields["web_search_results"] is not None
assert len(provider_fields["web_search_results"]) == 1
assert provider_fields["web_search_results"][0]["type"] == "web_search_tool_result"
# Should NOT be in tool_results
assert provider_fields.get("tool_results") is None
# ============ Compaction Tests ============
def test_compaction_block_extraction():
"""
Test that compaction blocks are correctly extracted from Anthropic response.
"""
config = AnthropicConfig()
completion_response = {
"id": "msg_compaction_test",
"type": "message",
"role": "assistant",
"model": "claude-opus-4-6",
"content": [
{
"type": "compaction",
"content": "Summary of the conversation: The user requested help building a web scraper..."
},
{
"type": "text",
"text": "I don't have access to real-time data, so I can't provide the current weather in San Francisco."
}
],
"stop_reason": "max_tokens",
"stop_sequence": None,
"usage": {
"input_tokens": 86,
"output_tokens": 100
}
}
text, citations, thinking_blocks, reasoning_content, tool_calls, web_search_results, tool_results, compaction_blocks = config.extract_response_content(
completion_response
)
# Verify compaction blocks are extracted
assert compaction_blocks is not None
assert len(compaction_blocks) == 1
assert compaction_blocks[0]["type"] == "compaction"
assert "Summary of the conversation" in compaction_blocks[0]["content"]
# Verify text content is extracted
assert "I don't have access to real-time data" in text
def test_compaction_block_in_provider_specific_fields():
"""
Test that compaction blocks are included in provider_specific_fields.
"""
import httpx
from litellm.types.utils import ModelResponse
config = AnthropicConfig()
completion_response = {
"id": "msg_compaction_provider_fields",
"type": "message",
"role": "assistant",
"model": "claude-opus-4-6",
"content": [
{
"type": "compaction",
"content": "Summary of the conversation: The user requested help building a web scraper..."
},
{
"type": "text",
"text": "Here is the response."
}
],
"stop_reason": "end_turn",
"usage": {
"input_tokens": 50,
"output_tokens": 25
}
}
raw_response = httpx.Response(status_code=200, headers={})
model_response = ModelResponse()
result = config.transform_parsed_response(
completion_response=completion_response,
raw_response=raw_response,
model_response=model_response,
json_mode=False,
prefix_prompt=None,
)
# Verify compaction_blocks is in provider_specific_fields
provider_fields = result.choices[0].message.provider_specific_fields
assert provider_fields is not None
assert "compaction_blocks" in provider_fields
assert len(provider_fields["compaction_blocks"]) == 1
assert provider_fields["compaction_blocks"][0]["type"] == "compaction"
assert "Summary of the conversation" in provider_fields["compaction_blocks"][0]["content"]
def test_multiple_compaction_blocks():
"""
Test that multiple compaction blocks are all extracted.
"""
config = AnthropicConfig()
completion_response = {
"content": [
{
"type": "compaction",
"content": "First summary..."
},
{
"type": "text",
"text": "Some text."
},
{
"type": "compaction",
"content": "Second summary..."
}
]
}
text, citations, thinking_blocks, reasoning_content, tool_calls, web_search_results, tool_results, compaction_blocks = config.extract_response_content(
completion_response
)
# Verify both compaction blocks are extracted
assert compaction_blocks is not None
assert len(compaction_blocks) == 2
assert compaction_blocks[0]["content"] == "First summary..."
assert compaction_blocks[1]["content"] == "Second summary..."
def test_compaction_block_request_transformation():
"""
Test that compaction blocks from provider_specific_fields are correctly
transformed back to Anthropic format in requests.
"""
from litellm.litellm_core_utils.prompt_templates.factory import (
anthropic_messages_pt,
)
messages = [
{
"role": "user",
"content": "What is the weather in San Francisco?"
},
{
"role": "assistant",
"content": [
{
"type": "text",
"text": "I don't have access to real-time data."
}
],
"provider_specific_fields": {
"compaction_blocks": [
{
"type": "compaction",
"content": "Summary of the conversation: The user requested help building a web scraper..."
}
]
}
},
{
"role": "user",
"content": "What about New York?"
}
]
result = anthropic_messages_pt(
messages=messages,
model="claude-opus-4-6",
llm_provider="anthropic"
)
# Find the assistant message
assistant_message = None
for msg in result:
if msg["role"] == "assistant":
assistant_message = msg
break
assert assistant_message is not None
assert "content" in assistant_message
assert isinstance(assistant_message["content"], list)
# Verify compaction block is at the beginning
assert assistant_message["content"][0]["type"] == "compaction"
assert "Summary of the conversation" in assistant_message["content"][0]["content"]
# Verify text content follows
text_blocks = [c for c in assistant_message["content"] if c.get("type") == "text"]
assert len(text_blocks) > 0
assert "I don't have access to real-time data" in text_blocks[0]["text"]
def test_compaction_with_context_management():
"""
Test that compaction works with context_management parameter.
"""
config = AnthropicConfig()
messages = [{"role": "user", "content": "Hello"}]
optional_params = {
"context_management": {
"edits": [
{
"type": "compact_20260112"
}
]
},
"max_tokens": 100
}
result = config.transform_request(
model="claude-opus-4-6",
messages=messages,
optional_params=optional_params,
litellm_params={},
headers={}
)
# Verify context_management is included
assert "context_management" in result
assert result["context_management"]["edits"][0]["type"] == "compact_20260112"
def test_compaction_block_with_other_content_types():
"""
Test that compaction blocks work alongside other content types like thinking blocks and tool calls.
"""
config = AnthropicConfig()
completion_response = {
"content": [
{
"type": "compaction",
"content": "Summary of previous conversation..."
},
{
"type": "thinking",
"thinking": "Let me think about this..."
},
{
"type": "text",
"text": "Based on my analysis..."
},
{
"type": "tool_use",
"id": "toolu_123",
"name": "get_weather",
"input": {"location": "San Francisco"}
}
]
}
text, citations, thinking_blocks, reasoning_content, tool_calls, web_search_results, tool_results, compaction_blocks = config.extract_response_content(
completion_response
)
# Verify all content types are extracted
assert compaction_blocks is not None
assert len(compaction_blocks) == 1
assert thinking_blocks is not None
assert len(thinking_blocks) == 1
assert "Based on my analysis" in text
assert len(tool_calls) == 1
assert tool_calls[0]["function"]["name"] == "get_weather"
def test_compaction_block_empty_list_not_added():
"""
Test that empty compaction_blocks list is not added to provider_specific_fields.
"""
import httpx
from litellm.types.utils import ModelResponse
config = AnthropicConfig()
# Response without compaction blocks
completion_response = {
"id": "msg_no_compaction",
"type": "message",
"role": "assistant",
"model": "claude-opus-4-6",
"content": [
{
"type": "text",
"text": "Just a regular response."
}
],
"stop_reason": "end_turn",
"usage": {
"input_tokens": 10,
"output_tokens": 5
}
}
raw_response = httpx.Response(status_code=200, headers={})
model_response = ModelResponse()
result = config.transform_parsed_response(
completion_response=completion_response,
raw_response=raw_response,
model_response=model_response,
json_mode=False,
prefix_prompt=None,
)
# Verify compaction_blocks is not in provider_specific_fields when there are none
provider_fields = result.choices[0].message.provider_specific_fields
if provider_fields:
assert "compaction_blocks" not in provider_fields or provider_fields.get("compaction_blocks") is None

View file

@ -235,3 +235,97 @@ class TestAzureAnthropicConfig:
assert result["max_tokens"] == 100
assert "messages" in result
def test_context_management_compact_beta_header(self):
"""Test that context_management with compact adds the correct beta header for Azure AI"""
config = AzureAnthropicConfig()
messages = [{"role": "user", "content": "Hello"}]
optional_params = {
"context_management": {
"edits": [
{
"type": "compact_20260112"
}
]
},
"max_tokens": 100
}
litellm_params = {"api_key": "test-key"}
headers = {"api-key": "test-key"}
with patch(
"litellm.llms.azure.common_utils.BaseAzureLLM._base_validate_azure_environment"
) as mock_validate:
mock_validate.return_value = {"api-key": "test-key"}
result = config.transform_request(
model="claude-opus-4-6",
messages=messages,
optional_params=optional_params,
litellm_params=litellm_params,
headers=headers,
)
# Verify context_management is included
assert "context_management" in result
assert result["context_management"]["edits"][0]["type"] == "compact_20260112"
def test_context_management_compact_beta_header_in_headers(self):
"""Test that compact beta header is added to headers for Azure AI"""
config = AzureAnthropicConfig()
messages = [{"role": "user", "content": "Hello"}]
optional_params = {
"context_management": {
"edits": [
{
"type": "compact_20260112"
}
]
},
"max_tokens": 100
}
# Test that the parent's update_headers_with_optional_anthropic_beta is called
# which should add the compact beta header
headers = {}
headers = config.update_headers_with_optional_anthropic_beta(
headers=headers,
optional_params=optional_params
)
# Verify compact beta header is present
assert "anthropic-beta" in headers
assert "compact-2026-01-12" in headers["anthropic-beta"]
def test_context_management_mixed_edits_beta_headers(self):
"""Test that context_management with both compact and other edits adds both beta headers"""
config = AzureAnthropicConfig()
messages = [{"role": "user", "content": "Hello"}]
optional_params = {
"context_management": {
"edits": [
{
"type": "compact_20260112"
},
{
"type": "replace",
"message_id": "msg_123",
"content": "new content"
}
]
},
"max_tokens": 100
}
headers = {}
headers = config.update_headers_with_optional_anthropic_beta(
headers=headers,
optional_params=optional_params
)
# Verify both beta headers are present
assert "anthropic-beta" in headers
assert "compact-2026-01-12" in headers["anthropic-beta"]
assert "context-management-2025-06-27" in headers["anthropic-beta"]

View file

@ -6,6 +6,9 @@ import pytest
sys.path.insert(
0, os.path.abspath("../../../../../..")
) # Adds the parent directory to the system path
from litellm.anthropic_beta_headers_manager import (
update_headers_with_filtered_beta,
)
from litellm.llms.vertex_ai.vertex_ai_partner_models.anthropic.transformation import (
VertexAIAnthropicConfig,
)
@ -37,8 +40,9 @@ def test_get_supported_params_thinking():
def test_vertex_ai_anthropic_web_search_header_in_completion():
"""Test that web search tool adds the required beta header for Vertex AI completion requests"""
from unittest.mock import MagicMock, patch
from litellm.llms.anthropic.common_utils import AnthropicModelInfo
# Create the config instance
model_info = AnthropicModelInfo()
@ -74,6 +78,76 @@ def test_vertex_ai_anthropic_web_search_header_in_completion():
"anthropic-beta with web-search should not be present for non-Vertex requests"
def test_vertex_ai_anthropic_context_management_compact_beta_header():
"""Test that context_management with compact adds the correct beta header for Vertex AI"""
config = VertexAIAnthropicConfig()
messages = [{"role": "user", "content": "Hello"}]
optional_params = {
"context_management": {
"edits": [
{
"type": "compact_20260112"
}
]
},
"max_tokens": 100,
"is_vertex_request": True
}
result = config.transform_request(
model="claude-opus-4-6",
messages=messages,
optional_params=optional_params,
litellm_params={},
headers={}
)
# Verify context_management is included
assert "context_management" in result
assert result["context_management"]["edits"][0]["type"] == "compact_20260112"
# Verify compact beta header is in anthropic_beta field
assert "anthropic_beta" in result
assert "compact-2026-01-12" in result["anthropic_beta"]
def test_vertex_ai_anthropic_context_management_mixed_edits():
"""Test that context_management with both compact and other edits adds both beta headers"""
config = VertexAIAnthropicConfig()
messages = [{"role": "user", "content": "Hello"}]
optional_params = {
"context_management": {
"edits": [
{
"type": "compact_20260112"
},
{
"type": "replace",
"message_id": "msg_123",
"content": "new content"
}
]
},
"max_tokens": 100,
"is_vertex_request": True
}
result = config.transform_request(
model="claude-opus-4-6",
messages=messages,
optional_params=optional_params,
litellm_params={},
headers={}
)
# Verify both beta headers are present
assert "anthropic_beta" in result
assert "compact-2026-01-12" in result["anthropic_beta"]
assert "context-management-2025-06-27" in result["anthropic_beta"]
def test_vertex_ai_anthropic_structured_output_header_not_added():
"""Test that structured output beta headers are NOT added for Vertex AI requests"""
from litellm.llms.anthropic.chat.transformation import AnthropicConfig
@ -115,3 +189,177 @@ def test_vertex_ai_anthropic_structured_output_header_not_added():
"Non-Vertex request SHOULD have anthropic-beta header for structured output"
assert result_non_vertex["anthropic-beta"] == "structured-outputs-2025-11-13", \
f"Expected 'structured-outputs-2025-11-13', got: {result_non_vertex.get('anthropic-beta')}"
def test_vertex_ai_claude_sonnet_4_5_structured_output_fix():
"""
Test fix for issue #18625: Claude Sonnet 4.5 on VertexAI should use tool-based
structured outputs instead of output_format parameter.
This test verifies that:
1. Claude Sonnet 4.5 uses tool-based structured outputs on VertexAI
2. output_format parameter is removed from the final request
3. The fix prevents "Extra inputs are not permitted" error
"""
config = VertexAIAnthropicConfig()
# Test data matching the issue report
response_format = {
"type": "json_schema",
"json_schema": {
"name": "questions",
"strict": True,
"schema": {
"type": "object",
"properties": {
"question": {
"type": "string"
},
"response": {
"type": "string"
}
},
"required": ["question", "response"],
"additionalProperties": False
}
}
}
messages = [
{"role": "user", "content": "Generate a question and answer about AI."}
]
# Test parameters that would trigger the issue
non_default_params = {
"response_format": response_format,
"max_tokens": 1000,
}
# Test 1: Verify map_openai_params forces tool-based approach for Claude Sonnet 4.5
optional_params = {}
result_params = config.map_openai_params(
non_default_params=non_default_params,
optional_params=optional_params,
model="claude-3-5-sonnet-20241022", # Claude Sonnet 4.5 model
drop_params=False,
)
# Should have tools and tool_choice (tool-based approach)
assert "tools" in result_params, "Tools should be present for structured output"
assert "tool_choice" in result_params, "Tool choice should be present for structured output"
assert "json_mode" in result_params, "JSON mode should be enabled"
# Verify the tool is the response format tool
tools = result_params["tools"]
assert len(tools) == 1, "Should have exactly one tool for response format"
assert tools[0]["name"] == "json_tool_call", "Tool should be named json_tool_call"
# Test 2: Verify transform_request removes output_format parameter
# Simulate what would happen if parent class added output_format
test_data = {
"model": "claude-3-5-sonnet-20241022",
"messages": messages,
"max_tokens": 1000,
"tools": tools,
"tool_choice": result_params["tool_choice"],
"output_format": { # This would be added by parent class for Sonnet 4.5
"type": "json_schema",
"schema": response_format["json_schema"]["schema"]
}
}
# Mock the parent transform_request to return data with output_format
original_transform = config.__class__.__bases__[0].transform_request
def mock_transform_request(self, model, messages, optional_params, litellm_params, headers):
# Return test data that includes output_format
return test_data.copy()
# Temporarily replace parent method
config.__class__.__bases__[0].transform_request = mock_transform_request
try:
final_data = config.transform_request(
model="claude-3-5-sonnet-20241022",
messages=messages,
optional_params=result_params,
litellm_params={},
headers={},
)
# Verify that output_format was removed (fixes the "Extra inputs are not permitted" error)
assert "output_format" not in final_data, "output_format should be removed for VertexAI"
assert "model" not in final_data, "model should be removed for VertexAI"
assert "tools" in final_data, "tools should still be present"
assert "tool_choice" in final_data, "tool_choice should still be present"
finally:
# Restore original method
config.__class__.__bases__[0].transform_request = original_transform
def test_vertex_ai_anthropic_other_models_still_use_tools():
"""
Test that other Anthropic models (non-Sonnet 4.5) on VertexAI also use tool-based
structured outputs, ensuring consistency across all models.
"""
config = VertexAIAnthropicConfig()
response_format = {
"type": "json_schema",
"json_schema": {
"name": "test_schema",
"schema": {
"type": "object",
"properties": {
"result": {"type": "string"}
}
}
}
}
# Test with Claude 3 Sonnet (not 4.5)
non_default_params = {"response_format": response_format}
optional_params = {}
result_params = config.map_openai_params(
non_default_params=non_default_params,
optional_params=optional_params,
model="claude-3-sonnet-20240229",
drop_params=False,
)
# Should still use tool-based approach
assert "tools" in result_params, "Claude 3 Sonnet should also use tool-based structured output"
assert "tool_choice" in result_params, "Tool choice should be present"
assert "json_mode" in result_params, "JSON mode should be enabled"
def test_vertex_ai_partner_models_anthropic_remove_prompt_caching_scope_beta_header():
"""
Test that remove_unsupported_beta correctly filters out prompt-caching-scope-2026-01-05
from the anthropic-beta headers.
"""
from litellm.llms.vertex_ai.vertex_ai_partner_models.anthropic.experimental_pass_through.transformation import (
VertexAIPartnerModelsAnthropicMessagesConfig,
)
# This beta header should be removed
PROMPT_CACHING_BETA_HEADER = "prompt-caching-scope-2026-01-05"
headers = {
"anthropic-beta": f"other-feature,{PROMPT_CACHING_BETA_HEADER},web-search-2025-03-05"
}
headers = update_headers_with_filtered_beta(headers, "vertex_ai")
beta_header = headers.get("anthropic-beta")
assert PROMPT_CACHING_BETA_HEADER not in (beta_header or ""), \
f"{PROMPT_CACHING_BETA_HEADER} should be filtered out"
assert "other-feature" in (beta_header or ""), \
"Other non-excluded beta headers should remain"
assert "web-search-2025-03-05" in (beta_header or ""), \
"Other non-excluded beta headers should remain"
# If prompt-caching was the only value, header should be removed completely
headers2 = {"anthropic-beta": PROMPT_CACHING_BETA_HEADER}
headers2 = update_headers_with_filtered_beta(headers2, "vertex_ai")
assert "anthropic-beta" not in headers2, "Header should be removed if no supported values remain"

View file

@ -201,6 +201,11 @@ ignored_keys = [
"metadata.usage_object",
"metadata.cold_storage_object_key",
"metadata.additional_usage_values.prompt_tokens_details.cache_creation_tokens",
"metadata.additional_usage_values.completion_tokens_details",
"metadata.additional_usage_values.prompt_tokens_details",
"metadata.additional_usage_values.cache_creation_input_tokens",
"metadata.additional_usage_values.cache_read_input_tokens",
"metadata.additional_usage_values.inference_geo",
"metadata.litellm_overhead_time_ms",
"metadata.cost_breakdown",
]

View file

@ -0,0 +1,306 @@
"""
Tests for the centralized Anthropic beta headers manager.
Design: JSON config lists UNSUPPORTED headers for each provider.
Headers not in the unsupported list are passed through.
Header transformations (e.g., advanced-tool-use -> tool-search-tool) happen in code, not in JSON.
"""
import pytest
from litellm.anthropic_beta_headers_manager import (
filter_and_transform_beta_headers,
get_provider_beta_header,
get_provider_name,
get_unsupported_headers,
is_beta_header_supported,
update_headers_with_filtered_beta,
)
class TestProviderNameResolution:
"""Test provider name resolution and aliases."""
def test_get_provider_name_direct(self):
"""Test direct provider names."""
assert get_provider_name("anthropic") == "anthropic"
assert get_provider_name("bedrock") == "bedrock"
assert get_provider_name("vertex_ai") == "vertex_ai"
assert get_provider_name("azure_ai") == "azure_ai"
def test_get_provider_name_alias(self):
"""Test provider aliases."""
# Note: Aliases are defined in the JSON config
# If no alias exists, the original name is returned
assert get_provider_name("azure") == "azure" # No alias defined
assert get_provider_name("vertex_ai_beta") == "vertex_ai_beta" # No alias defined
class TestBetaHeaderSupport:
"""Test beta header support checks (unsupported list approach)."""
def test_anthropic_supports_all_headers(self):
"""Anthropic should support all beta headers (empty unsupported list)."""
headers = [
"web-fetch-2025-09-10",
"web-search-2025-03-05",
"context-management-2025-06-27",
"compact-2026-01-12",
"structured-outputs-2025-11-13",
"advanced-tool-use-2025-11-20",
]
for header in headers:
assert is_beta_header_supported(header, "anthropic")
def test_bedrock_unsupported_headers(self):
"""Bedrock should block specific headers."""
# Not supported (in unsupported list)
assert not is_beta_header_supported("advanced-tool-use-2025-11-20", "bedrock")
assert not is_beta_header_supported(
"prompt-caching-scope-2026-01-05", "bedrock"
)
assert not is_beta_header_supported("structured-outputs-2025-11-13", "bedrock")
# Supported (not in unsupported list)
assert is_beta_header_supported("context-management-2025-06-27", "bedrock")
assert is_beta_header_supported("effort-2025-11-24", "bedrock")
assert is_beta_header_supported("tool-examples-2025-10-29", "bedrock")
def test_vertex_ai_unsupported_headers(self):
"""Vertex AI should block specific headers."""
# Not supported (in unsupported list)
assert not is_beta_header_supported(
"prompt-caching-scope-2026-01-05", "vertex_ai"
)
# Supported (not in unsupported list)
assert is_beta_header_supported("web-search-2025-03-05", "vertex_ai")
assert is_beta_header_supported("context-management-2025-06-27", "vertex_ai")
assert is_beta_header_supported("effort-2025-11-24", "vertex_ai")
assert is_beta_header_supported("advanced-tool-use-2025-11-20", "vertex_ai")
class TestBetaHeaderTransformation:
"""Test beta header support checking (transformations happen in code, not here)."""
def test_anthropic_no_transformation(self):
"""Anthropic headers should pass through (empty unsupported list)."""
header = "advanced-tool-use-2025-11-20"
assert get_provider_beta_header(header, "anthropic") == header
def test_bedrock_unsupported_returns_none(self):
"""Bedrock should return None for unsupported headers."""
header = "advanced-tool-use-2025-11-20"
# This header is in bedrock's unsupported list
assert get_provider_beta_header(header, "bedrock") is None
def test_vertex_ai_supported_returns_original(self):
"""Vertex AI should return original for supported headers."""
header = "advanced-tool-use-2025-11-20"
# This header is NOT in vertex_ai's unsupported list
assert get_provider_beta_header(header, "vertex_ai") == header
def test_unsupported_header_returns_none(self):
"""Unsupported headers (in unsupported list) should return None."""
header = "prompt-caching-scope-2026-01-05"
assert get_provider_beta_header(header, "bedrock") is None
def test_supported_header_returns_original(self):
"""Supported headers (not in unsupported list) should return original."""
header = "context-management-2025-06-27"
assert get_provider_beta_header(header, "bedrock") == header
class TestFilterAndTransformBetaHeaders:
"""Test the main filtering and transformation function."""
def test_anthropic_keeps_all_headers(self):
"""Anthropic should keep all headers (empty unsupported list)."""
headers = [
"web-fetch-2025-09-10",
"context-management-2025-06-27",
"structured-outputs-2025-11-13",
"some-new-future-header-2026-01-01", # Even unknown headers pass through
]
result = filter_and_transform_beta_headers(headers, "anthropic")
assert set(result) == set(headers)
def test_bedrock_filters_unsupported(self):
"""Bedrock should filter out headers in unsupported list."""
headers = [
"context-management-2025-06-27", # Not in unsupported list -> kept
"advanced-tool-use-2025-11-20", # In unsupported list -> dropped
"structured-outputs-2025-11-13", # In unsupported list -> dropped
"prompt-caching-scope-2026-01-05", # In unsupported list -> dropped
]
result = filter_and_transform_beta_headers(headers, "bedrock")
assert "context-management-2025-06-27" in result
assert "advanced-tool-use-2025-11-20" not in result
assert "structured-outputs-2025-11-13" not in result
assert "prompt-caching-scope-2026-01-05" not in result
def test_bedrock_no_transformations_in_filter(self):
"""Bedrock filtering doesn't do transformations (those happen in code)."""
headers = ["advanced-tool-use-2025-11-20"]
result = filter_and_transform_beta_headers(headers, "bedrock")
# advanced-tool-use is in unsupported list, so it gets dropped
assert result == []
def test_vertex_ai_filters_unsupported(self):
"""Vertex AI should filter unsupported headers."""
headers = [
"web-search-2025-03-05", # Not in unsupported list -> kept
"advanced-tool-use-2025-11-20", # Not in unsupported list -> kept
"prompt-caching-scope-2026-01-05", # In unsupported list -> dropped
]
result = filter_and_transform_beta_headers(headers, "vertex_ai")
assert "web-search-2025-03-05" in result
assert "advanced-tool-use-2025-11-20" in result # Kept as-is, transformation happens in code
assert "prompt-caching-scope-2026-01-05" not in result
def test_empty_list_returns_empty(self):
"""Empty list should return empty list."""
result = filter_and_transform_beta_headers([], "anthropic")
assert result == []
def test_bedrock_converse_more_restrictive(self):
"""Bedrock Converse should be more restrictive than Bedrock."""
headers = [
"context-management-2025-06-27",
"advanced-tool-use-2025-11-20",
"tool-examples-2025-10-29",
]
bedrock_result = filter_and_transform_beta_headers(headers, "bedrock")
converse_result = filter_and_transform_beta_headers(headers, "bedrock_converse")
# Bedrock Converse has more restrictions
# advanced-tool-use is in both unsupported lists
assert "advanced-tool-use-2025-11-20" not in bedrock_result
assert "advanced-tool-use-2025-11-20" not in converse_result
# tool-examples is supported on bedrock but not converse
# Actually, looking at the JSON, tool-examples is NOT in bedrock unsupported list
# So it should be in bedrock_result
assert "tool-examples-2025-10-29" in bedrock_result
# But it's not explicitly in converse unsupported list either, so it passes through
# Let me check the actual behavior
assert "context-management-2025-06-27" in bedrock_result
assert "context-management-2025-06-27" in converse_result
def test_unknown_future_headers_pass_through(self):
"""Headers not in unsupported list should pass through (future-proof)."""
headers = ["some-new-beta-2026-05-01", "another-feature-2026-06-01"]
result = filter_and_transform_beta_headers(headers, "anthropic")
assert set(result) == set(headers)
class TestUpdateHeadersWithFilteredBeta:
"""Test the headers update function."""
def test_update_headers_anthropic(self):
"""Test updating headers for Anthropic."""
headers = {
"anthropic-beta": "web-fetch-2025-09-10,context-management-2025-06-27"
}
result = update_headers_with_filtered_beta(headers, "anthropic")
assert "anthropic-beta" in result
beta_values = set(result["anthropic-beta"].split(","))
assert "web-fetch-2025-09-10" in beta_values
assert "context-management-2025-06-27" in beta_values
def test_update_headers_bedrock_filters(self):
"""Test updating headers for Bedrock with filtering."""
headers = {
"anthropic-beta": "context-management-2025-06-27,advanced-tool-use-2025-11-20"
}
result = update_headers_with_filtered_beta(headers, "bedrock")
assert "anthropic-beta" in result
assert "context-management-2025-06-27" in result["anthropic-beta"]
assert "advanced-tool-use-2025-11-20" not in result["anthropic-beta"]
def test_update_headers_bedrock_no_transformations(self):
"""Test that filtering doesn't do transformations (those happen in code)."""
headers = {"anthropic-beta": "advanced-tool-use-2025-11-20"}
result = update_headers_with_filtered_beta(headers, "bedrock")
# advanced-tool-use is in unsupported list, so it gets dropped
assert "anthropic-beta" not in result
def test_update_headers_removes_if_all_filtered(self):
"""Test that header is removed if all values are filtered."""
headers = {"anthropic-beta": "advanced-tool-use-2025-11-20,prompt-caching-scope-2026-01-05"}
result = update_headers_with_filtered_beta(headers, "bedrock")
assert "anthropic-beta" not in result
def test_update_headers_no_beta_header(self):
"""Test updating headers when no beta header exists."""
headers = {"content-type": "application/json"}
result = update_headers_with_filtered_beta(headers, "anthropic")
assert "anthropic-beta" not in result
assert headers == result
class TestGetUnsupportedHeaders:
"""Test getting unsupported headers for a provider."""
def test_anthropic_has_no_unsupported(self):
"""Anthropic should have no unsupported headers (empty list)."""
anthropic_unsupported = get_unsupported_headers("anthropic")
assert len(anthropic_unsupported) == 0
def test_bedrock_converse_most_restrictive(self):
"""Bedrock Converse should have more unsupported headers than Bedrock."""
bedrock_unsupported = get_unsupported_headers("bedrock")
converse_unsupported = get_unsupported_headers("bedrock_converse")
# Converse has more restrictions
assert len(converse_unsupported) >= len(bedrock_unsupported)
def test_all_providers_have_config(self):
"""All providers should have a configuration entry."""
providers = ["anthropic", "azure_ai", "bedrock", "bedrock_converse", "vertex_ai"]
for provider in providers:
unsupported = get_unsupported_headers(provider)
# Should return a list (even if empty)
assert isinstance(unsupported, list), f"Provider {provider} should return a list"
class TestEdgeCases:
"""Test edge cases and error handling."""
def test_unknown_provider(self):
"""Unknown provider with no config should pass through all headers."""
result = filter_and_transform_beta_headers(
["context-management-2025-06-27"], "unknown_provider"
)
# Unknown providers have no unsupported list, so headers pass through
assert "context-management-2025-06-27" in result
def test_whitespace_handling(self):
"""Headers with whitespace should be handled correctly."""
headers = [
" context-management-2025-06-27 ",
" web-search-2025-03-05 ",
]
result = filter_and_transform_beta_headers(headers, "anthropic")
assert len(result) == 2
def test_duplicate_headers(self):
"""Duplicate headers should be deduplicated."""
headers = [
"context-management-2025-06-27",
"context-management-2025-06-27",
]
result = filter_and_transform_beta_headers(headers, "anthropic")
assert len(result) == 1
def test_case_sensitivity(self):
"""Headers should be case-sensitive."""
# Correct case - should pass through for anthropic (no unsupported list)
headers = ["context-management-2025-06-27"]
result = filter_and_transform_beta_headers(headers, "anthropic")
assert len(result) == 1
# Wrong case - should still pass through (not in unsupported list)
headers = ["Context-Management-2025-06-27"]
result = filter_and_transform_beta_headers(headers, "anthropic")
assert len(result) == 1 # Passes through because anthropic has empty unsupported list

View file

@ -0,0 +1,183 @@
"""
Validate Claude Opus 4.6 model configuration entries.
"""
import json
import os
import litellm
def test_opus_4_6_model_pricing_and_capabilities():
json_path = os.path.join(os.path.dirname(__file__), "../../model_prices_and_context_window.json")
with open(json_path) as f:
model_data = json.load(f)
expected_models = {
"claude-opus-4-6": {
"provider": "anthropic",
"has_long_context_pricing": True,
"tool_use_system_prompt_tokens": 346,
"max_input_tokens": 1000000,
},
"claude-opus-4-6-20260205": {
"provider": "anthropic",
"has_long_context_pricing": True,
"tool_use_system_prompt_tokens": 346,
"max_input_tokens": 1000000,
},
"anthropic.claude-opus-4-6-v1": {
"provider": "bedrock_converse",
"has_long_context_pricing": True,
"tool_use_system_prompt_tokens": 346,
"max_input_tokens": 1000000,
},
"vertex_ai/claude-opus-4-6": {
"provider": "vertex_ai-anthropic_models",
"has_long_context_pricing": True,
"tool_use_system_prompt_tokens": 346,
"max_input_tokens": 1000000,
},
"azure_ai/claude-opus-4-6": {
"provider": "azure_ai",
"has_long_context_pricing": False,
"tool_use_system_prompt_tokens": 159,
"max_input_tokens": 200000,
},
}
for model_name, config in expected_models.items():
assert model_name in model_data, f"Missing model entry: {model_name}"
info = model_data[model_name]
assert info["litellm_provider"] == config["provider"]
assert info["mode"] == "chat"
assert info["max_input_tokens"] == config["max_input_tokens"]
assert info["max_output_tokens"] == 128000
assert info["max_tokens"] == 128000
assert info["input_cost_per_token"] == 5e-06
assert info["output_cost_per_token"] == 2.5e-05
assert info["cache_creation_input_token_cost"] == 6.25e-06
assert info["cache_read_input_token_cost"] == 5e-07
if config["has_long_context_pricing"]:
assert info["input_cost_per_token_above_200k_tokens"] == 1e-05
assert info["output_cost_per_token_above_200k_tokens"] == 3.75e-05
assert info["cache_creation_input_token_cost_above_200k_tokens"] == 1.25e-05
assert info["cache_read_input_token_cost_above_200k_tokens"] == 1e-06
assert info["supports_assistant_prefill"] is False
assert info["supports_function_calling"] is True
assert info["supports_prompt_caching"] is True
assert info["supports_reasoning"] is True
assert info["supports_tool_choice"] is True
assert info["supports_vision"] is True
assert info["tool_use_system_prompt_tokens"] == config["tool_use_system_prompt_tokens"]
def test_opus_4_6_bedrock_regional_model_pricing():
json_path = os.path.join(os.path.dirname(__file__), "../../model_prices_and_context_window.json")
with open(json_path) as f:
model_data = json.load(f)
expected_models = {
"global.anthropic.claude-opus-4-6-v1": {
"input_cost_per_token": 5e-06,
"output_cost_per_token": 2.5e-05,
"cache_creation_input_token_cost": 6.25e-06,
"cache_read_input_token_cost": 5e-07,
"input_cost_per_token_above_200k_tokens": 1e-05,
"output_cost_per_token_above_200k_tokens": 3.75e-05,
"cache_creation_input_token_cost_above_200k_tokens": 1.25e-05,
"cache_read_input_token_cost_above_200k_tokens": 1e-06,
},
"us.anthropic.claude-opus-4-6-v1": {
"input_cost_per_token": 5.5e-06,
"output_cost_per_token": 2.75e-05,
"cache_creation_input_token_cost": 6.875e-06,
"cache_read_input_token_cost": 5.5e-07,
"input_cost_per_token_above_200k_tokens": 1.1e-05,
"output_cost_per_token_above_200k_tokens": 4.125e-05,
"cache_creation_input_token_cost_above_200k_tokens": 1.375e-05,
"cache_read_input_token_cost_above_200k_tokens": 1.1e-06,
},
"eu.anthropic.claude-opus-4-6-v1": {
"input_cost_per_token": 5.5e-06,
"output_cost_per_token": 2.75e-05,
"cache_creation_input_token_cost": 6.875e-06,
"cache_read_input_token_cost": 5.5e-07,
"input_cost_per_token_above_200k_tokens": 1.1e-05,
"output_cost_per_token_above_200k_tokens": 4.125e-05,
"cache_creation_input_token_cost_above_200k_tokens": 1.375e-05,
"cache_read_input_token_cost_above_200k_tokens": 1.1e-06,
},
"apac.anthropic.claude-opus-4-6-v1": {
"input_cost_per_token": 5.5e-06,
"output_cost_per_token": 2.75e-05,
"cache_creation_input_token_cost": 6.875e-06,
"cache_read_input_token_cost": 5.5e-07,
"input_cost_per_token_above_200k_tokens": 1.1e-05,
"output_cost_per_token_above_200k_tokens": 4.125e-05,
"cache_creation_input_token_cost_above_200k_tokens": 1.375e-05,
"cache_read_input_token_cost_above_200k_tokens": 1.1e-06,
},
"apac.anthropic.claude-opus-4-6-v1": {
"input_cost_per_token": 5.5e-06,
"output_cost_per_token": 2.75e-05,
"cache_creation_input_token_cost": 6.875e-06,
"cache_read_input_token_cost": 5.5e-07,
"input_cost_per_token_above_200k_tokens": 1.1e-05,
"output_cost_per_token_above_200k_tokens": 4.125e-05,
"cache_creation_input_token_cost_above_200k_tokens": 1.375e-05,
"cache_read_input_token_cost_above_200k_tokens": 1.1e-06,
},
}
for model_name, expected in expected_models.items():
assert model_name in model_data, f"Missing model entry: {model_name}"
info = model_data[model_name]
assert info["litellm_provider"] == "bedrock_converse"
assert info["max_input_tokens"] == 1000000
assert info["max_output_tokens"] == 128000
assert info["max_tokens"] == 128000
assert info["supports_assistant_prefill"] is False
assert info["tool_use_system_prompt_tokens"] == 346
for key, value in expected.items():
assert info[key] == value
def test_opus_4_6_alias_and_dated_metadata_match():
json_path = os.path.join(os.path.dirname(__file__), "../../model_prices_and_context_window.json")
with open(json_path) as f:
model_data = json.load(f)
alias = model_data["claude-opus-4-6"]
dated = model_data["claude-opus-4-6-20260205"]
keys_to_match = [
"max_input_tokens",
"max_output_tokens",
"max_tokens",
"input_cost_per_token",
"output_cost_per_token",
"cache_creation_input_token_cost",
"cache_creation_input_token_cost_above_1hr",
"cache_read_input_token_cost",
"input_cost_per_token_above_200k_tokens",
"output_cost_per_token_above_200k_tokens",
"cache_creation_input_token_cost_above_200k_tokens",
"cache_read_input_token_cost_above_200k_tokens",
"supports_assistant_prefill",
"tool_use_system_prompt_tokens",
]
for key in keys_to_match:
assert alias[key] == dated[key], f"Mismatch for {key}"
def test_opus_4_6_bedrock_converse_registration():
assert "anthropic.claude-opus-4-6-v1" in litellm.BEDROCK_CONVERSE_MODELS
assert "global.anthropic.claude-opus-4-6-v1" in litellm.bedrock_converse_models
assert "us.anthropic.claude-opus-4-6-v1" in litellm.bedrock_converse_models
assert "eu.anthropic.claude-opus-4-6-v1" in litellm.bedrock_converse_models
assert "apac.anthropic.claude-opus-4-6-v1" in litellm.bedrock_converse_models

View file

@ -1,6 +1,10 @@
{
"compilerOptions": {
"lib": ["dom", "dom.iterable", "esnext"],
"lib": [
"dom",
"dom.iterable",
"esnext"
],
"allowJs": true,
"skipLibCheck": true,
"strict": true,
@ -10,7 +14,7 @@
"moduleResolution": "bundler",
"resolveJsonModule": true,
"isolatedModules": true,
"jsx": "preserve",
"jsx": "react-jsx",
"incremental": true,
"plugins": [
{
@ -18,9 +22,20 @@
}
],
"paths": {
"@/*": ["./src/*"]
}
"@/*": [
"./src/*"
]
},
"target": "ES2017"
},
"include": ["next-env.d.ts", "**/*.ts", "**/*.tsx", ".next/types/**/*.ts"],
"exclude": ["node_modules"]
"include": [
"next-env.d.ts",
"**/*.ts",
"**/*.tsx",
".next/types/**/*.ts",
".next/dev/types/**/*.ts"
],
"exclude": [
"node_modules"
]
}