Fixes issue where tool_result content blocks include explicit
'cache_control': null which breaks some Anthropic API channels.
Changes:
- Only include cache_control field when explicitly set and not None
- Prevents serialization of null values in tool_result text content
- Maintains backward compatibility with existing cache_control usage
Related issue: Anthropic tool_result conversion adds explicit null values
that cause compatibility issues with certain API implementations.
Co-Authored-By: Claude (claude-4.5-sonnet) <noreply@anthropic.com>
When Gemini uses implicit caching, it returns cachedContentTokenCount but
NOT cacheTokensDetails. Previously, text_tokens was not adjusted in this case,
causing costs to be calculated as if all tokens were non-cached.
This fix subtracts cachedContentTokenCount from text_tokens when no
cacheTokensDetails is present (implicit caching), ensuring correct cost
calculation with the reduced cache_read pricing.
The lazy loading implementation for encoding in __getattr__ was calling
tiktoken.get_encoding() directly without first setting TIKTOKEN_CACHE_DIR.
This caused tiktoken to attempt downloading the encoding file from the
internet instead of using the local copy bundled with litellm.
This fix uses _get_default_encoding() from _lazy_imports which properly
sets TIKTOKEN_CACHE_DIR before loading tiktoken, ensuring the local cache
is used.
As indicated by https://docs.litellm.ai/docs/exception_mapping,
BadRequestError is used as the base type for multiple exceptions. As
such, it should be tested last in handling retry policies.
This updates the integration test that validates retry policies work as
expected.
Fixes#19876
- Add whitelist-based filtering for anthropic_beta headers
- Only allow Bedrock-supported beta flags (computer-use, tool-search, etc.)
- Filter out unsupported flags like mcp-servers, structured-outputs
- Remove output_format parameter from Bedrock Invoke requests
- Force tool-based structured outputs when response_format is used
Fixes#16726
Added test cases for custom model names containing slashes in Vertex AI
passthrough URLs (e.g., gcp/google/gemini-2.5-flash).
Test cases:
- gcp/google/gemini-2.5-flash
- gcp/google/gemini-3-flash-preview
- custom/model
This follow-up to PR #16862 allows users to specify models that should use
the native provider's background mode instead of polling via cache.
Config example:
litellm_settings:
responses:
background_mode:
polling_via_cache: ["openai"]
native_background_mode: ["o4-mini-deep-research"]
ttl: 3600
When a model is in native_background_mode list, should_use_polling_for_request
returns False, allowing the request to fall through to native provider handling.
Committed-By-Agent: cursor
Extend advanced-tool-use header translation to include Claude Sonnet 4.5
in addition to Opus 4.5 on Bedrock Invoke API.
When Claude Code sends the advanced-tool-use-2025-11-20 header, it now
gets correctly translated to Bedrock-specific headers for both:
- Claude Opus 4.5
- Claude Sonnet 4.5
Headers translated:
- tool-search-tool-2025-10-19
- tool-examples-2025-10-29
Fixes defer_loading validation error on Bedrock with Sonnet 4.5.
Ref: https://platform.claude.com/docs/en/agents-and-tools/tool-use/tool-search-tool
The error message for DISABLE_ADMIN_ENDPOINTS incorrectly said
"DISABLING LLM API ENDPOINTS is an Enterprise feature" instead of
"DISABLING ADMIN ENDPOINTS is an Enterprise feature".
This was a copy-paste bug from the is_llm_api_route_disabled() function.
Added regression tests to verify both error messages are correct.
* feat: add feature to make silent calls
* add test or silent feat
* add docs for silent feat
* fix lint issues and UI logs
* add docs of ab testing and deep copy
