When Gemini uses implicit caching, it returns cachedContentTokenCount but
NOT cacheTokensDetails. Previously, text_tokens was not adjusted in this case,
causing costs to be calculated as if all tokens were non-cached.
This fix subtracts cachedContentTokenCount from text_tokens when no
cacheTokensDetails is present (implicit caching), ensuring correct cost
calculation with the reduced cache_read pricing.
The lazy loading implementation for encoding in __getattr__ was calling
tiktoken.get_encoding() directly without first setting TIKTOKEN_CACHE_DIR.
This caused tiktoken to attempt downloading the encoding file from the
internet instead of using the local copy bundled with litellm.
This fix uses _get_default_encoding() from _lazy_imports which properly
sets TIKTOKEN_CACHE_DIR before loading tiktoken, ensuring the local cache
is used.
As indicated by https://docs.litellm.ai/docs/exception_mapping,
BadRequestError is used as the base type for multiple exceptions. As
such, it should be tested last in handling retry policies.
This updates the integration test that validates retry policies work as
expected.
Fixes#19876
- Add whitelist-based filtering for anthropic_beta headers
- Only allow Bedrock-supported beta flags (computer-use, tool-search, etc.)
- Filter out unsupported flags like mcp-servers, structured-outputs
- Remove output_format parameter from Bedrock Invoke requests
- Force tool-based structured outputs when response_format is used
Fixes#16726
Added test cases for custom model names containing slashes in Vertex AI
passthrough URLs (e.g., gcp/google/gemini-2.5-flash).
Test cases:
- gcp/google/gemini-2.5-flash
- gcp/google/gemini-3-flash-preview
- custom/model
Extend advanced-tool-use header translation to include Claude Sonnet 4.5
in addition to Opus 4.5 on Bedrock Invoke API.
When Claude Code sends the advanced-tool-use-2025-11-20 header, it now
gets correctly translated to Bedrock-specific headers for both:
- Claude Opus 4.5
- Claude Sonnet 4.5
Headers translated:
- tool-search-tool-2025-10-19
- tool-examples-2025-10-29
Fixes defer_loading validation error on Bedrock with Sonnet 4.5.
Ref: https://platform.claude.com/docs/en/agents-and-tools/tool-use/tool-search-tool
The error message for DISABLE_ADMIN_ENDPOINTS incorrectly said
"DISABLING LLM API ENDPOINTS is an Enterprise feature" instead of
"DISABLING ADMIN ENDPOINTS is an Enterprise feature".
This was a copy-paste bug from the is_llm_api_route_disabled() function.
Added regression tests to verify both error messages are correct.
* feat: add feature to make silent calls
* add test or silent feat
* add docs for silent feat
* fix lint issues and UI logs
* add docs of ab testing and deep copy
Azure audio models were charging audio output tokens at the text token
rate instead of the correct audio token rate. This resulted in costs
being ~6.65x lower than expected.
The fix replaces Azure's custom cost calculation logic with the generic
cost calculator that properly handles text, audio, cached, reasoning,
and image tokens.
Fixes#19764
When using the responses API with provider-specific params (aws_*, vertex_*)
without explicitly passing custom_llm_provider, the code crashed with:
AttributeError: 'NoneType' object has no attribute 'startswith'
Root cause: local_vars was captured via locals() before get_llm_provider()
detected the provider from the model string (e.g., "bedrock/..."), so
custom_llm_provider remained None when processing provider-specific params.
Fix: Update local_vars["custom_llm_provider"] after get_llm_provider() call
so the detected provider is available for param processing.
Affected provider-specific params:
- aws_* (aws_region_name, aws_access_key_id, etc.) for Bedrock/SageMaker
- vertex_* (vertex_project, vertex_location, etc.) for Vertex AI
