The regex in get_vertex_model_id_from_url() was using [^/:]+
which stopped at the first slash, truncating model names like
'gcp/google/gemini-2.5-flash' to just 'gcp'. This caused
access_groups checks to fail for custom model names.
Changed the pattern to [^:]+ to allow slashes in model names,
only stopping at the colon before the action (e.g., :generateContent).
* Cleanup code for user cli auth, and make sure not to prompt user for team multiple times while polling
* Adding tests
* Cleanup normalize teams some more
When using langfuse_otel callback, empty traces were being sent to Langfuse
for requests that didn't result in actual LLM calls (e.g., auth operations,
health checks, failed requests). These traces contained only internal proxy
operations (auth, postgres, proxy_pre_call) with no useful LLM data.
Root cause: LangfuseOtelLogger extends OpenTelemetry, which sets itself as
the proxy's open_telemetry_logger. This caused create_litellm_proxy_request_started_span
to be called for every request, creating a parent span that was sent to Langfuse
even when no LLM call occurred.
Fix: Override create_litellm_proxy_request_started_span in LangfuseOtelLogger
to return None, preventing the creation of empty parent spans. This is consistent
with the existing overrides for async_service_success_hook and async_service_failure_hook
which already prevent service-level logs from being sent to Langfuse.
Fixes: Empty traces in Langfuse v3 when using langfuse_otel callback
When Gemini uses implicit caching, it returns cachedContentTokenCount but
NOT cacheTokensDetails. Previously, text_tokens was not adjusted in this case,
causing costs to be calculated as if all tokens were non-cached.
This fix subtracts cachedContentTokenCount from text_tokens when no
cacheTokensDetails is present (implicit caching), ensuring correct cost
calculation with the reduced cache_read pricing.
The lazy loading implementation for encoding in __getattr__ was calling
tiktoken.get_encoding() directly without first setting TIKTOKEN_CACHE_DIR.
This caused tiktoken to attempt downloading the encoding file from the
internet instead of using the local copy bundled with litellm.
This fix uses _get_default_encoding() from _lazy_imports which properly
sets TIKTOKEN_CACHE_DIR before loading tiktoken, ensuring the local cache
is used.
As indicated by https://docs.litellm.ai/docs/exception_mapping,
BadRequestError is used as the base type for multiple exceptions. As
such, it should be tested last in handling retry policies.
This updates the integration test that validates retry policies work as
expected.
Fixes#19876
- Add whitelist-based filtering for anthropic_beta headers
- Only allow Bedrock-supported beta flags (computer-use, tool-search, etc.)
- Filter out unsupported flags like mcp-servers, structured-outputs
- Remove output_format parameter from Bedrock Invoke requests
- Force tool-based structured outputs when response_format is used
Fixes#16726
Added test cases for custom model names containing slashes in Vertex AI
passthrough URLs (e.g., gcp/google/gemini-2.5-flash).
Test cases:
- gcp/google/gemini-2.5-flash
- gcp/google/gemini-3-flash-preview
- custom/model
Extend advanced-tool-use header translation to include Claude Sonnet 4.5
in addition to Opus 4.5 on Bedrock Invoke API.
When Claude Code sends the advanced-tool-use-2025-11-20 header, it now
gets correctly translated to Bedrock-specific headers for both:
- Claude Opus 4.5
- Claude Sonnet 4.5
Headers translated:
- tool-search-tool-2025-10-19
- tool-examples-2025-10-29
Fixes defer_loading validation error on Bedrock with Sonnet 4.5.
Ref: https://platform.claude.com/docs/en/agents-and-tools/tool-use/tool-search-tool
The error message for DISABLE_ADMIN_ENDPOINTS incorrectly said
"DISABLING LLM API ENDPOINTS is an Enterprise feature" instead of
"DISABLING ADMIN ENDPOINTS is an Enterprise feature".
This was a copy-paste bug from the is_llm_api_route_disabled() function.
Added regression tests to verify both error messages are correct.
* feat: add feature to make silent calls
* add test or silent feat
* add docs for silent feat
* fix lint issues and UI logs
* add docs of ab testing and deep copy
