OIDC providers like Logto may return opaque (non-JWT) access tokens,
which caused jwt.decode() to raise DecodeError and crash the SSO
callback with a 500 error. Catch DecodeError and skip JWT-based
extraction gracefully, since user info is already available from
the UserInfo endpoint.
Fixes#20724
* Add chat completion support for websearch
* Add chat completion tool calls support and response transformation
* Add new methods in chat completion
* Add chat completion tool format
* Add callback for websearch in completion method
* Add test for web search
* Potential fix for code scanning alert no. 4046: Clear-text logging of sensitive information
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
* Update litellm/integrations/websearch_interception/tools.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix: empty guardrails/policies arrays should not trigger enterprise license check (#20567)
* fix: empty guardrails/policies arrays should not trigger enterprise license check (#20304)
The UI sends empty arrays for enterprise-only fields (guardrails, policies,
logging) even when the user has not configured these features. The backend
`is not None` check treated `[]` as a truthy intent to use the feature,
falsely requiring an enterprise license for basic team operations.
Backend: Add `and updated_kv[field] != [] and updated_kv[field] != {}`
guards in `_update_metadata_fields` so empty collections are skipped.
UI: Conditionally omit guardrails, logging, and policies from the
payload when empty instead of defaulting to `[]`.
Fixes#20304
* fix: allow clearing fields with empty collections while skipping enterprise check
Address PR review feedback:
1. Move the empty-collection guard into _update_metadata_field (singular)
so that empty lists/dicts skip only the premium license check but still
get written into metadata. This lets users intentionally clear a
previously-set field (e.g. guardrails: []) without being blocked, while
the UI's default empty arrays still don't trigger a false enterprise
error.
2. Remove sys.path hack from test file; use standard imports that work
with pytest discovery.
3. Add tests verifying that empty collections are moved into metadata
(field clearing works) even though they bypass the premium check.
Fixes#20304
* fix critical CVE vulnerabliltes (#20683)
* fix: add hook to handle db case (#20635)
* Add team policy mapping for zguard (#20608)
* support policy mapping on team key level
* update document
* update document
* address comments
* update document
* add unit test for new feature
* add more test case
* feat: add support for anthropic_messages call type in prompt caching (#19233)
* feat: add support for anthropic_messages call type in prompt caching
* test: move anthropic_messages prompt caching test to main router test file
* add tutorial on using claude code with prompt cache routing
* docs: add SDK proxy authentication (OAuth2/JWT auto-refresh) documentation (#20680)
Adds documentation for the litellm.proxy_auth feature that automatically
obtains and refreshes OAuth2/JWT tokens when connecting to a LiteLLM Proxy.
* Fixes#20582 (#20663)
* fix: show error details instead of Data Not Available for failed requests (#20656)
* fix(ui): add null guard for models in API keys table (#20655)
The VirtualKeysTable crashed when rendering keys with null or undefined
models field. The className expression tried to access .length on null,
throwing a TypeError that broke the entire keys table.
Added Array.isArray() guard before accessing .length on the models value.
Fixes#20611
* Fix: Spend logs pickle error with Pydantic models and redaction (#20685)
* docs: add callback registration optimization to v1.81.9 release notes (#20681)
* docs: add callback registration optimization to v1.81.9 release notes
* Update v1.81.9.md
---------
Co-authored-by: Alexsander Hamir <alexsanderhamirgomesbaptista@gmail.com>
* Fix spend logs pickle error with Pydantic models
Replace copy.deepcopy() with Pydantic-safe serialization to avoid
"cannot pickle '_thread.RLock' object" errors when request/response
redaction is enabled.
Changes:
- Add _convert_to_json_serializable_dict() helper that uses
model_dump() for Pydantic models instead of pickle
- Replace copy.deepcopy() calls in request and response redaction
paths with the new helper function
- Recursively handles nested dicts, lists, and Pydantic models
Root cause: Pydantic v2 BaseModel instances contain internal
_thread.RLock objects for thread-safety. When copy.deepcopy()
attempts to pickle these objects, it fails because threading
primitives cannot be pickled.
Fixes#20647
* chore: remove unused copy import
Remove unused copy import that was causing lint failure. The copy.deepcopy()
calls were replaced with _convert_to_json_serializable_dict() helper function
in the previous commit, making the copy module no longer needed.
---------
Co-authored-by: ryan-crabbe <128659760+ryan-crabbe@users.noreply.github.com>
Co-authored-by: Alexsander Hamir <alexsanderhamirgomesbaptista@gmail.com>
* fix(vertex_ai): propagate extra_headers anthropic-beta to request body (#20666)
Vertex AI requires Anthropic beta flags in the request body
(anthropic_beta array), not as HTTP headers. The Bedrock handler
already extracts user-specified beta headers from the headers dict,
but the Vertex handler was missing this, causing extra_headers like
interleaved-thinking-2025-05-14 to be silently dropped.
This extracts anthropic-beta values from optional_params extra_headers
and merges them into the anthropic_beta request body field, and also
removes extra_headers from the request body since the parent's
transform_request spreads optional_params into data.
* fix(streaming): preserve interleaved thinking/redacted blocks
* test(streaming): build thinking chunks with typed Delta/StreamingChoices
* Fix video list pagination cursors not encoded with provider metadata
first_id and last_id in the video list response were returned as raw
provider IDs while data[].id was properly wrapped with
encode_video_id_with_provider(). This caused pagination to break when
clients passed unencoded cursors back as the `after` parameter.
- Encode first_id/last_id in transform_video_list_response
- Decode the `after` param in transform_video_list_request via
extract_original_video_id()
- Add 6 unit tests covering encoding, decoding, passthrough, and
full round-trip pagination
Fixes#20708
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(responses): preserve streamed tool deltas when id is omitted
* fix(responses): guard ambiguous tool-call index reuse
* Add compaction for vertex ai
* Add all new feat for v1/messages
* Add inference_geo as supported messages param
* Add inference based costing
* Add inference_geo as supported messages param
* Add support for fast param
* Add fast mode for other providers
* Add documentation for Fast Mode
* add missing indexes on VerificationToken table
* Fix structured response of tool call
* Add tests for WebSearch interception with chat completions API
* Add doc for chat completion web search
* Fix: is_web_search_tool_chat_completion
* Fix double json import
* Add new vercel ai anthropic models
* Fix: base_model name for body and deplyment name in URL
* Add output_config as supported param
* Add response schema for vercel ai sonnet 4.5
* handle when litellm_parrams might be none
* Fix : litellm/tests/test_litellm/llms/bedrock/chat/invoke_transformations/test_bedrock_chat_invoke_transformations_anthropic_claude3_transformation.py
* fix: Missing return statement for async streaming
* Fix: get_supported_anthropic_messages_params
* Fix mypy issues
* Fix mypy issues
* Add support for extra fields in Generic SSO via GENERIC_USER_EXTRA_ATTRIBUTES
Enables extraction of additional fields from the Generic SSO userinfo endpoint response beyond the standard 8 fields (id, email, name, etc.). Custom handlers can now access these fields via CustomOpenID.extra_fields dict.
Changes:
- Add extra_fields: Optional[Dict[str, Any]] to CustomOpenID type
- Add GENERIC_USER_EXTRA_ATTRIBUTES env var (comma-separated field names)
- Extract specified fields using get_nested_value() with dot notation support
- Add 4 test cases covering basic, nested, and missing field scenarios
- Update custom_sso.py example showing how to access extra_fields
Backward compatible: extra_fields is None when env var not set
* docs: Add documentation for GENERIC_USER_EXTRA_ATTRIBUTES
Document the new GENERIC_USER_EXTRA_ATTRIBUTES environment variable for Generic SSO
- Add to admin_ui_sso.md: explanation and usage examples
- Add to config_settings.md: environment variable reference
- Add to custom_sso.md: code example showing how to access extra_fields
- Includes examples for nested field paths with dot notation
---------
Co-authored-by: Sameer Kankute <sameer@berri.ai>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: Varun Chawla <34209028+veeceey@users.noreply.github.com>
Co-authored-by: Harshit Jain <48647625+Harshit28j@users.noreply.github.com>
Co-authored-by: jwang-gif <j.wang@zscaler.com>
Co-authored-by: nuernber <benjamin.nuernberger@jpl.nasa.gov>
Co-authored-by: Cesar Garcia <128240629+Chesars@users.noreply.github.com>
Co-authored-by: John Lathouwers <john.lathouwers@oracle.com>
Co-authored-by: ryan-crabbe <128659760+ryan-crabbe@users.noreply.github.com>
Co-authored-by: Alexsander Hamir <alexsanderhamirgomesbaptista@gmail.com>
Co-authored-by: Elias Högbom Aronsson <elias.aronson@gmail.com>
Co-authored-by: Emerson Gomes <emerson.gomes@thalesgroup.com>
Co-authored-by: tshushan <tshushan@outbrain.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Carlo Alberto Ferraris <cafxx@mercari.com>
When MCP SDK hits root-level /register, /authorize, /token without
server name prefix, auto-resolve to the single configured OAuth2
server. Also fix WWW-Authenticate header to use correct public URL
behind reverse proxy.
* fix(prometheus): sanitize label values to prevent metric scrape failures
Unicode characters like U+2028 (Line Separator) in Prometheus label values
break the text exposition format, causing scrapers (e.g. Datadog) to fail
parsing the entire /metrics endpoint. One bad label value causes ALL metrics
to be lost, not just the affected metric.
Add _sanitize_prometheus_label_value() and apply it in prometheus_label_factory()
and all direct .labels() call sites.
* fix(prometheus): handle non-string label values in sanitization
Coerce non-string values (int, bool, float) to str before applying
sanitization, preventing AttributeError on .replace() calls.
* fix(prometheus): run sanitization on coerced non-string values
Non-string values should be coerced to str and then sanitized (not
returned early), so their string representations also get cleaned.
* fix(prometheus): widen type hint to Optional[Any] for label value sanitization
first_id and last_id in the video list response were returned as raw
provider IDs while data[].id was properly wrapped with
encode_video_id_with_provider(). This caused pagination to break when
clients passed unencoded cursors back as the `after` parameter.
- Encode first_id/last_id in transform_video_list_response
- Decode the `after` param in transform_video_list_request via
extract_original_video_id()
- Add 6 unit tests covering encoding, decoding, passthrough, and
full round-trip pagination
Fixes#20708
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Vertex AI requires Anthropic beta flags in the request body
(anthropic_beta array), not as HTTP headers. The Bedrock handler
already extracts user-specified beta headers from the headers dict,
but the Vertex handler was missing this, causing extra_headers like
interleaved-thinking-2025-05-14 to be silently dropped.
This extracts anthropic-beta values from optional_params extra_headers
and merges them into the anthropic_beta request body field, and also
removes extra_headers from the request body since the parent's
transform_request spreads optional_params into data.
* feat: add support for anthropic_messages call type in prompt caching
* test: move anthropic_messages prompt caching test to main router test file
* add tutorial on using claude code with prompt cache routing
* fix: empty guardrails/policies arrays should not trigger enterprise license check (#20304)
The UI sends empty arrays for enterprise-only fields (guardrails, policies,
logging) even when the user has not configured these features. The backend
`is not None` check treated `[]` as a truthy intent to use the feature,
falsely requiring an enterprise license for basic team operations.
Backend: Add `and updated_kv[field] != [] and updated_kv[field] != {}`
guards in `_update_metadata_fields` so empty collections are skipped.
UI: Conditionally omit guardrails, logging, and policies from the
payload when empty instead of defaulting to `[]`.
Fixes#20304
* fix: allow clearing fields with empty collections while skipping enterprise check
Address PR review feedback:
1. Move the empty-collection guard into _update_metadata_field (singular)
so that empty lists/dicts skip only the premium license check but still
get written into metadata. This lets users intentionally clear a
previously-set field (e.g. guardrails: []) without being blocked, while
the UI's default empty arrays still don't trigger a false enterprise
error.
2. Remove sys.path hack from test file; use standard imports that work
with pytest discovery.
3. Add tests verifying that empty collections are moved into metadata
(field clearing works) even though they bypass the premium check.
Fixes#20304
The three loops in function_setup that called is_async_callable() on every
callback each request were redundant after the first request. Move the
async/sync routing into LoggingCallbackManager.add_litellm_*_callback()
so it happens once at registration time instead of on every request.
Fixes 15 failing tests in the MCP test suite:
1. **OAuth discoverable endpoints** (test_discoverable_endpoints.py):
- Added autouse fixture to mock IPAddressUtils.get_mcp_client_ip
- This bypasses IP-based access control which was blocking server lookup
- Fixes: test_authorize_*, test_token_*, test_oauth_*, test_register_*
2. **A2A endpoints** (test_a2a_endpoints.py):
- Fixed mock path for add_litellm_data_to_request
- Was patching litellm_pre_call_utils but function is called from common_request_processing
3. **MCP guardrail handler** (test_mcp_guardrail_handler.py):
- Updated tests to match new handler behavior
- Handler now passes tools (not texts) to guardrail
- Handler checks for mcp_tool_name (not messages array)
4. **MCP path-based segregation** (test_user_api_key_auth_mcp.py):
- Added client_ip to get_auth_context unpacking (7 values now)
- get_auth_context was updated to include client_ip
5. **MCP registry** (test_mcp_management_endpoints.py):
- Added mock for get_filtered_registry (not just get_registry)
- Registry endpoint uses get_filtered_registry for IP filtering
Co-authored-by: Shin <shin@openclaw.ai>
Add cheap .get() guards in should_run_callback() to short-circuit
the expensive EnterpriseCallbackControls.is_callback_disabled_dynamically()
call. When neither litellm_disabled_callbacks nor x-litellm-disable-callbacks
header is set (the common case), the enterprise function is never entered,
reducing should_run_callback from ~485ms to ~93-165ms across 54k calls.
CallTypes(call_type) was constructing an enum from string on every call,
taking ~4.6µs/call (69.6% of function time). Replace with a frozenset
membership test for ~0.8µs/call (8.3x faster).
* perf: Optimize get_litellm_params with sparse kwargs extraction
- Add _OPTIONAL_KWARGS_KEYS frozenset for O(1) lookups
- Replace 28 unconditional kwargs.get() calls with sparse extraction
- Only add kwargs keys that are actually present in the dict
- Simplify _get_base_model_from_litellm_call_metadata by removing redundant None checks
This reduces get_litellm_params() time by ~31% (743ms → 509ms across 6000 calls)
and Logging.__init__ total time by ~24% (1.61s → 1.23s).
* test: add unit tests for get_litellm_params sparse kwargs extraction
* perf: add early-exit guards in completion_cost for unused features
Skip function calls to get_cost_for_built_in_tools, _apply_cost_discount,
_apply_cost_margin, and _store_cost_breakdown_in_logging_obj when their
respective features are not configured. Reduces completion_cost() time
by ~20% (4.39s → 3.53s over 6K requests) for the common case where
built-in tools, discounts, margins, and logging object are not active.
* fix: always call get_cost_for_built_in_tools regardless of standard_built_in_tools_params
The function can detect web search usage from the usage object (e.g.
server_tool_use.web_search_requests, prompt_tokens_details.web_search_requests)
even when standard_built_in_tools_params is None, so guarding on it can
under-count cost for providers like Vertex AI and Anthropic.
Adds regression test for completion_cost with web search in usage but
no standard_built_in_tools_params.
* fix(tests): Mock async_container_create_handler for async router test
The test was mocking container_create_handler (sync), but
router.acreate_container uses _is_async=True which calls
async_container_create_handler. This caused the test to hit
the real OpenAI API.
Fixed by using AsyncMock on async_container_create_handler.
* fix(tests): Use uuid for unique model name in scientific notation test
The test was using a static "unique" model name which could cause
conflicts when running tests in parallel (-n 16 in CI). Using uuid
ensures truly unique names to prevent test pollution.
---------
Co-authored-by: Shin <shin@openclaw.ai>
