OIDC providers like Logto may return opaque (non-JWT) access tokens,
which caused jwt.decode() to raise DecodeError and crash the SSO
callback with a 500 error. Catch DecodeError and skip JWT-based
extraction gracefully, since user info is already available from
the UserInfo endpoint.
Fixes#20724
* Add chat completion support for websearch
* Add chat completion tool calls support and response transformation
* Add new methods in chat completion
* Add chat completion tool format
* Add callback for websearch in completion method
* Add test for web search
* Potential fix for code scanning alert no. 4046: Clear-text logging of sensitive information
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
* Update litellm/integrations/websearch_interception/tools.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix: empty guardrails/policies arrays should not trigger enterprise license check (#20567)
* fix: empty guardrails/policies arrays should not trigger enterprise license check (#20304)
The UI sends empty arrays for enterprise-only fields (guardrails, policies,
logging) even when the user has not configured these features. The backend
`is not None` check treated `[]` as a truthy intent to use the feature,
falsely requiring an enterprise license for basic team operations.
Backend: Add `and updated_kv[field] != [] and updated_kv[field] != {}`
guards in `_update_metadata_fields` so empty collections are skipped.
UI: Conditionally omit guardrails, logging, and policies from the
payload when empty instead of defaulting to `[]`.
Fixes#20304
* fix: allow clearing fields with empty collections while skipping enterprise check
Address PR review feedback:
1. Move the empty-collection guard into _update_metadata_field (singular)
so that empty lists/dicts skip only the premium license check but still
get written into metadata. This lets users intentionally clear a
previously-set field (e.g. guardrails: []) without being blocked, while
the UI's default empty arrays still don't trigger a false enterprise
error.
2. Remove sys.path hack from test file; use standard imports that work
with pytest discovery.
3. Add tests verifying that empty collections are moved into metadata
(field clearing works) even though they bypass the premium check.
Fixes#20304
* fix critical CVE vulnerabliltes (#20683)
* fix: add hook to handle db case (#20635)
* Add team policy mapping for zguard (#20608)
* support policy mapping on team key level
* update document
* update document
* address comments
* update document
* add unit test for new feature
* add more test case
* feat: add support for anthropic_messages call type in prompt caching (#19233)
* feat: add support for anthropic_messages call type in prompt caching
* test: move anthropic_messages prompt caching test to main router test file
* add tutorial on using claude code with prompt cache routing
* docs: add SDK proxy authentication (OAuth2/JWT auto-refresh) documentation (#20680)
Adds documentation for the litellm.proxy_auth feature that automatically
obtains and refreshes OAuth2/JWT tokens when connecting to a LiteLLM Proxy.
* Fixes#20582 (#20663)
* fix: show error details instead of Data Not Available for failed requests (#20656)
* fix(ui): add null guard for models in API keys table (#20655)
The VirtualKeysTable crashed when rendering keys with null or undefined
models field. The className expression tried to access .length on null,
throwing a TypeError that broke the entire keys table.
Added Array.isArray() guard before accessing .length on the models value.
Fixes#20611
* Fix: Spend logs pickle error with Pydantic models and redaction (#20685)
* docs: add callback registration optimization to v1.81.9 release notes (#20681)
* docs: add callback registration optimization to v1.81.9 release notes
* Update v1.81.9.md
---------
Co-authored-by: Alexsander Hamir <alexsanderhamirgomesbaptista@gmail.com>
* Fix spend logs pickle error with Pydantic models
Replace copy.deepcopy() with Pydantic-safe serialization to avoid
"cannot pickle '_thread.RLock' object" errors when request/response
redaction is enabled.
Changes:
- Add _convert_to_json_serializable_dict() helper that uses
model_dump() for Pydantic models instead of pickle
- Replace copy.deepcopy() calls in request and response redaction
paths with the new helper function
- Recursively handles nested dicts, lists, and Pydantic models
Root cause: Pydantic v2 BaseModel instances contain internal
_thread.RLock objects for thread-safety. When copy.deepcopy()
attempts to pickle these objects, it fails because threading
primitives cannot be pickled.
Fixes#20647
* chore: remove unused copy import
Remove unused copy import that was causing lint failure. The copy.deepcopy()
calls were replaced with _convert_to_json_serializable_dict() helper function
in the previous commit, making the copy module no longer needed.
---------
Co-authored-by: ryan-crabbe <128659760+ryan-crabbe@users.noreply.github.com>
Co-authored-by: Alexsander Hamir <alexsanderhamirgomesbaptista@gmail.com>
* fix(vertex_ai): propagate extra_headers anthropic-beta to request body (#20666)
Vertex AI requires Anthropic beta flags in the request body
(anthropic_beta array), not as HTTP headers. The Bedrock handler
already extracts user-specified beta headers from the headers dict,
but the Vertex handler was missing this, causing extra_headers like
interleaved-thinking-2025-05-14 to be silently dropped.
This extracts anthropic-beta values from optional_params extra_headers
and merges them into the anthropic_beta request body field, and also
removes extra_headers from the request body since the parent's
transform_request spreads optional_params into data.
* fix(streaming): preserve interleaved thinking/redacted blocks
* test(streaming): build thinking chunks with typed Delta/StreamingChoices
* Fix video list pagination cursors not encoded with provider metadata
first_id and last_id in the video list response were returned as raw
provider IDs while data[].id was properly wrapped with
encode_video_id_with_provider(). This caused pagination to break when
clients passed unencoded cursors back as the `after` parameter.
- Encode first_id/last_id in transform_video_list_response
- Decode the `after` param in transform_video_list_request via
extract_original_video_id()
- Add 6 unit tests covering encoding, decoding, passthrough, and
full round-trip pagination
Fixes#20708
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(responses): preserve streamed tool deltas when id is omitted
* fix(responses): guard ambiguous tool-call index reuse
* Add compaction for vertex ai
* Add all new feat for v1/messages
* Add inference_geo as supported messages param
* Add inference based costing
* Add inference_geo as supported messages param
* Add support for fast param
* Add fast mode for other providers
* Add documentation for Fast Mode
* add missing indexes on VerificationToken table
* Fix structured response of tool call
* Add tests for WebSearch interception with chat completions API
* Add doc for chat completion web search
* Fix: is_web_search_tool_chat_completion
* Fix double json import
* Add new vercel ai anthropic models
* Fix: base_model name for body and deplyment name in URL
* Add output_config as supported param
* Add response schema for vercel ai sonnet 4.5
* handle when litellm_parrams might be none
* Fix : litellm/tests/test_litellm/llms/bedrock/chat/invoke_transformations/test_bedrock_chat_invoke_transformations_anthropic_claude3_transformation.py
* fix: Missing return statement for async streaming
* Fix: get_supported_anthropic_messages_params
* Fix mypy issues
* Fix mypy issues
* Add support for extra fields in Generic SSO via GENERIC_USER_EXTRA_ATTRIBUTES
Enables extraction of additional fields from the Generic SSO userinfo endpoint response beyond the standard 8 fields (id, email, name, etc.). Custom handlers can now access these fields via CustomOpenID.extra_fields dict.
Changes:
- Add extra_fields: Optional[Dict[str, Any]] to CustomOpenID type
- Add GENERIC_USER_EXTRA_ATTRIBUTES env var (comma-separated field names)
- Extract specified fields using get_nested_value() with dot notation support
- Add 4 test cases covering basic, nested, and missing field scenarios
- Update custom_sso.py example showing how to access extra_fields
Backward compatible: extra_fields is None when env var not set
* docs: Add documentation for GENERIC_USER_EXTRA_ATTRIBUTES
Document the new GENERIC_USER_EXTRA_ATTRIBUTES environment variable for Generic SSO
- Add to admin_ui_sso.md: explanation and usage examples
- Add to config_settings.md: environment variable reference
- Add to custom_sso.md: code example showing how to access extra_fields
- Includes examples for nested field paths with dot notation
---------
Co-authored-by: Sameer Kankute <sameer@berri.ai>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: Varun Chawla <34209028+veeceey@users.noreply.github.com>
Co-authored-by: Harshit Jain <48647625+Harshit28j@users.noreply.github.com>
Co-authored-by: jwang-gif <j.wang@zscaler.com>
Co-authored-by: nuernber <benjamin.nuernberger@jpl.nasa.gov>
Co-authored-by: Cesar Garcia <128240629+Chesars@users.noreply.github.com>
Co-authored-by: John Lathouwers <john.lathouwers@oracle.com>
Co-authored-by: ryan-crabbe <128659760+ryan-crabbe@users.noreply.github.com>
Co-authored-by: Alexsander Hamir <alexsanderhamirgomesbaptista@gmail.com>
Co-authored-by: Elias Högbom Aronsson <elias.aronson@gmail.com>
Co-authored-by: Emerson Gomes <emerson.gomes@thalesgroup.com>
Co-authored-by: tshushan <tshushan@outbrain.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Carlo Alberto Ferraris <cafxx@mercari.com>
When MCP SDK hits root-level /register, /authorize, /token without
server name prefix, auto-resolve to the single configured OAuth2
server. Also fix WWW-Authenticate header to use correct public URL
behind reverse proxy.
* fix: empty guardrails/policies arrays should not trigger enterprise license check (#20304)
The UI sends empty arrays for enterprise-only fields (guardrails, policies,
logging) even when the user has not configured these features. The backend
`is not None` check treated `[]` as a truthy intent to use the feature,
falsely requiring an enterprise license for basic team operations.
Backend: Add `and updated_kv[field] != [] and updated_kv[field] != {}`
guards in `_update_metadata_fields` so empty collections are skipped.
UI: Conditionally omit guardrails, logging, and policies from the
payload when empty instead of defaulting to `[]`.
Fixes#20304
* fix: allow clearing fields with empty collections while skipping enterprise check
Address PR review feedback:
1. Move the empty-collection guard into _update_metadata_field (singular)
so that empty lists/dicts skip only the premium license check but still
get written into metadata. This lets users intentionally clear a
previously-set field (e.g. guardrails: []) without being blocked, while
the UI's default empty arrays still don't trigger a false enterprise
error.
2. Remove sys.path hack from test file; use standard imports that work
with pytest discovery.
3. Add tests verifying that empty collections are moved into metadata
(field clearing works) even though they bypass the premium check.
Fixes#20304
Fixes 15 failing tests in the MCP test suite:
1. **OAuth discoverable endpoints** (test_discoverable_endpoints.py):
- Added autouse fixture to mock IPAddressUtils.get_mcp_client_ip
- This bypasses IP-based access control which was blocking server lookup
- Fixes: test_authorize_*, test_token_*, test_oauth_*, test_register_*
2. **A2A endpoints** (test_a2a_endpoints.py):
- Fixed mock path for add_litellm_data_to_request
- Was patching litellm_pre_call_utils but function is called from common_request_processing
3. **MCP guardrail handler** (test_mcp_guardrail_handler.py):
- Updated tests to match new handler behavior
- Handler now passes tools (not texts) to guardrail
- Handler checks for mcp_tool_name (not messages array)
4. **MCP path-based segregation** (test_user_api_key_auth_mcp.py):
- Added client_ip to get_auth_context unpacking (7 values now)
- get_auth_context was updated to include client_ip
5. **MCP registry** (test_mcp_management_endpoints.py):
- Added mock for get_filtered_registry (not just get_registry)
- Registry endpoint uses get_filtered_registry for IP filtering
Co-authored-by: Shin <shin@openclaw.ai>
Keycloak (and similar OIDC providers) include role claims in the JWT
access token but not in the UserInfo endpoint response. Previously,
roles were only extracted from UserInfo, causing all SSO users to
default to internal_user_view_only regardless of their actual role.
Changes:
- Extract user roles from JWT access token in process_sso_jwt_access_token()
when UserInfo doesn't provide them (tries role_mappings first, then
GENERIC_USER_ROLE_ATTRIBUTE)
- Handle list-type role values in get_litellm_user_role() since Keycloak
returns roles as arrays (e.g. ["proxy_admin"] instead of "proxy_admin")
- Add 9 new unit tests covering role extraction and list handling
- Update 3 existing tests for new JWT decode behavior
Closes#20407
- process_mcp_request() now falls back to OAuth2 passthrough when Authorization header contains a non-LiteLLM token (catches HTTPException and ProxyException 401/403)
- MCPClient._get_auth_headers() adds missing MCPAuth.oauth2 case
* Warn when budget lookup fails; cache won't populate
- Add _log_budget_lookup_failure helper in auth_checks.py
- Log at WARNING in get_user_object, get_team_object, get_key_object
when DB lookups fail (schema mismatch, etc.)
- Add schema migration hint for prisma/db errors
- Add dry-run test for _log_budget_lookup_failure
* fix: skip budget lookup failure log for expected user-not-found case
Avoid logging 'cache will not be populated' when the user simply doesn't
exist - not caching is correct behavior in that case. Only log for
unexpected errors (schema, DB, etc.) where the message is meaningful.
* fix(a2a): use text/event-stream SSE format for message/stream endpoint
The A2A gateway's streaming response was using application/x-ndjson
Content-Type and raw NDJSON body format. The A2A protocol spec requires
text/event-stream with SSE framing (data: ...\n\n).
The official a2a-sdk client validates the Content-Type header and raises
SSEError when it doesn't contain text/event-stream.
Changes:
- Changed media_type from application/x-ndjson to text/event-stream
- Updated response body to use SSE framing (data: prefix + \n\n suffix)
- Added tests validating Content-Type and SSE body format
Fixes#20278
* Potential fix for code scanning alert no. 4045: Information exposure through an exception
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
* test(a2a): add manual SSE format validation script
Adds a manual test script that:
1. Starts a real A2A agent on port 10001
2. Starts LiteLLM proxy with the agent registered
3. Makes a streaming request to the proxy's A2A gateway
4. Validates Content-Type header is text/event-stream
5. Validates body uses SSE framing (data: ...\n\n)
Run: python tests/a2a_manual/test_a2a_sse_manual.py
---------
Co-authored-by: shin-bot-litellm <shin-bot-litellm@users.noreply.github.com>
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
The unified guardrail's streaming iterator hook processes every Nth
chunk (sampling_rate, default 5). On each sampled chunk it calls
process_output_streaming_response, which combines all accumulated text
into the first chunk and clears all subsequent chunks to "".
The hook then yielded `processed_items[-1]` — the last item, whose
content had been cleared to "". This permanently lost every Nth
chunk's content, causing random missing words/tokens in the client
output (observed in Roo Code, Open WebUI, etc.).
Fix: deep-copy the current chunk before guardrail processing runs,
then yield the original (unmodified) chunk. The guardrail validation
still executes and can block if it detects a problem, but the stream
content is preserved.
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
