gemini/gemini-live-2.5-flash-preview-native-audio-09-2025 uses mode='realtime'
but the schema in test_aaamodel_prices_and_context_window_json_is_valid did
not include 'realtime' as a valid enum value, causing a ValidationError.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- a2a_protocol/main.py: replace bare assert with descriptive RuntimeError
in _execute_a2a_send_with_retry so retry exhaustion gives a clear message
- fine_tuning/main.py: fix _resolve_fine_tuning_timeout return type from
float to Union[float, httpx.Timeout] to accurately reflect the passthrough path
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add a new "Authentication Headers" panel to AgentFormFields:
- Static Headers: key-value Form.List (always sent to the backend agent,
static wins on conflict with dynamic)
- Forward Client Headers: Select[tags] of header names to extract from
the client request and forward (extra_headers)
Update buildAgentDataFromForm to serialize both fields for the API.
Update parseAgentForForm to deserialize them back for editing.
Covers both the create wizard (add_agent_form) and the edit view (agent_info).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
In invoke_agent_a2a:
- Extract admin-configured extra_headers from client request by name
- Extract convention-based headers (x-a2a-{agent_id/name}-{header}) from client request
- Merge with static_headers (static wins on conflict)
- Pass merged headers down to asend_message and _handle_stream_message
In asend_message / asend_message_streaming:
- Accept agent_extra_headers kwarg
- Overlay onto LiteLLM internal headers before creating the httpx client
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Mirrors merge_mcp_headers from the MCP server utils.
Dynamic headers come first; static (admin-configured) headers overlay and win on conflict.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Update add_agent_to_db, patch_agent_in_db, and update_agent_in_db to
read and write the two new header fields when creating or updating agents.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add two new fields to LiteLLM_AgentsTable:
- static_headers (Json): admin-configured headers always sent to the backend agent
- extra_headers (String[]): header names to extract from the client request and forward
Extend AgentConfig, PatchAgentRequest, and AgentResponse with the same fields.
Also remove duplicate spec_path field from LiteLLM_MCPServerTable.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Vertex AI does not support the output_config parameter in its API.
This parameter is being added by Anthropic/Gemini transformations but needs
to be removed before sending requests to Vertex AI endpoints.
This fix addresses the "Extra inputs are not permitted" error (issue #22312)
when using Claude models with structured outputs on Vertex AI.
Changes:
- Drop output_config in Gemini model transformation
- Drop output_config in Anthropic partner model transformation
- Drop output_config in Anthropic experimental pass-through transformation
- Add comprehensive tests to verify output_config is dropped
Fixes: #22312
Made-with: Cursor
Adds bedrock_mantle_models to the model_list union and models_by_provider
dict so models are discoverable via litellm.model_list and
litellm.models_by_provider["bedrock_mantle"].
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds bedrock_mantle_models to the model_list union and models_by_provider
dict so models are discoverable via litellm.model_list and
litellm.models_by_provider["bedrock_mantle"].
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Azure AI Foundry's Anthropic endpoint does not support the scope field in
cache_control. Strip it from both system and messages before sending.
Made-with: Cursor
Bedrock does not support the scope field in cache_control (e.g. 'global' for
cross-request caching). Only type and ttl are supported per AWS docs.
- Remove scope from cache_control in both system and messages
- Extend _remove_ttl_from_cache_control to process system blocks
- Add test for scope removal
Made-with: Cursor
* feat(mcp): BYOK (Bring Your Own Key) for OpenAPI MCP servers with OAuth 2.1 flow
Adds per-user credential storage for BYOK MCP servers so external clients
can authenticate via standard OAuth 2.1 PKCE without needing a full identity
provider.
Backend:
- New DB table LiteLLM_MCPUserCredentials (user_id, server_id, credential_b64)
- is_byok, byok_description, byok_api_key_help_url fields on MCPServerTable
- OAuth 2.1 authorization server endpoints (/.well-known/oauth-authorization-server,
/.well-known/oauth-protected-resource, /v1/mcp/oauth/authorize, /v1/mcp/oauth/token)
- 401 challenge with WWW-Authenticate header when BYOK server has no credential
- CRUD endpoints: POST/DELETE /v1/mcp/server/{id}/user-credential
- has_user_credential annotated on GET /v1/mcp/server response
UI:
- ByokCredentialModal: 2-step Connect flow (access description + API key entry)
- BYOK toggle + description fields on admin MCP server create form
- Connect/Connected state in MCP server table
- BYOK Demo page (/tools/byok-demo) showing full OAuth 2.1 PKCE flow
* feat(mcp/byok): redesign OAuth authorize page to match 2-step Connect mockup
- Step 1: L→S logos, requested access checklist, How it works box, Continue button
- Step 2: API key input, Save toggle, Duration pills (1h/24h/7d/30d/until_revoked), security note
- Matches screenshots: white modal on dark bg, progress dots, dark CTA buttons
- Authorize handler now fetches byok_description and byok_api_key_help_url from server registry
- CLAUDE.md: replace SQL snippet with proper DB migration troubleshooting guidance
* fix: address greptile review feedback (greploop iteration 1)
- XSS: escape all user-supplied values in _build_authorize_html() with html.escape()
- Open redirect: validate redirect_uri scheme and URL-encode code/state in redirect
- N+1 query: batch BYOK credential lookup into single find_many() call
- Critical path DB: add 60s TTL in-memory cache to _check_byok_credential()
- Encrypt BYOK credentials at rest using encrypt_value_helper/decrypt_value_helper
* fix(byok): update OAuth popup with LiteLLM logo, MCP title suffix, remove emojis
* fix(byok-demo): fix token endpoint URL (/v1/mcp/oauth/token not /v1/mcp/token)
* feat(byok): inject stored BYOK credential as mcp_auth_header on tool execution
* feat(byok): use contextvars to inject per-user credential into OpenAPI tool closures; remove byok-demo from LiteLLM UI
OpenAPI tools have auth headers baked into their closures at registration time. BYOK servers have
no static auth token, so per-user credentials were never reaching the HTTP calls.
Fix: add _request_auth_header ContextVar in openapi_to_mcp_generator.py. create_tool_function now
reads this var at call time and overrides the Authorization header if set. execute_mcp_tool resolves
the MCP server and performs BYOK checks before the local-tool dispatch branch, then sets the
ContextVar around _handle_local_mcp_tool so the credential flows into the HTTP request.
Also remove the /tools/byok-demo page from the LiteLLM UI dashboard — the demo lives at
~/Downloads/litellm-byok-demo/index.html (served separately on port 8080).
* fix: address greptile review feedback (greploop iteration 2)
- Cache invalidation: add _invalidate_byok_cred_cache() and call it after
store_user_credential() in both token endpoint and management endpoint
- Unbounded cache: add _BYOK_CRED_CACHE_MAX_SIZE=4096 with clear-on-overflow
- Unbounded auth codes: add _AUTH_CODES_MAX_SIZE=1000 with 503 on overflow
- Double DB query: merge _check_byok_credential + _get_byok_credential into
single _get_byok_credential call; raise 401 inline if None returned
- Sidebar: remove byok-demo entry (page was deleted in prior commit)
- JWT comment: document why byok_session HS256 token can't be used as proxy auth
* fix: address greptile review feedback (greploop iteration 3)
- auth_type: pre-format Authorization header (Bearer/ApiKey/Basic) in server.py
before setting ContextVar so openapi_to_mcp_generator respects server auth_type
- cache invalidation on delete: call _invalidate_byok_cred_cache after
delete_user_credential so stale True entries don't persist for 60s
- ContextVar guard: only set _request_auth_header when mcp_auth_header is set,
avoiding unnecessary ContextVar overhead on non-BYOK tool calls
* fix: address greptile review feedback (greploop iteration 4)
- Unified credential cache: store actual credential value (Optional[str])
instead of just bool so _get_byok_credential also benefits from caching —
eliminates the DB hit on every BYOK tool call within the 60s TTL window
- Extracted _write_byok_cred_cache() helper for consistent cache writes
- Replaced has_user_credential with get_user_credential in _check_byok_credential
so one DB call satisfies both existence check and value retrieval
- Remove false 'encrypted at rest' claim from OAuth HTML and ByokCredentialModal
* Update tests/test_litellm/proxy/_experimental/mcp_server/test_byok_oauth_endpoints.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* Update tests/test_litellm/proxy/_experimental/mcp_server/test_byok_oauth_endpoints.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Adds provider documentation for bedrock_mantle including:
- API key and region configuration
- Supported models with pricing table
- SDK, streaming, and async usage examples
- LiteLLM Proxy config and usage
- Added to Bedrock category in sidebar
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
filter_helpers.ts was not populating the Organization ID filter dropdown
(always empty). TeamVirtualKeysTable was showing the team's org for all
keys instead of each key's own org.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add proxy-admin-configurable toggles to restrict internal users (and optionally
team admins) from accessing agent and vector store management features.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The /key/list API returns `org_id` (the Pydantic field name), but the UI
was reading `organization_id`, causing the Organization field to always
show "Not Set" and the Organization ID filter to never match.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds a localStorage-based toggle to hide the bouncing 🌑 icon next to
the version tag in the navbar, following the same pattern used for
hiding prompts, usage indicator, new feature badges, and blog posts.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(gemini): ensure image token accumulation in usage metadata
Fixed an issue where image tokens were being overwritten instead of accumulated in Gemini responses. Added support for both camelCase and snake_case token count keys. Fixes#22082.
* test: add regression test for image token accumulation and cleanup files
* fix(gemini): ensure consistent accumulation for responseTokensDetails
* fix(gemini): harden token count parsing and add vertex accumulation test
Parse tokenCount/token_count as int-safe values to satisfy mypy and avoid None/object arithmetic. Add regression test for duplicate modality accumulation in Vertex _calculate_usage.
mask was missing from get_supported_openai_params, causing it to be
dropped before reaching transform_image_edit_request where it is
already handled correctly for flux-pro-1.0-fill inpainting.
