The purge takes an injectable invalidate_token_cache callable defaulting to the manager's shared
invalidation, and MCPServerManager takes an injectable per_user_token_cache alongside the existing
per_user_oauth_token_store, so tests inject fakes instead of monkeypatching the global manager and
the module-level cache. The new identity helpers drop Any for object throughout
Review follow-ups on the stale-token invalidation. The backend identity now decrypts client_id and
client_secret before comparing: the stored values are NaCl-encrypted with a fresh nonce on every
write, so comparing ciphertext flagged every routine save as a mint-relevant change and purged
per-user tokens that were still valid. The identity also gains spec_path, the audience for OpenAPI
servers, and parses credentials stored as a JSON string
The purge now routes each (user, server) through the manager's invalidate_user_oauth_token_cache,
which becomes the single invalidation point covering both the legacy per-user token cache and the
v2 per-user OAuth token store; previously the purge evicted only the legacy cache while the revoke
path evicted only the v2 store, so each path left the other cache serving a replaced token until
its TTL. A credential row racing in between the find and the delete is now detected via the
delete_many count and logged; its cache entry expires by TTL
On the dashboard, CLEARED_ON_INVALIDATION and the staleness check move to types.tsx as the single
shared implementation for both forms. The edit form's transport handler now rechecks the identity
after its programmatic setFieldsValue calls, which antd does not report through onValuesChange, so
a token no longer survives a transport switch that clears the mint target. The create form rebuilds
formValues from the post-reset form state after an invalidation instead of publishing the pre-reset
snapshot, so the tool preview can no longer refetch with the discarded DCR client. Both transport
handlers now share the recheck, which also stops the create form from over-invalidating on an
http to sse swap that keeps the same url and therefore the same audience
An admin who ran Authorize & Fetch and then changed a field that determines which upstream OAuth
token gets minted kept using the stale token for tool preview, sessionStorage, and (on the backend)
the stored per-user credential and its cache. Grounded in RFC 8707/8693 and the MCP auth spec, a token
is bound to one tuple: resource/audience (url), OAuth mode/grant (auth_type, oauth_flow_type), the
authorization-server endpoints, and the OAuth client + scopes. A shared getOAuthAuthorizationIdentity
captures exactly those fields; transport (http/sse on the same url is the same audience) and
delegate_auth_to_upstream (a downstream-usage toggle never sent to the authorize request) are excluded.
UI: both the create and edit forms now discard the held token (React state / sessionStorage / hook,
plus the fetched token + DCR client in form.credentials) whenever the identity diverges from the one it
was authorized against, re-applying the admin's in-flight edit so it is never wiped. The check lives in
one shared helper so the two forms cannot drift.
Backend: editing an MCP server now compares the pre/post identity and, on a mint-relevant change, purges
every stored per-user OAuth credential for the server (DB row + per-user token cache) so no user
forwards a token minted for a resource/AS/client that no longer matches. Best-effort; a purge failure
never fails the update.
The gateway authorize/token/register gate and the preview header extraction each carried their own
inline copy of the oauth2 + client-forwarded mode set, which could drift from the discovery
constant the registry builders use; all three surfaces mean the same thing (modes that run the
upstream OAuth browser flow), so they now read the one constant
The redacted resource kept the path, but hosted MCP servers routinely embed the credential in the
path (for example /mcp/s/<token>/mcp), and mcp_tool_call_metadata is readable by a caller who can
invoke the tool, so the path leaked the upstream credential into spend logs. Only scheme, host, and
port are logged now
Authorization doubles as the admission fallback when x-litellm-api-key is absent, so a caller who
authenticated the preview request that way had their LiteLLM key forwarded to the upstream as the
oauth2/client-forwarded token. The preview now forwards Authorization only when the primary
admission header is present, which is how the dashboard has always sent it; with no primary header
there is no upstream token on the request at all. Applies to oauth2 and both client-forwarded
modes; parametrized regression test plus the admission header added to the existing extraction
tests to mirror the real UI request shape
* fix(bedrock-invoke): retain clear_tool_uses_20250919 context_management edits and emit context-management-2025-06-27 beta (LIT-3393)
Copy of #29206 by oss-agent-shin, rebased onto litellm_internal_staging so CircleCI can run.
Bedrock InvokeModel supports automatic tool-call clearing (clear_tool_uses_20250919) under the context-management-2025-06-27 beta, but LiteLLM stripped the edit and dropped the beta header, causing a Bedrock 400. This maps bedrock.context-management-2025-06-27 to itself in anthropic_beta_headers_config.json (bedrock_converse stays null) and rewrites _filter_context_management_for_bedrock_invoke around an allowlist of supported edit types that keeps each supported edit and adds its matching beta.
* test(bedrock-invoke): restore beta-headers config cache with a shared fixture in LIT-3393 tests
Greptile flagged that three of the four new tests reloaded the module-level
beta-headers config into local mode without restoring it on teardown, leaking
state into later tests in the same process. Move setup/teardown into a
local_beta_headers_config fixture used by all four tests.
---------
Co-authored-by: oss-agent-shin <ext-agent-shin@berri.ai>
* fix: rust ocr tests finally pass
* fix: move realtime dir
* fix(realtime): normalize azure realtime api_base to host for Foundry endpoints
The azure realtime handler appended the realtime path to api_base verbatim, so a
Foundry base carrying a project path (.../api/projects/<name>) produced an invalid
realtime URL and the websocket handshake hung. Normalize api_base to scheme and host
before building the realtime path so both Azure OpenAI and Foundry bases connect
Point the e2e realtime azure deployment at the GA gpt-realtime model and stop passing
the os.environ refs the realtime path never unwraps, resolving them from the gateway
env by name instead. Drop the local docker-compose scaffolding from the tree
* test(e2e): add Gateway.list_files and list_fine_tuning_jobs for the discovery suite
The discovery endpoints suite calls client.gateway.list_files and
list_fine_tuning_jobs, which did not exist on Gateway, so both tests errored with
AttributeError before reaching the proxy. Add the two GET wrappers using the
existing FileListResponse / FineTuningJobsResponse models
* revert(realtime): drop azure realtime api_base host-normalization
The azure realtime handshake failure was a config issue, not a litellm bug: the
realtime base was set to the Azure AI Foundry project endpoint (.../api/projects/<p>),
but the OpenAI-compatible realtime route lives at the resource root. litellm correctly
appends the realtime path to whatever base it is given, so pointing the realtime
deployment at the resource root is the fix and no core change is needed
* fix(ocr): route azure_ai doc-intelligence to its own endpoint at the source
get_llm_provider inherits AZURE_AI_API_BASE into api_base for every azure_ai/* OCR
model, but Azure Document Intelligence is a separate resource reached via
AZURE_DOCUMENT_INTELLIGENCE_ENDPOINT, so doc-intelligence requests went to the wrong
host. Stop inheriting the azure_ai base for doc-intelligence models so api_base stays
unset and both the rust bridge and the python get_complete_url fall back to the
document-intelligence endpoint. This drops the earlier _rust_bridge_api_base reorder,
which only covered the rust path and let the env silently override an explicit api_base
* refactor(ocr): consolidate azure doc-intelligence detection; keep explicit api_base
Extract is_azure_document_intelligence_model as the single source of truth for the azure_ai doc-intelligence sub-route so the check is no longer duplicated across _prepare_ocr_request and _rust_bridge_api_base, and gate the dynamic_api_base suppression on the caller not supplying an api_base so an explicit endpoint is always honoured. Restore xai to the realtime PROVIDERS as a documented disabled entry instead of dropping it silently, and add a regression test pinning doc-intelligence api_base resolution.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Mubashir Osmani <mubashir@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(guardrails/bedrock): honor disable_exception_on_block by raising ModifyResponseException
The Bedrock-specific GuardrailInterventionNormalStringError predates the
unified guardrails refactor and no proxy code path handles it, so a block
with the flag set surfaced as an uncaught Exception -> HTTP 500 in pre_call
mode and was silently discarded in during_call mode (model call proceeded
in the parallel asyncio.gather; the block hook's data["mock_response"]
mutation happened after route_request had already unpacked kwargs).
Convert the block to ModifyResponseException at the raise site inside
make_bedrock_api_request. That exception is the industry-standard proxy
contract already caught in proxy_server, anthropic_endpoints, response_api
_endpoints, and pass_through_endpoints; it turns into a 200 response with
finish_reason=content_filter and the block message as content, which is
exactly what the flag was documented to yield. Post-call blocks attach
the LLM response to original_response so the synthetic reply reports the
upstream call's real token usage instead of zero.
Deletes the now-orphaned GuardrailInterventionNormalStringError class and
the dead create_guardrail_blocked_response / mock_response plumbing in the
Bedrock hooks; updates the existing tests that had locked in the buggy
contract.
Resolves LIT-4186
* chore(guardrails/bedrock): drop dead str branch in _update_messages_with_updated_bedrock_guardrail_response
Follow-up to the disable_exception_on_block fix. That method used to
receive either a BedrockGuardrailResponse or a plain string (the block
message, when the flag was set). Now that a block always raises
ModifyResponseException before this method runs, the string branch is
unreachable; tighten the type to BedrockGuardrailResponse and delete
the guard.
* fix(guardrails/bedrock): streaming post_call block yields synthetic stream instead of surfacing as SSE 500
Regression from the LIT-4186 refactor: pre-refactor, the streaming
post_call iterator caught GuardrailInterventionNormalStringError locally
and replaced the assembled response with a synthetic content-filter
message, then re-emitted it as chunks via MockResponseIterator. After
the refactor the exception was re-raised as ModifyResponseException,
which async_streaming_data_generator serializes as a proxy 500 error
frame because the SSE response headers are already flushed by the time
the block fires.
Non-streaming paths still let ModifyResponseException propagate to the
endpoint handler (which converts it into a 200). Streaming can't do
that, so keep the local synthesis: on the exception, rebind the
assembled response to a ModelResponse whose single choice carries the
block message as content and finish_reason=content_filter, and let the
downstream MockResponseIterator emit it as chunks. Same shape a
non-streaming block produces.
Adds a mapped-file regression test that mutation-kills the raise
behavior and locks in the synthetic-stream contract.
* fix(guardrails/bedrock): preserve upstream usage on streaming post_call block
Non-streaming post_call blocks report the upstream LLM call's real
token usage via ModifyResponseException.original_response, which the
endpoint handler unwraps through _blocked_response_usage. Streaming
post_call synthesizes its own ModelResponse locally (the exception
can't escape the SSE generator), and previously left .usage unset,
so the client saw accurate billing on non-streaming blocks and zero
on streaming blocks -- silent revenue leak.
Copy the assembled response's .usage onto the synthetic block
response before yielding. Pre-refactor code had the same gap
(create_guardrail_blocked_response never set usage); this is a net
improvement, not a regression fix.
* fix(proxy): capture logging_obj before post_call_failure_hook pops it in ModifyResponseException streaming path
post_call_failure_hook removes litellm_logging_obj from request_data before
iterating callbacks (it's not serialisable). The streaming branch of the
ModifyResponseException handler read it from _data after that call, so it
always received None and CustomStreamWrapper.__init__ crashed with
AttributeError: NoneType has no attribute model_call_details.
Capture it before the hook runs so the streaming path gets a valid object.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
* test(proxy): add regression for streaming ModifyResponseException logging_obj capture
Covers the bug where logging_obj was read from request_data after
post_call_failure_hook had already popped it, causing CustomStreamWrapper
to crash with AttributeError.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
* test(proxy): drive real chat_completion in ModifyResponseException streaming logging_obj regression
The original test inlined the fix pattern (capture before pop) in its
own body rather than calling the actual chat_completion handler in
proxy_server.py, so a revert of the fix left the test passing.
Confirmed via mutation check: reverting the two-line source fix and
re-running left the test green.
Rewrite the test to drive chat_completion directly:
- patch _read_request_body so chat_completion sees the seeded dict
- patch ProxyBaseLLMRequestProcessing.base_process_llm_request to
raise ModifyResponseException with the same request_data
- patch proxy_logging_obj so post_call_failure_hook mutates the dict
the way production does (pops litellm_logging_obj)
- intercept CustomStreamWrapper.__init__ and assert logging_obj is
the non-None object seeded in request_data
Mutation-verified: reverting the source fix now surfaces the exact
production crash inside CustomStreamWrapper's __init__
(AttributeError: NoneType has no attribute model_call_details) rather
than a silently-passing test.
Addresses Greptile P1 on PR #32665.
---------
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
create_model now waits until the new deployment is servable on the data plane
(polls /v1/models) before returning, instead of assuming /model/new makes it
instantly callable. On a split control/data-plane proxy the gateway only sees a
model after its next DB reload, so an immediate call raced the reload and 400'd
with "Invalid model name passed" (embeddings, responses, messages, ocr, ...).
It also stops pinning model_info.id to the model_name, letting the proxy assign a
unique model_id. Re-registering a fixed-name deployment (the batch suite's
openai-batch et al.) after a failed teardown no longer collides on the model_id
unique constraint (prisma UniqueViolationError surfaced as the generic 500
"Failed to add model to db", erroring every batch_lifecycle case at setup)
The edit form decided auth mode from the saved mcpServer.auth_type in fetchTools while the
authorize flow used the current form value, so a token authorized after switching the form to a
client-forwarded mode was never forwarded as the x-mcp header until the server was saved. A shared
getEffectiveAuthType (form value falling back to the saved record) is now the single decision point
for token receipt and tool loading
The save path classified the staged token with getMcpOAuthMode, which returns null for
true_passthrough and oauth_delegate, so the staged token was dropped on save instead of being
committed to sessionStorage the way the create form's submit path does. The passthrough branch now
also covers the client-forwarded modes; the token still never enters the server row
* fix(guardrails/bedrock): honor disable_exception_on_block by raising ModifyResponseException
The Bedrock-specific GuardrailInterventionNormalStringError predates the
unified guardrails refactor and no proxy code path handles it, so a block
with the flag set surfaced as an uncaught Exception -> HTTP 500 in pre_call
mode and was silently discarded in during_call mode (model call proceeded
in the parallel asyncio.gather; the block hook's data["mock_response"]
mutation happened after route_request had already unpacked kwargs).
Convert the block to ModifyResponseException at the raise site inside
make_bedrock_api_request. That exception is the industry-standard proxy
contract already caught in proxy_server, anthropic_endpoints, response_api
_endpoints, and pass_through_endpoints; it turns into a 200 response with
finish_reason=content_filter and the block message as content, which is
exactly what the flag was documented to yield. Post-call blocks attach
the LLM response to original_response so the synthetic reply reports the
upstream call's real token usage instead of zero.
Deletes the now-orphaned GuardrailInterventionNormalStringError class and
the dead create_guardrail_blocked_response / mock_response plumbing in the
Bedrock hooks; updates the existing tests that had locked in the buggy
contract.
Resolves LIT-4186
* chore(guardrails/bedrock): drop dead str branch in _update_messages_with_updated_bedrock_guardrail_response
Follow-up to the disable_exception_on_block fix. That method used to
receive either a BedrockGuardrailResponse or a plain string (the block
message, when the flag was set). Now that a block always raises
ModifyResponseException before this method runs, the string branch is
unreachable; tighten the type to BedrockGuardrailResponse and delete
the guard.
* fix(guardrails/bedrock): streaming post_call block yields synthetic stream instead of surfacing as SSE 500
Regression from the LIT-4186 refactor: pre-refactor, the streaming
post_call iterator caught GuardrailInterventionNormalStringError locally
and replaced the assembled response with a synthetic content-filter
message, then re-emitted it as chunks via MockResponseIterator. After
the refactor the exception was re-raised as ModifyResponseException,
which async_streaming_data_generator serializes as a proxy 500 error
frame because the SSE response headers are already flushed by the time
the block fires.
Non-streaming paths still let ModifyResponseException propagate to the
endpoint handler (which converts it into a 200). Streaming can't do
that, so keep the local synthesis: on the exception, rebind the
assembled response to a ModelResponse whose single choice carries the
block message as content and finish_reason=content_filter, and let the
downstream MockResponseIterator emit it as chunks. Same shape a
non-streaming block produces.
Adds a mapped-file regression test that mutation-kills the raise
behavior and locks in the synthetic-stream contract.
* fix(guardrails/bedrock): preserve upstream usage on streaming post_call block
Non-streaming post_call blocks report the upstream LLM call's real
token usage via ModifyResponseException.original_response, which the
endpoint handler unwraps through _blocked_response_usage. Streaming
post_call synthesizes its own ModelResponse locally (the exception
can't escape the SSE generator), and previously left .usage unset,
so the client saw accurate billing on non-streaming blocks and zero
on streaming blocks -- silent revenue leak.
Copy the assembled response's .usage onto the synthetic block
response before yielding. Pre-refactor code had the same gap
(create_guardrail_blocked_response never set usage); this is a net
improvement, not a regression fix.
The App Router migration is complete: every page is a path route and the
legacy `?page=` switch is gone from the index. This closes it out.
The `/ui/` index (page.tsx) kept its own duplicate copy of teams state, a
teams fetch, and keys/addKey plumbing solely to feed a second `UserDashboard`
render for the `invitation_id` case. That was redundant: `ApiKeysDashboard`
already renders `UserDashboard` sourcing its own data, so the index is thinned
to just render `<ApiKeysDashboard />`. The login redirect, the legacy `?page=`
deep-link redirect for old bookmarks, and the post-login return-URL handling
stay on the index.
The invitation entry point now resolves in one place. Modern invitation links
already point at the dedicated `/onboarding` route; the dashboard layout now
redirects legacy `/ui/?invitation_id=` links there too (via `migratedHref`,
the same base-aware redirect the index uses for `?page=`), instead of
re-rendering that route's page component inline. This removes an import of one
route's `page.tsx` into another module, and lets the now-unreachable
`if (invitation_id) return <Onboarding/>` branch in the shared
`user_dashboard.tsx` be deleted along with its dead `Onboarding` import and
`searchParams` read. A layout test asserts the redirect and fails if it
regresses.
`legacyPageHref` and the sidebar's migrated-vs-legacy href fallback are left
in place; they are still live for the parent-category nav nodes (agentic,
tools, experimental, settings) that are not page routes.
eslint-metrics.json is resynced: -2 no-explicit-any from the removed `any`
casts, plus pre-existing drift the gate requires the snapshot to match.
* fix(ui): forward refs through ui primitives and fail tests on swallowed refs
Under React 18 a ref passed to a plain function component is dropped
with only a dev console warning, so Base UI render-prop triggers
composed over our shadcn-style primitives silently stop working (the
tooltip just never opens; ui/badge.tsx hit exactly this on the shared
DataTable branch). Label, Separator, Skeleton, UiLoadingSpinner and the
Table family now use React.forwardRef like Button and Input already
did, a contract test pins ref delivery for each, and setupTests turns
React's ref warning into a test failure so the next primitive that
swallows a ref fails CI instead of shipping a dead tooltip
* fix(ui): include captured ref warnings in the tripwire error
The afterEach tripwire threw a fixed message and discarded the collected
React warnings, so a failure never said which component swallowed the ref.
Append the captured warnings (component name + stack) to the thrown error.
The expanded reasoning block did not constrain its width or break long unbreakable tokens, so its inline-block bubble grew past its max width and pushed the whole page wider (#32481). Mirror the message body handling by capping the container width and breaking long words/code.
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The create/edit forms passed several large object literals inline as arguments (persist-state
JSON.stringify, storeMCPOAuthUserCredential, setToken, transport-clear setFieldsValue), tripping
local/no-large-inline-object-arg. Assigned each to a named variable at the call site - a pure,
behavior-preserving refactor verified by the create/edit suites - which lowers the whole-tree count so
the eslint baseline is 512 rather than being raised to accommodate them.
The create form wrote the upstream token obtained by Authorize & Fetch into
form.credentials for every mode, so for true_passthrough / oauth_delegate the
browser-held token leaked into the OAuth flow's getCredentials (preview requests)
and the redirect-persist cache, and was a step away from server-level credential
persistence. onTokenReceived now early-returns for the client-forwarded modes,
holding the token only in local state for preview (mirroring the edit form),
instead of writing it into form.credentials.
The config-YAML loader and the DB loader each defined their own local tuple
(oauth2, true_passthrough, oauth_delegate) to decide which auth types trigger
upstream OAuth endpoint discovery, under two different names. Hoisted them to a
single module constant _UPSTREAM_OAUTH_DISCOVERY_AUTH_TYPES so the two load paths
cannot drift on which modes get discovery.
The 'auth_type is true_passthrough or oauth_delegate' check was duplicated inline
across both server forms' browser-authorize temp payloads, the edit form's
onTokenReceived and tool-preview gate, PassthroughAuthorizeSection, and mcp_tools'
usesBrowserHeldToken. Extracted a single isClientForwardedTokenMode helper in
types.tsx and routed every site through it so the set of client-forwarded modes
lives in one place and cannot drift. Also replaced a pre-existing nested ternary
in the authorize button label surfaced by touching the file.
The edit form's getTemporaryPayload read the server's stored auth_type instead
of the value the admin selected in the dropdown, so an admin who switched an
existing oauth2 server to true_passthrough (or oauth_delegate) and ran the
browser authorize flow built the temporary OAuth-relay server as oauth2. That
made needs_user_oauth_token true and persisted the token to the DB, contrary to
the mode's browser-held contract, and left it inconsistent with onTokenReceived
and the submit payload, both of which already read the form value. It now reads
values.auth_type, matching the create form.
Two review findings on the passthrough modes.
The tool-call log records the upstream MCP server URL as mcp_server_resource,
which is persisted in spend-log metadata and sent to logging callbacks. A URL
carrying embedded userinfo or a secret query parameter would leak into logs, so
the value is now redacted to its bare resource identifier (scheme + host + path);
userinfo, query string, and fragment are stripped before it is logged.
The listing fan-out withholds the request-wide Authorization from a
true_passthrough / oauth_delegate server when another server in scope also
consumes it, so one bearer is not replayed across upstreams. The later
server.extra_headers copy loop did not honor that decision: a server listing
Authorization in extra_headers would re-copy the withheld bearer from raw_headers.
The withhold decision is now computed once and applied to both the forwarding
branch and the extra_headers loop.
Adds mcp_auth_mode and mcp_server_resource to StandardLoggingMCPToolCall so a
relayed passthrough/delegate request can be attributed in an audit to its mode
and its upstream target without logging any credential. Both are non-sensitive
metadata derived from the resolved server; the admission and upstream tokens
stay SecretStr and are never logged.
Two correctness fixes for the client-forwarded token modes.
The preemptive-401 connect gate for true_passthrough and oauth_delegate
only inspected the request-wide Authorization, so a caller who bound the
upstream token via the per-server x-mcp-{alias}-authorization header (the
mandatory shape in a multi-server aggregate, where the request-wide
Authorization is withheld) was spuriously 401'd at connect even though
egress already honors that header. The gate now recognizes the per-server
header for both modes via a shared helper, mode-correctly: true_passthrough
treats any Authorization or the per-server header as the upstream token,
oauth_delegate keeps requiring a distinct x-litellm-api-key so a lone
Authorization consumed for admission is never mistaken for an upstream
token. The preemptive raise is also gated to single-server scopes so a
multi-server aggregate degrades gracefully (the listing absorbs a
per-server failure) instead of one missing token 401-ing the whole connect.
The browser-only Authorize flow was writing the upstream access and refresh
token to LiteLLM_MCPUserCredentials, contradicting the modes' persist-nothing
contract: the temp OAuth-relay server was cached with a hardcoded oauth2
auth_type, so needs_user_oauth_token was true and the token exchange stored
it. The create and edit forms now send the real auth_type for these modes,
so the temp server is not oauth2, needs_user_oauth_token is false, and the
exchange skips storage while still returning the token to the browser
session.
The DB build already discovers authorization/token endpoints for
true_passthrough and oauth_delegate rows; the config.yaml load path
kept the oauth2-only gate, so a YAML-defined server in either mode
could not use the relayed authorize flow unless the YAML declared
authorization_url. Both paths now share the same auth type set.
The server detail page's Tool Testing Playground gated its browser-held
token handling on the legacy PKCE-passthrough shape, so a
true_passthrough or oauth_delegate server listed tools unauthenticated
and surfaced 'Failed to fetch MCP tools' with no way to authorize. The
playground now treats both modes as browser-held-token servers: it
reads the sessionStorage token established by the create/edit
browser-only Authorize, forwards it via the x-mcp-{alias}-authorization
header, evicts it on a 401, and shows its own Authorize gate when the
token is absent.
That gate's flow uses the gateway's relayed authorize/register/token
endpoints with the real server id, which previously 400ed for anything
but oauth2. Those endpoints now also accept the client-forwarded token
modes (the minted token is upstream-audienced and browser-held; DCR
persistence stays off on this path), and registry builds run the same
RFC 9728/8414 endpoint discovery for these modes that oauth2 rows get,
since their rows never store an authorization_url.
true_passthrough and oauth_delegate persist no upstream credentials, so
the create/edit forms had no way to preview tools or configure the tool
allowlist: tools/list went upstream unauthenticated and came back 401.
This reuses the existing OAuth authorize machinery in browser-only mode
for those two auth types: the admin authorizes against the upstream
(DCR/PKCE, with optional client credentials for IdPs without dynamic
registration), the token lands in sessionStorage exactly like the
legacy PKCE-passthrough path, and the tools preview forwards it via the
per-server x-mcp-{alias}-authorization header, which the passthrough
resolver arm already accepts. Nothing is written to the server row or
the per-user credential store; the create payload keeps excluding
credentials for these auth types via AUTH_TYPES_REQUIRING_CREDENTIALS.
The tools preview endpoint now also extracts the Authorization header
for the two new auth types so the browser-held token reaches the
passthrough arm during create-time previews.
Adds the two client-forwarded token modes to the MCP server create and
edit form auth dropdowns, and shows a warning when true_passthrough is
selected: the gateway performs no admission auth for that server, so
callers reach the upstream without a LiteLLM key and per-key/per-team
rate limits and spend tracking do not apply. The warning is a shared
component so the two forms cannot drift on the copy.
When an IdP denies SSO access it redirects back to /sso/callback with
error and error_description query params and no code param. The callback
previously fell through to the provider token exchange, which failed
with a generic "'code' parameter was not found in callback request"
400 that hides the real denial reason. Raise a 401 that surfaces the
IdP's error and description instead.
Ported from #26640 with conflicts resolved against current staging
The v2 OTel integration stamped litellm-specific error details as
error.code, error.stack_trace, and error.llm_provider, squatting on the
semconv-owned error.* namespace. They now live at
litellm.provider.error.code, litellm.provider.error.stack_trace, and
litellm.provider.error.llm_provider alongside the other vendor-extension
keys. error.type and error.message stay on the semconv keys.
* fix(guardrails): walk Responses-API text taxonomy in shared content helpers
Every guardrail sharing litellm/proxy/guardrails/_content_utils.py silently
drops all text on the /v1/responses path. AIM turns it into a loud 422 (
{"error":"No messages in the request"}); every other guardrail (Lakera v2,
Cato, Lasso, Repello, IBM, Azure Content Safety, enterprise secret
detection) scans an empty payload and lets the request through unscanned.
Three defects, all in _content_utils.py:
1. _iter_text_parts_in_content recognised only part.type == "text", but the
Responses API uses input_text (request) and output_text (assistant).
2. _coerce_input_to_messages gated on "every item has a role key"; any
Responses input list containing a function_call or function_call_output
item failed the check and was wrapped as one opaque blob.
3. build_inspection_messages forwarded any role through, including a bare
tool role missing tool_call_id, which validators like AIM's /fw/v1/analyze
reject with a schema error.
Fix walks the actual Responses item taxonomy (message, function_call,
function_call_output, bare content parts and strings), recognises
{text, input_text, output_text} everywhere, and coerces any role outside
{system, user, assistant} to user in the outbound inspection payload.
* style: ruff-format changed guardrail files
* test(guardrails): cover function_call_output string form; drop em-dash in new docstring
* fix(guardrails): map function_call_output straight to user role
Avoids ever materialising a schema-invalid bare tool message. The
downstream role-safety coercion in build_inspection_messages still
guards genuinely caller-supplied non-standard roles (developer,
function, custom values); add a regression test covering that path
so the coercion has real coverage after this simplification.
* test(guardrails): pin chat-completions tool-role coercion in build_inspection_messages
* docs(test): soften AIM-specific claims in LIT-4294 test docstrings
Ryan's review flagged that several test docstrings assert AIM's
/fw/v1/analyze validates + rejects specific schema violations. That
behavior is customer-reported in the LIT-4294 writeup, not directly
verified by us. Rephrase to attribute the AIM 422 to the customer's
writeup and describe the underlying constraint as the OpenAI chat
schema; any downstream API that validates against that schema rejects
the same shape.
* refactor(guardrails): move unsupported-role coercion into AIM only
The generic coercion in build_inspection_messages collapsed any role
outside {system, user, assistant} to user for every caller of the
helper. Combined with the pre-existing apply_redacted_messages_back
write-back behavior in Lakera/AIM/Cato, that turned a loud OpenAI 400
on chat-completions tool-message masking into a silent semantic
corruption of the outbound request (role tool with tool_call_id got
rewritten to bare role user, dropping the assistant + tool_calls
sibling).
AIM specifically requires the coercion because its /fw/v1/analyze
validates the payload against the OpenAI chat schema; other guardrails
either do not validate roles or do their own reconstruction. Move the
coercion to AimGuardrail._build_aim_inspection_messages so the shared
helper keeps caller roles intact and no new cross-guardrail role
corruption is introduced. The pre-existing apply_redacted_messages_back
structural flatten remains as separate follow-up work.
function_call_output items still synthesise role user in the shared
helper because they have no natural role field, which is a different
concern from coercing a caller-supplied role.
* refactor(guardrails): preserve role fidelity in shared _content_utils
Shared inspection helpers should extract text and preserve semantic
role signals; role coercion for third-party schema safety stays inside
the guardrail that needs it (AIM).
Three shared-helper changes:
- Bare content-part dicts (input_text/output_text) with an explicit role
keep it; only role-less parts default to user.
- Responses message items already had their role preserved; the
behavior is now covered by an explicit test.
- function_call_output items default to role tool (semantic equivalent
of the chat-completions tool message shape) instead of role user, so
Responses and chat completions produce symmetric inspection payloads.
A caller-supplied role on the item is still preserved.
AIM's schema-safe coercion in _build_aim_inspection_messages already
handles the resulting role tool: it collapses to user before the POST
to /fw/v1/analyze so AIM's OpenAI-schema validator does not reject the
bare tool message (no tool_call_id can survive the flatten). Added a
regression test in test_aim.py covering that path.
The in-workflow regex list was hard to maintain and, because it runs on pull_request, could be modified by the same PR it inspects. Path gating for the OSS daily branches now lives in repository branch protection settings, so this workflow keeps only the OSS-safe checks: the hardcoded-secret test and ruff
Bedrock Converse supports cachePoint ttl (1h GA for Claude 4.5+), and
_get_cache_point_block maps cache_control.ttl -> cachePoint.ttl, but the
model parameter its allow-list gate requires was only threaded through
the system-message path. Every message-level path either called
_get_cache_point_block without model= (8 call sites in
_bedrock_converse_messages_pt / _pt_async) or hardcoded
CachePointBlock(type="default") (tool-result blocks and
_convert_to_bedrock_tool_call_invoke), so a requested 1h ttl silently
degraded to the 5-minute default - exactly on the conversation-tail
breakpoint that long-running agents need to survive tool calls longer
than 5 minutes.
- pass model= at the 8 _get_cache_point_block call sites
- tool-result blocks: capture the cache_control dict (was a boolean)
and route through _get_cache_point_block so ttl survives
- _convert_to_bedrock_tool_call_invoke: accept optional model and route
per-tool-call cache_control through _get_cache_point_block
Completes the ttl support added for system messages (#19848, #20326):
message-level cache_control now behaves identically.
Note: message-level cache_control on a content-less assistant message
emits no cachePoint at all today; that pre-existing gap is orthogonal
to ttl and left out of scope (per-tool-call placement covers it).
Co-authored-by: Arash <arashne@glia-ai.com>
