mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-11 22:51:28 +00:00
536 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Tin
|
42388c3d68 |
refactor(mcp): align the invalidation code with the v2 DI and typing discipline
The purge takes an injectable invalidate_token_cache callable defaulting to the manager's shared invalidation, and MCPServerManager takes an injectable per_user_token_cache alongside the existing per_user_oauth_token_store, so tests inject fakes instead of monkeypatching the global manager and the module-level cache. The new identity helpers drop Any for object throughout |
||
|
Tin
|
48124734a0 |
fix(mcp): compare the token identity decrypted and invalidate every per-user token store
Review follow-ups on the stale-token invalidation. The backend identity now decrypts client_id and client_secret before comparing: the stored values are NaCl-encrypted with a fresh nonce on every write, so comparing ciphertext flagged every routine save as a mint-relevant change and purged per-user tokens that were still valid. The identity also gains spec_path, the audience for OpenAPI servers, and parses credentials stored as a JSON string The purge now routes each (user, server) through the manager's invalidate_user_oauth_token_cache, which becomes the single invalidation point covering both the legacy per-user token cache and the v2 per-user OAuth token store; previously the purge evicted only the legacy cache while the revoke path evicted only the v2 store, so each path left the other cache serving a replaced token until its TTL. A credential row racing in between the find and the delete is now detected via the delete_many count and logged; its cache entry expires by TTL On the dashboard, CLEARED_ON_INVALIDATION and the staleness check move to types.tsx as the single shared implementation for both forms. The edit form's transport handler now rechecks the identity after its programmatic setFieldsValue calls, which antd does not report through onValuesChange, so a token no longer survives a transport switch that clears the mint target. The create form rebuilds formValues from the post-reset form state after an invalidation instead of publishing the pre-reset snapshot, so the tool preview can no longer refetch with the discarded DCR client. Both transport handlers now share the recheck, which also stops the create form from over-invalidating on an http to sse swap that keeps the same url and therefore the same audience |
||
|
Tin
|
05f39bf942 |
fix(mcp): invalidate a browser-authorized upstream token when a mint-relevant field changes
An admin who ran Authorize & Fetch and then changed a field that determines which upstream OAuth token gets minted kept using the stale token for tool preview, sessionStorage, and (on the backend) the stored per-user credential and its cache. Grounded in RFC 8707/8693 and the MCP auth spec, a token is bound to one tuple: resource/audience (url), OAuth mode/grant (auth_type, oauth_flow_type), the authorization-server endpoints, and the OAuth client + scopes. A shared getOAuthAuthorizationIdentity captures exactly those fields; transport (http/sse on the same url is the same audience) and delegate_auth_to_upstream (a downstream-usage toggle never sent to the authorize request) are excluded. UI: both the create and edit forms now discard the held token (React state / sessionStorage / hook, plus the fetched token + DCR client in form.credentials) whenever the identity diverges from the one it was authorized against, re-applying the admin's in-flight edit so it is never wiped. The check lives in one shared helper so the two forms cannot drift. Backend: editing an MCP server now compares the pre/post identity and, on a mint-relevant change, purges every stored per-user OAuth credential for the server (DB row + per-user token cache) so no user forwards a token minted for a resource/AS/client that no longer matches. Best-effort; a purge failure never fails the update. |
||
|
Tin
|
d0f1c38d6a |
fix(mcp): log only the origin of the upstream MCP url in tool-call metadata
The redacted resource kept the path, but hosted MCP servers routinely embed the credential in the path (for example /mcp/s/<token>/mcp), and mcp_tool_call_metadata is readable by a caller who can invoke the tool, so the path leaked the upstream credential into spend logs. Only scheme, host, and port are logged now |
||
|
Tin
|
65d0dcfb82 |
fix(mcp): never forward an Authorization header that satisfied admission on the tools preview
Authorization doubles as the admission fallback when x-litellm-api-key is absent, so a caller who authenticated the preview request that way had their LiteLLM key forwarded to the upstream as the oauth2/client-forwarded token. The preview now forwards Authorization only when the primary admission header is present, which is how the dashboard has always sent it; with no primary header there is no upstream token on the request at all. Applies to oauth2 and both client-forwarded modes; parametrized regression test plus the admission header added to the existing extraction tests to mirror the real UI request shape |
||
|
Tin
|
7c52cde505 |
fix(mcp): redact upstream URL in tool-call logs and plug fan-out Authorization bypass
Two review findings on the passthrough modes. The tool-call log records the upstream MCP server URL as mcp_server_resource, which is persisted in spend-log metadata and sent to logging callbacks. A URL carrying embedded userinfo or a secret query parameter would leak into logs, so the value is now redacted to its bare resource identifier (scheme + host + path); userinfo, query string, and fragment are stripped before it is logged. The listing fan-out withholds the request-wide Authorization from a true_passthrough / oauth_delegate server when another server in scope also consumes it, so one bearer is not replayed across upstreams. The later server.extra_headers copy loop did not honor that decision: a server listing Authorization in extra_headers would re-copy the withheld bearer from raw_headers. The withhold decision is now computed once and applied to both the forwarding branch and the extra_headers loop. |
||
|
Tin
|
98818df418 |
fix(mcp): recognize per-server auth header at connect and stop persisting browser-authorize tokens
Two correctness fixes for the client-forwarded token modes.
The preemptive-401 connect gate for true_passthrough and oauth_delegate
only inspected the request-wide Authorization, so a caller who bound the
upstream token via the per-server x-mcp-{alias}-authorization header (the
mandatory shape in a multi-server aggregate, where the request-wide
Authorization is withheld) was spuriously 401'd at connect even though
egress already honors that header. The gate now recognizes the per-server
header for both modes via a shared helper, mode-correctly: true_passthrough
treats any Authorization or the per-server header as the upstream token,
oauth_delegate keeps requiring a distinct x-litellm-api-key so a lone
Authorization consumed for admission is never mistaken for an upstream
token. The preemptive raise is also gated to single-server scopes so a
multi-server aggregate degrades gracefully (the listing absorbs a
per-server failure) instead of one missing token 401-ing the whole connect.
The browser-only Authorize flow was writing the upstream access and refresh
token to LiteLLM_MCPUserCredentials, contradicting the modes' persist-nothing
contract: the temp OAuth-relay server was cached with a hardcoded oauth2
auth_type, so needs_user_oauth_token was true and the token exchange stored
it. The create and edit forms now send the real auth_type for these modes,
so the temp server is not oauth2, needs_user_oauth_token is false, and the
exchange skips storage while still returning the token to the browser
session.
|
||
|
Tin
|
367aa904de |
fix(ui): wire the tool playground and gateway authorize flow for the client-forwarded token modes
The server detail page's Tool Testing Playground gated its browser-held
token handling on the legacy PKCE-passthrough shape, so a
true_passthrough or oauth_delegate server listed tools unauthenticated
and surfaced 'Failed to fetch MCP tools' with no way to authorize. The
playground now treats both modes as browser-held-token servers: it
reads the sessionStorage token established by the create/edit
browser-only Authorize, forwards it via the x-mcp-{alias}-authorization
header, evicts it on a 401, and shows its own Authorize gate when the
token is absent.
That gate's flow uses the gateway's relayed authorize/register/token
endpoints with the real server id, which previously 400ed for anything
but oauth2. Those endpoints now also accept the client-forwarded token
modes (the minted token is upstream-audienced and browser-held; DCR
persistence stays off on this path), and registry builds run the same
RFC 9728/8414 endpoint discovery for these modes that oauth2 rows get,
since their rows never store an authorization_url.
|
||
|
Tin
|
22ab518071 |
feat(ui): browser-only Authorize & Fetch for the client-forwarded token modes
true_passthrough and oauth_delegate persist no upstream credentials, so
the create/edit forms had no way to preview tools or configure the tool
allowlist: tools/list went upstream unauthenticated and came back 401.
This reuses the existing OAuth authorize machinery in browser-only mode
for those two auth types: the admin authorizes against the upstream
(DCR/PKCE, with optional client credentials for IdPs without dynamic
registration), the token lands in sessionStorage exactly like the
legacy PKCE-passthrough path, and the tools preview forwards it via the
per-server x-mcp-{alias}-authorization header, which the passthrough
resolver arm already accepts. Nothing is written to the server row or
the per-user credential store; the create payload keeps excluding
credentials for these auth types via AUTH_TYPES_REQUIRING_CREDENTIALS.
The tools preview endpoint now also extracts the Authorization header
for the two new auth types so the browser-held token reaches the
passthrough arm during create-time previews.
|
||
|
tin-berri
|
4e6ec995e7
|
Merge pull request #31989 from BerriAI/litellm_mcp_passthrough_delegate_modes
feat(mcp): add true_passthrough and oauth_delegate auth modes |
||
|
Tin
|
b2ea36f4f1 |
fix(mcp): match sanitized per-server alias at the connect-time preemptive 401
The connect gate resolved x-mcp-{alias}-authorization by matching the raw
lowercased alias/server_name/name only, but dashboard clients send
x-mcp-{sanitize_mcp_alias_for_header(alias)}-authorization, and egress resolves
those through lookup_mcp_server_auth_in_headers, which also tries the sanitized
alias. So a per-server token bound with a sanitized alias (e.g. alias 'pt-server'
arriving as header key 'pt_server') was forwarded at egress but still triggered a
preemptive 401 at connect. _client_has_per_server_auth_header now resolves through
the same lookup_mcp_server_auth_in_headers egress uses, so connect and egress
agree on which header names match.
|
||
|
Tin
|
ddec3b2b8b |
fix(mcp): plug fan-out Authorization bypass in the extra_headers loop
The listing fan-out withholds the request-wide Authorization from a true_passthrough / oauth_delegate server when another server in scope also consumes it, so one bearer is not replayed across upstreams. The later server.extra_headers copy loop did not honor that decision: a server listing Authorization in extra_headers would re-copy the withheld bearer from raw_headers. The withhold decision is now computed once and applied to both the forwarding branch and the extra_headers loop. |
||
|
Tin
|
edf00bbe23 |
fix(mcp): recognize per-server auth header at the connect-time preemptive 401
The preemptive 401 for true_passthrough and oauth_delegate only inspected the
request-wide Authorization, so a caller who bound the upstream token via the
per-server x-mcp-{alias}-authorization header (the required shape in a
multi-server aggregate, where the request-wide Authorization is withheld) was
spuriously 401'd at initialize even though egress already honors that header.
The gate now recognizes the per-server header for both modes via a shared
helper, mode-correctly: true_passthrough treats any Authorization or the
per-server header as the upstream token, oauth_delegate keeps requiring a
distinct x-litellm-api-key so a lone Authorization consumed for admission is
never mistaken for an upstream token. The preemptive raise is also gated to
single-server scopes so a multi-server aggregate degrades gracefully instead
of one missing token 401-ing the whole connect.
|
||
|
Tin
|
4a25cce114 |
fix(mcp): reject duplicate Authorization headers at MCP ingress
For the client-forwarded token modes the gateway relays the caller's Authorization to the upstream, so a request carrying more than one Authorization header would make which token is forwarded ambiguous (the ASGI header list collapses to last-wins) and could diverge from what admission inspected. Multiple Authorization headers is malformed for bearer auth anyway (RFC 9110: not a comma-combinable field), so the ingress header converter now fails closed with a 400 instead of silently keeping one. Applies to every MCP request, not just passthrough. |
||
|
David Katz
|
c0327cded4 |
fix(mcp): pair token-endpoint client_secret with the same source as client_id
On re-auth against a server with a persisted DCR client, register_client_with_server
short-circuits and returns a placeholder client_secret ("dummy") that the browser
echoes back to /token. exchange_token_with_server overrode the caller's client_id
with the persisted one but still fell back to the caller's secret when the server
had none stored, so a persisted public PKCE client (which has no secret) was paired
with the literal string "dummy" and the IdP rejected the exchange with 401 on every
re-authorization; the proxy surfaced that as a 500. First connects and brand-new
servers worked because a real DCR registration ran and no placeholder existed.
Resolve the secret from the server whenever the server's client_id wins, so a
secretless public client sends no client_secret at all
|
||
|
Yassin Kortam
|
bfff5e8d86
|
fix(mcp): log MCP tool calls returning isError=true as failures (#32238)
An MCP tool call that completes with CallToolResult.isError=true correctly returns HTTP 200 per the MCP spec, but the shared post-call logging helper always fired async_success_handler, so the standard logging payload carried status=success and OTel (whose _parse_error only marks ERROR on status=failure) showed green spans for failed tools. The helper now checks the result after async_post_mcp_tool_call_hook runs (guardrails may flip isError there) and routes error results to the failure path: success gates are consumed so the @client wrapper cannot enqueue a success log, failure_handler and async_failure_handler fire with a new MCPToolResultError carrying the tool's first text content, and post_call_failure_hook records the failure the same way raised exceptions already do. Raised exceptions never reach the helper, so no double failure logging. HTTP wire behavior is unchanged Resolves LIT-4081 |
||
|
tin-berri
|
1fb2b4aef4
|
fix(mcp): drop the cached per-user OAuth token when the credential row changes (#32302)
* fix(mcp): drop the cached per-user OAuth token when the credential row changes The v2 authorization_code chain Cached(Refreshing(V2PerUserTokenStore)) caches a positive token until its expires_at (or 300s without one), and CachedOAuthTokenStore.invalidate had no callers, so a re-authorization or revocation wrote the DB while egress kept serving the replaced token from the in-process cache until its TTL. LazyPerUserOAuthTokenStore now exposes invalidate, MCPServerManager threads it to the write side, and the three credential write sites (the OAuth callback, the Tools-tab persist endpoint, and the revoke endpoint) drop the cache entry after the row changes. The v2 refresher's own persist stays untouched; RefreshingTokenStore already feeds the rotated token back into the cache in the same fetch * test(mcp): pin cache invalidation on the revoke already-gone branch Greptile's review flagged that only the happy-path delete asserted the invalidate; a refactor moving the call inside the try block would silently skip the cache drop when the row was already deleted by a concurrent request while the cache still held the revoked token. The new test fails on exactly that mutation * test(mcp): cover invalidate on the redis-backed lazy store path Codecov flagged the redis fast path of LazyPerUserOAuthTokenStore.invalidate as unexercised; the existing invalidate tests only ran the no-redis chain. The new test builds the redis chain via a fetch and asserts a subsequent invalidate reaches the same store instance without a rebuild |
||
|
tin-berri
|
d6cbf6e7e3
|
feat(ui): expose MCP max_concurrent_requests in server create and edit forms (#32397)
* feat(ui): expose MCP max_concurrent_requests in server create and edit forms The proxy has enforced a per-server outbound tool-call concurrency cap (max_concurrent_requests) across every MCP egress path since #31641, and the management API has accepted the field on create and update all along, but the dashboard offered no way to set it. Add an optional Max Concurrent Requests input to the MCP server create and edit forms; it applies to every auth type and transport, so it renders unconditionally rather than gated on auth mode. Clearing the field on edit sends null so the stored limit is unset. Also rebuild the per-server semaphore when the configured limit changes. Previously the semaphore was created once per server_id and never resized, so an edited limit only took effect after a proxy restart even though the new value was persisted and reloaded into the registry. * feat(ui): mark MCP max concurrent requests field label as optional * test(ui): stop OBO create-form tests from timing out on CI The token-exchange payload test and the Entra scope-required test filled five text fields with user.type, which dispatches a full keystroke sequence per character; every input event runs the antd form onValuesChange handler and re-renders the whole CreateMCPServer tree, roughly 120 renders per test. As the form grew the two tests reached 8s and 18s locally, which crosses the 30s vitest timeout on slower CI containers; ui_unit_tests failed twice this way. Switch the plain text fields to fireEvent.change (one input event per field), matching the existing stdio test pattern. Both tests assert form output, not keystroke behavior, and now run in about 3s each. |
||
|
tin-berri
|
f922be32f0
|
fix(mcp): accept integer progressToken in host progress capture (#32402) | ||
|
Tin
|
732832d342 |
fix(mcp): bind client-forwarded Authorization to a single upstream
In a listing fan-out over a scope containing more than one server that
consumes the caller's Authorization (true_passthrough, oauth_delegate,
or the legacy delegate/passthrough shapes), the request-wide bearer is
now withheld from the new modes instead of being replayed against every
upstream (RFC 9700 cross-resource replay). Explicitly-addressed
operations (tool call, get_prompt, read_resource, single-server routes)
keep forwarding it.
Multi-server aggregates use the per-server x-mcp-{alias}-authorization
header instead: its value now feeds the passthrough resolver arm as the
inbound token and wins over the request-wide header, binding one token
to one server.
|
||
|
Tin Chi Lo
|
9bf3907de5 |
fix(mcp): bind oauth_delegate discovery resource to the upstream
oauth_delegate forwards the caller's token to the upstream, which validates its audience, so the protected-resource metadata must keep resource pointing at the upstream (returned verbatim, like true_passthrough) rather than rewriting it to the gateway. Rewriting to the gateway asks the client to mint a token bound to the gateway audience, which a strict IdP (Entra) refuses to issue for an unregistered resource and a spec-compliant upstream rejects on receipt. The legacy is_oauth_passthrough opt-in keeps the gateway rewrite unchanged. |
||
|
Tin Chi Lo
|
446a6e8cdd |
fix(mcp): route true_passthrough and oauth_delegate through upstream OAuth discovery
Both modes advertised LiteLLM as the authorization server and answered initialize locally, so a client with no token connected empty and was never driven into the upstream OAuth flow. The protected-resource discovery now proxies the upstream metadata for both modes (verbatim for true_passthrough, resource rewritten to the gateway for oauth_delegate), and the preemptive 401 emits the matching challenge: oauth_delegate uses the gateway-proxied resource_metadata once admission passes, true_passthrough probes the upstream anonymously and surfaces its WWW-Authenticate verbatim so the client authorizes directly against the upstream |
||
|
Tin Chi Lo
|
50c90281f4 |
feat(mcp): add true_passthrough and oauth_delegate auth modes
Introduce two first-class MCP server auth_type values that make LiteLLM's role in upstream authentication explicit, added alongside the existing delegate_auth_to_upstream / oauth_passthrough flags without changing their behavior. true_passthrough is a transparent proxy: LiteLLM performs no admission auth, requires no x-litellm-api-key, mints/stores/refreshes nothing, and forwards the client's Authorization to the upstream exactly as received. oauth_delegate keeps normal LiteLLM admission (x-litellm-api-key / SSO / JWT) and then forwards the client's separate upstream Authorization unchanged; the admission credential is never forwarded upstream. Both modes forward the caller's token via the existing extra_headers path and defer egress credential resolution to v1 (the v2 to_server_spec returns None for them). Upstream 401/403 responses are surfaced rather than swallowed so upstream OAuth challenges are preserved. Servers in either mode require per-user auth, so userless health checks are skipped. |
||
|
tin-berri
|
db2402754a
|
feat(mcp): let users select the entra_obo token_exchange profile in the UI and API (#32144)
* feat(mcp): let users select the entra_obo token_exchange profile in the UI and API
The backend token_exchange arm supports two wire dialects via token_exchange_profile
("rfc8693" default, or "entra_obo" for Microsoft Entra's On-Behalf-Of, the RFC 7523
jwt-bearer grant), but it could only be set through config.yaml. This surfaces it to the
create/update REST API and the dashboard so an admin can create an entra_obo server there,
completing the parity started in the parent PR for the other token-exchange fields.
token_exchange_profile becomes a dedicated column on LiteLLM_MCPServerTable, mirroring the
sibling fields: it is added to the request models, read column-first in
build_mcp_server_from_table with the credentials-blob as a back-compat fallback and a
default of rfc8693, and carried through both runtime-to-table builders so registry
round-trips preserve it. It is a non-secret dialect selector, so it is not scrubbed from
non-admin or virtual-key responses.
In the dashboard a Profile dropdown (RFC 8693 vs Microsoft Entra OBO) is added to the
token-exchange section. Entra OBO carries the target resource in the scope, so selecting it
makes the scope required and hints the api://<app-id>/.default form, while audience and
subject_token_type (which that dialect ignores) are hidden.
* fix(mcp): extend the blob-to-column lift and non-admin scrubbing to token_exchange_profile
token_exchange_profile gets the same storage contract as the other three
token-exchange settings: the column is authoritative, a blob copy is the legacy
shape — lifted into the column on every write and stripped from the stored
blob — and switching auth_type away from token exchange clears it
(_AUTH_FLOW_SCOPED_FIELDS). Both restricted-view sanitizers scrub it for
uniformity, and the edit form's auth-switch payload nulling includes it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test(mcp): assert every token-exchange setting is configurable via config.yaml
Pins the config surface: token_exchange_endpoint, audience, subject_token_type
and token_exchange_profile load from top-level config keys onto the built
server and through to the resolver spec; omitted keys resolve to their
documented defaults (RFC 8693 subject token type, rfc8693 profile), and
token_exchange servers need no oauth2_flow.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
||
|
tin-berri
|
9652509e46
|
fix(mcp): apply outbound concurrency limit to OBO tool calls (#32071)
The token_exchange (OBO) branch of _call_regular_mcp_tool built its coroutine by calling _obo_call_tool_with_retry directly, outside the _limit_outbound_concurrency context manager that the regular branch uses. OBO tool calls (and the internal re-mint retry, which issues a second upstream call_tool) therefore bypassed the per-server max_concurrent_requests semaphore, so an authenticated caller could run unlimited concurrent tool calls against an OBO MCP server despite an admin-configured limit. Wrap the OBO coroutine in _limit_outbound_concurrency the same way the regular path does, holding one permit across the initial call, the on-401 re-mint, and the retry, so OBO calls honor the configured cap. |
||
|
tin-berri
|
ff6dc33291
|
feat(mcp): support oauth2_token_exchange auth type via REST API and dashboard (#31772)
* feat(mcp): support oauth2_token_exchange auth type via REST API and dashboard OAuth 2.0 Token Exchange (RFC 8693, a.k.a. OBO) for MCP servers could previously only be configured through config.yaml; the create/update REST API and the dashboard had no way to express it. This wires token_exchange_endpoint, audience, and subject_token_type end to end. These three are persisted as dedicated columns on LiteLLM_MCPServerTable, mirroring how token_url and oauth2_flow are stored, so the edit form prefills them and they are unaffected by the credentials-blob clearing on auth_type change. build_mcp_server_from_table reads the columns first and falls back to the credentials blob so servers persisted before the columns existed still load. client_id and client_secret continue to ride the existing encrypted credentials path. On the dashboard, "OAuth Token Exchange (OBO)" is a distinct auth-type option with its own field section. The McpOAuthMode classifier gains a token_exchange arm keyed off auth_type; the previous catch-all oauth2 mode was renamed from "obo" to "authorization_code" so the two on-behalf-of mechanisms are no longer conflated. The token-exchange IdP endpoint and audience are scrubbed from non-admin and virtual-key responses, matching how token_url is treated. * fix(ui): gate the MCP list-401 re-auth on authorization_code, not token_exchange The renamed 401 gate keyed off isTokenExchange, but that condition exists for authorization_code: when a stored per-user credential is present yet the tools/list 401s (the backend's refresh could not mint a token), the user must re-authorize via the browser flow. token_exchange has no gateway-side authorize step, so the Authorize gate never applied to it. The prior isObo flag was undefined (a compile error) and, per this file's convention and its tests, meant authorization_code; renaming it to isTokenExchange changed the behavior and broke the mcp_tools auth-gate test for an authorization_code server whose token expired with no usable refresh. Gate on isAuthorizationCode instead and drop the now-unused isTokenExchange * fix(mcp): clear flow-scoped endpoint config when a server's auth_type changes Switching an existing oauth2 server to oauth2_token_exchange left the old flow's token_url on the row. The OBO resolver treats token_exchange_endpoint or token_url as the configured exchange endpoint, so the stale value both suppressed the RFC 9728/8414 discovery this PR adds and sent the exchange grant (client credentials plus the user's subject token) to the previous flow's token endpoint update_mcp_server now mirrors its existing stale-credentials rule for the flow-scoped columns (authorization_url, token_url, registration_url, oauth2_flow, token_exchange_endpoint, audience, subject_token_type): when auth_type changes, each one is cleared unless the same request explicitly provides it, so a deliberate override in the switch request still wins. Updates that keep the auth_type never touch these columns, which keeps legacy OBO rows that use token_url as their exchange endpoint working The edit form sends explicit nulls for the previous flow's fields on an auth type switch; antd preserves unmounted field values by default, so without this the old token_url would be re-sent verbatim and read as an explicit override. Transitions are detected against the persisted auth_type, so saves that keep the auth type send nothing extra Reported by Cursor Bugbot on the PR * fix(mcp): lift legacy blob token-exchange settings into their columns on every write The three token-exchange settings live in dedicated columns but also exist on MCPCredentials as the pre-column REST shape. Writes now lift incoming blob values into the columns (an explicit top-level value wins, including an explicit null) and strip them from the stored blob; the same-auth credentials merge migrates legacy rows the same way. The read-time column-or-blob fallback then only ever serves rows current code has never written, so clearing a column to re-enable RFC 9728/8414 discovery can no longer be silently undone by a stale blob copy. Also asserts the auth-switch clearing fires on the external fields_set path (PUT /v1/mcp/server). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(mcp): single source for the RFC 8693 default subject_token_type The default was applied at four egress build sites plus two model defaults, each with its own copy of the literal. All sites now share DEFAULT_SUBJECT_TOKEN_TYPE from litellm.types.mcp. A DB-level DEFAULT is deliberately not used: Prisma writes explicit values on insert, so a column default would rarely apply, and NULL-means-RFC-default keeps existing rows correct. Also documents two review decisions in place: the audience column keeps the RFC 8693 parameter name (RFC 8707 resource indicators are already a separate concept named resource in the v2 egress types), and the migration's out-of-order timestamp is safe under prisma migrate deploy. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore: fix import sort order in outbound_credentials/types.py (I001 strict budget) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(mcp): purge legacy blob copies when a token-exchange column is written without credentials The migrate-on-write in the credentials merge lifts blob values into null columns, which is correct for legacy rows but could repopulate a column an admin had cleared in an earlier no-credentials update (that path never touched the blob, so the stale copy survived to be lifted later). An explicit token-exchange column write (set or clear) now migrates the row even when the update carries no credentials: untouched null columns are lifted, every blob copy is stripped, and unrelated blob keys stay as-is. A cleared column can then never be resurrected, because no write path leaves a blob copy behind. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(mcp): state the blob-to-column lift contract on the legacy credential keys The three token-exchange keys on MCPCredentials are the pre-column REST shape (the only REST shape from 2026-05 until this PR). Document on both the blob type and the request models that the dedicated columns are authoritative and that writes lift blob values into them and strip the stored copy. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(mcp): scrub subject_token_type in the non-admin and virtual-key sanitizers The other two token-exchange fields were cleared while subject_token_type was left visible. It is a public RFC 8693 URN with no disclosure value, but the sanitizers' rule is that these views receive no token-exchange config at all — cleared for uniformity. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
tin-berri
|
12801260ce
|
feat(MCP/UI): add OAuth flow selector on the MCP edit page (#32298)
* feat(ui): OAuth flow selector on the MCP edit page
The edit form had no flow selector: oauth_flow_type was watched but never registered,
so isM2MFlow was always false in edit mode and the flow could only be changed over
REST. That left the backfill's remediation for ambiguous legacy rows (client creds +
token_url, no interactive signal, left unstamped) without a dashboard path
The oauth2 section now opens with an OAuth Flow Type select. Explicit rows prefill
their stored value and re-persist it on save; legacy null rows show a placeholder
instead of a fake preselection, and an untouched save still writes nothing, so the
form never guesses on the admin's behalf. Choosing Machine-to-Machine (M2M) persists
oauth2_flow=client_credentials, choosing Interactive (PKCE) persists
authorization_code, which is exactly the assertion the backfill warning asks for.
Registering the field also brings the existing isM2MFlow gating in the edit form to
life, so M2M rows stop showing the interactive-only token-validation fields
Tests cover the prefill round-trip for both explicit values, the untouched null row
writing nothing, and both selections persisting on a legacy null-flow row
* fix(mcp): registry-to-table conversions must carry oauth2_flow
_build_mcp_server_table and the health-check table builder dropped oauth2_flow when
converting registry servers for GET /v1/mcp/server (list and by-id), so the dashboard
never received the persisted flow: the edit page could not prefill the selector, M2M
gating never activated, and the tools page classifier saw every oauth2 server as
interactive regardless of the column. Found live while proving the edit-selector
persistence path end to end; the write side was fine (PUT persists and the column
reads back correctly), the read side was dropping the field at the conversion
Both builders now carry oauth2_flow; regression test pins the conversion
* docs(mcp): flag _resolve_oauth2_flow as security-sensitive in its docstring
The prior wording ('not called directly by security sites') could read as if the
function has no security relevance, when it is the shape-inference engine both
request-time security helpers delegate to. Reword to state that plainly: it decides
M2M-vs-interactive for an unstamped row, must always be reached through
effective_oauth2_flow or resolve_oauth2_flow_for_request, and its M2M-shape branch
must not be weakened without accounting for those callers. Docstring-only; no logic
change
Raised by review on the stacked PR
* refactor(ui): extract oauth2FlowToFormValue helper for the MCP OAuth flow prefill
The edit form derived the OAuth Flow Type select value from the stored oauth2_flow
with a nested ternary duplicated at two call sites. Extract the mapping into a named
helper in types.tsx (next to getMcpOAuthMode and the flow constants): client_credentials
-> M2M, authorization_code -> Interactive, null/unset -> undefined so the select shows
its placeholder instead of a guessed default. The tool-config call site keeps its
null -> Interactive display fallback via a trailing ?? OAUTH_FLOW.INTERACTIVE, so
behavior is unchanged. Adds unit tests for the helper; the existing prefill/save tests
already cover the call sites
* feat(ui): surface and warn on an unset MCP oauth2_flow (server card + edit page)
An oauth2 MCP server whose oauth2_flow was never classified (legacy null row the
backfill left ambiguous) now advertises that it needs attention instead of silently
falling back. The server card shows an 'OAuth flow not set' warning tag for any
auth_type=oauth2 server with no oauth2_flow, so admins can spot them in the list
without opening each one. The edit page shows a warning alert directly under the new
OAuth Flow Type selector while the flow is unset, and it clears the moment a flow is
picked.
Delegate (delegate_auth_to_upstream) servers are excluded from both: they authenticate
via upstream PKCE passthrough and route to passthrough regardless of oauth2_flow, so
the M2M-vs-interactive classification does not apply and prompting for it would be a
false alarm. The edit page reads the delegate state from the watched switch when it is
mounted and falls back to the stored value otherwise (useWatch returns undefined for an
unmounted field).
Also adds end-to-end coverage of the null-flow chain the selector depends on:
build_mcp_server_from_table carries oauth2_flow=None verbatim into the GET response,
so the dashboard maps it to undefined and shows the placeholder rather than a guessed
default. Tests: backend null carry, the select prefill display for all three states,
the edit-page warning show/hide/clear-on-select and delegate exclusion, and the card
badge across oauth2/non-oauth2, stamped/unstamped, and delegate
|
||
|
tin-berri
|
7ce573e6e8
|
fix(mcp): stop 'Team doesn't exist' warnings for UI dashboard sessions (#32348)
UI session tokens carry the virtual team_id litellm-dashboard (UI_TEAM_ID), which is never persisted. The MCP team-permission helpers passed it to get_team_object anyway, so every dashboard MCP listing raised a 404 per lookup that was swallowed into per-server 'Failed to get allowed tools for server' warnings (plus the sibling 'allowed MCP servers for team' and 'MCP access groups for team' warnings) and wasted DB queries. The 404 also escaped past the key-level permission handling in get_allowed_tools_for_server, dropping key tool restrictions for such sessions. Short-circuit the virtual team before the DB lookup in the three helpers, mirroring the existing UI_TEAM_ID handling in agent_permission_handler. Also reject /team/new with the reserved team_id, since a real row would bind its budget and permissions to every UI session |
||
|
tin-berri
|
733c01902f
|
feat(mcp)!: oauth2_flow read verbatim from DB rows and required in config; inference reduced to the request-time backstop (#32292)
* refactor(mcp): read oauth2_flow verbatim from DB rows; inference stays config-only plus a logged backstop With every DB write site stamping oauth2_flow (#32283, #32288) and the startup backfill healing legacy null rows, the DB build no longer needs to re-derive the flow from field shape. build_mcp_server_from_table now reads the column verbatim via _explicit_oauth2_flow: unknown or null values resolve to None, which needs_user_oauth_token already treats as interactive, so an unstamped row degrades to the safe default instead of guessing M2M from a shape that a DCR-registered interactive server shares whenever discovery is down Field-shape inference survives in exactly two places. config.yaml-loaded servers keep it at load time: they are rebuilt from the config on every boot, so there is no row to backfill and load-time resolution is their write-time stamp. And the request-time backstop in _get_allowed_mcp_servers keeps a not-yet-backfilled M2M row blocking caller Authorization forwarding (the P1 property); it now logs a warning whenever it actually fires, which is the fire-rate signal for deleting it once deployments have booted past the backfill Regression tests pin that the DB build does not infer M2M from the credential shape and reads an explicit column value verbatim Fourth step of the oauth2_flow persistence sequence, stacked on the backfill * feat(mcp): deprecation warning when config-level M2M is inferred rather than declared A config.yaml oauth2 server whose credential shape decides client_credentials without an explicit oauth2_flow now logs a warning at load pointing the admin at the explicit declaration. First rung of the deprecation ladder: the docs make oauth2_flow the recommended path, the warning surfaces configs still relying on inference, and a future breaking release can turn it into a config validation error, at which point config-level shape inference dies entirely. Interactive omissions stay silent since the default matches inference there and nothing load-bearing is being guessed * feat(mcp)!: require explicit oauth2_flow for config-defined oauth2 servers A config.yaml server with auth_type oauth2 must now declare its flow; the load raises a config validation error naming both values and what each means: oauth2_flow: client_credentials for machine-to-machine (the proxy mints a shared token at token_url using client_id/client_secret) or oauth2_flow: authorization_code for interactive (per-user tokens via browser sign-in, including delegate_auth_to_upstream) This replaces the load-time shape inference for config servers entirely. The credential shape is genuinely ambiguous (a DCR-registered interactive server carries client creds + token_url with no authorization_url, identical to M2M), so the config asserts the answer instead of the proxy guessing it. With this, field-shape inference survives in exactly one place: the request-time security backstop, which is telemetry-gated for deletion BREAKING CHANGE: config-defined oauth2 MCP servers without oauth2_flow fail proxy startup with the error above. Add the one line to the server block; the error text says exactly which value to pick * test(mcp): pin the verbatim read for authorization_code alongside client_credentials Raised by review on the PR * fix(mcp): fail closed on the anonymous delegate gate for unstamped M2M-shaped servers Reading oauth2_flow verbatim (this PR) changed has_client_credentials from True to False for a legacy null-flow row that still carries the M2M credential shape. That value is what the anonymous upstream-delegate gate checks before skipping LiteLLM auth entirely, so an M2M-shaped delegate server that was never stamped would newly pass the gate: an unauthenticated caller could get it selected and then list/read upstream data using the client credentials the request-time backstop re-infers, running as LiteLLM's service account. This reopens the hole the gate's existing 'never delegate for M2M' guard was written to close The gate now resolves the flow (column first, shape fallback) instead of reading the bare column, mirroring the request-time backstop in _get_allowed_mcp_servers: both fail closed on the ambiguous M2M shape and are removed together once no null rows remain. A pure-PKCE delegate server (no stored credentials) resolves to a non-M2M flow and keeps its bypass, so the common delegate case is unaffected Tests: an unstamped M2M-shaped delegate server is denied the bypass (mutation-checked against the bare-column regression), and a pure-PKCE delegate server still bypasses Raised by review on the PR * fix(mcp): centralize the request-time oauth2_flow backstop across every security site Reading oauth2_flow verbatim made has_client_credentials unreliable for legacy null rows, and the backstop that compensates was applied at only one reader. Review found three more consequences of that per-site approach: - the anonymous-delegate allowlist in get_allowed_mcp_servers read the bare column, so an unstamped M2M-shape delegate server was surfaced to anonymous callers (High) - call_mcp_tool resolved allowed ids into MCPServer objects without the backstop, so a null-flow M2M-shape row kept has_client_credentials false on tool execution during a backfill gap, though the listing path was covered (High) - the request-time warning claimed the startup backfill would stamp the row next boot, but the backfill deliberately leaves the ambiguous M2M shape unstamped (Low) Rather than patch each site, introduce two helpers on MCPServerManager that are the single choke point for request-time resolution: effective_oauth2_flow(server) for the enum/boolean decisions (allowlist filter, anonymous-delegate gate) and resolve_oauth2_flow_for_request(server) for the egress object copy (listing and tool call). Both fail closed on the M2M shape and leave stamped rows and pure-PKCE rows untouched. The gate now shares effective_oauth2_flow instead of its inline resolution, and the corrected warning lives once inside resolve_oauth2_flow_for_request, so deleting the whole transitional layer later is a single-site change. Tests: helper unit coverage (stamped verbatim, null M2M-shape resolves, pure-PKCE stays None, stamped/pure-PKCE return the same object, corrected warning text), the anonymous allowlist excludes an unstamped M2M-shape delegate server, and the call path resolves the flow like the listing path. The two security-integration tests are mutation-checked against the bare-column regression. Raised by review on the PR |
||
|
tin-berri
|
6041d37414
|
fix(mcp): apply semantic filter to expanded litellm_proxy tools and show filtered-out count (#32285)
* fix(mcp): apply semantic filter to expanded litellm_proxy tools and show filtered-out count * fix(mcp): guard expansion-path filtering on enabled flag and isolate metadata emission |
||
|
Sameer Kankute
|
a78dc69a09
|
fix(mcp): alias/display-name tool routing, REST filters, BYOK auth (#32320)
* fix(mcp): resolve tool name prefix via known server prefixes, not string match When an MCP server's alias differs from its server_name, tool names are listed with the alias prefix but _execute_tool_calls compared that prefix against the server_name stored in tool_server_map. The mismatch silently skipped prefix stripping, forwarding the fully-prefixed tool name upstream and causing "Unknown tool" failures. Resolve the actual MCPServer object and strip using its known prefix forms (alias, server_name, server_id) instead. * fix(mcp): preserve tool overrides and scope REST tool listing Return saved tool display/description overrides from the server table API so the edit UI reloads them, resolve display names before prefix stripping on tool calls, and honor mcp_server_name and toolset_name filters on the REST tools list endpoint. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(mcp): inject BYOK credentials on Playground OpenAPI tool calls Playground and Responses API route MCP execution through call_tool, which skipped BYOK lookup and never set the OpenAPI auth ContextVar, so upstream calls went out unauthenticated despite a stored user credential. Co-authored-by: Cursor <cursoragent@cursor.com> * test(mcp): cover alias-mismatch prefix stripping and display-name reverse mapping Regression tests for _execute_tool_calls: an MCP server whose alias differs from its server_name must still have its tool-name prefix stripped correctly, and a tool called by its configured display name must resolve back to the original tool name before dispatch. * fix(mcp): validate tool display names against Bedrock's tool-name pattern A display name replaces the tool name sent to the LLM provider, so a value with spaces or other special characters saves successfully but fails every subsequent Bedrock tool call. Validate tool_name_to_display_name server-side (create/update payload) against Bedrock's [a-zA-Z0-9_-]+ constraint, and add matching inline validation plus a save-blocking guard in the Admin UI's create and edit MCP server forms. * style(mcp): fix ruff/prettier formatting on CI No logic changes; satisfies the format checks flagged on PR #32320. * fix(mcp): fix CI failures on PR - complexity budget and stale test mock Extract toolset-scope resolution and query-param normalization out of list_tool_rest_api into helpers to bring it back under the C901 complexity budget (was 18, now within the 15 threshold). Add the missing get_mcp_server_by_name stub to the streaming iterator test's mock manager; the alias-fallback resolution added for tool-name-prefix stripping calls it unconditionally when _get_mcp_server_from_tool_name misses. * test(mcp): cover BYOK OpenAPI auth-header helpers to close codecov patch gap _format_byok_openapi_auth_header, _openapi_forwarded_extra_headers, and _resolve_byok_mcp_auth_header were only exercised indirectly via a mocked call_tool test, leaving their branches (auth-type formatting, header forwarding/stripping, missing-credential 401) uncovered. * fix(mcp): resolve BYOK auth before queuing the during-hook task _resolve_byok_mcp_auth_header can raise a 401 when no credential is stored. Resolving it after during_hook_task was already queued meant a hook's side effects (audit logging, rate-limit bookkeeping) could run and record success for a tool call that then fails on the missing credential. * fix: correct mcp alias routing regressions --------- Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
tin-berri
|
5e73994441
|
fix(mcp_semantic_filter): keep tool names whole in filter response header (#32282)
The x-litellm-semantic-filter-tools response header was sliced mid-name at MAX_MCP_SEMANTIC_FILTER_TOOLS_HEADER_LENGTH with a trailing "...", so the admin UI test panel rendered the last selected tool name chopped. Truncate the CSV at a tool name boundary instead so the header only ever carries complete names, and note in the test panel how many selected tools did not fit in the header |
||
|
tin-berri
|
4e3ebbb164
|
feat(mcp): startup backfill stamping oauth2_flow on legacy null rows (#32290)
* feat(mcp): startup backfill stamping oauth2_flow on legacy null rows Rows created before the write-side stamps carry a null oauth2_flow and rely on read-time field-shape inference, which cannot tell a DCR-registered interactive server (client creds + token_url, no persisted authorization_url) from an M2M server unless endpoint discovery succeeds first; on a transient discovery failure those servers flip to client_credentials for that registry load The backfill classifies each null oauth2 row once, at rest, ordered by signal strength: per-user token rows (only the interactive flow mints them, so this is definitive and catches the DCR-trap cohort), then a persisted authorization_url, then a persisted registration_url (DCR implies interactive; this covers registered-but-never-signed-in rows), then the M2M credential shape mirroring the legacy inference, else the interactive default that matches how needs_user_oauth_token treats a null flow. Every stamp is logged with the rule that fired and written with updated_by=oauth2_flow_backfill for auditability Runs in _init_mcp_servers_in_db before the registry load so the first build of the boot classifies from the column, is isolated so a failure cannot block server loading, and is idempotent: a healed fleet exits after one indexed query. This unblocks deleting the read-time inference for DB rows in the follow-up Third step of the oauth2_flow persistence sequence, after #32283 and #32288 * fix(mcp): backfill leaves the ambiguous M2M shape unstamped instead of guessing client_credentials The credential shape (client_id + client_secret + token_url, no interactive signal) is shared by real M2M servers and DCR-registered interactive servers nobody has signed into: the DCR persist writes creds and token_url but not authorization_url or registration_url. Stamping client_credentials from that shape permanently mislabeled the interactive cohort, and once explicit the value is authoritative, so per-user traffic would run on the proxy's stored client credential with no discovery rescue and no backstop (it only guards null rows) The backfill now stamps only what it can prove. Interactive signals keep stamping authorization_code; the ambiguous shape is left null with an actionable warning naming the server and the fix (set oauth2_flow via the dashboard or PUT /v1/mcp/server). A true M2M row keeps working per-request through the security backstop while the warning nags; an interactive row keeps its Authorize button (null renders interactive), and one completed sign-in creates the per-user token that stamps it authorization_code at the next boot. Mirrors the config-level rule: M2M is asserted by a human, never guessed Raised by review on the PR * perf(mcp): batch the backfill stamps into one update_many per flow value The per-row update loop issued one DB round-trip per legacy row at startup; rows sharing a stamped value now go out as a single update_many, so the DB cost is constant in fleet size. Per-row logging keeps the rule that fired for each server Raised by review on the PR * fix(mcp): backfill stamps only rows still null at write time and counts only real OAuth token rows as sign-in proof Two review findings. The batched update_many matched on server_id alone, so an explicit oauth2_flow set between the backfill's read and its write (an admin PUT or a sign-in's DCR stamp landing in the boot window) would be overwritten with the inferred value; the where clause now also requires oauth2_flow to still be null, so an explicit value can never be clobbered under any interleaving And the per_user_tokens rule counted any LiteLLM_MCPUserCredentials row as proof of an interactive sign-in, but that table doubles as BYOK storage for user-supplied API keys; a BYOK-flavored row would have stamped an M2M-shaped server authorization_code. The rule now counts only rows whose payload decodes as a type oauth2 token via the existing _decode_oauth_payload discriminator, so bare keys, undecodable rows, and stale leftovers from a BYOK-to-oauth2 auth switch prove nothing Raised by review on the PR |
||
|
tin-berri
|
fc3c21e837
|
fix(mcp): forward short OAuth state upstream, keep session in a cookie (#32146)
* fix(mcp): forward short OAuth state upstream, keep session in a cookie Some upstream authorization servers reject the OAuth authorize request with "state parameter too long" because LiteLLM replaced the client's short state with its own long encrypted session blob (base_url, original state, PKCE, client redirect_uri) and sent that upstream as state. Forward a short random handle as the upstream state instead, and carry the encrypted session in a per-flow HttpOnly, SameSite=lax cookie bound to that handle. The browser replays the cookie on /callback, so the session is recovered without any server-side store and the client still gets its own original state back. /callback falls back to decoding state directly when no cookie is present, so flows in flight across a deploy keep working. Resolves LIT-4197 * test(mcp): cover /callback error path cookie read and clear The happy-path regression test already asserts the short-handle -> cookie round trip. Add a focused test for the IdP-error branch of /callback: it must recover the client's original state from the per-flow cookie (not the short handle), propagate the error to the client's redirect_uri, and expire the one-time cookie. Fails if the error path stops reading or clearing the cookie. |
||
|
tin-berri
|
f5ea72b1b8
|
fix(mcp): stamp oauth2_flow=authorization_code when persisting a DCR client registration (#32283)
Only the gateway-managed interactive flow reaches this persist (the public /register routes never pass persist_credentials), so the row it writes is authorization_code by definition. It was not recorded, which left the row as client creds + token_url with no persisted authorization_url and a null oauth2_flow: exactly the shape the legacy M2M inference in _resolve_oauth2_flow matches. The row normally survives because endpoint discovery backfills authorization_url in memory before the inference runs, but on any transient discovery failure at registry build the server flips to client_credentials for that load, routing per-user traffic to the M2M path Stamping the flow at the write site makes the classification explicit and permanent, so a DCR-registered interactive server no longer depends on discovery succeeding to classify correctly. First step of persisting oauth2_flow at every write site so the legacy inference can eventually be deleted |
||
|
tin-berri
|
2e38da6b3e
|
feat(mcp): add entra_obo profile to the token_exchange (OBO) arm (#31983)
* feat(mcp): add entra_obo profile to the token_exchange (OBO) arm
Microsoft Entra On-Behalf-Of uses the RFC 7523 jwt-bearer grant rather than RFC 8693, so the existing token_exchange arm cannot mint tokens against Entra. This adds an entra_obo profile on TokenExchangeConfig that switches the request form to Entra's jwt-bearer OBO dialect (the inbound token as assertion, the target resource carried in scope, and requested_token_use=on_behalf_of), while reusing the shared caching, single-flight, TTL, and fail-closed machinery. The exchanger is renamed from Rfc8693TokenExchanger to OboTokenExchanger since it now serves both dialects
The profile is threaded through the config-load path, the DB credentials blob, and the MCPCredentials request schema, so an operator can select it from YAML or the management API. It rides in the existing credentials JSON blob, so there is no new auth_type and no DB migration
Resolves LIT-4163
* feat(mcp): propagate the Entra Conditional Access step-up challenge on the OBO 401
An entra_obo exchange that the IdP rejects for Conditional Access returns a 4xx with
error=interaction_required and a claims blob the client must satisfy to step up. The arm
dropped both and emitted a static RFC 9728 challenge, so a CA-protected Entra upstream was
unreachable through the gateway. The provider now reads the RFC 6749 error code and the claims
string off the rejection body (error_description is still never carried; it can leak IdP
internals), threads them through SubjectTokenRejected -> CredError.unauthorized, and the
challenge builder folds them into WWW-Authenticate: the machine error only when it is a plain
OAuth token (guards against header injection from a hostile body) and the claims base64-encoded
in a claims parameter, the convention MSAL-family clients decode. With neither field the header
is byte-identical to the static challenge. The multi-server aggregate still absorbs a
step-up 401 to an empty listing; only single-server routes surface it
* fix(mcp): use error=insufficient_claims for the Entra step-up challenge
Per Microsoft's claims-challenge format, a WWW-Authenticate carrying a claims challenge must set
error=insufficient_claims (the value MSAL-family clients key on to recognize the challenge and
replay the claims), not the raw token-endpoint code. The challenge now sets insufficient_claims
whenever a claims blob is present and keeps invalid_token otherwise, with a step-up-accurate
error_description in the claims case. The presence of claims now drives the error value, so the
raw oauth_error no longer needs threading from the provider through CredError to the edge; that
plumbing is removed (the provider still reads the error code for its gateway-fault classification).
Both the error value and the base64 claims are fixed-alphabet, so nothing from the IdP body reaches
the header unescaped. Cross-checked field-by-field against Microsoft Learn; a bogus jwt-bearer OBO
POST to the real login.microsoftonline.com/common endpoint confirmed Entra recognizes the grant and
returns the error shape the parser reads. Follow-up: also emit authorization_uri alongside
resource_metadata for strict non-MCP MSAL clients (RFC 9728 resource_metadata already serves MCP)
* fix(mcp): filter blank scopes on the config-load path so entra_obo fails closed
The DB-build path already runs YAML/DB scopes through _extract_scopes (which drops blanks), but the
config-load path read server_config["scopes"] raw. A YAML `scopes: [""]` therefore reached the
exchanger as a ("",) tuple: non-empty, so the entra_obo `not config.scopes` precondition skipped its
fail-closed misconfigured path and the form builder POSTed an empty scope to the IdP instead of
failing before any network call. Config-load now filters blanks the same way, so an all-blank list
normalizes to None and the precondition fails closed. Regression test loads a blank-scope entra_obo
server and asserts the exchange returns misconfigured without POSTing
|
||
|
tin-berri
|
c737789b26
|
feat(mcp): discover the OBO token endpoint via RFC 9728 -> RFC 8414 (no IdP guessing) (#31762)
* feat(mcp): discover the OBO token endpoint via RFC 9728 to RFC 8414 (no IdP guessing) An oauth2_token_exchange server can now have its token endpoint discovered the same way the oauth2 (authorization_code) flow already does, instead of always requiring token_exchange_endpoint/token_url to be configured by hand. The existing _descovery_metadata chain (RFC 9728 protected-resource metadata -> RFC 8414 authorization-server metadata -> token_endpoint, SSRF-guarded via async_safe_get) is reused; both the config-load and DB-build paths gate on a new _obo_needs_endpoint_discovery so discovery runs only when no endpoint is configured, and an explicitly configured endpoint still wins and skips the round-trip. The discovered token endpoint lands on token_url, which _token_exchange_spec already reads, so no resolver change is needed. _resolve_oauth2_flow returns None for any non-oauth2 auth_type, so a discovered token_url on an OBO server is never mis-inferred as the M2M client_credentials flow. Discovery for OBO is authoritative only: the resolution order is explicitly configured endpoint, then RFC 9728 -> RFC 8414 advertisement, then fail closed (412, on the parent commit). The gateway never guesses the IdP. _descovery_metadata grows an allow_origin_fallback flag, kept True for the browser oauth2 flow (a human sees the redirect) but set False for token_exchange so the last-resort guess that treats the resource server's own origin as its authorization server is skipped; a subject token is never exchanged against an inferred endpoint. * fix(mcp): surface a failed OBO exchange at connect instead of an empty tool list A token_exchange server whose exchange fails with a subject present used to open the MCP session anyway and mask the failure as an empty tools/list. Single-server routes now run the exchange preemptively at the transport edge, where a rejected subject raises the RFC 9728 challenge and a gateway fault its public status; the multi-server aggregate keeps absorbing per-server auth failures. The exchanger caches the preflight result, so the session's list/call reuses it with no extra IdP round-trip. Discovery now also debug-logs the authorization server's advertised issuer, grant types, and client auth methods * fix(mcp): persist the discovered OBO token endpoint to the DB row A DB-backed oauth2_token_exchange server with no configured endpoint had its token_url resolved via RFC 9728 -> RFC 8414 only on the in-memory object returned from build_mcp_server_from_table; the row kept token_url=None, so every rebuild re-ran discovery and a transient upstream outage during a rebuild left the server with no endpoint until the next successful discovery. Write the discovered token_url back onto the row so the guard sees it on the next build. Best-effort and scoped to DB servers: config servers already persist in-memory, and the write-back never fires from a user connect (only from add/update/reload, all admin or system driven). Adds DB-path coverage for discovery firing when unset, skipping when the credentials endpoint is configured, the write-back, and its negative guards |
||
|
tin-berri
|
0e56fc39e2
|
feat(mcp): make token_exchange (OBO) production-ready - discovery threading + audit hardening + RFC 9728 challenge (#31622)
* feat(mcp): thread the caller token into tools/list discovery for token_exchange
A token_exchange (OBO) server's tools could not be discovered through the aggregator: the list path
never threaded the caller's token, so every tools/list hit the no-subject branch. v1 masked this with
its client_credentials fallback (discovery used a service token); v2 dropped that fallback, so listing
had no credential and the OBO server's tools never appeared - and an MCP client lists before it calls.
Thread the inbound subject_token into the list path the same way the call path does, gated on
auth_type oauth2_token_exchange so the caller's bearer never leaks into other modes:
_get_tools_from_server takes an oauth2_headers param, extracts the token via _extract_bearer_token, and
passes it to _create_mcp_client; server.py forwards oauth2_headers at the list call site.
authorization_code (resolves off identity plus stored token), the static/config modes, and the
background registry refresh are unaffected, and the list path's existing graceful degradation
(catch -> empty list) is preserved.
* fix(mcp): harden token_exchange OBO from the audit (strip, TTL/expires_in, subject_token_type)
- _should_strip_caller_authorization returns True for oauth2_token_exchange, so the inbound subject
token is never forwarded upstream raw - only the IdP-exchanged token is (matches authorization_code).
- _parse_expires_in accepts a JSON float / numeric-string expires_in, and _ttl_seconds caps the cache
TTL at the token's real remaining lifetime so a short-lived exchanged token is never served stale.
- to_server_spec normalizes a falsy subject_token_type to the default URN, parity with v1.
The subject/key disambiguation (never exchange the LiteLLM key; Authorization: Bearer <litellm-key>
support for /mcp) is intentionally a separate cross-cutting PR off staging, not part of this OBO work.
* fix(mcp): stop caller header bypassing OBO exchange; thread subject into prompts/resources
The per-server x-mcp-* override guard in _create_mcp_client only kept the v2 spec
for authorization_code, so a caller-supplied header silently disabled the RFC 8693
exchange on a token_exchange server and forwarded the raw bearer upstream. Extend
the guard to token_exchange so the exchange always runs and the caller cannot
substitute an arbitrary upstream credential.
prompts/list+get, resources/list+read, and resource-templates/list never threaded
the OBO subject token, so those operations failed closed (401 / empty) on a
token_exchange server. Thread the caller's bearer as the subject for those paths
too, gated on the token_exchange mode via a shared _obo_subject_token helper.
* fix(mcp): keep the OBO/authz_code resolver credential authoritative; centralize OpenAPI strip
A guardrail (e.g. MCPJWTSigner), static_headers, or any other injected Authorization could
shadow the resolver-owned credential for token_exchange / authorization_code servers, so the
upstream would receive e.g. the signer's JWT instead of the exchanged token and reject it. In
_create_mcp_client the resolver-owned credential now wins: a conflicting header is dropped and
the minted/stored token reaches upstream. No behavior change for none/passthrough/static modes,
where an injected Authorization still wins as before.
The OpenAPI/local _request_extra_headers forwarder gated its Authorization strip on
has_client_credentials only, so an OpenAPI-backed token_exchange server with
extra_headers:[Authorization] forwarded the raw subject token upstream and never exchanged. It
now uses the centralized _should_strip_caller_authorization so it matches the managed paths.
* feat(mcp): RFC 9728 challenge for token_exchange (OBO) unauthorized
OBO previously returned an opaque 401 (Bearer error="invalid_request") with no discovery
info, and any IdP exchange failure collapsed to a retryable 503. Now an OBO server behaves like
a standards-compliant OAuth resource server:
- A missing/rejected subject token returns the RFC 9728 / RFC 6750 challenge: 401 +
WWW-Authenticate: Bearer resource_metadata="...", error="invalid_token", so a spec-compliant
MCP client can discover the IdP, SSO, and retry with a fresh subject token.
- The protected-resource metadata for a token_exchange server advertises the JWT-auth issuer(s)
(JWT_ISSUER / litellm_jwtauth.issuers) as authorization_servers -- the IdP that issues and
validates the subject -- instead of the gateway.
- An IdP 4xx (subject rejected) is now a non-retryable 401 (the challenge) instead of a 503, so a
caller with a dead token re-authenticates rather than looping; 5xx/transport stays retryable 503.
* fix(mcp): emit the OBO RFC 9728 challenge preemptively so a no-subject client can discover the IdP
A token_exchange server's tools are not discoverable without a subject token (list is lenient ->
empty), and a tool-call-time 401 is wrapped into a JSON-RPC error so the WWW-Authenticate header is
lost. So a cold-start client never saw the challenge and could not start discovery. Add a
token_exchange branch to the preemptive-401: a no-subject connect to an OBO server now returns
401 + WWW-Authenticate: Bearer resource_metadata=..., error="invalid_token" at the transport level,
so a spec-compliant client discovers the IdP (the PRM advertises the JWT-auth issuer), SSOs, and
retries with a subject token. Verified live on the per-server endpoint; the with-subject connect
still proceeds (no challenge).
(Also formats two lines from earlier commits in this stack.)
* refactor(mcp): inject root_path into the OBO/OAuth challenge edge
The adapter's raise_user_oauth_challenge and raise_token_exchange_challenge
reached into os.getenv("SERVER_ROOT_PATH") via get_server_root_path(), a
hidden ambient read in a module that is meant to be a pure edge. That coupling
made the preemptive-challenge test order-dependent under xdist: a sibling test
sets SERVER_ROOT_PATH at import without cleanup, leaking the prefix into the
challenge URL and failing the exact-match assertion.
Resolve the root path at the imperative-shell call sites and pass it in
keyword-only, so both challenge builders become pure functions of their inputs.
Extract the shared resource_metadata path construction into a single
oauth_protected_resource_path helper, collapsing the duplicated prefix/name
logic the two functions carried.
Also reduce _create_mcp_client below the strict complexity ceiling by extracting
the v2 credential resolution into _resolve_v2_auth, and extract the OBO
protected-resource-metadata branch into _obo_protected_resource_response (which
shipped without coverage) so discovery can be unit-tested directly.
Tests are now hermetic: the adapter tests pass root_path as a real input rather
than monkeypatching the environment, the stale-session preemptive test asserts
structural invariants instead of the exact prefixed URL, and five new tests
cover the OBO PRM issuer branch end to end.
* feat(mcp): OBO cache-key tenant isolation, reactive 401 retry, v1-parity logs
From a pass over the OBO behavior contract. Three changes to the
token_exchange arm, none of which alters any other auth mode.
The exchanged-token cache key now folds in the caller's tenant alongside
the subject token and exchange config, so two tenants presenting the same
opaque token can never share a cache entry; cross-tenant isolation is
structural rather than incidental to subject-token uniqueness. tenant_id is
threaded from the resolver's Subject; it is keyword-only with an empty
default so the no-tenant case and the existing call sites are unchanged.
The tool-call path gains one reactive retry. When an upstream rejects the
injected token with a 401/403, the gateway invalidates the cached exchange,
re-mints once through the IdP by rebuilding the client, and retries the call
exactly once before surfacing the upstream error, so a token revoked or
rotated upstream mid-TTL self-heals without an infinite loop. It is gated
strictly to oauth2_token_exchange; passthrough, authorization_code,
client_credentials, api_key, and none keep their single-call behavior.
MCPClient.call_tool gains a raise_on_error flag (mirroring list_tools) so
the path can tell an upstream 401 apart from an ordinary tool error and
avoid re-running a non-idempotent tool on a non-auth failure.
The exchanger also emits the v1-parity log lines it had dropped (attempt
with server, endpoint and audience; success; cache hit), while never
logging the form, subject token, secret, or minted token.
* fix(mcp): fail closed with 412 when a token_exchange server has no endpoint
A true token_exchange (OBO) server must use only an explicitly configured
token endpoint; it must never guess an IdP or silently fall back to a weaker
source. Previously an OBO server with client credentials but no
token_exchange_endpoint/token_url deferred to v1, which no-op'd and let the
request connect to the upstream with no credential (an upstream 401 rather
than a clear gateway error).
Now such a server is owned by the v2 arm: _token_exchange_spec builds the spec
even when the endpoint is absent, and the exchanger fails closed with a
precondition_required error that maps to HTTP 412 before any upstream or IdP
call, with the caller's subject token never sent anywhere. A missing
client_id/secret still maps to misconfigured (500); a present-but-rejected
subject still maps to 401; an unreachable IdP still maps to 503. The no-subject
case keeps its existing 401 RFC 9728 challenge.
* feat(mcp): log a refused non-Bearer token_type in the OBO exchange
* fix(mcp): surface OBO/authorization_code list-time 401 as a challenge instead of masking it
* feat(mcp): classify RFC 6749 gateway-fault token-exchange errors as 500, not a caller 401
* test(mcp): absorb fixture uses 500 now that 401/403 are challenge-class at list time
* style(mcp): PEP 604 union in the OBO retry signature to keep the UP007 budget flat
|
||
|
tin-berri
|
f19bf2c984
|
feat(mcp): migrate the token_exchange (OBO) arm to the v2 resolver (#31526)
* feat(mcp): v2-native RFC 8693 token exchanger for the token_exchange mode Adds the pure Rfc8693TokenExchanger plus its composition root: the OBO exchange POSTs the RFC 8693 grant through an injected HTTP edge and returns the upstream-bound token as a typed Result, caching and single-flighting per (subject_token, server) so a repeated caller token skips the IdP round-trip. The audience is carried on TokenExchangeConfig and sent only when the operator set one, matching the spec default behavior. Errors are values: a missing endpoint or client credential is misconfigured, an IdP that returns no usable token is upstream_unavailable. * feat(mcp): migrate the token_exchange arm to the v2 resolve_credentials Routes RFC 8693 OBO servers through the v2 resolver: the resolver arm reads the caller's inbound token and swaps it via the injected TokenExchanger, to_server_spec maps a complete oauth2_token_exchange server (endpoint plus client credentials) to TokenExchangeConfig, and the egress wires the LazyTokenExchanger in. A token_exchange server with no caller token fails closed with a plain 401 rather than v1's fall-through to client_credentials, so the call site now scopes the per-server browser-OAuth challenge to authorization_code and lets other modes raise their own. * fix(mcp): bind the token-exchange cache key to the exchange config The exchanged-token cache was keyed only by (subject_token, server_id), so rotating a server's audience, scope, endpoint, client_id, or secret kept serving a token minted for the old config until TTL. The key now hashes the caller token together with the config that minted it, so a config change forces a fresh exchange. Everything is hashed, so no secret is held in the key. * refactor(mcp): build the token exchanger eagerly, dropping the lazy wrapper The token exchanger reads no runtime global at build time (its httpx client is acquired per call), unlike the per-user store, so it does not need lazy first-use construction. Building it once at egress construction removes the first-use init path entirely and keeps the process-lifetime cache. * fix(mcp): map non-object token-exchange JSON to a miss instead of a 500 The post adapter annotated the parsed body as a dict without checking it, so a valid-but-non-object JSON response (list/string/number) was returned as-is and crashed the field parsing with an AttributeError. It now validates the shape at the boundary and returns None for a non-object body, so a malformed IdP response surfaces as a typed upstream_unavailable rather than a server error. * fix(mcp): fail closed on a non-Bearer token_type in the OBO exchange * feat(mcp): honor token_endpoint_auth_method (client_secret_basic) in the v2 OBO exchange * feat(mcp): reject a non-access issued_token_type in the OBO exchange * fix: include token endpoint auth method in exchange cache key --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> |
||
|
ryan-crabbe-berri
|
57ca48a863
|
feat(mcp): add all-proxy-mcpservers sentinel to grant teams every MCP server (#32012)
* feat(mcp): add all-proxy-mcpservers sentinel to grant every MCP server Teams can now be scoped to the all-proxy-mcpservers sentinel so they gain access to every MCP server on the proxy without listing each id. The sentinel expands to the live registry at request time, so a server added later is picked up with no change to the team's stored permission. The team ceiling that validates a key's MCP scope expands the sentinel too, so a key can be scoped to any server (including one registered after the team) and still pass subset validation Expose the option in the team create and edit forms via a new exclusive "All Proxy MCP Servers" choice in MCPServerSelector, mirroring the existing "No MCP Servers" sentinel * Update litellm/proxy/management_helpers/object_permission_utils.py Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> * fix(mcp): honor all-proxy-mcpservers only on the team path, never per-key The sentinel was expanded inside the shared expand_permission_list, which also feeds the key, org, end_user and agent resolvers. A key whose stored object_permission ever held all-proxy-mcpservers (a stale write, a configured default, or a bug) would silently resolve to every MCP server at runtime, and a teamless key had nothing to cap it, so all servers got injected. Only write-time validation stripping the value stood between that value and a full grant Move the expansion out of expand_permission_list and into _get_allowed_mcp_servers_for_team so the sentinel is honored only where it is settable (a team). Anywhere else it now passes through as an inert literal that matches no registered server and is denied downstream. Reserved-id protection already blocks a real server from taking that id * fix(mcp): require proxy admin to grant a team the all-proxy MCP sentinel Granting a team every MCP server on the proxy is a proxy-wide authorization decision, but team create/update let any caller who can manage a team set object_permission.mcp_servers, with no ceiling check. Org admins reach /team/update by default (org_admin_allowed_routes) and _verify_team_access also admits team admins, so a non-proxy-admin could set all-proxy-mcpservers and self-grant their team access to every MCP server on the proxy, including servers never assigned to that team Gate the grant in new_team and update_team: a non-proxy-admin cannot add the all-proxy-mcpservers sentinel. The check is scoped to newly adding it, so a team a proxy admin already scoped to all-proxy can still be edited by a team admin without being forced to strip the sentinel. The UI only offers the "All Proxy MCP Servers" option to proxy admins in the team create and edit forms * fix(ui): render friendly all-proxy MCP label for non-admins editing an all-proxy team A team scoped to the all-proxy-mcpservers sentinel could be opened in the team edit form by a team admin or org admin (canEditTeam admits them), but the "All Proxy MCP Servers" option in MCPServerSelector was rendered only behind the proxy-admin-gated allowAllProxyMcpServers flag. For a non-proxy-admin the stored sentinel was hydrated into the selected value with no matching Select.Option, so antd showed the raw all-proxy-mcpservers literal as a chip, and adding another server could persist a mixed [all-proxy-mcpservers, <id>] value. Render the option whenever the sentinel is present in the value, not only when the caller may grant it, and drive the real-option disabling off presence too so the selection stays exclusive. A non-proxy-admin now sees the friendly label read-only and cannot build a mixed state; only a proxy admin can newly add it, which the backend already enforces. Adds regression tests: the selector shows the friendly option (not the raw literal) when the sentinel is stored but the grant flag is off, plus exclusive emit and disabled-real-options coverage, and MCPServerPermissions renders the green "All" state instead of the raw sentinel string. * fix(ui): drop redundant "All servers" hint from the all-proxy MCP chip antd renders a Select option's children inside the selected tag, so the all-proxy option showed both "All Proxy MCP Servers" and the green "All servers" type-hint in the chip, which say the same thing. Collapse the option to a single green "All Proxy MCP Servers" label so the dropdown row and the chip read cleanly without the duplication. * fix(ui): color the all-proxy MCP label blue to match server chips Use the same blue (#1890ff) as regular MCP server entries for the "All Proxy MCP Servers" option/chip instead of green. * fix(ui): make the all-proxy MCP permissions display blue, not green Match the blue used by the selector chip and regular server entries so the "All Proxy MCP Servers" badge and row in MCPServerPermissions are consistent across the team/key/org detail views. The red "Blocked" state for no-mcp-servers is unchanged. --------- Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> |
||
|
tin-berri
|
15ff389eb4
|
fix(mcp): persist DCR client_id so interactive OAuth token refresh works (#31912)
* fix(mcp): persist DCR client_id so interactive OAuth token refresh works Interactive authorization_code MCP servers register an OAuth client via Dynamic Client Registration (RFC 7591) during the authorize flow, but the minted client_id and the discovered token_url were returned to the caller and never written to the server row. The autonomous refresh_token grant reads client_id, client_secret and token_url off the server, so an expired access token could not be refreshed; the user was bounced back to re-authorize and tools/list returned zero tools Persist the DCR client_id (plus client_secret and token_endpoint_auth_method when the registration returns them) and the discovered token_url onto the server row, reusing the encrypt_credentials write that client_credentials and token exchange already use, then refresh the in-memory registry so the value is live at refresh time. Both the v1 refresher and the v2 AuthorizationCodeRefresher read those same fields, so egress needs no change * fix: reuse persisted MCP DCR clients * fix: reuse persisted MCP DCR clients --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> |
||
|
tin-berri
|
b9df7fa705
|
fix(mcp): surface tools/list 401 auth failures as a challenge on single-server routes (#31921)
A 401 while listing tools (a missing or expired per-user OAuth token, or an upstream 401 for any auth_type) was swallowed to an empty tool list, so a single-server client got a 200 with no tools and no WWW-Authenticate challenge instead of a 401 it could re-authenticate against. Only oauth pass-through and delegate-to-upstream oauth2 servers surfaced it; every other auth_type, and the missing-token case for all of them, masked it. The surface-vs-absorb decision now keys on the route, not the auth_type. An upstream 401 in _fetch_tools_with_timeout becomes an MCPUpstreamAuthError regardless of auth_type, and the per-user OAuth challenge raised during client creation (a bare HTTPException 401 carrying a WWW-Authenticate header) is converted to the same type in _get_tools_from_server. The challenge is scoped to 401: a 403 (authenticated but forbidden, e.g. insufficient scope) is not a re-auth signal and degrades to an empty list like any other non-auth error, and the stdio-allowlist 403 (no challenge header) stays absorbed. The existing routing then does the right thing: single-server routes turn the error into a 401 + WWW-Authenticate, while the multi-server aggregator absorbs it to an empty list so one unauthenticated server does not fail the whole listing. On the UI tools page, an OBO (per-user authorization_code) server now shows the Authorize gate when the list call returns 401, not only when no credential row exists. The backend already refreshes a still-refreshable token on the list call, so a 401 means there is no valid token and none could be minted (expired with no usable refresh token), which is exactly when the user must reauthorize. |
||
|
tin-berri
|
58de920921
|
feat(mcp): bound outbound tool-call concurrency per MCP server (#31641)
Add an optional per-server max_concurrent_requests that caps how many tool calls LiteLLM sends to one MCP server at once, so batch-processing backends are not overwhelmed by unbounded parallel dispatch. Excess calls queue on a per-server asyncio.Semaphore instead of being rejected. Unset or non-positive means unlimited, preserving existing behavior. Resolves LIT-2749 |
||
|
tin-berri
|
c370503091
|
fix(mcp): gate OAuth authorize/token/register/discovery on auth_type=oauth2 (#31736)
Some checks are pending
GitHub Actions Security Analysis / zizmor (push) Waiting to run
* fix(mcp): gate OAuth authorize/token/register/discovery on auth_type=oauth2 A non-oauth2 MCP server (notably auth_type=none, access-group gated) has no client_id and no authorization URL, yet the gateway OAuth endpoints did not check auth_type. authorize() raised "client_id is required" before the auth_type was ever examined, and the .well-known discovery builders always advertised authorization_servers / authorization_endpoint / token_endpoint / registration_endpoint, so spec-compliant MCP clients were pointed at an OAuth flow that can never succeed. Add an auth_type != oauth2 guard to the authorize, token, register, protected-resource and authorization-server paths (covering the internal UI OAuth endpoints too). The discovery guard sits after the OAuth pass-through branch so genuine pass-through servers keep proxying their upstream metadata. oauth2 servers are unaffected. * fix(mcp): accurate non-oauth2 message; 404 unknown discovery names to close enumeration oracle Address review feedback on the auth_type gate. The 400 message no longer claims access is governed by access groups, which is only true for auth_type=none; it now states that the gateway runs the OAuth client_id/authorize/token/register flow only for oauth2 servers and that the server is reached using its configured auth_type, which is accurate for every non-oauth2 type (api_key, oauth2_token_exchange, etc.). The discovery gate previously 404'd a named non-oauth2 server but still returned 200 metadata for an unknown name, which both serves a broken document for a typo and lets an unauthenticated caller enumerate non-OAuth server names by comparing 404 vs 200. A named discovery request now returns 200 only when it resolves to an oauth2 server; unknown (or hidden) and non-oauth2 names return the same 404. Root discovery and pass-through servers are unaffected. * Apply suggestions from code review Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> --------- Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> |
||
|
Sameer Kankute
|
fabe5c283a
|
fix(mcp): roll up MCP tool spend to user counters and usage UI (#31576)
* fix(mcp): roll up MCP tool spend to user counters and usage UI Direct REST MCP tool calls now fire success logging so spend_logs and user/team rollups include configured mcp_server_cost_info charges. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(mcp): gate key-info enrichment to requests missing user_id; fix import order - Only call _enrich_failure_metadata_with_key_info when user_api_key_user_id is absent, avoiding a cache/DB lookup on every normal LLM request. - Move LiteLLMProxyRequestSetup import to correct alphabetical position (I001). Co-authored-by: Cursor <cursoragent@cursor.com> * fix(mcp): scope MCP spend aggregate by api_key to prevent cross-tenant disclosure Add api_key = ANY($2) to the MCP session aggregate query so it is bounded by the same ownership already applied to the main page query. Co-authored-by: Cursor <cursoragent@cursor.com> * Fix spend logs for call and list mcp tools * Add tags in mcp logging * Fix ruff * fix(lint): replace List/Dict with list/dict in new annotations (UP006) Replace the 8 new UP006 violations introduced by the mcp-tags changes: - Optional[List[str]] → Optional[list[str]] for request_tags params - List[str] return type → list[str] in _get_parent_request_tags - Dict[str, Dict[...]] → dict[str, dict[...]] for mcp_spend_map annotation Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lint): keep call_tool_rest_api within complexity budget and narrow MCP spend enrichment except to PrismaError * fix(mcp): keep final streaming chunk when draining inner stream fails * fix: handle MCP logging edge cases * fix: propagate MCP logging cancellation --------- Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|
Sameer Kankute
|
b96f1aa686
|
fix(mcp): byom visibility, preview UX, and admin settings gating (#31809)
* fix(ui): show info message when MCP tool preview returns 403
Internal users submitting MCP servers hit an admin-only preview endpoint; replace the red connection error with a clear review notice while leaving other failures unchanged.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(mcp): let BYOM submitters see their approved servers
Approved user-submitted MCP servers defaulted to no access groups and allow_all_keys=false, so submitters could not see them after admin approval. Grant creator visibility for active submissions in get_allowed_mcp_servers.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Improve dialogue box
* fix(security): restrict MCP semantic filter settings to proxy admins
Add an explicit PROXY_ADMIN check on PATCH /update/mcp_semantic_filter_settings
and hide Semantic Filter and Network Settings tabs from non-admin users in
the MCP Servers UI.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(lint): use list[str] instead of List[str] to satisfy UP006 budget
Co-authored-by: Cursor <cursoragent@cursor.com>
* perf(mcp): cache BYOM submitter server lookup with 60s TTL
Co-authored-by: Cursor <cursoragent@cursor.com>
* style: fix ruff format and prettier formatting
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix: preserve approved BYOM server visibility
* fix(mcp): keep no-mcp-servers opt-out absolute and gate BYOM union by key scope
The autofix in
|
||
|
Krrish Dholakia
|
cca71a07c2
|
feat(mcp): add mcp_tool_search virtual tools for large tool catalogs (#31777)
* feat(mcp): add tool search virtual tools for large catalogs
When mcp_tool_search_enabled is set on a key's object_permission,
tools/list returns only mcp_tool_search and mcp_tool_call instead of
the full catalog. The LLM searches by keyword then calls discovered
tools by name, avoiding context bloat with 100+ tool deployments.
* fix(mcp): persist mcp_tool_search_enabled and route tool_call by name
The mcp_tool_search_enabled flag existed on the Pydantic models but the
Prisma schema lacked the column, so keys generated with the flag never
persisted it and tools/list kept returning the full catalog. Add the
column across all three schema.prisma copies plus a migration.
handle_mcp_tool_call passed server_name="" into call_tool, which built a
malformed prefixed name ("-<tool>") and failed to resolve the server.
Resolve the caller's allowed servers and dispatch through execute_mcp_tool
instead, matching how the normal /tools/call path routes.
* fix(mcp): filter list_tools to virtual tools on the protocol path
The REST surface (/mcp-rest/tools/list) returned only the two virtual
tools when mcp_tool_search_enabled was set, but the MCP protocol handler
(handle_list_tools, used by real MCP clients over streamable-http/SSE)
still returned the full catalog. Apply the same early return there so an
actual MCP client sees mcp_tool_search and mcp_tool_call instead of every
tool. call_tool was already intercepted on this path.
* fix(mcp): enforce IP + server filtering on virtual tool search/call
Review flagged that the virtual mcp_tool_search/mcp_tool_call path skipped
access controls the normal MCP flow applies. mcp_tool_call resolved allowed
servers from key permissions only, never applying IP filtering, so a caller
on a public IP could invoke a tool on a server marked
available_on_public_internet: false. mcp_tool_search listed the raw catalog
via global_mcp_server_manager.list_tools, exposing tool names/schemas that
/tools/list would hide and ignoring per-key/per-server tool filters.
Route both virtual handlers through the same filtered paths used by the
normal MCP flow: search now calls _list_mcp_tools and call resolves servers
via _get_allowed_mcp_servers, both threaded with the request client IP so
filter_server_ids_by_ip applies. execute_mcp_tool then enforces the server
allowlist and per-key tool permissions. Thread client_ip through
_list_mcp_tools/_get_tools_from_mcp_servers and pass it from the REST and
SSE call sites.
* fix(ci): ruff format server.py and sync dashboard API types
ruff format normalizes the list_tools client_ip changes in server.py, and
schema.d.ts gains the mcp_tool_search_enabled object-permission field so the
generated dashboard types match the proxy OpenAPI spec.
* style(mcp): drop quoted annotations and sort imports
Clears UP037 on the virtual tool handler signatures (redundant with
from __future__ import annotations) and I001 on the list_tools import block.
* refactor(mcp): extract virtual-tool dispatch and host progress capture
Pulls the mcp_tool_search/mcp_tool_call interception and the host
progress-callback setup out of mcp_server_tool_call into helpers, keeping
that handler under the strict cyclomatic-complexity ceiling after the
client_ip threading. No behavior change.
* test(mcp): cover SSE virtual-tool dispatch and host progress helpers
Adds unit tests for _dispatch_virtual_mcp_tool (non-virtual passthrough,
flag-disabled rejection, search/call routing with client_ip),
_capture_host_progress_callback, and the protocol list_tools virtual
early-return, covering the new server.py paths.
* fix(mcp): forward per-request auth headers through virtual tool handlers
The virtual mcp_tool_search/mcp_tool_call path intercepted the request
before the normal header extraction ran, so client-supplied per-request
auth (Authorization for upstream pass-through, x-mcp-auth-<alias>) was
dropped and execute_mcp_tool/_list_mcp_tools received None. Thread
mcp_auth_header, mcp_server_auth_headers, oauth2_headers, and raw_headers
from both the REST and SSE call sites through the handlers so upstream MCP
servers that require pass-through auth can be listed and called.
* fix(mcp): preserve requested server scope in virtual tool calls
A scoped MCP session (/mcp/<server>/ or header-scoped) carries an
mcp_servers scope that the normal call path passes into routing so the
session can only reach that server. The virtual-tool branch dropped it and
resolved with mcp_servers=None, letting a scoped session call mcp_tool_call
for any server the key can access. Thread the context mcp_servers scope
through _dispatch_virtual_mcp_tool into both handlers so search and call
resolve against the same scoped server set.
* fix(mcp): convert virtual tool errors to isError on the protocol path
The virtual-tool dispatch ran before the protocol handler's HTTPException
and guardrail handling, so a rejected virtual call (e.g. an out-of-scope
403 from execute_mcp_tool) raised out of mcp_server_tool_call and broke the
MCP JSON-RPC stream instead of returning an isError CallToolResult. Move
the dispatch inside the same try that wraps call_mcp_tool so virtual-tool
errors get the same isError conversion as normal tool calls.
* fix(mcp): spend-log virtual tool calls on the REST path
The REST virtual-tool branch returned before common_processing_pre_call_logic,
so execute_mcp_tool ran without a litellm_logging_obj and virtual mcp_tool_call
invocations were not spend-logged or guardrail-checked like normal calls. Run
the same pre-call pipeline in the call branch and thread the resulting
litellm_logging_obj through handle_mcp_tool_call into execute_mcp_tool.
* fix(mcp): reject virtual tool call when key has no accessible servers
handle_mcp_tool_call passed an empty allowed_mcp_servers list into
execute_mcp_tool; an unprefixed local tool name then fell through to the
local registry, which has no server permission check, so a key with only
mcp_tool_search_enabled and no server grants could run operator-configured
local tools by name. Reject with 403 before dispatch when no servers are
accessible, matching call_mcp_tool.
* docs(mcp): document virtual tool_search module and parity rule in AGENTS.md
* style(mcp): apply ruff format at repo line-length (120)
* fix(mcp): add mcp_tool_search_enabled to ObjectPermissionDict and customer test fixture
* chore: trigger CI
* fix(mcp): mirror pre-call pipeline, guard imports, coerce top_k, honor include_disabled_tools
- SSE mcp_tool_call now runs common_processing_pre_call_logic so it spend-logs and runs guardrails like the REST path (P1)
- coerce_top_k avoids ValueError on non-integer top_k from clients (both REST and SSE)
- guard mcp.types import in tool_search behind runtime/TYPE_CHECKING per package convention
- admin list with include_disabled_tools returns the real catalog even when mcp_tool_search_enabled is set
|
||
|
tin-berri
|
87de0e80a8
|
fix(mcp): stop one unauthenticated server from emptying the aggregate tools/list (#31684)
* fix(mcp): stop one unauthenticated server from emptying the aggregate tools/list On the aggregate MCP route (/mcp), the gateway fans out to every server the caller can access and flattens their tools. _fetch_and_filter_server_tools re-raises MCPUpstreamAuthError unconditionally (added with the OAuth passthrough feature in #28356) so it surfaces a 401 on single-server routes, but on the aggregate route that exception propagates through the asyncio.gather fan-out and the outer handler turns it into an empty list. The result: a single delegate/passthrough OAuth server the user has not authenticated (e.g. a delegate-auth server) zeroes the tools of every other server, including the ones that resolve fine, so the client connects and sees no tools. Surface the upstream auth error only when a single server was explicitly targeted (so that route still drives the upstream OAuth flow); across the aggregate, absorb it to [] for that one server so the rest still list their tools. This restores the graceful per-server degradation that predated #28356. Adds regression tests: the aggregate keeps a healthy server's tools when a sibling raises MCPUpstreamAuthError, and a single-server listing still surfaces it. * fix(mcp): decide aggregate vs single-server listing by route scope, not server count Addresses review: keying the surface-vs-absorb decision off the server count (len(allowed_mcp_servers), and even len(mcp_servers)) misclassifies an aggregate /mcp request from a key that can access exactly one server as a targeted single-server listing, so that one server's MCPUpstreamAuthError re-raises and empties the aggregate again for one-server permission sets. Use the path-derived single-server scope instead: _mcp_gateway_server_name, set by _gateway_initialize_instructions_request_scope only when the request path names exactly one upstream server (/<server>/mcp) and never from client headers, is None on the aggregate route (/mcp) regardless of how many servers the key can access. Single-server routes still surface the upstream-auth challenge; the aggregate absorbs it per server. Adds a regression test that an aggregate request with a single accessible server still absorbs, plus renames the single-server test to drive the route scope explicitly. The new test fails on the count-based logic. * fixing aggregation error * style(mcp): collapse single-line debug log to satisfy ruff format |
||
|
ryan-crabbe-berri
|
468d11f71d
|
feat(otel): emit a tools/list CLIENT span for MCP discovery under otel_v2 (#31525)
* feat(otel): emit a tools/list CLIENT span for MCP discovery under otel_v2
Under otel_v2 an MCP tools/call already produced a dedicated CLIENT span, but tools/list produced none. The discovery call surfaced only as the bare POST /{mcp_server_name}/mcp server span with no MCP attributes, indistinguishable from initialize and impossible to query by method
The list success event already reaches the v2 logger with call_type list_mcp_tools, but _emit_mcp_tool_call only matched call_mcp_tool, so listing fell through to the LLM-call path and emitted nothing. This adds a dedicated MCP_LIST_TOOLS span role with its own MCPListToolsSpanData, emitted from a sibling _emit_mcp_list_tools branch that mirrors the tools/call path
Per the OTel GenAI MCP semantic conventions the span is named tools/list (the method name alone, since there is no low-cardinality target), is a CLIENT span parented to the request span, and carries mcp.method.name plus the call id. It deliberately omits gen_ai.operation.name and gen_ai.tool.name, which the convention reserves for tool executions, since listing runs no tool
* fix(otel): anchor MCP spans to params._meta trace context, not the transport span
MCP streamable-HTTP multiplexes many JSON-RPC messages over one session, so the request-root anchor captured on initialize persisted and every later message's span (tools/call, tools/list) nested under it. A tools/list run 44s after the initialize rendered 44s to the right of its parent with a clock-skew warning, because the MCP message and the HTTP transport are independent lifecycles
Following the OTel GenAI MCP semantic conventions, an MCP span now parents to the W3C trace context the client propagated in the request's params._meta (a remote parent, per SEP-414), records the transport/session span as a span link rather than the parent, and starts its own root trace when nothing was propagated. The MCP gateway captures traceparent/tracestate/baggage from each message's params._meta into a per-message contextvar that the otel_v2 emitter reads; opentelemetry stays an optional dependency via guarded lazy imports
This applies to tools/call as well as the new tools/list span, since both shared the same transport-anchoring bug
* fix(otel): drop client baggage from MCP params._meta to prevent identity spoofing
The MCP trace propagation added a W3CBaggagePropagator, so resolve_mcp_span_context
extracted the client's W3C Baggage from params._meta into the span's parent context.
The LiteLLMBaggageSpanProcessor then stamps allowlisted baggage keys onto the span,
and the list-tools/tool-call mappers don't set those identity keys, so nothing
overwrites them. A malicious MCP client could send
params._meta.baggage: litellm.team.id=...,litellm.metadata.user_api_key_user_id=...
and have those identity attributes attributed to its spans.
Extract trace context only (traceparent/tracestate) in the propagator, and stop
collecting the baggage key at the source in _mcp_meta_trace_carrier. Parenting to the
client's trace context, the actual goal, needs only trace context; remote baggage had
no legitimate consumer here. Regression tests at both layers assert a spoofed
params._meta.baggage never lands as a span identity attribute.
* style(mcp): clear ruff strict-budget breach in otel trace-carrier helpers
The otel MCP trace-carrier helpers added in this branch pushed the BLE001 and
UP006 strict-rule totals past their ceilings. Use PEP 585 `dict[str, str]` instead
of `Dict`, and narrow the optional-import guards to `except ImportError` (the only
failure these can hit, matching the "when otel_v2 is unavailable" intent) instead of
a blind `except Exception`.
* fix(otel): stamp authenticated identity baggage onto MCP spans
Parenting MCP spans to the client's params._meta trace context over an empty
Context() meant the tool-call and tools/list spans carried no team/key/metadata
identity at all, so they couldn't be attributed or filtered by team in a traces
backend. The LLM-call span already re-seeds identity from the parsed, authenticated
StandardLoggingPayload rather than trusting ambient/remote context; extract that into
a shared _seed_identity_baggage helper and run both MCP emitters through it.
Identity comes only from the authenticated payload, never the client carrier, so this
keeps the earlier spoofing fix intact while restoring attribution. Regression tests
assert the authenticated team lands on both MCP spans and that a spoofed
params._meta.baggage value can't override it.
* refactor(otel): model MCP spans as roots that link the transport in SPAN_REGISTRY
|
||
|
tin-berri
|
7baf25526f
|
fix(mcp): support client_secret_basic for upstream OAuth token endpoints (#31635)
The MCP gateway authenticated to upstream OAuth token endpoints only with
client_secret_post (client_secret placed in the POST body). Providers that
require HTTP Basic client authentication (client_secret_basic, the OIDC
default) reject that with invalid_client, which surfaced as a 500 on the
/<server>/token exchange and broke both the initial authorization_code
exchange and refresh.
Add a per-server token_endpoint_auth_method ("client_secret_basic" |
"client_secret_post") and a single helper that builds the right headers and
body for the configured method, then route every upstream token-endpoint POST
through it: the inbound exchange and refresh in discoverable_endpoints, the v1
per-user refresh in db, the v2 authorization_code refresher, the M2M
client_credentials fetch, and the RFC 8693 token exchange. The default stays
client_secret_post so existing servers are unaffected; basic sends
Authorization: Basic base64(form-urlencode(client_id):form-urlencode(secret))
per RFC 6749 section 2.3.1 and omits the secret from the body.
client_secret_basic is a confidential-client method, so a server configured for
it with a missing client_id/secret raises rather than silently downgrading to a
body request (no-silent-fallback); the inbound endpoint maps that to a 400 and
the refresh paths to a failed-refresh / needs-reauth. A secretless client_id
under the default method stays valid for public clients authenticating with PKCE.
Resolves LIT-4091
|