Commit graph

624 commits

Author SHA1 Message Date
tin-berri
1af7a403c6
feat(mcp): start the named server's OAuth directly for a resource-scoped gateway flow (#39933)
An aggregate gateway DCR authorize whose RFC 8707 resource resolves to exactly one
gateway-managed oauth2 server sealed that server into the flow and then sent the browser to
the generic connect grid anyway, so the user had to find the server the client had already
named and click Connect.

The connect URL now carries only the flow handle. GET /authorize/flow classifies the sealed
flow as unscoped, interactive, M2M, or stale, and returns the matching state to the page.
Interactive flows require a live per-user vendor credential before minting and do not burn the
flow on an early submit. M2M flows use the gateway's configured service credential and finish
without an interactive OAuth trip. Stale flows fail closed instead of becoming unscoped.
The existing explicit Finish action and a new Cancel path preserve deliberate user intent.
2026-09-07 23:40:02 -07:00
mateo-berri
5e4dec4b88 merge: bring litellm_internal_staging into litellm_fix_agent_mcp_grants
Take staging's test_bedrock_knowledgebase_hook.py, which drops the duplicate
embedding_executor parameter that turned the lint check red, and make the two
cross-module helpers this branch added public (raise_denied_scoped_mcp_access
and routes_through_gateway) so the private-usage budget stays at its base count
2026-09-07 16:35:40 -07:00
tin-berri
cd681a573f
fix(mcp): encrypt stored static headers and stdio environment (#40164)
Encrypt secret maps at the shared persistence boundary, preserve plaintext API/runtime views, and extend rotation and migration scanning to legacy rows.

Co-authored-by: Claude Code <noreply@anthropic.com>
2026-09-07 16:03:06 -07:00
Mateo Wang
0e9e2c01f3
Merge pull request #38806 from BerriAI/litellm_fix_mcp_test_connection_oauth_bearer
fix(mcp): forward staged credentials on /mcp-rest/test/connection like /test/tools/list
2026-09-07 15:18:35 -07:00
Yassin Kortam
192e38fa7b
feat(skills): semantic search over the LiteLLM-hosted skill registry (#39401)
* feat(skills): semantic search over the LiteLLM-hosted skill registry

Adds GET /v1/skills?query= (custom_llm_provider=litellm_proxy) and a
skill_search MCP virtual tool, ranking the caller's accessible skills by
semantic similarity, mirroring the A2A agent registry search (LIT-6309).

Also fixes a pre-existing bug where create_skill() dropped description and
instructions for the litellm_proxy provider, which left every LiteLLM-hosted
skill with no searchable text.

* fix(mcp): coerce skill_search top_k instead of raising 500 on malformed input

The MCP-REST skill_search dispatch validated raw tool arguments through a
pydantic model directly, so a non-numeric top_k raised a ValidationError
that the endpoint's catch-all turned into an HTTP 500. Mirrors the
agent_search branch's tolerant coerce_top_k handling instead.

* fix(skills): enforce key limits on search embeddings and bound the semantic index

Semantic search embeddings now run the same pre_call_hook the /embeddings
route runs, so key rate limits, budgets and guardrails apply before the
embedding model is called. The shared SemanticTextIndex caps cached vectors
and evicts the least recently searched entries, and each skill's embedded
text is capped so one skill cannot inflate the embedding batch

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(skills): surface proxy 429s from search embeddings instead of a 503

ProxyRateLimitError is also an OpenAIError, so the search engine was folding
a key rate limit into skill_search_unavailable. Proxy HTTPExceptions now
propagate so the caller gets the same 429 the /embeddings route returns

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(skills): import assert_never from typing_extensions for Python 3.10

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(skills): embed the request as the pre-call hooks returned it, not the original text

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(skills): keep the litellm_proxy provider check for GET /v1/skills?query= inside llms/

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(skills): move the GET /v1/skills?query= endpoint tests under tests/test_litellm/proxy

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-07 12:28:38 -07:00
Roman D
55fe4a7894
feat(proxy): resolve root_path per request from a configured prefix list (SERVER_ROOT_PATHS) (#35935)
* feat(proxy): resolve root_path per request from SERVER_ROOT_PATHS

One deployment can encode exactly one client-visible URL path prefix
today: SERVER_ROOT_PATH is a scalar stamped onto the app at startup, so
a pod fronting several ingress prefixes 404s every prefix but one before
any handler runs, and MCP OAuth discovery can emit only one prefix's
URLs (RFC 9728 section 3 exact-match fails for the rest).

Add an opt-in outermost ASGI middleware that matches the request path
against a configured prefix list (SERVER_ROOT_PATHS, comma-separated) on
a segment boundary and sets scope["root_path"] for that request only.
Everything downstream is stock Starlette: route matching strips
root_path so routes stay registered root-relative, and request.base_url
re-includes it, so the discovery documents' resource and the 401
challenges' resource_metadata land under the prefix the client actually
called — with no discovery-builder changes.

LazyFeatureMiddleware now strips the scope root_path (falling back to
the cached SERVER_ROOT_PATH scalar) before feature prefix matching, so
lazily-registered routers — the MCP OAuth discovery router among them —
load under per-request prefixes.

Follow-up to the routing discussion on #35226; composes with, but does
not depend on, #35576.

* fix(proxy): import Sequence from collections.abc (ruff UP035 strict-budget gate)

* review(greptile): trim implementation commentary; fixture-own MCP registry state in tests

Addresses both P2s from the first Greptile pass:
- per_request_root_path_middleware.py (and the related _lazy_features /
  proxy_server comments) cut down to the constraints the code cannot
  express, per repo comment guidance
- the new discovery tests no longer clear/repopulate the shared MCP
  registry inline; a fixture snapshots it, hands the test an empty
  registry, and restores it afterwards so no state leaks between cases

* fix(lint): mutable-ok marker on the prefix accumulator (LIT002 type-discipline gate)

* fix(proxy): tie 401 challenges and get_custom_url to the per-request root_path

The per-request root_path middleware sets scope["root_path"] to the
prefix the client actually called, but the OAuth 401 challenges
(raise_user_oauth_challenge / raise_token_exchange_challenge) still
built their resource_metadata from SERVER_ROOT_PATH. On a pod fronting
several prefixes, the challenge advertised a discovery URL under a
different prefix than the discovery document served — the two
disagreed on where the resource metadata lives, and a strict RFC 9728
client refused the challenge. Route the challenges through a small
ContextVar the middleware populates so they read the same effective
root_path Starlette resolves the request under.

The same accessor fixes get_custom_url: when a request lives under a
SERVER_ROOT_PATHS-matched prefix, request.base_url already carries it,
so appending the SERVER_ROOT_PATH scalar on top produced e.g.
/tenant-a/legacy/sso/callback — a path that does not exist. Reading
the per-request prefix instead (and relying on join_paths's tail-dedup)
keeps SSO login/callback URLs under one prefix — the one the request
actually arrived on.

Fallback: outside a request (module-load-time UI URL builders,
background tasks) the ContextVar is unset and the accessor reads
SERVER_ROOT_PATH, matching get_server_root_path() so scalar-only
deployments are byte-identical.

* fix(mcp): challenge URL under per-request prefix must route, and mock parity

Two follow-ups to the review fix that made the 401 challenge use the
per-request root_path:

1. oauth_protected_resource_path must pick the URL structure that
   actually routes for the mechanism in use:
   - The scalar SERVER_ROOT_PATH deployment registers the well-known
     routes with the prefix INSERTED (via well_known_root_suffix at
     import time), matching RFC 8414 §3. The challenge URL must use the
     same insertion or a client fetching it 404s.
   - The per-request SERVER_ROOT_PATHS deployment can't register routes
     per prefix; PerRequestRootPathMiddleware strips the prefix from
     scope["path"] and the router matches the un-inserted route. The
     URL must place the prefix BEFORE .well-known so the strip leaves a
     matching path.
   The previous fix used the insertion form for both, which 404'd the
   discovery fetch on the per-request path — the discovery doc and the
   challenge would then disagree on where the resource metadata lives,
   the very failure the review flagged. End-to-end verified: the URL
   the challenge advertises routes and the doc's `resource` field
   equals the URL the client originally called (RFC 9728 §3).

2. get_request_root_path now delegates its fallback through
   get_server_root_path() instead of reading the env directly, so every
   existing `monkeypatch.setattr("litellm.proxy.utils.get_server_root_path"`
   test override keeps working. This unstubbed the mock on the /v2/login
   test that failed on the last CI run.

Plus the lint budget: annotate the local accumulator Final, tag the
scope["root_path"] rewrite as an intentional ASGI-contract mutation,
tag the reused `path`/`root_path` rebinds in LazyFeatureMiddleware, and
add reason strings to the two new PLC0415 lazy-import noqas.

* test(mcp): pin the reviewer's expected end-state — challenge URL routes, resource matches called URL

End-to-end regression test that mounts the discoverable router + the
per-request root_path middleware, hits an MCP endpoint that raises
raise_user_oauth_challenge, fetches the resource_metadata URL the
challenge advertises, and checks the returned document's `resource`
equals the URL the client originally called (RFC 9728 §3 exact match).

Covers /tenant-a, /tenant-b, and the unprefixed path on the same app so
a regression on any prefix — challenge URL 404s, or doc emits a
different prefix than the client called — fails at this test rather
than in a strict MCP client's discovery.

---------

Co-authored-by: gym-cmd <186399764+gym-cmd@users.noreply.github.com>
2026-09-07 11:54:59 -07:00
mubashir1osmani
a1e7293fa9
fix(mcp): apply key and team guardrails to MCP tool calls (#39629)
* fix(mcp): apply key and team guardrails to MCP tool calls

Guardrails attached to a virtual key or team were only enforced on LLM
routes. The synthetic request built for MCP tool call guardrail hooks
carried no guardrails in its metadata, so a guardrail with default_on
false never ran on tools/call even when the key explicitly listed it.
Resolve key, team, and project guardrails onto the synthetic request
with the same helper the chat path uses.

* fix(mcp): pass project metadata through without a mutable default

* fix(mcp): mark the request dict parameter mutable-ok with a reason

* test(mcp): explain the premium_user patch and tighten the helper docstring
2026-09-07 16:45:16 +00:00
tin-berri
60440ee4d3
feat(mcp): add opt-in per-server oauth relay discovery (#39936)
Resolves LIT-7074
2026-09-05 18:04:35 -07:00
ryan-crabbe-berri
1258d84221 refactor(ui): route the sidebar by pathname and shrink the ?page= shim to a redirect table
The sidebar and header were still keyed on legacy ?page= ids and mapped
back and forth through MIGRATED_PAGES, legacyPageHref and
legacyKeyForPathname. Leaves are now plain Next links to their path
route, the active item and breadcrumb come from usePathname, and the
setPage/defaultSelectedKey prop chain is gone.

The id-to-route table moves next to the dashboard root page as its only
consumer. That redirect now forwards the remaining query params instead
of dropping them, so deep links such as the proxy's MCP env-var setup
link (?page=mcp-servers&fill_env_vars=) no longer rely on the target page
reading the pre-redirect URL during its first render. The proxy builds
that link as /ui/mcp-servers?fill_env_vars= directly, and the Playground
warnings link to the real routes instead of relative ?page= URLs.

migratedHref is renamed uiHref, the /ui base-path helper it always was.
2026-09-05 16:44:11 -07:00
Yassin Kortam
9832d6e4a6
fix(mcp): scan and mask MCP tool call arguments in unified guardrails (#35142)
* fix(mcp): scan and mask MCP tool call arguments in unified guardrails

A guardrail configured with mode pre_mcp_call was handed only a synthetic
tool definition (name plus an empty parameters schema), so it never saw the
argument values it was configured to inspect, and any rewrite it returned was
discarded. Detection could not fire and masking could not take effect, while
the applied-guardrails metadata still reported the guardrail as having run.

Pass every string leaf of the tool call arguments as texts, and fold the
guardrail's rewritten leaves back into modified_arguments, which is the channel
the MCP call path reads to decide what to send upstream. The leaf walk reuses
the json_string_leaves / with_json_string_leaves helpers the tool result path
already uses, so both directions share one bounded traversal.

Two guardrails running concurrently under run_in_parallel scan the same payload
snapshot, so each returns a full replacement derived from the original leaf.
Rewrites of the same leaf to different values are rejected rather than silently
losing one redaction; a leaf that already holds this guardrail's own replacement
is convergent and still masks, which is what the bundled content filter does
when it rewrites the arguments itself as well as through texts.

* fix(mcp): annotate guardrail argument rewrites

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(tests): isolate MCP guardrail callback state

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* chore: ratchet LIT010 budget after merge

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(tests): remove duplicate Bedrock hook parameter

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(mcp): fail closed when guardrail rewrites cannot be mapped to MCP arguments

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(tests): patch the guardrail translation mappings cache where staging now keeps it

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-05 20:51:25 +00:00
Yassin Kortam
17e13126cc
feat(mcp): warn when an oauth2_id_jag server outruns the SSO provider's assertion capture (#35394)
* feat(mcp): warn when an oauth2_id_jag server outruns the SSO provider's assertion capture

Only the generic OIDC login path captures the IdP id_token that an oauth2_id_jag MCP
server spends as its RFC 8693 subject token. Under Google, Microsoft, SAML or no SSO at
all, registration succeeds and then every ID-JAG credential resolution fails for every
user, with nothing in the logs, the config or the API response to say why.

Report the mismatch from the two places it is knowable: when an oauth2_id_jag server is
created or updated through the management endpoint, and at SSO callback time when a login
hands the arm nothing while such a server is registered. Provider selection mirrors the
callback's precedence, so a generic client id sitting behind GOOGLE_CLIENT_ID does not
clear the warning.

* test(sso): update merged CLI diagnostic patch target

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* feat(mcp): warn about the ID-JAG capture gap for config-declared servers and on the SSO debug page (#39350)

* feat(sso): surface the ID-JAG capture gap on the SSO debug page

/sso/debug/callback is where an operator lands when they are already trying to work out
why ID-JAG is failing, so the reason belongs on it. The annotation appears only when the
active SSO provider captures no identity assertion AND an oauth2_id_jag server is
registered for that gap to break; a deployment without both renders the page it rendered
before, byte for byte. Only the provider name and the remedy are rendered, never a
configured value, and an unreachable MCP table costs the page its annotation rather than
the page itself.

The payload carries the one mutable-ok in this work. Conditionally including a member of a
JSON document has to construct a mapping, and the rejected alternatives are recorded on the
helper so the next reader does not rediscover them.

Held out of the diagnosability PR deliberately: that PR is already reviewed and green, and
this surface ships with the remaining config-load warning as one follow-up.

* feat(mcp): warn at config load when an oauth2_id_jag server outruns the SSO provider's assertion capture

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(sso): trim comments on the ID-JAG debug page diagnostic

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(sso): clean up merged imports

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(sso): satisfy type discipline for diagnostic payload

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* style(sso): keep the optional ID-JAG payload member on one line for ruff format

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(sso): use Python 3.10-compatible assert_never

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Yassin Kortam <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(sso): keep the ID-JAG capture-gap diagnostic out of the unauthenticated debug page

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(sso): inject the retention check and log via caplog so the ID-JAG tests pass the test-quality gate

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(sso): keep the debug-page outage test on the capture-gap path

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(sso): annotate the retention check type alias

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-05 12:43:09 -07:00
Yassin Kortam
110f654f34
feat(mcp): renew the stored SSO identity assertion behind ID-JAG (#35401)
* feat(mcp): renew the stored SSO identity assertion behind ID-JAG

The oauth2_id_jag arm asserts the id_token captured at the user's last
interactive SSO login, and nothing ever renewed it, so an agent holding a
brokered LiteLLM key could act for that user only until that token's exp.
The assertion already carried the IdP refresh token beside it; this
redeems it.

RefreshingSSOAssertionStore wraps the database reader and satisfies the
same protocol, so the egress arm is unchanged. Renewal is lazy and
single-flighted per user through the same RefreshCoordinator the
authorization_code arm uses, since an IdP that rotates refresh tokens
treats two concurrent redemptions as replay. A refusal leaves the expired
assertion in place so the reader still challenges the user; an
unreachable IdP surfaces as a store outage instead.

* fix(mcp): let a cross-replica loser settle the SSO assertion renewal itself

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(mcp): satisfy type discipline lint budget

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* chore(ci): rerun checks after docs main added the missing router setting row

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(mcp): answer a cross-replica loser retryable instead of re-electing it

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(mcp): bypass stale assertion cache during renewal

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* chore: ratchet type-discipline budget after merge

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Yassin Kortam <yassin.kortam@gmail.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-05 12:43:02 -07:00
tin-berri
e3b4a82ff9
Merge pull request #39926 from BerriAI/litellm_lit6981_none_url_auth
fix(mcp): reject URL credentials for none auth
2026-09-05 18:44:34 +00:00
yucheng-berri
fafd294878
fix(mcp): let config.yaml MCP servers pin server_id (#39286)
* fix(mcp): let config.yaml MCP servers pin server_id

A config-defined MCP server's id is a hash of server_name|url|transport|
auth_type|alias, recomputed on every config load, so editing any of those
fields mints a new id. Every key and team granted the old id via
object_permission.mcp_servers keeps pointing at an id that no longer exists,
and the server disappears from tools/list for them with nothing logged.

load_servers_from_config now uses an explicit server_id from the server's
config entry when present and falls back to the existing hash otherwise, so
grants survive url/name/alias edits. Rejected at config load: a blank or
non-string server_id, two entries claiming the same id, a pinned id already
held by a database-backed server, and a pinned id that is another entry's
server_name or alias (expand_permission_list matches ids before names, so
that one would capture the other server's grants). Because the database
registry loads after the config on startup, a database row that lands on a
pinned config id is reported as a warning from the database reload instead,
where it is decidable; the warning is latched on the shadowed set so the
config-reload timer does not reprint it every interval.

Deployments that do not set server_id keep the exact id they have today.

* fix(mcp): close two more pinned-id capture paths

A pinned server_id equal to an alias supplied through litellm_settings
mcp_aliases was accepted, because the collision index only held the entry's
own alias field. expand_permission_list matches ids before names, so grants
written for the aliased server resolved to the pinning one. mcp_aliases keys
whose target is a config server are now reserved the same way.

A pinned server_id equal to a database-backed server's name, server_name or
alias had the same effect against the database side, and could not be
rejected at config load because the database registry is not loaded yet. The
database reload now warns about it, latched like the existing shadow warning.

* fix(mcp): reserve only the aliases the loader actually assigns

Reserving every mcp_aliases key targeting a config server was too broad in
two ways: the mapping is ignored when the entry sets its own alias, and only
the first mapping for a server is ever applied. Both cases made a pinned
server_id that could never have collided abort proxy startup. Reserve only
the name load_servers_from_config will really assign.

The database capture warning also fired for a database server whose own id is
the config server_id. There the database row wins the id outright through
get_registry precedence, so the shadow warning above it is the accurate one
and the capture message contradicted it. Skip those rows.

Also mark the two litellm-internal patches in the reload test helper, which
the test-quality gate counts; the database reload has no other seam.

* fix(mcp): match the loader's alias check exactly, is None not falsiness

load_servers_from_config consults mcp_aliases only when the entry has no
alias key at all, so an entry setting alias: "" gets no mapped alias. The
collision index used falsiness and reserved the mapped name anyway, which
failed startup on a pinned server_id that could never have collided with it.

* fix(mcp): skip one identifier, not the whole database row

A database row can shadow one config server_id by id and capture another by
name at the same time. Skipping the entire row when its id shadowed a config
entry dropped the second warning, leaving the operator with half a diagnosis.
Skip only the identifier equal to the row's own id.

* fix(mcp): reject conflicting self-pinned server ids

* fix(mcp): validate config server names before building the identifier index

The collision check reads every entry's body up front, so a malformed
entry under an invalid name surfaced as an AttributeError instead of the
name validation error the loader gave before this change.
2026-09-04 23:52:33 -07:00
mateo-berri
822c8629ae Merge branch 'litellm_internal_staging' of https://github.com/BerriAI/litellm into litellm_deflake_20260902
# Conflicts:
#	tests/test_litellm/caching/test_redis_cache.py
2026-09-04 19:06:33 -07:00
devin-ai-integration[bot]
205a5e9d6c
feat(mcp): use x-mcp-<access_group>-* headers as default upstream credentials for group members (#39717)
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-04 12:45:24 -07:00
Yujong Lee
fae3d224eb Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_python_version_ci
# Conflicts:
#	basedpyright-code-budget.json
#	tests/sdk_function_trace/profiler.py
#	tests/sdk_function_trace/test_profiler.py
2026-09-04 09:01:13 -07:00
mateo
9835883e03 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_deflake_20260902
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-04 09:20:07 +00:00
Yassin Kortam
b7f53ce9a9
fix(mcp): pre-flight the ID-JAG credential at the transport edge (#35392) 2026-09-04 01:39:37 +00:00
devin-ai-integration[bot]
1add1b4655
perf(mcp): cache SSO identity assertion reads on the ID-JAG path (#39348)
* perf(mcp): cache SSO identity assertion reads on the ID-JAG path

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(mcp): guard sso assertion cache against stale relogin reads

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(mcp): keep sso assertion cache entries and generation markers in separate namespaces

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(router): rename duplicate get_configured_mode test so ruff F811 passes

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(mcp): use a process-wide epoch for sso assertion cache invalidation

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-03 17:25:37 -07:00
devin-ai-integration[bot]
35d20468cd
fix(mcp): normalize a schemed authentication_token on the v2 and OpenAPI static paths (#39345)
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-03 15:09:08 -07:00
devin-ai-integration[bot]
6da516e6f3
fix(mcp): strip inbound auth scheme case-insensitively before token exchange (#39346)
* fix(mcp): strip inbound auth scheme case-insensitively before token exchange

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(mcp): type the fake credential provider params in token exchange scheme tests

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-03 15:08:02 -07:00
devin-ai-integration[bot]
f2f65a6e8b
fix(mcp): resolve OAuth broker endpoints by server_id with IP access checks (#39432)
* fix(mcp): resolve OAuth broker endpoints by server_id with IP access checks

Resolve named OAuth lookups through server IDs while retaining client IP checks\n\nCo-authored-by: KK291860 <krishnakumar.kocherykumaran@sephora.com>

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ci: retrigger e2e pipeline

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-03 15:04:45 -07:00
tin-berri
7256bd307a
fix(mcp): scope allow-all servers to virtual keys (#39531) 2026-09-03 10:32:03 -07:00
mateo
d4a480fb6e Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_deflake_20260902 2026-09-03 09:18:54 +00:00
yassin
342b21266d Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_fix_agent_mcp_grants
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-03 05:51:11 +00:00
mateo-berri
a581399027 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_python_version_ci
# Conflicts:
#	basedpyright-code-budget.json
2026-09-02 21:34:20 -07:00
devin-ai-integration[bot]
291d02f8aa
fix(mcp): never exchange the LiteLLM virtual key as the upstream subject token (#39446) 2026-09-02 20:32:43 -07:00
mateo-berri
45988143bc Merge branch 'litellm_internal_staging' of https://github.com/BerriAI/litellm into litellm_fix_agent_mcp_grants
# Conflicts:
#	type-discipline-budget.json
2026-09-02 18:42:33 -07:00
mateo-berri
748075be4f Merge origin/litellm_internal_staging into litellm_python_version_ci 2026-09-02 18:21:48 -07:00
mateo-berri
d6b8cb93b3 fix(mcp): name the agent veto on scoped tool calls too 2026-09-02 18:18:18 -07:00
Yassin Kortam
44a6c659fb
fix(mcp): fence an outbound-token write against an overlapping invalidation (#35398)
get_or_compute single-flights concurrent misses under a per-key lock, but
invalidate() deletes outside it. A mint already awaiting the IdP when an
invalidation ran wrote its result into the slot afterwards, so a bearer minted
before the invalidation was served after it for its full TTL; the upstream-401
retry then re-presented the bearer the server had just rejected.

The cache now keeps a per-key generation beside the single-flight lock.
invalidate() bumps it and get_or_compute snapshots it inside the lock right
before minting, skipping the cache write when it no longer matches. The token
is still returned to the caller it was minted for, so only the caching is
fenced. Generation and lock share one object that a running mint binds to a
local, which pins the weak map's entry for exactly as long as a write can still
land; a generation held weakly on its own would be collected across the await
and the re-check would silently pass.
2026-09-02 17:45:58 -07:00
mateo-berri
5e2d98d49b chore: merge the branch's staging sync into the attribution fix 2026-09-02 17:45:20 -07:00
devin-ai-integration[bot]
8cbaba8863
fix(mcp): gate the connect-time OBO pre-flight on the key's allowed servers (#39447)
The pre-flight token exchange resolved its target from the requested path, so a key not entitled to that server still drove an outbound exchange at the IdP and populated the credential cache before the later access check denied it. Resolve the target through the same allowed-server set the rest of the route uses and skip the exchange when the requested server is not in it.

Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-02 17:39:36 -07:00
mateo-berri
0d6948241e fix(mcp): attribute the agent veto for differently cased server names 2026-09-02 17:35:51 -07:00
yassin
61e2e3e646 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_fix_agent_mcp_grants
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-03 00:25:04 +00:00
mateo-berri
c4ca4268a2 Merge branch 'litellm_internal_staging' of https://github.com/BerriAI/litellm into litellm_fix_mcp_test_connection_oauth_bearer
# Conflicts:
#	litellm/proxy/_experimental/mcp_server/rest_endpoints.py
2026-09-02 17:16:33 -07:00
Mateo Wang
c1a26f36ee
Merge pull request #38791 from BerriAI/litellm_fix_mcp_oauth_tool_fetch_auth
fix(mcp): cap tools preview and test-connection at the listing timeout and name the unreachable upstream
2026-09-02 16:47:10 -07:00
mateo-berri
2a1527c22f fix(mcp): match gateway-served /mcp/<name> URLs the way the gateway resolves them and narrow only gateway references 2026-09-02 16:03:31 -07:00
yassin
659ed03502 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_fix_agent_mcp_grants
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-02 22:15:20 +00:00
devin-ai-integration[bot]
a76cb6feaf
feat(mcp): semantic tool search for the native MCP Gateway (#39404)
The mcp_tool_search virtual tool only did substring token matching, so a native MCP client asking for "FX" could not find a tool described as "foreign exchange rates" even though the same catalog is ranked by embeddings on /responses and /chat/completions.

Adds litellm_settings.mcp_tool_search (embedding_model, top_k, similarity_threshold, core_tools). With an embedding model the caller's authorized catalog from _list_mcp_tools is ranked by cosine similarity of name plus description; configured core tools the caller can reach come first and do not consume top_k. Without an embedding model the keyword fallback keeps the old behavior. Settings are hot-reloadable from the DB, exposed on /get and /update mcp_tool_search_settings, and editable from the Admin UI under MCP Servers > Tool Search. The embedding index is shared with agent_search via a new SemanticTextIndex.

Resolves LIT-6751

Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-02 15:09:44 -07:00
yassin
615b404d12 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_fix_agent_mcp_grants
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-02 22:09:19 +00:00
devin-ai-integration[bot]
748c2026d7
fix(proxy): word database 503s by whether the fault is transient (#39256)
Permanent Prisma/query-engine faults keep the 503 status and no_db_connection type but stop claiming the database is temporarily unreachable. A permanent fault anywhere in the exception chain outranks the transport error that surfaced it. MCP bridge and DCR flows gain a faulted resolution state with matching wording. Resolves LIT-5208

Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-02 15:05:55 -07:00
mateo-berri
cdf8383d9e Merge branch 'litellm_internal_staging' of https://github.com/BerriAI/litellm into litellm_fix_mcp_oauth_tool_fetch_auth
# Conflicts:
#	litellm/proxy/_experimental/mcp_server/rest_endpoints.py
2026-09-02 14:58:08 -07:00
Yujong Lee
cfcaaa03d6 fix: resolve Python 3.14 OCR annotations and remaining matrix failures 2026-09-02 14:35:38 -07:00
mateo
25c5f0d993 test: deflake JWT tamper assertions and fuzzy picker widget driver
Tamper tests rewrote the last two base64url characters of the signature,
which on roughly 1 in 250 RS256 tokens (1 in 1000 HS256) only touched
padding bits, so the decoded signature was unchanged and still verified.
Corrupt the decoded signature bytes instead.

The fuzzy picker driver sent keys after fixed sleeps, so a slow worker
could receive the filter text before the widget had highlighted the match.
Wait on the widget's highlighted choice instead.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-02 10:05:50 +00:00
mateo-berri
b45670b4b4 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_fix_mcp_test_connection_oauth_bearer 2026-09-01 22:23:03 -07:00
mateo-berri
222cb293ca fix(mcp): name the access group that resolved when an agent vetoes a mixed scope 2026-09-01 22:14:18 -07:00
mateo-berri
3c2138f037 fix(mcp): count agent toolsets in the agent MCP ceiling and tool grants
An agent's object_permission.mcp_toolsets could be persisted through the
new edit form and PATCH /v1/agents but never reached the request-time
checks: _get_allowed_mcp_servers_for_agent read only mcp_servers and
mcp_access_groups, so an agent granted a toolset alone resolved to [] and
placed no ceiling on its keys, and _get_agent_tool_permissions_for_server
ignored the tools those toolsets grant. Both helpers now resolve toolsets
through the shared _toolset_tool_permissions / _toolset_tools_for_server
helpers the key, team, and org levels already use, and a declared toolset
that resolves to nothing raises UnloadableEntitlementError so the
resolver denies instead of reading the agent as unrestricted
2026-09-01 19:12:24 -07:00
mateo-berri
e4c6badca2 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_fix_agent_mcp_grants 2026-09-01 18:37:31 -07:00