get_or_compute single-flights concurrent misses under a per-key lock, but
invalidate() deletes outside it. A mint already awaiting the IdP when an
invalidation ran wrote its result into the slot afterwards, so a bearer minted
before the invalidation was served after it for its full TTL; the upstream-401
retry then re-presented the bearer the server had just rejected.
The cache now keeps a per-key generation beside the single-flight lock.
invalidate() bumps it and get_or_compute snapshots it inside the lock right
before minting, skipping the cache write when it no longer matches. The token
is still returned to the caller it was minted for, so only the caching is
fenced. Generation and lock share one object that a running mint binds to a
local, which pins the weak map's entry for exactly as long as a write can still
land; a generation held weakly on its own would be collected across the await
and the re-check would silently pass.
The pre-flight token exchange resolved its target from the requested path, so a key not entitled to that server still drove an outbound exchange at the IdP and populated the credential cache before the later access check denied it. Resolve the target through the same allowed-server set the rest of the route uses and skip the exchange when the requested server is not in it.
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The mcp_tool_search virtual tool only did substring token matching, so a native MCP client asking for "FX" could not find a tool described as "foreign exchange rates" even though the same catalog is ranked by embeddings on /responses and /chat/completions.
Adds litellm_settings.mcp_tool_search (embedding_model, top_k, similarity_threshold, core_tools). With an embedding model the caller's authorized catalog from _list_mcp_tools is ranked by cosine similarity of name plus description; configured core tools the caller can reach come first and do not consume top_k. Without an embedding model the keyword fallback keeps the old behavior. Settings are hot-reloadable from the DB, exposed on /get and /update mcp_tool_search_settings, and editable from the Admin UI under MCP Servers > Tool Search. The embedding index is shared with agent_search via a new SemanticTextIndex.
Resolves LIT-6751
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Permanent Prisma/query-engine faults keep the 503 status and no_db_connection type but stop claiming the database is temporarily unreachable. A permanent fault anywhere in the exception chain outranks the transport error that surfaced it. MCP bridge and DCR flows gain a faulted resolution state with matching wording. Resolves LIT-5208
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
GET /mcp-rest/tools/list without server_id returned only the tools of
the servers that answered and silently dropped any server whose listing
failed (for example an OAuth-protected server without credentials), so
clients could not tell a partial listing from a complete one.
The aggregate response now carries a server_outcomes map keyed by server
alias with the same classified outcome (ok/auth_required/forbidden/...)
that the MCP protocol path already puts in _meta. Healthy tools and the
HTTP 200 status are unchanged.
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(mcp): follow tools/list pagination from upstream servers
Adopts BerriAI/litellm#32244 by Jupiter363 onto litellm_internal_staging
with merge conflicts resolved
* fix(mcp): degrade buggy pagination to partial results and bound the preview walk
A repeated nextCursor now returns the tools collected so far instead of
discarding every page with a RuntimeError, an empty-string cursor is treated
as terminal, load_mcp_tools shares the same pagination walk instead of
returning only the first page, and the tools/list preview is bounded by the
listing timeout instead of only the per-request timeout times the page cap
* fix(mcp): annotate deliberate rebind for the preview timeout scope
* fix(mcp): bound the shared pagination walk with an overall listing deadline
The per-request session read timeout restarts on every page, so direct SDK
callers of list_tools and load_mcp_tools could run up to the page cap with
no overall bound. The walk now returns the tools collected so far when
max(MCP_CLIENT_TIMEOUT, MCP_TOOL_LISTING_TIMEOUT) expires
* fix(mcp): let a per-server timeout extend the pagination deadline
MCPClient carries a per-server timeout that can exceed the global default;
list_tools now passes max(self.timeout, MCP_TOOL_LISTING_TIMEOUT) into the
shared walk so a deliberately slow server is not silently truncated at the
global deadline
* fix(mcp): honor per-server timeouts in the preview deadline and test the walk sessionless
The preview deadline now extends with the created client's own timeout, and
the pagination walk's cap, repeated-cursor, and empty-cursor cases are tested
directly against the helper instead of through patched SDK internals
* fix(mcp): forward the preview request's per-server timeout to the temporary server model
The tools preview built its temporary MCPServer without the request's
timeout field, so the client factory always fell back to the global
default and a per-server timeout could never extend the preview's
listing deadline (or its per-request timeout).
* feat(mcp): support RS256 signing for MCP gateway session tokens
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style: ruff format session token modules
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(mcp): enforce key strength on rotated public keys and unique kids
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix: enforce MCP toolsets attached to a team, org, or internal user
object_permission.mcp_toolsets was resolved into servers and tools only at
the key level; every other principal read mcp_tool_permissions and silently
ignored its toolsets. A team/org/user toolset alongside a server grant was
inert (all tools callable), a toolset alone granted nothing, and an inert
team toolset let the org server list substitute for the empty team result,
handing the caller every org server.
Resolve toolsets at each level that resolves mcp_tool_permissions, union
their servers into that level's granted server set, and count a declared
key/team toolset toward has_lower_level_mcp_restrictions so the org list
can only cap, never substitute, even when the toolset resolves empty.
Resolves LIT-5749
* fix: deny when a team's declared MCP toolset cannot be resolved
The team server resolver swallowed UnloadableEntitlementError into an empty
list, so a dangling team toolset dropped the team ceiling instead of denying,
unlike the org and user paths. Re-raise it so the top-level resolver denies.
Also anchor the test-quality suppression comments on the patch opener lines
the gate reads, with per-seam reasons.
An MCP server behind an API gateway needs two credentials on one request: the
gateway's own token on a private header, and a separate bearer on Authorization
for the server behind it. Every arm that minted or held a token hardcoded
Authorization, and the conflict rule then dropped the operator's static
Authorization to make room, so the second credential never arrived.
ApiKeyConfig already modelled this as header_name plus value_prefix behind a
header() method. Extend that carrier to the four minted-token configs, have each
resolver arm ask its config which header to use instead of naming one, and drop
only the header the resolved credential is about to occupy.
Operators set it per server via upstream_token_header, plumbed through
config.yaml, the credentials blob, the management API and the admin form, on the
M2M, token-exchange, authorization-code and ID-JAG arms. It is non-secret so it
stays plaintext and round-trips on admin reads. Unset keeps today's behaviour.
Moving a credential off Authorization means it stops inheriting what Authorization
gets for free, so the slot now carries those protections itself. httpx drops
Authorization when a redirect crosses origin and keeps every other header, so a
custom slot is dropped by the client on the same condition, mirroring httpx's own
scheme/host/port rule with an agreement test that fails if the two ever diverge.
The v1 path also mirrors the v2 conflict rule, so an injected header cannot shadow
the credential the gateway resolved for that slot.
Which header a credential occupies, and what counts as being that header, was
answered independently in nine places by four hand-rolled comparisons. same_header,
has_header and without_header in litellm/types/mcp.py are now the one owner, shared
by both MCP stacks, and the client derives its slot once instead of three times.
The header name reaches egress verbatim, so the RFC 7230 grammar lives in one
place and is checked where servers are built: a bad value fails the config load
and the management API returns 400, rather than raising while a spec is built
and emptying the aggregate tool list for every other server. A blank means unset,
matching what the endpoint already accepts.
* fix(mcp): keep upstream OAuth Authorization when jwt signer hook injects one on tools/call
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(mcp): only treat server credential as occupying Authorization when it maps to that header
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The token exchange no longer joins deferred discovery when the token url is
already stored, so it cannot 503 over an unreachable issuer it needs nothing
from. After a request joins discovery, authorize and token now read the
resolved server for the DCR bridge relay decision and the rest of the flow,
so a registration endpoint resolved mid-request routes a front-door client
to its own redirect binding. The encrypt seam in the issuer-yield authorize
test now uses a real salt key instead of patching an SDK internal.
Document the intentional internal seams used by the DCR bridge admission tests and normalize import ordering.\n\nGenerated with AI\n\nCo-Authored-By: Codex
Preserve standard Authorization key validation while preventing client MCP credentials from receiving anonymous bridge admission.
Generated with AI
Co-Authored-By: Codex
* test: drop the cwd-relative sys.path.insert calls from the test suite
TQ003 stands at 1,077 across 1,058 files, and 1,015 of them are the same shape:
sys.path.insert(0, os.path.abspath("../..")) and its deeper siblings. The
argument resolves against the working directory rather than the file, so from
the repo root, where every job runs pytest, it inserts the directory two levels
above the checkout. It has never pointed at litellm. The package is installed
into the environment anyway, which is what actually makes the import work, and
what the rule's message has said all along.
Removing them leaves 1,634 imports of sys and os with no remaining reference,
and those go too, except where another test module imports the name back out of
the file. The rest of TQ003 is 62 call sites that resolve against __file__ or a
variable, which are a different question and are left alone.
Collection is identical either way: 45,871 tests and the same 51 pre-existing
collection errors before and after, and ruff reports no new undefined name.
* test: drop the duplicate imports the sys.path sweep exposed to F811
* test(pre-call-utils): restore the os import the new bedrock tests need
* fix(mcp): resolve admin OAuth sessions to the same server set the connect page shows
* fix(mcp): bind admitted admin rows through the entitlement ceiling, not the credential scope clause
* fix(mcp): strip root_path before matching the per-server MCP route spelling
The 401 challenge for a gateway-managed oauth2 MCP server advertises the
protected-resource metadata URL in the spelling the client connected on, so a
strict RFC 9728 section 3 client lands on a document whose `resource` equals the
URL it actually called. That spelling test compared `_original_path` against the
root-relative `/{server}/mcp` shape, but `_original_path` and `scope["path"]`
are raw request-line paths that still carry the deployment's `root_path`
On a SERVER_ROOT_PATH deployment the prefix therefore made the legacy test fail
and every request fell through to the standard `/mcp/{server}` branch. A client
connecting on `/litellm/github/mcp` was pointed at the standard-pattern
document, which serves `resource = {base}/litellm/mcp/github`; that is not the
URL the client called, so a strict client aborts discovery before the MCP
request fires
Route the path through `get_route_relative_request_path` first, which removes
`root_path` on a segment boundary the same way
`litellm.proxy.auth.auth_utils.get_request_route` already does for the rest of
the MCP auth path, so `/litellmfoo` is not truncated under `root_path=/litellm`
* fix(mcp): make the gateway-managed 401 challenge root-path aware
The gateway-managed authorization_code challenge in process_mcp_request
built its AS-metadata URL from two root-path-unaware pieces:
- it matched the caller's spelling against `scope["_original_path"]`, a
raw request-line path that still carries the deployment prefix, so on a
SERVER_ROOT_PATH deployment the `/mcp/{server}` branch never matched and
every request fell through to the legacy one-segment form
- it hardcoded `/.well-known/oauth-authorization-server` without the
root-path segment the discovery route decorators bake in, so the URL
404'd under a sub-path deployment regardless of which branch was taken
Route the spelling match through get_route_relative_request_path and the
well-known root through well_known_root_suffix, the same two helpers the
discovery route registrations derive their paths from, so the advertised
URL cannot drift from the route that serves it.
Root-mounted deployments are unaffected: both helpers are no-ops when
SERVER_ROOT_PATH is unset.
Co-Authored-By: Claude <noreply@anthropic.com>
---------
Co-authored-by: Claude <noreply@anthropic.com>
A test that asserts on the error inside its own except block passes when the
call stops raising, because nothing runs the handler. That is the exact case
the test exists to catch, so the regression lands green.
Rewrites all 111 such blocks into pytest.raises, which fails when the call
succeeds, and selects PT017 in ruff-tests.toml so no new one lands.
The dcr_bridge oauth_delegate connect flow completed for a signed-in user
with no litellm-side grant to the target server: every leg returned 200,
the DCR client showed connected, and tools/list then fail-closed to an
empty list with the upstream never contacted (#36358). The authorize leg
now admits the user the way MCP egress will (same reload_admitted_user
constructor, same get_allowed_mcp_servers resolver) and refuses with an
RFC 6749 access_denied redirect naming the remedy, before any upstream
OAuth runs or an envelope is minted. Availability faults (5xx) propagate;
unknown or deactivated users deny fail-closed
Promotes MCPRequestHandler reload_admitted_user to public: it already had
a cross-module consumer in ui_session_utils, and this gate adds a second,
so the private name no longer reflected its use. Ratchets the freed
reportPrivateUsage budget headroom down
tests/litellm/ was a second mirror beside tests/test_litellm/ that no workflow,
Makefile target, or CircleCI job ever named. Its other 33 files were reconciled
during August 2026; this one stayed behind under a ci-coverage-allowlist entry
asking a later pass to decide which of its five orphan behaviours still hold.
They no longer hold as written: 25 of its 32 cases fail against today's code,
because the file froze on the day it stopped being collected and the endpoints
kept moving. Three of the five are already covered by the live twin, and better.
test_get_request_base_url_xff_trust_gate parametrizes the trust gate in both
directions, including the exact untrusted-caller case the orphan asserted, and
the standard and legacy protected-resource shapes are both exercised through
use_standard_pattern.
The other two were the only tests anywhere for validate_trusted_redirect_uri
under that same gate, so they are ported rather than dropped, rebuilt on the
live file's request-mock conventions. Both directions are load-bearing: forcing
is_request_from_trusted_proxy to True fails the untrusted case, forcing it to
False fails the trusted one.
313 tests pass in the live file, up from 311. Dropping the dead file clears one
zero-assert TQ001 violation, so its ceiling ratchets down with it.
