* fix(mcp)!: forward the LiteLLM key when the admin configures it
Restores forwarding of the caller's LiteLLM key through explicit channels
(Extra Headers listing X-LiteLLM-API-Key or Authorization, x-mcp-{alias}-authorization
and x-mcp-auth) that #45401 and #46932 started stripping. Implicit Authorization
forwarding stays scrubbed for true_passthrough and unlisted servers, gateway
session tokens and bridge envelopes stay scrubbed, and guardrail and JWT signer
bearer reads now skip the admission key
* test(mcp): serve signer introspection from an in-process transport
* fix(mcp): copy oauth2 headers so a listed key never reaches a later server
* test(mcp): drive the shared-header regression through the real MCP client
---------
Co-authored-by: Joshua Valluru <326636767+joshua-berri@users.noreply.github.com>
The second server in this case is a plain MCP SDK server registered as
oauth_delegate. The gateway's first catalog listing waits for that server's
OAuth metadata lookup, which never resolves and runs for the full
MCP_METADATA_TIMEOUT of 10 seconds. The MCP SDK client also gives
server/discover 10 seconds, so the case passed or failed on about a tenth
of a second of head start.
Setting LITELLM_MCP_METADATA_TIMEOUT to 5 for this case keeps the lookup
well inside the client limit. The case still asserts that subscription
changes are delivered and tools are refreshed across both servers.
* fix(responses): preserve native stream spend on guardrail rejection
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(responses): type deferred logging dispatch once
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(responses): expect guarded rounds on one failure row
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(tests): format guarded billing regression
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(tests): preserve existing MCP unit formatting
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The MCP SDK server answers a notification POST with 202 before it hands the notification to the session. When the gateway sends DELETE right after `notifications/initialized`, the session can close in that gap, and the SDK logs an ASGI error that fails the test at teardown. Wrap the upstream with the existing `_acknowledge_notifications_after_dispatch` helper the other SDK peers already use, so the 202 only goes out once the session has the notification
Co-authored-by: Joshua Valluru <326636767+joshua-berri@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* fix(mcp): bill both rounds of a streamed chat completions MCP auto-execute
A streamed /v1/chat/completions request with an MCP tool set to require_approval "never" runs two model calls on one logging object, and the final-stream dedupe kept only the first round's spend row. The final usage chunk also showed only the second round.
The MCP stream iterator now holds the first round's assembled response, prices it, and logs one success row that carries the usage and cost of both rounds, with the fixed margin charged once. The final usage chunk shows the same summed usage. A single-round stream still logs its one row when the answer finishes.
When the follow-up round fails, the first round is logged from a copy of the logging object taken before the follow-up opened, and the failure row carries only the follow-up's partial usage with no second fixed margin. With proxy post-call guardrails armed, only (assembled, cache_hit) tuples are parked for the proxy, and the deferred log is armed only while the iterator reads, so the proxy's orphaned-callback cleanup never sees it after the Router's first-chunk read
* fix(mcp): send the usage chunk of a streamed chat MCP request that ends in one round
When a streamed chat completions MCP request ends in its first round (the model answers without a tool call, or no tool results come back), the iterator drained the inner stream to fire its spend log and dropped the usage chunk that stream produced, so a client that asked for include_usage got no usage chunk at all.
The drained usage chunk is now held and sent to the client after the final content chunk and before [DONE], the same place a plain stream sends it, so the last usage chunk matches the billed row. A two-round request still drops the first round's usage chunk and sends the merged one from the follow-up, and a stream without include_usage still sends none
* fix(mcp): hold round one's log for the guardrail, bill the chunk cost once, and keep round one's id
When the follow-up call of a streamed chat MCP auto-execute fails, round one's
success log now goes through the proxy's deferred guardrail release instead of
straight to the handlers, so a post-call guardrail runs on it and the row
carries it. Without a proxy release it logs directly as before
The last usage chunk now takes its cost from the combined row's billed total,
so a fixed cost margin is counted once in the chunk, as it is on the row. The
follow-up's usage chunk is held until the follow-up ends to make that possible
The combined row now keeps round one's id, the stream's first chunk id, which
matches how a single-round stream is logged
* fix(mcp): bill every chat MCP stream round once on failure, hang-up, and sync drop
When the follow-up call fails, round 1's success row is now logged on a copy of the
logging object that does not share the request's guardrail results, so a post-call
guardrail that reports on the failed request counts once instead of on both rows
A client hang-up during the follow-up now bills round 1 plus the follow-up's partial
answer on one success row with round 1's id and the fixed margin counted once, the
same way a plain stream's hang-up bills its partial answer
The last usage chunk now carries round 1's usage and cost when only round 1 is billed,
and a synchronous reader that drops the stream mid-answer bills round 1 on the loop it
read with instead of discarding it
The deferred release's logged_on argument is typed and every release closure accepts
it, and the parked dispatch constants and protocols both MCP billing paths use now live
in one module. The unit tests wait on a logging signal instead of a fixed delay and
check the billed row through the proxy stream hook instead of private callback state
* test(mcp): drive the unassembled follow-up through an upstream audio delta that is not base64
The test replaced litellm.stream_chunk_builder with a stub that raises. Round 2 now streams an audio delta whose data is not valid base64, so the real builder raises, the stream parks only best-effort usage, and the client's last usage chunk must carry round 1's tokens and cost
* fix(mcp): count the fixed margin once in the combined stream row's usage cost
* fix(mcp): bill round 2's partial answer when a client hangs up on a routed stream
On the proxy the Router wraps the MCP stream, and when a client hangs up during
round 2 the router's own cleanup closes that stream before the proxy's disconnect
cleanup runs. The close logged round 1 alone, so round 2's partial answer was
never billed. The MCP stream now bills round 1 plus round 2's partial answer as
soon as the hang-up's cancellation reaches its read, before anything closes it
* fix(mcp): count the follow-up's own messages in round 2's partial prompt
The follow-up stream shares the request's logger, so it took the client's
original messages as its prompt. When round 2 ended early, from a hang-up or a
broken upstream, its partial usage counted only those messages and left out the
assistant tool call and the tool results the follow-up actually sent. The
follow-up stream now counts the messages the follow-up call sent
* fix(mcp): bill round 2's partial when a routed stream's client hangs up between reads
On a proxy the stream the disconnect cleanup holds is the router's fallback wrapper, so the
cleanup never found the MCP stream's in-flight billing and released round 1 alone when the client
hung up while the proxy sat at a yield. The round billing now registers its in-flight biller on
the request's logging object, next to the parked release, and the cleanup looks it up there, so
any wrapper chain bills round 1 plus round 2's partial once
* fix(mcp): send the MCP stream usage chunk under the billed row id and count the partial off the loop
A streamed chat completion that runs a LiteLLM-hosted MCP tool bills both model calls on one row under round 1's
chatcmpl id, but the usage chunk it sent the client was round 2's held chunk and kept round 2's id, so a caller could
not join the usage chunk to its row. The usage chunk now carries round 1's id in every path that sends it, with or
without a deferred post-call guardrail, while its usage and cost and every content chunk's id stay as they were
When the client hangs up during round 2, assembling round 2's partial answer counts the follow-up messages, tool
result included, with the local tokenizer. That ran on the event loop and stalled every other request on the worker
for about a second on a large tool result. The assembly now runs on a worker thread through asyncify from both the
reader's cancellation and the proxy's disconnect cleanup, shielded from cancellation, and round 1 is checked again
after the await so a release that happened meanwhile is never billed twice
* test(mcp): wait on the loop with a hang guard instead of a 300 ms deadline
* fix(mcp): give a proxy-rejected follow-up's failure row round 1's own cost breakdown
* refactor(streaming): fold the partial response's empty guards into one check
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test(responses): audit #47445
Integration cells for billing every round of a streamed MCP auto-execute on one spend row, on a native Responses upstream and the chat bridge, across tool rounds, a rejected or broken follow-up, a post-call guardrail block, a client hang-up, the response cache, and the next bridged turn
* test(responses): check a broken follow-up bills its own nonzero usage
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Adds a real-proxy case where the MCP follow-up call fails on every router retry, checking that the one failure row bills the first call of every attempt on chat completions and /v1/responses
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test(proxy): cover the max_parallel_requests slot a streamed /v1/messages MCP auto-execute keeps
A streamed /v1/messages request that auto-executes MCP tools replays its answer
as a stream after its model rounds already logged success. The stream path frees
the request's max_parallel_requests slot only on a client disconnect, so after
message_stop the slot stays taken and the key's next request answers 429
The test streams such a replay to the end and expects the slot to be free. It
fails until the fix lands
* fix(proxy): free the max_parallel_requests slot when a replayed /v1/messages MCP stream completes
A streamed /v1/messages request that auto-executes MCP tools runs its model
rounds without streaming and replays the final answer as a stream. The first
round claims the request's only success log, and its limiter callback keeps the
slot because the tool call reused it afterwards. The stream path released the
slot only on a client disconnect, so after message_stop the key stayed at its
max_parallel_requests limit and its next request answered 429
Stream cleanup now releases the stashed slot when the stream completed and its
logging object already claimed the success log, which only happens when the
stream replays a response that was logged before it started. A real stream
still releases through its own success callback
* test(mcp): free the key's only parallel slot after a streamed auto-execute ends
Run a streamed MCP auto-execute on a key with max_parallel_requests 1 on
every LLM surface, then send the next request on that key and expect 200.
On main both /v1/messages surfaces answer 429 here, while /v1/responses and
/v1/chat/completions already pass
* test(proxy): keep only the MCP stream regression tests, since #48628 already frees the slot at stream end
* test(proxy): require the MCP stream regression test's response to stream and reach its end event
* refactor(mcp): drop the experimental label from the MCP server and client paths
Move litellm/proxy/_experimental/mcp_server to litellm/proxy/mcp_server and litellm/experimental_mcp_client to litellm/mcp_client, with the mirrored tests moved under tests/unit/proxy/mcp_server and tests/unit/mcp_client. Both old packages stay as shims that warn with DeprecationWarning, re-export the new package's public names and alias every submodule in sys.modules so deep imports keep resolving to the same objects. The OpenAPI docs gate now puts the merge base's enterprise package on PYTHONPATH so its base counts do not mix the checked-out enterprise code with the base litellm tree
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci(mcp): keep tests/unit/proxy/_experimental as a sharded root for the shim tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(mcp): alias namespace subpackages and bind attributes in the deprecated MCP shims
pkgutil.walk_packages skipped mcp_server/auth because it has no __init__.py, so old imports of litellm.proxy._experimental.mcp_server.auth.* failed, and sys.modules entries alone left litellm.experimental_mcp_client.tools unreachable as an attribute. Walk the package on disk and setattr each alias on its old parent.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(mcp): upgrade to MCP SDK 2.3.0 and raise the per-event SSE size cap
MCP SDK 2.x reads Streamable HTTP responses through httpx2's SSE parser,
which rejects any single event over 1 MiB, so a large tools/call result
failed with 'SSE stream ended without a response'. mcp 2.3.0 exposes
max_sse_event_size, and MCPClient now passes a 64 MiB default,
configurable with LITELLM_MCP_MAX_SSE_EVENT_SIZE_BYTES
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(mcp): fall back for invalid SSE event limits
* test(mcp): verify configured SSE limits and recovery
* chore(mcp): remove redundant dependency comment
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Joshua Valluru <326636767+joshua-berri@users.noreply.github.com>
* test(mcp): cover one spend row billing both rounds of a chat completions MCP auto-execute
* fix(mcp): bill both model calls of a chat completions MCP auto-execute on one spend row
* fix(mcp): keep provider usage fields when summing both chat rounds of an MCP auto-execute
* fix(mcp): log each auto-executed MCP tool call under its own spend-log id
A chat MCP auto-execute logged its tool calls under the request's call id, which is also the id of the request's failure row. When the follow-up call failed, the database kept the tool row and dropped the failure row that bills the paid first call
* test(mcp): cover the margin, cache, failure row and parallel tool rows of MCP auto-execute billing
* fix(mcp): keep the gateway handler off the SDK path when no tool is an MCP tool
completion(), responses() and anthropic messages imported the MCP gateway handler
whenever tools were passed, and responses() did so on every call. Since the handler
began importing the MCP catalog at module level in #42568, that import needs the
optional mcp package, so a base SDK install failed with ModuleNotFoundError on any
function-tool call and on every responses() call
A new import-safe pre-check skips the handler unless some tool is a dict of type mcp,
which is the only shape the handler's gateway decision can accept. MCP gateway tools
still reach the handler as before
Adds in-process regression tests for the three entry points and a function-tool
completion to the base SDK install check
* fix(mcp): leave one-shot tool iterables unread in the gateway pre-check
responses() declares tools as an Iterable and the handler scans the same
object a second time, so a pre-check that read a generator would hand the
handler one tool fewer. The pre-check now scans only collections and reports
anything else as possibly naming the gateway, which keeps the previous path
The Anthropic messages regression test now mocks the httpx layer with respx
and turns off the aiohttp transport, instead of replacing AsyncHTTPHandler
* fix(mcp): scan only lists and tuples in the gateway pre-check
The pre-check read any Collection, but a Collection that yields its items only
once would still be exhausted by the handler's second scan. Only the list and
tuple builtins are guaranteed to iterate the same way twice, so they are the
only inputs read now. Everything else keeps the previous path and is reported
as possibly naming the gateway
The unread-input test now covers a generator, an iterator, a one-shot
collection and a deque
* fix(mcp): scan only exact list and tuple types in the gateway pre-check
isinstance accepted list subclasses, and a subclass that iterates only once
would still be consumed by the handler's second scan. The fast path now reads
only objects whose type is exactly list or tuple. Every other input, subclasses
included, keeps the previous path and is reported as possibly naming the gateway
The unread-input test gains a one-shot list subclass and a tuple subclass
* fix(sdk): decide MCP gateway routing without importing the proxy handler
The pre-check now holds the handler's exact gateway test, so remote MCP tools, mcp 1.x installs and any tools iterable stay off the handler, and the base SDK check calls responses() with a remote MCP tool.
* fix(sdk): send /mcp/<name> remote MCP URLs to the provider when the gateway registry cannot load
A path-only /mcp/<name> match now counts as a gateway tool only when the MCP server registry imports, so SDK installs without the proxy extras pass remote MCP servers like Zapier to the provider. Also drops a dead test patch.
* test(sdk): send a /mcp/<name> remote MCP tool in the base SDK install check
* style(sdk): drop docstrings from the MCP gateway pre-check and its tests
* test(mcp): cover /mcp/<name> gateway routing on the proxy and Messages tools on a base SDK install
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test(integration): give each owned proxy a private database and Redis by default
owned_proxy and owned_proxy_process now boot on a scratch database and a
per-proxy redis-server unless the caller opts into the worker's state with
state=OwnedState.SHARED_WITH_WORKER, passes its own DATABASE_URL, or strips
DATABASE_URL. Under the replica profile the private database gets the
writer and reader roles so the reader path stays covered.
Gateway carries the database_url its proxy writes to, and the Scenario
cleanup checks and delete_key_if_present read from it, so they keep
checking something on a private database.
The 189 test files with owned proxy sites that depend on the worker's
database or Redis opt in with a shares_worker_state module marker, which a
module-scoped conftest fixture turns into the shared state. The 7 helper
modules with such sites opt in per call, since conftest fixtures and other
test files reach them.
* test(integration): reach session fixtures with the shared marker and keep the worker retry default
The module-scoped fixture that applied shares_worker_state ran after
session-scoped fixtures, so proxies those booted went private in marked
files. The marker is now applied around each marked item's setup, call
and teardown, which covers fixtures of every scope.
Owned proxies on the worker database inherited the worker gateway's
router_settings.num_retries of 0 from it. A private proxy now gets the same
default in its config copy unless the config sets num_retries itself.
Five providers files read the owned proxy's spend rows or response cache
through the worker's database or Redis and opt into sharing.
* test(mcp): move the MCP OAuth happy path from a live Linear e2e to an integration test
GitHub Actions now runs unit tests only, and the MCP OAuth happy path needed a
captured Linear browser login to click through a real consent page. The OAuth
flows it exercised are already integration tests against an owned
authorization-server double, except one behavior: the stored per-user token
must be served by a proxy process that never saw the consent, with no shared
Redis to fall back on. That case is now in test_mcp_oauth_flows.py
The e2e test, its owned-gateway helper, its workflow, the mcp_oauth_live marker
and the JUnit gate script only that workflow used are removed
* test(mcp): chat completions run an OAuth server tool with the caller's own stored token
* test(e2e): drop the Linear chat-completion MCP OAuth e2e and its helpers
The behavior it checked, a chat completion running an OAuth server's tool with
the caller's stored per-user token, is now an integration test in
test_mcp_llm_endpoints.py that needs no saved Linear browser login
* test(mcp): decode the stored per-user credential as urlsafe base64
The plaintext check used b64decode, which drops the - and _ of a urlsafe value
and raised on roughly one ciphertext in five
* test(mcp): drop the chat OAuth integration test that duplicates test_mcp_slots F2-chat
security/test_mcp_slots.py already sends a per-user OAuth token through
/v1/chat/completions and asserts the peer receives it. Every mutant the new test
killed was also killed by the existing suite, so it added no signal
* fix(test-harness): preserve sibling auth modules in isolated worker configs
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): expect servers in MCP connect-flow descriptions
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): expect ptu_hours in daily activity metrics after #43043
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): kill every serving worker once the SCIM push blocks on the row lock
* fix(cost): price exact cost-map names before price-free capability rule matches
* test(streaming): price a priced alias stream at its deployment model and keep the unpriceable case unmapped
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(cost): price an explicit base_model under its own provider after candidate reordering
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(cost): retain priced generalized candidates and skip explicit-pricing lookups
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(cost): assert custom pricing makes no remote model-info requests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(cost): keep the per-candidate pricing provider final
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(router): pin a Bedrock application profile prompt for the one-hour ttl its base model sends
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): assert the Nova Sonic tool-call argument delta carries the full arguments
* fix(cost): sort cost candidates from the static cost map and rules, without provider lookups
* test(integration): bill route-priced Bedrock chat streams and responses, and pin application profiles for an hour on messages and cache hits
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Yuneng Jiang <yuneng@berri.ai>
* fix(responses): log and bill every round of non-stream MCP auto-execute
Both model rounds run as internal sub-calls on the request's logging object, and the outer call logs the response the client received with usage, cost, and cost breakdown summed across the rounds. The internal-call gate now also covers the sync success callbacks, so an inner round no longer takes their first-wins slot
* fix(caching): let the parent call log and bill an internal sub-call's cache hit
A round of non-stream MCP auto-execute that the response cache served fired the
cache-hit callbacks on the shared logging object, so the spend log kept round 1's
function_call row and dropped the billed row for the answer. Internal sub-call cache
hits now fire no callbacks and carry a zero cost, and the summed cost breakdown leaves
out rounds served from the cache
* fix(logging): keep a parent row unique when its response came from the response cache
A RAG query or an MCP turn whose final round is served from the cache hands back the cached response's id, which already belongs to the row that cached it, so SpendLogs dropped the new row as a duplicate. Such a row now gets the cache-hit id suffix, and it is logged as a cache hit when it cost nothing.
* fix(logging): skip the zero-cost warning for a response served from the cache at no charge
A non-stream MCP auto-execute request whose rounds the response cache served is priced
at $0 before the parent marks it a cache hit, so it logged a false 'priced at $0'
warning and counted in litellm_zero_cost_requests_total. The diagnostic now treats a
response the cache served at no charge the same as a marked cache hit
* fix(responses): bill the paid first MCP round when a later step of the request fails
* fix(responses): log each MCP tool call under its own id so a failed follow-up keeps its spend row
The non-stream auto-execute path passed the request's call id to the tool call, so the tool's spend row and the request's failure row shared one request_id and the failure row was dropped on insert, leaving the paid first round unbilled.
* fix(responses): bill round 1 on a client disconnect, keep MCP tool rows in the request's session, and charge a fixed margin once per request
* fix(responses): keep bridged MCP auto-execute in one session and trace and continue its conversation
The bridged follow-up no longer chains on round 1, so a thinking model keeps its reasoning and the request stays in one session. The tool call log continues the request trace without renaming it, the tools are fetched once, and a later turn finds the MCP turn under the id the client got, skipping the tool rows.
* fix(responses): log the non-stream MCP tool listing in the request's session and trace
* fix(responses): find a bridged MCP turn by the id the client holds and log its listing in the bridged trace
* fix(responses): derive the MCP tool row id from the request's call id
The tool call's spend row now uses <call id>_<tool call id>, so it stays
linked to the request without colliding with the request's failure row.
Also tighten the internal cache-hit test to a fixed clock and typed params.
* fix(responses): bill a retried MCP attempt for its own rounds
A router retry or fallback reuses the request's logging object, so a failed attempt's stashed round 1 usage made the next attempt's success row log stale tokens. Each non-stream MCP attempt now starts without that stash.
* fix(responses): bill every failed MCP attempt's paid first round on the request failure
A retry or fallback now sets a failed attempt's stashed round 1 aside instead of dropping it, and the proxy's failure row adds it to the last attempt's own, so a request whose later attempt fails on its first call still bills the round the provider charged. A cached final round also keeps the paid first round's cost breakdown.
* fix(responses): keep MCP tool catalog and results out of the bridged next-turn replay
* fix(responses): keep MCP tool spend, client body keys, and Langfuse traces working on auto-execute
Count the MCP tools an auto-executed Responses API call ran in the tool spend rollup, accept a client body key named previous_response_client_ids without a 500, and point MCP listing and tool spans at the trace Langfuse gives the parent request, including x-litellm-session-id requests
* fix(responses): join MCP spans to the Langfuse trace when the request sends body metadata
* test(integration): cover non-stream MCP auto-execute billing, failure rows, and bridged follow-ups
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test(integration): hand every harness server a port from one per-worker block
About 45 places in the integration harness picked a port by binding port 0,
reading the number back, closing the socket and starting a server on that
number later. Another process could take the port in between, which surfaced
as "Address already in use" flakes. Earlier fixes retried around the race in
single call sites without removing it.
tests/integration/_support/ports.py now owns every port the harness picks
ahead of a server. Each pytest process claims a 200-port block between 10000
and 32767 by taking an exclusive flock on a per-block lock file in the system
temp directory, so parallel runs on one machine never share a block and the
lock goes away with the process. The range is above the fixed service ports
and below the Linux and macOS ephemeral ranges. Ports go out in order, are
never reused in a run, skip anything an outside process already holds, and
a process that exhausts its block claims the next free one.
The three _free_port copies, worker_endpoints' port-0 binds,
free_loopback_port, the owned proxy's lost-port-race retry and the owned
Redis retry loop are gone. Tests that restart a server on the same port now
allocate that port up front instead of reading it back from the first
server. TQ012 in scripts/check_test_quality.py rejects a port-0 bind read
back with getsockname anywhere under tests/integration.
* ci(integration): record listening sockets and the ephemeral port range
* Revert "ci(integration): record listening sockets and the ephemeral port range"
This reverts commit d3fafc3da2.
* fix(responses): open faked stream items empty so tool arguments and text arrive once
* fix(responses): open faked stream items in progress with a fresh empty list, and open mid-stream guardrail blocks empty
* fix(responses): open faked refusals, reasoning summaries, custom tool calls and MCP calls empty so each value arrives once
* test(integration): cover the empty-open Responses stream items on every fake, replay and block path
Integration cells for the faked /v1/responses stream, the chat and messages bridges, cache replay, guardrail block events and the MCP gateway call. Each opened item starts empty and each value arrives once, checked with raw httpx, the OpenAI SDK and the Anthropic SDK, plus upstream errors, 5 KB values, a client drop, a dropped upstream connection and a worker SIGKILL
* test(integration): cover mcp_runtime endpoint contracts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(integration): ruff format and import order in mcp runtime tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): assert out-of-path calls and untrusted forwarded headers are refused
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): move the untrusted forwarded-header discovery case into its own test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): hash the token row lookup, widen spend polling, and union open servers after listing
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): tighten mcp runtime contracts per review (sdk client, toolset scope, forwarded trust, callback html, session tools/call)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): assert the sdk unknown-path error without pinning the sdk's exception wrappers
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): cover trailing-path and group csv mcp paths, rfc 9207 iss-less and error-iss callbacks, and id_token exchange
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): cover lite logout spelling, team-mapped and unmapped IdP subjects
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): cover per-server mcp-session resource and root /token spelling
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): unpack the sealed client code in the step-up tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): cover mcp_management endpoint contracts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): require server_not_found for pending and rejected submissions
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): pin per-worker submission reloads, toolset and alias-collision spellings, raw key overrides
Assert the approving and rejecting worker immediately and only wait on the peer, add an outsider key and exact upstream calls, list by toolset_name, cover a config alias shadowing a DB server_name, send key_hash as a raw sk- key, and add BUG skips for API disallowed_tools and overrides on keys or teams without object_permission
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): pin XFF trusted-peer, load balancer chains and sibling refusal body
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): enforce output_policy trust chain, JSON-RPC XFF views, MCP tool-policy BUG pins
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(mcp): force a registry reload during the pending window, pin the admin default listing and the whole tools/call body
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): give chaos and timing tests a harness API and gate raw signals
Chaos tests killed and suspended proxy workers with raw os.kill and psutil
calls and bounded elapsed time with fixed literals, so they raced the uvicorn
worker healthcheck and failed on slow CI boxes
OwnedProxy now offers kill_worker and replacing_worker, which return only once
a replacement answers readiness with its own pid, and suspended, which is
refused unless the proxy was launched with a lenient worker healthcheck.
timing.py bounds durations against a measured control or the injected delay.
run.py measures the proxy import time once and derives readiness deadlines and
the per-test timeout from it, writing boot-calibration.json
check_test_quality.py adds TQ010 (raw process signals) and TQ011 (elapsed time
asserted under a numeric literal) for tests/integration outside _support, and
every call site is migrated so both report zero with no suppressions
* test(integration): steer the assistants wire burst onto both workers
The mixed-status burst assumed the OS would spread 48 connections across both workers and failed when one worker took them all. Each half is now sent while the other worker is suspended, and the test asserts an exact 24 and 24 split
* test(integration): restore product-enforced timing bounds lost in the migration
Some TQ011 migrations dropped or loosened a bound whose time is the behaviour
under test. The websocket session cap only kept its floor, so a cap closing at
300 s would have passed, and several timeouts were bounded by the injected stall
instead of the configured product value
assert_enforced_within bounds an elapsed time by the configured value plus
explicit slack, with an optional floor. The websocket cap, the MCP validation
timeout, the guardrail litellm_params timeout, the prisma command timeouts, the
baseline projection lock timeout and the key metadata statement timeout use it.
The device code refusal keeps its 15 s margin, and the langfuse retried prompt
fetch is now bounded by a cold fetch of a steady prompt as well as by the
ignored retry-after
* fix(tests): wait for the supervisor to reap killed workers and fix real-kill migrations
The lenient worker healthcheck was the whole case budget, so uvicorn's supervisor sat
in a ping on the killed worker for up to that long before respawning it. It now uses
the graceful stop budget, and a worker kill returns only after the victim is reaped,
so pooled connections no longer race the dead worker's socket.
Several tests on main picked the multiprocessing resource tracker as their victim and
never killed a worker. Now that they kill a real one, presidio counts streams cut by
the death, delete_model waits for the config reload that removes a deleted model from
a replacement worker, and the wedged coordination Redis test kills before the wedge
and waits for every parked publish to fail before resuming Redis.
The redis cluster test imports the harness through the tests package so it collects
in the unit job, and the per-model fallback rig exposes the owned proxy it now needs.
* test(observability): only require S3 deliveries the killed worker could not have buffered
A real SIGKILL drops the victim's unflushed batch, so requests it already answered can't be expected to land. The test now requires every request served after the replacement to land, some of the surviving worker's burst share to land, and nothing unsent to land
---------
Co-authored-by: Yuneng Jiang <yuneng@berri.ai>
The probe test asserted that every request the peer saw carried the forwarded token, so a credential-less tool discovery session from either gateway (startup mapping, an OpenAPI registration elsewhere, or the 30s MCP reload) landing in the window failed it intermittently on the CircleCI mcp layout
The caller now sends an allowlisted x-integration-caller marker, which the gateway forwards on the caller's listing and call sessions but never on the probe or on background discovery. Probes and marked requests must carry exactly the forwarded token, the single tools/call must be one of the caller's, and anything left over may not carry any Authorization at all. The admission key scan still covers every request
Verified on a local two-gateway rig sharing Postgres and Redis while a thread registered and deleted OpenAPI servers on both gateways. The old assertions failed 207 of 240 runs there with the CI signature, the new ones 0 of 240. Dropping the token on the call session, on the listing session, or on the probe still fails all six cases
Co-authored-by: Yuneng Jiang <yuneng@berri.ai>
* test: migrate proxy security, schema migration and MCP legacy tests
* test: run the MCP proxy e2e upstream under the SDK1 peer in integration-mcp
* test: warm the baseline store before timing the locked projection
* test(e2e): hold the stack exclusively while the semantic filter flag is flipped
* test(mcp): scope the in-process proxy e2e fixtures to their module
The session-scoped autouse fixture deleted DATABASE_URL and overrode LITELLM_MASTER_KEY for the rest of the worker, which broke every later test in the integration mcp suite
* test(e2e): assert the semantic filter keeps listed tools instead of a ranking
Which Datadog tool an OpenAI embedding ranks first is not something litellm owns, so the test now checks that the kept tools are a non-empty subset of the key's listing within top_k
* test(mcp): keep the in-process proxy e2e tests on one worker
loadgroup spread the file's tests across workers and interleaved them with other files, so the module-scoped in-process proxy was torn down and restarted between tests and requests timed out
* test(mcp): name the upstream requests that carried the wrong token
The bare all() assertion hid which request broke the forwarded token, which made an intermittent CircleCI failure undiagnosable
---------
Co-authored-by: yuneng <yuneng@berri.ai>
* test: move guardrails, image, audio, batches, search legacy tests to unit
* test: restore the custom guardrail and remaining chunk 1 unit rows
* test: move the ocr matrix to e2e
* test: drop comments from moved tests and fix vertex ocr credentials
* test: move litellm utils and unified google legacy tests to their tiers
* test: move logging callback and pass-through unit legacy tests to their tiers
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: move store_model_in_db legacy tests and start the admin ui migration
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: finish the proxy job rows and drop their CircleCI jobs
Moves the remaining multi-instance, openai endpoint, otel, pass-through and
top-level proxy tests into tests/integration with scripted upstreams, rewrites
the weak replacements to assert the outbound wire and exact responses, and
removes the build_and_test job group this PR owns from CircleCI
* test: tighten the key generation and error log assertions, rename migrated unit files to their mapped names
* test: shard the pagerduty unit tests and record the rust harness file on the coverage allowlist
* test: drive the SSO sign-in tests through a scripted IdP and isolate the cost center stand-in
* test: type the e2e beta header, prompt cache and secret manager values
* test: assert the missing model error message on the OpenAI routes
* test: drop the deleted otel base class import and restore mcp redaction and exact langfuse usage checks
* test: make the langfuse fixture directory a package for the layout check
* test: restore collected no-match cases, real langfuse and gcs payload checks, and gate the aws secret lane
* test: keep the ocr e2e pdf fixture next to the test
* test: put meta below covers on the streaming tool search e2e
* test: run the OCR matrix against a scripted upstream and fix two runner-only e2e failures
* test: count only OCR calls in the OCR matrix recorder
* test: pin OCR parse bodies and billed cost, group new e2e cases into classes
* ci: restore the python image anchor dropped in the main sync
---------
Co-authored-by: yuneng <yuneng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* perf(ci): distribute unit cases and reuse security sweep connections
* revert(ci): reject first speed trial after new failures
* perf(ci): avoid repeated test discovery and parsing
* perf(ci): isolate parallel security workers
* revert(ci): restore measured security execution isolation
* perf(tests): reuse source scans and deduplicate cache cleanup
* perf(ci): restore parallel security with safe log batching
* test(ci): isolate fixture state and schema comparisons
* test(ci): make schema and scheduler fixtures deterministic
* test(ci): drain queued logging before fixture teardown
* test(ci): preserve logging delivery without idle worker restarts
* test(ci): keep guardrail sinks available during parallel bursts
* test(ci): isolate full proxy lifecycle from gateway imports
* test(ci): preserve first failure responses and sweep evidence
* test(ci): restore documentation URL settings between cases
* test(ci): isolate open MCP tools in private gateways
* test(ci): group private MCP cases and handle exited workers
* test(ci): retain proxy exits and MCP failure details
* test(ci): repair merge interactions with main
Key the schema shard's full clone on artifact-name, since main removed the
unit-flag input. Give each owned Redis its own log file again. Hand the
realtime transcription tests main added a logging object whose dispatch
handlers are awaitable, so the per-test logging drain no longer waits on a
MagicMock task forever
* perf(ci): shard owned integration cases and close lifecycle leaks
* fix(ci): qualify shard execution and harden interrupted cleanup
* fix(ci): bound worker supervision and own fault traffic
* perf(ci): reuse unparametrized fixtures across bounded case groups
* fix(ci): preserve boot fault allowances and complete exit records
* test(ci): tolerate startup retries and retain readiness details
* perf(ci): balance integration shards by recorded duration
* fix(ci): load migration database helpers only for workers
* fix(mcp): never forward the caller's LiteLLM key to MCP servers
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(mcp): tidy caller-key scrub
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(mcp): avoid master-key scanner false positive
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(mcp): match admission when scrubbing the caller key
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(mcp): drop header copy churn
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(mcp): normalize bearer variants in caller-key match
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(mcp): inject admission header name and cover provider keys
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(mcp): exercise real REST header extraction
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(mcp): cover caller-key scrub on the real REST path
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(mcp): exclude gateway admission header from upstream forwarding
* fix(mcp): skip empty admission headers when scrubbing keys
* fix(mcp): use scrubbed server credentials for auth probes
* fix(mcp): match probe credential precedence to upstream calls
* test(mcp): simplify probe header assertion
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Joshua Valluru <326636767+joshua-berri@users.noreply.github.com>
* feat(guardrails): support logging_only mode for the Akto guardrail
* test(guardrails): wait for the spend row before reading Akto calls and cover mixed modes
* test(guardrails): cover Akto logging_only on MCP tool calls
* test(guardrails): type the Akto logging_only unit tests and inject the HTTP handler
* test(guardrails): cover failing Akto replies, provider failures and a mixed outage burst under logging_only
* test(guardrails): cover an unreachable Akto with fail_open under logging_only
* test(mcp): fix stale bridge-hook, applied-guardrails and pagination-revoke integration tests
* test(mcp): clear the spare direct grant when restoring the access-group policy
---------
Co-authored-by: yuneng <yuneng@berri.ai>
* fix(mcp): refresh server catalog for each gateway operation
* fix(mcp): reject configuration changes during scoped dispatch
* fix(mcp): refresh shared catalog state for each operation
* fix(mcp): refresh catalog before native alias routing
* docs(mcp): clarify native route resolution order
* test(mcp): align streaming fixture and generated API documentation
* fix(mcp): preserve discovery published during catalog refresh
* fix(mcp): coalesce queued catalog reads without a stale window
* fix(mcp): reconcile concurrent route changes when publishing catalog
* test(mcp): provide catalog scope in post-call hook fixtures
* fix(mcp): retain valid live routes and handlers during refresh
* fix(mcp): keep refreshed OpenAPI operation membership authoritative
* test(mcp): preserve logging fixtures after catalog integration
* fix(mcp): isolate transport test admission state and exhaust OAuth outcomes
* refactor(mcp): narrow catalog consistency change to ticket scope
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(mcp): gate catalog refresh on a database revision marker
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(mcp): refresh waiters that observed a newer catalog revision
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(mcp): awaitable catalog revision doubles in prisma mocks
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(mcp): satisfy catalog refresh lint and type gates
* test(mcp): provide catalog scope in toolset fixtures
* test(mcp): exercise temporary OAuth through catalog operations
* fix(mcp): share active catalog snapshots across discovery tasks
* fix(mcp): retain discovered routes across cached catalog operations
* fix(mcp): preserve local tool ownership across discovery
* refactor(mcp): satisfy tightened immutability lint ceiling
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(mcp): authorize local handlers by registered server ownership
* fix(mcp): check registered handler freshness and isolate fixtures
* fix(mcp): read catalog revisions and snapshots from the writer
* fix(mcp): resolve access groups from the operation catalog
* fix(mcp): preserve empty access group restrictions
* fix(mcp): retain rediscovered routes across concurrent updates
* fix(mcp): enforce registered ownership for local tool dispatch
* fix(mcp): preserve issuer discovery across catalog snapshots
---------
Co-authored-by: Joshua Valluru <326636767+joshua-berri@users.noreply.github.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(mcp): hand listed-tool metadata to pre-call hooks with per-caller catalog identity
Track the tools each MCP server listed per caller identity so pre_mcp_call and during_mcp_call hooks receive the tool description and input schema the client saw. Servers with no caller-dependent inputs share one slot; user identity, forwarded headers, stdio env, relayed bearers, and server-specific auth get their own. Local registry and OpenAPI paths pass the registered metadata and admin description overrides. The Agent 365 guardrail reads the new fields into its evaluate payload.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(mcp): drop the listed-tools empty sentinel and routine test docstrings
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(mcp): mark the listed-tools cache digest as a non-security hash
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(mcp): key the listed-tools cache by the OBO subject token
token_exchange servers list upstream with the caller's own Entra bearer, so two callers on one
LiteLLM key with different subjects were sharing a catalog slot
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(mcp): resolve the BYOK credential before keying the listed-tools slot
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(mcp): drop the OAuth discovery cache when a server definition changes
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(mcp): drop a diff-narrating comment
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(mcp): never validate a supplied header on the tools/list BYOK path
The pre-listing resolver ran the tool-call byok_auth_required check even
when the caller already supplied x-mcp-auth, and it ran outside the
per-server error boundary, so a single deprecated-header caller dropped
the server from the aggregate list. Listing now returns a supplied header
unchanged and falls back to the stored credential without raising
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(mcp): assert the BYOK listing lands in the caller's listed-tool slot
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(mcp): cover the deprecated string x-mcp-auth header on a BYOK tools/list
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(mcp): key the per-caller listed-tool slot by the hashed token
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(mcp): key discovery cache by the hashed token
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(mcp): key discovery caches per caller correctly and drop stale caches on server updates
Discovery-list cache identity now uses the hashed token instead of the raw
api_key and treats MCPJWTSigner-signed servers as per caller. Server
definition changes also drop the cached upstream OAuth metadata. OpenAPI
listings look tools up under the normalized registry prefix with the
separator, so an overlapping sibling prefix no longer leaks into the list.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(mcp): keep the discovery cache digest call unchanged so CodeQL matches the existing alert
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(mcp): derive the listed-tool caller identity from the discovery cache key
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(mcp): guard OAuth metadata cache writes with a per-server generation and drop unproven per-caller discovery keys
An upstream metadata fetch that started before a server edit could store its stale reply after
invalidate_oauth_metadata_cache ran. Invalidation now bumps a per-server generation and the fetch
only stores when the generation it captured before I/O is unchanged.
The MCPJWTSigner-based per-caller discovery classification and the api_key to token key change had no
reproduction (the signer only injects on tools/list, and UserAPIKeyAuth hashes api_key in place), so
both go back to the merge-base behavior.
Integration coverage under tests/integration/mcp: overlapping OpenAPI aliases, a config-declared
server name with a space, OAuth metadata refetch after a save, and the in-flight stale-write race
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(mcp): keep OAuth metadata generations only while a fetch is in flight
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(mcp): count queued OAuth metadata fetchers so invalidation survives lock handoff
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(mcp): keep a held OAuth metadata lock registered even when no fetcher slot claims it
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(mcp): prove a peer worker drops stale upstream OAuth metadata after a save elsewhere
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(mcp): return one masked text per scanned string in the selected-guardrail REST test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(mcp): fold the signed caller into the discovery digest instead of a second key hash
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(mcp): satisfy type discipline gate on listed-tool identity
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(mcp): hand tools/call hooks the exact catalog entry tools/list served
get_listed_tool re-applied the admin description override on top of the cached listing, so a
guardrail-masked description was restored to its original wording at call time, and the OpenAPI /
local-registry call path built its metadata from the registry instead of the guarded caller catalog.
Both paths now return the cached entry as served, falling back to the registry only when no listing
was recorded
Adds tests/integration/mcp/test_mcp_listed_tool_metadata.py (red on the prior head for the two
regressions, red on the merge base for the feature, green on this head)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(mcp): key OpenAPI listed-tool entries per caller so tools/call reads its own guarded listing
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(mcp): align listed-tool slot tests with per-caller keying
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(mcp): keep oauth2 listing on the minted or signed credential, not the stored BYOK secret
The listing helper that keys the per-caller catalog by the stored BYOK credential also handed that
credential to the upstream client, which on an oauth2 server short-circuited the client_credentials
mint and the MCPJWTSigner gate. Split the two: the catalog identity keeps the stored credential so
tools/call finds the caller's slot, while an oauth2 server's tools/list sends only the per-request
header, letting the M2M mint or signed JWT proceed as on main
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(mcp): oauth2 BYOK listing sends the minted token, not the stored secret, through the real proxy
Integration cell for the listing fix: a client_credentials BYOK server with a stored user credential,
one tools/list as that user, the peer must see a live minted bearer and one /token mint. Red at the
pre-fix tip (zero mints, stored secret upstream), green at the fixed head and at the merge base
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(mcp): keep the stored BYOK credential for catalog identity only on tools/list
Listing used the resolved stored credential both to key the caller's catalog slot and as the
upstream transport header, so REST api_key and bearer_token listings sent the user's secret
instead of the server's static token and the MCPJWTSigner gate went quiet. The upstream client
and the signer gate now read the caller-supplied mcp_auth_header for every auth type, exactly
as before the catalog existed, and the stored credential only names the slot tools/call reads
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(mcp): type the listed-tool metadata read from pre-call kwargs for the basedpyright gate
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(mcp): hand never-listed tools/call hooks name and arguments only
The local-registry call path fell back to the registry entry with the admin description override
when no tools/list had been recorded for the caller, so a pre_mcp_call guardrail scanned a
description the caller was never served and blocked OpenAPI calls that passed before, and base's
own selected-guardrail REST test failed on the two-text redaction. _registered_tool_metadata now
returns the listed entry or None, so a tools/call with no prior listing sends name and arguments
only as promised, and that REST test double goes back to its base shape
* fix(mcp): keep during_mcp_call hooks on name and arguments only
call_tool handed the caller's listed entry to the during-hook task as well, so during_mcp_call
guardrails scanned the description line and schema leaves of any listed tool after the upstream
call had already run, blocking calls that passed before whenever the policy matched the
description, returned a fixed-length texts list, or hit the depth guard on a deep schema. The
listed entry is only disclosed for pre_mcp_call, so the during task no longer receives it and its
request object carries no description or schema, as before
* fix(mcp): key the BYOK catalog slot by the client's header, not the stored credential
tools/list resolved the stored BYOK credential to pick the caller's catalog slot, which read the
credential store before the classified try block. With Postgres down and a cold per-worker cache
that made every REST tools/list on an is_byok server fail with tools=[] and no upstream call, and
the read seeded the per-worker cache (including a negative entry), so a tools/call on another
worker after a store, rotate or revoke on this one kept using the stale value.
The slot is now keyed by what the client supplied plus the caller's hashed key, on both sides.
_get_tools_from_server and call_tool take a keyword-only catalog_auth_header that defaults to
mcp_auth_header as received (the default is the builtin Ellipsis so it survives a module reload).
The /mcp fan-out and execute_mcp_tool, which swap the resolved credential into mcp_auth_header,
pass the client's value explicitly. What goes upstream is unchanged. _byok_catalog_auth_header is
gone.
* fix(mcp): drop a listed catalog recorded across a server save
_record_listed_tools ran after the awaited upstream fetch, so a PUT /v1/mcp/server that landed
mid-fetch had its invalidation undone when the fetch completed: hooks then saw the pre-save
description next to the post-save definition until the next listing, instead of name and
arguments only.
The manager now keeps a per-server listed-tools generation, bumped by
_invalidate_server_definition_caches. _get_tools_from_server reads it before the fetch and
_record_listed_tools skips the write when it moved; the next listing records normally.
* fix(mcp): drop the catalog again once a saved OpenAPI server's registry is rebuilt
add_server and update_server publish the saved definition before the OpenAPI registry entries are
rebuilt from the spec, so a listing recorded during that fetch held the pre-save entries under the
new generation. The generation is bumped a second time after the registry refresh.
The during-hook task no longer accepts a listed entry, the one-line wrapper over get_listed_tool is
inlined at its two call sites, and the per-server generation map is a plain dict.
* fix(mcp): keep discovery and OAuth metadata caches across an OpenAPI spec re-read
add_server and update_server ran the full server-definition invalidation a
second time after the awaited OpenAPI spec fetch, which also dropped the
prompts/resources/templates discovery entries and the OAuth protected-resource
metadata filled under the already-published definition, so the next request
went upstream again. Only the listed-tool catalog recorded during the fetch
holds pre-save entries, so the post-fetch pass now drops just that catalog
and bumps its generation via the new _drop_listed_tools helper, which the
full invalidation also calls.
* fix(mcp): look a called tool up in the listed catalog by its bare name only
get_listed_tool stripped the server prefix a second time when the exact
name was absent from the caller's listing, so a never-listed upstream tool
whose bare name starts with the server prefix resolved to the listed
sibling and that sibling's description and input schema reached the
pre-call hooks for a call to a different tool. Every caller already passes
the once-stripped bare name, so the lookup is now exact.
Tests that looked the catalog up by a prefixed name now use the bare name
the callers pass; two new tests pin the never-listed sibling case at the
manager and at the tools/call path.
* fix(mcp): record a listed-tool catalog only for a listing the caller is served
_get_tools_from_server now records the catalog into the caller's
listed-tools slot only when asked (record_listing=True), which the
served listings pass: the /mcp and Responses API tools/list handlers via
_get_tools_from_mcp_servers, MCPServerManager.list_tools, and the REST
listing via _list_server_tools. Four internal listings stop recording,
so a later tools/call hands pre_mcp_call hooks name and arguments only,
as on main:
- _list_tools_before_first_call, the implicit listing inside tools/call
when this worker does not yet expose the tool
- fetch_pinnable_tool_catalog, the admin pin snapshot listed without the
catalog guard and without description overrides
- _initialize_tool_name_to_mcp_server_name_mapping, the startup fill
- get_tools_for_server, used by the semantic tool filter
_create_prefixed_tools returns to its tool-name mapping job only; the
record follows it in _get_tools_from_server.
* fix(mcp): opt every listing out of catalog recording unless it is served
The aggregate listing and _list_mcp_tools now default to record_listing=False,
so a catalog fetched inside a tools/call no longer fills the caller's
listed-tools slot. The /mcp/proxy meta-tools (call_tool, search_tools,
get_tool_schema) and the tool-search virtual tool stop recording: /mcp/proxy
serves only the meta-tools and the search serves only its hits, so a later
pre_mcp_call hook was reading a description the caller never listed.
The tools/list handler, the Responses MCP handler and the /v1/mcp/tools
management listing opt in with record_listing=True, since each serves the
catalog to the caller.
* fix(mcp): key the listed-tool slot by the caller's admission identity and forwarded bearer
The slot a tools/list records for a later tools/call was keyed by (user_id, api_key)
only, so every team-only JWT caller shared one slot and one JWT user acting in two
teams shared a slot; a tools/call then handed pre_mcp_call hooks a description another
caller was served. The slot is now keyed by the hashed key, user, team and organization,
plus the admission credential of a caller admitted with neither a key nor a user.
The caller bearer split the slot only on client-forwarded-token and token-exchange
servers; a legacy delegated oauth2 server (delegate_auth_to_upstream without client
credentials) also forwards it upstream and served a different catalog per bearer into
one slot. The bearer now splits the slot on every server whose egress forwards it
(_consumes_caller_authorization) or exchanges it.
* fix(mcp): record only tools served by the bridge
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(mcp): keep bridge tool metadata request-local
* refactor(mcp): centralize listed catalog recording guard
* fix(mcp): preserve base TPM reservations for listed tool calls
* fix(mcp): preserve project token reservations for listed calls
* fix(mcp): record served catalogs and preserve call message bytes
* test(mcp): align listing expectation with deferred recording
* test(mcp): audit listed metadata across callers and bridge lifecycles
* test(mcp): preserve guardrail fixture worker affinity
* refactor(mcp): expose listed catalog recording API
* feat(mcp): pass served_tools through the anthropic messages bridge
/v1/messages auto-execution now hands this request's resolved tool
definitions to _execute_tool_calls, matching the Responses and chat
completions bridges: the pre_mcp_call hook receives the description
and input schema the model was shown for that call. Request-local
only; the shared listed-tools catalog is untouched.
* test(mcp): pin served_tools handoff on the anthropic messages bridge
Mirrors the credentials-forwarding test: the request's resolved tool
definitions must reach _execute_tool_calls under served_tools so
pre_mcp_call hooks judge the call on the description and input schema
the model was shown. Fails without the previous commit's one-liner.
* style(mcp): sort the local import block ruff flagged
* fix(mcp): keep the admin include_disabled_tools view off the listed-tools catalog
GET /mcp-rest/tools/list?include_disabled_tools=true is the admin-only
configuration view: apply_tool_filters is False, so it serves the full
server catalog. Recording that response into the caller's listed-tools
slot warmed tools/call metadata no runtime listing ever served,
breaking the only-a-served-listing-records invariant (Bugbot).
The record is now gated on apply_tool_filters; disabled tools stay
unreachable (the call-time allowlist 403 fires before hooks), so the
observable fix is the slot no longer warming from a settings view.
Verified live: the new test fails on the unfixed head and passes here,
and the rest of the listed-tool-metadata suite is unchanged.
---------
Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): answer the model-info refresh GET in the mixed MCP responses wire peer
The proxy's periodic model-info refresh sends GET /v1/models to the deployment api_base, which tripped the peer's /responses-only assertion when a tick landed mid-test
* test(e2e): wait for the api-keys URL after clicking Virtual Keys in onboarding
/ui already renders the Virtual Keys heading, so the helper returned before navigation finished. The late route change moved focus and closed the account menu popover in hideLiteAdmin
* test(proxy): stop two unit modules leaking app.openapi_schema and a session-wide Router
test_custom_openapi cached a stripped schema on app.openapi_schema and never cleared it, breaking later openapi route tests. test_proxy_reject_logging built a module-level Router that stayed in the live router registry all session and re-added cost-map keys during a reload. Reset the schema via monkeypatch and make the Router a function-scoped fixture
* refactor(mcp): extract upstream preparation and support modern clients
* fix(mcp): reject incompatible upstream transport before saving
* fix(mcp): serialize protocol validation with server updates
---------
Co-authored-by: Joshua Valluru <326636767+joshua-berri@users.noreply.github.com>
