diff --git a/docs/plans/provider-executed-mcp.md b/docs/plans/provider-executed-mcp.md index 98b786a02e5..328a44f9b32 100644 --- a/docs/plans/provider-executed-mcp.md +++ b/docs/plans/provider-executed-mcp.md @@ -1,6 +1,6 @@ # Provider-executed MCP registration -Status: registration and permission integration are approved and pushed through `f1bb8d8ddd`. Native request authorization is implemented and approved for commit and push +Status: registration, permission integration, and native request authorization are approved and pushed through `18cd30435c`. The provider registration UI and generated API schema refresh are verified and approved for commit and push ## Implementation plan @@ -10,9 +10,9 @@ Work through these layers in order on `litellm_provider_mcp_registration`. Updat | --- | --- | --- | | 1 | Settle registration fields, values, storage, and validation | Approved | | 2 | Implement persistence and execution-mode isolation before enabling provider registration | Database, isolation, and admin API committed; YAML registration and provider input model reviewed and approved | -| 3 | Management UI/API, stored tool options, and existing key/team/toolset selectors | Scoped retrieval, assignment, and stored tool selectors implemented and approved. Provider registration form remains separate | -| 4 | Authorize effective OpenAI Responses and Anthropic Messages MCP attachments through existing MCP permissions | Implemented and approved | -| 5 | Live provider tests, streaming admission tests, UI tests, and gateway regressions | Request admission and gateway regressions verified. Real OpenAI JSON/streaming passed. Real Anthropic calls blocked by invalid QA credential. Earlier browser QA remains incomplete | +| 3 | Management UI/API, stored tool options, and existing key/team/toolset selectors | Scoped retrieval, assignment, and stored tool selectors approved and pushed. Provider create/edit/detail UI verified and approved | +| 4 | Authorize effective OpenAI Responses and Anthropic Messages MCP attachments through existing MCP permissions | Approved and pushed in `18cd30435c` | +| 5 | Live provider tests, streaming admission tests, UI tests, and gateway regressions | Request admission and gateway regressions verified. Real OpenAI JSON/streaming passed. Real Anthropic legacy/modern JSON and SSE calls, cache revocation, and token accounting passed with the updated QA credential. Registration browser CRUD, empty-tool persistence, and config read-only QA passed without MCP discovery or execution | Request authorization enforces native attachments before provider dispatch using the completed permission integration. Preserve the current sources of truth and lookup paths. Object permissions remain database-backed and use the existing auth caches. Key assignment keeps its database lookup and gains recognition of provider-only YAML records in its config fallback. Scoped lists keep their existing permission calculation and hydrate authorized provider IDs from the already-loaded registration inventory. Provider tool selectors use stored allowed tools and descriptions. Gateway lookup, discovery, and execution retain their existing paths @@ -24,6 +24,48 @@ Chat Completions entry routes, response/SSE inspection, per-invocation argument The registry design review is deferred by agreement on 2026-11-02. This batch retains the executable registry and separate provider registration inventory. A later revision may publish both modes through one registry and derive executable views at gateway boundaries. No registry consolidation is included in this approval +## Registration UI review + +The existing Add New MCP Server flow offers LiteLLM-executed and Provider-executed registration. Provider mode mounts a separate compact form shared by create and edit. It collects name, optional alias/description, HTTPS URL, exact allowed tool names, access groups, and allow-all-keys. The payload contains no transport or credential settings. Create sends execution mode; update omits it because mode is immutable + +Provider cards show execution mode and allowed-tool count. Details show stored tools without mounting the gateway tool viewer. Connection tests, stored credentials, and gateway cost settings are hidden for provider registrations. YAML registrations retain their read-only settings. Empty allowed-tools lists retain the enforced deny-all policy + +Validation passed 302 focused UI tests, TypeScript, and the production build. Changed-file ESLint has no errors. A live browser against the proxy and a disposable PostgreSQL database verified create, stored-tool detail, edit to an empty list, reload, read-only YAML settings, and deletion. Browser traffic contained no MCP REST discovery/execution requests, and the provider endpoint recorded zero outbound connections. Before/after screenshots and the live harness are saved under the local registration UI review evidence directory + +The UI reuses the existing registration endpoints and shared MCP entity. This batch adds no backend routes, permission rules, database changes, or catalog changes. Public docs remain separate + +The UI is approved, and the remaining v1 work is final release review. Live Anthropic verification passed with the updated QA credential; see the follow-up below. The earlier OpenAI native MCP JSON and streaming checks passed. Chat Completions, response/SSE inspection, usage/billing changes, and registry consolidation remain outside this batch + +## Full-stack verification + +The 2026-11-02 admin sequence passed 22 live checks using the public OpenAI documentation MCP and `gpt-6-luna`. Both execution modes were registered through the UI for the same HTTPS endpoint. Scoped MCP protocol and REST calls returned real documentation. Gateway Responses execution sent function tools and returned tool output in a second provider request. Provider-native JSON and streaming calls executed successfully with one provider request each after registration and permission assignment + +Before provider registration, native URL requests rejected even for the admin. Missing server grants, narrower tool permissions, unregistered tools, changed URLs, unbounded scopes, and enforced empty lists rejected with zero provider requests. Provider registrations remained excluded from gateway REST execution, and gateway protocol calls still worked after both modes were registered + +Messages denial checks passed without provider dispatch. Direct Anthropic happy-path execution remains unverified because the designated API key returns 401. The authorized QA vault service account is deleted. The user's AWS profile is valid, but a standard Bedrock call is not equivalent proof of Anthropic MCP connector execution. The live harness, wire evidence, screenshots, and report are in the local `mcp-gap-matrix/full-stack-review` directory. No application changes were required by this run + +## Live risk and adversarial review + +The 2026-11-02 review covered the cumulative stack plus uncommitted UI against merge base `a392a39627`. Five independent lenses completed. One unintended issue was confirmed: the lazy OpenAPI snapshot and dashboard API types omitted the new mode/nullable transport contract. Both were regenerated through the documented commands; repeat generation was deterministic, TypeScript passed, and an independent scoped re-review found no unrelated schema changes + +The live A/B rig used two proxy processes with two workers each per side, real PostgreSQL with the catalog revision trigger, shared Redis per side, a forwarding recorder, real OpenAI, and the public documentation MCP. Gateway calls, agentic continuation, ordinary JSON/streaming inference, Messages adaptation, fallback, and key lifecycle retained their behavior. Provider-native allowed JSON/streaming calls executed; missing grants, unknown endpoints, registration/tool revocation and deletion rejected with zero provider requests, including after a shared response-cache hit + +Supplemental head and merged-current-main checks verified late callback URL mutation is blocked and key-tool revocation overrides the cache across both proxy processes. Identity-free SDK native calls still execute. Native success, cache hits, and denied requests produced the expected nonzero/zero spend records. Current main `a2d27881e8` merged cleanly in an isolated tree; the original branch was not merged or committed + +Two compatibility changes remain explicit: native URL attachments require provider registration and bounded grants, and MCP updates must omit `execution_mode`, including a same-value field copied from GET. The latter implements the existing v1 decision; no update-contract relaxation was made. A malformed mixed-config reload concern was classified as optional hardening after the base showed the same non-atomic update behavior + +Direct Anthropic native execution remains unverified. AWS CLI with `aws-dev` successfully ran a Claude control, but the tested Bedrock native request rejected `mcp_toolset`; this does not replace direct Anthropic connector proof. More unusual adapter/continuation/retry and long-lived config-reload paths remain individually listed in the local risk report. No blanket release-safety or full-CI claim is made. Evidence: `mcp-gap-matrix/risk-review-20261102/REPORT.md` + +## Live Anthropic follow-up, passed + +The updated designated QA key enabled a real Claude Haiku 5.5 base-versus-head run. Both legacy connector and modern MCP toolset formats executed the public documentation MCP successfully in JSON and SSE. The rig used real PostgreSQL, the catalog revision trigger, shared Redis, and two proxy processes with two workers each per side + +All 47 HTTP checks passed. Registered finite tools with key/team grants executed; unregistered URLs, missing grants, forbidden tools, changed URLs and unbounded scopes rejected before provider dispatch. Registration and key-tool revocation each blocked cached responses across both process ports. The branch recorded 17 native request denials with zero provider calls. Gateway automatic MCP execution also passed on base and head with `require_approval=never` + +All eight native JSON/SSE executions had identical final token totals between the recorded upstream response, client response and spend log. Successful calls recorded nonzero cost; the two cache hits and 17 denied calls recorded zero cost. The corresponding outbound request bodies matched base/head exactly. This is ordinary token-accounting proof, not comprehensive provider-specific tool-fee attribution + +The earlier sections describing a blocked Anthropic key record the state before this follow-up; that happy-path/cache-revocation gap is now closed. These calls used direct Anthropic, not Bedrock. Advanced retry/adapter paths remain separately bounded, and this was not another full gauntlet-mini run. No product-code fix was needed. Evidence is in `mcp-gap-matrix/risk-review-20261102/anthropic-followup/` + ## Request authorization review This batch adds first-class request authorization. It changes no registration schema, permission-combination policy, database lookup owner, or management inventory design. The admission logic lives in `litellm/proxy/mcp_server/auth/provider_mcp.py` and calls the existing server and tool permission resolvers diff --git a/litellm/proxy/_lazy_openapi_snapshot.json b/litellm/proxy/_lazy_openapi_snapshot.json index 41ab7fcaf88..e63a9825e54 100644 --- a/litellm/proxy/_lazy_openapi_snapshot.json +++ b/litellm/proxy/_lazy_openapi_snapshot.json @@ -36659,6 +36659,14 @@ "title": "MCPEnvVarScope", "type": "string" }, + "MCPExecutionMode": { + "enum": [ + "litellm", + "provider" + ], + "title": "MCPExecutionMode", + "type": "string" + }, "NewMCPServerRequest": { "properties": { "alias": { @@ -36849,6 +36857,10 @@ ], "title": "Env Vars" }, + "execution_mode": { + "$ref": "#/components/schemas/MCPExecutionMode", + "default": "litellm" + }, "extra_headers": { "anyOf": [ { @@ -37135,14 +37147,21 @@ "title": "Tool Name To Display Name" }, "transport": { - "default": "sse", - "enum": [ - "sse", - "http", - "stdio" + "anyOf": [ + { + "enum": [ + "sse", + "http", + "stdio" + ], + "type": "string" + }, + { + "type": "null" + } ], - "title": "Transport", - "type": "string" + "default": "sse", + "title": "Transport" }, "url": { "anyOf": [ @@ -38677,6 +38696,10 @@ ], "title": "Env Vars" }, + "execution_mode": { + "$ref": "#/components/schemas/MCPExecutionMode", + "default": "litellm" + }, "extra_headers": { "items": { "type": "string" @@ -39059,13 +39082,20 @@ "title": "Tool Name To Display Name" }, "transport": { - "enum": [ - "sse", - "http", - "stdio" + "anyOf": [ + { + "enum": [ + "sse", + "http", + "stdio" + ], + "type": "string" + }, + { + "type": "null" + } ], - "title": "Transport", - "type": "string" + "title": "Transport" }, "updated_at": { "anyOf": [ @@ -39487,6 +39517,14 @@ "title": "MCPEnvVarScope", "type": "string" }, + "MCPExecutionMode": { + "enum": [ + "litellm", + "provider" + ], + "title": "MCPExecutionMode", + "type": "string" + }, "NewMCPServerRequest": { "properties": { "alias": { @@ -39677,6 +39715,10 @@ ], "title": "Env Vars" }, + "execution_mode": { + "$ref": "#/components/schemas/MCPExecutionMode", + "default": "litellm" + }, "extra_headers": { "anyOf": [ { @@ -39963,14 +40005,21 @@ "title": "Tool Name To Display Name" }, "transport": { - "default": "sse", - "enum": [ - "sse", - "http", - "stdio" + "anyOf": [ + { + "enum": [ + "sse", + "http", + "stdio" + ], + "type": "string" + }, + { + "type": "null" + } ], - "title": "Transport", - "type": "string" + "default": "sse", + "title": "Transport" }, "url": { "anyOf": [ @@ -41876,6 +41925,10 @@ ], "title": "Env Vars" }, + "execution_mode": { + "$ref": "#/components/schemas/MCPExecutionMode", + "default": "litellm" + }, "extra_headers": { "items": { "type": "string" @@ -42258,13 +42311,20 @@ "title": "Tool Name To Display Name" }, "transport": { - "enum": [ - "sse", - "http", - "stdio" + "anyOf": [ + { + "enum": [ + "sse", + "http", + "stdio" + ], + "type": "string" + }, + { + "type": "null" + } ], - "title": "Transport", - "type": "string" + "title": "Transport" }, "updated_at": { "anyOf": [ @@ -42903,6 +42963,14 @@ "title": "MCPEnvVarScope", "type": "string" }, + "MCPExecutionMode": { + "enum": [ + "litellm", + "provider" + ], + "title": "MCPExecutionMode", + "type": "string" + }, "MCPGatewaySession": { "description": "One live stateful Streamable HTTP session held by this proxy worker.", "properties": { @@ -43720,6 +43788,10 @@ ], "title": "Env Vars" }, + "execution_mode": { + "$ref": "#/components/schemas/MCPExecutionMode", + "default": "litellm" + }, "extra_headers": { "anyOf": [ { @@ -44006,14 +44078,21 @@ "title": "Tool Name To Display Name" }, "transport": { - "default": "sse", - "enum": [ - "sse", - "http", - "stdio" + "anyOf": [ + { + "enum": [ + "sse", + "http", + "stdio" + ], + "type": "string" + }, + { + "type": "null" + } ], - "title": "Transport", - "type": "string" + "default": "sse", + "title": "Transport" }, "url": { "anyOf": [ @@ -44529,14 +44608,21 @@ "title": "Tool Name To Display Name" }, "transport": { - "default": "sse", - "enum": [ - "sse", - "http", - "stdio" + "anyOf": [ + { + "enum": [ + "sse", + "http", + "stdio" + ], + "type": "string" + }, + { + "type": "null" + } ], - "title": "Transport", - "type": "string" + "default": "sse", + "title": "Transport" }, "url": { "anyOf": [ @@ -46994,6 +47080,14 @@ "title": "MCPEnvVarScope", "type": "string" }, + "MCPExecutionMode": { + "enum": [ + "litellm", + "provider" + ], + "title": "MCPExecutionMode", + "type": "string" + }, "NewMCPServerRequest": { "properties": { "alias": { @@ -47184,6 +47278,10 @@ ], "title": "Env Vars" }, + "execution_mode": { + "$ref": "#/components/schemas/MCPExecutionMode", + "default": "litellm" + }, "extra_headers": { "anyOf": [ { @@ -47470,14 +47568,21 @@ "title": "Tool Name To Display Name" }, "transport": { - "default": "sse", - "enum": [ - "sse", - "http", - "stdio" + "anyOf": [ + { + "enum": [ + "sse", + "http", + "stdio" + ], + "type": "string" + }, + { + "type": "null" + } ], - "title": "Transport", - "type": "string" + "default": "sse", + "title": "Transport" }, "url": { "anyOf": [ diff --git a/ui/litellm-dashboard/src/app/(dashboard)/mcp-servers/_components/CreateMCPServer.tsx b/ui/litellm-dashboard/src/app/(dashboard)/mcp-servers/_components/CreateMCPServer.tsx index 07618abb9a4..137f7e0259d 100644 --- a/ui/litellm-dashboard/src/app/(dashboard)/mcp-servers/_components/CreateMCPServer.tsx +++ b/ui/litellm-dashboard/src/app/(dashboard)/mcp-servers/_components/CreateMCPServer.tsx @@ -52,6 +52,8 @@ import { StdioDisabledBanner, TransportSelectItems } from "./StdioAvailability"; import MCPPermissionManagement from "./MCPPermissionManagement"; import OpenAPIFormSection, { OpenAPIKeyTool } from "./OpenAPIFormSection"; import MCPLogoSelector from "./MCPLogoSelector"; +import ProviderMCPRegistrationForm from "./ProviderMCPRegistrationForm"; +import { Label } from "@/components/ui/label"; import EnvVarsSection from "./EnvVarsSection"; import { isAdminRole } from "@/utils/roles"; import { validateMCPServerUrl, validateMCPServerName } from "./utils"; @@ -84,6 +86,7 @@ interface CreateMCPServerProps { prefillData?: DiscoverableMCPServer | null; onBackToDiscovery?: () => void; stdioEnabled?: boolean; + executionModeSelector?: React.ReactNode; } const payloadErrorMessage = (result: Exclude): string => { @@ -104,7 +107,7 @@ const CREATE_DEFAULTS: MountedFormValues = { oauth_passthrough: false, }; -const CreateMCPServer: React.FC = ({ +const GatewayCreateMCPServer: React.FC = ({ userID, userRole, accessToken, @@ -116,6 +119,7 @@ const CreateMCPServer: React.FC = ({ prefillData, onBackToDiscovery, stdioEnabled = true, + executionModeSelector, }) => { const form = useForm({ mode: "onChange", defaultValues: CREATE_DEFAULTS }); const registry = useMountRegistry(); @@ -647,6 +651,7 @@ const CreateMCPServer: React.FC = ({ + {executionModeSelector}
@@ -1016,4 +1021,63 @@ const CreateMCPServer: React.FC = ({ ); }; +const CreateMCPServer: React.FC = (props) => { + const [executionMode, setExecutionMode] = useState<"litellm" | "provider">("litellm"); + const selectorId = React.useId(); + const { setModalVisible } = props; + const setVisible = React.useCallback( + (visible: boolean) => { + if (!visible) setExecutionMode("litellm"); + setModalVisible(visible); + }, + [setModalVisible], + ); + const executionModeSelector = isAdminRole(props.userRole) ? ( +
+ + +
+ ) : null; + if (executionMode !== "provider" || !isAdminRole(props.userRole)) { + return ( + + ); + } + return ( + + + + Add New MCP Server + + {executionModeSelector} + setVisible(false)} + onSuccess={(server) => { + setVisible(false); + props.onCreateSuccess(server); + }} + /> + + + ); +}; + export default CreateMCPServer; diff --git a/ui/litellm-dashboard/src/app/(dashboard)/mcp-servers/_components/MCPServerCard.tsx b/ui/litellm-dashboard/src/app/(dashboard)/mcp-servers/_components/MCPServerCard.tsx index a9bb2334372..319bd3747de 100644 --- a/ui/litellm-dashboard/src/app/(dashboard)/mcp-servers/_components/MCPServerCard.tsx +++ b/ui/litellm-dashboard/src/app/(dashboard)/mcp-servers/_components/MCPServerCard.tsx @@ -56,6 +56,8 @@ const MCPServerCard: FC = ({ onDelete, stdioEnabled = true, }) => { + const isProvider = server.execution_mode === "provider"; + const recheckHealth = isProvider ? undefined : onRecheckHealth; const alias = server.alias || server.server_name || ""; const name = server.server_name || alias || server.server_id; // Logo is sourced exclusively from the admin-set `mcp_info.logo_url`. @@ -113,7 +115,7 @@ const MCPServerCard: FC = ({ } }; - const hasMenu = !!onRecheckHealth || !!onDelete; + const hasMenu = !!recheckHealth || !!onDelete; // Card uses role="button" + nested