diff --git a/docs/my-website/docs/mcp.md b/docs/my-website/docs/mcp.md index 61d724c99de..0e8d9024e6d 100644 --- a/docs/my-website/docs/mcp.md +++ b/docs/my-website/docs/mcp.md @@ -19,8 +19,6 @@ LiteLLM Proxy provides an MCP Gateway that allows you to use a fixed endpoint fo |---------|-------------| | MCP Operations | • List Tools
• Call Tools | | Supported MCP Transports | • Streamable HTTP
• SSE
• Standard Input/Output (stdio) | -| MCP Tool Cost Tracking | ✅ Supported | -| Grouping MCPs (Access Groups) | ✅ Supported | | LiteLLM Permission Management | ✨ Enterprise Only
• By Key
• By Team
• By Organization | ## Adding your MCP @@ -42,7 +40,7 @@ LiteLLM supports the following MCP transports: style={{width: '80%', display: 'block', margin: '0'}} /> -#### Adding a stdio MCP Server +### Adding a stdio MCP Server For stdio MCP servers, select "Standard Input/Output (stdio)" as the transport type and provide the stdio configuration in JSON format: @@ -113,7 +111,6 @@ mcp_servers: -### Quick Start #### Connect via OpenAI Responses API Use the OpenAI Responses API to connect to your LiteLLM MCP server: @@ -128,7 +125,7 @@ curl --location 'https://api.openai.com/v1/responses' \ { "type": "mcp", "server_label": "litellm", - "server_url": "litellm_proxy", + "server_url": "/mcp", "require_approval": "never", "headers": { "x-litellm-api-key": "Bearer YOUR_LITELLM_API_KEY" @@ -158,7 +155,7 @@ curl --location '/v1/responses' \ { "type": "mcp", "server_label": "litellm", - "server_url": "litellm_proxy", + "server_url": "/mcp", "require_approval": "never", "headers": { "x-litellm-api-key": "Bearer YOUR_LITELLM_API_KEY" @@ -188,7 +185,7 @@ Use tools directly from Cursor IDE with LiteLLM MCP: { "mcpServers": { "LiteLLM": { - "url": "litellm_proxy", + "url": "/mcp", "headers": { "x-litellm-api-key": "Bearer $LITELLM_API_KEY" } @@ -200,25 +197,123 @@ Use tools directly from Cursor IDE with LiteLLM MCP: -#### How it works when server_url="litellm_proxy" +## Selecting MCP Servers/Groups via URL Namespacing -When server_url="litellm_proxy", LiteLLM bridges non-MCP providers to your MCP tools. +You can now directly access specific MCP servers and groups by specifying them in the MCP URL itself. This allows you to: +- Limit tool access to one or more specific MCP servers or groups using the URL +- Control which tools are available in different environments or use cases -- Tool Discovery: LiteLLM fetches MCP tools and converts them to OpenAI-compatible definitions -- LLM Call: Tools are sent to the LLM with your input; LLM selects which tools to call -- Tool Execution: LiteLLM automatically parses arguments, routes calls to MCP servers, executes tools, and retrieves results -- Response Integration: Tool results are sent back to LLM for final response generation -- Output: Complete response combining LLM reasoning with tool execution results +**This is the preferred method for MCP server/group selection.** -This enables MCP tool usage with any LiteLLM-supported provider, regardless of native MCP support. +The URL pattern is: -#### Auto-execution for require_approval: "never" +``` +/mcp/ +``` -Setting require_approval: "never" triggers automatic tool execution, returning the final response in a single API call without additional user interaction. +- You can specify one or more server/group names, separated by commas after `/mcp/`. +- Server/group names with spaces should be replaced with underscores. +- If you do not use this URL pattern, all available MCP servers will be accessible (unless restricted by other means). +- You can still use the `x-mcp-servers` header as an alternative (see below). + + + + +```bash title="cURL Example with URL Namespacing" showLineNumbers +curl --location 'https://api.openai.com/v1/responses' \ +--header 'Content-Type: application/json' \ +--header "Authorization: Bearer $OPENAI_API_KEY" \ +--data '{ + "model": "gpt-4o", + "tools": [ + { + "type": "mcp", + "server_label": "litellm", + "server_url": "/mcp/Zapier_Gmail", + "require_approval": "never", + "headers": { + "x-litellm-api-key": "Bearer YOUR_LITELLM_API_KEY" + } + } + ], + "input": "Run available tools", + "tool_choice": "required" +}' +``` + +In this example, the request will only have access to tools from the "Zapier_Gmail" MCP servers. + + + + + +```bash title="cURL Example with URL Namespacing" showLineNumbers +curl --location '/v1/responses' \ +--header 'Content-Type: application/json' \ +--header "Authorization: Bearer $LITELLM_API_KEY" \ +--data '{ + "model": "gpt-4o", + "tools": [ + { + "type": "mcp", + "server_label": "litellm", + "server_url": "/mcp/Zapier_Gmail,Group1", + "require_approval": "never", + "headers": { + "x-litellm-api-key": "Bearer YOUR_LITELLM_API_KEY" + } + } + ], + "input": "Run available tools", + "tool_choice": "required" +}' +``` + +This configuration restricts the request to only use tools from the specified MCP servers/groups via the URL. + + + + + +```json title="Cursor MCP Configuration with URL Namespacing" showLineNumbers +{ + "mcpServers": { + "LiteLLM": { + "url": "/mcp/Zapier_Gmail", + "headers": { + "x-litellm-api-key": "Bearer $LITELLM_API_KEY" + } + } + } +} +``` + +This configuration in Cursor IDE settings will limit tool access to only the specified MCP servers/groups via the URL. + + + + +:::info +**Note:** You can add multiple servers or access groups in the URL instead of just one, by making it comma-separated. This allows you to restrict access to several MCP servers/groups at once. +::: + +**Example:** + +```json title="Multiple Servers/Access Groups in URL" showLineNumbers +{ + "mcpServers": { + "LiteLLM": { + "url": "/mcp/Zapier_Gmail,dev_access_group,deepwiki_mcp", + "headers": { + "x-litellm-api-key": "Bearer $LITELLM_API_KEY" + } + } + } +} +``` - -### Specific MCP Servers +## Segregating MCP Server Access Using Headers You can choose to access specific MCP servers and only list their tools using the `x-mcp-servers` header. This header allows you to: - Limit tool access to one or more specific MCP servers @@ -243,7 +338,7 @@ curl --location 'https://api.openai.com/v1/responses' \ { "type": "mcp", "server_label": "litellm", - "server_url": "litellm_proxy", + "server_url": "/mcp", "require_approval": "never", "headers": { "x-litellm-api-key": "Bearer YOUR_LITELLM_API_KEY", @@ -272,7 +367,7 @@ curl --location '/v1/responses' \ { "type": "mcp", "server_label": "litellm", - "server_url": "litellm_proxy", + "server_url": "/mcp", "require_approval": "never", "headers": { "x-litellm-api-key": "Bearer YOUR_LITELLM_API_KEY", @@ -295,7 +390,7 @@ This configuration restricts the request to only use tools from the specified MC { "mcpServers": { "LiteLLM": { - "url": "litellm_proxy", + "url": "/mcp", "headers": { "x-litellm-api-key": "Bearer $LITELLM_API_KEY", "x-mcp-servers": "Zapier_Gmail,Server2" @@ -310,57 +405,7 @@ This configuration in Cursor IDE settings will limit tool access to only the spe -### Grouping MCPs (Access Groups) - -MCP Access Groups allow you to group multiple MCP servers together for easier management. - -#### 1. Create an Access Group - -To create an access group: -- Go to MCP Servers in the LiteLLM UI -- Click "Add a New MCP Server" -- Under "MCP Access Groups", create a new group (e.g., "dev_group") by typing it -- Add the same group name to other servers to group them together - - - -#### 2. Use Access Group in Cursor - -Include the access group name in the `x-mcp-servers` header: - -```json title="Cursor Configuration with Access Groups" showLineNumbers -{ - "mcpServers": { - "LiteLLM": { - "url": "litellm_proxy", - "headers": { - "x-litellm-api-key": "Bearer $LITELLM_API_KEY", - "x-mcp-servers": "dev_group" - } - } - } -} -``` - -This gives you access to all servers in the "dev_group" access group. - -#### Advanced: Connecting Access Groups to API Keys - -When creating API keys, you can assign them to specific access groups for permission management: - -- Go to "Keys" in the LiteLLM UI and click "Create Key" -- Select the desired MCP access groups from the dropdown -- The key will have access to all MCP servers in those groups -- This is reflected in the Test Key page - - - +--- ## Using your MCP with client side credentials @@ -385,7 +430,7 @@ curl --location 'https://api.openai.com/v1/responses' \ { "type": "mcp", "server_label": "litellm", - "server_url": "litellm_proxy", + "server_url": "/mcp", "require_approval": "never", "headers": { "x-litellm-api-key": "Bearer YOUR_LITELLM_API_KEY", @@ -416,7 +461,7 @@ curl --location '/v1/responses' \ { "type": "mcp", "server_label": "litellm", - "server_url": "litellm_proxy", + "server_url": "/mcp", "require_approval": "never", "headers": { "x-litellm-api-key": "Bearer YOUR_LITELLM_API_KEY", @@ -447,7 +492,7 @@ Use tools directly from Cursor IDE with LiteLLM MCP and include your MCP authent { "mcpServers": { "LiteLLM": { - "url": "litellm_proxy", + "url": "/mcp", "headers": { "x-litellm-api-key": "Bearer $LITELLM_API_KEY", "x-mcp-auth": "$MCP_AUTH_TOKEN" @@ -467,7 +512,7 @@ Connect to LiteLLM MCP using HTTP transport with MCP authentication: **Server URL:** ```text showLineNumbers -litellm_proxy +/mcp ``` **Headers:** @@ -494,7 +539,7 @@ from fastmcp import Client from fastmcp.client.transports import StreamableHttpTransport # Create the transport with your LiteLLM MCP server URL and auth headers -server_url = "litellm_proxy" +server_url = "/mcp" transport = StreamableHttpTransport( server_url, headers={ @@ -575,7 +620,7 @@ curl --location '/v1/responses' \ { "type": "mcp", "server_label": "litellm", - "server_url": "litellm_proxy", + "server_url": "/mcp", "require_approval": "never", "headers": { "x-litellm-api-key": "Bearer YOUR_LITELLM_API_KEY", @@ -590,37 +635,16 @@ curl --location '/v1/responses' \ -## MCP Cost Tracking +## ✨ MCP Cost Tracking -LiteLLM provides cost tracking for MCP tool calls, allowing you to monitor and control expenses associated with MCP operations. You can configure costs at two levels: +LiteLLM provides two ways to track costs for MCP tool calls: -- **Default cost per tool**: Set a uniform cost for all tools from a specific MCP server -- **Tool-specific costs**: Define individual costs for specific tools (e.g., `search_tool` costs $10, while `get_weather` costs $5) +| Method | When to Use | What It Does | +|--------|-------------|--------------| +| **Config-based Cost Tracking** | Simple cost tracking with fixed costs per tool/server | Automatically tracks costs based on configuration | +| **Custom Post-MCP Hook** | Dynamic cost tracking with custom logic | Allows custom cost calculations and response modifications | -### Configure cost tracking - -LiteLLM offers two approaches to track MCP tool costs, each designed for different use cases: - -| Method | Best For | Capabilities | -|--------|----------|-------------| -| **UI/Config-based Cost Tracking** | Simple, static cost tracking scenarios | • Set default costs for all server tools
• Configure individual tool costs
• Automatic cost tracking based on configuration | -| **Custom Post-MCP Hook** | Dynamic, complex cost tracking requirements | • Custom cost calculation logic
• Real-time cost adjustments
• Response modification capabilities | - -### Configuration on UI/config.yaml - - - - -On the UI when adding a new MCP server, you can navigate to the "Cost Configuration" tab to configure the cost for the MCP server. - - - - - - +### Config-based Cost Tracking Configure fixed costs for MCP servers directly in your config.yaml: @@ -650,9 +674,6 @@ mcp_servers: default_cost_per_query: 1.50 ``` - - - ### Custom Post-MCP Hook Use this when you need dynamic cost calculation or want to modify the MCP response before it's returned to the user. @@ -741,6 +762,7 @@ When Creating a Key, Team, or Organization, you can select the allowed MCP Serve style={{width: '80%', display: 'block', margin: '0'}} /> + ## LiteLLM Proxy - Walk through MCP Gateway LiteLLM exposes an MCP Gateway for admins to add all their MCP servers to LiteLLM. The key benefits of using LiteLLM Proxy with MCP are: