From ed356fdfc0222b52d334b9fe295c6d1addf9c0fb Mon Sep 17 00:00:00 2001 From: Ishaan Jaff Date: Sat, 13 Dec 2025 14:44:40 -0800 Subject: [PATCH] [Docs] Cursor Integration (#17939) * docs cursor * remove bloat * stash changes * docs fix * simpler docs * docs * docs cursor * add cursor/chat/completions --- docs/my-website/docs/proxy/cursor.md | 108 -------- .../docs/tutorials/cursor_integration.md | 241 ++++-------------- litellm/proxy/_types.py | 1 + 3 files changed, 51 insertions(+), 299 deletions(-) delete mode 100644 docs/my-website/docs/proxy/cursor.md diff --git a/docs/my-website/docs/proxy/cursor.md b/docs/my-website/docs/proxy/cursor.md deleted file mode 100644 index d01c1e62036..00000000000 --- a/docs/my-website/docs/proxy/cursor.md +++ /dev/null @@ -1,108 +0,0 @@ ---- -id: cursor -title: /cursor/chat/completions - Cursor Endpoint -description: Accept Responses API input from Cursor and return OpenAI Chat Completions output ---- - -LiteLLM provides a Cursor-specific endpoint to make Cursor IDE work seamlessly with the LiteLLM Proxy when using BYOK + custom `base_url`. - -- Accepts Requests in OpenAI Responses API input format (Cursor sends this) -- Returns Responses in OpenAI Chat Completions format (Cursor expects this) -- Supports streaming and non‑streaming - -## Endpoint - -- Path: `/cursor/chat/completions` -- Auth: Standard LiteLLM Proxy auth (`Authorization: Bearer `) -- Behavior: Internally routes to LiteLLM `/responses` flow and transforms output to Chat Completions - -## Why this exists - -When setting up Cursor with BYOK against a custom `base_url`, Cursor sends requests to the Chat Completions endpoint but in the OpenAI Responses API input shape. Without translation, Cursor won’t display streamed output. This endpoint bridges the formats: - -- Input: Responses API (`input`, tool calls, etc.) -- Output: Chat Completions (`choices`, `delta`, `finish_reason`, etc.) - -## Usage - -### Non-streaming - -```bash -curl -X POST https://litellm-internal/cursor/chat/completions \ - -H "Content-Type: application/json" \ - -H "Authorization: Bearer sk-1234" \ - -d '{ - "model": "gpt-4o", - "input": [{"role": "user", "content": "Hello"}] - }' -``` - -Example response (shape): - -```json -{ - "id": "chatcmpl-123", - "object": "chat.completion", - "created": 1733333333, - "model": "gpt-4o", - "choices": [ - { - "index": 0, - "message": { - "role": "assistant", - "content": "Hello! How can I help you?" - }, - "finish_reason": "stop" - } - ], - "usage": { - "prompt_tokens": 10, - "completion_tokens": 8, - "total_tokens": 18 - } -} -``` - -### Streaming - -```bash -curl -N -X POST https://litellm-internal/cursor/chat/completions \ - -H "Content-Type: application/json" \ - -H "Authorization: Bearer sk-1234" \ - -d '{ - "model": "gpt-4o", - "input": [{"role": "user", "content": "Hello"}], - "stream": true - }' -``` - -- Server-Sent Events (SSE) -- Emits `chat.completion.chunk` deltas (`choices[].delta`) and ends with `data: [DONE]` - -## Configuration - -### Base URL Setup - -**Important**: When configuring Cursor IDE to use this endpoint, you must include `/cursor` in the base URL. - -Cursor automatically appends `/chat/completions` to the base URL you provide. To ensure requests go to `/cursor/chat/completions`, configure your base URL in Cursor as: - -``` -Base URL: https://litellm-internal/cursor -``` - -This way, when Cursor appends `/chat/completions`, the full path becomes `/cursor/chat/completions`, which is the correct endpoint. - -**Example**: If your LiteLLM Proxy is running at `https://litellm-internal`, set the base URL in Cursor to `https://litellm-internal/cursor` (not just `https://litellm-internal`). - -### General Setup - -No special configuration is required beyond your normal LiteLLM Proxy setup. Ensure that: - -- Your `config.yaml` includes the models you want to call via this endpoint -- Your Cursor project uses your LiteLLM Proxy `base_url` (with `/cursor` included) and a valid API key - -## Notes -- This endpoint is intended specifically for Cursor’s request/response expectations. Other clients should continue to use `/v1/chat/completions` or `/v1/responses` as appropriate. - - diff --git a/docs/my-website/docs/tutorials/cursor_integration.md b/docs/my-website/docs/tutorials/cursor_integration.md index f0d87b050cf..3f462e1ee5d 100644 --- a/docs/my-website/docs/tutorials/cursor_integration.md +++ b/docs/my-website/docs/tutorials/cursor_integration.md @@ -1,226 +1,85 @@ ---- -sidebar_label: "Cursor IDE" +# Cursor Integration + +Route Cursor IDE requests through LiteLLM for unified logging, budget controls, and access to any model. + +:::info +**Supported modes:** Ask, Plan. Agent mode doesn't support custom API keys yet. +::: + +## Quick Reference + +| Setting | Value | +|---------|-------| +| Base URL | `/cursor` | +| API Key | Your LiteLLM Virtual Key | +| Model | Public Model Name from LiteLLM | + --- -import Tabs from '@theme/Tabs'; -import TabItem from '@theme/TabItem'; +## Setup -# Cursor IDE Integration with LiteLLM +### 1. Configure Base URL -This tutorial shows you how to integrate Cursor IDE with LiteLLM Proxy, allowing you to use any LiteLLM-supported model through Cursor's interface with BYOK (Bring Your Own Key) and custom base URL. +Open **Cursor → Settings → Cursor Settings → Models**. -## Benefits of using Cursor with LiteLLM +![](https://ajeuwbhvhr.cloudimg.io/https://colony-recorder.s3.amazonaws.com/files/2025-12-13/f725f154-588d-448d-a1d7-3c8bffaf3cf3/ascreenshot.jpeg?tl_px=0,0&br_px=1376,769&force_format=jpeg&q=100&width=1120.0&wat=1&wat_opacity=0.7&wat_gravity=northwest&wat_url=https://colony-recorder.s3.us-west-1.amazonaws.com/images/watermarks/FB923C_standard.png&wat_pad=263,73) -When you use Cursor IDE with LiteLLM you get the following benefits: - -**Developer Benefits:** -- Universal Model Access: Use any LiteLLM supported model (Anthropic, OpenAI, Vertex AI, Bedrock, etc.) through the Cursor IDE interface. -- Higher Rate Limits & Reliability: Load balance across multiple models and providers to avoid hitting individual provider limits, with fallbacks to ensure you get responses even if one provider fails. -- Streaming Support: Full streaming support with proper response transformation for Cursor's expected format. - -**Proxy Admin Benefits:** -- Centralized Management: Control access to all models through a single LiteLLM proxy instance without giving your developers API Keys to each provider. -- Budget Controls: Set spending limits and track costs across all Cursor usage. -- Request Logging: Track all requests made through Cursor for debugging and monitoring. - -## Prerequisites - -Before you begin, ensure you have: -- Cursor IDE installed -- A running LiteLLM Proxy instance with **HTTPS enabled** (HTTP is not supported) -- A valid LiteLLM Proxy API key -- An HTTPS domain for your LiteLLM Proxy (required by Cursor) - -## Quick Start Guide - -### Step 1: Install LiteLLM - -Install LiteLLM with proxy support: - -```bash -pip install litellm[proxy] -``` - -### Step 2: Configure LiteLLM Proxy - -Create a `config.yaml` file with your model configurations: - -```yaml showLineNumbers title="config.yaml" -model_list: - - model_name: gpt-4o - litellm_params: - model: gpt-4o - api_key: os.environ/OPENAI_API_KEY - - - model_name: claude-3-5-sonnet - litellm_params: - model: anthropic/claude-3-5-sonnet-20241022 - api_key: os.environ/ANTHROPIC_API_KEY - -general_settings: - master_key: sk-1234567890 # Change this to a secure key -``` - -### Step 3: Start LiteLLM Proxy - -Start the proxy server with HTTPS enabled: - -```bash -litellm --config config.yaml --port 4000 -``` - -:::warning HTTPS Required - -**Important**: Cursor IDE requires HTTPS connections. HTTP (`http://`) will not work. You must: -- Deploy your LiteLLM Proxy with HTTPS enabled -- Use a valid SSL certificate -- Access the proxy via an HTTPS domain (e.g., `https://your-proxy-domain.com`) - -For local development, you'll need to set up HTTPS (e.g., using a reverse proxy like nginx with SSL, or deploying to a cloud service with HTTPS). - -::: - -### Step 4: Configure Cursor IDE - -Configure Cursor IDE to use your LiteLLM proxy with the `/cursor/chat/completions` endpoint: - -1. Open Cursor IDE -2. Go to **Settings** → **Features** → **AI** -3. Enable **"Use Custom API"** or **"Bring Your Own Key"** -4. Set the following: - - **Base URL**: `https://your-proxy-domain.com/cursor` (⚠️ **Important**: Must use HTTPS and include `/cursor`) - - **API Key**: Your LiteLLM Proxy API key (e.g., `sk-1234567890`) - -:::warning HTTPS Required - -Cursor IDE **requires HTTPS** connections. HTTP (`http://`) will not work. You must: -- Use an HTTPS URL for your base URL (e.g., `https://your-proxy-domain.com/cursor`) -- Ensure your LiteLLM Proxy is accessible via HTTPS -- Have a valid SSL certificate configured - -::: - -**Example Configuration:** +Enable **Override OpenAI Base URL** and enter your proxy URL with `/cursor`: ``` -Base URL: https://your-proxy-domain.com/cursor -API Key: sk-1234567890 +https://your-litellm-proxy.com/cursor ``` -Replace `your-proxy-domain.com` with your actual HTTPS domain where LiteLLM Proxy is running. +![](https://colony-recorder.s3.amazonaws.com/files/2025-12-13/6580de2b-3a59-45b2-b7b6-3ab105d87e74/ascreenshot.jpeg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIA2JDELI43356LVVTC%2F20251213%2Fus-west-1%2Fs3%2Faws4_request&X-Amz-Date=20251213T224156Z&X-Amz-Expires=900&X-Amz-SignedHeaders=host&X-Amz-Signature=5a1af4ff63d38d51e06d398ed50f10161d690e3e57e9d67c1d23ce5b7ffdefd5) -:::info Why `/cursor` in the base URL? +### 2. Create Virtual Key -Cursor automatically appends `/chat/completions` to the base URL you provide. By setting the base URL to `https://your-proxy-domain.com/cursor`, Cursor will send requests to `/cursor/chat/completions`, which is the special endpoint that handles Cursor's Responses API input format and transforms it to Chat Completions output format. +In LiteLLM Dashboard, go to **Virtual Keys → + Create New Key**. -If you set the base URL to just `https://your-proxy-domain.com`, Cursor would send requests to `/chat/completions`, which won't work correctly with Cursor's request format. +![](https://ajeuwbhvhr.cloudimg.io/https://colony-recorder.s3.amazonaws.com/files/2025-12-13/1d8156bc-1b12-433f-936d-77f876142e3f/ascreenshot.jpeg?tl_px=0,0&br_px=1376,769&force_format=jpeg&q=100&width=1120.0&wat=1&wat_opacity=0.7&wat_gravity=northwest&wat_url=https://colony-recorder.s3.us-west-1.amazonaws.com/images/watermarks/FB923C_standard.png&wat_pad=240,182) +Name your key and select which models it can access. -::: +![](https://ajeuwbhvhr.cloudimg.io/https://colony-recorder.s3.amazonaws.com/files/2025-12-13/c45843db-b623-442b-b42b-3145ef3ba986/ascreenshot.jpeg?tl_px=0,151&br_px=1376,920&force_format=jpeg&q=100&width=1120.0&wat=1&wat_opacity=0.7&wat_gravity=northwest&wat_url=https://colony-recorder.s3.us-west-1.amazonaws.com/images/watermarks/FB923C_standard.png&wat_pad=453,277) -### Step 5: Test the Integration +Click **Create Key** then copy it immediately—you won't see it again. -1. Restart Cursor IDE to apply the settings -2. Open a code file and try using Cursor's AI features (completions, chat, etc.) -3. Your requests will now be routed through LiteLLM Proxy +![](https://ajeuwbhvhr.cloudimg.io/https://colony-recorder.s3.amazonaws.com/files/2025-12-13/4022504d-fdba-4e17-b16e-bf8e935cbcad/ascreenshot.jpeg?tl_px=0,101&br_px=1376,870&force_format=jpeg&q=100&width=1120.0&wat=1&wat_opacity=0.7&wat_gravity=northwest&wat_url=https://colony-recorder.s3.us-west-1.amazonaws.com/images/watermarks/FB923C_standard.png&wat_pad=512,277) -You can verify it's working by: -- Checking the LiteLLM Proxy logs for incoming requests -- Using Cursor's chat feature and seeing responses stream correctly -- Checking your LiteLLM dashboard for request logs and cost tracking +Paste it into the **OpenAI API Key** field in Cursor. -## How It Works +![](https://ajeuwbhvhr.cloudimg.io/https://colony-recorder.s3.amazonaws.com/files/2025-12-13/6b50fc92-9219-4868-aac2-a29d0c063e57/ascreenshot.jpeg?tl_px=251,235&br_px=1627,1004&force_format=jpeg&q=100&width=1120.0&wat=1&wat_opacity=0.7&wat_gravity=northwest&wat_url=https://colony-recorder.s3.us-west-1.amazonaws.com/images/watermarks/FB923C_standard.png&wat_pad=524,276) -The `/cursor/chat/completions` endpoint is specifically designed to handle Cursor's unique request format: +### 3. Add Custom Model -1. **Input**: Cursor sends requests in OpenAI Responses API format (with `input` field) -2. **Processing**: LiteLLM processes the request through its internal `/responses` flow -3. **Output**: The response is transformed to OpenAI Chat Completions format (with `choices` field) that Cursor expects +Click **+ Add Custom Model** in Cursor Settings. -This transformation happens automatically for both streaming and non-streaming responses. +![](https://ajeuwbhvhr.cloudimg.io/https://colony-recorder.s3.amazonaws.com/files/2025-12-13/4e46538e-a876-44c4-a133-bdae664510f3/ascreenshot.jpeg?tl_px=192,8&br_px=1569,777&force_format=jpeg&q=100&width=1120.0&wat=1&wat_opacity=0.7&wat_gravity=northwest&wat_url=https://colony-recorder.s3.us-west-1.amazonaws.com/images/watermarks/FB923C_standard.png&wat_pad=524,276) -## Advanced Configuration +Get the **Public Model Name** from LiteLLM Dashboard → Models + Endpoints. -### Using Different Models +![](https://ajeuwbhvhr.cloudimg.io/https://colony-recorder.s3.amazonaws.com/files/2025-12-13/2ee87f64-104a-4b37-8041-c92130a44896/ascreenshot.jpeg?tl_px=0,11&br_px=1376,780&force_format=jpeg&q=100&width=1120.0&wat=1&wat_opacity=0.7&wat_gravity=northwest&wat_url=https://colony-recorder.s3.us-west-1.amazonaws.com/images/watermarks/FB923C_standard.png&wat_pad=331,277) -You can configure Cursor to use different models by updating your `config.yaml`: +Paste the name in Cursor and enable the toggle. -```yaml showLineNumbers title="config.yaml" -model_list: - - model_name: gpt-4o - litellm_params: - model: gpt-4o - api_key: os.environ/OPENAI_API_KEY - - - model_name: claude-3-5-sonnet - litellm_params: - model: anthropic/claude-3-5-sonnet-20241022 - api_key: os.environ/ANTHROPIC_API_KEY - - - model_name: gemini-pro - litellm_params: - model: gemini/gemini-1.5-pro - api_key: os.environ/GEMINI_API_KEY -``` +![](https://ajeuwbhvhr.cloudimg.io/https://colony-recorder.s3.amazonaws.com/files/2025-12-13/5ab35f93-d417-423f-a359-9811ce18e2c3/ascreenshot.jpeg?tl_px=352,26&br_px=1728,795&force_format=jpeg&q=100&width=1120.0&wat=1&wat_opacity=0.7&wat_gravity=northwest&wat_url=https://colony-recorder.s3.us-west-1.amazonaws.com/images/watermarks/FB923C_standard.png&wat_pad=786,277) -Then in Cursor, you can specify which model to use in your requests. +### 4. Test -### Rate Limiting and Budgets +Open **Ask** mode with `Cmd+L` / `Ctrl+L` and select your model. -Set up rate limits and budgets in your `config.yaml`: +![](https://colony-recorder.s3.amazonaws.com/files/2025-12-13/d87ee25b-3c6d-4231-ba00-4d841d0612bc/ascreenshot.jpeg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIA2JDELI43356LVVTC%2F20251213%2Fus-west-1%2Fs3%2Faws4_request&X-Amz-Date=20251213T223855Z&X-Amz-Expires=900&X-Amz-SignedHeaders=host&X-Amz-Signature=75316b8cd2d451f476232bd0ca459c4b6877e788637bf228bbd7d8b319fd1427) -```yaml showLineNumbers title="config.yaml" -general_settings: - master_key: sk-1234567890 +Send a message. All requests now route through LiteLLM. -litellm_settings: - # Set max budget per user - max_budget: 100.0 - - # Set rate limits - rate_limit: 100 # requests per minute -``` +![](https://ajeuwbhvhr.cloudimg.io/https://colony-recorder.s3.amazonaws.com/files/2025-12-13/05a5853a-58ed-44bf-a5c2-c14f9003eace/ascreenshot.jpeg?tl_px=0,151&br_px=1728,1117&force_format=jpeg&q=100&width=1120.0) -### Request Logging - -All requests from Cursor will be logged by LiteLLM Proxy. You can: -- View logs in the LiteLLM Admin UI -- Export logs to your preferred logging service -- Track costs per user/team +--- ## Troubleshooting -### Cursor shows no output - -- **Check base URL**: Ensure it uses HTTPS and includes `/cursor` (e.g., `https://your-proxy-domain.com/cursor`, not `http://` or without `/cursor`) -- **Verify HTTPS**: Cursor requires HTTPS - HTTP connections will not work -- **Check API key**: Verify your LiteLLM Proxy API key is correct -- **Check proxy logs**: Look for errors in the LiteLLM Proxy logs - -### Requests failing - -- **Verify HTTPS is enabled**: Cursor requires HTTPS connections. Ensure your LiteLLM Proxy is accessible via HTTPS with a valid SSL certificate -- **Verify proxy is running**: Check that LiteLLM Proxy is accessible at your HTTPS base URL -- **Check SSL certificate**: Ensure your SSL certificate is valid and not expired -- **Check model configuration**: Ensure the model you're trying to use is configured in `config.yaml` -- **Check API keys**: Verify provider API keys are set correctly in environment variables - -### HTTP not working - -If you're trying to use HTTP (`http://`) and it's not working: -- **This is expected**: Cursor IDE requires HTTPS connections -- **Solution**: Deploy your LiteLLM Proxy with HTTPS enabled (use a reverse proxy like nginx, or deploy to a cloud service that provides HTTPS) - -### Streaming not working - -The `/cursor/chat/completions` endpoint automatically handles streaming. If streaming isn't working: -- Check that your model supports streaming -- Verify the proxy logs for any transformation errors -- Ensure Cursor IDE is up to date - -## Related Documentation - -- [Cursor Endpoint Documentation](/docs/proxy/cursor) - Detailed endpoint documentation -- [LiteLLM Proxy Setup](/docs/proxy/quick_start) - General proxy setup guide -- [Model Configuration](/docs/proxy/configs) - How to configure models - +| Issue | Solution | +|-------|----------| +| Model not responding | Check base URL ends with `/cursor` and key has model access | +| Auth errors | Regenerate key; ensure it starts with `sk-` | +| Agent mode not working | Expected—only Ask and Plan modes support custom keys | diff --git a/litellm/proxy/_types.py b/litellm/proxy/_types.py index d507b6b5ac3..dcbca1a8a0d 100644 --- a/litellm/proxy/_types.py +++ b/litellm/proxy/_types.py @@ -247,6 +247,7 @@ class LiteLLMRoutes(enum.Enum): "/openai/deployments/{model}/chat/completions", "/chat/completions", "/v1/chat/completions", + "/cursor/chat/completions", # completions "/engines/{model}/completions", "/openai/deployments/{model}/completions",