mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-06 02:48:13 +00:00
[Docs] Cursor Integration (#17939)
* docs cursor * remove bloat * stash changes * docs fix * simpler docs * docs * docs cursor * add cursor/chat/completions
This commit is contained in:
Ishaan Jaff
GitHub
parent
fab1b81b7f
commit
ed356fdfc0
3 changed files with 51 additions and 299 deletions
|
|
@ -1,108 +0,0 @@
|
|||
---
|
||||
id: cursor
|
||||
title: /cursor/chat/completions - Cursor Endpoint
|
||||
description: Accept Responses API input from Cursor and return OpenAI Chat Completions output
|
||||
---
|
||||
|
||||
LiteLLM provides a Cursor-specific endpoint to make Cursor IDE work seamlessly with the LiteLLM Proxy when using BYOK + custom `base_url`.
|
||||
|
||||
- Accepts Requests in OpenAI Responses API input format (Cursor sends this)
|
||||
- Returns Responses in OpenAI Chat Completions format (Cursor expects this)
|
||||
- Supports streaming and non‑streaming
|
||||
|
||||
## Endpoint
|
||||
|
||||
- Path: `/cursor/chat/completions`
|
||||
- Auth: Standard LiteLLM Proxy auth (`Authorization: Bearer <key>`)
|
||||
- Behavior: Internally routes to LiteLLM `/responses` flow and transforms output to Chat Completions
|
||||
|
||||
## Why this exists
|
||||
|
||||
When setting up Cursor with BYOK against a custom `base_url`, Cursor sends requests to the Chat Completions endpoint but in the OpenAI Responses API input shape. Without translation, Cursor won’t display streamed output. This endpoint bridges the formats:
|
||||
|
||||
- Input: Responses API (`input`, tool calls, etc.)
|
||||
- Output: Chat Completions (`choices`, `delta`, `finish_reason`, etc.)
|
||||
|
||||
## Usage
|
||||
|
||||
### Non-streaming
|
||||
|
||||
```bash
|
||||
curl -X POST https://litellm-internal/cursor/chat/completions \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "Authorization: Bearer sk-1234" \
|
||||
-d '{
|
||||
"model": "gpt-4o",
|
||||
"input": [{"role": "user", "content": "Hello"}]
|
||||
}'
|
||||
```
|
||||
|
||||
Example response (shape):
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "chatcmpl-123",
|
||||
"object": "chat.completion",
|
||||
"created": 1733333333,
|
||||
"model": "gpt-4o",
|
||||
"choices": [
|
||||
{
|
||||
"index": 0,
|
||||
"message": {
|
||||
"role": "assistant",
|
||||
"content": "Hello! How can I help you?"
|
||||
},
|
||||
"finish_reason": "stop"
|
||||
}
|
||||
],
|
||||
"usage": {
|
||||
"prompt_tokens": 10,
|
||||
"completion_tokens": 8,
|
||||
"total_tokens": 18
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Streaming
|
||||
|
||||
```bash
|
||||
curl -N -X POST https://litellm-internal/cursor/chat/completions \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "Authorization: Bearer sk-1234" \
|
||||
-d '{
|
||||
"model": "gpt-4o",
|
||||
"input": [{"role": "user", "content": "Hello"}],
|
||||
"stream": true
|
||||
}'
|
||||
```
|
||||
|
||||
- Server-Sent Events (SSE)
|
||||
- Emits `chat.completion.chunk` deltas (`choices[].delta`) and ends with `data: [DONE]`
|
||||
|
||||
## Configuration
|
||||
|
||||
### Base URL Setup
|
||||
|
||||
**Important**: When configuring Cursor IDE to use this endpoint, you must include `/cursor` in the base URL.
|
||||
|
||||
Cursor automatically appends `/chat/completions` to the base URL you provide. To ensure requests go to `/cursor/chat/completions`, configure your base URL in Cursor as:
|
||||
|
||||
```
|
||||
Base URL: https://litellm-internal/cursor
|
||||
```
|
||||
|
||||
This way, when Cursor appends `/chat/completions`, the full path becomes `/cursor/chat/completions`, which is the correct endpoint.
|
||||
|
||||
**Example**: If your LiteLLM Proxy is running at `https://litellm-internal`, set the base URL in Cursor to `https://litellm-internal/cursor` (not just `https://litellm-internal`).
|
||||
|
||||
### General Setup
|
||||
|
||||
No special configuration is required beyond your normal LiteLLM Proxy setup. Ensure that:
|
||||
|
||||
- Your `config.yaml` includes the models you want to call via this endpoint
|
||||
- Your Cursor project uses your LiteLLM Proxy `base_url` (with `/cursor` included) and a valid API key
|
||||
|
||||
## Notes
|
||||
- This endpoint is intended specifically for Cursor’s request/response expectations. Other clients should continue to use `/v1/chat/completions` or `/v1/responses` as appropriate.
|
||||
|
||||
|
||||
|
|
@ -1,226 +1,85 @@
|
|||
---
|
||||
sidebar_label: "Cursor IDE"
|
||||
# Cursor Integration
|
||||
|
||||
Route Cursor IDE requests through LiteLLM for unified logging, budget controls, and access to any model.
|
||||
|
||||
:::info
|
||||
**Supported modes:** Ask, Plan. Agent mode doesn't support custom API keys yet.
|
||||
:::
|
||||
|
||||
## Quick Reference
|
||||
|
||||
| Setting | Value |
|
||||
|---------|-------|
|
||||
| Base URL | `<LITELLM_PROXY_BASE_URL>/cursor` |
|
||||
| API Key | Your LiteLLM Virtual Key |
|
||||
| Model | Public Model Name from LiteLLM |
|
||||
|
||||
---
|
||||
|
||||
import Tabs from '@theme/Tabs';
|
||||
import TabItem from '@theme/TabItem';
|
||||
## Setup
|
||||
|
||||
# Cursor IDE Integration with LiteLLM
|
||||
### 1. Configure Base URL
|
||||
|
||||
This tutorial shows you how to integrate Cursor IDE with LiteLLM Proxy, allowing you to use any LiteLLM-supported model through Cursor's interface with BYOK (Bring Your Own Key) and custom base URL.
|
||||
Open **Cursor → Settings → Cursor Settings → Models**.
|
||||
|
||||
## Benefits of using Cursor with LiteLLM
|
||||
|
||||
|
||||
When you use Cursor IDE with LiteLLM you get the following benefits:
|
||||
|
||||
**Developer Benefits:**
|
||||
- Universal Model Access: Use any LiteLLM supported model (Anthropic, OpenAI, Vertex AI, Bedrock, etc.) through the Cursor IDE interface.
|
||||
- Higher Rate Limits & Reliability: Load balance across multiple models and providers to avoid hitting individual provider limits, with fallbacks to ensure you get responses even if one provider fails.
|
||||
- Streaming Support: Full streaming support with proper response transformation for Cursor's expected format.
|
||||
|
||||
**Proxy Admin Benefits:**
|
||||
- Centralized Management: Control access to all models through a single LiteLLM proxy instance without giving your developers API Keys to each provider.
|
||||
- Budget Controls: Set spending limits and track costs across all Cursor usage.
|
||||
- Request Logging: Track all requests made through Cursor for debugging and monitoring.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
Before you begin, ensure you have:
|
||||
- Cursor IDE installed
|
||||
- A running LiteLLM Proxy instance with **HTTPS enabled** (HTTP is not supported)
|
||||
- A valid LiteLLM Proxy API key
|
||||
- An HTTPS domain for your LiteLLM Proxy (required by Cursor)
|
||||
|
||||
## Quick Start Guide
|
||||
|
||||
### Step 1: Install LiteLLM
|
||||
|
||||
Install LiteLLM with proxy support:
|
||||
|
||||
```bash
|
||||
pip install litellm[proxy]
|
||||
```
|
||||
|
||||
### Step 2: Configure LiteLLM Proxy
|
||||
|
||||
Create a `config.yaml` file with your model configurations:
|
||||
|
||||
```yaml showLineNumbers title="config.yaml"
|
||||
model_list:
|
||||
- model_name: gpt-4o
|
||||
litellm_params:
|
||||
model: gpt-4o
|
||||
api_key: os.environ/OPENAI_API_KEY
|
||||
|
||||
- model_name: claude-3-5-sonnet
|
||||
litellm_params:
|
||||
model: anthropic/claude-3-5-sonnet-20241022
|
||||
api_key: os.environ/ANTHROPIC_API_KEY
|
||||
|
||||
general_settings:
|
||||
master_key: sk-1234567890 # Change this to a secure key
|
||||
```
|
||||
|
||||
### Step 3: Start LiteLLM Proxy
|
||||
|
||||
Start the proxy server with HTTPS enabled:
|
||||
|
||||
```bash
|
||||
litellm --config config.yaml --port 4000
|
||||
```
|
||||
|
||||
:::warning HTTPS Required
|
||||
|
||||
**Important**: Cursor IDE requires HTTPS connections. HTTP (`http://`) will not work. You must:
|
||||
- Deploy your LiteLLM Proxy with HTTPS enabled
|
||||
- Use a valid SSL certificate
|
||||
- Access the proxy via an HTTPS domain (e.g., `https://your-proxy-domain.com`)
|
||||
|
||||
For local development, you'll need to set up HTTPS (e.g., using a reverse proxy like nginx with SSL, or deploying to a cloud service with HTTPS).
|
||||
|
||||
:::
|
||||
|
||||
### Step 4: Configure Cursor IDE
|
||||
|
||||
Configure Cursor IDE to use your LiteLLM proxy with the `/cursor/chat/completions` endpoint:
|
||||
|
||||
1. Open Cursor IDE
|
||||
2. Go to **Settings** → **Features** → **AI**
|
||||
3. Enable **"Use Custom API"** or **"Bring Your Own Key"**
|
||||
4. Set the following:
|
||||
- **Base URL**: `https://your-proxy-domain.com/cursor` (⚠️ **Important**: Must use HTTPS and include `/cursor`)
|
||||
- **API Key**: Your LiteLLM Proxy API key (e.g., `sk-1234567890`)
|
||||
|
||||
:::warning HTTPS Required
|
||||
|
||||
Cursor IDE **requires HTTPS** connections. HTTP (`http://`) will not work. You must:
|
||||
- Use an HTTPS URL for your base URL (e.g., `https://your-proxy-domain.com/cursor`)
|
||||
- Ensure your LiteLLM Proxy is accessible via HTTPS
|
||||
- Have a valid SSL certificate configured
|
||||
|
||||
:::
|
||||
|
||||
**Example Configuration:**
|
||||
Enable **Override OpenAI Base URL** and enter your proxy URL with `/cursor`:
|
||||
|
||||
```
|
||||
Base URL: https://your-proxy-domain.com/cursor
|
||||
API Key: sk-1234567890
|
||||
https://your-litellm-proxy.com/cursor
|
||||
```
|
||||
|
||||
Replace `your-proxy-domain.com` with your actual HTTPS domain where LiteLLM Proxy is running.
|
||||
|
||||
|
||||
:::info Why `/cursor` in the base URL?
|
||||
### 2. Create Virtual Key
|
||||
|
||||
Cursor automatically appends `/chat/completions` to the base URL you provide. By setting the base URL to `https://your-proxy-domain.com/cursor`, Cursor will send requests to `/cursor/chat/completions`, which is the special endpoint that handles Cursor's Responses API input format and transforms it to Chat Completions output format.
|
||||
In LiteLLM Dashboard, go to **Virtual Keys → + Create New Key**.
|
||||
|
||||
If you set the base URL to just `https://your-proxy-domain.com`, Cursor would send requests to `/chat/completions`, which won't work correctly with Cursor's request format.
|
||||
|
||||
|
||||
Name your key and select which models it can access.
|
||||
|
||||
:::
|
||||
|
||||
|
||||
### Step 5: Test the Integration
|
||||
Click **Create Key** then copy it immediately—you won't see it again.
|
||||
|
||||
1. Restart Cursor IDE to apply the settings
|
||||
2. Open a code file and try using Cursor's AI features (completions, chat, etc.)
|
||||
3. Your requests will now be routed through LiteLLM Proxy
|
||||
|
||||
|
||||
You can verify it's working by:
|
||||
- Checking the LiteLLM Proxy logs for incoming requests
|
||||
- Using Cursor's chat feature and seeing responses stream correctly
|
||||
- Checking your LiteLLM dashboard for request logs and cost tracking
|
||||
Paste it into the **OpenAI API Key** field in Cursor.
|
||||
|
||||
## How It Works
|
||||
|
||||
|
||||
The `/cursor/chat/completions` endpoint is specifically designed to handle Cursor's unique request format:
|
||||
### 3. Add Custom Model
|
||||
|
||||
1. **Input**: Cursor sends requests in OpenAI Responses API format (with `input` field)
|
||||
2. **Processing**: LiteLLM processes the request through its internal `/responses` flow
|
||||
3. **Output**: The response is transformed to OpenAI Chat Completions format (with `choices` field) that Cursor expects
|
||||
Click **+ Add Custom Model** in Cursor Settings.
|
||||
|
||||
This transformation happens automatically for both streaming and non-streaming responses.
|
||||
|
||||
|
||||
## Advanced Configuration
|
||||
Get the **Public Model Name** from LiteLLM Dashboard → Models + Endpoints.
|
||||
|
||||
### Using Different Models
|
||||
|
||||
|
||||
You can configure Cursor to use different models by updating your `config.yaml`:
|
||||
Paste the name in Cursor and enable the toggle.
|
||||
|
||||
```yaml showLineNumbers title="config.yaml"
|
||||
model_list:
|
||||
- model_name: gpt-4o
|
||||
litellm_params:
|
||||
model: gpt-4o
|
||||
api_key: os.environ/OPENAI_API_KEY
|
||||
|
||||
- model_name: claude-3-5-sonnet
|
||||
litellm_params:
|
||||
model: anthropic/claude-3-5-sonnet-20241022
|
||||
api_key: os.environ/ANTHROPIC_API_KEY
|
||||
|
||||
- model_name: gemini-pro
|
||||
litellm_params:
|
||||
model: gemini/gemini-1.5-pro
|
||||
api_key: os.environ/GEMINI_API_KEY
|
||||
```
|
||||
|
||||
|
||||
Then in Cursor, you can specify which model to use in your requests.
|
||||
### 4. Test
|
||||
|
||||
### Rate Limiting and Budgets
|
||||
Open **Ask** mode with `Cmd+L` / `Ctrl+L` and select your model.
|
||||
|
||||
Set up rate limits and budgets in your `config.yaml`:
|
||||
|
||||
|
||||
```yaml showLineNumbers title="config.yaml"
|
||||
general_settings:
|
||||
master_key: sk-1234567890
|
||||
Send a message. All requests now route through LiteLLM.
|
||||
|
||||
litellm_settings:
|
||||
# Set max budget per user
|
||||
max_budget: 100.0
|
||||
|
||||
# Set rate limits
|
||||
rate_limit: 100 # requests per minute
|
||||
```
|
||||
|
||||
|
||||
### Request Logging
|
||||
|
||||
All requests from Cursor will be logged by LiteLLM Proxy. You can:
|
||||
- View logs in the LiteLLM Admin UI
|
||||
- Export logs to your preferred logging service
|
||||
- Track costs per user/team
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Cursor shows no output
|
||||
|
||||
- **Check base URL**: Ensure it uses HTTPS and includes `/cursor` (e.g., `https://your-proxy-domain.com/cursor`, not `http://` or without `/cursor`)
|
||||
- **Verify HTTPS**: Cursor requires HTTPS - HTTP connections will not work
|
||||
- **Check API key**: Verify your LiteLLM Proxy API key is correct
|
||||
- **Check proxy logs**: Look for errors in the LiteLLM Proxy logs
|
||||
|
||||
### Requests failing
|
||||
|
||||
- **Verify HTTPS is enabled**: Cursor requires HTTPS connections. Ensure your LiteLLM Proxy is accessible via HTTPS with a valid SSL certificate
|
||||
- **Verify proxy is running**: Check that LiteLLM Proxy is accessible at your HTTPS base URL
|
||||
- **Check SSL certificate**: Ensure your SSL certificate is valid and not expired
|
||||
- **Check model configuration**: Ensure the model you're trying to use is configured in `config.yaml`
|
||||
- **Check API keys**: Verify provider API keys are set correctly in environment variables
|
||||
|
||||
### HTTP not working
|
||||
|
||||
If you're trying to use HTTP (`http://`) and it's not working:
|
||||
- **This is expected**: Cursor IDE requires HTTPS connections
|
||||
- **Solution**: Deploy your LiteLLM Proxy with HTTPS enabled (use a reverse proxy like nginx, or deploy to a cloud service that provides HTTPS)
|
||||
|
||||
### Streaming not working
|
||||
|
||||
The `/cursor/chat/completions` endpoint automatically handles streaming. If streaming isn't working:
|
||||
- Check that your model supports streaming
|
||||
- Verify the proxy logs for any transformation errors
|
||||
- Ensure Cursor IDE is up to date
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- [Cursor Endpoint Documentation](/docs/proxy/cursor) - Detailed endpoint documentation
|
||||
- [LiteLLM Proxy Setup](/docs/proxy/quick_start) - General proxy setup guide
|
||||
- [Model Configuration](/docs/proxy/configs) - How to configure models
|
||||
|
||||
| Issue | Solution |
|
||||
|-------|----------|
|
||||
| Model not responding | Check base URL ends with `/cursor` and key has model access |
|
||||
| Auth errors | Regenerate key; ensure it starts with `sk-` |
|
||||
| Agent mode not working | Expected—only Ask and Plan modes support custom keys |
|
||||
|
|
|
|||
|
|
@ -247,6 +247,7 @@ class LiteLLMRoutes(enum.Enum):
|
|||
"/openai/deployments/{model}/chat/completions",
|
||||
"/chat/completions",
|
||||
"/v1/chat/completions",
|
||||
"/cursor/chat/completions",
|
||||
# completions
|
||||
"/engines/{model}/completions",
|
||||
"/openai/deployments/{model}/completions",
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue