+ +**Actions** +- Capture LiteLLM proxy logs alongside MCP-server logs (see [Error Log Example](./mcp_troubleshoot#error-log-example-failed-mcp-call)) to inspect the request/response pair and stack traces. +- From the LiteLLM server, run Method 2 ([`curl` smoke test](./mcp_troubleshoot#curl-smoke-test)) against the MCP endpoint to confirm basic connectivity. + +### Client Traffic Issues (Client → LiteLLM) +If only real client requests fail, determine whether LiteLLM ever reaches the MCP hop. + +#### MCP Protocol Sessions +Clients such as IDEs or agent runtimes speak the MCP protocol directly with LiteLLM. + +**Actions** +- Inspect LiteLLM access logs (see [Access Log Example](./mcp_troubleshoot#access-log-example-successful-mcp-call)) to verify the client request reached the proxy and which MCP server it targeted. +- Review LiteLLM error logs (see [Error Log Example](./mcp_troubleshoot#error-log-example-failed-mcp-call)) for TLS, authentication, or routing errors that block the request before the MCP call starts. +- Use the [MCP Inspector](./mcp_troubleshoot#mcp-inspector) to confirm the MCP server is reachable outside of the failing client. + +#### Responses/Completions with Embedded MCP Calls +During `/responses` or `/chat/completions`, LiteLLM may trigger MCP tool calls mid-request. An error could occur before the MCP call begins or after the MCP responds. + +**Actions** +- Check LiteLLM request logs (see [Access Log Example](./mcp_troubleshoot#access-log-example-successful-mcp-call)) to see whether an MCP attempt was recorded; if not, the problem lies in `Client → LiteLLM`. +- Validate MCP connectivity with the [MCP Inspector](./mcp_troubleshoot#mcp-inspector) to ensure the server responds. +- Reproduce the same MCP call via the LiteLLM Playground to confirm LiteLLM can complete the MCP hop independently. + +
+ +## Required Variables + +```python showLineNumbers title="Environment Variables" +os.environ["ABLITERATION_API_KEY"] = "" # your Abliteration API key +``` + +## Sample Usage + +```python showLineNumbers title="Abliteration Completion" +import os +from litellm import completion + +os.environ["ABLITERATION_API_KEY"] = "" + +response = completion( + model="abliteration/abliterated-model", + messages=[{"role": "user", "content": "Hello from LiteLLM"}], +) + +print(response) +``` + +## Sample Usage - Streaming + +```python showLineNumbers title="Abliteration Streaming Completion" +import os +from litellm import completion + +os.environ["ABLITERATION_API_KEY"] = "" + +response = completion( + model="abliteration/abliterated-model", + messages=[{"role": "user", "content": "Stream a short reply"}], + stream=True, +) + +for chunk in response: + print(chunk) +``` + +## Usage with LiteLLM Proxy Server + +1. Add the model to your proxy config: + +```yaml showLineNumbers title="config.yaml" +model_list: + - model_name: abliteration-chat + litellm_params: + model: abliteration/abliterated-model + api_key: os.environ/ABLITERATION_API_KEY +``` + +2. Start the proxy: + +```bash +litellm --config /path/to/config.yaml +``` + +## Direct API Usage (Bearer Token) + +Use the environment variable as a Bearer token against the OpenAI-compatible endpoint: +`https://api.abliteration.ai/v1/chat/completions`. + +```bash showLineNumbers title="cURL" +export ABLITERATION_API_KEY="" +curl https://api.abliteration.ai/v1/chat/completions \ + -H "Authorization: Bearer ${ABLITERATION_API_KEY}" \ + -H "Content-Type: application/json" \ + -d '{ + "model": "abliterated-model", + "messages": [{"role": "user", "content": "Hello from Abliteration"}] + }' +``` + +```python showLineNumbers title="Python (requests)" +import os +import requests + +api_key = os.environ["ABLITERATION_API_KEY"] + +response = requests.post( + "https://api.abliteration.ai/v1/chat/completions", + headers={ + "Authorization": f"Bearer {api_key}", + "Content-Type": "application/json", + }, + json={ + "model": "abliterated-model", + "messages": [{"role": "user", "content": "Hello from Abliteration"}], + }, + timeout=60, +) + +print(response.json()) +``` diff --git a/docs/my-website/docs/providers/anthropic.md b/docs/my-website/docs/providers/anthropic.md index cae8657f1a0..446d663c5ac 100644 --- a/docs/my-website/docs/providers/anthropic.md +++ b/docs/my-website/docs/providers/anthropic.md @@ -1692,9 +1692,9 @@ Assistant: ``` -## Usage - PDF +## Usage - PDF -Pass base64 encoded PDF files to Anthropic models using the `image_url` field. +Pass base64 encoded PDF files to Anthropic models using the `file` content type with a `file_data` field.
@@ -78,3 +78,135 @@ response = completion(
route= ""
)
```
+
+## Embedding
+
+```python
+from litellm import embedding
+import os
+
+os.environ["OPENROUTER_API_KEY"] = "your-api-key"
+
+response = embedding(
+ model="openrouter/openai/text-embedding-3-small",
+ input=["good morning from litellm", "this is another item"],
+)
+print(response)
+```
+
+## Image Generation
+
+OpenRouter supports image generation through select models like Google Gemini image generation models. LiteLLM transforms standard image generation requests to OpenRouter's chat completion format.
+
+### Supported Parameters
+
+- `size`: Maps to OpenRouter's `aspect_ratio` format
+ - `1024x1024` → `1:1` (square)
+ - `1536x1024` → `3:2` (landscape)
+ - `1024x1536` → `2:3` (portrait)
+ - `1792x1024` → `16:9` (wide landscape)
+ - `1024x1792` → `9:16` (tall portrait)
+
+- `quality`: Maps to OpenRouter's `image_size` format (Gemini models)
+ - `low` or `standard` → `1K`
+ - `medium` → `2K`
+ - `high` or `hd` → `4K`
+
+- `n`: Number of images to generate
+
+### Usage
+
+```python
+from litellm import image_generation
+import os
+
+os.environ["OPENROUTER_API_KEY"] = "your-api-key"
+
+# Basic image generation
+response = image_generation(
+ model="openrouter/google/gemini-2.5-flash-image",
+ prompt="A beautiful sunset over a calm ocean",
+)
+print(response)
+```
+
+### Advanced Usage with Parameters
+
+```python
+from litellm import image_generation
+import os
+
+os.environ["OPENROUTER_API_KEY"] = "your-api-key"
+
+# Generate high-quality landscape image
+response = image_generation(
+ model="openrouter/google/gemini-2.5-flash-image",
+ prompt="A serene mountain landscape with a lake",
+ size="1536x1024", # Landscape format
+ quality="high", # High quality (4K)
+)
+
+# Access the generated image
+image_data = response.data[0]
+if image_data.b64_json:
+ # Base64 encoded image
+ print(f"Generated base64 image: {image_data.b64_json[:50]}...")
+elif image_data.url:
+ # Image URL
+ print(f"Generated image URL: {image_data.url}")
+```
+
+### Using OpenRouter-Specific Parameters
+
+You can also pass OpenRouter-specific parameters directly using `image_config`:
+
+```python
+from litellm import image_generation
+import os
+
+os.environ["OPENROUTER_API_KEY"] = "your-api-key"
+
+response = image_generation(
+ model="openrouter/google/gemini-2.5-flash-image",
+ prompt="A futuristic cityscape at night",
+ image_config={
+ "aspect_ratio": "16:9", # OpenRouter native format
+ "image_size": "4K" # OpenRouter native format
+ }
+)
+print(response)
+```
+
+### Response Format
+
+The response follows the standard LiteLLM ImageResponse format:
+
+```python
+{
+ "created": 1703658209,
+ "data": [{
+ "b64_json": "iVBORw0KGgoAAAANSUhEUgAA...", # Base64 encoded image
+ "url": None,
+ "revised_prompt": None
+ }],
+ "usage": {
+ "input_tokens": 10,
+ "output_tokens": 1290,
+ "total_tokens": 1300
+ }
+}
+```
+
+### Cost Tracking
+
+OpenRouter provides cost information in the response, which LiteLLM automatically tracks:
+
+```python
+response = image_generation(
+ model="openrouter/google/gemini-2.5-flash-image",
+ prompt="A cute baby sea otter",
+)
+
+# Cost is available in the response metadata
+print(f"Request cost: ${response._hidden_params['additional_headers']['llm_provider-x-litellm-response-cost']}")
+```
diff --git a/docs/my-website/docs/providers/sap.md b/docs/my-website/docs/providers/sap.md
index ce8f3fa15dd..16f30a2e99c 100644
--- a/docs/my-website/docs/providers/sap.md
+++ b/docs/my-website/docs/providers/sap.md
@@ -12,102 +12,340 @@ LiteLLM supports SAP Generative AI Hub's Orchestration Service.
| Supported Endpoints | `/chat/completions`, `/embeddings` |
| API Reference | [SAP AI Core Documentation](https://help.sap.com/docs/sap-ai-core) |
+## Prerequisites
+
+Before you begin, ensure you have:
+
+1. **SAP BTP Account** with access to SAP AI Core
+2. **AI Core Service Instance** provisioned in your subaccount
+3. **Service Key** created for your AI Core instance (this contains your credentials)
+4. **Resource Group** with deployed AI models (check with your SAP administrator)
+
+:::tip Where to Find Your Credentials
+Your credentials come from the **Service Key** you create in SAP BTP Cockpit:
+
+1. Navigate to your **Subaccount** → **Instances and Subscriptions**
+2. Find your **AI Core** instance and click on it
+3. Go to **Service Keys** and create one (or use existing)
+4. The JSON contains all values needed below
+
+The service key JSON looks like this:
+
+```json
+{
+ "clientid": "sb-abc123...",
+ "clientsecret": "xyz789...",
+ "url": "https://myinstance.authentication.eu10.hana.ondemand.com",
+ "serviceurls": {
+ "AI_API_URL": "https://api.ai.prod.eu-central-1.aws.ml.hana.ondemand.com"
+ }
+}
+```
+
+:::info Resource Group
+The resource group is typically configured separately in your AI Core deployment, not in the service key itself. You can set it via the `AICORE_RESOURCE_GROUP` environment variable (defaults to "default").
+:::
+
+## Quick Start
+
+### Step 1: Install LiteLLM
+
+```bash
+pip install litellm
+```
+
+### Step 2: Set Your Credentials
+
+Choose **one** of these authentication methods:
+
+-AICORE_AUTH_URL = "https://* * * .authentication.sap.hana.ondemand.com/oauth/token", -AICORE_CLIENT_ID = " *** ", -AICORE_CLIENT_SECRET = " *** ", -AICORE_RESOURCE_GROUP = " *** ", -AICORE_BASE_URL = "https://api.ai.***.cfapps.sap.hana.ondemand.com/v2" --Other credential configuration options are also available. For more information, see the [SAP AI Core Documentation](https://help.sap.com/doc/generative-ai-hub-sdk/CLOUD/en-US/_reference/README_sphynx.html#configuration). -## Usage - LiteLLM Python SDK +### Proxy Usage -```python showLineNumbers title="SAP Chat Completion" -from litellm import completion -import os +When using the LiteLLM Proxy, you use the **friendly `model_name`** defined in your configuration. The proxy automatically handles the `sap/` prefix routing. -os.environ["AICORE_SERVICE_KEY"] = '{"clientid": "...", "clientsecret": "...", ...}' +```yaml +# In config.yaml, define the mapping +model_list: + - model_name: gpt-4o # ← Use this name in client requests + litellm_params: + model: sap/gpt-4o # ← Proxy handles the sap/ prefix +``` -response = completion( - model="sap/gpt-4", - messages=[{"role": "user", "content": "Hello from LiteLLM"}] +```python +# Client request - no sap/ prefix needed +client.chat.completions.create( + model="gpt-4o", # ✓ Correct for proxy usage + messages=[...] ) -print(response) ``` -```python showLineNumbers title="SAP Chat Completion - Streaming" +### Anthropic Models Special Syntax + +Anthropic models use a double-dash (`--`) prefix convention: + +| Provider | Model Example | LiteLLM Format | +|----------|---------------|----------------| +| OpenAI | GPT-4o | `sap/gpt-4o` | +| Anthropic | Claude 4.5 Sonnet | `sap/anthropic--claude-4.5-sonnet` | +| Google | Gemini 2.5 Pro | `sap/gemini-2.5-pro` | +| Mistral | Mistral Large | `sap/mistral-large` | + +### Quick Reference Table + +| Usage Type | Model Format | Example | +|------------|--------------|---------| +| Direct SDK | `sap/
.default})
diff --git a/docs/my-website/docs/tutorials/claude_non_anthropic_models.md b/docs/my-website/docs/tutorials/claude_non_anthropic_models.md
new file mode 100644
index 00000000000..75ac08e3094
--- /dev/null
+++ b/docs/my-website/docs/tutorials/claude_non_anthropic_models.md
@@ -0,0 +1,316 @@
+import Image from '@theme/IdealImage';
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Use Claude Code with Non-Anthropic Models
+
+This tutorial shows how to use Claude Code with non-Anthropic models like OpenAI, Gemini, and other LLM providers through LiteLLM proxy.
+
+:::info
+
+LiteLLM automatically translates between different provider formats, allowing you to use any supported LLM provider with Claude Code while maintaining the Anthropic Messages API format.
+
+:::
+
+## Prerequisites
+
+- [Claude Code](https://docs.anthropic.com/en/docs/claude-code/overview) installed
+- API keys for your chosen providers (OpenAI, Vertex AI, etc.)
+
+## Installation
+
+First, install LiteLLM with proxy support:
+
+```bash
+pip install 'litellm[proxy]'
+```
+
+## Configuration
+
+### 1. Setup config.yaml
+
+Create a configuration file with your preferred non-Anthropic models:
+
+