+
Call 100+ LLMs in OpenAI format. [Bedrock, Azure, OpenAI, VertexAI, Anthropic, Groq, etc.] +
Call all LLM APIs using the OpenAI format [Bedrock, Huggingface, VertexAI, TogetherAI, Azure, OpenAI, Groq etc.]
-
@@ -30,29 +30,17 @@
-
+### Python SDK
```shell
pip install litellm
@@ -62,253 +50,237 @@ pip install litellm
from litellm import completion
import os
-## set ENV variables
os.environ["OPENAI_API_KEY"] = "your-openai-key"
os.environ["ANTHROPIC_API_KEY"] = "your-anthropic-key"
-messages = [{ "content": "Hello, how are you?","role": "user"}]
+# OpenAI
+response = completion(model="openai/gpt-4o", messages=[{"role": "user", "content": "Hello!"}])
-# openai call
-response = completion(model="openai/gpt-4o", messages=messages)
-
-# anthropic call
-response = completion(model="anthropic/claude-sonnet-4-20250514", messages=messages)
-print(response)
+# Anthropic
+response = completion(model="anthropic/claude-sonnet-4-20250514", messages=[{"role": "user", "content": "Hello!"}])
```
-### Response (OpenAI Format)
+### AI Gateway (Proxy Server)
-```json
-{
- "id": "chatcmpl-1214900a-6cdd-4148-b663-b5e2f642b4de",
- "created": 1751494488,
- "model": "claude-sonnet-4-20250514",
- "object": "chat.completion",
- "system_fingerprint": null,
- "choices": [
- {
- "finish_reason": "stop",
- "index": 0,
- "message": {
- "content": "Hello! I'm doing well, thank you for asking. I'm here and ready to help with whatever you'd like to discuss or work on. How are you doing today?",
- "role": "assistant",
- "tool_calls": null,
- "function_call": null
- }
- }
- ],
- "usage": {
- "completion_tokens": 39,
- "prompt_tokens": 13,
- "total_tokens": 52,
- "completion_tokens_details": null,
- "prompt_tokens_details": {
- "audio_tokens": null,
- "cached_tokens": 0
- },
- "cache_creation_input_tokens": 0,
- "cache_read_input_tokens": 0
- }
-}
-```
-
-Call any model supported by a provider, with `model=| + | LiteLLM AI Gateway | +LiteLLM Python SDK | +
|---|---|---|
| Use Case | +Central service (LLM Gateway) to access multiple LLMs | +Use LiteLLM directly in your Python code | +
| Who Uses It? | +Gen AI Enablement / ML Platform Teams | +Developers building LLM projects | +
| Key Features | +Centralized API gateway with authentication and authorization, multi-tenant cost tracking and spend management per project/user, per-project customization (logging, guardrails, caching), virtual keys for secure access control, admin dashboard UI for monitoring and management | +Direct Python library integration in your codebase, Router with retry/fallback logic across multiple deployments (e.g. Azure/OpenAI) - Router, application-level load balancing and cost tracking, exception handling with OpenAI-compatible errors, observability callbacks (Lunary, MLflow, Langfuse, etc.) | +
Netflix |
+
| + | LiteLLM Proxy Server | +LiteLLM Python SDK | +
|---|---|---|
| Use Case | +Central service (LLM Gateway) to access multiple LLMs | +Use LiteLLM directly in your Python code | +
| Who Uses It? | +Gen AI Enablement / ML Platform Teams | +Developers building LLM projects | +
| Key Features | +• Centralized API gateway with authentication & authorization • Multi-tenant cost tracking and spend management per project/user • Per-project customization (logging, guardrails, caching) • Virtual keys for secure access control • Admin dashboard UI for monitoring and management |
+• Direct Python library integration in your codebase • Router with retry/fallback logic across multiple deployments (e.g. Azure/OpenAI) - Router • Application-level load balancing and cost tracking • Exception handling with OpenAI-compatible errors • Observability callbacks (Lunary, MLflow, Langfuse, etc.) |
+
@@ -251,7 +257,7 @@ response = completion(
{
"type": "image_url",
"image_url": {
- "url": "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg"
+ "url": "https://awsmp-logos.s3.amazonaws.com/seller-xw5kijmvmzasy/c233c9ade2ccb5491072ae232c814942.png"
}
}
]
@@ -543,7 +549,8 @@ print(response)
### Entra ID - use `azure_ad_token`
-This is a walkthrough on how to use Azure Active Directory Tokens - Microsoft Entra ID to make `litellm.completion()` calls
+This is a walkthrough on how to use Azure Active Directory Tokens - Microsoft Entra ID to make `litellm.completion()` calls.
+> **Note:** You can follow the same process below to use Azure Active Directory Tokens for all other Azure endpoints (e.g., chat, embeddings, image, audio, etc.) with LiteLLM.
Step 1 - Download Azure CLI
Installation instructions: https://learn.microsoft.com/en-us/cli/azure/install-azure-cli
diff --git a/docs/my-website/docs/providers/azure/azure_anthropic.md b/docs/my-website/docs/providers/azure/azure_anthropic.md
new file mode 100644
index 00000000000..4c722b30397
--- /dev/null
+++ b/docs/my-website/docs/providers/azure/azure_anthropic.md
@@ -0,0 +1,378 @@
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Azure Anthropic (Claude via Azure Foundry)
+
+LiteLLM supports Claude models deployed via Microsoft Azure Foundry, including Claude Sonnet 4.5, Claude Haiku 4.5, and Claude Opus 4.1.
+
+## Available Models
+
+Azure Foundry supports the following Claude models:
+
+- `claude-sonnet-4-5` - Anthropic's most capable model for building real-world agents and handling complex, long-horizon tasks
+- `claude-haiku-4-5` - Near-frontier performance with the right speed and cost for high-volume use cases
+- `claude-opus-4-1` - Industry leader for coding, delivering sustained performance on long-running tasks
+
+| Property | Details |
+|-------|-------|
+| Description | Claude models deployed via Microsoft Azure Foundry. Uses the same API as Anthropic's Messages API but with Azure authentication. |
+| Provider Route on LiteLLM | `azure_ai/` (add this prefix to Claude model names - e.g. `azure_ai/claude-sonnet-4-5`) |
+| Provider Doc | [Azure Foundry Claude Models ↗](https://learn.microsoft.com/en-us/azure/ai-services/foundry-models/claude) |
+| API Endpoint | `https://
@@ -78,3 +78,135 @@ response = completion(
route= ""
)
```
+
+## Embedding
+
+```python
+from litellm import embedding
+import os
+
+os.environ["OPENROUTER_API_KEY"] = "your-api-key"
+
+response = embedding(
+ model="openrouter/openai/text-embedding-3-small",
+ input=["good morning from litellm", "this is another item"],
+)
+print(response)
+```
+
+## Image Generation
+
+OpenRouter supports image generation through select models like Google Gemini image generation models. LiteLLM transforms standard image generation requests to OpenRouter's chat completion format.
+
+### Supported Parameters
+
+- `size`: Maps to OpenRouter's `aspect_ratio` format
+ - `1024x1024` → `1:1` (square)
+ - `1536x1024` → `3:2` (landscape)
+ - `1024x1536` → `2:3` (portrait)
+ - `1792x1024` → `16:9` (wide landscape)
+ - `1024x1792` → `9:16` (tall portrait)
+
+- `quality`: Maps to OpenRouter's `image_size` format (Gemini models)
+ - `low` or `standard` → `1K`
+ - `medium` → `2K`
+ - `high` or `hd` → `4K`
+
+- `n`: Number of images to generate
+
+### Usage
+
+```python
+from litellm import image_generation
+import os
+
+os.environ["OPENROUTER_API_KEY"] = "your-api-key"
+
+# Basic image generation
+response = image_generation(
+ model="openrouter/google/gemini-2.5-flash-image",
+ prompt="A beautiful sunset over a calm ocean",
+)
+print(response)
+```
+
+### Advanced Usage with Parameters
+
+```python
+from litellm import image_generation
+import os
+
+os.environ["OPENROUTER_API_KEY"] = "your-api-key"
+
+# Generate high-quality landscape image
+response = image_generation(
+ model="openrouter/google/gemini-2.5-flash-image",
+ prompt="A serene mountain landscape with a lake",
+ size="1536x1024", # Landscape format
+ quality="high", # High quality (4K)
+)
+
+# Access the generated image
+image_data = response.data[0]
+if image_data.b64_json:
+ # Base64 encoded image
+ print(f"Generated base64 image: {image_data.b64_json[:50]}...")
+elif image_data.url:
+ # Image URL
+ print(f"Generated image URL: {image_data.url}")
+```
+
+### Using OpenRouter-Specific Parameters
+
+You can also pass OpenRouter-specific parameters directly using `image_config`:
+
+```python
+from litellm import image_generation
+import os
+
+os.environ["OPENROUTER_API_KEY"] = "your-api-key"
+
+response = image_generation(
+ model="openrouter/google/gemini-2.5-flash-image",
+ prompt="A futuristic cityscape at night",
+ image_config={
+ "aspect_ratio": "16:9", # OpenRouter native format
+ "image_size": "4K" # OpenRouter native format
+ }
+)
+print(response)
+```
+
+### Response Format
+
+The response follows the standard LiteLLM ImageResponse format:
+
+```python
+{
+ "created": 1703658209,
+ "data": [{
+ "b64_json": "iVBORw0KGgoAAAANSUhEUgAA...", # Base64 encoded image
+ "url": None,
+ "revised_prompt": None
+ }],
+ "usage": {
+ "input_tokens": 10,
+ "output_tokens": 1290,
+ "total_tokens": 1300
+ }
+}
+```
+
+### Cost Tracking
+
+OpenRouter provides cost information in the response, which LiteLLM automatically tracks:
+
+```python
+response = image_generation(
+ model="openrouter/google/gemini-2.5-flash-image",
+ prompt="A cute baby sea otter",
+)
+
+# Cost is available in the response metadata
+print(f"Request cost: ${response._hidden_params['additional_headers']['llm_provider-x-litellm-response-cost']}")
+```
diff --git a/docs/my-website/docs/providers/ovhcloud.md b/docs/my-website/docs/providers/ovhcloud.md
index 6c42208f2cc..94625b0f2ed 100644
--- a/docs/my-website/docs/providers/ovhcloud.md
+++ b/docs/my-website/docs/providers/ovhcloud.md
@@ -311,6 +311,21 @@ response = embedding(
print(response.data)
```
+### Audio Transcription
+
+```python
+from litellm import transcription
+
+audio_file = open("path/to/your/audio.wav", "rb")
+
+response = transcription(
+ model="ovhcloud/whisper-large-v3-turbo",
+ file=audio_file
+)
+
+print(response.text)
+```
+
## Usage with LiteLLM Proxy Server
Here's how to call a OVHCloud AI Endpoints model with the LiteLLM Proxy Server
diff --git a/docs/my-website/docs/providers/poe.md b/docs/my-website/docs/providers/poe.md
new file mode 100644
index 00000000000..ba4089ae6a4
--- /dev/null
+++ b/docs/my-website/docs/providers/poe.md
@@ -0,0 +1,139 @@
+# Poe
+
+## Overview
+
+| Property | Details |
+|-------|-------|
+| Description | Poe is Quora's AI platform that provides access to more than 100 models across text, image, video, and voice modalities through a developer-friendly API. |
+| Provider Route on LiteLLM | `poe/` |
+| Link to Provider Doc | [Poe Website ↗](https://poe.com) |
+| Base URL | `https://api.poe.com/v1` |
+| Supported Operations | [`/chat/completions`](#sample-usage) |
+
+Hello, world!
-This is a test of the
Hello, world!
-This is a test of the
Hello, world!
-This is a test of the
Hello, world!
-This is a test of the
Hello, world!
+This is a test of the
Hello!
How are you?
Hello!
How are you?
-
-
-```python
-import os
-from litellm import completion
-
-os.environ["WATSONX_URL"] = ""
-os.environ["WATSONX_APIKEY"] = ""
-
-## Call WATSONX `/text/chat` endpoint - supports function calling
-response = completion(
- model="watsonx/meta-llama/llama-3-1-8b-instruct",
- messages=[{ "content": "what is your favorite colour?","role": "user"}],
- project_id="
+
+
+```python showLineNumbers title="Chat Completion"
+import os
+from litellm import completion
+
+os.environ["WATSONX_URL"] = ""
+os.environ["WATSONX_APIKEY"] = ""
+
+response = completion(
+ model="watsonx/meta-llama/llama-3-1-8b-instruct",
+ messages=[{ "content": "what is your favorite colour?","role": "user"}],
+ project_id=".default})
diff --git a/docs/my-website/docs/tutorials/claude_non_anthropic_models.md b/docs/my-website/docs/tutorials/claude_non_anthropic_models.md
new file mode 100644
index 00000000000..75ac08e3094
--- /dev/null
+++ b/docs/my-website/docs/tutorials/claude_non_anthropic_models.md
@@ -0,0 +1,316 @@
+import Image from '@theme/IdealImage';
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Use Claude Code with Non-Anthropic Models
+
+This tutorial shows how to use Claude Code with non-Anthropic models like OpenAI, Gemini, and other LLM providers through LiteLLM proxy.
+
+:::info
+
+LiteLLM automatically translates between different provider formats, allowing you to use any supported LLM provider with Claude Code while maintaining the Anthropic Messages API format.
+
+:::
+
+## Prerequisites
+
+- [Claude Code](https://docs.anthropic.com/en/docs/claude-code/overview) installed
+- API keys for your chosen providers (OpenAI, Vertex AI, etc.)
+
+## Installation
+
+First, install LiteLLM with proxy support:
+
+```bash
+pip install 'litellm[proxy]'
+```
+
+## Configuration
+
+### 1. Setup config.yaml
+
+Create a configuration file with your preferred non-Anthropic models:
+
+POST http://localhost:4000/v1/vector_stores/{vector_store_id}/files
+
+```python
+from openai import OpenAI
+
+client = OpenAI(
+ base_url="http://localhost:4000", # LiteLLM proxy or OpenAI base
+ api_key="sk-1234"
+)
+
+vector_store_file = client.vector_stores.files.create(
+ vector_store_id="vs_69172088a18c8191ab3e2621aa87d1ee",
+ file_id="file-NDbEDJTfqVh7S4Ugi3CGYw",
+ chunking_strategy={
+ "type": "static",
+ "static": {
+ "max_chunk_size_tokens": 800,
+ "chunk_overlap_tokens": 400,
+ },
+ },
+)
+
+print(vector_store_file)
+```
+
+## List vector store files
+
+GET http://localhost:4000/v1/vector_stores/{vector_store_id}/files
+
+Parameters:
+
+- `vector_store_id` (path, required)
+- `after` / `before` (query, optional) – pagination cursors
+- `filter` (query, optional) – `in_progress`, `completed`, `failed`, `cancelled`
+- `limit` (query, optional, default `20`, range `1-100`)
+- `order` (query, optional, default `desc`)
+
+```python
+vector_store_files = client.vector_stores.files.list(
+ vector_store_id="vs_abc123"
+)
+print(vector_store_files)
+```
+
+## Retrieve vector store file
+
+GET http://localhost:4000/v1/vector_stores/{vector_store_id}/files/{file_id}
+
+```python
+vector_store_file = client.vector_stores.files.retrieve(
+ vector_store_id="vs_abc123",
+ file_id="file-abc123"
+)
+print(vector_store_file)
+```
+
+## Delete vector store file
+
+DELETE http://localhost:4000/v1/vector_stores/{vector_store_id}/files/{file_id}
+
+```python
+deleted_vector_store_file = client.vector_stores.files.delete(
+ vector_store_id="vs_abc123",
+ file_id="file-abc123"
+)
+print(deleted_vector_store_file)
+```
+
+## Proxy-only endpoints
+
+When you need raw content chunks or attribute updates, call the LiteLLM Proxy directly.
+
+### Retrieve file content
+
+```bash
+curl -X GET "http://localhost:4000/v1/vector_stores/\{vector_store_id\}/files/\{file_id\}/content" \
+ -H "Authorization: Bearer sk-1234"
+```
+
+### Update file attributes
+
+```bash
+curl -X POST "http://localhost:4000/v1/vector_stores/\{vector_store_id\}/files/\{file_id\}" \
+ -H "Authorization: Bearer sk-1234" \
+ -H "Content-Type: application/json" \
+ -d '{
+ "attributes": {
+ "category": "support-faq",
+ "language": "en"
+ }
+ }'
+```
diff --git a/docs/my-website/docs/vector_stores/create.md b/docs/my-website/docs/vector_stores/create.md
index 19b4f39cd9e..7025c490a32 100644
--- a/docs/my-website/docs/vector_stores/create.md
+++ b/docs/my-website/docs/vector_stores/create.md
@@ -14,6 +14,7 @@ Create a vector store which can be used to store and search document chunks for
| End-user Tracking | ✅ | |
| Support LLM Providers (OpenAI `/vector_stores` API) | **OpenAI** | Full vector stores API support across providers |
| Support LLM Providers (Passthrough API) | [**Azure AI**](/docs/providers/azure_ai/azure_ai_vector_stores_passthrough) | Full vector stores API support across providers |
+| Support LLM Providers (Dataset Management) | [**RAGFlow**](/docs/providers/ragflow_vector_store.md) | Dataset creation and management (search not supported) |
## Usage
diff --git a/docs/my-website/docs/vector_stores/search.md b/docs/my-website/docs/vector_stores/search.md
index 2ffc8ef12e5..3286b3b01e5 100644
--- a/docs/my-website/docs/vector_stores/search.md
+++ b/docs/my-website/docs/vector_stores/search.md
@@ -12,7 +12,7 @@ Search a vector store for relevant chunks based on a query and file attributes f
| Cost Tracking | ✅ | Tracked per search operation |
| Logging | ✅ | Works across all integrations |
| End-user Tracking | ✅ | |
-| Support LLM Providers | **OpenAI, Azure OpenAI, Bedrock, Vertex RAG Engine, Azure AI, Milvus** | Full vector stores API support across providers |
+| Support LLM Providers | **OpenAI, Azure OpenAI, Bedrock, Vertex RAG Engine, Azure AI, Milvus, Gemini** | Full vector stores API support across providers |
## Usage
@@ -164,6 +164,41 @@ print(response)
[See full Milvus vector store documentation](../providers/milvus_vector_stores.md)
+
+
+This is a React page
- Hi {recipient_email},
+
+ Your LiteLLM API key has crossed its soft budget limit of {soft_budget}.
+
+ Current Spend: {spend}
+ Soft Budget: {soft_budget}
+ {max_budget_info}
+
+
+ ⚠️ Note: Your API requests will continue to work, but you should monitor your usage closely. + If you reach your maximum budget, requests will be rejected. +
+ + You can view your usage and manage your budget in the LiteLLM Dashboard. Hi {recipient_email},
+
+ Your LiteLLM API key has reached {percentage}% of its maximum budget.
+
+ Current Spend: {spend}
+ Maximum Budget: {max_budget}
+ Alert Threshold: {alert_threshold} ({percentage}%)
+
+
+ ⚠️ Warning: You are approaching your maximum budget limit. + Once you reach your maximum budget of {max_budget}, all API requests will be rejected. +
+ + You can view your usage and manage your budget in the LiteLLM Dashboard.