scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-06 02:48:13 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 103

added doc versioning

Browse source
This commit is contained in:
shivam 2026-01-31 15:15:58 -08:00
parent d12ce3cd5d
commit 6b0b99a594
5128 changed files with 1238456 additions and 457 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

10
docs/my-website/docs/a2a.md
View file

@ -7,7 +7,7 @@ import Image from '@theme/IdealImage';
Add A2A Agents on LiteLLM AI Gateway, Invoke agents in A2A Protocol, track request/response logs in LiteLLM Logs. Manage which Teams, Keys can access which Agents onboarded.
<Image
img={require('../img/a2a_gateway.png')}
img={require('@site/img/a2a_gateway.png')}
style={{width: '80%', display: 'block', margin: '0', borderRadius: '8px'}}
/>
@ -39,7 +39,7 @@ You can add A2A-compatible agents through the LiteLLM Admin UI.
3. Enter the agent name (e.g., `ij-local`) and the URL of your A2A agent
<Image
img={require('../img/add_agent_1.png')}
img={require('@site/img/add_agent_1.png')}
style={{width: '80%', display: 'block', margin: '0'}}
/>
@ -189,7 +189,7 @@ The logs show:
- **Latency and cost** metrics
<Image
img={require('../img/agent2.png')}
img={require('@site/img/agent2.png')}
style={{width: '100%', display: 'block', margin: '2rem auto'}}
/>
@ -296,14 +296,14 @@ With header forwarding enabled, you'll see:
**Trace Grouping in Langfuse:**
<Image
img={require('../img/a2a_trace_grouping.png')}
img={require('@site/img/a2a_trace_grouping.png')}
style={{width: '80%', display: 'block', margin: '0', borderRadius: '8px'}}
/>
**Agent Spend Attribution:**
<Image
img={require('../img/a2a_agent_spend.png')}
img={require('@site/img/a2a_agent_spend.png')}
style={{width: '80%', display: 'block', margin: '0', borderRadius: '8px'}}
/>

8
docs/my-website/docs/a2a_agent_permissions.md
View file

@ -32,7 +32,7 @@ This example shows how to create a key with agent permissions and test access.
3. Copy the **Agent ID**
<Image
img={require('../img/agent_id.png')}
img={require('@site/img/agent_id.png')}
style={{width: '80%', display: 'block', margin: '0', borderRadius: '8px'}}
/>
@ -67,7 +67,7 @@ Response:
3. Select the agents you want to allow
<Image
img={require('../img/agent_key.png')}
img={require('@site/img/agent_key.png')}
style={{width: '80%', display: 'block', margin: '0', borderRadius: '8px'}}
/>
@ -130,7 +130,7 @@ Restrict all keys belonging to a team to only access specific agents.
3. Select the agents you want to allow for this team
<Image
img={require('../img/agent_key.png')}
img={require('@site/img/agent_key.png')}
style={{width: '80%', display: 'block', margin: '0', borderRadius: '8px'}}
/>
@ -169,7 +169,7 @@ Response:
2. Select the **Team** from the dropdown
<Image
img={require('../img/agent_team.png')}
img={require('@site/img/agent_team.png')}
style={{width: '80%', display: 'block', margin: '0', borderRadius: '8px'}}
/>

6
docs/my-website/docs/a2a_cost_tracking.md
View file

@ -106,7 +106,7 @@ Navigate to the Agent Usage tab in the Admin UI to view agent-level spend analyt
Go to the Usage page in the Admin UI (`PROXY_BASE_URL/ui/?login=success&page=new_usage`) and click on the **Agent Usage** tab.
<Image img={require('../img/agent_usage_ui_navigation.png')} />
<Image img={require('@site/img/agent_usage_ui_navigation.png')} />
### 2. View Agent Analytics
@ -117,7 +117,7 @@ The Agent Usage dashboard provides:
- **Model usage breakdown**: Understand which models each agent uses
- **Activity metrics**: Track requests, tokens, and success rates per agent
<Image img={require('../img/agent_usage_analytics.png')} />
<Image img={require('@site/img/agent_usage_analytics.png')} />
### 3. Filter by Agent
@ -127,7 +127,7 @@ Use the agent filter dropdown to view spend for specific agents:
- View filtered analytics, spend logs, and activity metrics
- Compare spend across different agents
<Image img={require('../img/agent_usage_filter.png')} />
<Image img={require('@site/img/agent_usage_filter.png')} />
## Cost Configuration Options

4
docs/my-website/docs/benchmarks.md
View file

@ -28,11 +28,11 @@ In these tests the baseline latency characteristics are measured against a fake-
| Custom | LiteLLM Overhead Duration (ms) | 12 | 29 | 43 | 14.74 | 1035.7 |
| | Aggregated | 100 | 430 | 930 | 138.6 | 2071.4 |
<!-- <Image img={require('../img/1_instance_proxy.png')} /> -->
<!-- <Image img={require('@site/img/1_instance_proxy.png')} /> -->
<!-- ## **Horizontal Scaling - 10K RPS**
<Image img={require('../img/instances_vs_rps.png')} /> -->
<Image img={require('@site/img/instances_vs_rps.png')} /> -->
### 4 Instances

18
docs/my-website/docs/completion/knowledgebase.md
View file

@ -5,7 +5,7 @@ import Image from '@theme/IdealImage';
# Using Vector Stores (Knowledge Bases)
<Image
img={require('../../img/kb.png')}
img={require('@site/img/kb.png')}
style={{width: '100%', display: 'block', margin: '2rem auto'}}
/>
<p style={{textAlign: 'left', color: '#666'}}>
@ -100,7 +100,7 @@ vector_store_registry:
On the LiteLLM UI, Navigate to Experimental > Vector Stores > Create Vector Store. On this page you can create a vector store with a name, vector store id and credentials.
<Image
img={require('../../img/kb_2.png')}
img={require('@site/img/kb_2.png')}
style={{width: '50%'}}
/>
@ -180,7 +180,7 @@ Ensure you have a Bedrock Knowledge Base created in your AWS account with the ap
3. Enter your Bedrock Knowledge Base ID in the **"Vector Store ID"** field
<Image
img={require('../../img/kb_2.png')}
img={require('@site/img/kb_2.png')}
style={{width: '60%', display: 'block'}}
/>
@ -194,7 +194,7 @@ Ensure you have a Bedrock Knowledge Base created in your AWS account with the ap
<div style={{margin: '20px 0', padding: '10px', border: '1px solid #ddd', borderRadius: '8px', display: 'inline-block', boxShadow: '0 2px 8px rgba(0,0,0,0.1)'}}>
<Image
img={require('../../img/kb_vertex1.png')}
img={require('@site/img/kb_vertex1.png')}
style={{width: '60%', display: 'block'}}
/>
</div>
@ -204,7 +204,7 @@ Ensure you have a Bedrock Knowledge Base created in your AWS account with the ap
<div style={{margin: '20px 0', padding: '10px', border: '1px solid #ddd', borderRadius: '8px', display: 'inline-block', boxShadow: '0 2px 8px rgba(0,0,0,0.1)'}}>
<Image
img={require('../../img/kb_vertex2.png')}
img={require('@site/img/kb_vertex2.png')}
style={{width: '60%', display: 'block'}}
/>
</div>
@ -217,7 +217,7 @@ Ensure you have a Bedrock Knowledge Base created in your AWS account with the ap
<div style={{margin: '20px 0', padding: '10px', border: '1px solid #ddd', borderRadius: '8px', display: 'inline-block', boxShadow: '0 2px 8px rgba(0,0,0,0.1)'}}>
<Image
img={require('../../img/kb_vertex3.png')}
img={require('@site/img/kb_vertex3.png')}
style={{width: '60%', display: 'block'}}
/>
</div>
@ -261,7 +261,7 @@ Once your litellm-pg-vector-store is deployed:
<div style={{margin: '20px 0', padding: '10px', border: '1px solid #ddd', borderRadius: '8px', display: 'inline-block', boxShadow: '0 2px 8px rgba(0,0,0,0.1)'}}>
<Image
img={require('../../img/kb_pg1.png')}
img={require('@site/img/kb_pg1.png')}
style={{width: '60%', display: 'block'}}
/>
</div>
@ -282,7 +282,7 @@ Once your litellm-pg-vector-store is deployed:
<div style={{margin: '20px 0', padding: '10px', border: '1px solid #ddd', borderRadius: '8px', display: 'inline-block', boxShadow: '0 2px 8px rgba(0,0,0,0.1)'}}>
<Image
img={require('../../img/kb_openai1.png')}
img={require('@site/img/kb_openai1.png')}
style={{width: '60%', display: 'block'}}
/>
</div>
@ -298,7 +298,7 @@ LiteLLM allows you to view your vector store usage in the LiteLLM UI on the `Log
After completing a request with a vector store, navigate to the `Logs` page on LiteLLM. Here you should be able to see the query sent to the vector store and corresponding response with scores.
<Image
img={require('../../img/kb_4.png')}
img={require('@site/img/kb_4.png')}
style={{width: '80%'}}
/>
<p style={{textAlign: 'left', color: '#666'}}>

4
docs/my-website/docs/default_code_snippet.md
View file

@ -3,8 +3,8 @@ displayed_sidebar: tutorialSidebar
---
# Get Started
import QueryParamReader from '../src/components/queryParamReader.js'
import TokenComponent from '../src/components/queryParamToken.js'
import QueryParamReader from '@site/src/components/queryParamReader.js'
import TokenComponent from '@site/src/components/queryParamToken.js'
:::info

10
docs/my-website/docs/enterprise.md
View file

@ -17,7 +17,7 @@ Get free 7-day trial key [here](https://www.litellm.ai/enterprise#trial)
Includes all enterprise features.
<Image img={require('../img/enterprise_vs_oss_2.png')} />
<Image img={require('@site/img/enterprise_vs_oss_2.png')} />
[**Procurement available via AWS / Azure Marketplace**](./data_security.md#legalcompliance-faqs)
@ -102,17 +102,17 @@ Pricing is based on usage. We can figure out a price that works for your team, o
### 1. Create keys
<Image img={require('../img/litellm_hosted_ui_create_key.png')} />
<Image img={require('@site/img/litellm_hosted_ui_create_key.png')} />
### 2. Add Models
<Image img={require('../img/litellm_hosted_ui_add_models.png')}/>
<Image img={require('@site/img/litellm_hosted_ui_add_models.png')}/>
### 3. Track spend
<Image img={require('../img/litellm_hosted_usage_dashboard.png')} />
<Image img={require('@site/img/litellm_hosted_usage_dashboard.png')} />
### 4. Configure load balancing
<Image img={require('../img/litellm_hosted_ui_router.png')} />
<Image img={require('@site/img/litellm_hosted_ui_router.png')} />

2
docs/my-website/docs/guides/code_interpreter.md
View file

@ -93,7 +93,7 @@ for file in files.data:
The LiteLLM Admin UI includes built-in Code Interpreter support.
<Image img={require('../../img/code_interp.png')} />
<Image img={require('@site/img/code_interp.png')} />
**Steps:**

4
docs/my-website/docs/load_test.md
View file

@ -39,7 +39,7 @@ model_list:
Set Users=100, Ramp Up Users=10, Host=Base URL of your LiteLLM Proxy
<Image img={require('../img/locust_load_test.png')} />
<Image img={require('@site/img/locust_load_test.png')} />
6. Expected Results
@ -48,5 +48,5 @@ model_list:
Avg → /health/readiness is `219ms`
<Image img={require('../img/litellm_load_test.png')} />
<Image img={require('@site/img/litellm_load_test.png')} />

6
docs/my-website/docs/load_test_advanced.md
View file

@ -85,7 +85,7 @@ litellm_settings:
6. Expected results
<Image img={require('../img/locust_load_test1.png')} />
<Image img={require('@site/img/locust_load_test1.png')} />
## Load test - Endpoints with Rate Limits
@ -149,7 +149,7 @@ litellm_settings:
Head to the locust UI on http://0.0.0.0:8089 and use the following settings
<Image img={require('../img/locust_load_test2_setup.png')} />
<Image img={require('@site/img/locust_load_test2_setup.png')} />
6. Expected results
- Successful responses in 1 minute = 19,800 = (69415 - 49615)
@ -157,7 +157,7 @@ litellm_settings:
- Median response time = 70ms
- Average response time = 640ms
<Image img={require('../img/locust_load_test2.png')} />
<Image img={require('@site/img/locust_load_test2.png')} />
## Prometheus Metrics for debugging load tests

10
docs/my-website/docs/mcp.md
View file

@ -7,7 +7,7 @@ import Image from '@theme/IdealImage';
LiteLLM Proxy provides an MCP Gateway that allows you to use a fixed endpoint for all MCP tools and control MCP access by Key, Team.
<Image
img={require('../img/mcp_2.png')}
img={require('@site/img/mcp_2.png')}
style={{width: '100%', display: 'block', margin: '2rem auto'}}
/>
<p style={{textAlign: 'left', color: '#666'}}>
@ -80,7 +80,7 @@ LiteLLM supports the following MCP transports:
- Standard Input/Output (stdio)
<Image
img={require('../img/add_mcp.png')}
img={require('@site/img/add_mcp.png')}
style={{width: '80%', display: 'block', margin: '0'}}
/>
@ -110,7 +110,7 @@ This video walks through adding and using an SSE MCP server on LiteLLM UI and us
For stdio MCP servers, select "Standard Input/Output (stdio)" as the transport type and provide the stdio configuration in JSON format:
<Image
img={require('../img/add_stdio_mcp.png')}
img={require('@site/img/add_stdio_mcp.png')}
style={{width: '80%', display: 'block', margin: '0'}}
/>
@ -124,7 +124,7 @@ LiteLLM attempts [OAuth 2.0 Authorization Server Discovery](https://datatracker.
**Customize the OAuth flow when needed:**
<Image
img={require('../img/mcp_oauth.png')}
img={require('@site/img/mcp_oauth.png')}
style={{width: '80%', display: 'block', margin: '0'}}
/>
@ -138,7 +138,7 @@ LiteLLM attempts [OAuth 2.0 Authorization Server Discovery](https://datatracker.
Sometimes your MCP server needs specific headers on every request. Maybe it's an API key, maybe it's a custom header the server expects. Instead of configuring auth, you can just set them directly.
<Image
img={require('../img/static_headers.png')}
img={require('@site/img/static_headers.png')}
style={{width: '80%', display: 'block', margin: '0'}}
/>

8
docs/my-website/docs/mcp_control.md
View file

@ -31,7 +31,7 @@ LiteLLM supports managing permissions for MCP Servers by Keys, Teams, Organizati
When Creating a Key, Team, or Organization, you can select the allowed MCP Servers that the entity has access to.
<Image
img={require('../img/mcp_key.png')}
img={require('@site/img/mcp_key.png')}
style={{width: '80%', display: 'block', margin: '0'}}
/>
@ -108,7 +108,7 @@ Some MCP servers are meant to be shared broadly—think internal knowledge bases
3. Toggle **Allow All LiteLLM Keys** on.
<Image
img={require('../img/mcp_allow_all_ui.png')}
img={require('@site/img/mcp_allow_all_ui.png')}
style={{width: '80%', display: 'block', margin: '1rem auto'}}
alt="MCP server configuration in Admin UI"
/>
@ -585,7 +585,7 @@ To create an access group:
- Add the same group name to other servers to group them together
<Image
img={require('../img/mcp_create_access_group.png')}
img={require('@site/img/mcp_create_access_group.png')}
style={{width: '80%', display: 'block', margin: '0'}}
/>
@ -620,7 +620,7 @@ When creating API keys, you can assign them to specific access groups for permis
- This is reflected in the Test Key page
<Image
img={require('../img/mcp_key_access_group.png')}
img={require('@site/img/mcp_key_access_group.png')}
style={{width: '80%', display: 'block', margin: '0'}}
/>

4
docs/my-website/docs/mcp_troubleshoot.md
View file

@ -14,7 +14,7 @@ Pin down where the failure occurs before adjusting settings so you do not mix sy
Failures shown on the MCP creation form or within the MCP Tool Testing Playground mean the LiteLLM proxy cannot reach the MCP server. Typical causes are misconfiguration (transport, headers, credentials), MCP/server outages, network/firewall blocks, or inaccessible OAuth metadata.
<Image
img={require('../img/mcp_tool_testing_playground.png')}
img={require('@site/img/mcp_tool_testing_playground.png')}
style={{width: '80%', display: 'block', margin: '0'}}
/>
@ -44,7 +44,7 @@ During `/responses` or `/chat/completions`, LiteLLM may trigger MCP tool calls m
- Reproduce the same MCP call via the LiteLLM Playground to confirm LiteLLM can complete the MCP hop independently.
<Image
img={require('../img/mcp_playground.png')}
img={require('@site/img/mcp_playground.png')}
style={{width: '80%', display: 'block', margin: '0'}}
/>

2
docs/my-website/docs/observability/argilla.md
View file

@ -95,7 +95,7 @@ litellm_settings:
## Example Output
<Image img={require('../../img/argilla.png')} />
<Image img={require('@site/img/argilla.png')} />
## Add sampling rate to Argilla calls

2
docs/my-website/docs/observability/arize_integration.md
View file

@ -7,7 +7,7 @@ import TabItem from '@theme/TabItem';
AI Observability and Evaluation Platform
<Image img={require('../../img/arize.png')} />
<Image img={require('@site/img/arize.png')} />

2
docs/my-website/docs/observability/athina_integration.md
View file

@ -13,7 +13,7 @@ https://github.com/BerriAI/litellm
[Athina](https://athina.ai/) is an evaluation framework and production monitoring platform for your LLM-powered app. Athina is designed to enhance the performance and reliability of AI applications through real-time monitoring, granular analytics, and plug-and-play evaluations.
<Image img={require('../../img/athina_dashboard.png')} />
<Image img={require('@site/img/athina_dashboard.png')} />
## Getting Started

4
docs/my-website/docs/observability/azure_sentinel.md
View file

@ -4,7 +4,7 @@ import TabItem from '@theme/TabItem';
# Azure Sentinel
<Image img={require('../../img/sentinel.png')} />
<Image img={require('@site/img/sentinel.png')} />
LiteLLM supports logging to Azure Sentinel via the Azure Monitor Logs Ingestion API. Azure Sentinel uses Log Analytics workspaces for data storage, so logs sent to the workspace will be available in Sentinel for security monitoring and analysis.
@ -108,7 +108,7 @@ LiteLLM_CL
You should see following logs in Azure Workspace.
<Image img={require('../../img/sentinel.png')} />
<Image img={require('@site/img/sentinel.png')} />
## Environment Variables

6
docs/my-website/docs/observability/datadog.md
View file

@ -117,7 +117,7 @@ curl --location 'http://0.0.0.0:4000/chat/completions' \
Expected output on Datadog
<Image img={require('../../img/dd_small1.png')} />
<Image img={require('@site/img/dd_small1.png')} />
### Redacting Messages and Responses
@ -161,11 +161,11 @@ curl --location 'http://0.0.0.0:4000/chat/completions' \
On the Datadog LLM Observability page, you should see that both input messages and output responses are redacted, while metadata (token counts, timing, model info) remains visible.
<Image img={require('../../img/dd_llm_obs.png')} />
<Image img={require('@site/img/dd_llm_obs.png')} />
<Image img={require('../../img/dd_llm_obs.png')} />
<Image img={require('@site/img/dd_llm_obs.png')} />
## Datadog Cloud Cost Management

2
docs/my-website/docs/observability/deepeval_integration.md
View file

@ -15,7 +15,7 @@ import Image from '@theme/IdealImage';
- Run evaluations to measure and improve performance
- Track costs and latency to optimize resource usage
<Image img={require('../../img/deepeval_dashboard.png')} />
<Image img={require('@site/img/deepeval_dashboard.png')} />
### Quickstart

2
docs/my-website/docs/observability/gcs_bucket_integration.md
View file

@ -59,7 +59,7 @@ curl --location 'http://0.0.0.0:4000/chat/completions' \
## Expected Logs on GCS Buckets
<Image img={require('../../img/gcs_bucket.png')} />
<Image img={require('@site/img/gcs_bucket.png')} />
### Fields Logged on GCS Buckets

4
docs/my-website/docs/observability/lago.md
View file

@ -6,7 +6,7 @@ import TabItem from '@theme/TabItem';
[Lago](https://www.getlago.com/) offers a self-hosted and cloud, metering and usage-based billing solution.
<Image img={require('../../img/lago.jpeg')} />
<Image img={require('@site/img/lago.jpeg')} />
## Quick Start
Use just 1 lines of code, to instantly log your responses **across all providers** with Lago
@ -150,7 +150,7 @@ print(response)
</Tabs>
<Image img={require('../../img/lago_2.png')} />
<Image img={require('@site/img/lago_2.png')} />
## Advanced - Lagos Logging object

2
docs/my-website/docs/observability/langfuse_integration.md
View file

@ -8,7 +8,7 @@ Langfuse ([GitHub](https://github.com/langfuse/langfuse)) is an open-source LLM
Example trace in Langfuse using multiple models via LiteLLM:
<Image img={require('../../img/langfuse-example-trace-multiple-models-min.png')} />
<Image img={require('@site/img/langfuse-example-trace-multiple-models-min.png')} />
:::info

2
docs/my-website/docs/observability/langfuse_otel_integration.md
View file

@ -7,7 +7,7 @@ import Image from '@theme/IdealImage';
The Langfuse OpenTelemetry integration allows you to send LiteLLM traces and observability data to Langfuse using the OpenTelemetry protocol. This provides a standardized way to collect and analyze your LLM usage data.
<Image img={require('../../img/langfuse_otel.png')} />
<Image img={require('@site/img/langfuse_otel.png')} />
## Features

2
docs/my-website/docs/observability/langsmith_integration.md
View file

@ -9,7 +9,7 @@ import TabItem from '@theme/TabItem';
An all-in-one developer platform for every step of the application lifecycle
https://smith.langchain.com/
<Image img={require('../../img/langsmith_new.png')} />
<Image img={require('@site/img/langsmith_new.png')} />
:::info
We want to learn how we can make the callbacks better! Meet the LiteLLM [founders](https://calendly.com/d/4mp-gd3-k5k/berriai-1-1-onboarding-litellm-hosted-version) or

4
docs/my-website/docs/observability/levo_integration.md
View file

@ -10,10 +10,10 @@ import TabItem from '@theme/TabItem';
<div className="levo-logo-container" style={{ marginTop: '0.5rem', marginBottom: '1rem' }}>
<div className="levo-logo-light">
<Image img={require('../../img/levo_logo.png')} />
<Image img={require('@site/img/levo_logo.png')} />
</div>
<div className="levo-logo-dark">
<Image img={require('../../img/levo_logo_dark.png')} />
<Image img={require('@site/img/levo_logo_dark.png')} />
</div>
</div>

2
docs/my-website/docs/observability/literalai_integration.md
View file

@ -4,7 +4,7 @@ import Image from '@theme/IdealImage';
[Literal AI](https://literalai.com) is a collaborative observability, evaluation and analytics platform for building production-grade LLM apps.
<Image img={require('../../img/literalai.png')} />
<Image img={require('@site/img/literalai.png')} />
## Pre-Requisites

2
docs/my-website/docs/observability/logfire_integration.md
View file

@ -5,7 +5,7 @@ import Image from '@theme/IdealImage';
Logfire is open Source Observability & Analytics for LLM Apps
Detailed production traces and a granular view on quality, cost and latency
<Image img={require('../../img/logfire.png')} />
<Image img={require('@site/img/logfire.png')} />
:::info
We want to learn how we can make the callbacks better! Meet the LiteLLM [founders](https://calendly.com/d/4mp-gd3-k5k/berriai-1-1-onboarding-litellm-hosted-version) or

2
docs/my-website/docs/observability/lunary_integration.md
View file

@ -118,7 +118,7 @@ def my_chain(chain_input):
my_chain("Chain input")
```
<Image img={require('../../img/lunary-trace.png')} />
<Image img={require('@site/img/lunary-trace.png')} />
## Usage with LiteLLM Proxy Server
### Step1: Install dependencies and set your environment variables

4
docs/my-website/docs/observability/mlflow.md
View file

@ -9,7 +9,7 @@ import Image from '@theme/IdealImage';
MLflow’s integration with LiteLLM supports advanced observability compatible with OpenTelemetry.
<Image img={require('../../img/mlflow_tracing.png')} />
<Image img={require('@site/img/mlflow_tracing.png')} />
## Getting Started
@ -102,7 +102,7 @@ response = litellm.completion(
)
```
<Image img={require('../../img/mlflow_tool_calling_tracing.png')} />
<Image img={require('@site/img/mlflow_tool_calling_tracing.png')} />
## Evaluation

4
docs/my-website/docs/observability/openmeter.md
View file

@ -6,7 +6,7 @@ import TabItem from '@theme/TabItem';
[OpenMeter](https://openmeter.io/) is an Open Source Usage-Based Billing solution for AI/Cloud applications. It integrates with Stripe for easy billing.
<Image img={require('../../img/openmeter.png')} />
<Image img={require('@site/img/openmeter.png')} />
:::info
We want to learn how we can make the callbacks better! Meet the LiteLLM [founders](https://calendly.com/d/4mp-gd3-k5k/berriai-1-1-onboarding-litellm-hosted-version) or
@ -94,4 +94,4 @@ curl --location 'http://0.0.0.0:4000/chat/completions' \
</Tabs>
<Image img={require('../../img/openmeter_img_2.png')} />
<Image img={require('@site/img/openmeter_img_2.png')} />

4
docs/my-website/docs/observability/opentelemetry_integration.md
View file

@ -6,7 +6,7 @@ import TabItem from '@theme/TabItem';
OpenTelemetry is a CNCF standard for observability. It connects to any observability tool, such as Jaeger, Zipkin, Datadog, New Relic, Traceloop, Levo AI and others.
<Image img={require('../../img/traceloop_dash.png')} />
<Image img={require('@site/img/traceloop_dash.png')} />
:::note Change in v1.81.0
@ -122,7 +122,7 @@ for successful + failed requests
click under `litellm_request` in the trace
<Image img={require('../../img/otel_debug_trace.png')} />
<Image img={require('@site/img/otel_debug_trace.png')} />
### Not seeing traces land on Integration

4
docs/my-website/docs/observability/opik_integration.md
View file

@ -6,7 +6,7 @@ import Image from '@theme/IdealImage';
Opik is an open source end-to-end [LLM Evaluation Platform](https://www.comet.com/site/products/opik/?utm_source=litelllm&utm_medium=docs&utm_content=intro_paragraph) that helps developers track their LLM prompts and responses during both development and production. Users can define and run evaluations to test their LLMs apps before deployment to check for hallucinations, accuracy, context retrevial, and more!
<Image img={require('../../img/opik.png')} />
<Image img={require('@site/img/opik.png')} />
:::info
We want to learn how we can make the callbacks better! Meet the LiteLLM [founders](https://calendly.com/d/4mp-gd3-k5k/berriai-1-1-onboarding-litellm-hosted-version) or
@ -235,7 +235,7 @@ When you create an API key in LiteLLM Proxy, you can attach Opik-specific metada
Go to 'Virtual Keys', click on your choosen api key and edit 'Settings'.
Now save the opik metadata as user api key metdata.
<Image img={require('../../img/opik_key_metadata.png')} />
<Image img={require('@site/img/opik_key_metadata.png')} />
**Step 2: Use the key - Opik metadata is automatically applied**

2
docs/my-website/docs/observability/promptlayer_integration.md
View file

@ -13,7 +13,7 @@ https://github.com/BerriAI/litellm
Promptlayer is a platform for prompt engineers. Log OpenAI requests. Search usage history. Track performance. Visually manage prompt templates.
<Image img={require('../../img/promptlayer.png')} />
<Image img={require('@site/img/promptlayer.png')} />
## Use Promptlayer to log requests across all LLM Providers (OpenAI, Azure, Anthropic, Cohere, Replicate, PaLM)

4
docs/my-website/docs/observability/raw_request_response.md
View file

@ -56,7 +56,7 @@ litellm_settings:
**Expected Log**
<Image img={require('../../img/raw_request_log.png')}/>
<Image img={require('@site/img/raw_request_log.png')}/>
## Return Raw Response Headers
@ -121,4 +121,4 @@ curl -X POST 'http://0.0.0.0:4000/chat/completions' \
**Expected Response**
<Image img={require('../../img/raw_response_headers.png')}/>
<Image img={require('@site/img/raw_response_headers.png')}/>

2
docs/my-website/docs/observability/sentry.md
View file

@ -17,7 +17,7 @@ Track exceptions for:
- litellm.acompletion() - async completion()
- Streaming completion() & acompletion() calls
<Image img={require('../../img/sentry.png')} />
<Image img={require('@site/img/sentry.png')} />
## Usage

2
docs/my-website/docs/observability/slack_integration.md
View file

@ -2,7 +2,7 @@ import Image from '@theme/IdealImage';
# Slack - Logging LLM Input/Output, Exceptions
<Image img={require('../../img/slack.png')} />
<Image img={require('@site/img/slack.png')} />
:::info
We want to learn how we can make the callbacks better! Meet the LiteLLM [founders](https://calendly.com/d/4mp-gd3-k5k/berriai-1-1-onboarding-litellm-hosted-version) or

2
docs/my-website/docs/observability/wandb_integration.md
View file

@ -13,7 +13,7 @@ https://github.com/BerriAI/litellm
Weights & Biases helps AI developers build better models faster https://wandb.ai
<Image img={require('../../img/wandb.png')} />
<Image img={require('@site/img/wandb.png')} />
:::info
We want to learn how we can make the callbacks better! Meet the LiteLLM [founders](https://calendly.com/d/4mp-gd3-k5k/berriai-1-1-onboarding-litellm-hosted-version) or

2
docs/my-website/docs/providers/bedrock_vector_store.md
View file

@ -78,7 +78,7 @@ vector_store_registry:
On the LiteLLM UI, Navigate to Experimental > Vector Stores > Create Vector Store. On this page you can create a vector store with a name, vector store id and credentials.
<Image
img={require('../../img/kb_2.png')}
img={require('@site/img/kb_2.png')}
style={{width: '50%'}}
/>

2
docs/my-website/docs/providers/gemini.md
View file

@ -2091,7 +2091,7 @@ You can now specify a custom Time-To-Live (TTL) for your cached content using th
### Architecture Diagram
<Image img={require('../../img/gemini_context_caching.png')} />
<Image img={require('@site/img/gemini_context_caching.png')} />
**Notes:**

2
docs/my-website/docs/providers/huggingface.md
View file

@ -14,7 +14,7 @@ LiteLLM supports running inference across multiple services for models hosted on
### Serverless Inference Providers
You can check available models for an inference provider by going to [huggingface.co/models](https://huggingface.co/models), clicking the "Other" filter tab, and selecting your desired provider:
![Filter models by Inference Provider](../../img/hf_filter_inference_providers.png)
![Filter models by Inference Provider](@site/img/hf_filter_inference_providers.png)
For example, you can find all Fireworks supported models [here](https://huggingface.co/models?inference_provider=fireworks-ai&sort=trending).

2
docs/my-website/docs/providers/ragflow_vector_store.md
View file

@ -74,7 +74,7 @@ vector_store_registry:
On the LiteLLM UI, Navigate to Experimental > Vector Stores > Create Vector Store. On this page you can create a vector store with a name, vector store id and credentials.
<Image
img={require('../../img/kb_2.png')}
img={require('@site/img/kb_2.png')}
style={{width: '50%'}}
/>

6
docs/my-website/docs/providers/vertex.md
View file

@ -3115,7 +3115,7 @@ Trying to deploy LiteLLM on Google Cloud Run? Tutorial [here](https://docs.litel
1. Figure out the Service Account bound to the Google Cloud Run service
<Image img={require('../../img/gcp_acc_1.png')} />
<Image img={require('@site/img/gcp_acc_1.png')} />
2. Get the FULL EMAIL address of the corresponding Service Account
@ -3123,11 +3123,11 @@ Trying to deploy LiteLLM on Google Cloud Run? Tutorial [here](https://docs.litel
Click `Add Principal`
<Image img={require('../../img/gcp_acc_2.png')}/>
<Image img={require('@site/img/gcp_acc_2.png')}/>
4. Specify the Service Account as the principal and Vertex AI User as the role
<Image img={require('../../img/gcp_acc_3.png')}/>
<Image img={require('@site/img/gcp_acc_3.png')}/>
Once that's done, when you deploy the new container in the Google Cloud Run service, LiteLLM will have automatic access to all Vertex AI endpoints.

2
docs/my-website/docs/proxy/access_control.md
View file

@ -4,7 +4,7 @@ import Image from '@theme/IdealImage';
Role-based access control (RBAC) is based on Organizations, Teams and Internal User Roles
<Image img={require('../../img/litellm_user_heirarchy.png')} style={{ width: '100%', maxWidth: '4000px' }} />
<Image img={require('@site/img/litellm_user_heirarchy.png')} style={{ width: '100%', maxWidth: '4000px' }} />
- `Organizations` are the top-level entities that contain Teams.

26
docs/my-website/docs/proxy/admin_ui_sso.md
View file

@ -42,7 +42,7 @@ You can get your domain specific auth/token/userinfo endpoints at `<YOUR-OKTA-DO
On Okta, add the 'callback_url' as `<proxy_base_url>/sso/callback`
<Image img={require('../../img/okta_callback_url.png')} />
<Image img={require('@site/img/okta_callback_url.png')} />
</TabItem>
<TabItem value="google" label="Google SSO">
@ -220,7 +220,7 @@ PROXY_BASE_URL=https://litellm-api.up.railway.app
```
#### Step 4. Test flow
<Image img={require('../../img/litellm_ui_3.gif')} />
<Image img={require('@site/img/litellm_ui_3.gif')} />
### Restrict Email Subdomains w/ SSO
@ -238,7 +238,7 @@ Set a Proxy Admin when SSO is enabled. Once SSO is enabled, the `user_id` for us
#### Step 1: Copy your ID from the UI
<Image img={require('../../img/litellm_ui_copy_id.png')} />
<Image img={require('@site/img/litellm_ui_copy_id.png')} />
#### Step 2: Set it in your .env as the PROXY_ADMIN_ID
@ -252,7 +252,7 @@ If you plan to change this ID, please update the user role via API `/user/update
#### Step 3: See all proxy keys
<Image img={require('../../img/litellm_ui_admin.png')} />
<Image img={require('@site/img/litellm_ui_admin.png')} />
:::info
@ -292,7 +292,7 @@ general_settings:
**Step 2. Invite view-only users**
<Image img={require('../../img/admin_ui_viewer.png')} />
<Image img={require('@site/img/admin_ui_viewer.png')} />
### Custom Branding Admin UI
@ -300,7 +300,7 @@ Use your companies custom branding on the LiteLLM Admin UI
We allow you to
- Customize the UI Logo
- Customize the UI color scheme
<Image img={require('../../img/litellm_custom_ai.png')} />
<Image img={require('@site/img/litellm_custom_ai.png')} />
#### Set Custom Logo
We allow you to pass a local image or a an http/https url of your image
@ -319,8 +319,8 @@ UI_LOGO_PATH="ui_images/logo.jpg"
#### Or set your logo directly from Admin UI:
<div style={{ display: 'flex', gap: '12px', alignItems: 'center' }}>
<Image img={require('../../img/admin_settings_ui_theme.png')} />
<Image img={require('../../img/admin_settings_ui_theme_logo.png')} />
<Image img={require('@site/img/admin_settings_ui_theme.png')} />
<Image img={require('@site/img/admin_settings_ui_theme_logo.png')} />
</div>
#### Set Custom Color Theme
@ -405,7 +405,7 @@ PROXY_BASE_URL=https://mydomain.com
If you need to access the UI via username/password when SSO is on navigate to `/fallback/login`. This route will allow you to sign in with your username/password credentials.
<Image img={require('../../img/fallback_login.png')} />
<Image img={require('@site/img/fallback_login.png')} />
### Debugging SSO JWT fields
@ -413,7 +413,7 @@ If you need to access the UI via username/password when SSO is on navigate to `/
If you need to inspect the JWT fields received from your SSO provider by LiteLLM, follow these instructions. This guide walks you through setting up a debug callback to view the JWT data during the SSO process.
<Image img={require('../../img/debug_sso.png')} style={{ width: '500px', height: 'auto' }} />
<Image img={require('@site/img/debug_sso.png')} style={{ width: '500px', height: 'auto' }} />
<br />
1. Add `/sso/debug/callback` as a redirect URL in your SSO provider
@ -464,7 +464,7 @@ Centralize role management by defining user permissions in Azure Entra ID. LiteL
- `internal_user` - Can create/view/delete own keys
- `internal_user_viewer` - Can view own keys (read-only)
<Image img={require('../../img/app_roles.png')} style={{ width: '900px', height: 'auto' }} />
<Image img={require('@site/img/app_roles.png')} style={{ width: '900px', height: 'auto' }} />
---
@ -477,7 +477,7 @@ Centralize role management by defining user permissions in Azure Entra ID. LiteL
5. Under **Select a role**, choose the app role you created (e.g., `proxy_admin_viewer`)
6. Click **Assign** to save
<Image img={require('../../img/app_role2.png')} style={{ width: '900px', height: 'auto' }} />
<Image img={require('@site/img/app_role2.png')} style={{ width: '900px', height: 'auto' }} />
---
@ -487,7 +487,7 @@ Centralize role management by defining user permissions in Azure Entra ID. LiteL
2. LiteLLM will automatically extract the app role from the JWT token
3. The user will be assigned the corresponding role (you can verify this in the UI by checking the user profile dropdown)
<Image img={require('../../img/app_role3.png')} style={{ width: '900px', height: 'auto' }} />
<Image img={require('@site/img/app_role3.png')} style={{ width: '900px', height: 'auto' }} />
**Note:** The role from Entra ID will take precedence over any existing role in the LiteLLM database. This ensures your SSO provider is the authoritative source for user roles.

22
docs/my-website/docs/proxy/ai_hub.md
View file

@ -12,7 +12,7 @@ This feature is **available in v1.74.3-stable and above**.
Admin can select models/agents to expose on public AI hub → Users go to the public url and see what's available.
<Image img={require('../../img/final_public_model_hub_view.png')} />
<Image img={require('@site/img/final_public_model_hub_view.png')} />
## Models
@ -22,23 +22,23 @@ Admin can select models/agents to expose on public AI hub → Users go to the pu
Navigate to the Model Hub page in the Admin UI (`PROXY_BASE_URL/ui/?login=success&page=model-hub-table`)
<Image img={require('../../img/model_hub_admin_view.png')} />
<Image img={require('@site/img/model_hub_admin_view.png')} />
#### 2. Select the models you want to expose
Click on `Select Models to Make Public` and select the models you want to expose.
<Image img={require('../../img/make_public_modal.png')} />
<Image img={require('@site/img/make_public_modal.png')} />
#### 3. Confirm the changes
<Image img={require('../../img/make_public_modal_confirmation.png')} />
<Image img={require('@site/img/make_public_modal_confirmation.png')} />
#### 4. Success!
Go to the public url (`PROXY_BASE_URL/ui/model_hub_table`) and see available models.
<Image img={require('../../img/final_public_model_hub_view.png')} />
<Image img={require('@site/img/final_public_model_hub_view.png')} />
### API Endpoints
@ -62,7 +62,7 @@ Create an agent that follows the [A2A spec](https://a2a.dev/).
<Tabs>
<TabItem value="ui" label="UI">
<Image img={require('../../img/add_agent.png')} />
<Image img={require('@site/img/add_agent.png')} />
</TabItem>
<TabItem value="api" label="API">
@ -140,11 +140,11 @@ Make the agent discoverable on the AI Hub.
Navigate to the Agents Tab on the AI Hub page
<Image img={require('../../img/ai_hub_with_agents.png')} />
<Image img={require('@site/img/ai_hub_with_agents.png')} />
Select the agents you want to make public and click on `Make Public` button.
<Image img={require('../../img/make_agents_public.png')} />
<Image img={require('@site/img/make_agents_public.png')} />
</TabItem>
<TabItem value="api" label="API">
@ -197,7 +197,7 @@ Users can now discover the agent via the public endpoint.
<Tabs>
<TabItem value="ui" label="UI">
<Image img={require('../../img/public_agent_hub.png')} />
<Image img={require('@site/img/public_agent_hub.png')} />
</TabItem>
<TabItem value="api" label="API">
@ -255,7 +255,7 @@ Go here for instructions: [MCP Overview](../mcp#adding-your-mcp)
Navigate to AI Hub page, and select the MCP tab (`PROXY_BASE_URL/ui/?login=success&page=mcp-server-table`)
<Image img={require('../../img/mcp_server_on_ai_hub.png')} />
<Image img={require('@site/img/mcp_server_on_ai_hub.png')} />
</TabItem>
<TabItem value="api" label="API">
@ -278,7 +278,7 @@ Users can now discover the MCP server via the public endpoint (`PROXY_BASE_URL/u
<Tabs>
<TabItem value="ui" label="UI">
<Image img={require('../../img/mcp_on_public_ai_hub.png')} />
<Image img={require('@site/img/mcp_on_public_ai_hub.png')} />
</TabItem>
<TabItem value="api" label="API">

6
docs/my-website/docs/proxy/alerting.md
View file

@ -130,7 +130,7 @@ curl http://0.0.0.0:4000/chat/completions \
Step 3. Check slack for Expected Alert
<Image img={require('../../img/soft_budget_alert.png')}/>
<Image img={require('@site/img/soft_budget_alert.png')}/>
@ -162,7 +162,7 @@ response = client.chat.completions.create(
**Expected Response**
<Image img={require('../../img/alerting_metadata.png')}/>
<Image img={require('@site/img/alerting_metadata.png')}/>
### Select specific alert types
@ -324,7 +324,7 @@ curl --location 'http://0.0.0.0:4000/health/services?service=slack' \
**Expected Response**
<Image img={require('../../img/ms_teams_alerting.png')}/>
<Image img={require('@site/img/ms_teams_alerting.png')}/>
### Discord Webhooks

2
docs/my-website/docs/proxy/architecture.md
View file

@ -6,7 +6,7 @@ import TabItem from '@theme/TabItem';
## High Level architecture
<Image img={require('../../img/litellm_gateway.png')} style={{ width: '100%', maxWidth: '4000px' }} />
<Image img={require('@site/img/litellm_gateway.png')} style={{ width: '100%', maxWidth: '4000px' }} />
### Request Flow

4
docs/my-website/docs/proxy/auto_routing.md
View file

@ -6,7 +6,7 @@ import TabItem from '@theme/TabItem';
LiteLLM can auto select the best model for a request based on rules you define.
<Image alt="Auto Routing" img={require('../../img/auto_router.png')} style={{ borderRadius: '8px', marginBottom: '1em', maxWidth: '100%' }} />
<Image alt="Auto Routing" img={require('@site/img/auto_router.png')} style={{ borderRadius: '8px', marginBottom: '1em', maxWidth: '100%' }} />
## LiteLLM Python SDK
@ -144,7 +144,7 @@ Configure the following required fields:
#### Route Configuration
<Image alt="Auto Router Setup" img={require('../../img/auto_router2.png')} style={{ borderRadius: '8px', marginBottom: '1em', maxWidth: '100%' }} />
<Image alt="Auto Router Setup" img={require('@site/img/auto_router2.png')} style={{ borderRadius: '8px', marginBottom: '1em', maxWidth: '100%' }} />
<br />

2
docs/my-website/docs/proxy/billing.md
View file

@ -149,7 +149,7 @@ print(response)
**See Results on Lago**
<Image img={require('../../img/lago_2.png')} style={{ width: '500px', height: 'auto' }} />
<Image img={require('@site/img/lago_2.png')} style={{ width: '500px', height: 'auto' }} />
## Advanced - Lago Logging object

2
docs/my-website/docs/proxy/call_hooks.md
View file

@ -257,7 +257,7 @@ general_settings:
**Result**
<Image img={require('../../img/end_user_enforcement.png')}/>
<Image img={require('@site/img/end_user_enforcement.png')}/>
## Advanced - Return rejected message as response

2
docs/my-website/docs/proxy/control_plane_and_data_plane.md
View file

@ -27,7 +27,7 @@ When scaling LiteLLM for production use, you may want to deploy multiple instanc
### Typical Deployment Scenario
<Image img={require('../../img/scaling_architecture.png')} />
<Image img={require('@site/img/scaling_architecture.png')} />
### Benefits of This Architecture

6
docs/my-website/docs/proxy/cost_tracking.md
View file

@ -124,7 +124,7 @@ That's IT. Now Verify your spend was tracked
Expect to see `x-litellm-response-cost` in the response headers with calculated cost
<Image img={require('../../img/response_cost_img.png')} />
<Image img={require('@site/img/response_cost_img.png')} />
</TabItem>
<TabItem value="db" label="DB + UI">
@ -150,7 +150,7 @@ The following spend gets tracked in Table `LiteLLM_SpendLogs`
Navigate to the Usage Tab on the LiteLLM UI (found on https://your-proxy-endpoint/ui) and verify you see spend tracked under `Usage`
<Image img={require('../../img/admin_ui_spend.png')} />
<Image img={require('@site/img/admin_ui_spend.png')} />
</TabItem>
</Tabs>
@ -332,7 +332,7 @@ Requirements:
**Note:** By default, LiteLLM will track `User-Agent` as a custom tag for cost tracking. This enables viewing usage for tools like Claude Code, Gemini CLI, etc.
<Image img={require('../../img/claude_cli_tag_usage.png')} />
<Image img={require('@site/img/claude_cli_tag_usage.png')} />
### Client-side spend tag

2
docs/my-website/docs/proxy/custom_pricing.md
View file

@ -48,7 +48,7 @@ litellm /path/to/config.yaml
**Step 3: View Spend Logs**
<Image img={require('../../img/spend_logs_table.png')} />
<Image img={require('@site/img/spend_logs_table.png')} />
## Cost Per Token (e.g. Azure)

2
docs/my-website/docs/proxy/custom_prompt_management.md
View file

@ -10,7 +10,7 @@ Connect LiteLLM to your prompt management system with custom hooks.
<Image
img={require('../../img/custom_prompt_management.png')}
img={require('@site/img/custom_prompt_management.png')}
style={{width: '100%', display: 'block', margin: '2rem auto'}}
/>

2
docs/my-website/docs/proxy/custom_root_ui.md
View file

@ -35,7 +35,7 @@ After running the proxy you can access it on `http://0.0.0.0:4000/api/v1/` (sinc
### 3. Verify Running on correct path
<Image img={require('../../img/custom_root_path.png')} />
<Image img={require('@site/img/custom_root_path.png')} />
**That's it**, that's all you need to run the proxy on a custom root path

8
docs/my-website/docs/proxy/customer_usage.md
View file

@ -18,7 +18,7 @@ Customer Usage enables you to track spend and usage for individual customers (en
- Set budgets and rate limits per customer
- Monitor customer usage patterns and trends
<Image img={require('../../img/customer_usage.png')} />
<Image img={require('@site/img/customer_usage.png')} />
## How to Track Spend
@ -105,7 +105,7 @@ Navigate to the Customer Usage tab in the Admin UI to view customer-level spend
Go to the Usage page in the Admin UI (`PROXY_BASE_URL/ui/?login=success&page=new_usage`) and click on the **Customer Usage** tab.
<Image img={require('../../img/customer_usage_ui_navigation.png')} />
<Image img={require('@site/img/customer_usage_ui_navigation.png')} />
#### 2. View Customer Analytics
@ -116,7 +116,7 @@ The Customer Usage dashboard provides:
- **Model usage breakdown**: Understand which models each customer uses
- **Activity metrics**: Track requests, tokens, and success rates per customer
<Image img={require('../../img/customer_usage_analytics.png')} />
<Image img={require('@site/img/customer_usage_analytics.png')} />
#### 3. Filter by Customer
@ -126,7 +126,7 @@ Use the customer filter dropdown to view spend for specific customers:
- View filtered analytics, spend logs, and activity metrics
- Compare spend across different customers
<Image img={require('../../img/customer_usage_filter.png')} />
<Image img={require('@site/img/customer_usage_filter.png')} />
## Use Cases

2
docs/my-website/docs/proxy/customers.md
View file

@ -212,7 +212,7 @@ Create and assign customers to pricing tiers.
- Click on '+ Create Budget'.
- Create your pricing tier (e.g. 'my-free-tier' with budget $4). This means each user on this pricing tier will have a max budget of $4.
<Image img={require('../../img/create_budget_modal.png')} />
<Image img={require('@site/img/create_budget_modal.png')} />
</TabItem>
<TabItem value="api" label="API">

4
docs/my-website/docs/proxy/db_deadlocks.md
View file

@ -27,7 +27,7 @@ LiteLLM writes `UPDATE` and `UPSERT` queries to the DB. When using 10+ instances
Each instance will accumulate the spend updates for a key, user, team, etc and write the updates to a redis queue.
<Image img={require('../../img/deadlock_fix_1.png')} style={{ width: '900px', height: 'auto' }} />
<Image img={require('@site/img/deadlock_fix_1.png')} style={{ width: '900px', height: 'auto' }} />
<p style={{textAlign: 'left', color: '#666'}}>
Each instance writes updates to redis
</p>
@ -47,7 +47,7 @@ A single instance will acquire a lock on the DB and flush all elements in the re
- Note: Only 1 instance can acquire the lock at a time, this limits the number of instances that can write to the DB at once
<Image img={require('../../img/deadlock_fix_2.png')} style={{ width: '900px', height: 'auto' }} />
<Image img={require('@site/img/deadlock_fix_2.png')} style={{ width: '900px', height: 'auto' }} />
<p style={{textAlign: 'left', color: '#666'}}>
A single instance flushes the redis queue to the DB
</p>

2
docs/my-website/docs/proxy/deleted_keys_teams.md
View file

@ -2,7 +2,7 @@ import Image from '@theme/IdealImage';
# Deleted Keys & Teams Audit Logs
<Image img={require('../../img/ui_deleted_keys_table.png')} />
<Image img={require('@site/img/ui_deleted_keys_table.png')} />
View deleted API keys and teams along with their spend and budget information at the time of deletion for auditing and compliance purposes.

16
docs/my-website/docs/proxy/email.md
View file

@ -5,7 +5,7 @@ import TabItem from '@theme/TabItem';
# Email Notifications
<Image
img={require('../../img/email_2_0.png')}
img={require('@site/img/email_2_0.png')}
style={{width: '70%', display: 'block', margin: '0 0 2rem 0'}}
/>
<p style={{textAlign: 'left', color: '#666'}}>
@ -131,7 +131,7 @@ EMAIL_BUDGET_ALERT_TTL=86400
This email is send when you create a new user on LiteLLM Proxy.
<Image
img={require('../../img/email_event_1.png')}
img={require('@site/img/email_event_1.png')}
style={{width: '70%', display: 'block', margin: '0 0 2rem 0'}}
/>
@ -140,7 +140,7 @@ This email is send when you create a new user on LiteLLM Proxy.
On the LiteLLM Proxy UI, go to Users > Create User > Enter the user's email address > Create User.
<Image
img={require('../../img/new_user_email.png')}
img={require('@site/img/new_user_email.png')}
style={{width: '70%', display: 'block', margin: '0 0 2rem 0'}}
/>
@ -149,7 +149,7 @@ On the LiteLLM Proxy UI, go to Users > Create User > Enter the user's email addr
This email is sent when you create a new API key for a user on LiteLLM Proxy.
<Image
img={require('../../img/email_event_2.png')}
img={require('@site/img/email_event_2.png')}
style={{width: '70%', display: 'block', margin: '0 0 2rem 0'}}
/>
@ -158,14 +158,14 @@ This email is sent when you create a new API key for a user on LiteLLM Proxy.
On the LiteLLM Proxy UI, go to Virtual Keys > Create API Key > Select User ID
<Image
img={require('../../img/key_email.png')}
img={require('@site/img/key_email.png')}
style={{width: '70%', display: 'block', margin: '0 0 2rem 0'}}
/>
On the Create Key Modal, Select Advanced Settings > Set Send Email to True.
<Image
img={require('../../img/key_email_2.png')}
img={require('@site/img/key_email_2.png')}
style={{width: '70%', display: 'block', margin: '0 0 2rem 0'}}
/>
@ -174,7 +174,7 @@ On the Create Key Modal, Select Advanced Settings > Set Send Email to True.
This email is sent when you rotate an API key for a user on LiteLLM Proxy.
<Image
img={require('../../img/email_regen2.png')}
img={require('@site/img/email_regen2.png')}
style={{maxHeight: '600px', width: 'auto', display: 'block', margin: '0 0 2rem 0'}}
/>
@ -189,7 +189,7 @@ Ensure there is a `user_id` attached to the key. This would have been set when c
:::
<Image
img={require('../../img/email_regen.png')}
img={require('@site/img/email_regen.png')}
style={{width: '70%', display: 'block', margin: '0 0 2rem 0'}}
/>

2
docs/my-website/docs/proxy/endpoint_activity.md
View file

@ -17,7 +17,7 @@ Endpoint Activity enables you to track spend and usage for individual API endpoi
- Identify which endpoints are getting the most activity
- View trend data showing endpoint usage over time
<Image img={require('../../img/ui_endpoint_activity.png')} />
<Image img={require('@site/img/ui_endpoint_activity.png')} />
## How Endpoint Activity Works

4
docs/my-website/docs/proxy/enterprise.md
View file

@ -638,7 +638,7 @@ In your environment, set:
DOCS_FILTERED="True" # only shows openai routes to user
```
<Image img={require('../../img/custom_swagger.png')} style={{ width: '900px', height: 'auto' }} />
<Image img={require('@site/img/custom_swagger.png')} style={{ width: '900px', height: 'auto' }} />
## Enable Blocked User Lists
@ -765,7 +765,7 @@ Share a public page of available models and agents for users
[Learn more](./ai_hub.md)
<Image img={require('../../img/model_hub.png')} style={{ width: '900px', height: 'auto' }}/>
<Image img={require('@site/img/model_hub.png')} style={{ width: '900px', height: 'auto' }}/>
## [BETA] AWS Key Manager - Key Decryption

6
docs/my-website/docs/proxy/guardrails/aporia_api.md
View file

@ -15,20 +15,20 @@ Create two projects on [Aporia](https://guardrails.aporia.com/)
1. Pre LLM API Call - Set all the policies you want to run on pre LLM API call
2. Post LLM API Call - Set all the policies you want to run post LLM API call
<Image img={require('../../../img/aporia_projs.png')} />
<Image img={require('@site/img/aporia_projs.png')} />
### Pre-Call: Detect PII
Add the `PII - Prompt` to your Pre LLM API Call project
<Image img={require('../../../img/aporia_pre.png')} />
<Image img={require('@site/img/aporia_pre.png')} />
### Post-Call: Detect Profanity in Responses
Add the `Toxicity - Response` to your Post LLM API Call project
<Image img={require('../../../img/aporia_post.png')} />
<Image img={require('@site/img/aporia_post.png')} />
## 2. Define Guardrails on your LiteLLM config.yaml

8
docs/my-website/docs/proxy/guardrails/litellm_content_filter.md
View file

@ -28,7 +28,7 @@ import Image from '@theme/IdealImage';
Click "Add New Guardrail" and select "LiteLLM Content Filter" as your guardrail provider.
<Image img={require('../../../img/create_guard.gif')} alt="Select LiteLLM Content Filter" />
<Image img={require('@site/img/create_guard.gif')} alt="Select LiteLLM Content Filter" />
### Step 2: Configure Pattern Detection
@ -36,13 +36,13 @@ Select the prebuilt entities you want to block or mask. In this example, we sele
If you need to block a custom entity, you can add a custom regex pattern by clicking "Add custom regex".
<Image img={require('../../../img/add_Guard2.gif')} alt="Select prebuilt entities or add custom regex" />
<Image img={require('@site/img/add_Guard2.gif')} alt="Select prebuilt entities or add custom regex" />
### Step 3: Add Blocked Keywords
Enter specific keywords you want to block. This is useful if you have policies to block certain words or phrases.
<Image img={require('../../../img/create_guard3.gif')} alt="Add blocked keywords" />
<Image img={require('@site/img/create_guard3.gif')} alt="Add blocked keywords" />
### Step 4: Test Your Guardrail
@ -52,7 +52,7 @@ Test examples:
- **Blocked keyword test**: Entering "hi blue" will trigger the block since we set "blue" as a blocked keyword
- **Pattern detection test**: Entering "Hi ishaan@berri.ai" will trigger the email pattern detector
<Image img={require('../../../img/add_guard5.gif')} alt="Test guardrail in playground" />
<Image img={require('@site/img/add_guard5.gif')} alt="Test guardrail in playground" />
## LiteLLM Config.yaml Setup

10
docs/my-website/docs/proxy/guardrails/pii_masking_v2.md
View file

@ -33,7 +33,7 @@ For this guardrail you need a deployed Presidio Analyzer and Presido Anonymizer
On the LiteLLM UI, navigate to Guardrails. Click "Add Guardrail". On this dropdown select "Presidio PII" and enter your presidio analyzer and anonymizer endpoints.
<Image
img={require('../../../img/presidio_1.png')}
img={require('@site/img/presidio_1.png')}
style={{width: '80%', display: 'block', margin: '0'}}
/>
@ -45,7 +45,7 @@ On the LiteLLM UI, navigate to Guardrails. Click "Add Guardrail". On this dropdo
Now select the entity types you want to mask. See the [supported actions here](#supported-actions)
<Image
img={require('../../../img/presidio_2.png')}
img={require('@site/img/presidio_2.png')}
style={{width: '50%', display: 'block', margin: '0'}}
/>
@ -117,7 +117,7 @@ My credit card is 4111-1111-1111-1111 and my email is test@example.com.
```
<Image
img={require('../../../img/presidio_3.png')}
img={require('@site/img/presidio_3.png')}
style={{width: '100%', display: 'block', margin: '0'}}
/>
@ -207,7 +207,7 @@ Once your guardrail is live in production, you will also be able to trace your g
On the LiteLLM logs page you can see that the PII content was masked for this specific request. And you can see detailed tracing for the guardrail. This allows you to monitor entity types masked with their corresponding confidence score and the duration of the guardrail execution.
<Image
img={require('../../../img/presidio_4.png')}
img={require('@site/img/presidio_4.png')}
style={{width: '60%', display: 'block', margin: '0'}}
/>
@ -216,7 +216,7 @@ On the LiteLLM logs page you can see that the PII content was masked for this sp
When connecting Litellm to Langfuse, you can see the guardrail information on the Langfuse Trace.
<Image
img={require('../../../img/presidio_5.png')}
img={require('@site/img/presidio_5.png')}
style={{width: '60%', display: 'block', margin: '0'}}
/>

4
docs/my-website/docs/proxy/guardrails/quick_start.md
View file

@ -419,11 +419,11 @@ Monitor which guardrails were executed and whether they passed or failed. e.g. g
#### Traced Guardrail Success
<Image img={require('../../../img/gd_success.png')} />
<Image img={require('@site/img/gd_success.png')} />
#### Traced Guardrail Failure
<Image img={require('../../../img/gd_fail.png')} />
<Image img={require('@site/img/gd_fail.png')} />

2
docs/my-website/docs/proxy/guardrails/test_playground.md
View file

@ -4,7 +4,7 @@ import Image from '@theme/IdealImage';
Test and compare multiple guardrails in real-time with an interactive playground interface.
<Image img={require('../../../img/guardrail_playground.png')} alt="Guardrail Test Playground" />
<Image img={require('@site/img/guardrail_playground.png')} alt="Guardrail Test Playground" />
## How to Use the Guardrail Testing Playground

2
docs/my-website/docs/proxy/image_handling.md
View file

@ -4,7 +4,7 @@ import TabItem from '@theme/TabItem';
# Image URL Handling
<Image img={require('../../img/image_handling.png')} style={{ width: '900px', height: 'auto' }} />
<Image img={require('@site/img/image_handling.png')} style={{ width: '900px', height: 'auto' }} />
Some LLM API's don't support url's for images, but do support base-64 strings.

2
docs/my-website/docs/proxy/jwt_auth_arch.md
View file

@ -14,7 +14,7 @@ import TabItem from '@theme/TabItem';
:::
<Image img={require('../../img/control_model_access_jwt.png')} style={{ width: '100%', maxWidth: '4000px' }} />
<Image img={require('@site/img/control_model_access_jwt.png')} style={{ width: '100%', maxWidth: '4000px' }} />
## Example Token

2
docs/my-website/docs/proxy/keys_teams_router_settings.md
View file

@ -16,7 +16,7 @@ With key-level and team-level router settings, you can now:
- **Apply different reliability settings** (cooldowns, allowed failures) per key or team
- **Override global settings** when needed for specific use cases
<Image img={require('../../img/ui_granular_router_settings.png')} />
<Image img={require('@site/img/ui_granular_router_settings.png')} />
## Summary

2
docs/my-website/docs/proxy/litellm_managed_files.md
View file

@ -419,7 +419,7 @@ No, as of `v1.71.2` users can only view/edit/delete files they have created.
<Image img={require('../../img/managed_files_arch.png')} style={{ width: '800px', height: 'auto' }} />
<Image img={require('@site/img/managed_files_arch.png')} style={{ width: '800px', height: 'auto' }} />
## See Also

18
docs/my-website/docs/proxy/litellm_prompt_management.md
View file

@ -18,7 +18,7 @@ Use the LiteLLM AI Gateway to create, manage and version your prompts.
- **Type**: Prompt type (e.g., db)
- **Actions**: Delete and manage prompt options (admin only)
![Prompt Table](../../img/prompt_table.png)
![Prompt Table](@site/img/prompt_table.png)
## Create a Prompt
@ -40,7 +40,7 @@ Respond as jack sparrow would
This will instruct the model to respond in the style of Captain Jack Sparrow from Pirates of the Caribbean.
![Add Prompt with Developer Message](../../img/add_prompt.png)
![Add Prompt with Developer Message](@site/img/add_prompt.png)
### Step 3: Add Prompt Messages
@ -58,7 +58,7 @@ Give me a recipe for {{dish}}
The UI will automatically detect variables in your prompt and display them in the **Detected variables** section.
![Add Prompt with Variables](../../img/add_prompt_var.png)
![Add Prompt with Variables](@site/img/add_prompt_var.png)
### Step 5: Test Your Prompt
@ -68,11 +68,11 @@ Before saving, you can test your prompt directly in the UI:
2. Type a message in the chat interface to test the prompt
3. The assistant will respond using your configured model, developer message, and substituted variables
![Test Prompt with Variables](../../img/add_prompt_use_var1.png)
![Test Prompt with Variables](@site/img/add_prompt_use_var1.png)
The result will show the model's response with your variables substituted:
![Prompt Test Results](../../img/add_prompt_use_var.png)
![Prompt Test Results](@site/img/add_prompt_use_var.png)
### Step 6: Save Your Prompt
@ -309,7 +309,7 @@ Click on any prompt ID in the prompts table to view its details page. This page
- **Last Updated**: Timestamp of the most recent update
- **LiteLLM Parameters**: The raw JSON configuration
![Prompt Details](../../img/edit_prompt.png)
![Prompt Details](@site/img/edit_prompt.png)
### Update a Prompt
@ -325,7 +325,7 @@ To update an existing prompt:
4. Test your changes in the chat interface on the right
5. Click the **Update** button to save the new version
![Edit Prompt in Studio](../../img/edit_prompt2.png)
![Edit Prompt in Studio](@site/img/edit_prompt2.png)
Each time you click **Update**, a new version is created (v1 → v2 → v3, etc.) while maintaining the same prompt ID.
@ -337,7 +337,7 @@ To view all versions of a prompt:
2. Click the **History** button in the top right
3. A **Version History** panel will open on the right side
![Version History Panel](../../img/edit_prompt3.png)
![Version History Panel](@site/img/edit_prompt3.png)
The version history panel displays:
- **Latest version** (marked with a "Latest" badge and "Active" status)
@ -357,7 +357,7 @@ To view or restore an older version:
- The model and parameters used
- All variables defined at that time
![View Older Version](../../img/edit_prompt4.png)
![View Older Version](@site/img/edit_prompt4.png)
The selected version will be highlighted with an "Active" badge in the version history panel.

24
docs/my-website/docs/proxy/logging.md
View file

@ -146,11 +146,11 @@ curl -L -X POST 'http://0.0.0.0:4000/v1/chat/completions' \
**Logging Tool**
<Image img={require('../../img/message_redaction_logging.png')}/>
<Image img={require('@site/img/message_redaction_logging.png')}/>
**Spend Logs**
<Image img={require('../../img/message_redaction_spend_logs.png')} />
<Image img={require('@site/img/message_redaction_spend_logs.png')} />
### Redacting UserAPIKeyInfo
@ -390,7 +390,7 @@ litellm --test
Expected output on Langfuse
<Image img={require('../../img/langfuse_small.png')} />
<Image img={require('@site/img/langfuse_small.png')} />
### Logging Metadata to Langfuse
@ -735,7 +735,7 @@ print(response)
You will see `raw_request` in your Langfuse Metadata. This is the RAW CURL command sent from LiteLLM to your LLM API provider
<Image img={require('../../img/debug_langfuse.png')} />
<Image img={require('@site/img/debug_langfuse.png')} />
## OpenTelemetry
@ -1084,7 +1084,7 @@ print(response)
Search for Trace=`80e1afed08e019fc1110464cfa66635c` on your OTEL Collector
<Image img={require('../../img/otel_parent.png')} />
<Image img={require('@site/img/otel_parent.png')} />
##### Forwarding `Traceparent HTTP Header` to LLM APIs
@ -1170,7 +1170,7 @@ curl --location 'http://0.0.0.0:4000/chat/completions' \
#### Expected Logs on GCS Buckets
<Image img={require('../../img/gcs_bucket.png')} />
<Image img={require('@site/img/gcs_bucket.png')} />
#### Fields Logged on GCS Buckets
@ -1304,7 +1304,7 @@ curl -X POST 'http://0.0.0.0:4000/chat/completions' \
5. Check trace on platform:
<Image img={require('../../img/deepeval_visible_trace.png')} />
<Image img={require('@site/img/deepeval_visible_trace.png')} />
## s3 Buckets
@ -1566,7 +1566,7 @@ curl --location 'http://0.0.0.0:4000/chat/completions' \
#### Expected Logs on Azure Data Lake Storage
<Image img={require('../../img/azure_blob.png')} />
<Image img={require('@site/img/azure_blob.png')} />
#### Fields Logged on Azure Data Lake Storage
@ -2050,7 +2050,7 @@ ModelResponse(
## Custom Callback APIs [Async]
<Image
img={require('../../img/callback_api.png')}
img={require('@site/img/callback_api.png')}
style={{width: '100%', display: 'block', margin: '2rem auto'}}
/>
<p style={{textAlign: 'left', color: '#666'}}>
@ -2174,7 +2174,7 @@ curl --location 'http://0.0.0.0:4000/chat/completions' \
'
```
Expect to see your log on Langfuse
<Image img={require('../../img/langsmith_new.png')} />
<Image img={require('@site/img/langsmith_new.png')} />
## Arize AI
@ -2222,7 +2222,7 @@ curl --location 'http://0.0.0.0:4000/chat/completions' \
'
```
Expect to see your log on Langfuse
<Image img={require('../../img/langsmith_new.png')} />
<Image img={require('@site/img/langsmith_new.png')} />
## Langtrace
@ -2380,7 +2380,7 @@ curl --location 'http://0.0.0.0:4000/chat/completions' \
'
```
<Image img={require('../../img/openmeter_img_2.png')} />
<Image img={require('@site/img/openmeter_img_2.png')} />
## DynamoDB

4
docs/my-website/docs/proxy/model_access_groups.md
View file

@ -431,7 +431,7 @@ You can also manage access groups through the LiteLLM Admin UI.
When adding a model to the database, assign it to an access group using the "Model Access Group" field:
![Add Model with Access Group](../../img/add_model_access.png)
![Add Model with Access Group](@site/img/add_model_access.png)
In this example, `gpt-4` is added to the `production-models` access group.
@ -439,7 +439,7 @@ In this example, `gpt-4` is added to the `production-models` access group.
When creating an API key, specify the access group in the "Models" field:
![Create Key with Access Group](../../img/add_model_key.png)
![Create Key with Access Group](@site/img/add_model_key.png)
The key will have access to all models in the `production-models` group.

22
docs/my-website/docs/proxy/model_compare_ui.md
View file

@ -12,7 +12,7 @@ This feature is **available in v1.80.0-stable and above**.
The Model Compare Playground UI enables side-by-side comparison of up to 3 different LLM models simultaneously. Configure models, parameters, and test prompts to evaluate and compare model responses with detailed metrics including latency, token usage, and cost.
<Image img={require('../../img/ui_model_compare_overview.png')} />
<Image img={require('@site/img/ui_model_compare_overview.png')} />
## Getting Started
@ -22,7 +22,7 @@ The Model Compare Playground UI enables side-by-side comparison of up to 3 diffe
Go to the Playground page in the Admin UI (`PROXY_BASE_URL/ui/?login=success&page=llm-playground`)
<Image img={require('../../img/ui_playground_navigation.png')} />
<Image img={require('@site/img/ui_playground_navigation.png')} />
#### 2. Switch to Compare Tab
@ -40,7 +40,7 @@ You can compare up to 3 models simultaneously. For each comparison panel:
- Select a model from your configured endpoints
- Models are loaded from your LiteLLM proxy configuration
<Image img={require('../../img/ui_model_compare_select_model.png')} />
<Image img={require('@site/img/ui_model_compare_select_model.png')} />
#### 2. Configure Model Parameters
@ -56,13 +56,13 @@ Each model panel supports individual parameter configuration:
- Enable "Use Advanced Params" to configure additional model-specific parameters
- Supports all parameters available for the selected model/provider
<Image img={require('../../img/ui_model_compare_model_parameters.png')} />
<Image img={require('@site/img/ui_model_compare_model_parameters.png')} />
#### 3. Apply Parameters Across Models
Use the "Sync Settings Across Models" toggle to synchronize parameters (tags, guardrails, temperature, max tokens, etc.) across all comparison panels for consistent testing.
<Image img={require('../../img/ui_model_compare_sync_across_models.png')} />
<Image img={require('@site/img/ui_model_compare_sync_across_models.png')} />
### Guardrails
@ -73,7 +73,7 @@ Configure and test guardrails directly in the playground:
3. Test how different models respond to guardrail filtering
4. Compare guardrail behavior across models
<Image img={require('../../img/ui_model_compare_guardrails_config.png')} />
<Image img={require('@site/img/ui_model_compare_guardrails_config.png')} />
### Tags
@ -82,7 +82,7 @@ Apply tags to organize and filter your comparisons:
1. Select tags from the tag dropdown
2. Tags help categorize and track different test scenarios
<Image img={require('../../img/ui_model_compare_tags_config.png')} />
<Image img={require('@site/img/ui_model_compare_tags_config.png')} />
### Vector Stores
@ -92,7 +92,7 @@ Configure vector store retrieval for RAG (Retrieval Augmented Generation) compar
2. Compare how different models utilize retrieved context
3. Evaluate RAG performance across models
<Image img={require('../../img/ui_model_compare_vector_stores_config.png')} />
<Image img={require('@site/img/ui_model_compare_vector_stores_config.png')} />
## Running Comparisons
@ -104,7 +104,7 @@ Type your test prompt in the message input area. You can:
- Use suggested prompts for quick testing
- Build multi-turn conversations
<Image img={require('../../img/ui_model_compare_enter_prompt.png')} />
<Image img={require('@site/img/ui_model_compare_enter_prompt.png')} />
### 2. Send Request
@ -118,7 +118,7 @@ Responses appear side-by-side in each model panel, making it easy to compare:
- Response length and structure
- Model-specific formatting
<Image img={require('../../img/ui_model_compare_responses.png')} />
<Image img={require('@site/img/ui_model_compare_responses.png')} />
## Comparison Metrics
@ -146,7 +146,7 @@ If cost tracking is enabled in your LiteLLM configuration, you'll see:
- Cost breakdown by input/output tokens
- Comparison of costs across models
<Image img={require('../../img/ui_model_compare_cost_metrics.png')} />
<Image img={require('@site/img/ui_model_compare_cost_metrics.png')} />
## Use Cases

2
docs/my-website/docs/proxy/multi_tenant_architecture.md
View file

@ -31,7 +31,7 @@ Organizations with multi-tenant architectures face several challenges when deplo
## How LiteLLM Solves Multi-Tenancy
<Image img={require('../../img/litellm_user_heirarchy.png')} style={{ width: '100%', maxWidth: '4000px' }} />
<Image img={require('@site/img/litellm_user_heirarchy.png')} style={{ width: '100%', maxWidth: '4000px' }} />
LiteLLM implements a hierarchical multi-tenant architecture with four levels:

4
docs/my-website/docs/proxy/multiple_admins.md
View file

@ -6,7 +6,7 @@ import Image from '@theme/IdealImage';
# ✨ Audit Logs
<Image
img={require('../../img/release_notes/ui_audit_log.png')}
img={require('@site/img/release_notes/ui_audit_log.png')}
style={{width: '100%', display: 'block', margin: '2rem auto'}}
/>
@ -51,7 +51,7 @@ curl -X POST 'http://0.0.0.0:4000/key/delete' \
On the LiteLLM UI, navigate to Logs -> Audit Logs. You should see the audit log for the key deletion.
<Image
img={require('../../img/key_delete.png')}
img={require('@site/img/key_delete.png')}
style={{width: '100%', display: 'block', margin: '2rem auto'}}
/>

4
docs/my-website/docs/proxy/pagerduty.md
View file

@ -75,7 +75,7 @@ curl -i --location 'http://0.0.0.0:4000/chat/completions' \
'
```
<Image img={require('../../img/pagerduty_fail.png')} />
<Image img={require('@site/img/pagerduty_fail.png')} />
### LLM Hanging Alert
@ -100,7 +100,7 @@ curl -i --location 'http://0.0.0.0:4000/chat/completions' \
'
```
<Image img={require('../../img/pagerduty_hanging.png')} />
<Image img={require('@site/img/pagerduty_hanging.png')} />

4
docs/my-website/docs/proxy/pass_through.md
View file

@ -31,7 +31,7 @@ To create a pass through endpoint:
- `Target URL`: The URL where requests will be forwarded
<Image
img={require('../../img/pt_1.png')}
img={require('@site/img/pt_1.png')}
style={{width: '60%', display: 'block', margin: '2rem auto'}}
/>
@ -63,7 +63,7 @@ Configure the required authentication and pricing:
- This enables cost tracking and billing for your users
<Image
img={require('../../img/pt_2.png')}
img={require('@site/img/pt_2.png')}
style={{width: '60%', display: 'block', margin: '2rem auto'}}
/>

4
docs/my-website/docs/proxy/pass_through_guardrails.md
View file

@ -20,7 +20,7 @@ You can configure guardrails on pass-through endpoints either via the **UI** (re
Go to **Models + Endpoints** → Click **+ Add Pass-Through Endpoint**
<Image img={require('../../img/pt_guard1.png')} alt="Add guardrails to pass-through endpoint" />
<Image img={require('@site/img/pt_guard1.png')} alt="Add guardrails to pass-through endpoint" />
Scroll to the **Guardrails** section and select which guardrails to enforce.
@ -30,7 +30,7 @@ By default, you don't need to specify fields - LiteLLM will JSON dump the entire
#### 2. Target Specific Fields (Optional)
<Image img={require('../../img/pt_guard2.png')} alt="Configure field-level targeting" />
<Image img={require('@site/img/pt_guard2.png')} alt="Configure field-level targeting" />
To check only specific fields instead of the entire payload:

4
docs/my-website/docs/proxy/perf.md
View file

@ -4,8 +4,8 @@ import Image from '@theme/IdealImage';
### Throughput - 30% Increase
LiteLLM proxy + Load Balancer gives **30% increase** in throughput compared to Raw OpenAI API
<Image img={require('../../img/throughput.png')} />
<Image img={require('@site/img/throughput.png')} />
### Latency Added - 0.00325 seconds
LiteLLM proxy adds **0.00325 seconds** latency as compared to using the Raw OpenAI API
<Image img={require('../../img/latency.png')} />
<Image img={require('@site/img/latency.png')} />

2
docs/my-website/docs/proxy/prod.md
View file

@ -294,7 +294,7 @@ Or [watch on Loom](https://www.loom.com/share/b08be303331246b88fdc053940d03281?s
### High Level Architecture
<Image alt="Separate Health App Architecture" img={require('../../img/separate_health_app_architecture.png')} style={{ borderRadius: '8px', marginBottom: '1em', maxWidth: '100%' }} />
<Image alt="Separate Health App Architecture" img={require('@site/img/separate_health_app_architecture.png')} style={{ borderRadius: '8px', marginBottom: '1em', maxWidth: '100%' }} />
## Extras

8
docs/my-website/docs/proxy/prometheus.md
View file

@ -406,7 +406,7 @@ litellm_settings:
On starting up LiteLLM if your metrics were correctly configured, you should see the following on your container logs
<Image
img={require('../../img/prom_config.png')}
img={require('@site/img/prom_config.png')}
style={{width: '100%', display: 'block', margin: '2rem auto'}}
/>
@ -525,11 +525,11 @@ https://github.com/BerriAI/litellm/tree/main/cookbook/litellm_proxy_server/grafa
Here is a screenshot of the metrics you can monitor with the LiteLLM Grafana Dashboard
<Image img={require('../../img/grafana_1.png')} />
<Image img={require('@site/img/grafana_1.png')} />
<Image img={require('../../img/grafana_2.png')} />
<Image img={require('@site/img/grafana_2.png')} />
<Image img={require('../../img/grafana_3.png')} />
<Image img={require('@site/img/grafana_3.png')} />
## Deprecated Metrics

6
docs/my-website/docs/proxy/prompt_management.md
View file

@ -450,7 +450,7 @@ model_list:
If the model is specified in the Langfuse config, it will be used.
<Image img={require('../../img/langfuse_prompt_management_model_config.png')} />
<Image img={require('@site/img/langfuse_prompt_management_model_config.png')} />
```yaml
model_list:
@ -470,7 +470,7 @@ model_list:
- `prompt_id`: The ID of the prompt that will be used for the request.
<Image img={require('../../img/langfuse_prompt_id.png')} />
<Image img={require('@site/img/langfuse_prompt_id.png')} />
## What will the formatted prompt look like?
@ -488,7 +488,7 @@ If the Langfuse prompt is a list, it will be sent as is (Langfuse chat prompts a
## Architectural Overview
<Image img={require('../../img/prompt_management_architecture_doc.png')} />
<Image img={require('@site/img/prompt_management_architecture_doc.png')} />
## API Reference

28
docs/my-website/docs/proxy/self_serve.md
View file

@ -13,7 +13,7 @@ import TabItem from '@theme/TabItem';
Go to `Internal Users` -> `+New User`
<Image img={require('../../img/add_internal_user.png')} style={{ width: '800px', height: 'auto' }} />
<Image img={require('@site/img/add_internal_user.png')} style={{ width: '800px', height: 'auto' }} />
</TabItem>
<TabItem value="api" label="API">
@ -61,7 +61,7 @@ Internal User Roles:
Copy the invitation link with the user
<Image img={require('../../img/invitation_link.png')} style={{ width: '800px', height: 'auto' }} />
<Image img={require('@site/img/invitation_link.png')} style={{ width: '800px', height: 'auto' }} />
</TabItem>
<TabItem value="api" label="API">
@ -110,7 +110,7 @@ Use [Email Notifications](./email.md) to email users onboarding links
3. User logs in via email + password auth
<Image img={require('../../img/ui_clean_login.png')} style={{ width: '500px', height: 'auto' }} />
<Image img={require('@site/img/ui_clean_login.png')} style={{ width: '500px', height: 'auto' }} />
@ -123,7 +123,7 @@ LiteLLM Enterprise: Enable [SSO login](./ui.md#setup-ssoauth-for-ui)
4. User can now create their own keys
<Image img={require('../../img/ui_self_serve_create_key.png')} style={{ width: '800px', height: 'auto' }} />
<Image img={require('@site/img/ui_self_serve_create_key.png')} style={{ width: '800px', height: 'auto' }} />
## Allow users to View Usage, Caching Analytics
@ -131,23 +131,23 @@ LiteLLM Enterprise: Enable [SSO login](./ui.md#setup-ssoauth-for-ui)
Set their role to `Admin Viewer` - this means they can only view usage, caching analytics
<Image img={require('../../img/ui_invite_user.png')} style={{ width: '800px', height: 'auto' }} />
<Image img={require('@site/img/ui_invite_user.png')} style={{ width: '800px', height: 'auto' }} />
<br />
2. Share invitation link with user
<Image img={require('../../img/ui_invite_link.png')} style={{ width: '800px', height: 'auto' }} />
<Image img={require('@site/img/ui_invite_link.png')} style={{ width: '800px', height: 'auto' }} />
<br />
3. User logs in via email + password auth
<Image img={require('../../img/ui_clean_login.png')} style={{ width: '500px', height: 'auto' }} />
<Image img={require('@site/img/ui_clean_login.png')} style={{ width: '500px', height: 'auto' }} />
<br />
4. User can now view Usage, Caching Analytics
<Image img={require('../../img/ui_usage.png')} style={{ width: '800px', height: 'auto' }} />
<Image img={require('@site/img/ui_usage.png')} style={{ width: '800px', height: 'auto' }} />
## Available Roles
@ -224,7 +224,7 @@ Set `PROXY_LOGOUT_URL` in your .env if you want users to get redirected to a spe
export PROXY_LOGOUT_URL="https://www.google.com"
```
<Image img={require('../../img/ui_logout.png')} style={{ width: '400px', height: 'auto' }} />
<Image img={require('@site/img/ui_logout.png')} style={{ width: '400px', height: 'auto' }} />
### Set default max budget for internal users
@ -241,11 +241,11 @@ This sets a max budget of $10 USD for internal users when they sign up.
You can also manage these settings visually in the UI:
<Image img={require('../../img/default_user_settings_admin_ui.png')} style={{ width: '700px', height: 'auto' }} />
<Image img={require('@site/img/default_user_settings_admin_ui.png')} style={{ width: '700px', height: 'auto' }} />
This budget only applies to personal keys created by that user - seen under `Default Team` on the UI.
<Image img={require('../../img/max_budget_for_internal_users.png')} style={{ width: '500px', height: 'auto' }} />
<Image img={require('@site/img/max_budget_for_internal_users.png')} style={{ width: '500px', height: 'auto' }} />
This budget does not apply to keys created under non-default teams.
@ -263,7 +263,7 @@ Go to `Internal Users` -> `Default User Settings` and set the default team to th
Let's also set the default models to `no-default-models`. This means a user can only create keys within a team.
<Image img={require('../../img/default_user_settings_with_default_team.png')} style={{ width: '1000px', height: 'auto' }} />
<Image img={require('@site/img/default_user_settings_with_default_team.png')} style={{ width: '1000px', height: 'auto' }} />
</TabItem>
<TabItem value="yaml" label="YAML">
@ -294,7 +294,7 @@ You can do this when creating a new team, or by updating an existing team.
<Tabs>
<TabItem value="ui" label="UI">
<Image img={require('../../img/create_default_team.png')} style={{ width: '600px', height: 'auto' }} />
<Image img={require('@site/img/create_default_team.png')} style={{ width: '600px', height: 'auto' }} />
</TabItem>
<TabItem value="api" label="API">
@ -323,7 +323,7 @@ You can do this when creating a new team, or by updating an existing team.
<Tabs>
<TabItem value="ui" label="UI">
<Image img={require('../../img/create_team_member_rate_limits.png')} style={{ width: '600px', height: 'auto' }} />
<Image img={require('@site/img/create_team_member_rate_limits.png')} style={{ width: '600px', height: 'auto' }} />
</TabItem>
<TabItem value="api" label="API">

2
docs/my-website/docs/proxy/service_accounts.md
View file

@ -39,7 +39,7 @@ general_settings:
### 2. Create Service Account Key on LiteLLM Proxy Admin UI
<Image img={require('../../img/create_service_account.png')} />
<Image img={require('@site/img/create_service_account.png')} />
### 3. Test Service Account Key

4
docs/my-website/docs/proxy/spend_logs_deletion.md
View file

@ -75,7 +75,7 @@ If Redis is enabled, LiteLLM uses it to make sure only one instance runs the cle
- If no lock is present:
- Cleanup still runs (useful for single-node setups)
![Working of spend log deletions](../../img/spend_log_deletion_working.png)
![Working of spend log deletions](@site/img/spend_log_deletion_working.png)
*Working of spend log deletions*
### Step 2. Batch Deletion
@ -101,5 +101,5 @@ SPEND_LOG_CLEANUP_BATCH_SIZE=2000
This would allow up to 200,000 logs to be deleted in one run.
![Batch deletion of old logs](../../img/spend_log_deletion_multi_pod.jpg)
![Batch deletion of old logs](@site/img/spend_log_deletion_multi_pod.jpg)
*Batch deletion of old logs*

2
docs/my-website/docs/proxy/tag_budgets.md
View file

@ -75,7 +75,7 @@ curl -X POST 'http://0.0.0.0:4000/tag/new' \
Navigate to the **Tag Management** page and click **Create New Tag**. Fill in the tag details and set your budget:
<Image
img={require('../../img/tag_budget1.png')}
img={require('@site/img/tag_budget1.png')}
style={{width: '80%', display: 'block', margin: '0'}}
/>

6
docs/my-website/docs/proxy/team_budgets.md
View file

@ -69,7 +69,7 @@ Response
</TabItem>
<TabItem value="UI" label="Admin UI">
<Image img={require('../../img/create_team_gif_good.gif')} />
<Image img={require('@site/img/create_team_gif_good.gif')} />
</TabItem>
@ -112,7 +112,7 @@ Response
</TabItem>
<TabItem value="UI" label="Admin UI">
<Image img={require('../../img/create_key_in_team.gif')} />
<Image img={require('@site/img/create_key_in_team.gif')} />
</TabItem>
</Tabs>
@ -155,7 +155,7 @@ On the 2nd response - expect to see the following exception
</TabItem>
<TabItem value="UI" label="Admin UI">
<Image img={require('../../img/test_key_budget.gif')} />
<Image img={require('@site/img/test_key_budget.gif')} />
</TabItem>
</Tabs>

14
docs/my-website/docs/proxy/team_logging.md
View file

@ -36,7 +36,7 @@ Team 3 -> Disabled Logging (for GDPR compliance)
Create a team called "AI Agents"
<Image
img={require('../../img/team_logging1.png')}
img={require('@site/img/team_logging1.png')}
style={{width: '100%', display: 'block', margin: '2rem auto'}}
/>
@ -48,7 +48,7 @@ Create a team called "AI Agents"
We will create a key for the team "AI Agents". The team logging settings will be used for all keys created for the team.
<Image
img={require('../../img/team_logging2.png')}
img={require('@site/img/team_logging2.png')}
style={{width: '80%', display: 'block', margin: '2rem auto', border: '1px solid #E5E7EB'}}
/>
@ -60,7 +60,7 @@ We will create a key for the team "AI Agents". The team logging settings will be
Use the new key to make a test LLM API Request, we expect to see the logs on your logging provider configured in step 1.
<Image
img={require('../../img/team_logging3.png')}
img={require('@site/img/team_logging3.png')}
style={{width: '100%', display: 'block', margin: '2rem auto'}}
/>
@ -71,7 +71,7 @@ Use the new key to make a test LLM API Request, we expect to see the logs on you
Navigate to your configured logging provider and check if you received the logs from step 2.
<Image
img={require('../../img/team_logging4.png')}
img={require('@site/img/team_logging4.png')}
style={{width: '100%', display: 'block', margin: '2rem auto'}}
/>
@ -265,7 +265,7 @@ Use the `/key/generate` or `/key/update` endpoints to add logging callbacks to a
When creating a key, you can configure the specific logging settings for the key. These logging settings will be used for all requests made with this key.
<Image
img={require('../../img/key_logging.png')}
img={require('@site/img/key_logging.png')}
style={{width: '100%', display: 'block', margin: '2rem auto'}}
/>
<br />
@ -276,7 +276,7 @@ When creating a key, you can configure the specific logging settings for the key
Use the new key to make a test LLM API Request, we expect to see the logs on your logging provider configured in step 1.
<Image
img={require('../../img/key_logging2.png')}
img={require('@site/img/key_logging2.png')}
style={{width: '100%', display: 'block', margin: '2rem auto'}}
/>
@ -287,7 +287,7 @@ Use the new key to make a test LLM API Request, we expect to see the logs on you
Navigate to your configured logging provider and check if you received the logs from step 2.
<Image
img={require('../../img/key_logging_arize.png')}
img={require('@site/img/key_logging_arize.png')}
style={{width: '100%', display: 'block', margin: '2rem auto'}}
/>

6
docs/my-website/docs/proxy/ui.md
View file

@ -6,7 +6,7 @@ import TabItem from '@theme/TabItem';
Create keys, track spend, add models without worrying about the config / CRUD endpoints.
<Image img={require('../../img/litellm_ui_create_key.png')} />
<Image img={require('@site/img/litellm_ui_create_key.png')} />
## Quick Start
@ -33,7 +33,7 @@ http://0.0.0.0:4000/ui # <proxy_base_url>/ui
Your Proxy Swagger is available on the root of the Proxy: e.g.: `http://localhost:4000/`
<Image img={require('../../img/ui_link.png')} />
<Image img={require('@site/img/ui_link.png')} />
### 4. Change default username + password
@ -88,4 +88,4 @@ Useful, if your security team has additional restrictions on UI usage.
**Expected Response**
<Image img={require('../../img/admin_ui_disabled.png')}/>
<Image img={require('@site/img/admin_ui_disabled.png')}/>

6
docs/my-website/docs/proxy/ui/bulk_edit_users.md
View file

@ -10,15 +10,15 @@ Assign existing users to a default team and default model access.
### 1. Select the users you want to edit
<Image img={require('../../../img/bulk_select_users.png')} />
<Image img={require('@site/img/bulk_select_users.png')} />
### 2. Select the team you want to assign to the users
<Image img={require('../../../img/select_default_team.png')} />
<Image img={require('@site/img/select_default_team.png')} />
### 3. Click the bulk edit button
<Image img={require('../../../img/success_bulk_edit.png')} />
<Image img={require('@site/img/success_bulk_edit.png')} />

10
docs/my-website/docs/proxy/ui_credentials.md
View file

@ -12,7 +12,7 @@ You can add LLM provider credentials on the UI. Once you add credentials you can
Go to Models -> LLM Credentials -> Add Credential
<Image img={require('../../img/ui_cred_add.png')} />
<Image img={require('@site/img/ui_cred_add.png')} />
### 2. Add credentials
@ -20,14 +20,14 @@ Select your LLM provider, enter your API Key and click "Add Credential"
**Note: Credentials are based on the provider, if you select Vertex AI then you will see `Vertex Project`, `Vertex Location` and `Vertex Credentials` fields**
<Image img={require('../../img/ui_add_cred_2.png')} />
<Image img={require('@site/img/ui_add_cred_2.png')} />
### 3. Use credentials when adding a model
Go to Add Model -> Existing Credentials -> Select your credential in the dropdown
<Image img={require('../../img/ui_cred_3.png')} />
<Image img={require('@site/img/ui_cred_3.png')} />
## Create a Credential from an existing model
@ -38,13 +38,13 @@ Use this if you have already created a model and want to store the model credent
Go to Models -> Select your model -> Credential -> Create Credential
<Image img={require('../../img/ui_cred_4.png')} />
<Image img={require('@site/img/ui_cred_4.png')} />
### 2. Use new credential when adding a model
Go to Add Model -> Existing Credentials -> Select your credential in the dropdown
<Image img={require('../../img/use_model_cred.png')} />
<Image img={require('@site/img/use_model_cred.png')} />
## Frequently Asked Questions

4
docs/my-website/docs/proxy/ui_logs.md
View file

@ -8,7 +8,7 @@ import TabItem from '@theme/TabItem';
View Spend, Token Usage, Key, Team Name for Each Request to LiteLLM
<Image img={require('../../img/ui_request_logs.png')}/>
<Image img={require('@site/img/ui_request_logs.png')}/>
## Overview
@ -32,7 +32,7 @@ general_settings:
store_prompts_in_spend_logs: true
```
<Image img={require('../../img/ui_request_logs_content.png')}/>
<Image img={require('@site/img/ui_request_logs_content.png')}/>
## Stop storing Error Logs in DB

2
docs/my-website/docs/proxy/ui_logs_sessions.md
View file

@ -7,7 +7,7 @@ import TabItem from '@theme/TabItem';
Group requests into sessions. This allows you to group related requests together.
<Image img={require('../../img/ui_session_logs.png')}/>
<Image img={require('@site/img/ui_session_logs.png')}/>
## Usage

2
docs/my-website/docs/proxy/user_management_heirarchy.md
View file

@ -3,7 +3,7 @@ import Image from '@theme/IdealImage';
# User Management Hierarchy
<Image img={require('../../img/litellm_user_heirarchy.png')} style={{ width: '100%', maxWidth: '4000px' }} />
<Image img={require('@site/img/litellm_user_heirarchy.png')} style={{ width: '100%', maxWidth: '4000px' }} />
LiteLLM supports a hierarchy of users, teams, organizations, and budgets.

4
docs/my-website/docs/proxy/virtual_keys.md
View file

@ -597,7 +597,7 @@ curl 'http://0.0.0.0:4000/key/generate' \
On the LiteLLM UI, Navigate to the Keys page and click on `Generate Key` > `Key Lifecycle` > `Enable Auto Rotation`
<Image
img={require('../../img/key_r.png')}
img={require('@site/img/key_r.png')}
style={{width: '30%', display: 'block', margin: '0'}}
/>
@ -628,7 +628,7 @@ curl 'http://0.0.0.0:4000/key/update' \
On the LiteLLM UI, Navigate to the Keys page. Select the key you want to update and click on `Edit Settings` > `Auto-Rotation Settings`
<Image
img={require('../../img/key_u.png')}
img={require('@site/img/key_u.png')}
style={{width: '30%', display: 'block', margin: '0'}}
/>

2
docs/my-website/docs/router_architecture.md
View file

@ -6,7 +6,7 @@ import TabItem from '@theme/TabItem';
## High Level architecture
<Image img={require('../img/router_architecture.png')} style={{ width: '100%', maxWidth: '4000px' }} />
<Image img={require('@site/img/router_architecture.png')} style={{ width: '100%', maxWidth: '4000px' }} />
### Request Flow

4
docs/my-website/docs/secret_managers/cyberark.md
View file

@ -73,7 +73,7 @@ When you create a virtual key in the LiteLLM UI, it automatically gets stored in
In this example, we create a key named `litellm-cyber-ark-secret-key`:
<Image img={require('../../img/cyberark1.png')} alt="Creating virtual key in LiteLLM UI" />
<Image img={require('@site/img/cyberark1.png')} alt="Creating virtual key in LiteLLM UI" />
**Step 2:** Verify the secret exists in CyberArk
@ -89,7 +89,7 @@ curl -H "Authorization: Token token=\"$TOKEN\"" \
The response shows `litellm-cyber-ark-secret-key` exists in CyberArk:
<Image img={require('../../img/cyberark2.png')} alt="Virtual key stored in CyberArk API" />
<Image img={require('@site/img/cyberark2.png')} alt="Virtual key stored in CyberArk API" />
The virtual key is stored with the full path: `default:variable:litellm/litellm-cyber-ark-secret-key`

8
docs/my-website/docs/secret_managers/hashicorp_vault.md
View file

@ -181,7 +181,7 @@ For example, for `AZURE_API_KEY`, the secret should be stored as:
}
```
<Image img={require('../../img/hcorp.png')} />
<Image img={require('@site/img/hcorp.png')} />
**Writing Secrets**
@ -189,20 +189,20 @@ When a Virtual Key is Created / Deleted on LiteLLM, LiteLLM will automatically c
- Create Virtual Key on LiteLLM either through the LiteLLM Admin UI or API
<Image img={require('../../img/hcorp_create_virtual_key.png')} />
<Image img={require('@site/img/hcorp_create_virtual_key.png')} />
- Check Hashicorp Vault for secret
LiteLLM stores secret under the `prefix_for_stored_virtual_keys` path (default: `litellm/`)
<Image img={require('../../img/hcorp_virtual_key.png')} />
<Image img={require('@site/img/hcorp_virtual_key.png')} />
### Team-specific overrides
When running the LiteLLM proxy you can override the Vault location per team. Use the [Team-Level Secret Manager Settings](./overview.md#team-level-secret-manager-settings) flow in the dashboard and configure the panel shown below:
<Image img={require('../../img/secret_manager_hashicorp_vault_settings.png')} />
<Image img={require('@site/img/secret_manager_hashicorp_vault_settings.png')} />
Use the following structure for the JSON payload:

Some files were not shown because too many files have changed in this diff Show more