docs auto router

This commit is contained in:
Ishaan Jaff 2025-07-25 09:04:27 -07:00
parent f2e9834dc6
commit 1a15bd4885

View file

@ -1,4 +1,6 @@
import Image from '@theme/IdealImage';
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
# Auto Routing
@ -165,6 +167,51 @@ Configure each route with:
- **Score Threshold** - The minimum similarity score (0.0-1.0) required to trigger this route
### Usage
Once added developers need to select the model=`auto_router1` in the `model` field of the LLM API request.
<Tabs>
<TabItem value="openai" label="OpenAI Python v1.0.0+">
```python
import openai
client = openai.OpenAI(
api_key="sk-1234", # replace with your LiteLLM API key
base_url="http://localhost:4000"
)
# This request will be auto-routed based on the content
response = client.chat.completions.create(
model="auto_router1",
messages=[
{
"role": "user",
"content": "how to code a program in python"
}
]
)
print(response)
```
</TabItem>
<TabItem value="curl" label="Curl Request">
```shell
curl -X POST http://localhost:4000/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $LITELLM_API_KEY" \
-d '{
"model": "auto_router1",
"messages": [{"role": "user", "content": "how to code a program in python"}]
}'
```
</TabItem>
</Tabs>
## How It Works
1. When a request comes in, LiteLLM generates embeddings for the input message