mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-09 03:18:44 +00:00
docs auto router
This commit is contained in:
parent
f2e9834dc6
commit
1a15bd4885
1 changed files with 47 additions and 0 deletions
|
|
@ -1,4 +1,6 @@
|
|||
import Image from '@theme/IdealImage';
|
||||
import Tabs from '@theme/Tabs';
|
||||
import TabItem from '@theme/TabItem';
|
||||
|
||||
# Auto Routing
|
||||
|
||||
|
|
@ -165,6 +167,51 @@ Configure each route with:
|
|||
- **Score Threshold** - The minimum similarity score (0.0-1.0) required to trigger this route
|
||||
|
||||
|
||||
### Usage
|
||||
|
||||
Once added developers need to select the model=`auto_router1` in the `model` field of the LLM API request.
|
||||
|
||||
<Tabs>
|
||||
<TabItem value="openai" label="OpenAI Python v1.0.0+">
|
||||
|
||||
```python
|
||||
import openai
|
||||
client = openai.OpenAI(
|
||||
api_key="sk-1234", # replace with your LiteLLM API key
|
||||
base_url="http://localhost:4000"
|
||||
)
|
||||
|
||||
# This request will be auto-routed based on the content
|
||||
response = client.chat.completions.create(
|
||||
model="auto_router1",
|
||||
messages=[
|
||||
{
|
||||
"role": "user",
|
||||
"content": "how to code a program in python"
|
||||
}
|
||||
]
|
||||
)
|
||||
|
||||
print(response)
|
||||
```
|
||||
</TabItem>
|
||||
|
||||
<TabItem value="curl" label="Curl Request">
|
||||
|
||||
```shell
|
||||
curl -X POST http://localhost:4000/v1/chat/completions \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "Authorization: Bearer $LITELLM_API_KEY" \
|
||||
-d '{
|
||||
"model": "auto_router1",
|
||||
"messages": [{"role": "user", "content": "how to code a program in python"}]
|
||||
}'
|
||||
```
|
||||
</TabItem>
|
||||
</Tabs>
|
||||
|
||||
|
||||
|
||||
## How It Works
|
||||
|
||||
1. When a request comes in, LiteLLM generates embeddings for the input message
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue