diff --git a/docs/my-website/docs/proxy/auto_routing.md b/docs/my-website/docs/proxy/auto_routing.md index 3169047a6a4..7325dc8227e 100644 --- a/docs/my-website/docs/proxy/auto_routing.md +++ b/docs/my-website/docs/proxy/auto_routing.md @@ -1,4 +1,6 @@ import Image from '@theme/IdealImage'; +import Tabs from '@theme/Tabs'; +import TabItem from '@theme/TabItem'; # Auto Routing @@ -165,6 +167,51 @@ Configure each route with: - **Score Threshold** - The minimum similarity score (0.0-1.0) required to trigger this route +### Usage + +Once added developers need to select the model=`auto_router1` in the `model` field of the LLM API request. + + + + +```python +import openai +client = openai.OpenAI( + api_key="sk-1234", # replace with your LiteLLM API key + base_url="http://localhost:4000" +) + +# This request will be auto-routed based on the content +response = client.chat.completions.create( + model="auto_router1", + messages=[ + { + "role": "user", + "content": "how to code a program in python" + } + ] +) + +print(response) +``` + + + + +```shell +curl -X POST http://localhost:4000/v1/chat/completions \ +-H "Content-Type: application/json" \ +-H "Authorization: Bearer $LITELLM_API_KEY" \ +-d '{ + "model": "auto_router1", + "messages": [{"role": "user", "content": "how to code a program in python"}] +}' +``` + + + + + ## How It Works 1. When a request comes in, LiteLLM generates embeddings for the input message