diff --git a/README.md b/README.md index a7311882bd0..5a796257654 100644 --- a/README.md +++ b/README.md @@ -141,12 +141,56 @@ response = client.chat.completions.create( ) ``` -[**Docs: LLM Providers**](https://docs.litellm.ai/docs/providers) + + +
+Auto Routing - Choose a model for each prompt (Python SDK + AI Gateway) + +### Python SDK + +[**Docs: Autorouter SDK**](https://docs.litellm.ai/docs/autorouter_sdk) + +Install with `uv add litellm` and set `OPENAI_API_KEY` for the models below. + +```python +import asyncio +from litellm import Router + +router = Router(model_list=[ + {"model_name": "fast", "litellm_params": {"model": "openai/gpt-6-luna"}}, + {"model_name": "capable", "litellm_params": {"model": "openai/gpt-6.1-sol"}}, + {"model_name": "auto", "litellm_params": { + "model": "auto_router/complexity_router", + "complexity_router_config": { + "classifier_type": "heuristic", + "tiers": {"SIMPLE": "fast", "MEDIUM": "fast", + "COMPLEX": "capable", "REASONING": "capable"}, + "return_raw_model_name": True, + }, + "complexity_router_default_model": "capable", + }}, +]) +response = asyncio.run(router.acompletion( + model="auto", messages=[{"role": "user", "content": "What is 2+2?"}], +)) +print(response.model, response.choices[0].message.content) +``` + +### AI Gateway (Proxy Server) + +[**Create an Auto Router**](https://docs.litellm.ai/docs/auto_router/setup), then call its model name. Replace `auto` and the URL below with yours, and set `LITELLM_API_KEY` to your virtual key. + +```shell +curl http://localhost:4000/v1/chat/completions \ + -H "Authorization: Bearer $LITELLM_API_KEY" \ + -H "Content-Type: application/json" \ + -d '{"model":"auto","messages":[{"role":"user","content":"What is 2+2?"}]}' +```
-Agents - Invoke A2A Agents (Python SDK + AI Gateway) +A2A - Invoke A2A Agents (Python SDK + AI Gateway) [**Supported Providers**](https://docs.litellm.ai/docs/a2a#add-a2a-agents) - LangGraph, Vertex AI Agent Engine, Azure AI Foundry, Bedrock AgentCore, Pydantic AI @@ -442,7 +486,7 @@ You can use LiteLLM through either the Proxy Server or Python SDK. Both give you LiteLLM AI Gateway -LiteLLM Python SDK +LiteLLM Python SDK