diff --git a/docs/my-website/docs/guides/index.md b/docs/my-website/docs/guides/index.md index 294bdf1412b..b39234e4c87 100644 --- a/docs/my-website/docs/guides/index.md +++ b/docs/my-website/docs/guides/index.md @@ -5,62 +5,102 @@ sidebar_label: Overview import NavigationCards from '@site/src/components/NavigationCards'; -**Guides** are focused references for specific LiteLLM SDK features and proxy configuration options. Each guide is self-contained โ€” jump to any topic without reading the others first. +**Guides** are focused references organized by the job you are trying to do with LiteLLM: make requests, use tools, handle media, manage context, or operate the gateway safely. -> Looking for step-by-step integration walkthroughs? See [Tutorials โ†’](/docs/tutorials) +> New to LiteLLM or not sure whether you need the SDK or Gateway path first? Start at [Learn โ†’](/docs/learn) --- -## Core Features +## Start Here --- -## Configuration & Cost +## Build With LiteLLM + +--- + +## Operate & Extend + + \ No newline at end of file +/> diff --git a/docs/my-website/docs/learn/gateway_quickstart.md b/docs/my-website/docs/learn/gateway_quickstart.md new file mode 100644 index 00000000000..f29f6e7d483 --- /dev/null +++ b/docs/my-website/docs/learn/gateway_quickstart.md @@ -0,0 +1,159 @@ +--- +title: Gateway Quickstart +sidebar_label: Gateway Quickstart +description: Start LiteLLM Gateway, add models and keys, then connect applications and SDKs to one shared endpoint. +--- + +import NavigationCards from '@site/src/components/NavigationCards'; + +Use this path if you need one shared OpenAI-compatible endpoint for a team or platform. + +If you need a Docker or database-first setup, use the [Docker + Database tutorial](/docs/proxy/docker_quick_start). Otherwise, use the steps below to get to a working request fast. + +## 1. Install The Gateway + +```bash +pip install 'litellm[proxy]' +``` + +## 2. Set One Provider Key + +```bash +export OPENAI_API_KEY="your-api-key" +``` + +## 3. Create `config.yaml` + +```yaml +model_list: + - model_name: gpt-4o-mini + litellm_params: + model: openai/gpt-4o-mini + api_key: os.environ/OPENAI_API_KEY + +general_settings: + master_key: sk-1234 +``` + +## 4. Start The Gateway + +```bash +litellm --config config.yaml +``` + +You should see the proxy start on `http://0.0.0.0:4000`. + +## 5. Send Your First Request + +```bash +curl -X POST 'http://0.0.0.0:4000/chat/completions' \ + -H 'Content-Type: application/json' \ + -H 'Authorization: Bearer sk-1234' \ + -d '{ + "model": "gpt-4o-mini", + "messages": [ + {"role": "user", "content": "Hello from LiteLLM Gateway"} + ] + }' +``` + +## 6. Check The Response + +If the request succeeds, the proxy returns `200 OK` with the same OpenAI-style response shape LiteLLM uses in the SDK. + +The assistant text will be in: + +```json +choices[0].message.content +``` + +It looks like this: + +```json +{ + "id": "chatcmpl-abc123", + "object": "chat.completion", + "created": 1677858242, + "model": "gpt-4o-mini", + "choices": [ + { + "index": 0, + "message": { + "role": "assistant", + "content": "Hello! How can I help?" + }, + "finish_reason": "stop" + } + ], + "usage": { + "prompt_tokens": 12, + "completion_tokens": 9, + "total_tokens": 21 + } +} +``` + +`id`, `created`, token counts, and message text will vary by request. + +## 7. Add Keys And The UI + +If you need virtual keys, spend tracking, or the admin UI, add a database next. + +- Add `database_url` under `general_settings` +- Use [Virtual keys](/docs/proxy/virtual_keys) for key creation and budgets +- Use [Admin UI](/docs/proxy/ui) to manage models and keys +- Use the [Docker + Database tutorial](/docs/proxy/docker_quick_start) if you want a fuller setup + +## 8. Pick Your Next Step + + + +## When To Use The SDK Path Instead + +If you only need to call models from one application and do not need centralized auth or shared infrastructure, start with the [SDK Quickstart](/docs/learn/sdk_quickstart) instead. diff --git a/docs/my-website/docs/learn/index.md b/docs/my-website/docs/learn/index.md index bb8ffe851e5..e36ec906f32 100644 --- a/docs/my-website/docs/learn/index.md +++ b/docs/my-website/docs/learn/index.md @@ -6,106 +6,148 @@ slug: /learn import NavigationCards from '@site/src/components/NavigationCards'; -LiteLLM gives you a single OpenAI-compatible interface to 100+ LLM providers. Use this page to find the right starting point for where you are. +LiteLLM gives you one OpenAI-compatible interface for 100+ LLM providers. Start with the path that matches your setup. --- -## Get Started +## Start Here -Set up your environment and make your first LLM call. +Pick one path first. + + + +--- + +## Common Tasks + +Jump to a specific task. + +--- + +## Explore + +Use these when you want examples or tool-specific walkthroughs. + + - -[View all getting started tutorials โ†’](/docs/tutorials/getting_started) - ---- - -## Guides - -Focused references for SDK features and proxy configuration. Each guide is self-contained. - - - -[View all guides โ†’](/docs/guides) - ---- - -## Tutorials - -Step-by-step walkthroughs for integrating LiteLLM with external tools and services. - - -[View all tutorials โ†’](/docs/tutorials) - --- -## Production +## Docs Map -Add observability, access control, and safety to your deployment. See the [Production section](/docs/tutorials#production) in Tutorials for Observability & Safety, Proxy & Gateway, and more. +Use these when you already know the type of doc you want. + + + +Not sure where to start? Use [SDK Quickstart](/docs/learn/sdk_quickstart) for app code or [Gateway Quickstart](/docs/learn/gateway_quickstart) for shared infrastructure. diff --git a/docs/my-website/docs/learn/sdk_quickstart.md b/docs/my-website/docs/learn/sdk_quickstart.md new file mode 100644 index 00000000000..3726f1f22b9 --- /dev/null +++ b/docs/my-website/docs/learn/sdk_quickstart.md @@ -0,0 +1,137 @@ +--- +title: SDK Quickstart +sidebar_label: SDK Quickstart +description: Make your first LiteLLM SDK call, then jump to the right docs for the next feature you need. +--- + +import NavigationCards from '@site/src/components/NavigationCards'; + +Use this path if you are integrating LiteLLM directly into application code. + +## 1. Install LiteLLM + +```bash +pip install litellm +``` + +## 2. Set Provider Credentials + +Start with one provider and set its environment variables. + +- OpenAI: `OPENAI_API_KEY` +- Anthropic: `ANTHROPIC_API_KEY` +- Azure OpenAI: `AZURE_API_KEY`, `AZURE_API_BASE`, `AZURE_API_VERSION` +- Bedrock: standard AWS credentials +- Vertex AI: `VERTEXAI_PROJECT`, `VERTEXAI_LOCATION` + +If you have not picked a provider yet, browse [all supported providers](/docs/providers). + +## 3. Make Your First Call + +```python +from litellm import completion +import os + +os.environ["OPENAI_API_KEY"] = "your-api-key" + +response = completion( + model="openai/gpt-4o", + messages=[{"role": "user", "content": "Hello, how are you?"}], +) + +print(response.choices[0].message.content) +``` + +## 4. Check The Response + +The line below: + +```python +print(response.choices[0].message.content) +``` + +prints the assistant text, for example: + +```text +Hello! I'm doing well, thanks for asking. +``` + +The full response is an OpenAI-style `ModelResponse` object. It looks like this: + +```json +{ + "id": "chatcmpl-abc123", + "object": "chat.completion", + "created": 1677858242, + "model": "gpt-4o", + "choices": [ + { + "index": 0, + "message": { + "role": "assistant", + "content": "Hello! I'm doing well, thanks for asking." + }, + "finish_reason": "stop" + } + ], + "usage": { + "prompt_tokens": 13, + "completion_tokens": 12, + "total_tokens": 25 + } +} +``` + +`id`, `created`, token counts, and message text will vary by request. For the full output reference, see [completion output](/docs/completion/output). + +Need more provider examples? See the main [Getting Started](/docs/#quick-start) page. + +## 5. Pick Your Next Step + + + +## When To Use Gateway Instead + +Use LiteLLM Gateway if you need centralized auth, virtual keys, spend tracking, shared logging, or one OpenAI-compatible endpoint for multiple apps. + +[Go to Gateway Quickstart โ†’](/docs/learn/gateway_quickstart) diff --git a/docs/my-website/docs/tutorials/index.md b/docs/my-website/docs/tutorials/index.md index 1d32ae5a940..9c3fc7a35b6 100644 --- a/docs/my-website/docs/tutorials/index.md +++ b/docs/my-website/docs/tutorials/index.md @@ -7,7 +7,35 @@ import NavigationCards from '@site/src/components/NavigationCards'; **Tutorials** are step-by-step walkthroughs for integrating LiteLLM with external tools, frameworks, and services โ€” or building complete end-to-end workflows. -> Need help with a specific LiteLLM SDK feature or proxy config? See [Guides โ†’](/docs/guides) +> Need help choosing the right path before you start? See [Learn โ†’](/docs/learn) + +--- + +## Start Here + + --- diff --git a/docs/my-website/sidebars.js b/docs/my-website/sidebars.js index b64291ac1df..27d7c6b34f3 100644 --- a/docs/my-website/sidebars.js +++ b/docs/my-website/sidebars.js @@ -17,11 +17,6 @@ const sidebars = { integrationsSidebar: [ { type: "doc", id: "integrations/index" }, { type: "doc", id: "integrations/community" }, - { - type: "doc", - id: "integrations/websearch_interception", - label: "Web Search Integration" - }, { type: "category", label: "Observability", @@ -1140,6 +1135,16 @@ const learnSidebar = { learnSidebar: [ // โ”€โ”€ Landing page โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ { type: "doc", id: "learn/index", label: "Learn" }, + { + type: "category", + label: "Start Here", + collapsible: true, + collapsed: false, + items: [ + "learn/sdk_quickstart", + "learn/gateway_quickstart", + ], + }, // โ”€โ”€ Guides โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ { @@ -1151,118 +1156,151 @@ const learnSidebar = { items: [ { type: "category", - label: "Completion Basics", + label: "Core Requests", collapsible: true, collapsed: false, link: { type: "generated-index", - title: "Completion Basics", - description: "Streaming, function calling, JSON mode, vision, audio, and more", - slug: "/guides/completion_basics" + title: "Core Requests", + description: "Streaming, batching, structured outputs, and reasoning behavior", + slug: "/guides/core_request_response_patterns" }, items: [ "completion/stream", - "completion/function_call", - "completion/json_mode", - "completion/vision", - "completion/audio", "completion/batching", - "completion/prefix", - "completion/predict_outputs", - "completion/provider_specific_params", + "completion/json_mode", "reasoning_content", - "completion/drop_params", - "completion/model_alias", - "completion/mock_requests", - "completion/reliable_completions", ], }, { type: "category", - label: "Documents, Images & Messages", + label: "Tool Calling", collapsible: true, collapsed: true, link: { type: "generated-index", - title: "Documents, Images & Messages", - description: "Document understanding, image generation, message trimming, and sanitization", - slug: "/guides/input_output_handling" + title: "Tool Calling", + description: "Function calling, web tools, interception patterns, computer use, code interpreter, and tool-call hygiene", + slug: "/guides/tools_integrations" }, items: [ - "completion/document_understanding", - "completion/image_generation_chat", - "completion/message_trimming", + "completion/function_call", + "completion/web_search", + { + type: "doc", + id: "integrations/websearch_interception", + label: "Web Search Interception", + }, + "completion/web_fetch", + "completion/computer_use", + "guides/code_interpreter", "completion/message_sanitization", ], }, { type: "category", - label: "Prompt Optimization", + label: "Multimodal I/O", collapsible: true, collapsed: true, link: { type: "generated-index", - title: "Prompt Optimization", - description: "Prompt caching and prompt formatting", - slug: "/guides/prompt_optimization" + title: "Multimodal I/O", + description: "Vision, audio, PDFs, image generation, and video generation", + slug: "/guides/multimodal_io" }, items: [ + "completion/vision", + "completion/audio", + "completion/document_understanding", + "completion/image_generation_chat", + "proxy/veo_video_generation", + ], + }, + { + type: "category", + label: "Retrieval & Knowledge", + collapsible: true, + collapsed: true, + link: { + type: "generated-index", + title: "Retrieval & Knowledge", + description: "Vector stores, file search, citations, and knowledge-base routing", + slug: "/guides/retrieval_knowledge" + }, + items: [ + "completion/knowledgebase", + ], + }, + { + type: "category", + label: "Prompts & Context", + collapsible: true, + collapsed: true, + link: { + type: "generated-index", + title: "Prompts & Context", + description: "Prompt caching, trimming, formatting, assistant prefill, and predicted outputs", + slug: "/guides/prompts_context" + }, + items: [ + "completion/prefix", + "completion/predict_outputs", + "completion/message_trimming", "completion/prompt_caching", "completion/prompt_formatting", ], }, { type: "category", - label: "AI Capabilities", + label: "Compatibility & Extensibility", collapsible: true, collapsed: true, link: { type: "generated-index", - title: "AI Capabilities", - description: "Web search, code interpreter, knowledge base, and more", - slug: "/guides/ai_capabilities" - }, - items: [ - "completion/web_search", - "completion/web_fetch", - "completion/computer_use", - "completion/knowledgebase", - "guides/code_interpreter", - "proxy/veo_video_generation", - ], - }, - { - type: "category", - label: "Models & Configuration", - collapsible: true, - collapsed: true, - link: { - type: "generated-index", - title: "Models & Configuration", - description: "Fine-tuned models, security settings, and adapters", - slug: "/guides/models_configuration" + title: "Compatibility & Extensibility", + description: "Provider-specific params, model aliases, fine-tuned models, and adapters", + slug: "/guides/compatibility_extensibility" }, items: [ + "completion/provider_specific_params", + "completion/drop_params", + "completion/model_alias", "guides/finetuned_models", - "guides/security_settings", "extras/creating_adapters", ], }, { type: "category", - label: "Budgets & Cost", + label: "Reliability, Testing & Spend", collapsible: true, collapsed: true, link: { type: "generated-index", - title: "Budgets & Cost", - description: "Set spend limits and track costs", - slug: "/guides/budgets_cost" + title: "Reliability, Testing & Spend", + description: "Retries, fallbacks, mock responses, and budget controls", + slug: "/guides/reliability_testing_spend" }, items: [ + "completion/mock_requests", + "completion/reliable_completions", "budget_manager", ], }, + { + type: "category", + label: "Security & Network", + collapsible: true, + collapsed: true, + link: { + type: "generated-index", + title: "Security & Network", + description: "SSL, custom CA bundles, HTTP proxy settings, and per-service verification", + slug: "/guides/security_network" + }, + items: [ + "guides/security_settings", + ], + }, ], },