scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-07 08:26:10 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 36

docs: update sidebar structure and enhance guides

Browse source
This commit is contained in:
Arindam200 2026-03-18 01:46:00 +05:30
parent d6b4015ed6
commit 26dce15f07
6 changed files with 610 additions and 166 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

104
docs/my-website/docs/guides/index.md
View file

@ -5,62 +5,102 @@ sidebar_label: Overview
import NavigationCards from '@site/src/components/NavigationCards';
**Guides** are focused references for specific LiteLLM SDK features and proxy configuration options. Each guide is self-contained — jump to any topic without reading the others first.
**Guides** are focused references organized by the job you are trying to do with LiteLLM: make requests, use tools, handle media, manage context, or operate the gateway safely.
> Looking for step-by-step integration walkthroughs? See [Tutorials →](/docs/tutorials)
> New to LiteLLM or not sure whether you need the SDK or Gateway path first? Start at [Learn →](/docs/learn)
---
## Core Features
## Start Here
<NavigationCards
columns={2}
columns={3}
items={[
{
icon: "⚡",
title: "Completion Basics",
description: "Streaming, function calling, JSON mode, vision, audio, and more.",
to: "/docs/guides/completion_basics",
icon: "🐍",
title: "SDK Quickstart",
description: "Install LiteLLM, make your first call, then branch into feature guides.",
to: "/docs/learn/sdk_quickstart",
},
{
icon: "📥",
title: "Documents, Images & Messages",
description: "Document understanding, image generation, message trimming, and sanitization.",
to: "/docs/guides/input_output_handling",
icon: "🖥️",
title: "Gateway Quickstart",
description: "Start the proxy, add keys, then move into proxy-specific guides.",
to: "/docs/learn/gateway_quickstart",
},
{
icon: "🔄",
title: "Prompt Optimization",
description: "Prompt caching and prompt formatting for better performance.",
to: "/docs/guides/prompt_optimization",
},
{
icon: "🤖",
title: "AI Capabilities",
description: "Web search, code interpreter, knowledge base, and more.",
to: "/docs/guides/ai_capabilities",
icon: "🛠️",
title: "Need Walkthroughs?",
description: "Use Tutorials for end-to-end integrations instead of feature references.",
to: "/docs/tutorials",
},
]}
/>
---
## Configuration & Cost
## Build With LiteLLM
<NavigationCards
columns={2}
columns={3}
items={[
{
icon: "⚡",
title: "Core Requests",
description: "Streaming, batching, structured outputs, and reasoning behavior.",
to: "/docs/guides/core_request_response_patterns",
},
{
icon: "🛠️",
title: "Tool Calling",
description: "Function calling, web tools, interception patterns, computer use, code interpreter, and tool-call hygiene.",
to: "/docs/guides/tools_integrations",
},
{
icon: "🖼️",
title: "Multimodal I/O",
description: "Vision, audio, PDFs, image generation, and video generation.",
to: "/docs/guides/multimodal_io",
},
{
icon: "📚",
title: "Retrieval & Knowledge",
description: "Vector stores, file search, citations, and knowledge-base routing.",
to: "/docs/guides/retrieval_knowledge",
},
{
icon: "🧠",
title: "Prompts & Context",
description: "Prompt caching, trimming, formatting, assistant prefill, and predicted outputs.",
to: "/docs/guides/prompts_context",
},
]}
/>
---
## Operate & Extend
<NavigationCards
columns={3}
items={[
{
icon: "🎛️",
title: "Models & Configuration",
description: "Fine-tuned models, security settings, and adapters.",
to: "/docs/guides/models_configuration",
title: "Compatibility & Extensibility",
description: "Provider-specific params, model aliases, fine-tuned models, and adapters.",
to: "/docs/guides/compatibility_extensibility",
},
{
icon: "💰",
title: "Budgets & Cost",
description: "Set spend limits and track costs across teams and deployments.",
to: "/docs/guides/budgets_cost",
icon: "🧪",
title: "Reliability, Testing & Spend",
description: "Retries, fallbacks, mock responses, and budget controls.",
to: "/docs/guides/reliability_testing_spend",
},
{
icon: "🔒",
title: "Security & Network",
description: "SSL, custom CA bundles, HTTP proxy settings, and per-service verification.",
to: "/docs/guides/security_network",
},
]}
/>
/>

159
docs/my-website/docs/learn/gateway_quickstart.md Normal file
View file

@ -0,0 +1,159 @@
---
title: Gateway Quickstart
sidebar_label: Gateway Quickstart
description: Start LiteLLM Gateway, add models and keys, then connect applications and SDKs to one shared endpoint.
---
import NavigationCards from '@site/src/components/NavigationCards';
Use this path if you need one shared OpenAI-compatible endpoint for a team or platform.
If you need a Docker or database-first setup, use the [Docker + Database tutorial](/docs/proxy/docker_quick_start). Otherwise, use the steps below to get to a working request fast.
## 1. Install The Gateway
```bash
pip install 'litellm[proxy]'
```
## 2. Set One Provider Key
```bash
export OPENAI_API_KEY="your-api-key"
```
## 3. Create `config.yaml`
```yaml
model_list:
- model_name: gpt-4o-mini
litellm_params:
model: openai/gpt-4o-mini
api_key: os.environ/OPENAI_API_KEY
general_settings:
master_key: sk-1234
```
## 4. Start The Gateway
```bash
litellm --config config.yaml
```
You should see the proxy start on `http://0.0.0.0:4000`.
## 5. Send Your First Request
```bash
curl -X POST 'http://0.0.0.0:4000/chat/completions' \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer sk-1234' \
-d '{
"model": "gpt-4o-mini",
"messages": [
{"role": "user", "content": "Hello from LiteLLM Gateway"}
]
}'
```
## 6. Check The Response
If the request succeeds, the proxy returns `200 OK` with the same OpenAI-style response shape LiteLLM uses in the SDK.
The assistant text will be in:
```json
choices[0].message.content
```
It looks like this:
```json
{
"id": "chatcmpl-abc123",
"object": "chat.completion",
"created": 1677858242,
"model": "gpt-4o-mini",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "Hello! How can I help?"
},
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 12,
"completion_tokens": 9,
"total_tokens": 21
}
}
```
`id`, `created`, token counts, and message text will vary by request.
## 7. Add Keys And The UI
If you need virtual keys, spend tracking, or the admin UI, add a database next.
- Add `database_url` under `general_settings`
- Use [Virtual keys](/docs/proxy/virtual_keys) for key creation and budgets
- Use [Admin UI](/docs/proxy/ui) to manage models and keys
- Use the [Docker + Database tutorial](/docs/proxy/docker_quick_start) if you want a fuller setup
## 8. Pick Your Next Step
<NavigationCards
columns={3}
items={[
{
icon: "🎛️",
title: "Model Config",
description: "Add more models and gateway settings.",
to: "/docs/proxy/configs",
},
{
icon: "🔑",
title: "Virtual Keys",
description: "Create keys, budgets, and access controls.",
to: "/docs/proxy/virtual_keys",
},
{
icon: "📈",
title: "Add Logging",
description: "Capture logs, spend, and traces.",
to: "/docs/proxy/logging",
},
{
icon: "🔀",
title: "Load Balance",
description: "Route across deployments, regions, or providers.",
to: "/docs/proxy/load_balancing",
},
{
icon: "🛡️",
title: "Add Guardrails",
description: "Add safety checks and policy enforcement.",
to: "/docs/proxy/guardrails/quick_start",
},
{
icon: "🖥️",
title: "Connect SDKs",
description: "Point LiteLLM or OpenAI-compatible clients to the gateway.",
to: "/docs/providers/litellm_proxy",
},
{
icon: "📊",
title: "Reliability",
description: "Configure retries, fallbacks, and timeouts.",
to: "/docs/proxy/reliability",
},
]}
/>
## When To Use The SDK Path Instead
If you only need to call models from one application and do not need centralized auth or shared infrastructure, start with the [SDK Quickstart](/docs/learn/sdk_quickstart) instead.

188
docs/my-website/docs/learn/index.md
View file

@ -6,106 +6,148 @@ slug: /learn
import NavigationCards from '@site/src/components/NavigationCards';
LiteLLM gives you a single OpenAI-compatible interface to 100+ LLM providers. Use this page to find the right starting point for where you are.
LiteLLM gives you one OpenAI-compatible interface for 100+ LLM providers. Start with the path that matches your setup.
---
## Get Started
## Start Here
Set up your environment and make your first LLM call.
Pick one path first.
<NavigationCards
columns={2}
items={[
{
icon: "🐍",
title: "SDK Quickstart",
description: "Use LiteLLM directly in application code.",
listDescription: [
"Install",
"First request",
"Next SDK features",
],
to: "/docs/learn/sdk_quickstart",
},
{
icon: "🖥️",
title: "Gateway Quickstart",
description: "Run LiteLLM as a shared gateway.",
listDescription: [
"Start proxy",
"Add models and keys",
"Connect clients",
],
to: "/docs/learn/gateway_quickstart",
},
]}
/>
---
## Common Tasks
Jump to a specific task.
<NavigationCards
columns={3}
items={[
{
icon: "⚡",
title: "Set Up Your Environment",
description: "Get API keys and configure your environment before making your first LLM call.",
to: "/docs/tutorials/installation",
title: "Stream Responses",
description: "Return tokens as they are generated.",
to: "/docs/completion/stream",
},
{
icon: "🧰",
title: "Use Tools",
description: "Add function calling to your app.",
to: "/docs/completion/function_call",
},
{
icon: "🔀",
title: "Add Routing",
description: "Retries, fallbacks, and load balancing.",
to: "/docs/routing",
},
{
icon: "🔑",
title: "Set Up Keys",
description: "Gateway auth, virtual keys, and access control.",
to: "/docs/proxy/virtual_keys",
},
{
icon: "📈",
title: "Add Logging",
description: "Capture request logs and spend data.",
to: "/docs/proxy/logging",
},
{
icon: "🌐",
title: "Choose A Provider",
description: "Find provider-specific auth and params.",
to: "/docs/providers",
},
]}
/>
---
## Explore
Use these when you want examples or tool-specific walkthroughs.
<NavigationCards
columns={3}
items={[
{
icon: "🧪",
title: "Build an LLM Playground",
description: "Compare multiple LLM providers side by side in under 10 minutes with Streamlit.",
title: "Playground",
description: "Compare providers side by side.",
to: "/docs/tutorials/first_playground",
},
{
icon: "📞",
title: "Make Your First Call",
description: "Use the completion() function to call OpenAI, Anthropic, Vertex, or 100+ providers — same interface, one line of code.",
to: "/docs/#quick-start",
},
]}
/>
[View all getting started tutorials →](/docs/tutorials/getting_started)
---
## Guides
Focused references for SDK features and proxy configuration. Each guide is self-contained.
<NavigationCards
columns={3}
items={[
{
icon: "⚡",
title: "Completion Basics",
description: "Streaming, function calling, JSON mode, vision, audio, and more.",
to: "/docs/guides/completion_basics",
},
{
icon: "🎛️",
title: "Models & Configuration",
description: "Fine-tuned models, security settings, and adapters.",
to: "/docs/guides/models_configuration",
},
{
icon: "💰",
title: "Budgets & Cost",
description: "Set spend limits and track costs across teams and deployments.",
to: "/docs/guides/budgets_cost",
},
]}
/>
[View all guides →](/docs/guides)
---
## Tutorials
Step-by-step walkthroughs for integrating LiteLLM with external tools and services.
<NavigationCards
columns={3}
items={[
{
icon: "🤖",
title: "Agent SDKs & Frameworks",
description: "OpenAI Agents SDK, Claude Agent SDK, Google ADK, CopilotKit, and more.",
title: "Agent SDKs",
description: "OpenAI Agents SDK, Claude Agent SDK, ADK, and more.",
to: "/docs/agent_sdks",
},
{
icon: "🛠️",
title: "AI Coding Tools",
description: "Claude Code, Cursor, GitHub Copilot, Gemini CLI, OpenCode, and more.",
description: "Claude Code, Cursor, Copilot, Gemini CLI, and more.",
to: "/docs/ai_tools",
},
{
icon: "🐍",
title: "Provider Tutorials",
description: "Set up Azure OpenAI, HuggingFace, TogetherAI, local models, and more.",
to: "/docs/tutorials/provider_tutorials",
},
]}
/>
[View all tutorials →](/docs/tutorials)
---
## Production
## Docs Map
Add observability, access control, and safety to your deployment. See the [Production section](/docs/tutorials#production) in Tutorials for Observability & Safety, Proxy & Gateway, and more.
Use these when you already know the type of doc you want.
<NavigationCards
columns={3}
items={[
{
icon: "📚",
title: "Guides",
description: "Feature reference.",
to: "/docs/guides",
},
{
icon: "🛠️",
title: "Tutorials",
description: "Step-by-step integrations.",
to: "/docs/tutorials",
},
{
icon: "🌐",
title: "Providers",
description: "Provider-specific auth and params.",
to: "/docs/providers",
},
]}
/>
Not sure where to start? Use [SDK Quickstart](/docs/learn/sdk_quickstart) for app code or [Gateway Quickstart](/docs/learn/gateway_quickstart) for shared infrastructure.

137
docs/my-website/docs/learn/sdk_quickstart.md Normal file
View file

@ -0,0 +1,137 @@
---
title: SDK Quickstart
sidebar_label: SDK Quickstart
description: Make your first LiteLLM SDK call, then jump to the right docs for the next feature you need.
---
import NavigationCards from '@site/src/components/NavigationCards';
Use this path if you are integrating LiteLLM directly into application code.
## 1. Install LiteLLM
```bash
pip install litellm
```
## 2. Set Provider Credentials
Start with one provider and set its environment variables.
- OpenAI: `OPENAI_API_KEY`
- Anthropic: `ANTHROPIC_API_KEY`
- Azure OpenAI: `AZURE_API_KEY`, `AZURE_API_BASE`, `AZURE_API_VERSION`
- Bedrock: standard AWS credentials
- Vertex AI: `VERTEXAI_PROJECT`, `VERTEXAI_LOCATION`
If you have not picked a provider yet, browse [all supported providers](/docs/providers).
## 3. Make Your First Call
```python
from litellm import completion
import os
os.environ["OPENAI_API_KEY"] = "your-api-key"
response = completion(
model="openai/gpt-4o",
messages=[{"role": "user", "content": "Hello, how are you?"}],
)
print(response.choices[0].message.content)
```
## 4. Check The Response
The line below:
```python
print(response.choices[0].message.content)
```
prints the assistant text, for example:
```text
Hello! I'm doing well, thanks for asking.
```
The full response is an OpenAI-style `ModelResponse` object. It looks like this:
```json
{
"id": "chatcmpl-abc123",
"object": "chat.completion",
"created": 1677858242,
"model": "gpt-4o",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "Hello! I'm doing well, thanks for asking."
},
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 13,
"completion_tokens": 12,
"total_tokens": 25
}
}
```
`id`, `created`, token counts, and message text will vary by request. For the full output reference, see [completion output](/docs/completion/output).
Need more provider examples? See the main [Getting Started](/docs/#quick-start) page.
## 5. Pick Your Next Step
<NavigationCards
columns={3}
items={[
{
icon: "⚡",
title: "Stream Responses",
description: "Receive tokens incrementally with stream=True.",
to: "/docs/completion/stream",
},
{
icon: "🧰",
title: "Use Tools",
description: "Add function calling in a provider-agnostic way.",
to: "/docs/completion/function_call",
},
{
icon: "📦",
title: "Return JSON",
description: "Constrain responses to structured JSON output.",
to: "/docs/completion/json_mode",
},
{
icon: "🔀",
title: "Add Routing",
description: "Use retries, fallbacks, and load balancing in app code.",
to: "/docs/routing",
},
{
icon: "🌐",
title: "Choose A Provider",
description: "Find provider-specific auth, model naming, and params.",
to: "/docs/providers",
},
{
icon: "🖥️",
title: "Use Gateway",
description: "Send requests through a shared LiteLLM gateway.",
to: "/docs/learn/gateway_quickstart",
},
]}
/>
## When To Use Gateway Instead
Use LiteLLM Gateway if you need centralized auth, virtual keys, spend tracking, shared logging, or one OpenAI-compatible endpoint for multiple apps.
[Go to Gateway Quickstart →](/docs/learn/gateway_quickstart)

30
docs/my-website/docs/tutorials/index.md
View file

@ -7,7 +7,35 @@ import NavigationCards from '@site/src/components/NavigationCards';
**Tutorials** are step-by-step walkthroughs for integrating LiteLLM with external tools, frameworks, and services — or building complete end-to-end workflows.
> Need help with a specific LiteLLM SDK feature or proxy config? See [Guides →](/docs/guides)
> Need help choosing the right path before you start? See [Learn →](/docs/learn)
---
## Start Here
<NavigationCards
columns={3}
items={[
{
icon: "🐍",
title: "SDK Quickstart",
description: "Use this if you want to make your first LiteLLM call before following integration tutorials.",
to: "/docs/learn/sdk_quickstart",
},
{
icon: "🖥️",
title: "Gateway Quickstart",
description: "Use this if your tutorial depends on the LiteLLM proxy or shared infrastructure.",
to: "/docs/learn/gateway_quickstart",
},
{
icon: "📚",
title: "Need Feature References?",
description: "Use Guides when you know the capability you need and just want the doc for it.",
to: "/docs/guides",
},
]}
/>
---

158
docs/my-website/sidebars.js
View file

@ -17,11 +17,6 @@ const sidebars = {
integrationsSidebar: [
{ type: "doc", id: "integrations/index" },
{ type: "doc", id: "integrations/community" },
{
type: "doc",
id: "integrations/websearch_interception",
label: "Web Search Integration"
},
{
type: "category",
label: "Observability",
@ -1140,6 +1135,16 @@ const learnSidebar = {
learnSidebar: [
// ── Landing page ──────────────────────────────────────────────────
{ type: "doc", id: "learn/index", label: "Learn" },
{
type: "category",
label: "Start Here",
collapsible: true,
collapsed: false,
items: [
"learn/sdk_quickstart",
"learn/gateway_quickstart",
],
},
// ── Guides ────────────────────────────────────────────────────────
{
@ -1151,118 +1156,151 @@ const learnSidebar = {
items: [
{
type: "category",
label: "Completion Basics",
label: "Core Requests",
collapsible: true,
collapsed: false,
link: {
type: "generated-index",
title: "Completion Basics",
description: "Streaming, function calling, JSON mode, vision, audio, and more",
slug: "/guides/completion_basics"
title: "Core Requests",
description: "Streaming, batching, structured outputs, and reasoning behavior",
slug: "/guides/core_request_response_patterns"
},
items: [
"completion/stream",
"completion/function_call",
"completion/json_mode",
"completion/vision",
"completion/audio",
"completion/batching",
"completion/prefix",
"completion/predict_outputs",
"completion/provider_specific_params",
"completion/json_mode",
"reasoning_content",
"completion/drop_params",
"completion/model_alias",
"completion/mock_requests",
"completion/reliable_completions",
],
},
{
type: "category",
label: "Documents, Images & Messages",
label: "Tool Calling",
collapsible: true,
collapsed: true,
link: {
type: "generated-index",
title: "Documents, Images & Messages",
description: "Document understanding, image generation, message trimming, and sanitization",
slug: "/guides/input_output_handling"
title: "Tool Calling",
description: "Function calling, web tools, interception patterns, computer use, code interpreter, and tool-call hygiene",
slug: "/guides/tools_integrations"
},
items: [
"completion/document_understanding",
"completion/image_generation_chat",
"completion/message_trimming",
"completion/function_call",
"completion/web_search",
{
type: "doc",
id: "integrations/websearch_interception",
label: "Web Search Interception",
},
"completion/web_fetch",
"completion/computer_use",
"guides/code_interpreter",
"completion/message_sanitization",
],
},
{
type: "category",
label: "Prompt Optimization",
label: "Multimodal I/O",
collapsible: true,
collapsed: true,
link: {
type: "generated-index",
title: "Prompt Optimization",
description: "Prompt caching and prompt formatting",
slug: "/guides/prompt_optimization"
title: "Multimodal I/O",
description: "Vision, audio, PDFs, image generation, and video generation",
slug: "/guides/multimodal_io"
},
items: [
"completion/vision",
"completion/audio",
"completion/document_understanding",
"completion/image_generation_chat",
"proxy/veo_video_generation",
],
},
{
type: "category",
label: "Retrieval & Knowledge",
collapsible: true,
collapsed: true,
link: {
type: "generated-index",
title: "Retrieval & Knowledge",
description: "Vector stores, file search, citations, and knowledge-base routing",
slug: "/guides/retrieval_knowledge"
},
items: [
"completion/knowledgebase",
],
},
{
type: "category",
label: "Prompts & Context",
collapsible: true,
collapsed: true,
link: {
type: "generated-index",
title: "Prompts & Context",
description: "Prompt caching, trimming, formatting, assistant prefill, and predicted outputs",
slug: "/guides/prompts_context"
},
items: [
"completion/prefix",
"completion/predict_outputs",
"completion/message_trimming",
"completion/prompt_caching",
"completion/prompt_formatting",
],
},
{
type: "category",
label: "AI Capabilities",
label: "Compatibility & Extensibility",
collapsible: true,
collapsed: true,
link: {
type: "generated-index",
title: "AI Capabilities",
description: "Web search, code interpreter, knowledge base, and more",
slug: "/guides/ai_capabilities"
},
items: [
"completion/web_search",
"completion/web_fetch",
"completion/computer_use",
"completion/knowledgebase",
"guides/code_interpreter",
"proxy/veo_video_generation",
],
},
{
type: "category",
label: "Models & Configuration",
collapsible: true,
collapsed: true,
link: {
type: "generated-index",
title: "Models & Configuration",
description: "Fine-tuned models, security settings, and adapters",
slug: "/guides/models_configuration"
title: "Compatibility & Extensibility",
description: "Provider-specific params, model aliases, fine-tuned models, and adapters",
slug: "/guides/compatibility_extensibility"
},
items: [
"completion/provider_specific_params",
"completion/drop_params",
"completion/model_alias",
"guides/finetuned_models",
"guides/security_settings",
"extras/creating_adapters",
],
},
{
type: "category",
label: "Budgets & Cost",
label: "Reliability, Testing & Spend",
collapsible: true,
collapsed: true,
link: {
type: "generated-index",
title: "Budgets & Cost",
description: "Set spend limits and track costs",
slug: "/guides/budgets_cost"
title: "Reliability, Testing & Spend",
description: "Retries, fallbacks, mock responses, and budget controls",
slug: "/guides/reliability_testing_spend"
},
items: [
"completion/mock_requests",
"completion/reliable_completions",
"budget_manager",
],
},
{
type: "category",
label: "Security & Network",
collapsible: true,
collapsed: true,
link: {
type: "generated-index",
title: "Security & Network",
description: "SSL, custom CA bundles, HTTP proxy settings, and per-service verification",
slug: "/guides/security_network"
},
items: [
"guides/security_settings",
],
},
],
},