mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-06 08:16:43 +00:00
Litellm stable docs 06 14 2025 p2 (#11738)
* docs: track which items need docs * docs(anthropic.md): add tool_choice="none" to docs * docs: add docs for new anthropic + perplexity features * docs: cleanup mistral reasoning docs * docs: add links to docs * docs(index.md): update docs * docs: refactor to add a new 'integrations' tab to docs * refactor(docs/): create separate tab for integrations make it easier to highlight new integrations * docs: sort sidebar * docs: update
This commit is contained in:
Krish Dholakia
GitHub
parent
4efdfc00ab
commit
d15664c8e6
11 changed files with 300 additions and 220 deletions
|
|
@ -7,6 +7,8 @@ For companies that need SSO, user management and professional support for LiteLL
|
|||
Get free 7-day trial key [here](https://www.litellm.ai/#trial)
|
||||
:::
|
||||
|
||||
## Enterprise Features
|
||||
|
||||
Includes all enterprise features.
|
||||
|
||||
<Image img={require('../img/enterprise_vs_oss.png')} />
|
||||
|
|
@ -18,32 +20,13 @@ This covers:
|
|||
- [**Enterprise Features**](./proxy/enterprise)
|
||||
- ✅ **Feature Prioritization**
|
||||
- ✅ **Custom Integrations**
|
||||
- ✅ **Professional Support - Dedicated discord + slack**
|
||||
- ✅ **Professional Support - Dedicated Slack/Teams channel**
|
||||
|
||||
|
||||
Deployment Options:
|
||||
## Self-Hosted
|
||||
|
||||
**Self-Hosted**
|
||||
1. Manage Yourself - you can deploy our Docker Image or build a custom image from our pip package, and manage your own infrastructure. In this case, we would give you a license key + provide support via a dedicated support channel.
|
||||
Manage Yourself - you can deploy our Docker Image or build a custom image from our pip package, and manage your own infrastructure. In this case, we would give you a license key + provide support via a dedicated support channel.
|
||||
|
||||
2. We Manage - you give us subscription access on your AWS/Azure/GCP account, and we manage the deployment.
|
||||
|
||||
**Managed**
|
||||
|
||||
You can use our cloud product where we setup a dedicated instance for you.
|
||||
|
||||
## Frequently Asked Questions
|
||||
|
||||
### SLA's + Professional Support
|
||||
|
||||
Professional Support can assist with LLM/Provider integrations, deployment, upgrade management, and LLM Provider troubleshooting. We can’t solve your own infrastructure-related issues but we will guide you to fix them.
|
||||
|
||||
- 1 hour for Sev0 issues - 100% production traffic is failing
|
||||
- 6 hours for Sev1 - <100% production traffic is failing
|
||||
- 24h for Sev2-Sev3 between 7am – 7pm PT (Monday through Saturday) - setup issues e.g. Redis working on our end, but not on your infrastructure.
|
||||
- 72h SLA for patching vulnerabilities in the software.
|
||||
|
||||
**We can offer custom SLAs** based on your needs and the severity of the issue
|
||||
|
||||
### What’s the cost of the Self-Managed Enterprise edition?
|
||||
|
||||
|
|
@ -58,8 +41,72 @@ You just deploy [our docker image](https://docs.litellm.ai/docs/proxy/deploy) an
|
|||
LITELLM_LICENSE="eyJ..."
|
||||
```
|
||||
|
||||
No data leaves your environment.
|
||||
**No data leaves your environment.**
|
||||
|
||||
|
||||
## Hosted LiteLLM Proxy
|
||||
|
||||
LiteLLM maintains the proxy, so you can focus on your core products.
|
||||
|
||||
We provide a dedicated proxy for your team, and manage the infrastructure.
|
||||
|
||||
### **Status**: GA
|
||||
|
||||
Our proxy is already used in production by customers.
|
||||
|
||||
See our status page for [**live reliability**](https://status.litellm.ai/)
|
||||
|
||||
### **Benefits**
|
||||
- **No Maintenance, No Infra**: We'll maintain the proxy, and spin up any additional infrastructure (e.g.: separate server for spend logs) to make sure you can load balance + track spend across multiple LLM projects.
|
||||
- **Reliable**: Our hosted proxy is tested on 1k requests per second, making it reliable for high load.
|
||||
- **Secure**: LiteLLM is SOC-2 Type 2 and ISO 27001 certified, to make sure your data is as secure as possible.
|
||||
|
||||
### Supported data regions for LiteLLM Cloud
|
||||
|
||||
You can find [supported data regions litellm here](../docs/data_security#supported-data-regions-for-litellm-cloud)
|
||||
|
||||
|
||||
## Frequently Asked Questions
|
||||
|
||||
### SLA's + Professional Support
|
||||
|
||||
Professional Support can assist with LLM/Provider integrations, deployment, upgrade management, and LLM Provider troubleshooting. We can’t solve your own infrastructure-related issues but we will guide you to fix them.
|
||||
|
||||
- 1 hour for Sev0 issues - 100% production traffic is failing
|
||||
- 6 hours for Sev1 - <100% production traffic is failing
|
||||
- 24h for Sev2-Sev3 between 7am – 7pm PT (Monday through Saturday) - setup issues e.g. Redis working on our end, but not on your infrastructure.
|
||||
- 72h SLA for patching vulnerabilities in the software.
|
||||
|
||||
**We can offer custom SLAs** based on your needs and the severity of the issue
|
||||
|
||||
## Data Security / Legal / Compliance FAQs
|
||||
|
||||
[Data Security / Legal / Compliance FAQs](./data_security.md)
|
||||
[Data Security / Legal / Compliance FAQs](./data_security.md)
|
||||
|
||||
|
||||
### Pricing
|
||||
|
||||
Pricing is based on usage. We can figure out a price that works for your team, on the call.
|
||||
|
||||
[**Contact Us to learn more**](https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat)
|
||||
|
||||
|
||||
|
||||
## **Screenshots**
|
||||
|
||||
### 1. Create keys
|
||||
|
||||
<Image img={require('../img/litellm_hosted_ui_create_key.png')} />
|
||||
|
||||
### 2. Add Models
|
||||
|
||||
<Image img={require('../img/litellm_hosted_ui_add_models.png')}/>
|
||||
|
||||
### 3. Track spend
|
||||
|
||||
<Image img={require('../img/litellm_hosted_usage_dashboard.png')} />
|
||||
|
||||
|
||||
### 4. Configure load balancing
|
||||
|
||||
<Image img={require('../img/litellm_hosted_ui_router.png')} />
|
||||
|
|
|
|||
|
|
@ -1,66 +0,0 @@
|
|||
import Image from '@theme/IdealImage';
|
||||
|
||||
# Hosted LiteLLM Proxy
|
||||
|
||||
LiteLLM maintains the proxy, so you can focus on your core products.
|
||||
|
||||
## [**Get Onboarded**](https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat)
|
||||
|
||||
This is in alpha. Schedule a call with us, and we'll give you a hosted proxy within 30 minutes.
|
||||
|
||||
[**🚨 Schedule Call**](https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat)
|
||||
|
||||
### **Status**: Alpha
|
||||
|
||||
Our proxy is already used in production by customers.
|
||||
|
||||
See our status page for [**live reliability**](https://status.litellm.ai/)
|
||||
|
||||
### **Benefits**
|
||||
- **No Maintenance, No Infra**: We'll maintain the proxy, and spin up any additional infrastructure (e.g.: separate server for spend logs) to make sure you can load balance + track spend across multiple LLM projects.
|
||||
- **Reliable**: Our hosted proxy is tested on 1k requests per second, making it reliable for high load.
|
||||
- **Secure**: LiteLLM is currently undergoing SOC-2 compliance, to make sure your data is as secure as possible.
|
||||
|
||||
## Data Privacy & Security
|
||||
|
||||
You can find our [data privacy & security policy for cloud litellm here](../docs/data_security#litellm-cloud)
|
||||
|
||||
## Supported data regions for LiteLLM Cloud
|
||||
|
||||
You can find [supported data regions litellm here](../docs/data_security#supported-data-regions-for-litellm-cloud)
|
||||
|
||||
### Pricing
|
||||
|
||||
Pricing is based on usage. We can figure out a price that works for your team, on the call.
|
||||
|
||||
[**🚨 Schedule Call**](https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat)
|
||||
|
||||
## **Screenshots**
|
||||
|
||||
### 1. Create keys
|
||||
|
||||
<Image img={require('../img/litellm_hosted_ui_create_key.png')} />
|
||||
|
||||
### 2. Add Models
|
||||
|
||||
<Image img={require('../img/litellm_hosted_ui_add_models.png')}/>
|
||||
|
||||
### 3. Track spend
|
||||
|
||||
<Image img={require('../img/litellm_hosted_usage_dashboard.png')} />
|
||||
|
||||
|
||||
### 4. Configure load balancing
|
||||
|
||||
<Image img={require('../img/litellm_hosted_ui_router.png')} />
|
||||
|
||||
#### [**🚨 Schedule Call**](https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat)
|
||||
|
||||
## Feature List
|
||||
|
||||
- Easy way to add/remove models
|
||||
- 100% uptime even when models are added/removed
|
||||
- custom callback webhooks
|
||||
- your domain name with HTTPS
|
||||
- Ability to create/delete User API keys
|
||||
- Reasonable set monthly cost
|
||||
5
docs/my-website/docs/integrations/index.md
Normal file
5
docs/my-website/docs/integrations/index.md
Normal file
|
|
@ -0,0 +1,5 @@
|
|||
# Integrations
|
||||
|
||||
This section covers integrations with various tools and services that can be used with LiteLLM (either Proxy or SDK).
|
||||
|
||||
Click into each section to learn more about the integrations.
|
||||
|
|
@ -664,6 +664,63 @@ response = completion(
|
|||
)
|
||||
```
|
||||
|
||||
### Disable Tool Calling
|
||||
|
||||
You can disable tool calling by setting the `tool_choice` to `"none"`.
|
||||
|
||||
<Tabs>
|
||||
<TabItem value="sdk" label="SDK">
|
||||
|
||||
```python
|
||||
from litellm import completion
|
||||
|
||||
response = completion(
|
||||
model="anthropic/claude-3-opus-20240229",
|
||||
messages=messages,
|
||||
tools=tools,
|
||||
tool_choice="none",
|
||||
)
|
||||
|
||||
```
|
||||
</TabItem>
|
||||
<TabItem value="proxy" label="Proxy">
|
||||
|
||||
1. Setup config.yaml
|
||||
|
||||
```yaml
|
||||
model_list:
|
||||
- model_name: anthropic-claude-model
|
||||
litellm_params:
|
||||
model: anthropic/claude-3-opus-20240229
|
||||
api_key: os.environ/ANTHROPIC_API_KEY
|
||||
```
|
||||
|
||||
2. Start proxy
|
||||
|
||||
```bash
|
||||
litellm --config /path/to/config.yaml
|
||||
```
|
||||
|
||||
3. Test it!
|
||||
|
||||
Replace `anything` with your LiteLLM Proxy Virtual Key, if [setup](../proxy/virtual_keys).
|
||||
|
||||
```bash
|
||||
curl http://0.0.0.0:4000/v1/chat/completions \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "Authorization: Bearer anything" \
|
||||
-d '{
|
||||
"model": "anthropic-claude-model",
|
||||
"messages": [{"role": "user", "content": "Who won the World Cup in 2022?"}],
|
||||
"tools": [{"type": "mcp", "server_label": "deepwiki", "server_url": "https://mcp.deepwiki.com/mcp", "require_approval": "never"}],
|
||||
"tool_choice": "none"
|
||||
}'
|
||||
```
|
||||
</TabItem>
|
||||
</Tabs>
|
||||
|
||||
|
||||
|
||||
### MCP Tool Calling
|
||||
|
||||
Here's how to use MCP tool calling with Anthropic:
|
||||
|
|
|
|||
|
|
@ -8,9 +8,16 @@ Call your custom torch-serve / internal LLM APIs via LiteLLM
|
|||
- For modifying incoming/outgoing calls on proxy, [go here](../proxy/call_hooks.md)
|
||||
:::
|
||||
|
||||
Supported Routes:
|
||||
- `/v1/chat/completions` -> `litellm.completion`
|
||||
- `/v1/completions` -> `litellm.text_completion`
|
||||
- `/v1/embeddings` -> `litellm.embedding`
|
||||
- `/v1/images/generations` -> `litellm.image_generation`
|
||||
|
||||
|
||||
## Quick Start
|
||||
|
||||
```python
|
||||
```python showLineNumbers
|
||||
import litellm
|
||||
from litellm import CustomLLM, completion, get_llm_provider
|
||||
|
||||
|
|
|
|||
|
|
@ -205,11 +205,13 @@ assert isinstance(
|
|||
)
|
||||
```
|
||||
|
||||
## Reasoning Capabilities (Magistral Models)
|
||||
## Reasoning
|
||||
|
||||
Mistral's Magistral models support advanced reasoning capabilities that allow the model to think step-by-step before providing answers. LiteLLM provides seamless integration with these reasoning features through OpenAI-compatible parameters.
|
||||
Mistral does not directly support reasoning, instead it recommends a specific [system prompt](https://docs.mistral.ai/capabilities/reasoning/) to use with their magistral models. By setting the `reasoning_effort` parameter, LiteLLM will prepend the system prompt to the request.
|
||||
|
||||
### Supported Magistral Models
|
||||
If an existing system message is provided, LiteLLM will send both as a list of system messages (you can verify this by enabling `litellm._turn_on_debug()`).
|
||||
|
||||
### Supported Models
|
||||
|
||||
| Model Name | Function Call |
|
||||
|----------------|--------------------------------------------------------------|
|
||||
|
|
|
|||
|
|
@ -39,6 +39,69 @@ for chunk in response:
|
|||
print(chunk)
|
||||
```
|
||||
|
||||
## Reasoning Effort
|
||||
|
||||
Requires v1.72.6+
|
||||
|
||||
:::info
|
||||
|
||||
See full guide on Reasoning with LiteLLM [here](../reasoning_content)
|
||||
|
||||
:::
|
||||
|
||||
You can set the reasoning effort by setting the `reasoning_effort` parameter.
|
||||
|
||||
<Tabs>
|
||||
<TabItem value="sdk" label="SDK">
|
||||
|
||||
```python
|
||||
from litellm import completion
|
||||
import os
|
||||
|
||||
os.environ['PERPLEXITYAI_API_KEY'] = ""
|
||||
response = completion(
|
||||
model="perplexity/sonar-reasoning",
|
||||
messages=messages,
|
||||
reasoning_effort="high"
|
||||
)
|
||||
print(response)
|
||||
```
|
||||
</TabItem>
|
||||
<TabItem value="proxy" label="Proxy">
|
||||
|
||||
1. Setup config.yaml
|
||||
|
||||
```yaml
|
||||
model_list:
|
||||
- model_name: perplexity-sonar-reasoning-model
|
||||
litellm_params:
|
||||
model: perplexity/sonar-reasoning
|
||||
api_key: os.environ/PERPLEXITYAI_API_KEY
|
||||
```
|
||||
|
||||
2. Start proxy
|
||||
|
||||
```bash
|
||||
litellm --config /path/to/config.yaml
|
||||
```
|
||||
|
||||
3. Test it!
|
||||
|
||||
Replace `anything` with your LiteLLM Proxy Virtual Key, if [setup](../proxy/virtual_keys).
|
||||
|
||||
```bash
|
||||
curl http://0.0.0.0:4000/v1/chat/completions \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "Authorization: Bearer anything" \
|
||||
-d '{
|
||||
"model": "perplexity-sonar-reasoning-model",
|
||||
"messages": [{"role": "user", "content": "Who won the World Cup in 2022?"}],
|
||||
"reasoning_effort": "high"
|
||||
}'
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
</Tabs>
|
||||
|
||||
## Supported Models
|
||||
All models listed here https://docs.perplexity.ai/docs/model-cards are supported. Just do `model=perplexity/<model-name>`.
|
||||
|
|
|
|||
|
|
@ -40,7 +40,7 @@ LiteLLM will standardize the `reasoning_content` in the response and `thinking_b
|
|||
## Quick Start
|
||||
|
||||
<Tabs>
|
||||
<TabItem value="anthropic" label="Anthropic">
|
||||
<TabItem value="sdk" label="SDK">
|
||||
|
||||
```python showLineNumbers
|
||||
from litellm import completion
|
||||
|
|
@ -58,25 +58,6 @@ response = completion(
|
|||
print(response.choices[0].message.content)
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
<TabItem value="mistral" label="Mistral">
|
||||
|
||||
```python showLineNumbers
|
||||
from litellm import completion
|
||||
import os
|
||||
|
||||
os.environ["MISTRAL_API_KEY"] = ""
|
||||
|
||||
response = completion(
|
||||
model="mistral/magistral-medium-2506",
|
||||
messages=[
|
||||
{"role": "user", "content": "What is 15 multiplied by 7? Show your reasoning."},
|
||||
],
|
||||
reasoning_effort="medium",
|
||||
)
|
||||
print(response.choices[0].message.content)
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
<TabItem value="proxy" label="PROXY">
|
||||
|
||||
|
|
|
|||
|
|
@ -120,16 +120,16 @@ const config = {
|
|||
label: 'Docs',
|
||||
},
|
||||
{
|
||||
sidebarId: 'tutorialSidebar',
|
||||
sidebarId: 'integrationsSidebar',
|
||||
position: 'left',
|
||||
label: 'Enterprise',
|
||||
to: "docs/enterprise"
|
||||
label: 'Integrations',
|
||||
to: "docs/integrations"
|
||||
},
|
||||
{
|
||||
sidebarId: 'tutorialSidebar',
|
||||
position: 'left',
|
||||
label: 'Hosted',
|
||||
to: "docs/hosted"
|
||||
label: 'Enterprise',
|
||||
to: "docs/enterprise"
|
||||
},
|
||||
{ to: '/release_notes', label: 'Release Notes', position: 'left' },
|
||||
{
|
||||
|
|
|
|||
|
|
@ -65,47 +65,47 @@ This version is not out yet.
|
|||
|
||||
### Updated Models
|
||||
#### Bugs
|
||||
- **Watsonx**
|
||||
- **[Watsonx](../../docs/providers/watsonx)**
|
||||
- Ignore space id on Watsonx deployments (throws json errors) - [PR](https://github.com/BerriAI/litellm/pull/11527)
|
||||
- **Ollama**
|
||||
- **[Ollama](../../docs/providers/ollama)**
|
||||
- Set tool call id for streaming calls - [PR](https://github.com/BerriAI/litellm/pull/11528)
|
||||
- **Gemini (VertexAI + Google AI Studio)**
|
||||
- **Gemini ([VertexAI](../../docs/providers/vertex) + [Google AI Studio](../../docs/providers/gemini))**
|
||||
- Fix tool call indexes - [PR](https://github.com/BerriAI/litellm/pull/11558)
|
||||
- Handle empty string for arguments in function calls - [PR](https://github.com/BerriAI/litellm/pull/11601)
|
||||
- Add audio/ogg mime type support when inferring from file url’s - [PR](https://github.com/BerriAI/litellm/pull/11635)
|
||||
- **Custom LLM**
|
||||
- **[Custom LLM](../../docs/providers/custom_llm_server)**
|
||||
- Fix passing api_base, api_key, litellm_params_dict to custom_llm embedding methods - [PR](https://github.com/BerriAI/litellm/pull/11450) s/o [ElefHead](https://github.com/ElefHead)
|
||||
- **Huggingface**
|
||||
- **[Huggingface](../../docs/providers/huggingface)**
|
||||
- Add /chat/completions to endpoint url when missing - [PR](https://github.com/BerriAI/litellm/pull/11630)
|
||||
- **Deepgram**
|
||||
- **[Deepgram](../../docs/providers/deepgram)**
|
||||
- Support async httpx calls - [PR](https://github.com/BerriAI/litellm/pull/11641)
|
||||
- **Anthropic**
|
||||
- **[Anthropic](../../docs/providers/anthropic)**
|
||||
- Append prefix (if set) to assistant content start - [PR](https://github.com/BerriAI/litellm/pull/11719)
|
||||
|
||||
#### Features
|
||||
- **VertexAI**
|
||||
- **[VertexAI](../../docs/providers/vertex)**
|
||||
- Support vertex credentials set via env var on passthrough - [PR](https://github.com/BerriAI/litellm/pull/11527)
|
||||
- Support for choosing ‘global’ region when model is only available there - [PR](https://github.com/BerriAI/litellm/pull/11566)
|
||||
- Anthropic passthrough cost calculation + token tracking - [PR](https://github.com/BerriAI/litellm/pull/11611)
|
||||
- Support ‘global’ vertex region on passthrough - [PR](https://github.com/BerriAI/litellm/pull/11661)
|
||||
- **Anthropic**
|
||||
- ‘none’ tool choice param support - [PR](https://github.com/BerriAI/litellm/pull/11695)
|
||||
- **Perplexity**
|
||||
- Add ‘reasoning_effort’ support - [PR](https://github.com/BerriAI/litellm/pull/11562)
|
||||
- **Mistral**
|
||||
- Add mistral reasoning support - [PR](https://github.com/BerriAI/litellm/pull/11642)
|
||||
- **SGLang**
|
||||
- **[Anthropic](../../docs/providers/anthropic)**
|
||||
- ‘none’ tool choice param support - [PR](https://github.com/BerriAI/litellm/pull/11695), [Get Started](../../docs/providers/anthropic#disable-tool-calling)
|
||||
- **[Perplexity](../../docs/providers/perplexity)**
|
||||
- Add ‘reasoning_effort’ support - [PR](https://github.com/BerriAI/litellm/pull/11562), [Get Started](../../docs/providers/perplexity#reasoning-effort)
|
||||
- **[Mistral](../../docs/providers/mistral)**
|
||||
- Add mistral reasoning support - [PR](https://github.com/BerriAI/litellm/pull/11642), [Get Started](../../docs/providers/mistral#reasoning)
|
||||
- **[SGLang](../../docs/providers/openai_compatible)**
|
||||
- Map context window exceeded error for proper handling - [PR](https://github.com/BerriAI/litellm/pull/11575/)
|
||||
- **Deepgram**
|
||||
- **[Deepgram](../../docs/providers/deepgram)**
|
||||
- Provider specific params support - [PR](https://github.com/BerriAI/litellm/pull/11638)
|
||||
- **Azure**
|
||||
- **[Azure](../../docs/providers/azure)**
|
||||
- Return content safety filter results - [PR](https://github.com/BerriAI/litellm/pull/11655)
|
||||
---
|
||||
|
||||
## LLM API Endpoints
|
||||
|
||||
#### Bugs
|
||||
- **Chat Completion**
|
||||
- **[Chat Completion](../../docs/completion/input)**
|
||||
- Streaming - Ensure consistent ‘created’ across chunks - [PR](https://github.com/BerriAI/litellm/pull/11528)
|
||||
#### Features
|
||||
- **MCP**
|
||||
|
|
@ -113,11 +113,11 @@ This version is not out yet.
|
|||
- Add permission management for MCP List + Call Tool operations - [PR](https://github.com/BerriAI/litellm/pull/11682), [Docs](../../docs/mcp#-mcp-permission-management)
|
||||
- Streamable HTTP server support - [PR](https://github.com/BerriAI/litellm/pull/11628), [PR](https://github.com/BerriAI/litellm/pull/11645), [Docs](../../docs/mcp#using-your-mcp)
|
||||
- Use Experimental dedicated Rest endpoints for list, calling MCP tools - [PR](https://github.com/BerriAI/litellm/pull/11684)
|
||||
- **Responses API**
|
||||
- NEW API Endpoint - List input items - [PR](https://github.com/BerriAI/litellm/pull/11602)
|
||||
- **[Responses API](../../docs/response_api)**
|
||||
- NEW API Endpoint - List input items - [PR](https://github.com/BerriAI/litellm/pull/11602) [NEEDS DOCS]
|
||||
- Background mode for OpenAI + Azure OpenAI - [PR](https://github.com/BerriAI/litellm/pull/11640)
|
||||
- Langfuse/other Logging support on responses api requests - [PR](https://github.com/BerriAI/litellm/pull/11685)
|
||||
- **Chat Completions**
|
||||
- **[Chat Completions](../../docs/completion/input)**
|
||||
- Bridge for Responses API - allows calling codex-mini via `/chat/completions` and `/v1/messages` - [PR](https://github.com/BerriAI/litellm/pull/11632), [PR](https://github.com/BerriAI/litellm/pull/11685)
|
||||
|
||||
|
||||
|
|
@ -126,9 +126,9 @@ This version is not out yet.
|
|||
## Spend Tracking
|
||||
|
||||
#### Bugs
|
||||
- **End Users**
|
||||
- **[End Users](../../docs/proxy/customers)**
|
||||
- Update enduser spend and budget reset date based on budget duration - [PR](https://github.com/BerriAI/litellm/pull/8460) (s/o [laurien16](https://github.com/laurien16))
|
||||
- **Custom Pricing**
|
||||
- **[Custom Pricing](../../docs/proxy/custom_pricing)**
|
||||
- Convert scientific notation str to int - [PR](https://github.com/BerriAI/litellm/pull/11655)
|
||||
|
||||
---
|
||||
|
|
@ -136,11 +136,11 @@ This version is not out yet.
|
|||
## Management Endpoints / UI
|
||||
|
||||
#### Bugs
|
||||
- **Users**
|
||||
- **[Users](../../docs/proxy/users)**
|
||||
- `/user/info` - fix passing user with `+` in user id
|
||||
- Add admin-initiated password reset flow - [PR](https://github.com/BerriAI/litellm/pull/11618)
|
||||
- Fixes default user settings UI rendering error - [PR](https://github.com/BerriAI/litellm/pull/11674)
|
||||
- **Budgets**
|
||||
- **[Budgets](../../docs/proxy/users)**
|
||||
- Correct success message when new user budget is created - [PR](https://github.com/BerriAI/litellm/pull/11608)
|
||||
|
||||
#### Features
|
||||
|
|
@ -163,15 +163,15 @@ This version is not out yet.
|
|||
## Logging / Guardrails Integrations
|
||||
|
||||
#### Bugs
|
||||
- **Arize**
|
||||
- **[Arize](../../docs/observability/arize_integration)**
|
||||
- Change space_key header to space_id - [PR](https://github.com/BerriAI/litellm/pull/11595) (s/o [vanities](https://github.com/vanities))
|
||||
- **Prometheus**
|
||||
- **[Prometheus](../../docs/proxy/prometheus)**
|
||||
- Fix total requests increment - [PR](https://github.com/BerriAI/litellm/pull/11718)
|
||||
|
||||
#### Features
|
||||
- **Lasso Guardrails**
|
||||
- **[Lasso Guardrails](../../docs/proxy/guardrails/lasso_security)**
|
||||
- [NEW] Lasso Guardrails support - [PR](https://github.com/BerriAI/litellm/pull/11565)
|
||||
- **Users**
|
||||
- **[Users](../../docs/proxy/users)**
|
||||
- New `organizations` param on `/user/new` - allows adding users to orgs on creation - [PR](https://github.com/BerriAI/litellm/pull/11572/files)
|
||||
- **Prevent double logging when using bridge logic** - [PR](https://github.com/BerriAI/litellm/pull/11687)
|
||||
|
||||
|
|
@ -180,11 +180,11 @@ This version is not out yet.
|
|||
## Performance / Reliability Improvements
|
||||
|
||||
#### Bugs
|
||||
- **Tag based routing**
|
||||
- **[Tag based routing](../../docs/proxy/tag_routing)**
|
||||
- Do not consider ‘default’ models when request specifies a tag - [PR](https://github.com/BerriAI/litellm/pull/11454) (s/o [thiagosalvatore](https://github.com/thiagosalvatore))
|
||||
|
||||
#### Features
|
||||
- **Caching**
|
||||
- **[Caching](../../docs/caching/all_caches)**
|
||||
- New optional ‘litellm[caching]’ pip install for adding disk cache dependencies - [PR](https://github.com/BerriAI/litellm/pull/11600)
|
||||
|
||||
---
|
||||
|
|
|
|||
|
|
@ -14,7 +14,57 @@
|
|||
/** @type {import('@docusaurus/plugin-content-docs').SidebarsConfig} */
|
||||
const sidebars = {
|
||||
// // By default, Docusaurus generates a sidebar from the docs folder structure
|
||||
|
||||
integrationsSidebar: [
|
||||
{ type: "doc", id: "integrations/index" },
|
||||
{
|
||||
type: "category",
|
||||
label: "Observability",
|
||||
items: [
|
||||
{
|
||||
type: "autogenerated",
|
||||
dirName: "observability"
|
||||
}
|
||||
],
|
||||
},
|
||||
{
|
||||
type: "category",
|
||||
label: "[Beta] Guardrails",
|
||||
items: [
|
||||
"proxy/guardrails/quick_start",
|
||||
...[
|
||||
"proxy/guardrails/aim_security",
|
||||
"proxy/guardrails/aporia_api",
|
||||
"proxy/guardrails/bedrock",
|
||||
"proxy/guardrails/lasso_security",
|
||||
"proxy/guardrails/guardrails_ai",
|
||||
"proxy/guardrails/lakera_ai",
|
||||
"proxy/guardrails/pangea",
|
||||
"proxy/guardrails/pii_masking_v2",
|
||||
"proxy/guardrails/secret_detection",
|
||||
"proxy/guardrails/custom_guardrail",
|
||||
"proxy/guardrails/prompt_injection",
|
||||
].sort(),
|
||||
],
|
||||
},
|
||||
{
|
||||
type: "category",
|
||||
label: "Alerting & Monitoring",
|
||||
items: [
|
||||
"proxy/prometheus",
|
||||
"proxy/alerting",
|
||||
"proxy/pagerduty"
|
||||
].sort()
|
||||
},
|
||||
{
|
||||
type: "category",
|
||||
label: "[Beta] Prompt Management",
|
||||
items: [
|
||||
"proxy/prompt_management",
|
||||
"proxy/custom_prompt_management"
|
||||
].sort()
|
||||
},
|
||||
|
||||
],
|
||||
// But you can create a sidebar manually
|
||||
tutorialSidebar: [
|
||||
{ type: "doc", id: "index" }, // NEW
|
||||
|
|
@ -139,31 +189,10 @@ const sidebars = {
|
|||
items: [
|
||||
"proxy/logging",
|
||||
"proxy/logging_spec",
|
||||
"proxy/team_logging",
|
||||
"proxy/prometheus",
|
||||
"proxy/alerting",
|
||||
"proxy/pagerduty"],
|
||||
},
|
||||
{
|
||||
type: "category",
|
||||
label: "[Beta] Guardrails",
|
||||
items: [
|
||||
"proxy/guardrails/quick_start",
|
||||
...[
|
||||
"proxy/guardrails/aim_security",
|
||||
"proxy/guardrails/aporia_api",
|
||||
"proxy/guardrails/bedrock",
|
||||
"proxy/guardrails/lasso_security",
|
||||
"proxy/guardrails/guardrails_ai",
|
||||
"proxy/guardrails/lakera_ai",
|
||||
"proxy/guardrails/pangea",
|
||||
"proxy/guardrails/pii_masking_v2",
|
||||
"proxy/guardrails/secret_detection",
|
||||
"proxy/guardrails/custom_guardrail",
|
||||
"proxy/guardrails/prompt_injection",
|
||||
].sort(),
|
||||
"proxy/team_logging"
|
||||
],
|
||||
},
|
||||
|
||||
{
|
||||
type: "category",
|
||||
label: "Secret Managers",
|
||||
|
|
@ -455,14 +484,7 @@ const sidebars = {
|
|||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
type: "category",
|
||||
label: "[Beta] Prompt Management",
|
||||
items: [
|
||||
"proxy/prompt_management",
|
||||
"proxy/custom_prompt_management"
|
||||
],
|
||||
},
|
||||
|
||||
{
|
||||
type: "category",
|
||||
label: "Load Testing",
|
||||
|
|
@ -473,44 +495,6 @@ const sidebars = {
|
|||
"load_test_rpm",
|
||||
]
|
||||
},
|
||||
{
|
||||
type: "category",
|
||||
label: "Logging & Observability",
|
||||
items: [
|
||||
"observability/agentops_integration",
|
||||
"observability/langfuse_integration",
|
||||
"observability/langfuse_otel_integration",
|
||||
"observability/lunary_integration",
|
||||
"observability/deepeval_integration",
|
||||
"observability/mlflow",
|
||||
"observability/gcs_bucket_integration",
|
||||
"observability/langsmith_integration",
|
||||
"observability/literalai_integration",
|
||||
"observability/opentelemetry_integration",
|
||||
"observability/logfire_integration",
|
||||
"observability/argilla",
|
||||
"observability/arize_integration",
|
||||
"observability/phoenix_integration",
|
||||
"debugging/local_debugging",
|
||||
"observability/raw_request_response",
|
||||
"observability/custom_callback",
|
||||
"observability/humanloop",
|
||||
"observability/scrub_data",
|
||||
"observability/braintrust",
|
||||
"observability/sentry",
|
||||
"observability/lago",
|
||||
"observability/helicone_integration",
|
||||
"observability/openmeter",
|
||||
"observability/promptlayer_integration",
|
||||
"observability/wandb_integration",
|
||||
"observability/slack_integration",
|
||||
"observability/athina_integration",
|
||||
"observability/greenscale_integration",
|
||||
"observability/supabase_integration",
|
||||
`observability/telemetry`,
|
||||
"observability/opik_integration",
|
||||
],
|
||||
},
|
||||
{
|
||||
type: "category",
|
||||
label: "Tutorials",
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue