scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-10 03:28:53 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 84

New stable release - release notes (#8148)

Browse source 
* docs(v1.59.8-stable): add release note

* docs(index.md): cleanup new stable release, release notes
This commit is contained in:
Krish Dholakia 2025-01-31 10:02:59 -08:00 • committed by GitHub
parent b8b927f23b
commit a699000a4b
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
18 changed files with 394 additions and 12 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

31
docs/my-website/docs/image_variations.md Normal file
View file

@ -0,0 +1,31 @@
# [BETA] Image Variations
OpenAI's `/image/variations` endpoint is now supported.
## Quick Start
```python
from litellm import image_variation
import os
# set env vars
os.environ["OPENAI_API_KEY"] = ""
os.environ["TOPAZ_API_KEY"] = ""
# openai call
response = image_variation(
model="dall-e-2", image=image_url
)
# topaz call
response = image_variation(
model="topaz/Standard V2", image=image_url
)
print(response)
```
## Supported Providers
- OpenAI
- Topaz

61
docs/my-website/docs/providers/bedrock.md
View file

@ -2,7 +2,7 @@ import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
# AWS Bedrock
ALL Bedrock models (Anthropic, Meta, Mistral, Amazon, etc.) are Supported
ALL Bedrock models (Anthropic, Meta, Deepseek, Mistral, Amazon, etc.) are Supported
| Property | Details |
|-------|-------|
@ -1331,6 +1331,65 @@ Here's an example of using a bedrock model with LiteLLM. For a complete list, re
| Mistral 7B Instruct | `completion(model='bedrock/mistral.mistral-7b-instruct-v0:2', messages=messages)` | `os.environ['AWS_ACCESS_KEY_ID']`, `os.environ['AWS_SECRET_ACCESS_KEY']`, `os.environ['AWS_REGION_NAME']` |
| Mixtral 8x7B Instruct | `completion(model='bedrock/mistral.mixtral-8x7b-instruct-v0:1', messages=messages)` | `os.environ['AWS_ACCESS_KEY_ID']`, `os.environ['AWS_SECRET_ACCESS_KEY']`, `os.environ['AWS_REGION_NAME']` |
### Deepseek
Native Deepseek support will be [available soon](https://github.com/BerriAI/litellm/pull/8132)
Till then, use this:
<Tabs>
<TabItem value="sdk" label="SDK">
```python
import litellm
response = litellm.completion(
model="bedrock/meta.DeepSeek-R1-Distill-Llama-8B",
model_id="provisioned-model-arn",
messages=[{"content": "Hello, how are you?", "role": "user"}]
)
```
</TabItem>
<TabItem value="proxy" label="PROXY">
1. Setup config.yaml
```yaml
model_list:
- model_name: bedrock-deepseek
litellm_params:
model: bedrock/meta.DeepSeek-R1-Distill-Llama-8B
model_id: provisioned-model-arn
```
2. Start proxy server
```bash
litellm --config config.yaml
# RUNNING on http://0.0.0.0:4000
```
3. Test it!
```bash
curl -L -X POST 'http://0.0.0.0:4000/v1/chat/completions' \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer sk-1234' \
-d '{
"model": "bedrock-deepseek",
"messages": [
{
"role": "user",
"content": "What'\''s the weather like in Boston today 236589?"
}
]
}'
```
</TabItem>
</Tabs>
## Bedrock Embedding
### API keys

2
docs/my-website/docs/providers/deepseek.md
View file

@ -76,7 +76,7 @@ resp = completion(
)
print(
resp.choices[0].message.provider_specific_fields["reasoning_content"]
resp.choices[0].message.reasoning_content
)
```

70
docs/my-website/docs/providers/ollama.md
View file

@ -238,6 +238,76 @@ Ollama supported models: https://github.com/ollama/ollama
| Nous-Hermes 13B | `completion(model='ollama/nous-hermes:13b', messages, api_base="http://localhost:11434", stream=True)` |
| Wizard Vicuna Uncensored | `completion(model='ollama/wizard-vicuna', messages, api_base="http://localhost:11434", stream=True)` |
### JSON Schema support
<Tabs>
<TabItem value="sdk" label="SDK">
```python
from litellm import completion
response = completion(
model="ollama_chat/deepseek-r1",
messages=[{ "content": "respond in 20 words. who are you?","role": "user"}],
response_format={"type": "json_schema", "json_schema": {"schema": {"type": "object", "properties": {"name": {"type": "string"}}}}},
)
print(response)
```
</TabItem>
<TabItem value="proxy" label="PROXY">
1. Setup config.yaml
```yaml
model_list:
- model_name: "deepseek-r1"
litellm_params:
model: "ollama_chat/deepseek-r1"
api_base: "http://localhost:11434"
```
2. Start proxy
```bash
litellm --config /path/to/config.yaml
# RUNNING ON http://0.0.0.0:4000
```
3. Test it!
```python
from pydantic import BaseModel
from openai import OpenAI
client = OpenAI(
api_key="anything", # 👈 PROXY KEY (can be anything, if master_key not set)
base_url="http://0.0.0.0:4000" # 👈 PROXY BASE URL
)
class Step(BaseModel):
explanation: str
output: str
class MathReasoning(BaseModel):
steps: list[Step]
final_answer: str
completion = client.beta.chat.completions.parse(
model="deepseek-r1",
messages=[
{"role": "system", "content": "You are a helpful math tutor. Guide the user through the solution step by step."},
{"role": "user", "content": "how can I solve 8x + 7 = -23"}
],
response_format=MathReasoning,
)
math_reasoning = completion.choices[0].message.parsed
```
</TabItem>
</Tabs>
## Ollama Vision Models
| Model Name | Function Call |
|------------------|--------------------------------------|

27
docs/my-website/docs/providers/topaz.md Normal file
View file

@ -0,0 +1,27 @@
# Topaz
| Property | Details |
|-------|-------|
| Description | Professional-grade photo and video editing powered by AI. |
| Provider Route on LiteLLM | `topaz/` |
| Provider Doc | [Topaz ↗](https://www.topazlabs.com/enhance-api) |
| API Endpoint for Provider | https://api.topazlabs.com |
| Supported OpenAI Endpoints | `/image/variations` |
## Quick Start
```python
from litellm import image_variation
import os
os.environ["TOPAZ_API_KEY"] = ""
response = image_variation(
model="topaz/Standard V2", image=image_url
)
```
## Supported OpenAI Params
- `response_format`
- `size` (widthxheight)

11
docs/my-website/docs/proxy/health.md
View file

@ -314,6 +314,17 @@ Example Response:
"I'm alive!"
```
## `/health/services`
Use this admin-only endpoint to check if a connected service (datadog/slack/langfuse/etc.) is healthy.
```bash
curl -L -X GET 'http://0.0.0.0:4000/health/services?service=datadog' -H 'Authorization: Bearer sk-1234'
```
[**API Reference**](https://litellm-api.up.railway.app/#/health/health_services_endpoint_health_services_get)
## Advanced - Call specific models
To check health of specific models, here's how to call them:

15
docs/my-website/docs/proxy/virtual_keys.md
View file

@ -559,6 +559,21 @@ curl 'http://localhost:4000/key/sk-1234/regenerate' \
[**👉 API REFERENCE DOCS**](https://litellm-api.up.railway.app/#/key%20management/regenerate_key_fn_key__key__regenerate_post)
### Temporary Budget Increase
Use the `/key/update` endpoint to increase the budget of an existing key.
```bash
curl -L -X POST 'http://localhost:4000/key/update' \
-H 'Authorization: Bearer sk-1234' \
-H 'Content-Type: application/json' \
-d '{"key": "sk-b3Z3Lqdb_detHXSUp4ol4Q", "temp_budget_increase": 100, "temp_budget_expiry": "10d"}'
```
[API Reference](https://litellm-api.up.railway.app/#/key%20management/update_key_fn_key_update_post)
### Restricting Key Generation
Use this to control who can generate keys. Useful when letting others create keys on the UI.

2
docs/my-website/release_notes/v1.55.10/index.md
View file

@ -6,7 +6,7 @@ authors:
- name: Krrish Dholakia
title: CEO, LiteLLM
url: https://www.linkedin.com/in/krish-d/
image_url: https://media.licdn.com/dms/image/v2/C5103AQHYMXJfHTf4Ng/profile-displayphoto-shrink_800_800/profile-displayphoto-shrink_800_800/0/1517455593871?e=1741824000&v=beta&t=udmat6jS-s3EQZp1DTykf7NZmf-3sefD_I9B1aMjE5Y
image_url: https://media.licdn.com/dms/image/v2/D4D03AQGrlsJ3aqpHmQ/profile-displayphoto-shrink_400_400/B4DZSAzgP7HYAg-/0/1737327772964?e=1743638400&v=beta&t=39KOXMUFedvukiWWVPHf3qI45fuQD7lNglICwN31DrI
- name: Ishaan Jaffer
title: CTO, LiteLLM
url: https://www.linkedin.com/in/reffajnaahsi/

2
docs/my-website/release_notes/v1.55.8-stable/index.md
View file

@ -6,7 +6,7 @@ authors:
- name: Krrish Dholakia
title: CEO, LiteLLM
url: https://www.linkedin.com/in/krish-d/
image_url: https://media.licdn.com/dms/image/v2/C5103AQHYMXJfHTf4Ng/profile-displayphoto-shrink_800_800/profile-displayphoto-shrink_800_800/0/1517455593871?e=1741824000&v=beta&t=udmat6jS-s3EQZp1DTykf7NZmf-3sefD_I9B1aMjE5Y
image_url: https://media.licdn.com/dms/image/v2/D4D03AQGrlsJ3aqpHmQ/profile-displayphoto-shrink_400_400/B4DZSAzgP7HYAg-/0/1737327772964?e=1743638400&v=beta&t=39KOXMUFedvukiWWVPHf3qI45fuQD7lNglICwN31DrI
- name: Ishaan Jaffer
title: CTO, LiteLLM
url: https://www.linkedin.com/in/reffajnaahsi/

2
docs/my-website/release_notes/v1.56.1/index.md
View file

@ -6,7 +6,7 @@ authors:
- name: Krrish Dholakia
title: CEO, LiteLLM
url: https://www.linkedin.com/in/krish-d/
image_url: https://media.licdn.com/dms/image/v2/C5103AQHYMXJfHTf4Ng/profile-displayphoto-shrink_800_800/profile-displayphoto-shrink_800_800/0/1517455593871?e=1741824000&v=beta&t=udmat6jS-s3EQZp1DTykf7NZmf-3sefD_I9B1aMjE5Y
image_url: https://media.licdn.com/dms/image/v2/D4D03AQGrlsJ3aqpHmQ/profile-displayphoto-shrink_400_400/B4DZSAzgP7HYAg-/0/1737327772964?e=1743638400&v=beta&t=39KOXMUFedvukiWWVPHf3qI45fuQD7lNglICwN31DrI
- name: Ishaan Jaffer
title: CTO, LiteLLM
url: https://www.linkedin.com/in/reffajnaahsi/

2
docs/my-website/release_notes/v1.56.3/index.md
View file

@ -6,7 +6,7 @@ authors:
- name: Krrish Dholakia
title: CEO, LiteLLM
url: https://www.linkedin.com/in/krish-d/
image_url: https://media.licdn.com/dms/image/v2/C5103AQHYMXJfHTf4Ng/profile-displayphoto-shrink_800_800/profile-displayphoto-shrink_800_800/0/1517455593871?e=1741824000&v=beta&t=udmat6jS-s3EQZp1DTykf7NZmf-3sefD_I9B1aMjE5Y
image_url: https://media.licdn.com/dms/image/v2/D4D03AQGrlsJ3aqpHmQ/profile-displayphoto-shrink_400_400/B4DZSAzgP7HYAg-/0/1737327772964?e=1743638400&v=beta&t=39KOXMUFedvukiWWVPHf3qI45fuQD7lNglICwN31DrI
- name: Ishaan Jaffer
title: CTO, LiteLLM
url: https://www.linkedin.com/in/reffajnaahsi/

2
docs/my-website/release_notes/v1.56.4/index.md
View file

@ -6,7 +6,7 @@ authors:
- name: Krrish Dholakia
title: CEO, LiteLLM
url: https://www.linkedin.com/in/krish-d/
image_url: https://media.licdn.com/dms/image/v2/C5103AQHYMXJfHTf4Ng/profile-displayphoto-shrink_800_800/profile-displayphoto-shrink_800_800/0/1517455593871?e=1741824000&v=beta&t=udmat6jS-s3EQZp1DTykf7NZmf-3sefD_I9B1aMjE5Y
image_url: https://media.licdn.com/dms/image/v2/D4D03AQGrlsJ3aqpHmQ/profile-displayphoto-shrink_400_400/B4DZSAzgP7HYAg-/0/1737327772964?e=1743638400&v=beta&t=39KOXMUFedvukiWWVPHf3qI45fuQD7lNglICwN31DrI
- name: Ishaan Jaffer
title: CTO, LiteLLM
url: https://www.linkedin.com/in/reffajnaahsi/

2
docs/my-website/release_notes/v1.57.3/index.md
View file

@ -6,7 +6,7 @@ authors:
- name: Krrish Dholakia
title: CEO, LiteLLM
url: https://www.linkedin.com/in/krish-d/
image_url: https://media.licdn.com/dms/image/v2/C5103AQHYMXJfHTf4Ng/profile-displayphoto-shrink_800_800/profile-displayphoto-shrink_800_800/0/1517455593871?e=1741824000&v=beta&t=udmat6jS-s3EQZp1DTykf7NZmf-3sefD_I9B1aMjE5Y
image_url: https://media.licdn.com/dms/image/v2/D4D03AQGrlsJ3aqpHmQ/profile-displayphoto-shrink_400_400/B4DZSAzgP7HYAg-/0/1737327772964?e=1743638400&v=beta&t=39KOXMUFedvukiWWVPHf3qI45fuQD7lNglICwN31DrI
- name: Ishaan Jaffer
title: CTO, LiteLLM
url: https://www.linkedin.com/in/reffajnaahsi/

2
docs/my-website/release_notes/v1.57.7/index.md
View file

@ -6,7 +6,7 @@ authors:
- name: Krrish Dholakia
title: CEO, LiteLLM
url: https://www.linkedin.com/in/krish-d/
image_url: https://media.licdn.com/dms/image/v2/C5103AQHYMXJfHTf4Ng/profile-displayphoto-shrink_800_800/profile-displayphoto-shrink_800_800/0/1517455593871?e=1741824000&v=beta&t=udmat6jS-s3EQZp1DTykf7NZmf-3sefD_I9B1aMjE5Y
image_url: https://media.licdn.com/dms/image/v2/D4D03AQGrlsJ3aqpHmQ/profile-displayphoto-shrink_400_400/B4DZSAzgP7HYAg-/0/1737327772964?e=1743638400&v=beta&t=39KOXMUFedvukiWWVPHf3qI45fuQD7lNglICwN31DrI
- name: Ishaan Jaffer
title: CTO, LiteLLM
url: https://www.linkedin.com/in/reffajnaahsi/

2
docs/my-website/release_notes/v1.57.8-stable/index.md
View file

@ -6,7 +6,7 @@ authors:
- name: Krrish Dholakia
title: CEO, LiteLLM
url: https://www.linkedin.com/in/krish-d/
image_url: https://media.licdn.com/dms/image/v2/C5103AQHYMXJfHTf4Ng/profile-displayphoto-shrink_800_800/profile-displayphoto-shrink_800_800/0/1517455593871?e=1741824000&v=beta&t=udmat6jS-s3EQZp1DTykf7NZmf-3sefD_I9B1aMjE5Y
image_url: https://media.licdn.com/dms/image/v2/D4D03AQGrlsJ3aqpHmQ/profile-displayphoto-shrink_400_400/B4DZSAzgP7HYAg-/0/1737327772964?e=1743638400&v=beta&t=39KOXMUFedvukiWWVPHf3qI45fuQD7lNglICwN31DrI
- name: Ishaan Jaffer
title: CTO, LiteLLM
url: https://www.linkedin.com/in/reffajnaahsi/

2
docs/my-website/release_notes/v1.59.0/index.md
View file

@ -6,7 +6,7 @@ authors:
- name: Krrish Dholakia
title: CEO, LiteLLM
url: https://www.linkedin.com/in/krish-d/
image_url: https://media.licdn.com/dms/image/v2/C5103AQHYMXJfHTf4Ng/profile-displayphoto-shrink_800_800/profile-displayphoto-shrink_800_800/0/1517455593871?e=1741824000&v=beta&t=udmat6jS-s3EQZp1DTykf7NZmf-3sefD_I9B1aMjE5Y
image_url: https://media.licdn.com/dms/image/v2/D4D03AQGrlsJ3aqpHmQ/profile-displayphoto-shrink_400_400/B4DZSAzgP7HYAg-/0/1737327772964?e=1743638400&v=beta&t=39KOXMUFedvukiWWVPHf3qI45fuQD7lNglICwN31DrI
- name: Ishaan Jaffer
title: CTO, LiteLLM
url: https://www.linkedin.com/in/reffajnaahsi/

161
docs/my-website/release_notes/v1.59.8-stable/index.md Normal file
View file

@ -0,0 +1,161 @@
---
title: v1.59.8-stable
slug: v1.59.8-stable
date: 2025-01-31T10:00:00
authors:
- name: Krrish Dholakia
title: CEO, LiteLLM
url: https://www.linkedin.com/in/krish-d/
image_url: https://media.licdn.com/dms/image/v2/D4D03AQGrlsJ3aqpHmQ/profile-displayphoto-shrink_400_400/B4DZSAzgP7HYAg-/0/1737327772964?e=1743638400&v=beta&t=39KOXMUFedvukiWWVPHf3qI45fuQD7lNglICwN31DrI
- name: Ishaan Jaffer
title: CTO, LiteLLM
url: https://www.linkedin.com/in/reffajnaahsi/
image_url: https://media.licdn.com/dms/image/v2/D4D03AQGiM7ZrUwqu_Q/profile-displayphoto-shrink_800_800/profile-displayphoto-shrink_800_800/0/1675971026692?e=1741824000&v=beta&t=eQnRdXPJo4eiINWTZARoYTfqh064pgZ-E21pQTSy8jc
tags: [admin ui, logging, db schema]
hide_table_of_contents: false
---
import Image from '@theme/IdealImage';
# v1.59.8-stable
:::info
Get a 7 day free trial for LiteLLM Enterprise [here](https://litellm.ai/#trial).
**no call needed**
:::
## New Models / Updated Models
1. New OpenAI `/image/variations` endpoint BETA support [Docs](../../docs/image_variations)
2. Topaz API support on OpenAI `/image/variations` BETA endpoint [Docs](../../docs/providers/topaz)
3. Deepseek - r1 support w/ reasoning_content ([Deepseek API](../../docs/providers/deepseek#reasoning-models), [Vertex AI](../../docs/providers/vertex.md#model-garden), [Bedrock](../../docs/providers/bedrock#deepseek))
4. Azure - Add azure o1 pricing [See Here](https://github.com/BerriAI/litellm/blob/b8b927f23bc336862dacb89f59c784a8d62aaa15/model_prices_and_context_window.json#L952)
5. Anthropic - handle `-latest` tag in model for cost calculation
6. Gemini-2.0-flash-thinking - add model pricing (it’s 0.0) [See Here](https://github.com/BerriAI/litellm/blob/b8b927f23bc336862dacb89f59c784a8d62aaa15/model_prices_and_context_window.json#L3393)
7. Bedrock - add stability sd3 model pricing [See Here](https://github.com/BerriAI/litellm/blob/b8b927f23bc336862dacb89f59c784a8d62aaa15/model_prices_and_context_window.json#L6814) (s/o [Marty Sullivan](https://github.com/marty-sullivan))
8. Bedrock - add us.amazon.nova-lite-v1:0 to model cost map [See Here](https://github.com/BerriAI/litellm/blob/b8b927f23bc336862dacb89f59c784a8d62aaa15/model_prices_and_context_window.json#L5619)
9. TogetherAI - add new together_ai llama3.3 models [See Here](https://github.com/BerriAI/litellm/blob/b8b927f23bc336862dacb89f59c784a8d62aaa15/model_prices_and_context_window.json#L6985)
## LLM Translation
1. LM Studio -> fix async embedding call
2. Gpt 4o models - fix response_format translation
3. Bedrock nova - expand supported document types to include .md, .csv, etc. [Start Here](../../docs/providers/bedrock#usage---pdf--document-understanding)
4. Bedrock - docs on IAM role based access for bedrock - [Start Here](https://docs.litellm.ai/docs/providers/bedrock#sts-role-based-auth)
5. Bedrock - cache IAM role credentials when used
6. Google AI Studio (`gemini/`) - support gemini 'frequency_penalty' and 'presence_penalty'
7. Azure O1 - fix model name check
8. WatsonX - ZenAPIKey support for WatsonX [Docs](../../docs/providers/watsonx.md)
9. Ollama Chat - support json schema response format [Start Here](../../docs/providers/ollama.md#json-schema-support)
10. Bedrock - return correct bedrock status code and error message if error during streaming
11. Anthropic - Supported nested json schema on anthropic calls
12. OpenAI - `metadata` param preview support
1. SDK - enable via `litellm.enable_preview_features = True`
2. PROXY - enable via `litellm_settings::enable_preview_features: true`
13. Replicate - retry completion response on status=processing
## Spend Tracking Improvements
1. Bedrock - QA asserts all bedrock regional models have same `supported_` as base model
2. Bedrock - fix bedrock converse cost tracking w/ region name specified
3. Spend Logs reliability fix - when `user` passed in request body is int instead of string
4. Ensure ‘base_model’ cost tracking works across all endpoints
5. Fixes for Image generation cost tracking
6. Anthropic - fix anthropic end user cost tracking
7. JWT / OIDC Auth - add end user id tracking from jwt auth
## Management Endpoints / UI
1. allows team member to become admin post-add (ui + endpoints)
2. New edit/delete button for updating team membership on UI
3. If team admin - show all team keys
4. Model Hub - clarify cost of models is per 1m tokens
5. Invitation Links - fix invalid url generated
6. New - SpendLogs Table Viewer - allows proxy admin to view spend logs on UI
1. New spend logs - allow proxy admin to ‘opt in’ to logging request/response in spend logs table - enables easier abuse detection
2. Show country of origin in spend logs
3. Add pagination + filtering by key name/team name
7. `/key/delete` - allow team admin to delete team keys
8. Internal User ‘view’ - fix spend calculation when team selected
9. Model Analytics is now on Free
10. Usage page - shows days when spend = 0, and round spend on charts to 2 sig figs
11. Public Teams - allow admins to expose teams for new users to ‘join’ on UI - [Start Here](https://docs.litellm.ai/docs/proxy/public_teams)
12. Guardrails
1. set/edit guardrails on a virtual key
2. Allow setting guardrails on a team
3. Set guardrails on team create + edit page
13. Support temporary budget increases on `/key/update` - new `temp_budget_increase` and `temp_budget_expiry` fields - [Start Here](../../docs/proxy/virtual_keys.md#temporary-budget-increase)
14. Support writing new key alias to AWS Secret Manager - on key rotation [Start Here](../../docs/secret.md#aws-secret-manager)
## Helm
1. add securityContext and pull policy values to migration job (s/o https://github.com/Hexoplon)
2. allow specifying envVars on values.yaml
3. new helm lint test
## Logging / Guardrail Integrations
1. Log the used prompt when prompt management used. [Start Here](../../docs/proxy/prompt_management.md)
2. Support s3 logging with team alias prefixes - [Start Here](https://docs.litellm.ai/docs/proxy/logging#team-alias-prefix-in-object-key)
3. Prometheus [Start Here](../../docs/proxy/prometheus.md)
1. fix litellm_llm_api_time_to_first_token_metric not populating for bedrock models
2. emit remaining team budget metric on regular basis (even when call isn’t made) - allows for more stable metrics on Grafana/etc.
3. add key and team level budget metrics
4. emit `litellm_overhead_latency_metric`
5. Emit `litellm_team_budget_reset_at_metric` and `litellm_api_key_budget_remaining_hours_metric`
4. Datadog - support logging spend tags to Datadog. [Start Here](../../docs/proxy/enterprise.md#tracking-spend-for-custom-tags)
5. Langfuse - fix logging request tags, read from standard logging payload
6. GCS - don’t truncate payload on logging
7. New GCS Pub/Sub logging support [Start Here](https://docs.litellm.ai/docs/proxy/logging#google-cloud-storage---pubsub-topic)
8. Add AIM Guardrails support [Start Here](../../docs/proxy/guardrails/aim_security.md)
## Security
1. New Enterprise SLA for patching security vulnerabilities. [See Here](../../docs/enterprise.md#slas--professional-support)
2. Hashicorp - support using vault namespace for TLS auth. [Start Here](../../docs/secret.md#hashicorp-vault)
3. Azure - DefaultAzureCredential support
## Health Checks
1. Cleanup pricing-only model names from wildcard route list - prevent bad health checks
2. Allow specifying a health check model for wildcard routes - https://docs.litellm.ai/docs/proxy/health#wildcard-routes
3. New ‘health_check_timeout ‘ param with default 1min upperbound to prevent bad model from health check to hang and cause pod restarts. [Start Here](../../docs/proxy/health.md#health-check-timeout)
4. Datadog - add data dog service health check + expose new `/health/services` endpoint. [Start Here](../../docs/proxy/health.md#healthservices)
## Performance / Reliability improvements
1. 3x increase in RPS - moving to orjson for reading request body
2. LLM Routing speedup - using cached get model group info
3. SDK speedup - using cached get model info helper - reduces CPU work to get model info
4. Proxy speedup - only read request body 1 time per request
5. Infinite loop detection scripts added to codebase
6. Bedrock - pure async image transformation requests
7. Cooldowns - single deployment model group if 100% calls fail in high traffic - prevents an o1 outage from impacting other calls
8. Response Headers - return
1. `x-litellm-timeout`
2. `x-litellm-attempted-retries`
3. `x-litellm-overhead-duration-ms`
4. `x-litellm-response-duration-ms`
9. ensure duplicate callbacks are not added to proxy
10. Requirements.txt - bump certifi version
## General Proxy Improvements
1. JWT / OIDC Auth - new `enforce_rbac` param,allows proxy admin to prevent any unmapped yet authenticated jwt tokens from calling proxy. [Start Here](../../docs/proxy/token_auth.md#enforce-role-based-access-control-rbac)
2. fix custom openapi schema generation for customized swagger’s
3. Request Headers - support reading `x-litellm-timeout` param from request headers. Enables model timeout control when using Vercel’s AI SDK + LiteLLM Proxy. [Start Here](../../docs/proxy/request_headers.md#litellm-headers)
4. JWT / OIDC Auth - new `role` based permissions for model authentication. [See Here](https://docs.litellm.ai/docs/proxy/jwt_auth_arch)
## Complete Git Diff
This is the diff between v1.57.8-stable and v1.59.8-stable.
Use this to see the changes in the codebase.
[**Git Diff**](https://github.com/BerriAI/litellm/compare/v1.57.8-stable...v1.59.8-stable)

10
docs/my-website/sidebars.js
View file

@ -206,6 +206,7 @@ const sidebars = {
"providers/perplexity",
"providers/friendliai",
"providers/galadriel",
"providers/topaz",
"providers/groq",
"providers/github",
"providers/deepseek",
@ -278,7 +279,14 @@ const sidebars = {
},
"text_completion",
"embedding/supported_embedding",
"image_generation",
{
type: "category",
label: "Image",
items: [
"image_generation",
"image_variations",
]
},
{
type: "category",
label: "Audio",