mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-10 22:41:41 +00:00
Updated release notes (#12594)
* docs(index.md): cleanup docs * docs(index.md): add more highlights * docs(model_hub.md): cleanup doc
This commit is contained in:
parent
b757bb68f9
commit
78baeae6d8
9 changed files with 112 additions and 0 deletions
39
docs/my-website/docs/proxy/model_hub.md
Normal file
39
docs/my-website/docs/proxy/model_hub.md
Normal file
|
|
@ -0,0 +1,39 @@
|
|||
import Image from '@theme/IdealImage';
|
||||
import Tabs from '@theme/Tabs';
|
||||
import TabItem from '@theme/TabItem';
|
||||
|
||||
# Model Hub
|
||||
|
||||
Tell developers what models are available on the proxy.
|
||||
|
||||
This feature is **available in v1.74.3-stable and above**.
|
||||
|
||||
## Overview
|
||||
|
||||
Admin can select models to expose on public model hub -> Users can go to the public url (`/ui/model_hub_table`) and see available models.
|
||||
|
||||
<Image img={require('../../img/final_public_model_hub_view.png')} />
|
||||
|
||||
## How to use
|
||||
|
||||
### 1. Go to the Admin UI
|
||||
|
||||
Navigate to the Model Hub page in the Admin UI (`PROXY_BASE_URL/ui/?login=success&page=model-hub-table`)
|
||||
|
||||
<Image img={require('../../img/model_hub_admin_view.png')} />
|
||||
|
||||
### 2. Select the models you want to expose
|
||||
|
||||
Click on `Make Public` and select the models you want to expose.
|
||||
|
||||
<Image img={require('../../img/make_public_modal.png')} />
|
||||
|
||||
### 3. Confirm the changes
|
||||
|
||||
<Image img={require('../../img/make_public_modal_confirmation.png')} />
|
||||
|
||||
### 4. Success!
|
||||
|
||||
Go to the public url (`PROXY_BASE_URL/ui/model_hub_table`) and see available models.
|
||||
|
||||
<Image img={require('../../img/final_public_model_hub_view.png')} />
|
||||
BIN
docs/my-website/img/final_public_model_hub_view.png
Normal file
BIN
docs/my-website/img/final_public_model_hub_view.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 301 KiB |
BIN
docs/my-website/img/make_public_modal.png
Normal file
BIN
docs/my-website/img/make_public_modal.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 180 KiB |
BIN
docs/my-website/img/make_public_modal_confirmation.png
Normal file
BIN
docs/my-website/img/make_public_modal_confirmation.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 130 KiB |
BIN
docs/my-website/img/model_hub_admin_view.png
Normal file
BIN
docs/my-website/img/model_hub_admin_view.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 357 KiB |
BIN
docs/my-website/img/model_hub_public.png
Normal file
BIN
docs/my-website/img/model_hub_public.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 336 KiB |
BIN
docs/my-website/img/release_notes/mcp_access_groups.png
Normal file
BIN
docs/my-website/img/release_notes/mcp_access_groups.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 144 KiB |
|
|
@ -43,6 +43,78 @@ pip install litellm==1.74.3rc2
|
|||
|
||||
---
|
||||
|
||||
## Key Highlights
|
||||
|
||||
- **MCP: Model Access Groups** - Add mcp servers to access groups, for easily managing access to users and teams.
|
||||
- **MCP: Tool Cost Tracking** - Set prices for each MCP tool.
|
||||
- **Model Hub v2** - New OSS Model Hub for telling developers what models are available on the proxy.
|
||||
- **Bytez** - New LLM API Provider.
|
||||
- **Dashscope API** - Call Alibaba's qwen models via new Dashscope API Provider.
|
||||
|
||||
---
|
||||
|
||||
## MCP Gateway: Model Access Groups
|
||||
|
||||
<Image
|
||||
img={require('../../img/release_notes/mcp_access_groups.png')}
|
||||
style={{width: '80%', display: 'block', margin: '0'}}
|
||||
/>
|
||||
|
||||
<br/>
|
||||
|
||||
v1.74.3-stable adds support for adding MCP servers to access groups, this makes it **easier for Proxy Admins** to manage access to MCP servers across users and teams.
|
||||
|
||||
For **developers**, this means you can now connect to multiple MCP servers by passing the access group name in the `x-mcp-servers` header.
|
||||
|
||||
Read more [here](https://docs.litellm.ai/docs/mcp#grouping-mcps-access-groups)
|
||||
|
||||
---
|
||||
|
||||
## MCP Gateway: Tool Cost Tracking
|
||||
|
||||
<Image
|
||||
img={require('../../img/release_notes/mcp_tool_cost_tracking.png')}
|
||||
style={{width: '80%', display: 'block', margin: '0'}}
|
||||
/>
|
||||
|
||||
<br/>
|
||||
|
||||
This release adds cost tracking for MCP tool calls. This is great for **Proxy Admins** giving MCP access to developers as you can now attribute MCP tool call costs to specific LiteLLM keys and teams.
|
||||
|
||||
You can set:
|
||||
- **Uniform server cost**: Set a uniform cost for all tools from a server
|
||||
- **Individual tool cost**: Define individual costs for specific tools (e.g., search_tool costs $10, get_weather costs $5).
|
||||
- **Dynamic costs**: For use cases where you want to set costs based on the MCP's response, you can write a custom post mcp call hook to parse responses and set costs dynamically.
|
||||
|
||||
[Get started](https://docs.litellm.ai/docs/mcp#mcp-cost-tracking)
|
||||
|
||||
---
|
||||
|
||||
## Model Hub v2
|
||||
|
||||
<Image
|
||||
img={require('../../img/release_notes/model_hub_v2.png')}
|
||||
style={{width: '100%', display: 'block', margin: '2rem auto'}}
|
||||
/>
|
||||
|
||||
<br/>
|
||||
|
||||
v1.74.3-stable introduces a new OSS Model Hub for telling developers what models are available on the proxy.
|
||||
|
||||
This is great for **Proxy Admins** as you can now tell developers what models are available on the proxy.
|
||||
|
||||
This improves on the previous model hub by enabling:
|
||||
- The ability to show **Developers** models, even if they don't have a LiteLLM key.
|
||||
- The ability for **Proxy Admins** to select specific models to be public on the model hub.
|
||||
- Improved search and filtering capabilities:
|
||||
- search for models by partial name (e.g. `xai grok-4`)
|
||||
- filter by provider and feature (e.g. 'vision' models)
|
||||
- sort by cost (e.g. cheapest vision model from OpenAI)
|
||||
|
||||
[Get started](../../docs/proxy/model_hub)
|
||||
|
||||
---
|
||||
|
||||
|
||||
## New Models / Updated Models
|
||||
|
||||
|
|
|
|||
|
|
@ -166,6 +166,7 @@ const sidebars = {
|
|||
"proxy/ui",
|
||||
"proxy/admin_ui_sso",
|
||||
"proxy/custom_root_ui",
|
||||
"proxy/model_hub",
|
||||
"proxy/self_serve",
|
||||
"proxy/public_teams",
|
||||
"tutorials/scim_litellm",
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue