mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-11 03:38:38 +00:00
Add sync models GitHub documentation with Loom video and cross-references (#15191)
- Add comprehensive sync_models_github.md with API endpoints and examples - Include Loom video tutorial for Admin UI sync process - Add cross-references from model_management.md, cost_tracking.md, and ui.md - Provide both manual and automated sync options - Include Python SDK usage examples
This commit is contained in:
parent
8095de506a
commit
7a176804f3
4 changed files with 83 additions and 0 deletions
|
|
@ -8,6 +8,10 @@ Track spend for keys, users, and teams across 100+ LLMs.
|
|||
|
||||
LiteLLM automatically tracks spend for all known models. See our [model cost map](https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json)
|
||||
|
||||
:::tip Keep Pricing Data Updated
|
||||
[Sync model pricing data from GitHub](../sync_models_github.md) to ensure accurate cost tracking.
|
||||
:::
|
||||
|
||||
### How to Track Spend with LiteLLM
|
||||
|
||||
**Step 1**
|
||||
|
|
|
|||
|
|
@ -19,6 +19,10 @@ model_list:
|
|||
|
||||
Retrieve detailed information about each model listed in the `/model/info` endpoint, including descriptions from the `config.yaml` file, and additional model info (e.g. max tokens, cost per input token, etc.) pulled from the model_info you set and the [litellm model cost map](https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json). Sensitive details like API keys are excluded for security purposes.
|
||||
|
||||
:::tip Sync Model Data
|
||||
Keep your model pricing data up to date by [syncing models from GitHub](../sync_models_github.md).
|
||||
:::
|
||||
|
||||
<Tabs
|
||||
defaultValue="curl"
|
||||
values={[
|
||||
|
|
|
|||
61
docs/my-website/docs/proxy/sync_models_github.md
Normal file
61
docs/my-website/docs/proxy/sync_models_github.md
Normal file
|
|
@ -0,0 +1,61 @@
|
|||
# Syncing Models to GitHub model_context_window
|
||||
|
||||
Sync model pricing data from GitHub's `model_prices_and_context_window.json` file outside of the LiteLLM UI.
|
||||
|
||||
> **📹 Video Tutorial**: [Watch how to sync models via the Admin UI](https://www.loom.com/share/ba41acc1882d41b284bbddbb0e9c27ce?sid=bdae351e-2026-4e39-932b-fcb185ff612c)
|
||||
|
||||
## Quick Start
|
||||
|
||||
**Manual sync:**
|
||||
```bash
|
||||
curl -X POST "https://your-proxy-url/reload/model_cost_map" \
|
||||
-H "Authorization: Bearer YOUR_ADMIN_TOKEN" \
|
||||
-H "Content-Type: application/json"
|
||||
```
|
||||
|
||||
**Automatic sync every 6 hours:**
|
||||
```bash
|
||||
curl -X POST "https://your-proxy-url/schedule/model_cost_map_reload?hours=6" \
|
||||
-H "Authorization: Bearer YOUR_ADMIN_TOKEN" \
|
||||
-H "Content-Type: application/json"
|
||||
```
|
||||
|
||||
## API Endpoints
|
||||
|
||||
| Endpoint | Method | Description |
|
||||
|----------|--------|-------------|
|
||||
| `/reload/model_cost_map` | POST | Manual sync |
|
||||
| `/schedule/model_cost_map_reload?hours={hours}` | POST | Schedule periodic sync |
|
||||
| `/schedule/model_cost_map_reload` | DELETE | Cancel scheduled sync |
|
||||
| `/schedule/model_cost_map_reload/status` | GET | Check sync status |
|
||||
|
||||
**Authentication:** Requires admin role or master key
|
||||
|
||||
## Python Example
|
||||
|
||||
```python
|
||||
import requests
|
||||
|
||||
def sync_models(proxy_url, admin_token):
|
||||
response = requests.post(
|
||||
f"{proxy_url}/reload/model_cost_map",
|
||||
headers={"Authorization": f"Bearer {admin_token}"}
|
||||
)
|
||||
return response.json()
|
||||
|
||||
# Usage
|
||||
result = sync_models("https://your-proxy-url", "your-admin-token")
|
||||
print(result['message'])
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
**Custom model cost map URL:**
|
||||
```bash
|
||||
export LITELLM_MODEL_COST_MAP_URL="https://raw.githubusercontent.com/BerriAI/litellm/main/model_prices_and_context_window.json"
|
||||
```
|
||||
|
||||
**Use local model cost map:**
|
||||
```bash
|
||||
export LITELLM_LOCAL_MODEL_COST_MAP=True
|
||||
```
|
||||
|
|
@ -54,6 +54,20 @@ Allow others to create/delete their own keys.
|
|||
|
||||
[**Go Here**](./self_serve.md)
|
||||
|
||||
## Model Management
|
||||
|
||||
The Admin UI provides comprehensive model management capabilities:
|
||||
|
||||
- **Add Models**: Add new models through the UI without restarting the proxy
|
||||
- **Model Hub**: Make models public for developers to discover available models
|
||||
- **Price Data Sync**: Keep model pricing data up to date by syncing from GitHub
|
||||
|
||||
For detailed information on model management, see [Model Management](./model_management.md).
|
||||
|
||||
:::tip Sync Model Pricing Data
|
||||
[Sync model pricing data from GitHub](./sync_models_github.md) to keep your model cost information current.
|
||||
:::
|
||||
|
||||
## Disable Admin UI
|
||||
|
||||
Set `DISABLE_ADMIN_UI="True"` in your environment to disable the Admin UI.
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue