From 7a176804f3a8a56f28d096f0a00b75c97959e3dc Mon Sep 17 00:00:00 2001 From: Teddy Amkie <38896345+TeddyAmkie@users.noreply.github.com> Date: Sat, 4 Oct 2025 12:32:01 -0700 Subject: [PATCH] Add sync models GitHub documentation with Loom video and cross-references (#15191) - Add comprehensive sync_models_github.md with API endpoints and examples - Include Loom video tutorial for Admin UI sync process - Add cross-references from model_management.md, cost_tracking.md, and ui.md - Provide both manual and automated sync options - Include Python SDK usage examples --- docs/my-website/docs/proxy/cost_tracking.md | 4 ++ .../my-website/docs/proxy/model_management.md | 4 ++ .../docs/proxy/sync_models_github.md | 61 +++++++++++++++++++ docs/my-website/docs/proxy/ui.md | 14 +++++ 4 files changed, 83 insertions(+) create mode 100644 docs/my-website/docs/proxy/sync_models_github.md diff --git a/docs/my-website/docs/proxy/cost_tracking.md b/docs/my-website/docs/proxy/cost_tracking.md index 35db752cbb6..85147e12c66 100644 --- a/docs/my-website/docs/proxy/cost_tracking.md +++ b/docs/my-website/docs/proxy/cost_tracking.md @@ -8,6 +8,10 @@ Track spend for keys, users, and teams across 100+ LLMs. LiteLLM automatically tracks spend for all known models. See our [model cost map](https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json) +:::tip Keep Pricing Data Updated +[Sync model pricing data from GitHub](../sync_models_github.md) to ensure accurate cost tracking. +::: + ### How to Track Spend with LiteLLM **Step 1** diff --git a/docs/my-website/docs/proxy/model_management.md b/docs/my-website/docs/proxy/model_management.md index a8cc66ae765..6a87dda2f42 100644 --- a/docs/my-website/docs/proxy/model_management.md +++ b/docs/my-website/docs/proxy/model_management.md @@ -19,6 +19,10 @@ model_list: Retrieve detailed information about each model listed in the `/model/info` endpoint, including descriptions from the `config.yaml` file, and additional model info (e.g. max tokens, cost per input token, etc.) pulled from the model_info you set and the [litellm model cost map](https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json). Sensitive details like API keys are excluded for security purposes. +:::tip Sync Model Data +Keep your model pricing data up to date by [syncing models from GitHub](../sync_models_github.md). +::: + **📹 Video Tutorial**: [Watch how to sync models via the Admin UI](https://www.loom.com/share/ba41acc1882d41b284bbddbb0e9c27ce?sid=bdae351e-2026-4e39-932b-fcb185ff612c) + +## Quick Start + +**Manual sync:** +```bash +curl -X POST "https://your-proxy-url/reload/model_cost_map" \ + -H "Authorization: Bearer YOUR_ADMIN_TOKEN" \ + -H "Content-Type: application/json" +``` + +**Automatic sync every 6 hours:** +```bash +curl -X POST "https://your-proxy-url/schedule/model_cost_map_reload?hours=6" \ + -H "Authorization: Bearer YOUR_ADMIN_TOKEN" \ + -H "Content-Type: application/json" +``` + +## API Endpoints + +| Endpoint | Method | Description | +|----------|--------|-------------| +| `/reload/model_cost_map` | POST | Manual sync | +| `/schedule/model_cost_map_reload?hours={hours}` | POST | Schedule periodic sync | +| `/schedule/model_cost_map_reload` | DELETE | Cancel scheduled sync | +| `/schedule/model_cost_map_reload/status` | GET | Check sync status | + +**Authentication:** Requires admin role or master key + +## Python Example + +```python +import requests + +def sync_models(proxy_url, admin_token): + response = requests.post( + f"{proxy_url}/reload/model_cost_map", + headers={"Authorization": f"Bearer {admin_token}"} + ) + return response.json() + +# Usage +result = sync_models("https://your-proxy-url", "your-admin-token") +print(result['message']) +``` + +## Configuration + +**Custom model cost map URL:** +```bash +export LITELLM_MODEL_COST_MAP_URL="https://raw.githubusercontent.com/BerriAI/litellm/main/model_prices_and_context_window.json" +``` + +**Use local model cost map:** +```bash +export LITELLM_LOCAL_MODEL_COST_MAP=True +``` \ No newline at end of file diff --git a/docs/my-website/docs/proxy/ui.md b/docs/my-website/docs/proxy/ui.md index a093b226a27..f7419d20740 100644 --- a/docs/my-website/docs/proxy/ui.md +++ b/docs/my-website/docs/proxy/ui.md @@ -54,6 +54,20 @@ Allow others to create/delete their own keys. [**Go Here**](./self_serve.md) +## Model Management + +The Admin UI provides comprehensive model management capabilities: + +- **Add Models**: Add new models through the UI without restarting the proxy +- **Model Hub**: Make models public for developers to discover available models +- **Price Data Sync**: Keep model pricing data up to date by syncing from GitHub + +For detailed information on model management, see [Model Management](./model_management.md). + +:::tip Sync Model Pricing Data +[Sync model pricing data from GitHub](./sync_models_github.md) to keep your model cost information current. +::: + ## Disable Admin UI Set `DISABLE_ADMIN_UI="True"` in your environment to disable the Admin UI.