diff --git a/docs/my-website/img/release_notes/mcp_ui.png b/docs/my-website/img/release_notes/mcp_ui.png index 6731fba71be..8f4cd4ea198 100644 Binary files a/docs/my-website/img/release_notes/mcp_ui.png and b/docs/my-website/img/release_notes/mcp_ui.png differ diff --git a/docs/my-website/img/release_notes/ui_usage.png b/docs/my-website/img/release_notes/ui_usage.png new file mode 100644 index 00000000000..ac39ffb9189 Binary files /dev/null and b/docs/my-website/img/release_notes/ui_usage.png differ diff --git a/docs/my-website/release_notes/v1.65.0-stable/index.md b/docs/my-website/release_notes/v1.65.0-stable/index.md index 36e571be933..ec5f3bd4410 100644 --- a/docs/my-website/release_notes/v1.65.0-stable/index.md +++ b/docs/my-website/release_notes/v1.65.0-stable/index.md @@ -18,12 +18,7 @@ import Image from '@theme/IdealImage'; v1.65.0-stable is live now. Here are the key highlights of this release: - **MCP Support**: Support for adding and using MCP servers on the LiteLLM proxy. -- **Custom Prompt Management**: Connect LiteLLM to your prompt management system with custom hooks. - - - - - +- **UI view total usage after 1M+ logs**: You can now view usage analytics after crossing 1M+ logs in DB. ## Model Context Protocol (MCP) @@ -39,21 +34,71 @@ Read more about MCP [here](https://docs.litellm.ai/docs/mcp). Expose and use MCP servers through LiteLLM

-## Custom Prompt Management +## UI view total usage after 1M+ logs -This release allows you to connect LiteLLM to any prompt management service through our custom prompt management hooks. As proxy admin all you need to do is implement a `get_chat_completion_prompt` hook which accepts a prompt_id and prompt_variables and returns a formatted prompt. +This release brings the ability to view total usage analytics even after exceeding 1M+ logs in your database. We've implemented a scalable architecture that stores only aggregate usage data, resulting in significantly more efficient queries and reduced database CPU utilization. -Read more about custom prompt management [here](https://docs.litellm.ai/docs/proxy/custom_prompt_management).

- Connect LiteLLM to your prompt management service through custom hooks + View total usage after 1M+ logs

+- How this works: + - We now aggregate usage data into a dedicated DailyUserSpend table, significantly reducing query load and CPU usage even beyond 1M+ logs. + +- Daily Spend Breakdown API: + + - Retrieve granular daily usage data (by model, provider, and API key) with a single endpoint. + Example Request: + + ```shell title="Daily Spend Breakdown API" showLineNumbers + curl -L -X GET 'http://localhost:4000/user/daily/activity?start_date=2025-03-20&end_date=2025-03-27' \ + -H 'Authorization: Bearer sk-...' + ``` + + ```json title="Daily Spend Breakdown API Response" showLineNumbers + { + "results": [ + { + "date": "2025-03-27", + "metrics": { + "spend": 0.0177072, + "prompt_tokens": 111, + "completion_tokens": 1711, + "total_tokens": 1822, + "api_requests": 11 + }, + "breakdown": { + "models": { + "gpt-4o-mini": { + "spend": 1.095e-05, + "prompt_tokens": 37, + "completion_tokens": 9, + "total_tokens": 46, + "api_requests": 1 + }, + "providers": { "openai": { ... }, "azure_ai": { ... } }, + "api_keys": { "3126b6eaf1...": { ... } } + } + } + ], + "metadata": { + "total_spend": 0.7274667, + "total_prompt_tokens": 280990, + "total_completion_tokens": 376674, + "total_api_requests": 14 + } + } + ``` + + + + ## New Models / Updated Models - Support for Vertex AI gemini-2.0-flash-lite & Google AI Studio gemini-2.0-flash-lite [PR](https://github.com/BerriAI/litellm/pull/9523) - Support for Vertex AI Fine-Tuned LLMs [PR](https://github.com/BerriAI/litellm/pull/9542)