diff --git a/docs/my-website/img/release_notes/1_78_0_perf.png b/docs/my-website/img/release_notes/1_78_0_perf.png new file mode 100644 index 00000000000..ed84c3a420a Binary files /dev/null and b/docs/my-website/img/release_notes/1_78_0_perf.png differ diff --git a/docs/my-website/release_notes/v1.78.0-stable/index.md b/docs/my-website/release_notes/v1.78.0-stable/index.md index 0ea4008b96b..e9a471f45b5 100644 --- a/docs/my-website/release_notes/v1.78.0-stable/index.md +++ b/docs/my-website/release_notes/v1.78.0-stable/index.md @@ -59,15 +59,37 @@ pip install litellm==1.78.0.rc.1 ## Key Highlights - **MCP Gateway - Control Tool Access by Team, Key** - Control MCP tool access by team/key. +- **Performance Improvements** - 70% Lower p99 Latency - **GPT-5 Pro & GPT-Image-1-Mini** - Day 0 support for OpenAI's GPT-5 Pro (400K context) and gpt-image-1-mini image generation -- **UI Performance Boost** - Replaces bloated key list calls with lean key aliases endpoint, Turbopack for faster development, and major UI refactors - **EnkryptAI Guardrails** - New guardrail integration for content moderation - **Tag-Based Budgets** - Support for setting budgets based on request tags -- **Azure AD & SSO** - Enhanced Azure AD default credentials selection and EntraID app roles support --- -## Performance +### MCP Gateway - Control Tool Access by Team, Key + + + +
+ +Proxy admins can now control MCP tool access by team or key. This makes it easy to grant different teams selective access to tools from the same MCP server. + +For example, you can now give your Engineering team access to `list_repositories`, `create_issue`, and `search_code` tools, while Sales only gets `search_code` and `close_issue` tools. + +This makes it easier for Proxy Admins to govern MCP Tool Access. + +[Get Started](../../docs/mcp_control#set-allowed-tools-for-a-key-team-or-organization) + +--- + +## Performance - 70% Lower p99 Latency + + + +
This release cuts p99 latency by 70% on LiteLLM AI Gateway, making it even better for low-latency use cases. @@ -75,7 +97,7 @@ These gains come from two key enhancements: **Reliable Sessions** -`shared_session` is now consistently used across all calls, resolving JSON errors, enabling connection pooling, and actively managing session lifecycles—detecting stale connections, retrying expired sessions, and preventing 'Session is closed' errors. +Added support for shared sessions with aiohttp. The shared_session parameter is now consistently used across all calls, enabling connection pooling. **Faster Routing** @@ -112,23 +134,6 @@ View the complete configuration: [gist.github.com/AlexsanderHamir/config.yaml]( View the complete load testing script: [gist.github.com/AlexsanderHamir/no_cache_hits.py](https://gist.github.com/AlexsanderHamir/42c33d7a4dc7a57f56a78b560dee3a42) --- -### MCP Gateway - Control Tool Access by Team, Key - - - -
- -Proxy admins can now control MCP tool access by team or key. This makes it easy to grant different teams selective access to tools from the same MCP server. - -For example, you can now give your Engineering team access to `list_repositories`, `create_issue`, and `search_code` tools, while Sales only gets `search_code` and `close_issue` tools. - -This makes it easier for Proxy Admins to govern MCP Tool Access. - -[Get Started](../../docs/mcp_control#set-allowed-tools-for-a-key-team-or-organization) - ## New Models / Updated Models diff --git a/litellm/model_prices_and_context_window_backup.json b/litellm/model_prices_and_context_window_backup.json index 6b2de335f66..645ecb9779f 100644 --- a/litellm/model_prices_and_context_window_backup.json +++ b/litellm/model_prices_and_context_window_backup.json @@ -4841,7 +4841,7 @@ "input_cost_per_token_above_200k_tokens": 6e-06, "litellm_provider": "anthropic", "max_input_tokens": 1000000, - "max_output_tokens": 1000000, + "max_output_tokens": 64000, "max_tokens": 1000000, "mode": "chat", "output_cost_per_token": 1.5e-05,