mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-05 02:41:56 +00:00
4a81ec4982
* feat(proxy): add per-MCP-server RPM rate limiting for keys and teams
Adds mcp_rpm_limit, a dict keyed by MCP server name (alias if set, else the
configured name) that caps requests per minute per server for a key or team.
The v3 rate limiter builds a per-server descriptor only when a limit is
configured for the server being called, so other servers stay uncapped and no
TPM reservation is engaged. Server identity is surfaced into the request data
via mcp_rate_limit_server_name so the limiter can resolve it.
* fix(proxy): gate MCP rpm descriptors on call_mcp_tool; document mcp_rpm_limit param
Only honor mcp_server_name when the call is an actual MCP tool call. Without
this, a normal LLM request could inject mcp_server_name in its body to consume
a target server's MCP quota and 429 legitimate tool calls. Also adds the
mcp_rpm_limit parameter docstring to update_key, new_user, and user_update so
the API docs validator passes.
* Fix MCP rate limit quota handling
* Delete scripts/test_mcp_rpm_limit.sh
* docs(proxy): clarify mcp_rpm_limit is enforced for keys and teams, not per user
* fix(proxy): accept mcp_rpm_limit in generate_key_helper_fn
NewUserRequest and GenerateKeyRequest inherit mcp_rpm_limit from
GenerateRequestBase, so /user/new and /key/generate forwarded the field
to generate_key_helper_fn, which did not accept it and returned a 500
("unexpected keyword argument 'mcp_rpm_limit'"). Accept the param and
store it in metadata, matching model_rpm_limit/model_tpm_limit, so the
limit is persisted where get_key_mcp_rpm_limit reads it.
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
|
||
|---|---|---|
| .. | ||
| policy_endpoints | ||
| scim | ||
| search_endpoints | ||
| usage_endpoints | ||
| test_access_group_endpoints.py | ||
| test_access_group_management.py | ||
| test_activity_tenant_scoping.py | ||
| test_budget_endpoints.py | ||
| test_cache_settings_endpoints.py | ||
| test_callback_management_endpoints.py | ||
| test_common_daily_activity.py | ||
| test_common_utils.py | ||
| test_compliance_endpoints.py | ||
| test_config_override_endpoints.py | ||
| test_cost_tracking_settings.py | ||
| test_customer_budget.py | ||
| test_customer_endpoints.py | ||
| test_delete_callbacks_endpoint.py | ||
| test_delete_verification_tokens_failed.py | ||
| test_entraid_app_roles.py | ||
| test_internal_user_endpoints.py | ||
| test_key_management_endpoints.py | ||
| test_mcp_management_endpoints.py | ||
| test_model_management_endpoints.py | ||
| test_org_admin_team_access.py | ||
| test_organization_endpoints.py | ||
| test_policy_endpoints.py | ||
| test_project_org_authz.py | ||
| test_router_settings_endpoints.py | ||
| test_tag_management_endpoints.py | ||
| test_team_callback_endpoints.py | ||
| test_team_default_params.py | ||
| test_team_endpoints.py | ||
| test_tool_management_endpoints.py | ||
| test_ui_sso.py | ||
| test_workflow_management_endpoints.py | ||