mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-14 23:21:35 +00:00
4a81ec4982
* feat(proxy): add per-MCP-server RPM rate limiting for keys and teams
Adds mcp_rpm_limit, a dict keyed by MCP server name (alias if set, else the
configured name) that caps requests per minute per server for a key or team.
The v3 rate limiter builds a per-server descriptor only when a limit is
configured for the server being called, so other servers stay uncapped and no
TPM reservation is engaged. Server identity is surfaced into the request data
via mcp_rate_limit_server_name so the limiter can resolve it.
* fix(proxy): gate MCP rpm descriptors on call_mcp_tool; document mcp_rpm_limit param
Only honor mcp_server_name when the call is an actual MCP tool call. Without
this, a normal LLM request could inject mcp_server_name in its body to consume
a target server's MCP quota and 429 legitimate tool calls. Also adds the
mcp_rpm_limit parameter docstring to update_key, new_user, and user_update so
the API docs validator passes.
* Fix MCP rate limit quota handling
* Delete scripts/test_mcp_rpm_limit.sh
* docs(proxy): clarify mcp_rpm_limit is enforced for keys and teams, not per user
* fix(proxy): accept mcp_rpm_limit in generate_key_helper_fn
NewUserRequest and GenerateKeyRequest inherit mcp_rpm_limit from
GenerateRequestBase, so /user/new and /key/generate forwarded the field
to generate_key_helper_fn, which did not accept it and returned a 500
("unexpected keyword argument 'mcp_rpm_limit'"). Accept the param and
store it in metadata, matching model_rpm_limit/model_tpm_limit, so the
limit is persisted where get_key_mcp_rpm_limit reads it.
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
|
||
|---|---|---|
| .. | ||
| auth | ||
| guardrail_translation | ||
| test_byok_oauth_endpoints.py | ||
| test_callback_oauth_error_responses.py | ||
| test_db_credentials.py | ||
| test_discoverable_endpoints.py | ||
| test_is_tool_name_prefixed.py | ||
| test_jwt_mcp_enforcement.py | ||
| test_jwt_mcp_simple.py | ||
| test_mcp_cost_calculator.py | ||
| test_mcp_custom_fields.py | ||
| test_mcp_debug.py | ||
| test_mcp_discovery.py | ||
| test_mcp_header_alias_utils.py | ||
| test_mcp_hook_extra_headers.py | ||
| test_mcp_metadata_preservation.py | ||
| test_mcp_oauth_passthrough.py | ||
| test_mcp_oauth_passthrough_cold_start.py | ||
| test_mcp_oauth_passthrough_tools.py | ||
| test_mcp_partial_update.py | ||
| test_mcp_server.py | ||
| test_mcp_server_manager.py | ||
| test_mcp_session_logging.py | ||
| test_mcp_sigv4_auth.py | ||
| test_mcp_stale_session.py | ||
| test_mcp_toolset_scope.py | ||
| test_oauth2_token_cache.py | ||
| test_openapi_to_mcp_generator.py | ||
| test_openapi_tool_auth.py | ||
| test_rest_endpoints.py | ||
| test_semantic_tool_filter.py | ||
| test_short_mcp_tool_prefix.py | ||
| test_ui_session_utils.py | ||