mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-23 00:41:40 +00:00
477ca5b0d0
The parallel request limiter previously rendered the offending virtual
key's full 64-char SHA-256 hash verbatim into the customer-facing
``detail`` field of every 429 response, e.g.
{"error":{"message":"Rate limit exceeded for api_key: \
523544f141d47ff188ff366337ddd3c9b44968b565d83a1c9b6fa56c543d3042. ..."}}
The hash cannot be reversed to recover the raw ``sk-...`` secret, but
exposing it in an HTTP error body is still a real downside:
* lets a third party fingerprint which key is hitting limits across
customers;
* discloses LiteLLM's internal key-storage strategy
(SHA-256-of-raw-key) to anyone watching error bodies;
* violates the principle of least information for an error surface
that customers / integrators read.
``redact_user_api_key_info=True`` does not cover this path — that flag
only applies to Langfuse callback metadata and a few logging surfaces,
not the rate-limit response shape (see GH #27884 for the user report).
This patch adds a small ``_sanitize_descriptor_value_for_response``
static helper. When the offending descriptor is ``api_key``, the
customer-facing detail now reads:
Rate limit exceeded for api_key: sk-...3d3042. Limit type: ...
— keeping the last 6 hex chars so an operator reading both the 429
body and the structured proxy log (which still includes the full hash
at debug level) can correlate, but no longer round-tripping the full
identifier. Non-key descriptors (``user_id`` / ``team_id`` / ``model``)
flow through untouched — those are user-supplied scoping values, not
key material.
A ``verbose_proxy_logger.debug`` call preserves the full descriptor
for operator-side correlation; that log is gated by the proxy's debug
flag and never reaches the customer.
Closes #27884.
Test plan
---------
* Added ``test_429_body_does_not_leak_full_api_key_hash`` — integration
test through ``async_pre_call_hook`` that asserts the 64-char hash
is absent from the response detail, the redacted form preserves the
last-6 correlation suffix, and the ``sk-...`` prefix signals the
redaction.
* Added ``test_sanitize_descriptor_value_redacts_api_key`` — unit test
that the sanitiser leaves non-key descriptors alone and tolerates
the ``unknown`` fallback emitted when the resolver can't find a
matching descriptor.
* Existing ``test_missing_descriptor_fallback`` and
``test_multiple_rate_limits_per_descriptor`` continue to pass
(they assert on substring prefixes, not the hash value).
Test results
------------
``pytest tests/test_litellm/proxy/hooks/test_parallel_request_limiter_v3.py``
runs 51 passed + 1 skipped (the pre-existing skip), 13 warnings.
|
||
|---|---|---|
| .. | ||
| _experimental/mcp_server | ||
| agent_endpoints | ||
| anthropic_endpoints | ||
| auth | ||
| client | ||
| common_utils | ||
| db | ||
| discovery_endpoints | ||
| experimental/mcp_server | ||
| google_endpoints | ||
| guardrails | ||
| health_endpoints | ||
| hooks | ||
| image_endpoints | ||
| management_endpoints | ||
| management_helpers | ||
| memory | ||
| middleware | ||
| openai_files_endpoint | ||
| pass_through_endpoints | ||
| policy_engine | ||
| prompts | ||
| public_endpoints | ||
| rag_endpoints | ||
| realtime_endpoints | ||
| response_api_endpoints | ||
| spend_tracking | ||
| test_configs | ||
| types_utils | ||
| ui_crud_endpoints | ||
| vector_store_endpoints | ||
| __init__.py | ||
| conftest.py | ||
| test_aiohttp_cleanup_closed.py | ||
| test_aiohttp_session_recovery.py | ||
| test_api_key_masking_in_errors.py | ||
| test_audio_speech_prometheus_hooks.py | ||
| test_batch_expiry.py | ||
| test_batch_metadata_none_fix.py | ||
| test_batch_retrieve_bedrock.py | ||
| test_budget_reservation.py | ||
| test_caching_routes.py | ||
| test_chat_completion_metadata.py | ||
| test_common_request_processing.py | ||
| test_cors_config.py | ||
| test_custom_proxy.py | ||
| test_dynamic_mcp_route.py | ||
| test_empty_model_list.py | ||
| test_enforce_user_param.py | ||
| test_fallback_management_endpoints.py | ||
| test_fastapi_offline_routes.py | ||
| test_filter_models_by_team_access_group.py | ||
| test_health_check_functions.py | ||
| test_health_check_max_tokens.py | ||
| test_langfuse_passthrough_security.py | ||
| test_lazy_openapi_snapshot.py | ||
| test_litellm_pre_call_utils.py | ||
| test_max_budget_env_var.py | ||
| test_mcp_asgi_response.py | ||
| test_model_dump_with_preserved_fields.py | ||
| test_model_id_header_propagation.py | ||
| test_model_info_default_limits.py | ||
| test_model_level_guardrails.py | ||
| test_openapi_schema_validation.py | ||
| test_pricing_field_strip.py | ||
| test_prometheus_cleanup.py | ||
| test_provider_url_destination_guard.py | ||
| test_proxy_cli.py | ||
| test_proxy_logging_hook_detection.py | ||
| test_proxy_server.py | ||
| test_proxy_types.py | ||
| test_proxy_utils.py | ||
| test_pyroscope.py | ||
| test_redis_auth_cache_flag.py | ||
| test_response_model_sanitization.py | ||
| test_route_a2a_models.py | ||
| test_route_llm_request.py | ||
| test_sensitive_route_auth.py | ||
| test_shared_health_check.py | ||
| test_spend_log_cleanup.py | ||
| test_swagger_chat_completions.py | ||
| test_team_member_update.py | ||
| test_team_org_move.py | ||
| test_tools_allowlist_enforcement.py | ||
| test_update_llm_router_resilience.py | ||
| test_utils.py | ||