mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-22 00:31:44 +00:00
477ca5b0d0
The parallel request limiter previously rendered the offending virtual
key's full 64-char SHA-256 hash verbatim into the customer-facing
``detail`` field of every 429 response, e.g.
{"error":{"message":"Rate limit exceeded for api_key: \
523544f141d47ff188ff366337ddd3c9b44968b565d83a1c9b6fa56c543d3042. ..."}}
The hash cannot be reversed to recover the raw ``sk-...`` secret, but
exposing it in an HTTP error body is still a real downside:
* lets a third party fingerprint which key is hitting limits across
customers;
* discloses LiteLLM's internal key-storage strategy
(SHA-256-of-raw-key) to anyone watching error bodies;
* violates the principle of least information for an error surface
that customers / integrators read.
``redact_user_api_key_info=True`` does not cover this path — that flag
only applies to Langfuse callback metadata and a few logging surfaces,
not the rate-limit response shape (see GH #27884 for the user report).
This patch adds a small ``_sanitize_descriptor_value_for_response``
static helper. When the offending descriptor is ``api_key``, the
customer-facing detail now reads:
Rate limit exceeded for api_key: sk-...3d3042. Limit type: ...
— keeping the last 6 hex chars so an operator reading both the 429
body and the structured proxy log (which still includes the full hash
at debug level) can correlate, but no longer round-tripping the full
identifier. Non-key descriptors (``user_id`` / ``team_id`` / ``model``)
flow through untouched — those are user-supplied scoping values, not
key material.
A ``verbose_proxy_logger.debug`` call preserves the full descriptor
for operator-side correlation; that log is gated by the proxy's debug
flag and never reaches the customer.
Closes #27884.
Test plan
---------
* Added ``test_429_body_does_not_leak_full_api_key_hash`` — integration
test through ``async_pre_call_hook`` that asserts the 64-char hash
is absent from the response detail, the redacted form preserves the
last-6 correlation suffix, and the ``sk-...`` prefix signals the
redaction.
* Added ``test_sanitize_descriptor_value_redacts_api_key`` — unit test
that the sanitiser leaves non-key descriptors alone and tolerates
the ``unknown`` fallback emitted when the resolver can't find a
matching descriptor.
* Existing ``test_missing_descriptor_fallback`` and
``test_multiple_rate_limits_per_descriptor`` continue to pass
(they assert on substring prefixes, not the hash value).
Test results
------------
``pytest tests/test_litellm/proxy/hooks/test_parallel_request_limiter_v3.py``
runs 51 passed + 1 skipped (the pre-existing skip), 13 warnings.
|
||
|---|---|---|
| .. | ||
| agent_tests | ||
| audio_tests | ||
| basic_proxy_startup_tests | ||
| batches_tests | ||
| benchmarks | ||
| code_coverage_tests | ||
| documentation_tests | ||
| enterprise | ||
| guardrails_tests | ||
| image_gen_tests | ||
| litellm | ||
| litellm-proxy-extras | ||
| litellm_core_utils | ||
| litellm_utils_tests | ||
| llm_responses_api_testing | ||
| llm_translation | ||
| load_tests | ||
| local_testing | ||
| logging_callback_tests | ||
| mcp_tests | ||
| multi_instance_e2e_tests | ||
| ocr_tests | ||
| old_proxy_tests/tests | ||
| openai_endpoints_tests | ||
| otel_tests | ||
| pass_through_tests | ||
| pass_through_unit_tests | ||
| proxy_admin_ui_tests | ||
| proxy_e2e_anthropic_messages_tests | ||
| proxy_security_tests | ||
| proxy_unit_tests | ||
| router_unit_tests | ||
| scim_tests | ||
| search_tests | ||
| spend_tracking_tests | ||
| store_model_in_db_tests | ||
| test_litellm | ||
| unified_google_tests | ||
| vector_store_tests | ||
| windows_tests | ||
| __init__.py | ||
| _flush_vcr_cache.py | ||
| _vcr_conftest_common.py | ||
| _vcr_redis_persister.py | ||
| eval_swe_bench.py | ||
| gettysburg.wav | ||
| large_text.py | ||
| openai_batch_completions.jsonl | ||
| README.MD | ||
| test_budget_management.py | ||
| test_callbacks_on_proxy.py | ||
| test_config.py | ||
| test_debug_warning.py | ||
| test_default_encoding_non_root.py | ||
| test_end_users.py | ||
| test_entrypoint.py | ||
| test_fallbacks.py | ||
| test_gpt5_azure_temperature_support.py | ||
| test_health.py | ||
| test_keys.py | ||
| test_litellm_proxy_responses_config.py | ||
| test_logging.conf | ||
| test_models.py | ||
| test_new_vector_store_endpoints.py | ||
| test_openai_endpoints.py | ||
| test_organizations.py | ||
| test_otel_thread_leak.py | ||
| test_passthrough_endpoints.py | ||
| test_presidio_latency.py | ||
| test_proxy_server_non_root.py | ||
| test_ratelimit.py | ||
| test_resource_cleanup.py | ||
| test_service_logger_otel.py | ||
| test_spend_logs.py | ||
| test_team.py | ||
| test_team_logging.py | ||
| test_team_members.py | ||
| test_users.py | ||
In total litellm runs 1000+ tests
[02/20/2025] Update:
To make it easier to contribute and map what behavior is tested,
we've started mapping the litellm directory in tests/test_litellm
This folder can only run mock tests.