mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-11 03:38:38 +00:00
47771aa369
* feat(router)!: redact internal model_group/fallback names from exception messages
The Router was unconditionally appending internal config names onto
exception.message:
- "Received Model Group=..."
- "Available Model Group Fallbacks=..."
- "No fallback model group found... Fallbacks={...}"
- "context_window_fallbacks={...}"
- Deployment-timeout messages including model_group
- Fallback failure detail listing fallback chain
ProxyException forwards .message verbatim to clients, so gateways were
leaking their model_name / fallback wiring in every failed call.
Fix: gate all five mutation sites on a new
`litellm.expose_router_debug_in_errors` flag (default False). Set to
True to restore upstream debug behavior for local debugging.
Why: matches the redaction posture this codebase already has for
upstream model identifiers (cf. _litellm_returned_model_name) and
removes the last common error-path leak of internal model_group names.
Breaking change marker (!): if anything parses "Received Model Group="
out of client error messages, flip the flag on or migrate to the
x-litellm-* response headers instead.
Tests: 7 cases covering each of the 5 redaction sites + the flag-on
inverse path, plus a "default off" sanity check.
* test(router): cover sites 1 + 3 of expose_router_debug_in_errors gate
Addresses Greptile / codecov feedback on #30418: patch coverage was
55.6% with 4 lines uncovered in litellm/router.py. The existing tests
exercised sites 2 (ContextWindowExceededError), 4 (no-fallback-found),
and 5 (Received Model Group) — both default and flag-on. Sites 1 and 3
were declared in the PR description as covered by "site 5 also fires"
but the gate body lines for each (the `e.message +=` inside the
`if litellm.expose_router_debug_in_errors:` branch) only execute when
the flag is on AND the specific exception path is taken, which neither
existing test triggered.
Added 4 new tests (default + flag-on × 2 sites):
- test_default_does_not_leak_deployment_timeout_debug
- test_flag_on_leaks_deployment_timeout_debug
- test_default_does_not_leak_content_policy_fallback_hint
- test_flag_on_leaks_content_policy_fallback_hint
Trigger details:
- Site 1 (litellm.Timeout in _acompletion) is reached via the
Router-supported `mock_timeout=True` + `timeout=0.001` kwargs on
`acompletion(...)`. Cannot embed a Timeout instance in model_list
because Router.__init__ deep-copies it and Timeout.__reduce__ does
not preserve the required positional args.
- Site 3 (ContentPolicyViolationError without content_policy_fallbacks
set, in async_function_with_fallbacks_common_utils) is reached by
passing a `mock_response=litellm.ContentPolicyViolationError(...)`
instance via the call-site kwarg — same deepcopy-avoidance reason.
11/11 tests pass locally. Patch coverage on litellm/router.py for this
PR's diff should now be 100%.
* chore(router): flip expose_router_debug_in_errors default to True
Addresses @Sameerlite's review on #30418 — maintain backward
compat on the wire. Redact becomes opt-in via setting the flag
to False; the historical behavior (leak internal model_group /
fallback wiring through exception messages) is preserved as the
default.
- litellm/__init__.py: default flipped to True, docstring rewritten
with deprecation note pointing at a future flip to False (redact
by default) in a major release.
- tests/test_litellm/test_router_exception_redaction.py: fixture
resets to True (was False); the "off" tests now explicitly set
False; the "default_leaks_*" tests rely on the fixture default.
test_flag_defaults_off -> test_flag_defaults_on.
- No router.py change needed; the gate keys off the same flag,
only the default changes.
- PR title no longer needs the breaking-change `!` marker — no
client sees a behavior change at default settings.
11/11 pass locally.
* ci: retrigger workflows after base branch change to litellm_internal_staging
|
||
|---|---|---|
| .. | ||
| agent_tests | ||
| audio_tests | ||
| basic_proxy_startup_tests | ||
| batches_tests | ||
| benchmarks | ||
| code_coverage_tests | ||
| documentation_tests | ||
| enterprise | ||
| guardrails_tests | ||
| image_gen_tests | ||
| integration | ||
| litellm | ||
| litellm-proxy-extras | ||
| litellm_core_utils | ||
| litellm_utils_tests | ||
| llm_responses_api_testing | ||
| llm_translation | ||
| load_tests | ||
| local_testing | ||
| logging_callback_tests | ||
| mcp_tests | ||
| multi_instance_e2e_tests | ||
| ocr_tests | ||
| old_proxy_tests/tests | ||
| openai_endpoints_tests | ||
| otel_tests | ||
| pass_through_tests | ||
| pass_through_unit_tests | ||
| proxy_admin_ui_tests | ||
| proxy_behavior | ||
| proxy_e2e_anthropic_messages_tests | ||
| proxy_migration_tests | ||
| proxy_security_tests | ||
| proxy_unit_tests | ||
| router_unit_tests | ||
| scim_tests | ||
| search_tests | ||
| spend_tracking_tests | ||
| store_model_in_db_tests | ||
| test_litellm | ||
| unified_google_tests | ||
| vector_store_tests | ||
| windows_tests | ||
| __init__.py | ||
| _flush_vcr_cache.py | ||
| _live_test_helpers.py | ||
| _openai_record_replay_proxy.py | ||
| _vcr_conftest_common.py | ||
| _vcr_redis_persister.py | ||
| eval_swe_bench.py | ||
| gettysburg.wav | ||
| large_text.py | ||
| openai_batch_completions.jsonl | ||
| README.MD | ||
| test_anthropic_compaction_usage.py | ||
| test_budget_management.py | ||
| test_callbacks_on_proxy.py | ||
| test_config.py | ||
| test_debug_warning.py | ||
| test_default_encoding_non_root.py | ||
| test_end_users.py | ||
| test_entrypoint.py | ||
| test_fallbacks.py | ||
| test_gpt5_azure_temperature_support.py | ||
| test_health.py | ||
| test_keys.py | ||
| test_litellm_proxy_responses_config.py | ||
| test_logging.conf | ||
| test_models.py | ||
| test_new_vector_store_endpoints.py | ||
| test_openai_endpoints.py | ||
| test_organizations.py | ||
| test_otel_thread_leak.py | ||
| test_passthrough_endpoints.py | ||
| test_presidio_latency.py | ||
| test_proxy_server_non_root.py | ||
| test_ratelimit.py | ||
| test_resource_cleanup.py | ||
| test_service_logger_otel.py | ||
| test_spend_logs.py | ||
| test_team.py | ||
| test_team_logging.py | ||
| test_team_members.py | ||
| test_users.py | ||
In total litellm runs 1000+ tests
[02/20/2025] Update:
To make it easier to contribute and map what behavior is tested,
we've started mapping the litellm directory in tests/test_litellm
This folder can only run mock tests.