litellm/tests
user 25f93bed91
security: prevent API key leaks in error tracebacks, logs, and alerts
Gemini API keys embedded in URLs as ?key= query parameters leak through
httpx error tracebacks, which are then captured by traceback.format_exc()
and forwarded to logging callbacks, Slack/Teams alerts, and HTTP client
responses.

Short-term: all httpx.HTTPStatusError handlers now raise
MaskedHTTPStatusError(...) from None, which masks the URL and breaks
exception chaining so the original error never appears in tracebacks.

Long-term: moved all Gemini/Vertex URL constructions from ?key={api_key}
to x-goog-api-key header (Google's documented auth method), so the key
is never in the URL at all. WebSocket realtime is the only exception
since WS clients cannot use custom headers.

Additionally hardened all outbound credential paths:
- WebSocket close reasons now pass through _redact_string()
- Callback pipeline (failure_handler) redacts traceback_exception and
  error_str before forwarding to integrations (Langfuse, Datadog, etc.)
- Slack/Teams alert messages redacted in send_llm_exception_alert,
  ProxyLogging.failure_handler, and post_call_failure_hook
- HTTP error responses in proxy SSE and health endpoints redacted
- Exception messages in exception_mapping_utils redacted
- print_verbose() stdout output redacted when set_verbose=True
- HTTPHandler.put() now has MaskedHTTPStatusError (was missing)
2026-04-14 23:09:17 +00:00
..
agent_tests refactor: make unit test 2026-03-28 21:44:46 -07:00
audio_tests test: cleanup test impacted by deployment name change 2026-03-30 18:07:50 -07:00
basic_proxy_startup_tests build: migrate packaging, CI, and Docker from Poetry to uv (#25007) 2026-04-09 11:46:23 -07:00
batches_tests test fix us.anthropic.claude-haiku-4-5-20251001-v1:0 (#24931) 2026-04-01 11:01:03 -07:00
benchmarks Add CodSpeed performance benchmarks (#23676) 2026-03-14 18:44:36 -07:00
code_coverage_tests build: migrate packaging, CI, and Docker from Poetry to uv (#25007) 2026-04-09 11:46:23 -07:00
documentation_tests [Fix] Scope documentation workflow to match CircleCI and add missing router settings 2026-03-28 11:23:53 -07:00
enterprise Litellm oss staging 04 02 2026 p1 (#25055) 2026-04-08 21:37:10 -07:00
guardrails_tests Add Akto Guardrails to LiteLLM (#23250) 2026-03-17 14:38:04 -07:00
image_gen_tests fix: cleanup tests 2026-03-30 16:24:35 -07:00
litellm security: prevent API key leaks in error tracebacks, logs, and alerts 2026-04-14 23:09:17 +00:00
litellm-proxy-extras [Infra] Mirror litellm_table_patch source changes (no binaries) 2026-03-27 16:45:12 -07:00
litellm_core_utils Fix: empty assistant message for converse API 2026-02-04 09:50:58 +05:30
litellm_utils_tests fix: tighten handling of environment references in request parameters 2026-04-11 21:41:41 -07:00
llm_responses_api_testing test: remove bad e2e tests - e2e failing due to low rate limits on ci/cd 2026-03-30 19:18:10 -07:00
llm_translation Merge pull request #25396 from BerriAI/litellm_bedrock-normalize-custom-tool-schema 2026-04-14 10:21:15 -07:00
load_tests test: update to new vertex ai keys 2026-03-28 20:19:05 -07:00
local_testing Merge pull request #25665 from BerriAI/litellm_oss_staging_04_13_2026_p1 2026-04-14 23:50:08 +05:30
logging_callback_tests [Test Fix] fix gov pricing tests (#25022) 2026-04-02 15:55:55 -07:00
mcp_tests feat(mcp): add per-user OAuth token storage for interactive MCP flows 2026-04-09 23:58:35 -07:00
multi_instance_e2e_tests
ocr_tests test: update to new vertex ai keys 2026-03-28 20:19:05 -07:00
old_proxy_tests/tests fix: cleanup tests 2026-03-30 16:24:35 -07:00
openai_endpoints_tests E2E test to assert response headers from the openai files change 2026-04-10 18:45:00 +00:00
otel_tests fix(tests): increase MAX_CALLS and reduce sleep in flaky e2e budget test 2026-03-13 00:04:31 -07:00
pass_through_tests Litellm fix update bedrock models (#24947) 2026-04-01 19:22:54 -07:00
pass_through_unit_tests Litellm fix update bedrock models (#24947) 2026-04-01 19:22:54 -07:00
proxy_admin_ui_tests [Fix] Remove neon CLI dependency and pin all JS dependencies 2026-04-01 16:15:32 -07:00
proxy_e2e_anthropic_messages_tests test fix us.anthropic.claude-haiku-4-5-20251001-v1:0 (#24931) 2026-04-01 11:01:03 -07:00
proxy_security_tests
proxy_unit_tests fix(proxy): use model_group for model_max_budget spend tracking cache key (#25549) 2026-04-11 19:37:58 -07:00
router_unit_tests fix(router): address remaining Greptile review comments 2026-03-27 20:11:28 +05:30
scim_tests
search_tests Merge remote main, resolve conflict keeping new unit tests 2026-03-09 15:20:20 -07:00
spend_tracking_tests [Staging] - Ishaan March 17th (#23903) 2026-03-18 15:09:01 -07:00
store_model_in_db_tests [Fix] CI: fix helm not found and MCP tests on Python 3.9 2026-04-10 21:32:22 -07:00
test_litellm security: prevent API key leaks in error tracebacks, logs, and alerts 2026-04-14 23:09:17 +00:00
unified_google_tests test: update to new vertex ai keys 2026-03-28 20:19:05 -07:00
vector_store_tests test: update test apis 2026-03-28 18:57:27 -07:00
windows_tests
__init__.py
eval_swe_bench.py feat: add litellm.compress() — BM25-based prompt compression with retrieval tool (#25637) 2026-04-13 12:23:54 -07:00
gettysburg.wav
large_text.py
openai_batch_completions.jsonl
README.MD
test_budget_management.py Fix budget reset test 2026-04-14 20:50:42 +05:30
test_callbacks_on_proxy.py
test_config.py
test_debug_warning.py
test_default_encoding_non_root.py fix: tiktoken cache nonroot offline (#23498) 2026-03-14 10:48:36 -07:00
test_end_users.py
test_entrypoint.py
test_fallbacks.py Revert "fix: prevent error when max_fallbacks exceeds available models (#20071)" 2026-02-03 15:15:30 +05:30
test_gpt5_azure_temperature_support.py [Fix] Remove deprecated o1-preview from O-series test and deduplicate is_o_series check 2026-03-12 13:22:00 -07:00
test_health.py
test_keys.py Adding retries to flaky tests 2026-01-22 15:21:44 -08:00
test_litellm_proxy_responses_config.py
test_logging.conf
test_models.py test: replace test_add_and_delete_models integration test with mock 2026-03-30 21:30:57 -07:00
test_new_vector_store_endpoints.py fix(ci): stabilize CI tests - conditional import, mock fixes, timing adjustments 2026-03-13 00:01:25 +00:00
test_openai_endpoints.py
test_organizations.py Adding retries to flaky tests 2026-01-22 15:21:44 -08:00
test_otel_thread_leak.py Fix thread leak in OpenTelemetry dynamic header path (#19946) 2026-01-28 10:35:37 -08:00
test_passthrough_endpoints.py
test_presidio_latency.py fix(presidio): reuse HTTP connections to prevent OOMs (#19964) 2026-01-28 16:08:53 -08:00
test_proxy_server_non_root.py deactivating non root tests 2026-01-23 22:55:36 -08:00
test_ratelimit.py [Fix] test_ratelimit: skip over-limit cases that race with background RPM tracking 2026-04-11 13:08:59 -07:00
test_resource_cleanup.py
test_service_logger_otel.py fix(langfuse_otel): prevent empty proxy request spans from being sent to Langfuse 2026-01-28 15:35:35 +01:00
test_spend_logs.py
test_team.py [Fix] Fail fast when team member spend not flushed in time 2026-03-13 12:40:17 -07:00
test_team_logging.py test: cleanup dead tests 2026-03-28 20:49:02 -07:00
test_team_members.py
test_users.py Litellm fix update bedrock models (#24947) 2026-04-01 19:22:54 -07:00

In total litellm runs 1000+ tests

[02/20/2025] Update:

To make it easier to contribute and map what behavior is tested,

we've started mapping the litellm directory in tests/test_litellm

This folder can only run mock tests.