litellm/tests/e2e
mubashir1osmani 497ad46171 test(e2e): cover presidio PII masking (pre_call + post_call)
Registers a presidio guardrail per request (default_on=False) with the
analyzer/anonymizer bases supplied in the registration params, so the test
controls its own dependency and needs no proxy restart.

presidio.pre_call.masks: a repeat-verbatim request comes back with the
<EMAIL_ADDRESS> placeholder and never the raw email, proving the prompt was
anonymized before the model saw it.

presidio.post_call.masks: with apply_to_output the model's own emitted email
is masked on the way out, so the caller never receives the raw value.

Both verified live against real presidio analyzer + anonymizer containers.
logging_only is intentionally not covered: /spend/logs exposes no prompt
messages to read back the masked log, and a logging_only run also masked the
response, contradicting its contract; noted in the module docstring for a
follow-up.
2026-07-21 11:02:55 -07:00
..
access_control test(true_rabbit): cover passthrough headers, batch assume-role, gemini, vllm, bedrock guardrails, batch rate-limit mapping (#33843) 2026-07-20 16:15:55 -07:00
batches test(true_rabbit): cover passthrough headers, batch assume-role, gemini, vllm, bedrock guardrails, batch rate-limit mapping (#33843) 2026-07-20 16:15:55 -07:00
claude_code refactor(e2e): fold claude_code HTTP probes onto shared Gateway methods (#33760) 2026-07-18 19:03:01 +00:00
coverage_registry test(e2e): add JWT auth journey suite against a real Keycloak IdP 2026-07-20 21:11:48 -07:00
guardrails test(e2e): cover presidio PII masking (pre_call + post_call) 2026-07-21 11:02:55 -07:00
llm_translation fix(e2e): use client.proxy not client.gateway in mid-conversation messages test 2026-07-20 19:50:16 -07:00
load fix(e2e): migrate load suite from e2e_gateway to ProxyClient (#33839) 2026-07-18 19:41:03 +00:00
logging test(true_rabbit): cover passthrough headers, batch assume-role, gemini, vllm, bedrock guardrails, batch rate-limit mapping (#33843) 2026-07-20 16:15:55 -07:00
management test(e2e): add JWT auth journey suite against a real Keycloak IdP 2026-07-20 21:11:48 -07:00
mcp test(e2e): harden stage flakes for batches, UI, and MCP (#33831) 2026-07-18 19:11:54 +00:00
other test(e2e): add other suite covering master-key auth and health lifecycle 2026-07-21 10:32:55 -07:00
quota_management test(true_rabbit): cover passthrough headers, batch assume-role, gemini, vllm, bedrock guardrails, batch rate-limit mapping (#33843) 2026-07-20 16:15:55 -07:00
router test(e2e): add reliability suite covering fallback, timeout, and cache behavior (#34023) 2026-07-20 23:06:46 +00:00
CLAUDE.md test(e2e): add other suite covering master-key auth and health lifecycle 2026-07-21 10:32:55 -07:00
conftest.py test(e2e): guard destructive spend-log truncate behind an explicit opt-in (#33751) 2026-07-20 08:47:39 -07:00
CONTRIBUTING.md chore(e2e): remove tests/e2e/docker-compose.yml (#33837) 2026-07-18 12:50:23 -07:00
e2e_config.py test(e2e): add JWT auth journey suite against a real Keycloak IdP 2026-07-20 21:11:48 -07:00
e2e_db.py test(e2e): guard destructive spend-log truncate behind an explicit opt-in (#33751) 2026-07-20 08:47:39 -07:00
e2e_http.py test(true_rabbit): cover passthrough headers, batch assume-role, gemini, vllm, bedrock guardrails, batch rate-limit mapping (#33843) 2026-07-20 16:15:55 -07:00
junit_properties.py refactor(e2e): replace bespoke result reporter with standard JUnit report (#33758) 2026-07-17 20:53:22 +00:00
lifecycle.py test(e2e): rename Gateway to ProxyClient and expose it as a session-scoped fixture (#33750) 2026-07-18 18:41:18 +00:00
models.py test(e2e): add other suite covering master-key auth and health lifecycle 2026-07-21 10:32:55 -07:00
proxy_client.py test(e2e): cover model update persisting to /model/info (#34017) 2026-07-20 21:13:54 +00:00
pytest.ini test(e2e): add Locust throughput load test that runs last (#33748) 2026-07-18 11:57:41 -07:00
transport.py test(true_rabbit): cover passthrough headers, batch assume-role, gemini, vllm, bedrock guardrails, batch rate-limit mapping (#33843) 2026-07-20 16:15:55 -07:00