litellm/tests/e2e
mubashir1osmani 05c31dcbc4 test(e2e): cover block_code_execution and openai_moderation guardrails
Extends the guardrails suite with two built-in guardrails registered per
request (default_on=False, opted in via the chat body's guardrails selector)
so neither intercepts unrelated traffic on the shared proxy.

block_code_execution.pre_call.blocks: a python code block plus a run-this
request is intercepted with the canned content-blocked message and the model
never runs, while the same code block asked about with don't-run-it reaches
the model. Verified live.

openai_moderations.pre_call.blocks: a flagged prompt is rejected 400 naming
the moderation policy while a benign prompt passes. The guardrail calls
OpenAI's moderation API; verifying it needs an OpenAI key with moderation
quota (this account currently 429s the moderation endpoint).

Adds a shared create_backend_model helper and a generic register() plus
per-request guardrails/max_tokens on the client so more built-ins can reuse
the same path.
2026-07-21 10:47:57 -07:00
..
access_control test(true_rabbit): cover passthrough headers, batch assume-role, gemini, vllm, bedrock guardrails, batch rate-limit mapping (#33843) 2026-07-20 16:15:55 -07:00
batches test(true_rabbit): cover passthrough headers, batch assume-role, gemini, vllm, bedrock guardrails, batch rate-limit mapping (#33843) 2026-07-20 16:15:55 -07:00
claude_code refactor(e2e): fold claude_code HTTP probes onto shared Gateway methods (#33760) 2026-07-18 19:03:01 +00:00
coverage_registry test(e2e): add JWT auth journey suite against a real Keycloak IdP 2026-07-20 21:11:48 -07:00
guardrails test(e2e): cover block_code_execution and openai_moderation guardrails 2026-07-21 10:47:57 -07:00
llm_translation fix(e2e): use client.proxy not client.gateway in mid-conversation messages test 2026-07-20 19:50:16 -07:00
load fix(e2e): migrate load suite from e2e_gateway to ProxyClient (#33839) 2026-07-18 19:41:03 +00:00
logging test(true_rabbit): cover passthrough headers, batch assume-role, gemini, vllm, bedrock guardrails, batch rate-limit mapping (#33843) 2026-07-20 16:15:55 -07:00
management test(e2e): add JWT auth journey suite against a real Keycloak IdP 2026-07-20 21:11:48 -07:00
mcp test(e2e): harden stage flakes for batches, UI, and MCP (#33831) 2026-07-18 19:11:54 +00:00
other test(e2e): add other suite covering master-key auth and health lifecycle 2026-07-21 10:32:55 -07:00
quota_management test(true_rabbit): cover passthrough headers, batch assume-role, gemini, vllm, bedrock guardrails, batch rate-limit mapping (#33843) 2026-07-20 16:15:55 -07:00
router test(e2e): add reliability suite covering fallback, timeout, and cache behavior (#34023) 2026-07-20 23:06:46 +00:00
CLAUDE.md test(e2e): add other suite covering master-key auth and health lifecycle 2026-07-21 10:32:55 -07:00
conftest.py test(e2e): guard destructive spend-log truncate behind an explicit opt-in (#33751) 2026-07-20 08:47:39 -07:00
CONTRIBUTING.md chore(e2e): remove tests/e2e/docker-compose.yml (#33837) 2026-07-18 12:50:23 -07:00
e2e_config.py test(e2e): add JWT auth journey suite against a real Keycloak IdP 2026-07-20 21:11:48 -07:00
e2e_db.py test(e2e): guard destructive spend-log truncate behind an explicit opt-in (#33751) 2026-07-20 08:47:39 -07:00
e2e_http.py test(true_rabbit): cover passthrough headers, batch assume-role, gemini, vllm, bedrock guardrails, batch rate-limit mapping (#33843) 2026-07-20 16:15:55 -07:00
junit_properties.py refactor(e2e): replace bespoke result reporter with standard JUnit report (#33758) 2026-07-17 20:53:22 +00:00
lifecycle.py test(e2e): rename Gateway to ProxyClient and expose it as a session-scoped fixture (#33750) 2026-07-18 18:41:18 +00:00
models.py test(e2e): add other suite covering master-key auth and health lifecycle 2026-07-21 10:32:55 -07:00
proxy_client.py test(e2e): cover model update persisting to /model/info (#34017) 2026-07-20 21:13:54 +00:00
pytest.ini test(e2e): add Locust throughput load test that runs last (#33748) 2026-07-18 11:57:41 -07:00
transport.py test(true_rabbit): cover passthrough headers, batch assume-role, gemini, vllm, bedrock guardrails, batch rate-limit mapping (#33843) 2026-07-20 16:15:55 -07:00