litellm/tests/e2e
ryan-crabbe-berri 871629e8f3 test(e2e): run the UI Playwright suite against the shared e2e gateway
The suite now targets LITELLM_PROXY_URL (default http://localhost:4000),
the same live gateway the python e2e suites use, instead of provisioning
its own postgres, mock LLM server and SQL-seeded proxy. globalSetup seeds
every e2e-* user, team, org, key and model deployment through the
management API with the master key, so any gateway with a master key and
store_model_in_db works; seed.sql and the mock LLM server are gone. The
two deployments (e2e-ui-openai, e2e-ui-anthropic) route to real providers
per the harness's no-mocks rule, and the key creation spec now asserts a
real completion instead of the mock's canned body.

Seeding is idempotent against a persistent gateway: teardown deletes
per-entity (batch deletes are all-or-nothing and one missing id used to
strand every other row), the modelHub spec resets the public model groups
list it fills, and the MCP spec purges its dead-URL servers whose failing
health checks kept networkidle navigation from settling on reruns. Both
Playwright configs pin workers: 1 since several specs mutate global
gateway state; the suite documents that it needs the gateway to itself
while it runs.

Two specs are also caught up with UI refactors that had moved their
buttons into dropdown menus (key deletion under "More key actions",
credential editing under the row's actions menu).

run_e2e.sh keeps a provisioning mode for CI and machines with no gateway,
and skips provisioning entirely when LITELLM_PROXY_URL is set. The
CircleCI jobs drop their seed and mock-server steps; the mock server's
removal also lets tests/e2e/ui leave the basedpyright exclude list.
2026-07-21 18:17:01 -07:00
..
access_control test(true_rabbit): cover passthrough headers, batch assume-role, gemini, vllm, bedrock guardrails, batch rate-limit mapping (#33843) 2026-07-20 16:15:55 -07:00
batches test(true_rabbit): cover passthrough headers, batch assume-role, gemini, vllm, bedrock guardrails, batch rate-limit mapping (#33843) 2026-07-20 16:15:55 -07:00
claude_code refactor(e2e): fold claude_code HTTP probes onto shared Gateway methods (#33760) 2026-07-18 19:03:01 +00:00
coverage_registry chore(tests): replace a customer name and domain with neutral placeholders 2026-07-21 15:09:15 -07:00
guardrails test(e2e): add Other suite and Guardrails coverage incl. an MCP tool-call guardrail (#34149) 2026-07-21 14:06:29 -07:00
llm_translation chore(tests): replace a customer name and domain with neutral placeholders 2026-07-21 15:09:15 -07:00
load fix(e2e): migrate load suite from e2e_gateway to ProxyClient (#33839) 2026-07-18 19:41:03 +00:00
logging test(e2e): add Other suite and Guardrails coverage incl. an MCP tool-call guardrail (#34149) 2026-07-21 14:06:29 -07:00
management test(e2e): cover model, tag and access group routes for Management/UI coverage (#34118) 2026-07-21 21:59:34 +00:00
mcp test(e2e): add Other suite and Guardrails coverage incl. an MCP tool-call guardrail (#34149) 2026-07-21 14:06:29 -07:00
other test(e2e): add Other suite and Guardrails coverage incl. an MCP tool-call guardrail (#34149) 2026-07-21 14:06:29 -07:00
quota_management test(true_rabbit): cover passthrough headers, batch assume-role, gemini, vllm, bedrock guardrails, batch rate-limit mapping (#33843) 2026-07-20 16:15:55 -07:00
router test(e2e): add reliability suite covering fallback, timeout, and cache behavior (#34023) 2026-07-20 23:06:46 +00:00
ui test(e2e): run the UI Playwright suite against the shared e2e gateway 2026-07-21 18:17:01 -07:00
CLAUDE.md test(e2e): run the UI Playwright suite against the shared e2e gateway 2026-07-21 18:17:01 -07:00
conftest.py test(e2e): guard destructive spend-log truncate behind an explicit opt-in (#33751) 2026-07-20 08:47:39 -07:00
CONTRIBUTING.md chore(e2e): remove tests/e2e/docker-compose.yml (#33837) 2026-07-18 12:50:23 -07:00
e2e_config.py test(true_rabbit): cover passthrough headers, batch assume-role, gemini, vllm, bedrock guardrails, batch rate-limit mapping (#33843) 2026-07-20 16:15:55 -07:00
e2e_db.py test(e2e): guard destructive spend-log truncate behind an explicit opt-in (#33751) 2026-07-20 08:47:39 -07:00
e2e_http.py test(true_rabbit): cover passthrough headers, batch assume-role, gemini, vllm, bedrock guardrails, batch rate-limit mapping (#33843) 2026-07-20 16:15:55 -07:00
junit_properties.py refactor(e2e): replace bespoke result reporter with standard JUnit report (#33758) 2026-07-17 20:53:22 +00:00
lifecycle.py test(e2e): rename Gateway to ProxyClient and expose it as a session-scoped fixture (#33750) 2026-07-18 18:41:18 +00:00
models.py test(e2e): add Other suite and Guardrails coverage incl. an MCP tool-call guardrail (#34149) 2026-07-21 14:06:29 -07:00
otel_client.py test(e2e): add Other suite and Guardrails coverage incl. an MCP tool-call guardrail (#34149) 2026-07-21 14:06:29 -07:00
proxy_client.py test(e2e): cover model update persisting to /model/info (#34017) 2026-07-20 21:13:54 +00:00
pytest.ini test(e2e): add Locust throughput load test that runs last (#33748) 2026-07-18 11:57:41 -07:00
transport.py test(e2e): cover model, tag and access group routes for Management/UI coverage (#34118) 2026-07-21 21:59:34 +00:00