litellm/tests/e2e
mateo-berri b2b69d4f4d fix(e2e/ui): seed the playwright fixtures over the management API
The ui suite has never gotten past globalSetup on stage. It logs in as five
roles, and four of them are password users that only ever existed because
run_e2e.sh and CircleCI piped fixtures/seed.sql into a database they had direct
access to. The packaged e2e image has no such access, so on stage only the
master-key admin could log in; the second role sat on /ui/login until
waitForURL hit its 30s timeout and the whole run died before a single test.

Confirmed against the stage gateway: POST /login as admin returns 303, as
adminviewer@test.local returns 401, and /user/info 404s for every seeded id.

Create the same budget, organization, users, teams and keys through the
management endpoints in globalSetup instead, so the suite seeds whatever
deployment it can reach rather than whatever database it can open. seed.sql is
gone and the psql steps go with it; the API path is also the more faithful
fixture, since seed.sql hand-wrote team rows no product code path produces
(populated admins/members arrays, unhashed key tokens).

Two behaviors the seeder has to work around, both verified live: /team/new
auto-adds the caller as an admin, which would have made the proxy admin a
member of the team the suite needs it to be a stranger to, so that membership
is removed after each team; and /team/delete and /user/delete are
all-or-nothing, so one unknown id in a batch deletes nothing, which means each
id gets its own request for the cleanup to survive a partial state.

Verified end to end against the stage gateway: seeding runs in ~6s, all five
role logins return 303, the four teams come back with exactly their seeded
members and no stray admin, all five keys resolve to the right owner and team,
and re-running after deleting a seeded team and key restores both.
2026-07-31 21:09:44 -07:00
..
a2a test(e2e): stop racing control-plane writes across the mcp, a2a, guardrail and passthrough suites (#34833) 2026-07-27 21:19:49 +00:00
access_control test(true_rabbit): cover passthrough headers, batch assume-role, gemini, vllm, bedrock guardrails, batch rate-limit mapping (#33843) 2026-07-20 16:15:55 -07:00
batches test(e2e): skip the batch rate-limiter spend-row test pending LIT-5027 (#35301) 2026-07-31 01:20:04 +00:00
claude_code refactor(e2e): fold claude_code HTTP probes onto shared Gateway methods (#33760) 2026-07-18 19:03:01 +00:00
coverage_registry feat(proxy): add GET /management/v1/budgets (#35310) 2026-07-31 11:47:05 -07:00
guardrails test(e2e): remove the Presidio guardrail suite (#35129) 2026-07-29 14:20:39 -07:00
llm_translation test(e2e): skip passthrough headers test until stage can route custom paths to provider creds (#34980) 2026-07-28 11:36:09 -07:00
load test(e2e): skip the throughput SLO load test pending LIT-5054 (#35381) 2026-07-31 17:35:46 +00:00
logging test(e2e): make MCP and prometheus e2e tests robust to data-plane sync lag (#34854) 2026-07-27 19:22:52 -07:00
management feat(proxy): add GET /management/v1/budgets (#35310) 2026-07-31 11:47:05 -07:00
mcp test(e2e): skip the three Datadog MCP tool-call tests pending LIT-5052 (#35380) 2026-07-31 17:19:20 +00:00
other test(e2e): add Other suite and Guardrails coverage incl. an MCP tool-call guardrail (#34149) 2026-07-21 14:06:29 -07:00
quota_management test(e2e): align budget e2e with the team-key budget hierarchy (#35276) 2026-07-30 21:09:50 +00:00
router test(e2e): add reliability suite covering fallback, timeout, and cache behavior (#34023) 2026-07-20 23:06:46 +00:00
ui fix(e2e/ui): seed the playwright fixtures over the management API 2026-07-31 21:09:44 -07:00
CLAUDE.md fix(e2e): exclude skipped tests from coverage-registry numerator 2026-07-30 22:19:30 -07:00
conftest.py test(e2e): cover key max_budget blocks on personal, team, and team-member keys (#33895) 2026-07-22 17:48:18 -07:00
CONTRIBUTING.md chore(e2e): remove tests/e2e/docker-compose.yml (#33837) 2026-07-18 12:50:23 -07:00
e2e_config.py fix(e2e): stop tests from breaking the shared proxy for every suite after them (#34664) 2026-07-25 23:12:55 +00:00
e2e_db.py test(e2e): guard destructive spend-log truncate behind an explicit opt-in (#33751) 2026-07-20 08:47:39 -07:00
e2e_http.py Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_fix_responses_bridge_streaming_contract 2026-07-27 14:34:49 -07:00
junit_properties.py refactor(e2e): replace bespoke result reporter with standard JUnit report (#33758) 2026-07-17 20:53:22 +00:00
lifecycle.py test(e2e): cover key max_budget blocks on personal, team, and team-member keys (#33895) 2026-07-22 17:48:18 -07:00
models.py test(e2e): cover MCP access-group tool selection at key creation (#34480) 2026-07-24 16:18:40 -07:00
otel_client.py test(e2e): add Other suite and Guardrails coverage incl. an MCP tool-call guardrail (#34149) 2026-07-21 14:06:29 -07:00
proxy_client.py test(e2e): cover model update persisting to /model/info (#34017) 2026-07-20 21:13:54 +00:00
pytest.ini test(e2e): add weekly session-anomaly load test against real providers 2026-07-21 14:54:02 -07:00
transport.py test(e2e): cover /v1/images/edits (#34476) 2026-07-25 10:38:07 -07:00