litellm/tests/e2e
mubashir1osmani 12b3d163b6 test(e2e/ui): derive the target from the environment and self-seed its users
tests/e2e/ui could only ever run against a locally provisioned stack, so it has
never run in the e2e pod. Three things pinned it there.

playwright.config.ts hardcoded baseURL http://localhost:4000 and globalSetup.ts
hardcoded the same host for /update/ui_settings and /ui/login. Both now derive
from LITELLM_PROXY_URL, falling back to localhost:4000 so local runs are
unchanged.

Storage state paths were repo-relative, so globalSetup wrote
admin.storageState.json into cwd. That is read-only in the runner image under
readOnlyRootFilesystem. They now sit under PLAYWRIGHT_STATE_DIR, defaulting to
"." for local runs. The per-role constants the specs import directly were
switched over too, so writer and readers stay in agreement.

globalSetup logged in as five roles whose users only exist if fixtures/seed.sql
was applied, and it throws when a login fails, so against an unseeded target the
whole suite died before a single spec ran. It now creates those users through the
proxy API first. ProxyAdmin is untouched since it logs in as "admin" with the
master key and needs no user row.

Verified against a live proxy that the seeding path works: /user/new returns 200,
/user/update sets the password, and /v2/login then authenticates as that user.
Worth knowing that /user/new accepts a password field but does not persist it
("User has no password set" on login), which is why the password is applied in a
second call.

All 25 spec files and 86 tests still collect, and the resolved values are correct
(trailing slash stripped, storage state redirected).

Teams, keys and the org from seed.sql are not self-seeded yet, so specs that
reference those ids still need them present on the target.
2026-07-25 11:39:20 -07:00
..
a2a test(a2a): assert the property agent returns listings, not just non-empty text (#34515) 2026-07-24 11:03:28 -07:00
access_control test(true_rabbit): cover passthrough headers, batch assume-role, gemini, vllm, bedrock guardrails, batch rate-limit mapping (#33843) 2026-07-20 16:15:55 -07:00
batches refactor(e2e): drop require_env, read os.environ where a cred is used (#34413) 2026-07-23 19:14:22 +00:00
claude_code refactor(e2e): fold claude_code HTTP probes onto shared Gateway methods (#33760) 2026-07-18 19:03:01 +00:00
coverage_registry test(e2e): cover /vllm passthrough files + batches (skipped, needs backend) (#34432) 2026-07-25 10:31:50 -07:00
guardrails test(e2e): make the bedrock guardrail test match the guardrail it points at (#34568) 2026-07-25 10:32:20 -07:00
llm_translation test(e2e): set reasoning_effort=none for gpt-5.6 chat tool calls (#34569) 2026-07-25 10:34:45 -07:00
load test(e2e): move Admin UI Playwright suite to tests/e2e/ui (#34196) 2026-07-22 19:43:10 +00:00
logging test(e2e): add Other suite and Guardrails coverage incl. an MCP tool-call guardrail (#34149) 2026-07-21 14:06:29 -07:00
management test(musty_leopard): cover customer chat/messages cost + streaming paths (#34164) 2026-07-21 18:57:11 -07:00
mcp test(e2e): cover MCP access-group tool selection at key creation (#34480) 2026-07-24 16:18:40 -07:00
other test(e2e): add Other suite and Guardrails coverage incl. an MCP tool-call guardrail (#34149) 2026-07-21 14:06:29 -07:00
quota_management refactor(e2e): drop require_env, read os.environ where a cred is used (#34413) 2026-07-23 19:14:22 +00:00
router test(e2e): add reliability suite covering fallback, timeout, and cache behavior (#34023) 2026-07-20 23:06:46 +00:00
ui test(e2e/ui): derive the target from the environment and self-seed its users 2026-07-25 11:39:20 -07:00
CLAUDE.md test(e2e): drive a real Linear OAuth MCP through chat completions under both ingress headers 2026-07-22 23:29:10 -07:00
conftest.py test(e2e): cover key max_budget blocks on personal, team, and team-member keys (#33895) 2026-07-22 17:48:18 -07:00
CONTRIBUTING.md chore(e2e): remove tests/e2e/docker-compose.yml (#33837) 2026-07-18 12:50:23 -07:00
e2e_config.py refactor(e2e): drop require_env, read os.environ where a cred is used (#34413) 2026-07-23 19:14:22 +00:00
e2e_db.py test(e2e): guard destructive spend-log truncate behind an explicit opt-in (#33751) 2026-07-20 08:47:39 -07:00
e2e_http.py Merge pull request #34052 from BerriAI/litellm_a2a_e2e_tests 2026-07-21 18:23:48 -07:00
junit_properties.py refactor(e2e): replace bespoke result reporter with standard JUnit report (#33758) 2026-07-17 20:53:22 +00:00
lifecycle.py test(e2e): cover key max_budget blocks on personal, team, and team-member keys (#33895) 2026-07-22 17:48:18 -07:00
models.py test(e2e): cover MCP access-group tool selection at key creation (#34480) 2026-07-24 16:18:40 -07:00
otel_client.py test(e2e): add Other suite and Guardrails coverage incl. an MCP tool-call guardrail (#34149) 2026-07-21 14:06:29 -07:00
proxy_client.py test(e2e): cover model update persisting to /model/info (#34017) 2026-07-20 21:13:54 +00:00
pytest.ini test(e2e): add weekly session-anomaly load test against real providers 2026-07-21 14:54:02 -07:00
transport.py test(e2e): cover 12 non-core LLM coverage registry cells (#34123) 2026-07-22 00:43:41 +00:00