litellm/tests/e2e
yuneng-jiang 0721163cac
test(e2e/ui): cover team-scoped model visibility, re-editing litellm params, and model health checks (#40039)
* test(e2e/ui): cover team-scoped model visibility, re-editing litellm params, and model health checks

Three Models and Endpoints flows had no end-to-end coverage, and all three
keep coming back as bug reports.

modelsByTeam walks an internal user through the Current team control and
asserts the table lists exactly what each team grants. It creates one
deployment that belongs to no team, proves that deployment is visible under
Personal, then proves it is absent under both seeded teams, so an empty
table cannot pass the same assertions.

editLitellmParams adds a temperature and a custom pair to a deployment,
saves, then re-edits the temperature and drops the custom pair. It checks
both update request bodies, polls the stored deployment until the new
temperature is there, reloads the page to confirm the second save is what
renders, and sends one chat completion to prove the deployment still serves.

modelHealthStatus runs the health check on a reachable deployment and on one
pointed at a dead port, asserts the healthy and unhealthy cells and the two
detail dialogs, and reloads to confirm both statuses are stored.

Every deployment these specs create carries a unique name and is deleted in
afterEach, including on the failure path.

* test(e2e/ui): find health rows across every page of the health table

The health table pages server-side at 50 rows with no search box, so on a
proxy carrying more deployments than that the two deployments the spec
creates can land on a later page and the lookup finds nothing.

Row lookups now walk the pages, using the table's own page indicator to
know when to advance and when to wrap back to the first page.

* test(e2e/ui): build the created deployment ids without mutating the array

* test(ui): scope model deployments to Playwright fixtures
2026-09-08 22:49:40 -07:00
..
a2a test(e2e): settle control-plane writes across every replica, not just one 2026-08-07 19:36:30 -07:00
access_control test(e2e): require a 200 inside the regenerate grace window and drop the helper docstrings 2026-09-05 11:53:22 -07:00
batches test(e2e/batches): run the list assertion when a batch completes before cancel 2026-09-04 18:50:47 -07:00
claude_code fix(e2e): wait for every gateway before using a new model and keep the network rerun 2026-09-05 16:10:40 -07:00
coverage_registry Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_/guardrail-automation-testing-3ecd3d 2026-09-07 11:38:59 -07:00
gateway ci(e2e): declare the embedding model the access_control canary calls 2026-09-05 18:58:07 -07:00
guardrails test(guardrails): document audit prompt storage prerequisite 2026-09-07 11:43:32 -07:00
llm_translation Merge pull request #39953 from BerriAI/litellm_/litellm-e2e-flaky-test-2159ae 2026-09-05 16:04:45 -07:00
load test(e2e): move load/perf testing out of the main suite and drop the vllm passthrough test (#35820) 2026-08-04 14:12:25 -07:00
logging test(e2e): cover presidio post_call, tool_permission, and weave logging cells (#39279) 2026-09-05 13:03:28 -07:00
management test(e2e): prove the virtual key lifecycle on every gateway replica (#40023) 2026-09-07 11:30:46 -07:00
mcp test(e2e): drop coverage registry cell for the alias-grant test 2026-09-01 09:03:47 -07:00
other test(e2e): add Other suite and Guardrails coverage incl. an MCP tool-call guardrail (#34149) 2026-07-21 14:06:29 -07:00
quota_management test(e2e): cover model access group budgets against a live proxy 2026-08-29 15:46:49 -07:00
router fix(router): skip the refusing deployment when retrying a non-transient error 2026-09-05 22:25:13 -07:00
ui test(e2e/ui): cover team-scoped model visibility, re-editing litellm params, and model health checks (#40039) 2026-09-08 22:49:40 -07:00
CLAUDE.md test(e2e): judge /v1/messages streaming on the clock, not on the provider's delta count 2026-09-05 14:44:21 -07:00
conftest.py ci(e2e): record the e2e suite weekly and replay it on weekdays with zero egress (#38163) 2026-08-24 23:49:03 -04:00
CONTRIBUTING.md test(guardrails): document audit prompt storage prerequisite 2026-09-07 11:43:32 -07:00
e2e_config.py chore: merge litellm_internal_staging into litellm_/e2e-test-performance-7d53be 2026-09-05 16:15:11 -07:00
e2e_db.py test(e2e): guard destructive spend-log truncate behind an explicit opt-in (#33751) 2026-07-20 08:47:39 -07:00
e2e_http.py test(e2e): drop the explanatory sentence from the StreamingResponse docstring 2026-09-05 14:53:12 -07:00
fixture_bundle.py feat(e2e): record and replay streamed provider responses chunk-for-chunk 2026-08-24 12:51:44 -07:00
fixture_canonical.py test(e2e): key multipart uploads by structured part identity 2026-08-21 19:32:02 -07:00
fixture_mode.py feat(e2e): move record/replay to the provider edge (LIT-5745) 2026-08-19 18:39:15 -07:00
junit_properties.py Refuse a source path carrying a colon 2026-09-01 16:02:54 -07:00
lifecycle.py test(e2e): pin openai_passthrough routing, cost logging, and file list isolation 2026-08-20 03:03:35 -07:00
models.py Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_/guardrail-automation-testing-3ecd3d 2026-09-07 11:38:59 -07:00
otel_client.py test(e2e): harden the suite against response-cache cross-talk, slow providers and single upstream blips (#37957) 2026-08-22 14:47:03 -07:00
provider_edge.py fix(e2e): record a streamed chunk only after its downstream write lands 2026-08-24 13:26:58 -07:00
proxy_client.py test(e2e): prove the virtual key lifecycle on every gateway replica (#40023) 2026-09-07 11:30:46 -07:00
pytest.ini ci(e2e): record the e2e suite weekly and replay it on weekdays with zero egress (#38163) 2026-08-24 23:49:03 -04:00
test_e2e_http.py test(e2e): annotate new stream-timing locals as Final and trim the docstrings 2026-09-05 14:49:56 -07:00
test_fixture_bundle.py feat(e2e): record and replay streamed provider responses chunk-for-chunk 2026-08-24 12:51:44 -07:00
test_fixture_canonical.py test(e2e): pin query params and multipart form fields as replay match-key identity 2026-08-20 15:59:34 -04:00
test_fixture_mode.py feat(e2e): move record/replay to the provider edge (LIT-5745) 2026-08-19 18:39:15 -07:00
test_junit_properties.py test(e2e): read JUnit properties off the real collected pytest Item 2026-09-01 19:12:07 -07:00
test_provider_edge.py fix(e2e): record a streamed chunk only after its downstream write lands 2026-08-24 13:26:58 -07:00
test_proxy_client.py test(e2e): prove the virtual key lifecycle on every gateway replica (#40023) 2026-09-07 11:30:46 -07:00
transport.py test(e2e): harden the suite against response-cache cross-talk, slow providers and single upstream blips (#37957) 2026-08-22 14:47:03 -07:00