litellm/tests/e2e/perf
yuneng-jiang b18188dee2
test(e2e): retire the quiet_stack lock and batch the claude_code model registration (#49984)
* test(e2e): retire the quiet_stack lock

quiet_stack held an exclusive host-wide lock for six tests, draining every
other xdist worker while they ran. On the full e2e lane that left 35-38% of
worker time idle. Each of those tests now lives where it can run without
pausing the suite

- the two memory tests move to perf/ behind the perf marker, so the perf lane
  runs them on its own fresh stack. Their refusing deployments use a literal
  key because the perf stack carries no provider keys
- the stored-request size check from the memory test becomes an integration
  test on an owned proxy that stores prompts
- the discount and margin spend-row tests are dropped from e2e. Their
  breakdown assertions join the existing integration pricing test, which
  already owns the discount and margin repricing flow
- the Bedrock guardrail cost test stays in e2e without the pricing reset
  fixture, since nothing else on the stack writes pricing config now
- the MCP semantic tool filter test turns on a proxy-wide setting, so it gets
  a new proxy_wide_config opt-in marker and runs after the main run

* test(e2e): register the claude_code deployments as one batch

The claude_code session fixture registered its 24 deployments one at a time on every xdist worker, and each one waited out the 15 s propagation window on its own, about 6 minutes per worker. ProxyClient.register_models posts the whole batch, polls each model until every replica serves it, then waits out propagation once from the last write. A model that fails to post or never becomes servable comes back as ModelNotRegistered, and an unservable one is deleted, so the fixture keeps reporting per-deployment failures the way it did
2026-11-04 09:55:53 -08:00
..
cases test(perf): ramp the load cases until the pod stops keeping up and floor sustained_rps (#47758) 2026-10-20 22:31:08 -07:00
setup test(perf): ramp the load cases until the pod stops keeping up and floor sustained_rps (#47758) 2026-10-20 22:31:08 -07:00
conftest.py test(e2e): rename load to perf and consolidate the opt-in into E2E_PERF_TESTS (#46077) 2026-10-12 13:54:03 -07:00
load_client.py test(e2e): rename load to perf and consolidate the opt-in into E2E_PERF_TESTS (#46077) 2026-10-12 13:54:03 -07:00
locust_load.py test(e2e): rename load to perf and consolidate the opt-in into E2E_PERF_TESTS (#46077) 2026-10-12 13:54:03 -07:00
locustfile.py test(e2e): rename load to perf and consolidate the opt-in into E2E_PERF_TESTS (#46077) 2026-10-12 13:54:03 -07:00
phase_budget.py test(e2e): rename load to perf and consolidate the opt-in into E2E_PERF_TESTS (#46077) 2026-10-12 13:54:03 -07:00
proxy_usage.py test(e2e): rename load to perf and consolidate the opt-in into E2E_PERF_TESTS (#46077) 2026-10-12 13:54:03 -07:00
session_anomaly.py test(e2e): perf rig (out-of-process mocks, PerformanceTestCase runner, per-request limits) and the small_bodies_5rps case (#46713) 2026-10-15 20:14:40 +00:00
test_database_upgrade.py test(e2e): retire the quiet_stack lock and batch the claude_code model registration (#49984) 2026-11-04 09:55:53 -08:00
test_perf_cases.py test(e2e): retire the quiet_stack lock and batch the claude_code model registration (#49984) 2026-11-04 09:55:53 -08:00
test_redis_chaos_e2e.py test(e2e): rename load to perf and consolidate the opt-in into E2E_PERF_TESTS (#46077) 2026-10-12 13:54:03 -07:00
test_reliability_memory_e2e.py test(e2e): retire the quiet_stack lock and batch the claude_code model registration (#49984) 2026-11-04 09:55:53 -08:00
test_ui_journeys.py test(e2e): retire the quiet_stack lock and batch the claude_code model registration (#49984) 2026-11-04 09:55:53 -08:00
test_weekly_session_anomaly_e2e.py test(e2e): rename load to perf and consolidate the opt-in into E2E_PERF_TESTS (#46077) 2026-10-12 13:54:03 -07:00
UI_QUERIES.md test(perf): let the UI journey fixture run on a restore upgraded by a released migrations image (#47628) 2026-10-20 19:49:58 +00:00
weekly_anomaly_config.yml test(e2e): rename load to perf and consolidate the opt-in into E2E_PERF_TESTS (#46077) 2026-10-12 13:54:03 -07:00