litellm/tests/e2e/perf/setup
Kerry Lu b99d2cc06e
test(perf): ramp the load cases until the pod stops keeping up and floor sustained_rps (#47758)
* test(perf): add small_bodies_30rps and mixed_size_requests_4rps load cases

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(perf): run the load cases at 50 rps small and 5 rps mixed, the pod's knee

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(perf): ramp the load cases until the pod stops keeping up and report sustained_rps

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(perf): hand the ramp to the observation and widen the mixed ramp to 14 rps

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(perf): floor the small ramp at 20 rps and emit the per-step readings

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(perf): emit the ramp's hold bound and each step's unexpected statuses

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(perf): read each ramp step's cpu, redis latency and connections from the samplers' windows

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(perf): floor mixed_size_requests_ramp at 8 rps from the lane's 10 rps sustained step

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(perf): compare step readings with math.isclose so the e2e basedpyright gate stays clean

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-20 22:31:08 -07:00
..
corpus test(e2e): perf rig (out-of-process mocks, PerformanceTestCase runner, per-request limits) and the small_bodies_5rps case (#46713) 2026-10-15 20:14:40 +00:00
mocks test(e2e): perf rig (out-of-process mocks, PerformanceTestCase runner, per-request limits) and the small_bodies_5rps case (#46713) 2026-10-15 20:14:40 +00:00
__init__.py test(e2e): perf rig (out-of-process mocks, PerformanceTestCase runner, per-request limits) and the small_bodies_5rps case (#46713) 2026-10-15 20:14:40 +00:00
datastore.py test(perf): ramp the load cases until the pod stops keeping up and floor sustained_rps (#47758) 2026-10-20 22:31:08 -07:00
event_loop.py test(perf): emit per-location seconds, offending locations and the ten longest holds as event_loop_stalls breakdowns (#47757) 2026-10-21 02:54:46 +00:00
locustfile.py test(perf): ramp the load cases until the pod stops keeping up and floor sustained_rps (#47758) 2026-10-20 22:31:08 -07:00
observe.py test(perf): ramp the load cases until the pod stops keeping up and floor sustained_rps (#47758) 2026-10-20 22:31:08 -07:00
performance_test_case.py test(perf): ramp the load cases until the pod stops keeping up and floor sustained_rps (#47758) 2026-10-20 22:31:08 -07:00
plan.py test(perf): ramp the load cases until the pod stops keeping up and floor sustained_rps (#47758) 2026-10-20 22:31:08 -07:00
podcpu.py test(perf): ramp the load cases until the pod stops keeping up and floor sustained_rps (#47758) 2026-10-20 22:31:08 -07:00
provision.py test(e2e): perf rig (out-of-process mocks, PerformanceTestCase runner, per-request limits) and the small_bodies_5rps case (#46713) 2026-10-15 20:14:40 +00:00
ramp.py test(perf): ramp the load cases until the pod stops keeping up and floor sustained_rps (#47758) 2026-10-20 22:31:08 -07:00
run.py test(perf): ramp the load cases until the pod stops keeping up and floor sustained_rps (#47758) 2026-10-20 22:31:08 -07:00
series.py test(perf): ramp the load cases until the pod stops keeping up and floor sustained_rps (#47758) 2026-10-20 22:31:08 -07:00
telemetry.py test(perf): ramp the load cases until the pod stops keeping up and floor sustained_rps (#47758) 2026-10-20 22:31:08 -07:00
ui_journeys.py test(perf): tag a UI journey sample with its API response, not a Next.js prefetch under /ui/ (#47650) 2026-10-20 14:59:34 -07:00
ui_query_fixture.py test(perf): record the row count and size of each table the Admin UI journeys query (#47683) 2026-10-21 00:10:39 +00:00
ui_query_isolation.py test(perf): measure Admin UI journey latency in a browser on the 1TB SpendLogs restore (#47486) 2026-10-20 16:18:44 +00:00
upgrade.py test(perf): ramp the load cases until the pod stops keeping up and floor sustained_rps (#47758) 2026-10-20 22:31:08 -07:00
upgrade_worker.py test(perf): qualify database upgrades with shared 5 RPS traffic (#46802) 2026-10-17 11:39:00 -07:00
wire.py test(perf): ramp the load cases until the pod stops keeping up and floor sustained_rps (#47758) 2026-10-20 22:31:08 -07:00