Commit graph

5 commits

Author SHA1 Message Date
Kerry Lu
b99d2cc06e
test(perf): ramp the load cases until the pod stops keeping up and floor sustained_rps (#47758)
* test(perf): add small_bodies_30rps and mixed_size_requests_4rps load cases

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(perf): run the load cases at 50 rps small and 5 rps mixed, the pod's knee

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(perf): ramp the load cases until the pod stops keeping up and report sustained_rps

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(perf): hand the ramp to the observation and widen the mixed ramp to 14 rps

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(perf): floor the small ramp at 20 rps and emit the per-step readings

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(perf): emit the ramp's hold bound and each step's unexpected statuses

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(perf): read each ramp step's cpu, redis latency and connections from the samplers' windows

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(perf): floor mixed_size_requests_ramp at 8 rps from the lane's 10 rps sustained step

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(perf): compare step readings with math.isclose so the e2e basedpyright gate stays clean

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-20 22:31:08 -07:00
Kerry Lu
f24036c99d
test(perf): emit the database upgrade run's perf rows and upgrade timings as JUnit properties (#47696)
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-21 00:47:04 +00:00
Kerry Lu
3c0e08146d
test(perf): set every perf ceiling to 1.2x the lane median (#47590)
* test(perf): set every perf ceiling to 1.2x the lane median

* test(perf): derive the upgrade verdict test's samples from the case ceilings

* test(perf): keep memory_growth_megabytes at 100 and 600, the metric swings 60 MB between reruns

* test(perf): keep small litellm_overhead_seconds p99 at 1.2x the lane max, it is the 3rd slowest of 300 requests

---------

Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-20 17:19:22 +00:00
Kerry Lu
dd22dc9132
test(perf): measure Admin UI journey latency in a browser on the 1TB SpendLogs restore (#47486)
* test(perf): measure Admin UI journey latency in a browser on the 1TB restore

* test(perf): settle on rendered rows, chart bars and the usage view trigger

* test(perf): cover the UI journey summaries in the e2e harness tests

* test(perf): fail UI journeys on failed samples and p50/p90 ceilings

* test(perf): add Usage, Models and Keys journeys and seed realistic keys, teams, users and models on the disposable restore

---------

Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-20 16:18:44 +00:00
Kerry Lu
b5037f28d1
test(perf): qualify database upgrades with shared 5 RPS traffic (#46802)
* test(perf): qualify database upgrades with shared 5 RPS traffic

* test(perf): share regular latency and memory limits for upgrades

* fix(perf): escape index format placeholder in upgrade worker

* fix(perf): handle uninitialized multiprocess failure counters

* test(perf): match persisted upgrade requests by call ID

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: kerry <kerry@berri.ai>
2026-10-17 11:39:00 -07:00