litellm/tests/e2e_harness
Kerry Lu b99d2cc06e
test(perf): ramp the load cases until the pod stops keeping up and floor sustained_rps (#47758)
* test(perf): add small_bodies_30rps and mixed_size_requests_4rps load cases

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(perf): run the load cases at 50 rps small and 5 rps mixed, the pod's knee

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(perf): ramp the load cases until the pod stops keeping up and report sustained_rps

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(perf): hand the ramp to the observation and widen the mixed ramp to 14 rps

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(perf): floor the small ramp at 20 rps and emit the per-step readings

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(perf): emit the ramp's hold bound and each step's unexpected statuses

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(perf): read each ramp step's cpu, redis latency and connections from the samplers' windows

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(perf): floor mixed_size_requests_ramp at 8 rps from the lane's 10 rps sustained step

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(perf): compare step readings with math.isclose so the e2e basedpyright gate stays clean

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-20 22:31:08 -07:00
..
batches test(e2e): move the harness self-tests out of tests/e2e (#45172) 2026-10-07 22:20:02 +00:00
claude_code test(e2e): move the harness self-tests out of tests/e2e (#45172) 2026-10-07 22:20:02 +00:00
coverage_registry test(e2e): move the harness self-tests out of tests/e2e (#45172) 2026-10-07 22:20:02 +00:00
guardrails test(e2e): move the harness self-tests out of tests/e2e (#45172) 2026-10-07 22:20:02 +00:00
logging test(e2e): move the harness self-tests out of tests/e2e (#45172) 2026-10-07 22:20:02 +00:00
migrations refactor(lens): connect LiteLLM to the independent Lens service (#45529) 2026-10-09 16:17:05 -07:00
perf test(perf): ramp the load cases until the pod stops keeping up and floor sustained_rps (#47758) 2026-10-20 22:31:08 -07:00
AGENTS.md test(e2e): rename load to perf and consolidate the opt-in into E2E_PERF_TESTS (#46077) 2026-10-12 13:54:03 -07:00
pytest.ini test(e2e): attach perf case metrics and limits as JUnit properties (#46774) 2026-10-15 19:12:07 -07:00
test_e2e_http.py test(e2e): move the harness self-tests out of tests/e2e (#45172) 2026-10-07 22:20:02 +00:00
test_fixture_bundle.py test(e2e): move the harness self-tests out of tests/e2e (#45172) 2026-10-07 22:20:02 +00:00
test_fixture_canonical.py test(e2e): move the harness self-tests out of tests/e2e (#45172) 2026-10-07 22:20:02 +00:00
test_fixture_mode.py test(e2e): move the harness self-tests out of tests/e2e (#45172) 2026-10-07 22:20:02 +00:00
test_idp.py test(e2e): move the harness self-tests out of tests/e2e (#45172) 2026-10-07 22:20:02 +00:00
test_junit_properties.py test(e2e): move the harness self-tests out of tests/e2e (#45172) 2026-10-07 22:20:02 +00:00
test_provider_edge.py test(e2e): move the harness self-tests out of tests/e2e (#45172) 2026-10-07 22:20:02 +00:00
test_proxy_client.py test(e2e): move the harness self-tests out of tests/e2e (#45172) 2026-10-07 22:20:02 +00:00
test_stack_lock.py test(e2e): move the harness self-tests out of tests/e2e (#45172) 2026-10-07 22:20:02 +00:00