litellm/tests/e2e/load
yassin 55e82feb8a test(e2e): add Locust throughput load test that runs last
Add a tests/e2e/load suite that drives concurrent POST /chat/completions through the live proxy with Locust and asserts an aggregate throughput SLO, filling the gap CodSpeed (no-IO SDK benchmarks) cannot cover. Traffic targets a mock deployment so the number reflects proxy overhead, not provider latency; every knob (users, spawn rate, duration, RPS floor, failure ratio) is env-overridable so the same test runs on berrie-litellm-stage EKS or a local compose stack.

Locust runs as a subprocess (it monkey-patches the stdlib with gevent, which deadlocks pytest in-process). The load-marked test is collected last via pytest_collection_modifyitems so it never perturbs latency-sensitive suites. Adds locust to the e2e-dev group and covers reliability.perf.throughput.under_slo.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-07-17 17:27:19 +00:00
..
conftest.py test(e2e): add Locust throughput load test that runs last 2026-07-17 17:27:19 +00:00
load_client.py test(e2e): add Locust throughput load test that runs last 2026-07-17 17:27:19 +00:00
locust_load.py test(e2e): add Locust throughput load test that runs last 2026-07-17 17:27:19 +00:00
locustfile.py test(e2e): add Locust throughput load test that runs last 2026-07-17 17:27:19 +00:00
test_chat_completions_throughput_e2e.py test(e2e): add Locust throughput load test that runs last 2026-07-17 17:27:19 +00:00