mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-14 23:21:35 +00:00
55e82feb8a
Add a tests/e2e/load suite that drives concurrent POST /chat/completions through the live proxy with Locust and asserts an aggregate throughput SLO, filling the gap CodSpeed (no-IO SDK benchmarks) cannot cover. Traffic targets a mock deployment so the number reflects proxy overhead, not provider latency; every knob (users, spawn rate, duration, RPS floor, failure ratio) is env-overridable so the same test runs on berrie-litellm-stage EKS or a local compose stack. Locust runs as a subprocess (it monkey-patches the stdlib with gevent, which deadlocks pytest in-process). The load-marked test is collected last via pytest_collection_modifyitems so it never perturbs latency-sensitive suites. Adds locust to the e2e-dev group and covers reliability.perf.throughput.under_slo. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| access_control | ||
| batches | ||
| claude_code | ||
| coverage_registry | ||
| grafana | ||
| llm_translation | ||
| load | ||
| logging | ||
| management | ||
| quota_management | ||
| router | ||
| bob_the_builder.py | ||
| CLAUDE.md | ||
| conftest.py | ||
| CONTRIBUTING.md | ||
| docker-compose.yml | ||
| e2e_config.py | ||
| e2e_gateway.py | ||
| e2e_http.py | ||
| e2e_result_reporter.py | ||
| lifecycle.py | ||
| models.py | ||
| pytest.ini | ||
| transport.py | ||