mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-14 23:21:35 +00:00
3d35eee560
The floor was an absolute fleet number, so it asserted replicas x per-replica rate and went red on how many gateway pods happened to be warm rather than on the request path. The test now measures one replica first, with a short serial pass that only ever occupies a single pod, and requires the concurrent phase to reach at least that rate. A serial latency budget carries the request-path assertion the floor used to imply, and both hold at one replica or seven. Zero-error runs that "sustained 16.7 RPS" were queueing, not slow requests: the load model is a mock_response deployment with no upstream, a single-worker replica serves it in about 57ms, and 100 closed-loop users against 1/0.057 RPS of capacity sit at 6s each by Little's law. The runner also kept locust's --json summary and threw away everything else, so a run where 93% of requests failed said nothing about what they got. It now passes --csv, reads the failure breakdown back, and reports locust's own generator-saturation warnings, both folded into the assertion messages. Resolves LIT-5054 |
||
|---|---|---|
| .. | ||
| conftest.py | ||
| load_client.py | ||
| load_constants.py | ||
| locust_load.py | ||
| locustfile.py | ||
| session_anomaly.py | ||
| test_chat_completions_throughput_e2e.py | ||
| test_locust_load.py | ||
| test_session_anomaly.py | ||
| test_weekly_session_anomaly_e2e.py | ||
| weekly_anomaly_config.yml | ||