litellm/tests/e2e/perf/setup
Nate Armstrong f13d0430e8
test(e2e): attach perf case metrics and limits as JUnit properties (#46774)
* test(e2e): attach perf case metrics and limits as JUnit properties

The perf case only printed its metrics table, so a green run left no numbers
behind. Each observed metric and its limit now land as <property> entries on
the test's JUnit testcase, named perf_<metric>[_p<percentile>] and
perf_<metric>[_p<percentile>]_limit, so project-releaser can collect them

* test(e2e): compare perf properties exactly instead of through pytest.approx

basedpyright flags pytest.approx as partially unknown in the e2e lint step,
so the harness test now uses metric values that are exact in binary floating
point
2026-10-15 19:12:07 -07:00
..
corpus test(e2e): perf rig (out-of-process mocks, PerformanceTestCase runner, per-request limits) and the small_bodies_5rps case (#46713) 2026-10-15 20:14:40 +00:00
mocks test(e2e): perf rig (out-of-process mocks, PerformanceTestCase runner, per-request limits) and the small_bodies_5rps case (#46713) 2026-10-15 20:14:40 +00:00
__init__.py test(e2e): perf rig (out-of-process mocks, PerformanceTestCase runner, per-request limits) and the small_bodies_5rps case (#46713) 2026-10-15 20:14:40 +00:00
datastore.py test(e2e): perf rig (out-of-process mocks, PerformanceTestCase runner, per-request limits) and the small_bodies_5rps case (#46713) 2026-10-15 20:14:40 +00:00
locustfile.py test(e2e): perf users start staggered with jitter, tool-call streams get a first token, small_bodies_5rps sends from the mixed case's callers (#46776) 2026-10-15 17:14:20 -07:00
observe.py test(e2e): attach perf case metrics and limits as JUnit properties (#46774) 2026-10-15 19:12:07 -07:00
performance_test_case.py test(e2e): perf rig (out-of-process mocks, PerformanceTestCase runner, per-request limits) and the small_bodies_5rps case (#46713) 2026-10-15 20:14:40 +00:00
plan.py test(e2e): perf rig (out-of-process mocks, PerformanceTestCase runner, per-request limits) and the small_bodies_5rps case (#46713) 2026-10-15 20:14:40 +00:00
podcpu.py test(e2e): perf rig (out-of-process mocks, PerformanceTestCase runner, per-request limits) and the small_bodies_5rps case (#46713) 2026-10-15 20:14:40 +00:00
provision.py test(e2e): perf rig (out-of-process mocks, PerformanceTestCase runner, per-request limits) and the small_bodies_5rps case (#46713) 2026-10-15 20:14:40 +00:00
run.py test(e2e): perf rig (out-of-process mocks, PerformanceTestCase runner, per-request limits) and the small_bodies_5rps case (#46713) 2026-10-15 20:14:40 +00:00
wire.py test(e2e): perf rig (out-of-process mocks, PerformanceTestCase runner, per-request limits) and the small_bodies_5rps case (#46713) 2026-10-15 20:14:40 +00:00