mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-18 04:44:37 +00:00
f13d0430e8
* test(e2e): attach perf case metrics and limits as JUnit properties The perf case only printed its metrics table, so a green run left no numbers behind. Each observed metric and its limit now land as <property> entries on the test's JUnit testcase, named perf_<metric>[_p<percentile>] and perf_<metric>[_p<percentile>]_limit, so project-releaser can collect them * test(e2e): compare perf properties exactly instead of through pytest.approx basedpyright flags pytest.approx as partially unknown in the e2e lint step, so the harness test now uses metric values that are exact in binary floating point
9 lines
643 B
INI
9 lines
643 B
INI
[pytest]
|
|
# Tests of the tests/e2e harness itself. They import harness modules by bare name
|
|
# (`from e2e_http import ...`, `from batch_cleanup import ...`) exactly as the suites
|
|
# do, so tests/e2e and each suite folder that owns a module under test go on the path.
|
|
addopts = --strict-markers --strict-config -p no:cacheprovider
|
|
pythonpath = ../e2e ../e2e/batches ../e2e/guardrails ../e2e/logging ../e2e/perf
|
|
markers =
|
|
covers(cell_id, *, exercised_on=()): coverage-registry cell(s) a test covers; exercised here only to prove the collector and the JUnit properties read it
|
|
cli_determinism: drives the real claude CLI for several seconds
|