mirror of
https://github.com/BerriAI/litellm.git
synced 2026-08-28 05:25:59 +00:00
a43f128a74
Introduce the e2e coverage denominator: 282 behavior cells across the six tracking modules (LLMs, MCPs, Management/UI, Reliability & Performance, Logging & Guardrails, Other), one validated YAML row each, plus a collector that diffs the registry against @pytest.mark.covers markers and reports coverage per module. The registry rows validate against a pydantic discriminated union so a row cannot carry a field from another module. The collector is static: a collect-only pass reads the markers, so it runs no test and needs no live proxy. Register the covers marker suite-wide so that pass works under --strict-markers. This is a draft for review. Tiers are proposed rather than signed off, and a few cells still need a support check or a prune.
8 lines
430 B
Python
8 lines
430 B
Python
"""The e2e coverage registry: the denominator for e2e test coverage.
|
|
|
|
`schema.py` defines one validated row per customer-noticeable behavior (a "cell").
|
|
The `*.yaml` files hold the rows, one file per id-prefix. `registry.py` loads and
|
|
validates them; `collector.py` diffs the registry against the `@pytest.mark.covers`
|
|
markers on the live tests and reports coverage per module. See tests/e2e/CLAUDE.md
|
|
for the naming grammar.
|
|
"""
|