litellm/tests/e2e
yuneng-jiang a43f128a74
test(e2e): add coverage registry and collector (#32304)
Introduce the e2e coverage denominator: 282 behavior cells across the six
tracking modules (LLMs, MCPs, Management/UI, Reliability & Performance,
Logging & Guardrails, Other), one validated YAML row each, plus a collector
that diffs the registry against @pytest.mark.covers markers and reports
coverage per module.

The registry rows validate against a pydantic discriminated union so a row
cannot carry a field from another module. The collector is static: a
collect-only pass reads the markers, so it runs no test and needs no live
proxy. Register the covers marker suite-wide so that pass works under
--strict-markers.

This is a draft for review. Tiers are proposed rather than signed off, and a
few cells still need a support check or a prune.
2026-07-07 15:51:20 -04:00
..
access_control test(e2e): migrate access-control and inference-endpoint regression tests (#32016) 2026-07-05 01:39:10 +00:00
batches Merge pull request #32166 from BerriAI/litellm_e2e_batches_ocr_model_registration 2026-07-05 02:15:10 +00:00
budgets feat(proxy): add key-level budget_fallbacks to reroute requests when a per-model budget is exceeded (#31783) 2026-07-03 12:20:12 -07:00
coverage_registry test(e2e): add coverage registry and collector (#32304) 2026-07-07 15:51:20 -04:00
llm_translation test(e2e): probe the full spend read surface including schema-hidden routes (#32267) 2026-07-06 14:02:05 -07:00
logging test(e2e): close coverage gaps across chat/responses, provider features, batches, prometheus, and langfuse eviction (#32165) 2026-07-04 18:56:52 -07:00
management test(e2e): add management suite covering key/team/user/org lifecycle and route permissions (#32300) 2026-07-06 19:11:27 -07:00
realtime test: add realtime proxy e2e suite across providers (#30960) 2026-06-26 09:36:49 -07:00
spend_tracking test(e2e): probe the full spend read surface including schema-hidden routes (#32267) 2026-07-06 14:02:05 -07:00
CLAUDE.md test(e2e): add management suite covering key/team/user/org lifecycle and route permissions (#32300) 2026-07-06 19:11:27 -07:00
conftest.py test(e2e): add coverage registry and collector (#32304) 2026-07-07 15:51:20 -04:00
CONTRIBUTING.md chore(e2e): untrack gateway config and document e2e test location (#31914) 2026-07-02 19:22:02 -07:00
docker-compose.yml chore(e2e): untrack gateway config and document e2e test location (#31914) 2026-07-02 19:22:02 -07:00
e2e_config.py test: add e2e tests for spend, budgets and llms (#30869) 2026-06-24 15:01:57 -07:00
e2e_gateway.py test(e2e): probe the full spend read surface including schema-hidden routes (#32267) 2026-07-06 14:02:05 -07:00
e2e_http.py test(e2e): add live batches suite across providers and routing scenarios (#30958) 2026-07-02 08:05:23 -07:00
lifecycle.py test(e2e): add live batches suite across providers and routing scenarios (#30958) 2026-07-02 08:05:23 -07:00
models.py test(e2e): add management suite covering key/team/user/org lifecycle and route permissions (#32300) 2026-07-06 19:11:27 -07:00
pytest.ini test: add e2e tests for spend, budgets and llms (#30869) 2026-06-24 15:01:57 -07:00
test_e2e_gateway.py test(e2e): probe the full spend read surface including schema-hidden routes (#32267) 2026-07-06 14:02:05 -07:00
test_lifecycle.py test: add e2e tests for spend, budgets and llms (#30869) 2026-06-24 15:01:57 -07:00
test_transport.py test(e2e): probe the full spend read surface including schema-hidden routes (#32267) 2026-07-06 14:02:05 -07:00
transport.py test(e2e): probe the full spend read surface including schema-hidden routes (#32267) 2026-07-06 14:02:05 -07:00