litellm/tests/e2e
Yassin Kortam 442fdc181e
docs(tests/e2e): align docs with the hard-fail-on-dead-proxy contract and scope the no-unit-tests rule (#33755)
The e2e docs claimed `e2e`-marked tests skip when no proxy answers the
liveness probe, but the harness has always hard-failed: conftest.py's
pytest_runtest_setup calls pytest.fail, its module docstring states
"hard failures only ... never skip", and logging/conftest.py forbids
skipping outright. Align the docs to the code so the single most
important contract reads the same everywhere; a dead proxy turns a run
red instead of being silently skipped and mistaken for a pass. The
per-suite conftest docstrings that described the shared hook as a
"proxy liveness skip" are corrected to "liveness gate" for the same
reason.

Also scope the no-unit-tests hard rule to what it means: never
substitute a unit test for e2e feature coverage, while explicitly
allowing tests that cover the harness itself (e.g.
coverage_registry/test_collector.py), which carry no e2e marker and
run whether or not a proxy is up.

No product code and no harness logic changed.

Resolves LIT-4554
2026-07-17 12:56:10 -07:00
..
access_control docs(tests/e2e): align docs with the hard-fail-on-dead-proxy contract and scope the no-unit-tests rule (#33755) 2026-07-17 12:56:10 -07:00
batches docs(tests/e2e): align docs with the hard-fail-on-dead-proxy contract and scope the no-unit-tests rule (#33755) 2026-07-17 12:56:10 -07:00
claude_code test(e2e): harness fixes for stage job green (skips + router/UI/budget) (#33634) 2026-07-16 20:30:30 -07:00
coverage_registry test(e2e): budget refusals are 429 for bare keys and team caps block every team key (#33632) 2026-07-17 11:29:16 -07:00
llm_translation docs(tests/e2e): align docs with the hard-fail-on-dead-proxy contract and scope the no-unit-tests rule (#33755) 2026-07-17 12:56:10 -07:00
logging test(e2e): datadog log delivery for streamed routes, read back from the real datadog api (#33566) 2026-07-16 19:37:07 -07:00
management docs(tests/e2e): align docs with the hard-fail-on-dead-proxy contract and scope the no-unit-tests rule (#33755) 2026-07-17 12:56:10 -07:00
quota_management docs(tests/e2e): align docs with the hard-fail-on-dead-proxy contract and scope the no-unit-tests rule (#33755) 2026-07-17 12:56:10 -07:00
router docs(tests/e2e): align docs with the hard-fail-on-dead-proxy contract and scope the no-unit-tests rule (#33755) 2026-07-17 12:56:10 -07:00
bob_the_builder.py test: litellm fix failing tests (#32577) 2026-07-09 13:54:45 -07:00
CLAUDE.md docs(tests/e2e): align docs with the hard-fail-on-dead-proxy contract and scope the no-unit-tests rule (#33755) 2026-07-17 12:56:10 -07:00
conftest.py feat(e2e): emit structured E2E_RESULT lines for package status history (#33578) 2026-07-16 15:01:01 -07:00
CONTRIBUTING.md docs(tests/e2e): align docs with the hard-fail-on-dead-proxy contract and scope the no-unit-tests rule (#33755) 2026-07-17 12:56:10 -07:00
docker-compose.yml test: e2e staging leftovers (#33613) 2026-07-16 17:52:41 -07:00
e2e_config.py test(e2e): datadog log delivery for streamed routes, read back from the real datadog api (#33566) 2026-07-16 19:37:07 -07:00
e2e_gateway.py fix(e2e/claude_code): unblock stage collection, align proxy env names, register compat models (#33433) 2026-07-16 11:05:31 -07:00
e2e_http.py test(e2e): otel trace completeness on streaming chat, messages, and responses (LIT-3787) (#33234) 2026-07-14 20:23:54 -07:00
e2e_result_reporter.py feat(e2e): emit structured E2E_RESULT lines for package status history (#33578) 2026-07-16 15:01:01 -07:00
lifecycle.py test(e2e): add live batches suite across providers and routing scenarios (#30958) 2026-07-02 08:05:23 -07:00
models.py test: e2e staging leftovers (#33613) 2026-07-16 17:52:41 -07:00
pytest.ini refactor(e2e): move budgets and spend_tracking suites under quota_management 2026-07-11 16:14:21 -07:00
transport.py test(e2e): probe the full spend read surface including schema-hidden routes (#32267) 2026-07-06 14:02:05 -07:00