litellm/tests/e2e
Yassin Kortam 53a2138d05 test(e2e): cover router strategy selection and context-window fallback
Adds live coverage for four routing-strategy registry cells (cost, tpm,
simple-shuffle weight, latency) and for context-window fallback rerouting. Each
routing test builds a model group whose deployments differ in only the dimension
the strategy under test reads, drives the strategy per request through
router_settings_override, and asserts which deployment served the call from the
x-litellm-model-id header. The fallback test overflows a real gpt-4 deployment's
context window and asserts the same prompt is served by the gpt-5.5 fallback

least_busy.picks_lowest_traffic is deliberately left uncovered. Proving it needs
the router's in-flight tally for a deployment to be observable at the moment of
the pick, and the proxy exposes no such read: the tally is absent from the
in-memory cache dump, prometheus is off, and spend rows only land once a call
finishes. Every barrier available instead observes an earlier layer, which would
make the assertion a probability bet rather than a proof
2026-07-27 17:06:27 -07:00
..
a2a fix(e2e): stop tests from breaking the shared proxy for every suite after them (#34664) 2026-07-25 23:12:55 +00:00
access_control test(true_rabbit): cover passthrough headers, batch assume-role, gemini, vllm, bedrock guardrails, batch rate-limit mapping (#33843) 2026-07-20 16:15:55 -07:00
batches fix(e2e): stop tests from breaking the shared proxy for every suite after them (#34664) 2026-07-25 23:12:55 +00:00
claude_code refactor(e2e): fold claude_code HTTP probes onto shared Gateway methods (#33760) 2026-07-18 19:03:01 +00:00
coverage_registry fix(e2e): stop tests from breaking the shared proxy for every suite after them (#34664) 2026-07-25 23:12:55 +00:00
guardrails fix(e2e): stop tests from breaking the shared proxy for every suite after them (#34664) 2026-07-25 23:12:55 +00:00
llm_translation fix(e2e): stop tests from breaking the shared proxy for every suite after them (#34664) 2026-07-25 23:12:55 +00:00
load test(e2e): move Admin UI Playwright suite to tests/e2e/ui (#34196) 2026-07-22 19:43:10 +00:00
logging test(e2e): add Other suite and Guardrails coverage incl. an MCP tool-call guardrail (#34149) 2026-07-21 14:06:29 -07:00
management fix(e2e): stop tests from breaking the shared proxy for every suite after them (#34664) 2026-07-25 23:12:55 +00:00
mcp test(e2e): cover MCP access-group tool selection at key creation (#34480) 2026-07-24 16:18:40 -07:00
other test(e2e): add Other suite and Guardrails coverage incl. an MCP tool-call guardrail (#34149) 2026-07-21 14:06:29 -07:00
quota_management refactor(e2e): drop require_env, read os.environ where a cred is used (#34413) 2026-07-23 19:14:22 +00:00
router test(e2e): cover router strategy selection and context-window fallback 2026-07-27 17:06:27 -07:00
ui fix(e2e/ui): resolve dashboard base URL from env instead of hardcoding localhost (#34739) 2026-07-27 10:19:32 -07:00
CLAUDE.md test(e2e): drive a real Linear OAuth MCP through chat completions under both ingress headers 2026-07-22 23:29:10 -07:00
conftest.py test(e2e): cover key max_budget blocks on personal, team, and team-member keys (#33895) 2026-07-22 17:48:18 -07:00
CONTRIBUTING.md chore(e2e): remove tests/e2e/docker-compose.yml (#33837) 2026-07-18 12:50:23 -07:00
e2e_config.py fix(e2e): stop tests from breaking the shared proxy for every suite after them (#34664) 2026-07-25 23:12:55 +00:00
e2e_db.py test(e2e): guard destructive spend-log truncate behind an explicit opt-in (#33751) 2026-07-20 08:47:39 -07:00
e2e_http.py test(e2e): cover /v1/images/edits (#34476) 2026-07-25 10:38:07 -07:00
junit_properties.py refactor(e2e): replace bespoke result reporter with standard JUnit report (#33758) 2026-07-17 20:53:22 +00:00
lifecycle.py test(e2e): cover key max_budget blocks on personal, team, and team-member keys (#33895) 2026-07-22 17:48:18 -07:00
models.py test(e2e): cover MCP access-group tool selection at key creation (#34480) 2026-07-24 16:18:40 -07:00
otel_client.py test(e2e): add Other suite and Guardrails coverage incl. an MCP tool-call guardrail (#34149) 2026-07-21 14:06:29 -07:00
proxy_client.py test(e2e): cover model update persisting to /model/info (#34017) 2026-07-20 21:13:54 +00:00
pytest.ini test(e2e): add weekly session-anomaly load test against real providers 2026-07-21 14:54:02 -07:00
transport.py test(e2e): cover /v1/images/edits (#34476) 2026-07-25 10:38:07 -07:00