litellm/tests/e2e
Yassin Kortam acb54a03e2 test(e2e): cover key lifecycle and proxy config lifecycle behavior
Adds six coverage-registry cells to the `other` suite, all driven against a
live proxy:

- other.key_mgmt.regenerate.grace_period_honored: a key rotated with a
  grace_period keeps authenticating for the window and is refused after it, with
  a sibling key rotated without a grace period as the control so the survival
  cannot be an auth-cache artifact
- other.key_mgmt.spend_reset.resets_to_value: reset_spend puts a key's recorded
  spend back and a key its own max_budget had blocked serves again
- other.auth.virtual_key.route_group_allowed: the allow side of allowed_routes;
  the llm_api_routes group reaches chat, embeddings and the model catalog while a
  management route stays refused 403
- other.lifecycle.startup.config_loads: the config file's model_list is live in
  the routing catalog (db_model false) and serves a real completion, and the
  file's store_model_in_db is in effect
- other.lifecycle.startup.env_vars_resolved: the redis cache built once at
  startup from os.environ/ host and port references actually serves a repeat
  request; the cache block is used because credential references get resolved
  again at call time and cannot witness boot-time resolution
- other.config.runtime_update.applies_at_runtime: /config/update reaches the
  running router (read back off the live object, not a config row), routes a
  completion through the alias it added, and takes it back again

Spend accounting is buffered, so the reset test settles on a value that has
stopped moving before it snapshots, then asserts only what a late flush cannot
violate: the reset reports a previous value no smaller than the settled one,
reports zero, and /key/info is polled until it agrees. Exact equality against a
single sample would go red on correct behavior.

/config/update is the only writer of model_group_alias, it rewrites the whole map
and offers neither a per-key write nor a version to compare against. Every write
therefore compares the aliases the test does not own against a baseline as full
name and target pairs, before the write and again after; anything added, removed
or repointed stops the test naming the difference and nothing is written, so a
concurrent writer's change is never reverted with a stale view. A second alias of
the test's own stands across the exchange and is asserted to survive. Everything
else is namespaced per run and deleted on teardown, so the shared proxy is left
as it was found.
2026-07-27 17:37:46 -07:00
..
a2a fix(e2e): stop tests from breaking the shared proxy for every suite after them (#34664) 2026-07-25 23:12:55 +00:00
access_control test(true_rabbit): cover passthrough headers, batch assume-role, gemini, vllm, bedrock guardrails, batch rate-limit mapping (#33843) 2026-07-20 16:15:55 -07:00
batches fix(e2e): stop tests from breaking the shared proxy for every suite after them (#34664) 2026-07-25 23:12:55 +00:00
claude_code refactor(e2e): fold claude_code HTTP probes onto shared Gateway methods (#33760) 2026-07-18 19:03:01 +00:00
coverage_registry fix(e2e): stop tests from breaking the shared proxy for every suite after them (#34664) 2026-07-25 23:12:55 +00:00
guardrails fix(e2e): stop tests from breaking the shared proxy for every suite after them (#34664) 2026-07-25 23:12:55 +00:00
llm_translation fix(e2e): stop tests from breaking the shared proxy for every suite after them (#34664) 2026-07-25 23:12:55 +00:00
load test(e2e): move Admin UI Playwright suite to tests/e2e/ui (#34196) 2026-07-22 19:43:10 +00:00
logging test(e2e): add Other suite and Guardrails coverage incl. an MCP tool-call guardrail (#34149) 2026-07-21 14:06:29 -07:00
management fix(e2e): stop tests from breaking the shared proxy for every suite after them (#34664) 2026-07-25 23:12:55 +00:00
mcp test(e2e): cover MCP access-group tool selection at key creation (#34480) 2026-07-24 16:18:40 -07:00
other test(e2e): cover key lifecycle and proxy config lifecycle behavior 2026-07-27 17:37:46 -07:00
quota_management refactor(e2e): drop require_env, read os.environ where a cred is used (#34413) 2026-07-23 19:14:22 +00:00
router test(e2e): add reliability suite covering fallback, timeout, and cache behavior (#34023) 2026-07-20 23:06:46 +00:00
ui fix(e2e/ui): resolve dashboard base URL from env instead of hardcoding localhost (#34739) 2026-07-27 10:19:32 -07:00
CLAUDE.md test(e2e): drive a real Linear OAuth MCP through chat completions under both ingress headers 2026-07-22 23:29:10 -07:00
conftest.py test(e2e): cover key max_budget blocks on personal, team, and team-member keys (#33895) 2026-07-22 17:48:18 -07:00
CONTRIBUTING.md chore(e2e): remove tests/e2e/docker-compose.yml (#33837) 2026-07-18 12:50:23 -07:00
e2e_config.py fix(e2e): stop tests from breaking the shared proxy for every suite after them (#34664) 2026-07-25 23:12:55 +00:00
e2e_db.py test(e2e): guard destructive spend-log truncate behind an explicit opt-in (#33751) 2026-07-20 08:47:39 -07:00
e2e_http.py test(e2e): cover /v1/images/edits (#34476) 2026-07-25 10:38:07 -07:00
junit_properties.py refactor(e2e): replace bespoke result reporter with standard JUnit report (#33758) 2026-07-17 20:53:22 +00:00
lifecycle.py test(e2e): cover key max_budget blocks on personal, team, and team-member keys (#33895) 2026-07-22 17:48:18 -07:00
models.py test(e2e): cover MCP access-group tool selection at key creation (#34480) 2026-07-24 16:18:40 -07:00
otel_client.py test(e2e): add Other suite and Guardrails coverage incl. an MCP tool-call guardrail (#34149) 2026-07-21 14:06:29 -07:00
proxy_client.py test(e2e): cover model update persisting to /model/info (#34017) 2026-07-20 21:13:54 +00:00
pytest.ini test(e2e): add weekly session-anomaly load test against real providers 2026-07-21 14:54:02 -07:00
transport.py test(e2e): cover /v1/images/edits (#34476) 2026-07-25 10:38:07 -07:00