litellm/tests/e2e
ryan-crabbe-berri dbb5b813c1
test(e2e): budget reset diagonal for team, org, user, and #32005 team-member keys (#33771)
* test(e2e): budget reset diagonal for team, org, user, and #32005 team-member keys

Adds E2E-7/8/10/11 from the budget-level x key-kind coverage matrix: each budget
level serves traffic again after its budget_duration window elapses, walking the
same ladder as the enforcement diagonal. New registry rows and tests cover the
team, organization, and internal-user reset rungs, plus the #32005 interplay
where a team-member key frozen by its owner's user budget comes back when the
user's window renews; the bare-key and per-team-member rungs already had coverage

Each case isolates the cap to one entity, drives spend to a budget_exceeded
block, then polls past the window until a call succeeds, holding every refusal
as a budget block so a reset that no-ops (stays blocked forever) or crashes
(leaks a 5xx) fails the test. budget_duration becomes an optional param on the
budget_client create_team / create_user / create_org helpers

* test(e2e): fold the reset diagonal into test_budget_reset_e2e.py and address greptile nits

Move the team / org / user / #32005 reset cases out of the standalone
test_budget_reset_diagonal_e2e.py and into test_budget_reset_e2e.py, absorbing
the pre-existing bare-key reset into the same TestBudgetResetDiagonal spec class
so the whole reset ladder reads as one file (mirroring how the enforcement
diagonal lives in test_budget_enforcement_e2e.py) and the drive/poll helpers are
defined once instead of duplicated across reset files.

Greptile nits: bound the drive phase to under one window (12 attempts x 2s < 30s)
so a block is observed before the reset job can fire, and replace the bare assert
in the poll loop with a pytest.fail that prints the HTTP status, so a provider 429
or a crashed reset path is distinguishable from a budget block at a glance.

* test(e2e): trim reset diagonal docstrings back to the file's original style

* test(e2e): inline single-use drive-loop bounds

* test(e2e): cut the reset module docstring to one line

* test(e2e): make the org reset test wait for a scheduled window (bugbot)

/organization/new stores budget_duration without scheduling budget_reset_at, so
the reset job's NULL catch-up branch zeroes org spend on its first 5-10s tick;
the org reset test could pass off that catch-up instead of a real window roll
(tracked as LIT-4570). The test now reads the org's budget_id and polls
/budget/info until budget_reset_at is scheduled before driving spend, so the
recovery it observes can only come from a genuine window expiry. Verified live:
the org case now runs ~33s (a full window) instead of beating the rescheduler
2026-07-18 03:02:22 +00:00
..
access_control docs(tests/e2e): align docs with the hard-fail-on-dead-proxy contract and scope the no-unit-tests rule (#33755) 2026-07-17 12:56:10 -07:00
batches docs(tests/e2e): align docs with the hard-fail-on-dead-proxy contract and scope the no-unit-tests rule (#33755) 2026-07-17 12:56:10 -07:00
claude_code test(e2e): skip flaky OpenAI GPT cells; raise multi-window max_tokens (#33799) 2026-07-17 19:07:18 -07:00
coverage_registry test(e2e): budget reset diagonal for team, org, user, and #32005 team-member keys (#33771) 2026-07-18 03:02:22 +00:00
llm_translation docs(tests/e2e): align docs with the hard-fail-on-dead-proxy contract and scope the no-unit-tests rule (#33755) 2026-07-17 12:56:10 -07:00
logging test(e2e): datadog log delivery for streamed routes, read back from the real datadog api (#33566) 2026-07-16 19:37:07 -07:00
management docs(tests/e2e): align docs with the hard-fail-on-dead-proxy contract and scope the no-unit-tests rule (#33755) 2026-07-17 12:56:10 -07:00
mcp test(e2e): mcp suite for key-without-access denial (#33752) 2026-07-17 16:04:43 -07:00
quota_management test(e2e): budget reset diagonal for team, org, user, and #32005 team-member keys (#33771) 2026-07-18 03:02:22 +00:00
router docs(tests/e2e): align docs with the hard-fail-on-dead-proxy contract and scope the no-unit-tests rule (#33755) 2026-07-17 12:56:10 -07:00
CLAUDE.md test(e2e): mcp suite for key-without-access denial (#33752) 2026-07-17 16:04:43 -07:00
conftest.py refactor(e2e): remove bob_the_builder; drive remediation from a Grafana alert (provisioned outside the repo) (#33749) 2026-07-17 14:23:21 -07:00
CONTRIBUTING.md docs(tests/e2e): align docs with the hard-fail-on-dead-proxy contract and scope the no-unit-tests rule (#33755) 2026-07-17 12:56:10 -07:00
docker-compose.yml test(e2e): mcp suite for key-without-access denial (#33752) 2026-07-17 16:04:43 -07:00
e2e_config.py test(e2e): datadog log delivery for streamed routes, read back from the real datadog api (#33566) 2026-07-16 19:37:07 -07:00
e2e_gateway.py fix(e2e/claude_code): unblock stage collection, align proxy env names, register compat models (#33433) 2026-07-16 11:05:31 -07:00
e2e_http.py test(e2e): otel trace completeness on streaming chat, messages, and responses (LIT-3787) (#33234) 2026-07-14 20:23:54 -07:00
junit_properties.py refactor(e2e): replace bespoke result reporter with standard JUnit report (#33758) 2026-07-17 20:53:22 +00:00
lifecycle.py test(e2e): add live batches suite across providers and routing scenarios (#30958) 2026-07-02 08:05:23 -07:00
models.py test(e2e): mcp suite for key-without-access denial (#33752) 2026-07-17 16:04:43 -07:00
pytest.ini refactor(e2e): move budgets and spend_tracking suites under quota_management 2026-07-11 16:14:21 -07:00
transport.py test(e2e): probe the full spend read surface including schema-hidden routes (#32267) 2026-07-06 14:02:05 -07:00