litellm/tests/e2e
ryan-crabbe-berri 595e72472a
test(e2e): assert the long budget window keeps blocking after the short window resets (#33832)
* test(e2e): assert the long budget window keeps blocking after the short window resets

The multi-window budget tests proved the tight window blocks and self-heals
but never asserted the other direction: a long (1d) window whose cap the
accumulated spend already crossed must keep refusing calls even inside a
fresh short window. Adds one test per file (key and team) that drives spend
to a block, waits for the short window's reset_at to strictly advance (the
reset job zeroes that window's counter in the same pass), then polls until
the refusal is attributed to the 1d window ("over 1d budget"), failing
immediately if any call succeeds or a non-budget error leaks. Harness gains
per-window reset_at readback: BudgetWindowState in models.py and
key_window_reset_at / team_window_reset_at on BudgetClient.

* refactor(e2e): hoist shared budget-suite helpers into budget_client

drive_to_block and as_datetime existed as five and four per-file copies in
the budgets suite; both move to budget_client with each file keeping a thin
delegating wrapper so call sites and per-file pacing stay unchanged. The
three /team/info readers in budget_client now share a private _team_info.
Also guard the long-window reset_at snapshots with explicit non-None asserts
so the midnight-roll diagnostic cannot misreport when the window is missing
from the info response (greptile P2s).

* docs(e2e): tighten the multi-window module docstrings

* refactor(e2e): type window reset_at as datetime and expose plain window readers

BudgetWindowState.reset_at becomes a pydantic-parsed datetime, so the
multi-window tests compare real datetimes instead of hand-parsing strings.
The duration-keyed accessors are replaced by two plain readers,
key_budget_windows and team_budget_windows, with the pure window_reset_at
lookup exported; the client no longer encodes one test's access pattern.

* test(e2e): name the tiny short-window cap and comment the wait loops

* test(e2e): surface the 429 budget-block assert in the multi-window tests

drive_to_block now returns the blocking response so a test body can assert
on its shape; the two long-window tests assert status 429 explicitly, which
also pins the multi-window enforcement path's HTTP mapping (the enforcement
suite only covers the single-budget path). Other callers ignore the return
and are unchanged.

* refactor(e2e): scope this PR to the multi-window test, drop the cross-suite hoist

The helper hoist rewrote four unrelated budget test files (reset, reset_advances,
team_member_reset, user_across_keys) to pull drive_to_block and as_datetime out
of budget_client, which is refactor churn beyond this PR's multi-window scope.
This restores those four to their pre-PR state and gives the two multi-window
tests their own inline drive-to-block loop again, so the PR touches only the
multi-window feature: its two tests plus the budget_client window readers and the
reset_at datetime typing they actually use. The suite-wide helper dedup can land
on its own PR

* docs(e2e): number the long-window key test steps inline

* docs(e2e): number the long-window team test steps inline
2026-07-18 18:00:15 -07:00
..
access_control test(e2e): rename Gateway to ProxyClient and expose it as a session-scoped fixture (#33750) 2026-07-18 18:41:18 +00:00
batches test(e2e): harden stage flakes for batches, UI, and MCP (#33831) 2026-07-18 19:11:54 +00:00
claude_code refactor(e2e): fold claude_code HTTP probes onto shared Gateway methods (#33760) 2026-07-18 19:03:01 +00:00
coverage_registry test(e2e): spendlog cost for streaming /v1/messages via responses bridge (#33753) 2026-07-18 14:12:26 -07:00
llm_translation test(e2e): cover /v1/responses OpenAI vision and Anthropic basic (#33838) 2026-07-18 14:00:43 -07:00
load fix(e2e): migrate load suite from e2e_gateway to ProxyClient (#33839) 2026-07-18 19:41:03 +00:00
logging test(e2e): rename Gateway to ProxyClient and expose it as a session-scoped fixture (#33750) 2026-07-18 18:41:18 +00:00
management test(e2e): harden stage flakes for batches, UI, and MCP (#33831) 2026-07-18 19:11:54 +00:00
mcp test(e2e): harden stage flakes for batches, UI, and MCP (#33831) 2026-07-18 19:11:54 +00:00
quota_management test(e2e): assert the long budget window keeps blocking after the short window resets (#33832) 2026-07-18 18:00:15 -07:00
router test(e2e): rename Gateway to ProxyClient and expose it as a session-scoped fixture (#33750) 2026-07-18 18:41:18 +00:00
CLAUDE.md test(e2e): spendlog cost for streaming /v1/messages via responses bridge (#33753) 2026-07-18 14:12:26 -07:00
conftest.py test(e2e): add Locust throughput load test that runs last (#33748) 2026-07-18 11:57:41 -07:00
CONTRIBUTING.md chore(e2e): remove tests/e2e/docker-compose.yml (#33837) 2026-07-18 12:50:23 -07:00
e2e_config.py test(e2e): harden stage flakes for batches, UI, and MCP (#33831) 2026-07-18 19:11:54 +00:00
e2e_http.py test(e2e): cover /v1/responses openai basic nonstream and stream (#33830) 2026-07-18 13:25:19 -07:00
junit_properties.py refactor(e2e): replace bespoke result reporter with standard JUnit report (#33758) 2026-07-17 20:53:22 +00:00
lifecycle.py test(e2e): rename Gateway to ProxyClient and expose it as a session-scoped fixture (#33750) 2026-07-18 18:41:18 +00:00
models.py test(e2e): assert the long budget window keeps blocking after the short window resets (#33832) 2026-07-18 18:00:15 -07:00
proxy_client.py test(e2e): spendlog cost for streaming /v1/messages via responses bridge (#33753) 2026-07-18 14:12:26 -07:00
pytest.ini test(e2e): add Locust throughput load test that runs last (#33748) 2026-07-18 11:57:41 -07:00
transport.py test(e2e): probe the full spend read surface including schema-hidden routes (#32267) 2026-07-06 14:02:05 -07:00