DataTable loses the comment that narrated its sticky header classes. The
table scrolling e2e spec now types every management API response it
reads, seeds rows through an immutable reduce instead of pushing into
arrays, and deletes what it seeded in each test's finally block instead
of draining a shared mutable list in afterEach.
Refs LIT-4738
Claude-Session: https://claude.ai/code/session_018yW93iDaEMhoQUXcYjus7D
Virtual Keys, Teams, Request Logs and Tags now hand DataTable a bounded
flex chain and use fillHeight, so the app shell main stays the only page
scroller, the rows scroll under a pinned header and the pagination footer
sits at the bottom of the page. DataTable keeps the sticky header inside
its own scroller in maxBodyHeight mode too, which is what let the header
scroll away with the rows on Keys, Teams and Models. Model Hub, Vector
Stores and the team detail keys tab drop their 75vh boxes and flow with
the page scroller.
Adds an e2e spec that fails on the merge base for every one of those
pages and passes at this tip.
Refs LIT-4738
Claude-Session: https://claude.ai/code/session_018yW93iDaEMhoQUXcYjus7D
PR #39604 renamed the Internal Users search box placeholder to "Search by email or ID…" but left searchUsers.spec.ts looking for the old "Search by email…" copy, so e2e_ui_testing has been red on litellm_internal_staging since it merged. Point the locator at the shipped placeholder
`POST /v1/agents/{id}/make_public` appended the agent id to
`litellm.public_agent_groups` and only then called `get_config()`, which
re-applies the DB's `litellm_settings` over the module globals and threw the
append away. The config it saved was therefore a no-op: the endpoint answered
200 with an empty `public_agent_groups`, the agent never reached
`GET /public/agent_hub`, and re-publishing never hit the "already public" 400.
Read the config first, derive the new list from the refreshed globals, save it,
then update the global
Also fixes the e2e model hub spec, which is flaky for a second reason: the
"Make Models Public" modal preselects the groups that are already public, so a
blind click on "Select All" cleared them and left "Next" disabled for the full
15s action timeout. Check the box instead of toggling it, and wait for "Next"
to be enabled before clicking
* fix(ui): paginate request logs by session groups server-side
The logs table server-paginated raw spend logs and then collapsed
multi-call sessions client-side, so a page could render 3 rows while
the footer claimed 25 and sessions straddled pages. Adds an opt-in
group_by_session param to /spend/logs/ui that pages and counts one
representative row per session (DISTINCT ON, newest non-MCP call),
keeps the bounded count contract, enriches whole-session llm/agent
composition counts, and deletes the client-side collapse pipeline.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QxT89fiygmzz2ALcjpu7Ve
* feat(ui): add a 10 rows-per-page option and default request logs to it
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QxT89fiygmzz2ALcjpu7Ve
* fix(ui): key session aggregates per api key in the logs enrichment
Grouped pagination splits a reused session id into one row per api key,
but the enrichment still aggregated by session_id alone, so both rows
showed combined spend and counts. The aggregate query now groups by
(session_id, api_key), the count folds into it (the separate group_by
query is deleted), and each row reads its own key's totals.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QxT89fiygmzz2ALcjpu7Ve
* fix(ui): treat an empty api_key as a real session group value
The spend-log schema defaults api_key to an empty string; truthiness
guards in the enrichment treated it as missing, so keyless multi-call
sessions lost their count and spend. Only None means missing now.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QxT89fiygmzz2ALcjpu7Ve
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
The default password policy from #39381 (12+ chars with upper, lower, number, and special) makes globalSetup's POST /user/update with password "test" fail with a 400, so every role login in the UI e2e suite has been failing since that merge. Seed the roles, the noteam@test.local login, and the second-admin spec with policy-compliant passwords and update the scrypt hash in seed.sql to match.
tests/e2e/test_junit_properties.py fed a hand-rolled FakeItem to
result_properties and attach_result_properties, both typed pytest.Item,
so uv run basedpyright tests/e2e reported 3 reportArgumentType errors on
litellm_internal_staging and every make check that scopes a litellm/ or
tests/e2e/ Python file failed.
Each test now looks up its own collected Item in request.session.items
and applies the covers marker at run time through request.applymarker,
so the coverage registry's collect-only pass never sees the test ids and
the production functions keep their pytest.Item signatures. No casts, no
ignores.
Resolves LIT-6669
`path:line` cannot represent a path that itself contains a colon, and the
one way pytest produces one is a Windows absolute location: separator
normalization turns `C:\app\e2e\a2a\test_x.py` into `C:/app/...`, which
slipped past the leading-slash check and composed the nonsense repo path
`tests/e2e/C:/app/e2e/a2a/test_x.py`.
Reject the colon itself rather than special-casing a drive letter: it is
the character the format reserves, so no path containing one was ever
linkable.
Claude-Session: https://claude.ai/code/session_017dTKXwJkzhtVLzDhePHsKG
The JUnit report is the only thing that leaves the e2e run, and it says
where a test's results came from but never where its code lives. A reader
looking at `test_cell_claimed_only_by_a_skipped_test_is_uncovered` on the
status page has a name and nothing else -- no file, no line, no way to
reach the source short of grepping the repo by hand.
Pytest knows the location; the report format loses it. The `xunit1` family
wrote `file=` and `line=` onto every `<testcase>`, and the `xunit2` default
this suite runs on drops both. Switching families back would change the
document for every consumer of the same XML -- the Buildkite Test Engine
upload and the Loki pipeline included -- so add the location the way this
suite already adds `package` and `covers`: as a `<property>`, which is
purely additive.
`source` is repo-relative and one-based (`tests/e2e/a2a/test_x.py:41`), so
a consumer can build a link without knowing how pytest was started. That
takes normalizing the two launch shapes -- the runner image runs from its
own copy at /app/e2e, a developer runs from the repo root -- which is the
same normalization `package_from_nodeid` was already doing in reverse, now
factored into `suite_parts` so the two cannot drift apart. Paths that
escape the suite, and tests pytest reports no line for, emit an empty
string: a test with no link beats a link that 404s.
Claude-Session: https://claude.ai/code/session_017dTKXwJkzhtVLzDhePHsKG
Two P0 rows in the reliability coverage registry had no test.
reliability.retry.timeout.succeeds_within_retries gets a new file. The model
group is a pair: an always-timing-out deployment holding all of the group's
shuffle weight, and a healthy backup at weight 0. The weighted pick always opens
on the timing-out one, its first Timeout benches it via an allowed_fails_policy
of TimeoutErrorAllowedFails 0, and the retry falls through to the only
deployment left, so the outcome is a completion plus a reported retry with no
random first pick in the middle.
reliability.fallback.context_window.routes_to_fallback joins the existing
fallbacks spec. It registers a genuinely small-context OpenAI deployment, sends
a prompt past its limit so the provider refuses it on length, and reroutes with
context_window_fallbacks, which is the setting that handles that refusal rather
than plain fallbacks.
Both drive real provider calls through router_settings_override, so no config
change and no second proxy is needed. Reliability & Performance goes 16/36 to
18/36.
Claude-Session: https://claude.ai/code/session_01QvQzYztinxj8ZuD5YxbVdL
* test(e2e/ui): cover the Usage page activity tabs
Usage had one test, on Top Virtual Keys. The Key, Model and Endpoint Activity
tabs are the ones an admin reads to answer where the spend went, and none of
them was covered.
Also fixes waitForKeyInDailyActivity, which only read the first page of
/user/daily/activity. The route paginates, so once a run generates more keys
than one page holds, the helper spins for its full 120 seconds and then blames
the rollup for a key the rollup wrote correctly. The Usage page itself already
walks every page; the helper now matches it.
* test(e2e/ui): route the user-creation call through SERVER_ROOT_PATH
Review caught /user/new posting to the server root, which misses the proxy
when it is mounted under a prefix. traffic.ts already had the helper for
this; it is now exported so specs making their own management calls can use
it too.
Also drops the mutable accumulators from the daily-activity paging, which
the repo conventions ask for.
---------
Co-authored-by: Claude <noreply@anthropic.com>
The Teams tests covered creating, deleting and membership, but nothing on the
Settings tab, which is the form that posts the whole team back. That is the
shape behind the reports of a team losing its metadata or its model aliases
after an unrelated edit.
Each test creates its own team rather than editing a seeded one. The limits
test pins the models and members the edit had no business touching, and the
alias test calls the new alias with a team key instead of trusting the
readback, since an alias the router never resolves reads the same either way.
Co-authored-by: Claude <noreply@anthropic.com>
* test(e2e/ui): stop the suite failing on things that are not regressions
Five tests in the UI suite fail for reasons that have nothing to do with the
product being broken, which is enough to keep the whole leg red.
Two need a premium proxy and fail hard without one: Regenerate Key renders
disabled when the proxy is unlicensed, and /model/new refuses a team-scoped
deployment. Both now skip without LITELLM_LICENSE, the way three other tests
in this suite already do.
Three consumed a seeded fixture: Delete key, Delete a team and remove a member
each destroyed the row they needed, so the retries CI runs with were guaranteed
to fail and the suite could not run twice against one database. They now create
what they destroy.
Top Virtual Keys ranks by spend and every mock deployment costs $0, so which
keys make the list came down to how ties happened to sort. It now sends its
traffic through a priced deployment and earns its place.
* test(e2e/ui): clean up the fixtures these tests create
Review caught two leaks: the priced deployment the usage test registers and
the user the team-admin test adds both outlived the run, so repeated runs
grew shared state that later routing and rosters can see.
Also brings in the paginated daily-activity read. /user/daily/activity pages
its per-key breakdown and the helper only read the first page, so the usage
test spent its full timeout blaming the rollup for a key the rollup wrote.
* test(e2e/ui): read the licence from the proxy, not the runner
Review pointed out that checking LITELLM_LICENSE in the runner's environment
describes the wrong machine: Playwright can be pointed at a proxy configured
somewhere else, and then the skip either hides coverage or runs a premium
test against an unlicensed target.
The admin session JWT already carries the premium_user claim the dashboard
itself reads to enable these controls, so both skips now use that.
* test(e2e/ui): clean up fixtures on the failure path too
Review caught both cleanups sitting at the end of the test body, where a
failing assertion skips them, and both discarding the response so a refused
delete passed quietly. They move to afterEach and assert the delete landed.
The priced deployment matters most: left behind it keeps its custom pricing
and goes on changing what later runs route and what they cost.
* test(e2e/ui): wait for the priced deployment to become routable
The Top Virtual Keys test registered a priced deployment and sent the key's
traffic through it on the next line, so on the deployed stack it failed with
"no healthy deployments for e2e-usage-priced-...": /model/new had written
the row but the router had not picked it up yet.
Polls a ping until the deployment answers before the test sends the request
it measures, matching what the addModel spec already does for a model added
through the UI. A ping that fails writes no spend log, so the retries cannot
move the ranking this test asserts.
* test(e2e/ui): register fixtures for cleanup before the step that can fail
Review found both helpers handing their id back to the caller to record, with
a failure-prone call in between: the priced deployment was registered after
the routability wait, and the added user after /team/member_add. Either
failing left the resource in the shared database with nothing tracking it.
Both now take the teardown list and add themselves as soon as the resource
exists, so the afterEach removes it however the rest of setup goes.
* test(e2e/ui): resolve the priced deployment for teardown by name
Review pointed out the remaining gap: /model/new can persist the deployment
and still answer non-2xx, and the id was only recorded after the response was
asserted, so that path left it behind with its custom pricing.
The name is now claimed before the request and teardown looks it up in
/model/info, so a create that saved without answering 2xx is still removed and
one that never saved is simply not there.
* test(e2e/ui): claim the member id before creating the user
/user/new can persist the user and still answer non-2xx, and the id is chosen
by the test rather than returned by the proxy, so registering it before the
call is what closes the last create-failure path.
Teardown now skips an id whose user is not there, so claiming it up front
cannot fail a run where the create never landed.
* test(e2e/ui): wait out the router reload when resolving a deployment to delete
/model/info answers from the router, not from the database, and /model/new
catches and logs a failed in-request reload while still answering 2xx. A
deployment can therefore be persisted and absent from the listing until the
next reload, which is where teardown was giving up and leaking it.
Teardown now retries the lookup for a little over one
PROXY_CONFIG_RELOAD_INTERVAL_SECONDS before treating the name as never
persisted, so the only names it skips are the ones that really are not there.
* test(e2e/ui): prove a stored credential survives a config reload before using it
The Test Connect assertion has been failing intermittently on the full-suite
runs. Artifacts from litellm-e2e-ui build 165 show the UI sending
litellm_credential_name and the proxy answering with the credential unapplied:
raw_request_api_base was https://api.openai.com/v1/ rather than the mock base
the credential carries, and the call died on an upstream 404 for the model. The
same credential had resolved on three probes eight seconds earlier.
The proxy's periodic credential refresh takes a database snapshot, prunes any
in-memory credential missing from it, then re-adds the snapshot. A credential
created while that is in flight gets pruned and stays gone until the next tick,
and load_credentials_from_list fails open onto the ambient key, so nothing in
the error names the credential.
The existing pre-check asked for three consecutive probe successes, but they
completed in under a second, so they could not span a refresh. Space them so
the run covers a whole interval, which is what proves the credential survived a
refresh and is therefore stable.
* test(e2e/ui): find a database-only deployment through the search listing
/model/info answers from the router, so a deployment that reached the database
while /model/new's in-request reload failed is invisible there, and waiting on
the next reload only helps if reconciliation eventually picks it up.
/v2/model/info?search= runs a bounded query against the model table and
deliberately returns rows the router does not hold, so it resolves those
deployments to the id /model/delete needs. Falling back to it removes the wait
as well: absent from both listings now means the deployment never persisted.
* test(e2e/ui): delete the temporary member without a lookup that can skip it
Teardown asked /user/info first and treated any non-2xx as absence, so a
transient failure on the lookup silently skipped the delete and left the user
behind, which is the leak the claimed id was meant to close.
/user/delete answers 404 for an id that is not there, so it can carry both
cases on its own: 404 means the create never persisted, and anything else that
is not 2xx now fails the teardown instead of passing quietly.
* test(e2e/ui): reach the database fallback when the router listing fails
Asserting on /model/info threw before the fallback could run, so a failure on
the router-backed listing aborted teardown and left the deployment persisted,
which is the leak the fallback was added to close.
The router listing is best-effort now: an unreadable response just falls
through to the search-backed one. That listing is the authoritative answer to
whether the deployment exists, so it is the one that has to be readable, and a
name missing from it is a create that never persisted.
---------
Co-authored-by: Claude <noreply@anthropic.com>
The Logs page had coverage for opening a request and for the End User filter,
but nothing for the filters an on-call engineer actually reaches for: whose
key made the request, and which requests failed.
Each test mints its own keys and asserts on request ids it generated itself,
so a filter that quietly does nothing fails on the other key's row still being
on screen rather than passing because our own row happens to be there.
Co-authored-by: Claude <noreply@anthropic.com>
* test(e2e/ui): cover creating, testing and deleting a guardrail
The Guardrails page had no browser coverage. The RC checklist covers it by
hand against a live Presidio, which is why it has always been skipped in CI.
These drive the LiteLLM content filter instead, which runs inside the proxy,
so the whole flow is exercised without a third-party moderation service. The
create test does not stop at the table row: it sends a prompt carrying the
keyword it just banned and asserts the gateway refuses it, then sends a clean
prompt through the same guardrail and asserts it is served.
* test(e2e/ui): delete the guardrails these tests create
Review caught the fixtures being left behind. Guardrails are database rows
that show up in the table and in the playground's list, so a run that leaves
them changes what the next run sees.
Also trims the comments that restated what the helpers already say.
* test(e2e/ui): fail the run when guardrail teardown does not delete
Review caught the afterEach discarding the DELETE response, so a failed
cleanup finished quietly and left the guardrail for the next run to trip on.
* test(e2e/ui): wait for a new guardrail to reach the request path
The wizard test drove one chat completion immediately after creating the
guardrail and required a 400. A trace from the deployed stack shows the
record is stored correctly (blocked_words, action BLOCK, block_on_violation)
and the call six seconds later is still served unguarded, so the first
request can land before the proxy picks the guardrail up.
Polls the same call to the same 400 instead, which keeps the assertion and
lets the refresh land. If it never blocks, this stays red, which is what we
want it to say.
---------
Co-authored-by: Claude <noreply@anthropic.com>
* test(e2e/ui): cover the Budgets page create, edit and delete flows
The Budgets page had no browser coverage at all, so an admin creating or
editing a spend cap through the UI was only exercised by hand at RC time.
Each test reads the budget back from /budget/list, a different route from
the one the table renders, so a row that only exists in the table's cache
does not pass. The edit test pins the rate limits an unrelated spend-cap
edit has no business touching.
* test(e2e/ui): trim comments that restate the test steps
Review flagged the explanatory comments as restating ordinary setup rather
than explaining anything. Keeps the two that carry the regression rationale
for an assertion and drops the rest.
---------
Co-authored-by: Claude <noreply@anthropic.com>
A single-instance run cannot reproduce the two-region setup, but the
regression is fully visible in one: the alias must survive to /key/info
unrewritten, and the alias-granted key must still list the server's
tools. The broken write path stored the resolved server id instead.
mgmt.cache_settings.update.happy_path and
mgmt.config_override.hashicorp_vault.happy_path were the last two uncovered
Management/UI cells, and neither can be covered against the shared proxy the
e2e suites run on. Both routes reconfigure the whole process rather than a
resource the test owns.
/cache/settings persists whatever it receives into a row that outranks the YAML
cache_params and is re-applied on a timer, so a partial write downgrades a TLS
cluster to a plaintext standalone node and every later Redis call hangs. That is
what took out 60 of 72 tests on 2026-07-25 and got the original test removed in
PR #34664.
/config_overrides/hashicorp_vault has the same shape: a POST sets the HCP_VAULT_*
env vars, swaps litellm.secret_manager_client process-wide, and writes a row the
config-reload poll re-applies, so every os.environ/ lookup on the pod resolves
against the test's Vault until the DELETE lands. Its constructor also never
dials Vault, so a POST to a bogus address still returns 200 and a smoke test
built on it would pass for the wrong reason.
Keeping rows we have decided not to cover only inflates the denominator, so drop
them and record the reasoning where someone would go to write the test. Filing
the isolated-proxy harness they both need separately; the cells come back with
it.
Management/UI goes 75/77 to 75/75, headline 402/544 to 402/542.
Replace table tbody and data-slot locators with getByRole, restore prior
public MCP hub entries instead of clearing the whitelist on cleanup, seed
the public agent via the append-semantics per-agent route, and rework
mutable cleanup state into const-scoped try/finally blocks
UsersTable overrides DataTable's default noDataMessage with its own
EmptyState, so the row reads "No users found" rather than "No results".
Assert that, and pair it with the seeded user being absent so the check
cannot pass while the filter silently does nothing.
Two assertions were checking the wrong thing. The anchoring tests read
getByRole("listbox"), which resolves to SelectPrimitive.List; that sits at
full content height inside the popup that clips and scrolls it, so the box
overlapped the trigger even when nothing visible did. Measure the popup.
The SSO-ID search expected zero rows, but DataTable renders a "No results"
message row when a filter matches nothing, so the count is one. Assert the
empty state the user actually sees.
Adds Playwright coverage for the RC checklist items an audit marked
automatable today: Playground to Logs hand-off, public Agent/MCP hub
tabs, team models in the Playground dropdown via a team key, Add Model
with a stored credential, internal user team key creation, a second
admin account, team model deletion, and Presidio guardrail CRUD without
a live sidecar. Seeds e2e-team-keygen with the /key/generate member
permission so the internal user key flow avoids the team-list cache lag
The consolidated popup test only asserted the options never cover the
trigger, so opening above the trigger with room below it, the regression
PR #38554 fixed, would have passed. Split it back into a below-trigger
case and a cramped-viewport case. The header test accepted a single pixel
of vertical intersection; require the refresh control's centre to sit
within the tab row instead.
The migration smoke waited on `getByRole("button", { expanded: false })`
after clicking it. Playwright re-resolves that locator on every retry, so
once the clicked group flipped to expanded it matched the next collapsed
group instead, and the assertion could never pass. Count the remaining
collapsed groups and wait for that count to drop by one.
The UI e2e suite had a class of assertions that pin how the dashboard is
built rather than what it does, so an ordinary refactor turns them red
without any user-visible change.
Geometry. The auto-router template select had two tests made of pixel
arithmetic plus a data-side="bottom" check, which is Base UI's own
positioner signal. The regression they guard (#38554) is a popup opening
on top of the control that spawned it, so both cases collapse to one
invariant: the options never cover the trigger. It now runs at both
viewport heights and reads the popup as role=listbox. The models header
test compared the tabs and refresh centers within 2px, which a padding
change flips; it now asserts the two share a row.
Structure. The logs drawer test walked xpath=../../.. from a text node
and read collapsed state off chevron icon classes. SectionHeader now
renders a real disclosure button with aria-expanded, and its two copy
buttons carry distinct names instead of both being "Copy". Sidebar group
toggles expose aria-expanded too, so the migration spec can ask for a
collapsed group by state rather than by nesting depth.
Positional lookups. keyRow.locator("button").first(), row.locator("td")
.first() and getByTestId(grid).locator("div").first() all named a
position where they meant an action; they now name the control. Table
scoping moves from "table tbody" to role=row.
Timing. Nine waitForTimeout calls are gone. Every assertion that followed
them already retried to its own timeout, so the sleeps only slowed the
run down.
Both files under tests/users/ were wrapped in test.skip("...", () => {}),
which registers one skipped test and never runs the body, so the four
tests inside had never executed and were written against a UI that has
since changed (the search placeholder is "Search by email…", the ID
filters moved into a drawer, pagination is labelled "Go to previous
page"). Rewritten against the current surface: the suite goes from 104
collected tests to 107.
Left in place deliberately: the chip and dialog-footer data-slot
selectors, because the accessible names they work around live in
components/ui/, which is shadcn CLI-managed and not hand-edited.
The route-level regression test returns a real prisma row from a mocked
update and asserts both routes serialize it to a 200 with the toggled
blocked flag, which is exactly the path that raised AttributeError before
the validator guard. Also binds the loop variable in the e2e poll lambda
(ruff B023).
The driver retries rate-limit-shaped failures twice with a 65s backoff, but
RATE_LIMIT_SHAPED_RE only matches 429 / rate limit / too many requests /
throttled / CLI timeout. A saturated provider does not always say any of
those: litellm-e2e-pr build 182 turned a green cell red on
status 503: litellm.ServiceUnavailableError: BedrockException -
{"message":"Bedrock is unable to process your request."}
which matched nothing and so was never retried.
Add TRANSIENT_UPSTREAM_SHAPED_RE for 503, 529, "service unavailable",
"overloaded" and Bedrock's "unable to process your request", and have the
retry loop ask is_retryable_shaped (either shape) instead of
is_rate_limit_shaped.
Kept as a second pattern rather than widened into the first on purpose. The
conftest feeds RATE_LIMIT_SHAPED_RE into the rate-limit summary, which the
binary-search helper reads to decide whether to lower a provider's request
rate. A 503 says the provider is out of capacity, not that we are asking too
often, so folding it in there would keep ratcheting our rate down against a
condition our rate never caused.
Retry-loop logging now names which shape fired, so a saturated upstream is
distinguishable from a throttled one in the run output.
Verified against the literal failure text from build 182: is_retryable_shaped
is True while is_rate_limit_shaped stays False.
Note this covers CLI-driven rows only. HTTP-probe rows (tool_search,
count_tokens) have no retry layer at all, so the specific cell that failed in
build 182 is still unprotected -- that is a separate change to http_probe.
The first pass accepted any empty completion whose finish_reason was "length",
which also swallowed a fallback that produced nothing at all. Require the
response to have billed completion tokens as well, so empty content is
accepted only when the budget was demonstrably spent on non-visible reasoning.
Asserts on completion_tokens rather than reasoning_tokens because the latter
is provider-optional; with empty content and a refusal of null, consumed
completion tokens are reasoning by elimination, since visible text would be
content. Both counts are reported in the failure message.
Folds the three body accessors onto one _parsed helper instead of re-parsing
per call, and adds completion_tokens_of / reasoning_tokens_of alongside.
max_tokens=64 caps reasoning plus visible output on gpt-5.5, so the fallback
target can legitimately return finish_reason="length" with empty content.
litellm-e2e build 90 hit exactly that: the response cost of $0.002005 backs
out to 64 completion tokens at gpt-5.5's $3e-05/token, i.e. the whole budget
spent reasoning about "say hi" with none left to answer. The fallback itself
worked -- 200, served by gpt-5.5-2026-04-23, x-litellm-attempted-fallbacks
present -- so the only thing that failed was an assertion about OpenAI's token
budgeting rather than about routing.
Raise the reliability helper's budget to 512 and accept empty content only
when finish_reason is "length". Empty content under any other finish_reason
still fails, so the tests keep catching a fallback that returns nothing for a
reason we do control.
The relaxed assertion lives in the helper both reliability fallback tests
share, so test_timeout_routes_to_fallback is covered too; it has the same
shape and had not tripped yet.
Four cases in tests/e2e/quota_management/budgets, driving real OpenAI calls
through a group whose shared pool is drained to exhaustion: the spender key
stays blocked, a key that spent nothing of its own is blocked by the same
pool, a sibling group with no budget keeps serving, and the budget read
reports the spend drawn against the group.
Adds set/get/delete access group budget to BudgetClient and the four
matching rows to the coverage registry.
The sidebar toggle sits absolutely positioned over the drawer's flex row. With
the sidebar expanded it lands on the sidebar header, but once collapsed it lands
on the drawer header, which is sticky at z-chrome (10). The named-z-scale
refactor moved the toggle from z-20 to z-raised (1), so from then on the header
painted over it and swallowed the click: collapse the trace list and there was
no way to bring it back.
Moves the toggle to z-floating (30) and folds the two mirrored buttons into one,
since they only ever differed by icon, label and handler.
Covered by a Playwright spec, which is the tier that can see the layering: the
button stays visible and enabled either way, so the pre-fix failure is a click
interception that jsdom cannot reproduce.
