* test(e2e/ui): cover member role and budget edits, member permission delegation, and team guardrail removal
Three Playwright specs for the Teams flows enterprise customers hit most, each
owning its fixtures and proving the mutation through a read-back rather than a
toast.
- teamMemberEdit: an admin edits a member's team role and per-member budget,
and both survive a reload of the Members table
- memberPermissions: a plain member is refused /key/generate for their team,
a team admin grants it on the Member Permissions tab, and the member then
creates a team key that serves a real completion
- teamGuardrailRemoval: clearing a team's only guardrail on the Settings tab
really clears it, and traffic the guardrail refused starts serving again
* test(e2e/ui): make the new team specs safe to run in parallel
Fixture ids came from Date.now(), so two repeats starting in the same
millisecond minted the same user id: one got a 409 and the loser's teardown
deleted the user the other was still signed in as. Ids now carry a random
suffix.
Also move the member-permissions setup inside the cleanup-protected block so a
half-finished setup cannot leak a team, and close both browser contexts the
test opens.
CI flagged the spec flaky twice more. Both were the same defect in different
places: the Logs drawer renders several nodes per string and the first in DOM
order is often hidden, so .first() waited 20s on an invisible element. The
entity assertions had a second problem on top, since getByText("EMAIL_ADDRESS")
substring-matched the masked prompt div, whose text contains <EMAIL_ADDRESS>,
rather than the entity chip.
Route every drawer assertion through onlyVisible, and match the entity type
and score exactly, which is what the panel renders them as: entity_type and
"Score: N.NN" each get their own span.
Verified on a live stack: 6 of 6 solo runs and the guardrails folder 5 of 5.
Mutating the analyzer to detect nothing turns the spec red on the raw address
reaching the spend log, so the assertions still carry their weight.
The Logs drawer renders the masked prompt in three nodes and the first one
in DOM order is hidden, so the previous commit's .first() traded a strict
mode violation for a locator that waits 20s on an invisible element. Local
runs against a warm stack failed on it every time, resolving the node 22
times and reporting "unexpected value hidden" each time.
Use onlyVisible, the helper this spec already uses for the playground
selectors, which filters to the visible node before taking the first. Solo
runs go 3 for 3 and the guardrails folder passes 5 of 5.
Also drop the 30s wait on the PII step added while chasing this: cold start
was never the cause, and it left the spec sitting on a dead locator longer.
The Logs drawer renders the masked prompt in three places, so matching it
without narrowing raised a strict mode violation instead of asserting
visibility. CI caught it as a flake: the spec failed its first attempt on
b0e53bfbe4 and passed on retry, which is a locator defect rather than a
timing one and would have gone red on any run that saw all three nodes.
Narrow to the first match, matching the guardrail-name assertion above it.
The leak checks below stay on toHaveCount(0), which is unaffected by
multiple matches and is what actually proves nothing raw reached the drawer.
Both readiness loops broke out on success and fell through on timeout, so a
mock Presidio server that failed to bind left the run going with nothing
serving /analyze. The guardrail then errored at request time and the failure
surfaced as an unrelated Playwright assertion in presidioUserStory.spec.ts
rather than as the missing fixture it actually was.
Fail the local runner with the port in the message, and add the matching wait
step to both CircleCI UI jobs, which had no readiness check at all.
The guardrail section of the release checklist is done by hand every cut:
create a Presidio guardrail through the wizard, send a sentence with PII from
the playground, then open Logs and check the guardrail caught it. Nothing
covered that path, so a break anywhere along it surfaced only when someone
happened to repeat the steps.
Adds a spec that walks it once and turns the eyeball checks into assertions.
The strongest of them is the leak check: it reads the request back and fails if
the stored prompt still carries the raw address or number, which is what the
manual step is really looking for.
The stack gains a Presidio stand-in that answers the two routes the guardrail
calls, detecting a fixed regex set with a Luhn check on card numbers. Real
Presidio's detection quality is Presidio's business, and pinning the UI lane to
it would mean two heavy containers with spaCy models on every CI run for a test
that is about LiteLLM's integration. The real analyzer stays covered in the
Python lane. The stand-in returns the same entities at the same spans as the
real one for the checklist's sentence, and driving the real guardrail against it
produces the same record shape, so a test written against it is written against
the product's real behavior.
Making the stand-in return the text unmasked turns the spec red on the raw
address reaching the spend log, so the leak assertion reads live data.
run_e2e.sh and both CircleCI UI jobs start the stand-in alongside the mock LLM.
Its port is overridable like the others so two checkouts can run at once.
The presidio suite proved masking happened by reading the served answer, and the
UI suite proved the wizard's "Select All & Mask" produced a row. Neither checked
the thing an operator actually looks at afterwards: the audit trail.
Adds an e2e test asserting the spend log carries the pre_call guardrail record
for a masked request: status, provider, per-entity masked counts, and the
detected-entity list the dashboard's guardrail panel renders its scores from.
It keys off the x-litellm-applied-guardrails response header rather than masked
text in the answer, because whether the model echoes the prompt back is a model
decision, not a guardrail one. Both a mutation that stops writing the record and
one that empties the entity list turn it red.
Two records land on one log, pre_call and post_call, so the assertion selects on
mode as well as name; picking by name alone could hand it the empty post_call
record depending on write order.
On the UI side, the Presidio wizard test now reads the stored guardrail back and
asserts every persisted entity carries the MASK action. A row appearing in the
table did not prove the entity selection survived the save, so a wizard that
persisted an empty pii_entities_config would have passed.
SpendLogRow gains a typed metadata field. guardrail_response is left as object
because each provider writes its own shape there (presidio a list of entities,
bedrock an assessment object, a failed run the exception string); a union narrow
enough to be useful would fail to parse the others and break every suite that
reads a spend log. The caller validates the shape it expects with a TypeAdapter.
run_e2e.sh now pins PROXY_BASE_URL to the stack's own origin. The proxy builds
its post-login redirect from that variable when it is set, so a value inherited
from a developer's .env sent the browser off the relocated stack and the suite's
login step timed out on every port but 4000.
test_no_linear_scans_in_router: #39674 renamed heuristic_v2_router_limit_violation
to auto_router_capability_violation, so the allowlist entry stopped matching and the
same admin-only scan tripped the static check. Rename the entry to follow it.
tableScrolling.spec.ts: 9ba6cab889 (LIT-4738) gave the Tags and Model Hub tables
client-side pagination at 25 rows, so the 40 seeded rows no longer render on one
page. Select 50 rows per page before counting, as the Logs case already does.
DataTable loses the comment that narrated its sticky header classes. The
table scrolling e2e spec now types every management API response it
reads, seeds rows through an immutable reduce instead of pushing into
arrays, and deletes what it seeded in each test's finally block instead
of draining a shared mutable list in afterEach.
Refs LIT-4738
Claude-Session: https://claude.ai/code/session_018yW93iDaEMhoQUXcYjus7D
Virtual Keys, Teams, Request Logs and Tags now hand DataTable a bounded
flex chain and use fillHeight, so the app shell main stays the only page
scroller, the rows scroll under a pinned header and the pagination footer
sits at the bottom of the page. DataTable keeps the sticky header inside
its own scroller in maxBodyHeight mode too, which is what let the header
scroll away with the rows on Keys, Teams and Models. Model Hub, Vector
Stores and the team detail keys tab drop their 75vh boxes and flow with
the page scroller.
Adds an e2e spec that fails on the merge base for every one of those
pages and passes at this tip.
Refs LIT-4738
Claude-Session: https://claude.ai/code/session_018yW93iDaEMhoQUXcYjus7D
PR #39604 renamed the Internal Users search box placeholder to "Search by email or ID…" but left searchUsers.spec.ts looking for the old "Search by email…" copy, so e2e_ui_testing has been red on litellm_internal_staging since it merged. Point the locator at the shipped placeholder
`POST /v1/agents/{id}/make_public` appended the agent id to
`litellm.public_agent_groups` and only then called `get_config()`, which
re-applies the DB's `litellm_settings` over the module globals and threw the
append away. The config it saved was therefore a no-op: the endpoint answered
200 with an empty `public_agent_groups`, the agent never reached
`GET /public/agent_hub`, and re-publishing never hit the "already public" 400.
Read the config first, derive the new list from the refreshed globals, save it,
then update the global
Also fixes the e2e model hub spec, which is flaky for a second reason: the
"Make Models Public" modal preselects the groups that are already public, so a
blind click on "Select All" cleared them and left "Next" disabled for the full
15s action timeout. Check the box instead of toggling it, and wait for "Next"
to be enabled before clicking
* fix(ui): paginate request logs by session groups server-side
The logs table server-paginated raw spend logs and then collapsed
multi-call sessions client-side, so a page could render 3 rows while
the footer claimed 25 and sessions straddled pages. Adds an opt-in
group_by_session param to /spend/logs/ui that pages and counts one
representative row per session (DISTINCT ON, newest non-MCP call),
keeps the bounded count contract, enriches whole-session llm/agent
composition counts, and deletes the client-side collapse pipeline.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QxT89fiygmzz2ALcjpu7Ve
* feat(ui): add a 10 rows-per-page option and default request logs to it
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QxT89fiygmzz2ALcjpu7Ve
* fix(ui): key session aggregates per api key in the logs enrichment
Grouped pagination splits a reused session id into one row per api key,
but the enrichment still aggregated by session_id alone, so both rows
showed combined spend and counts. The aggregate query now groups by
(session_id, api_key), the count folds into it (the separate group_by
query is deleted), and each row reads its own key's totals.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QxT89fiygmzz2ALcjpu7Ve
* fix(ui): treat an empty api_key as a real session group value
The spend-log schema defaults api_key to an empty string; truthiness
guards in the enrichment treated it as missing, so keyless multi-call
sessions lost their count and spend. Only None means missing now.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QxT89fiygmzz2ALcjpu7Ve
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
The default password policy from #39381 (12+ chars with upper, lower, number, and special) makes globalSetup's POST /user/update with password "test" fail with a 400, so every role login in the UI e2e suite has been failing since that merge. Seed the roles, the noteam@test.local login, and the second-admin spec with policy-compliant passwords and update the scrypt hash in seed.sql to match.
* test(e2e/ui): cover the Usage page activity tabs
Usage had one test, on Top Virtual Keys. The Key, Model and Endpoint Activity
tabs are the ones an admin reads to answer where the spend went, and none of
them was covered.
Also fixes waitForKeyInDailyActivity, which only read the first page of
/user/daily/activity. The route paginates, so once a run generates more keys
than one page holds, the helper spins for its full 120 seconds and then blames
the rollup for a key the rollup wrote correctly. The Usage page itself already
walks every page; the helper now matches it.
* test(e2e/ui): route the user-creation call through SERVER_ROOT_PATH
Review caught /user/new posting to the server root, which misses the proxy
when it is mounted under a prefix. traffic.ts already had the helper for
this; it is now exported so specs making their own management calls can use
it too.
Also drops the mutable accumulators from the daily-activity paging, which
the repo conventions ask for.
---------
Co-authored-by: Claude <noreply@anthropic.com>
The Teams tests covered creating, deleting and membership, but nothing on the
Settings tab, which is the form that posts the whole team back. That is the
shape behind the reports of a team losing its metadata or its model aliases
after an unrelated edit.
Each test creates its own team rather than editing a seeded one. The limits
test pins the models and members the edit had no business touching, and the
alias test calls the new alias with a team key instead of trusting the
readback, since an alias the router never resolves reads the same either way.
Co-authored-by: Claude <noreply@anthropic.com>
* test(e2e/ui): stop the suite failing on things that are not regressions
Five tests in the UI suite fail for reasons that have nothing to do with the
product being broken, which is enough to keep the whole leg red.
Two need a premium proxy and fail hard without one: Regenerate Key renders
disabled when the proxy is unlicensed, and /model/new refuses a team-scoped
deployment. Both now skip without LITELLM_LICENSE, the way three other tests
in this suite already do.
Three consumed a seeded fixture: Delete key, Delete a team and remove a member
each destroyed the row they needed, so the retries CI runs with were guaranteed
to fail and the suite could not run twice against one database. They now create
what they destroy.
Top Virtual Keys ranks by spend and every mock deployment costs $0, so which
keys make the list came down to how ties happened to sort. It now sends its
traffic through a priced deployment and earns its place.
* test(e2e/ui): clean up the fixtures these tests create
Review caught two leaks: the priced deployment the usage test registers and
the user the team-admin test adds both outlived the run, so repeated runs
grew shared state that later routing and rosters can see.
Also brings in the paginated daily-activity read. /user/daily/activity pages
its per-key breakdown and the helper only read the first page, so the usage
test spent its full timeout blaming the rollup for a key the rollup wrote.
* test(e2e/ui): read the licence from the proxy, not the runner
Review pointed out that checking LITELLM_LICENSE in the runner's environment
describes the wrong machine: Playwright can be pointed at a proxy configured
somewhere else, and then the skip either hides coverage or runs a premium
test against an unlicensed target.
The admin session JWT already carries the premium_user claim the dashboard
itself reads to enable these controls, so both skips now use that.
* test(e2e/ui): clean up fixtures on the failure path too
Review caught both cleanups sitting at the end of the test body, where a
failing assertion skips them, and both discarding the response so a refused
delete passed quietly. They move to afterEach and assert the delete landed.
The priced deployment matters most: left behind it keeps its custom pricing
and goes on changing what later runs route and what they cost.
* test(e2e/ui): wait for the priced deployment to become routable
The Top Virtual Keys test registered a priced deployment and sent the key's
traffic through it on the next line, so on the deployed stack it failed with
"no healthy deployments for e2e-usage-priced-...": /model/new had written
the row but the router had not picked it up yet.
Polls a ping until the deployment answers before the test sends the request
it measures, matching what the addModel spec already does for a model added
through the UI. A ping that fails writes no spend log, so the retries cannot
move the ranking this test asserts.
* test(e2e/ui): register fixtures for cleanup before the step that can fail
Review found both helpers handing their id back to the caller to record, with
a failure-prone call in between: the priced deployment was registered after
the routability wait, and the added user after /team/member_add. Either
failing left the resource in the shared database with nothing tracking it.
Both now take the teardown list and add themselves as soon as the resource
exists, so the afterEach removes it however the rest of setup goes.
* test(e2e/ui): resolve the priced deployment for teardown by name
Review pointed out the remaining gap: /model/new can persist the deployment
and still answer non-2xx, and the id was only recorded after the response was
asserted, so that path left it behind with its custom pricing.
The name is now claimed before the request and teardown looks it up in
/model/info, so a create that saved without answering 2xx is still removed and
one that never saved is simply not there.
* test(e2e/ui): claim the member id before creating the user
/user/new can persist the user and still answer non-2xx, and the id is chosen
by the test rather than returned by the proxy, so registering it before the
call is what closes the last create-failure path.
Teardown now skips an id whose user is not there, so claiming it up front
cannot fail a run where the create never landed.
* test(e2e/ui): wait out the router reload when resolving a deployment to delete
/model/info answers from the router, not from the database, and /model/new
catches and logs a failed in-request reload while still answering 2xx. A
deployment can therefore be persisted and absent from the listing until the
next reload, which is where teardown was giving up and leaking it.
Teardown now retries the lookup for a little over one
PROXY_CONFIG_RELOAD_INTERVAL_SECONDS before treating the name as never
persisted, so the only names it skips are the ones that really are not there.
* test(e2e/ui): prove a stored credential survives a config reload before using it
The Test Connect assertion has been failing intermittently on the full-suite
runs. Artifacts from litellm-e2e-ui build 165 show the UI sending
litellm_credential_name and the proxy answering with the credential unapplied:
raw_request_api_base was https://api.openai.com/v1/ rather than the mock base
the credential carries, and the call died on an upstream 404 for the model. The
same credential had resolved on three probes eight seconds earlier.
The proxy's periodic credential refresh takes a database snapshot, prunes any
in-memory credential missing from it, then re-adds the snapshot. A credential
created while that is in flight gets pruned and stays gone until the next tick,
and load_credentials_from_list fails open onto the ambient key, so nothing in
the error names the credential.
The existing pre-check asked for three consecutive probe successes, but they
completed in under a second, so they could not span a refresh. Space them so
the run covers a whole interval, which is what proves the credential survived a
refresh and is therefore stable.
* test(e2e/ui): find a database-only deployment through the search listing
/model/info answers from the router, so a deployment that reached the database
while /model/new's in-request reload failed is invisible there, and waiting on
the next reload only helps if reconciliation eventually picks it up.
/v2/model/info?search= runs a bounded query against the model table and
deliberately returns rows the router does not hold, so it resolves those
deployments to the id /model/delete needs. Falling back to it removes the wait
as well: absent from both listings now means the deployment never persisted.
* test(e2e/ui): delete the temporary member without a lookup that can skip it
Teardown asked /user/info first and treated any non-2xx as absence, so a
transient failure on the lookup silently skipped the delete and left the user
behind, which is the leak the claimed id was meant to close.
/user/delete answers 404 for an id that is not there, so it can carry both
cases on its own: 404 means the create never persisted, and anything else that
is not 2xx now fails the teardown instead of passing quietly.
* test(e2e/ui): reach the database fallback when the router listing fails
Asserting on /model/info threw before the fallback could run, so a failure on
the router-backed listing aborted teardown and left the deployment persisted,
which is the leak the fallback was added to close.
The router listing is best-effort now: an unreadable response just falls
through to the search-backed one. That listing is the authoritative answer to
whether the deployment exists, so it is the one that has to be readable, and a
name missing from it is a create that never persisted.
---------
Co-authored-by: Claude <noreply@anthropic.com>
The Logs page had coverage for opening a request and for the End User filter,
but nothing for the filters an on-call engineer actually reaches for: whose
key made the request, and which requests failed.
Each test mints its own keys and asserts on request ids it generated itself,
so a filter that quietly does nothing fails on the other key's row still being
on screen rather than passing because our own row happens to be there.
Co-authored-by: Claude <noreply@anthropic.com>
* test(e2e/ui): cover creating, testing and deleting a guardrail
The Guardrails page had no browser coverage. The RC checklist covers it by
hand against a live Presidio, which is why it has always been skipped in CI.
These drive the LiteLLM content filter instead, which runs inside the proxy,
so the whole flow is exercised without a third-party moderation service. The
create test does not stop at the table row: it sends a prompt carrying the
keyword it just banned and asserts the gateway refuses it, then sends a clean
prompt through the same guardrail and asserts it is served.
* test(e2e/ui): delete the guardrails these tests create
Review caught the fixtures being left behind. Guardrails are database rows
that show up in the table and in the playground's list, so a run that leaves
them changes what the next run sees.
Also trims the comments that restated what the helpers already say.
* test(e2e/ui): fail the run when guardrail teardown does not delete
Review caught the afterEach discarding the DELETE response, so a failed
cleanup finished quietly and left the guardrail for the next run to trip on.
* test(e2e/ui): wait for a new guardrail to reach the request path
The wizard test drove one chat completion immediately after creating the
guardrail and required a 400. A trace from the deployed stack shows the
record is stored correctly (blocked_words, action BLOCK, block_on_violation)
and the call six seconds later is still served unguarded, so the first
request can land before the proxy picks the guardrail up.
Polls the same call to the same 400 instead, which keeps the assertion and
lets the refresh land. If it never blocks, this stays red, which is what we
want it to say.
---------
Co-authored-by: Claude <noreply@anthropic.com>
* test(e2e/ui): cover the Budgets page create, edit and delete flows
The Budgets page had no browser coverage at all, so an admin creating or
editing a spend cap through the UI was only exercised by hand at RC time.
Each test reads the budget back from /budget/list, a different route from
the one the table renders, so a row that only exists in the table's cache
does not pass. The edit test pins the rate limits an unrelated spend-cap
edit has no business touching.
* test(e2e/ui): trim comments that restate the test steps
Review flagged the explanatory comments as restating ordinary setup rather
than explaining anything. Keeps the two that carry the regression rationale
for an assertion and drops the rest.
---------
Co-authored-by: Claude <noreply@anthropic.com>
Replace table tbody and data-slot locators with getByRole, restore prior
public MCP hub entries instead of clearing the whitelist on cleanup, seed
the public agent via the append-semantics per-agent route, and rework
mutable cleanup state into const-scoped try/finally blocks
UsersTable overrides DataTable's default noDataMessage with its own
EmptyState, so the row reads "No users found" rather than "No results".
Assert that, and pair it with the seeded user being absent so the check
cannot pass while the filter silently does nothing.
Two assertions were checking the wrong thing. The anchoring tests read
getByRole("listbox"), which resolves to SelectPrimitive.List; that sits at
full content height inside the popup that clips and scrolls it, so the box
overlapped the trigger even when nothing visible did. Measure the popup.
The SSO-ID search expected zero rows, but DataTable renders a "No results"
message row when a filter matches nothing, so the count is one. Assert the
empty state the user actually sees.
Adds Playwright coverage for the RC checklist items an audit marked
automatable today: Playground to Logs hand-off, public Agent/MCP hub
tabs, team models in the Playground dropdown via a team key, Add Model
with a stored credential, internal user team key creation, a second
admin account, team model deletion, and Presidio guardrail CRUD without
a live sidecar. Seeds e2e-team-keygen with the /key/generate member
permission so the internal user key flow avoids the team-list cache lag
The consolidated popup test only asserted the options never cover the
trigger, so opening above the trigger with room below it, the regression
PR #38554 fixed, would have passed. Split it back into a below-trigger
case and a cramped-viewport case. The header test accepted a single pixel
of vertical intersection; require the refresh control's centre to sit
within the tab row instead.
The migration smoke waited on `getByRole("button", { expanded: false })`
after clicking it. Playwright re-resolves that locator on every retry, so
once the clicked group flipped to expanded it matched the next collapsed
group instead, and the assertion could never pass. Count the remaining
collapsed groups and wait for that count to drop by one.
The UI e2e suite had a class of assertions that pin how the dashboard is
built rather than what it does, so an ordinary refactor turns them red
without any user-visible change.
Geometry. The auto-router template select had two tests made of pixel
arithmetic plus a data-side="bottom" check, which is Base UI's own
positioner signal. The regression they guard (#38554) is a popup opening
on top of the control that spawned it, so both cases collapse to one
invariant: the options never cover the trigger. It now runs at both
viewport heights and reads the popup as role=listbox. The models header
test compared the tabs and refresh centers within 2px, which a padding
change flips; it now asserts the two share a row.
Structure. The logs drawer test walked xpath=../../.. from a text node
and read collapsed state off chevron icon classes. SectionHeader now
renders a real disclosure button with aria-expanded, and its two copy
buttons carry distinct names instead of both being "Copy". Sidebar group
toggles expose aria-expanded too, so the migration spec can ask for a
collapsed group by state rather than by nesting depth.
Positional lookups. keyRow.locator("button").first(), row.locator("td")
.first() and getByTestId(grid).locator("div").first() all named a
position where they meant an action; they now name the control. Table
scoping moves from "table tbody" to role=row.
Timing. Nine waitForTimeout calls are gone. Every assertion that followed
them already retried to its own timeout, so the sleeps only slowed the
run down.
Both files under tests/users/ were wrapped in test.skip("...", () => {}),
which registers one skipped test and never runs the body, so the four
tests inside had never executed and were written against a UI that has
since changed (the search placeholder is "Search by email…", the ID
filters moved into a drawer, pagination is labelled "Go to previous
page"). Rewritten against the current surface: the suite goes from 104
collected tests to 107.
Left in place deliberately: the chip and dialog-footer data-slot
selectors, because the accessible names they work around live in
components/ui/, which is shadcn CLI-managed and not hand-edited.
The sidebar toggle sits absolutely positioned over the drawer's flex row. With
the sidebar expanded it lands on the sidebar header, but once collapsed it lands
on the drawer header, which is sticky at z-chrome (10). The named-z-scale
refactor moved the toggle from z-20 to z-raised (1), so from then on the header
painted over it and swallowed the click: collapse the trace list and there was
no way to bring it back.
Moves the toggle to z-floating (30) and folds the two mirrored buttons into one,
since they only ever differed by icon, label and handler.
Covered by a Playwright spec, which is the tier that can see the layering: the
button stays visible and enabled either way, so the pre-fix failure is a click
interception that jsdom cannot reproduce.
Both anchoring tests read the trigger's box before the click and the popup's box the
instant it turns visible. Base UI places the popup asynchronously and opening it can
shift the trigger, so both boxes could be sampled before the layout settled. The run
on 1eedaa3a43 missed by 4.2px (expected >= 446.015, got 441.799) on a tree with no UI
changes at all, having passed on 21092d633b, which differs only in a deleted python
test and a budget json.
Each assertion now re-reads both boxes under expect.poll. The conditions themselves
are unchanged: the popup must sit at or below the trigger's bottom edge in the first
test and must not overlap it in the second. Polling cannot mask a genuinely misplaced
popup, since one that never lands correctly still fails when the poll times out.
The shared SelectContent wrapper defaulted alignItemWithTrigger to true,
which puts Base UI's positioner into item-aligned mode and places the
popup so the active item sits on top of the trigger. In that mode the
side and sideOffset the wrapper passes two lines above are ignored, and
the popup reports data-side="none".
The overlap only becomes visible once the items are tall enough to
matter, which is why the autorouter Template picker shows it clearly:
its options are three-line cards, so the popup covers both the select
box and its own label.
No call site in the dashboard asked for item-aligned mode. 21 of them
across 15 files already passed alignItemWithTrigger={false} by hand to
undo the default, and the remaining 127 inherited the bug. Flipping the
default makes side and sideOffset live, so collision handling works and
a select with no room below now flips above the trigger rather than
covering it. The 21 hand-written opt-outs are deleted as redundant.
litellm-e2e-ui 68 failed the test this PR was meant to stabilise: "fallback
never took effect", streak 4 of a required 5, 60s timeout. Requiring a
consecutive streak of 200s after the fallback is set was wrong. It asserts that
the fallback path succeeds five times running, which is a reliability claim the
test never intended to make, and the path is inherently retry-ish because the
broken primary is attempted first on every call. One intermittent non-200
resets the streak, so a mostly-working fallback never converges.
The two directions are not symmetric:
before the write proving NO replica serves it -> needs every replica
after the write proving the fallback serves it -> one success is the claim
So the control keeps a multi-sample window and the success assertion goes back
to polling for a first sighting, on the wider 60s budget rather than the
original 30s that expired on litellm-e2e-ui 63.
Also drops the two local rebinds Greptile flagged against the repo's
no-reassignment convention: the streak counter is gone with the helper it lived
in, and the cache-round loop is now a lazy generator consumed by next().
Two e2e tests fail on timing rather than on litellm behaviour. Measured over the
last ~35 litellm-e2e / litellm-e2e-ui runs:
routerSettings.spec.ts:254 9/35 runs (7 flaky-on-retry, 2 hard failures)
test_cost_headers_e2e.py 1/29 runs it appeared in
Router fallback control
-----------------------
The e2e stack runs replicaCount 2 with proxy_config_reload_interval_seconds 7,
and every request is routed independently, so an observation of the new config
only proves the replica that served it reloaded. patchRouterSettings returns as
soon as /config/update returns, and clearBrokenFallback never waits at all, so a
retry's one-shot control assertion could be answered by a sibling replica still
holding the previous attempt's fallback. That is exactly the observed pair of
errors: "fallback never took effect" on the first attempt and "broken primary
unexpectedly succeeded on its own" on the retry.
Both assertions now poll for a consecutive streak spanning more than one reload
cycle, mirroring the PROPAGATION_TIMEOUT / settle_propagation doctrine the Python
suite already applies in e2e_config.py.
Cost-header cache read
----------------------
The prime and measure calls fired back to back with no gap, and each retry threw
away the prefix it had just paid to prime in favour of a fresh one. OpenAI
publishes a primed prefix asynchronously and routes cache lookups by
prompt_cache_key, so the test was rerolling the least likely path to a hit.
Each round now pins a prompt_cache_key and re-reads the same primed prefix up to
CACHE_REREADS times before rotating, so a fresh prefix is spent only after the
primed one has genuinely failed to become readable.
No production code changes; prompt_cache_key is added to the e2e ChatBody model,
which serializes exclude_none and so is inert for every other caller.
* test(ui): repoint the e2e locators at the post-antd form controls
Nine Playwright specs went red after the tremor and antd removals, none of
them because the product broke. The specs selected on markup those libraries
owned: tremor's TextInput stamped data-testid="base-input", the antd toast
facade rendered .ant-notification, and the team member modal's email field was
an .ant-select. Removing the libraries deleted those hooks silently.
Repoint each onto a user-facing locator that survives the next migration:
getByLabel for the key name, the MCP tool argument and the two cache pricing
fields, getByRole("combobox") for the team member email search, and the toast
container for the add-model success message.
The pricing fields needed a source change to be reachable at all. antd's
Form.Item used to assign the field name as the input id and tie the label to
it; the react-hook-form rewrite renders FieldLabel with no htmlFor and lets
FormField generate an opaque control id, so both cache inputs lost their
accessible name and could only be told apart by placeholder, which they share.
Pin the id back to the field name and point the label at it.
* fix(ui): let FormField own the pricing field label instead of hand-rendering one
The previous commit gave the cache cost inputs an accessible name by rendering
a FieldLabel with htmlFor next to the FormField. FieldLabel forwards Label's
props, and Label only accepts children, so next build failed type checking.
FormField already renders a label wired to the control id it generates, and
FormField.test.tsx covers that association, so passing label through is both
type-safe and less markup. The read-only branch keeps its plain FieldLabel,
which has no control to point at.
The log details drawer moved off Ant Design in 03d2b16bc, so its section
header renders lucide ChevronUp/ChevronDown rather than antd's UpOutlined
and DownOutlined. The collapse test still waited on .anticon-up and
.anticon-down, which no longer exist anywhere under view_logs, so it
failed on every run and burned all three attempts identically.
Point the three assertions at .lucide-chevron-up and .lucide-chevron-down,
matching how the dashboard's other suites address lucide icons.
The playground, logs drawer and AI Hub modal moved off antd, so the specs
that reached for .ant-select, .ant-drawer-content, .ant-modal and
.ant-radio-button-wrapper no longer match anything and time out.
Address the same controls through their accessible role and name instead,
which holds across the component library swap and reads closer to what a
user does.
Rebuilds the thirteen form-free components under common_components on
the in-repo shadcn layer, so they inherit the dashboard's design tokens
instead of styling themselves through Ant Design and Tremor.
SearchSelect and the three dropdowns that wrap it now forward an
optional input id, so an antd Form.Item label still resolves to its
control. The e2e steps that reached into antd's Select and Modal
internals now go through the test id, role and data-slot.
