* test(ui): split the vitest suite into unit, component and integration tiers
Every test file booted jsdom, including the ~1800 that assert pure functions
and never render. They now run as a separate vitest project in the node
environment, where the whole tier finishes in under four seconds.
The tiers are vitest projects rather than a naming convention, so CI can run
them as independent jobs. A .test.ts that renders React, a hook test being the
usual case, is listed explicitly and stays in the jsdom tier.
* test(ui): report per-test duration against a per-tier budget
A timeout only catches a hung test, and it has to stay generous enough to
survive a loaded runner, so it never reports the multi-second render tests that
make CI fail the moment the box is busy. Budgets are separate and far tighter:
50ms unit, 1s component, 3s integration.
The counts are laptop measurements, so the CI job is report-only for now.
Flipping it to blocking is one line once CI has published its own numbers.
* test(ui): run the tiers as separate CI jobs and stop clicking popups by text
The old job ran every file in one process, so the single slowest file set the
wall clock and a bigger box bought nothing. The tiers now run as separate jobs
with the component tier sharded four ways.
getByText and findByText match hidden nodes, so they resolve against a closed
Base UI popup whose positioner still carries pointer-events: none, and the
click lands or not depending on how far the open transition got. Two files
failed this way, one three runs in five and one every run. Querying the option
by role waits for it to be visible, and both are now stable. A lint rule keeps
the pattern from coming back.
* test(ui): give React Testing Library's async queries a CI-sized window
findBy* and waitFor run on asyncUtilTimeout, which defaults to 1000ms and is
independent of vitest's testTimeout. Raising the vitest timeout therefore did
nothing for them: a query still gave up after one second while the test had 59
seconds of budget left, which is why a loaded runner produced 'Unable to find
role=...' rather than a timeout.
UserSearchModal is the worked example. The role query it makes resolves in
249ms on a laptop and blew past 1000ms on CI, failing the run at 1494ms. Five
seconds keeps the same assertions and only widens the window a failing query
waits before reporting; a passing query still resolves the moment the element
appears.
* test(ui): calibrate the tier budgets from real CI numbers and report by default
The first CI run showed the laptop counts were badly off: component 176 local
against 326 on CI, integration 87 against 128. The maxima now come from that
run with headroom.
continue-on-error still painted the check red, which is the opposite of the
point, so the report-only decision moves into test-budgets.json as an explicit
enforce flag. The job passes and prints the counts; flipping enforce to true
makes it a gate.
* docs(ui): drop the CLAUDE.md edits from the tier split
Keeping this PR to the vitest, CI and test changes.
* ci(ui): run every tier in one job instead of eight check rows
Sharding bought nothing. Measured on the first run of this branch, the
component tier unsharded finishes in 198s while the integration tier is floored
at 384s by a single file, so integration was always the critical path and the
four component shards only added rows. One job running every project comes in
around 384s against the 426s the split jobs took.
Eight rows named things like 'component (2)' also told a reviewer nothing, on a
PR page that already carries forty checks.
The job keeps the id ui-unit-tests because guard-internal-staging requires that
exact context; renaming the jobs had silently stopped it reporting, which would
have blocked every merge on a check that no longer existed. The workflow's
display name becomes UI Tests since it runs more than unit tests.
The tier split itself is untouched: it lives in the vitest projects config, so
the unit tier still runs in node with no jsdom, and each tier keeps its own
timeout and budget.
* fix(ui): stop the type check from running the whole suite a second time
test:types was 'vitest --run --typecheck.only'. Under test.projects that flag
is ignored and the root-level typecheck block is not inherited, so the step
collected each project's normal include and ran all 8464 runtime tests instead
of type-checking. It took 542s on CI against 33s on the flat config it
replaced, and the job then ran the same suite again in the next step.
Typecheck now belongs to a project of its own, with an empty include so it
contributes no runtime tests, and the CI job runs one vitest invocation for all
four. The type tier adds about 3s to a full run and reports 'Type Errors: no
errors' rather than a suite of tests.
Verified it still catches things: breaking SortingState in DataTable.test-d.tsx
fails with 'Type number is not assignable to type string' and exit 1, and
restoring it passes.
* test(ui): scope the split down to the vitest tier projects
Removes everything from this branch that was not the tier split.
The three lint rules brought 1381 lines of grandfathered suppressions in
eslint-suppressions.json, which is 81% of the branch's added lines and
debt nobody is going to pay down. The per-test duration budget does not
scale as a CI step. Both are gone, along with the two query rewrites the
no-click-by-text rule forced: those files pass 10/10 at this base, quiet
and under load, so there was no failure behind them.
The workflow is byte-identical to the base again. It already runs
npm run test:types and then vitest related on pull requests, so PR cost
is unchanged; the split only repoints test:types at the new project.
That project is required, not optional: vitest silently ignores
--typecheck.only under test.projects, so without it the type script
collects the whole suite instead of the one typed file.
Restores the base 60s testTimeout on the unit tier. The 5s cap was not
part of the split and failed ChatShell.serverRootPath.test.ts, a 960ms
test, under load.
* test(ui): raise vitest test and hook timeouts for CI headroom
The UI unit suite runs about 3x slower on the CI runner than locally, which
put the slowest cases right on the 30s per-test limit. TeamInfo's pass
through routes case takes ~8s locally and has been failing on staging at the
timeout across consecutive runs even though it passes reliably when run
directly.
Raise testTimeout to 60s and set hookTimeout to 30s so the current slow cases
have headroom. This is a stopgap while the suite gets split into proper tiers,
not a fix for the underlying per-test cost.
* test(ui): query agent form panels with findByRole like the rest of the file
The panel helper was the only synchronous query in add_agent_form's
integration test; every other lookup already retries via findBy. On the CI
runner the second case has been failing with "Unable to find an accessible
element with the role button and name /Cost Configuration/" against a modal
whose body had not rendered.
Make the helper retry like its siblings and await it at each call site.
DataTable accepted any mix of its 40-odd props and rejected the incoherent
combinations at runtime, from a validator that threw during the first render.
A caller only found out it had wired server sorting without a `sorting` prop
when the page blew up in front of them.
Split the public prop type into mode-keyed unions instead, so the compiler
rejects those combinations at the call site. `validateDataTableConfig` and
`DataTableConfigError` go away; the component body reads an unchanged flat
`DataTableResolvedProps`, which every union member is assignable to, so there
is no narrowing inside it.
All 44 existing call sites typecheck against the new union unchanged, which
`next build` covers. That build only typechecks the app module graph, so the
prop type itself needed a gate of its own: `npm run test:types` runs vitest's
typecheck mode over `*.test-d.tsx`, and the unit workflow now runs it. The
four guards deleted from `DataTable.test.tsx` come back there as compile-time
assertions, and loosening the union back to the flat shape fails all five.
* chore(build): move the Admin UI toolchain to Node 24
Node 18 and Node 20 both reached end of life (2025-04-30 and 2026-04-30), and
the release images along with every CI lane were still building on them. Node 24
is the current LTS through 2028-04-30, so this moves the four UI build images,
the CircleCI lanes, and the four GitHub Actions workflows onto it
Node 24 also ships npm 11.17, which is the first line that implements the
min-release-age setting this repo already carries in its .npmrc files. On npm 10
the key is parsed and discarded, so the release-age gate has had no effect
regardless of its value. Tightening the dashboard's engines range and turning on
engine-strict makes an unsupported npm fail loudly rather than skip the gate
quietly, and a new step in the UI build workflow probes an impossible cooldown
so an inert setting cannot pass unnoticed again
Node 24's bundled undici tightened its brand check on RequestInit.signal, which
rejects the AbortSignal jsdom installs and broke the two cases in
src/lib/http/api.test.ts that rebase a request onto a runtime base url. Under
jsdom the Request global comes from Node while AbortSignal comes from jsdom;
tests/jsdomFetchEnv.ts delegates to the jsdom environment and then restores
Node's native AbortController and AbortSignal so both come from one realm.
Upgrading jsdom does not address this, as jsdom still does not own Request
The workflows now read ui/litellm-dashboard/.nvmrc instead of repeating a
literal, so the Node version has a single source of truth, and ui/Dockerfile is
pinned by digest to match the other three build images. The lockfile changes are
npm 11 normalising the engines range and dropping optional peer entries it no
longer records
* fix(build): point every Admin UI build script at .nvmrc
The enterprise Docker path was left on Node 18. docker/build_admin_ui.sh runs
only when enterprise/enterprise_ui/enterprise_colors.json is present, which it
never is in the OSS tree, so neither CI nor a default image build reaches it;
it pinned nvm to v18.17.0 and then built the dashboard, which now requires Node
24, so a customized enterprise image would have failed EBADENGINE
All three UI build scripts now resolve the version from
ui/litellm-dashboard/.nvmrc rather than carrying their own pin, so the Node
version has a single home across Docker, CI, and local builds. build_ui.sh was
on v20 and build_ui_custom_path.sh on v18.17.0
Also drops the dependency-cooldown probe from the UI build workflow. The
engines floor plus engine-strict already fails an unsupported npm loudly at
install time, so the probe was redundant, and treating any nonzero exit from a
live registry call as proof of enforcement made it unsound besides
Relocates ui/litellm-dashboard/e2e_tests to tests/e2e/ui so all end to end
suites live under tests/e2e. The suite stays in TypeScript and becomes a
self-contained npm package with its own package.json, lockfile and tsconfig
instead of leaning on the dashboard's toolchain; the dashboard drops its
@playwright/test dependency, e2e scripts and knip/vitest/tsconfig carve-outs.
CI paths follow the move: both CircleCI jobs (main e2e and the
SERVER_ROOT_PATH migration smoke) and the test_server_root_path workflow now
install and run Playwright from tests/e2e/ui, with the node cache keyed on
both lockfiles. classify_changes.sh treats tests/e2e/ui as client so spec
edits keep skipping backend jobs. The suite's mock LLM fixture is excluded
from the e2e basedpyright zero-error gate in pyrightconfig.json since it
belongs to the TS suite, not the typed Python harness.
* fix(ui): bundle provider logos as static imports and unify fallback in Logo component
providerLogoMap values are now content-hashed bundle URLs emitted by
static imports instead of /ui/assets/logos/ path strings, so any
deployment that serves the app JS also serves the logos: dev server,
proxy /ui mount, server_root_path sub-paths, and the split-chart nginx
image where the old route 404d in production. A missing file is now a
build error instead of a silent runtime 404.
resolveLogoSrc passes /_next/ URLs through untouched so bundled values
never get double-prefixed with the server root path. The new Logo
molecule owns resolution and the letter-avatar fallback and warns with
the failing URL on load error; ProviderLogo delegates to it. The three
bare img sites in the agents wizard render through Logo, fixing their
broken-image bug.
Dashscope now uses qwen.png, RunwayML the on-disk runway.png, and the
GradientAI entry is removed (no plausible asset exists). soniox.svg and
ai21.svg drop a single mismatched intrinsic dimension attribute that
Turbopack's import-time image parser rejects. Dead logoSrc lookup in
AddModelForm deleted. Vitest resolves image imports to Next's
StaticImageData shape via a config plugin so tests exercise the same
/_next/ URLs as production.
* fix(ui): retry logo load when src changes after an error
Track which src errored instead of a boolean so a Logo instance whose
source changes in place (agents modal title) attempts the new URL
rather than staying on the letter-avatar until remount.
The ui_unit_tests CircleCI job logged ~45k lines for a single run, most of
it React act() warnings, antd deprecation notices and component stack traces
emitted as console output by passing tests, which buried real failures.
Set silent: "passed-only" (Vitest 3.2+) gated on process.env.CI so console
output from passing tests is suppressed while a failing test still prints its
logs and full stack trace. Also drop two stray console.log calls in
UsagePageView that dumped the whole currentUser object on every render in
production, not just tests.
Verified by running the suite the way CI does
(CI=true npm run test -- --run --pool forks --poolOptions.forks.maxForks=6):
45,075 lines before, 981 after, all 4075 tests still passing. A throwaway
failing test confirms its console.log and assertion diff remain visible.
The ui_unit_tests job runs vitest with maxForks=8 on an 8-vCPU xlarge
container, leaving no headroom for the main vitest process that services
worker RPCs. Under full CPU saturation the coordinator misses the
onTaskUpdate ack, vitest raises "Timeout calling onTaskUpdate" as an
unhandled error, and the job exits 1 even though every test passes.
Lower maxForks to 6 so the coordinator, jsdom, and OS keep two cores, and
raise teardownTimeout to 60s for extra slack on heavy runs.
Raise vitest testTimeout from 10s to 30s and drop per-test timeout overrides
across UI unit tests. Group CreateUserButton and TeamInfo tests under nested
describe blocks to make the most flaky suites easier to scan.
