mirror of
https://github.com/BerriAI/litellm.git
synced 2026-08-28 05:25:59 +00:00
7675ba8717
* test(ui): split the vitest suite into unit, component and integration tiers Every test file booted jsdom, including the ~1800 that assert pure functions and never render. They now run as a separate vitest project in the node environment, where the whole tier finishes in under four seconds. The tiers are vitest projects rather than a naming convention, so CI can run them as independent jobs. A .test.ts that renders React, a hook test being the usual case, is listed explicitly and stays in the jsdom tier. * test(ui): report per-test duration against a per-tier budget A timeout only catches a hung test, and it has to stay generous enough to survive a loaded runner, so it never reports the multi-second render tests that make CI fail the moment the box is busy. Budgets are separate and far tighter: 50ms unit, 1s component, 3s integration. The counts are laptop measurements, so the CI job is report-only for now. Flipping it to blocking is one line once CI has published its own numbers. * test(ui): run the tiers as separate CI jobs and stop clicking popups by text The old job ran every file in one process, so the single slowest file set the wall clock and a bigger box bought nothing. The tiers now run as separate jobs with the component tier sharded four ways. getByText and findByText match hidden nodes, so they resolve against a closed Base UI popup whose positioner still carries pointer-events: none, and the click lands or not depending on how far the open transition got. Two files failed this way, one three runs in five and one every run. Querying the option by role waits for it to be visible, and both are now stable. A lint rule keeps the pattern from coming back. * test(ui): give React Testing Library's async queries a CI-sized window findBy* and waitFor run on asyncUtilTimeout, which defaults to 1000ms and is independent of vitest's testTimeout. Raising the vitest timeout therefore did nothing for them: a query still gave up after one second while the test had 59 seconds of budget left, which is why a loaded runner produced 'Unable to find role=...' rather than a timeout. UserSearchModal is the worked example. The role query it makes resolves in 249ms on a laptop and blew past 1000ms on CI, failing the run at 1494ms. Five seconds keeps the same assertions and only widens the window a failing query waits before reporting; a passing query still resolves the moment the element appears. * test(ui): calibrate the tier budgets from real CI numbers and report by default The first CI run showed the laptop counts were badly off: component 176 local against 326 on CI, integration 87 against 128. The maxima now come from that run with headroom. continue-on-error still painted the check red, which is the opposite of the point, so the report-only decision moves into test-budgets.json as an explicit enforce flag. The job passes and prints the counts; flipping enforce to true makes it a gate. * docs(ui): drop the CLAUDE.md edits from the tier split Keeping this PR to the vitest, CI and test changes. * ci(ui): run every tier in one job instead of eight check rows Sharding bought nothing. Measured on the first run of this branch, the component tier unsharded finishes in 198s while the integration tier is floored at 384s by a single file, so integration was always the critical path and the four component shards only added rows. One job running every project comes in around 384s against the 426s the split jobs took. Eight rows named things like 'component (2)' also told a reviewer nothing, on a PR page that already carries forty checks. The job keeps the id ui-unit-tests because guard-internal-staging requires that exact context; renaming the jobs had silently stopped it reporting, which would have blocked every merge on a check that no longer existed. The workflow's display name becomes UI Tests since it runs more than unit tests. The tier split itself is untouched: it lives in the vitest projects config, so the unit tier still runs in node with no jsdom, and each tier keeps its own timeout and budget. * fix(ui): stop the type check from running the whole suite a second time test:types was 'vitest --run --typecheck.only'. Under test.projects that flag is ignored and the root-level typecheck block is not inherited, so the step collected each project's normal include and ran all 8464 runtime tests instead of type-checking. It took 542s on CI against 33s on the flat config it replaced, and the job then ran the same suite again in the next step. Typecheck now belongs to a project of its own, with an empty include so it contributes no runtime tests, and the CI job runs one vitest invocation for all four. The type tier adds about 3s to a full run and reports 'Type Errors: no errors' rather than a suite of tests. Verified it still catches things: breaking SortingState in DataTable.test-d.tsx fails with 'Type number is not assignable to type string' and exit 1, and restoring it passes. * test(ui): scope the split down to the vitest tier projects Removes everything from this branch that was not the tier split. The three lint rules brought 1381 lines of grandfathered suppressions in eslint-suppressions.json, which is 81% of the branch's added lines and debt nobody is going to pay down. The per-test duration budget does not scale as a CI step. Both are gone, along with the two query rewrites the no-click-by-text rule forced: those files pass 10/10 at this base, quiet and under load, so there was no failure behind them. The workflow is byte-identical to the base again. It already runs npm run test:types and then vitest related on pull requests, so PR cost is unchanged; the split only repoints test:types at the new project. That project is required, not optional: vitest silently ignores --typecheck.only under test.projects, so without it the type script collects the whole suite instead of the one typed file. Restores the base 60s testTimeout on the unit tier. The 5s cap was not part of the split and failed ChatShell.serverRootPath.test.ts, a 960ms test, under load. |
||
|---|---|---|
| .. | ||
| litellm-dashboard | ||
| Dockerfile | ||
| nginx.conf | ||