GitNexus/gitnexus/test/helpers/cli-entry.ts
Gergő Magyar 8402963198
fix(ci): shard platform-sensitive matrix + spawn built CLI to fix Windows cross-platform timeout (#2394)
* fix(ci): shard platform-sensitive matrix + spawn built CLI to fix Windows cross-platform timeout

The `windows-latest (platform-sensitive)` job was hitting its 15-min internal
vitest watchdog in run-cross-platform.ts. It's cumulative slowness, not a hang:
the fixed 72-file suite is dominated by ~50 CLI/worker process spawns, and
Windows is ~5x slower than macOS at process startup (macOS ran the same set in
~3min of tests). Two complementary changes bring it back under the watchdog with
headroom, without touching any test assertion:

- Shard the platform-sensitive matrix (windows/macos × shard [1,2]) and forward
  `--shard=i/2` through run-cross-platform.ts to vitest, which partitions the
  fixed file list deterministically (sha1, equal file-count) — halving each
  runner. macOS/Ubuntu were already under budget.
- New test/helpers/cli-entry.ts (`CLI_SPAWN_PREFIX`): spawn the built
  `dist/cli/index.js` when `GITNEXUS_E2E_CLI=dist` (set on the cross-platform job,
  which already builds) instead of `node --import tsx src/cli/index.ts`, which
  re-transpiles the whole CLI on every spawn. Defaults to tsx-on-source so local
  runs always reflect current source; `GITNEXUS_E2E_CLI=dist` on an unbuilt tree
  throws an actionable "run npm run build" error. dist is opt-in only — never
  inferred from a generic `CI` env — so an ambient `CI=1` can't silently run a
  stale build. Converted 8 spawn-based e2e suites; added test/unit/cli-entry.test.ts.

The Ubuntu coverage job leaves `GITNEXUS_E2E_CLI` unset, so the tsx-on-source path
stays exercised in CI too (both entry points covered).

Measured on Linux: cli-limit-e2e 121.5s→91s, cli-e2e 289s→217s (~25%); larger on
Windows where the transpile is a bigger share of each spawn.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(ci): derive platform-sensitive shard count from one source (#2394)

The shard total was hardcoded in three coupled, unenforced places (matrix
length, job-name suffix, --shard denominator); editing one without the others
silently dropped a shard's tests with green CI. Add a checkout-free shard-plan
job whose single TOTAL generates both the shard index list (consumed via
fromJSON) and the /N denominator (job name + --shard arg), so they cannot
drift. Asserts TOTAL>=1 to rule out an empty-matrix silent skip. No behavior
change — still 2 shards per OS.

Addresses PR #2394 tri-review finding F2.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(ci): 3 shards for real Windows headroom + honest sharding comments (#2394)

vitest shards by file COUNT, not runtime, so the heaviest spawn suites cluster
into one shard: live CI showed Windows shard 1/2 at 12m12s (~81% of the 15-min
watchdog) vs shard 2/2 at 3m0s. The old comments claimed "comfortable/generous
headroom", which the count-based split doesn't deliver at 2 shards. Bump TOTAL
to 3 (one line, single source) so even the busiest Windows shard clears the
watchdog, and reword the comments to describe count-based (not time-based)
sharding.

Addresses PR #2394 tri-review finding F1.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(ci): extract testable parseShardArg from run-cross-platform (#2394)

The --shard parse/forward glue had no unit test. Extract it into a pure
scripts/shard-arg.ts (mirroring the computeSpawnPrefix extraction precedent) so
the branch logic is lockable without the script's top-level execFileSync, and
add test/unit/shard-arg.test.ts (absent -> undefined, valid token -> passed
through, found amid other args). Behavior unchanged; U4 adds the malformed
fail-loud on top.

Addresses PR #2394 tri-review finding F3.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(ci): fail loud on a malformed --shard arg (#2394)

A shard-shaped-but-malformed arg (--shard=1, --shard, --shard=abc) was silently
ignored, dropping the shard flag so both legs ran the full unsharded ~50-spawn
suite — re-arming the Windows watchdog timeout with no signal. parseShardArg now
throws an actionable error on any --shard/--shard=… arg that fails the strict
regex (unrelated flags like --shardx= pass through), and the call site in
run-cross-platform.ts catches it into console.error + exit 1, kept outside the
execFileSync try so the message isn't swallowed by that catch's watchdog-only
branch.

Addresses PR #2394 tri-review finding F4.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(test): fail loud on an unknown GITNEXUS_E2E_CLI value (#2394)

computeSpawnPrefix silently degraded any unknown GITNEXUS_E2E_CLI value to
tsx-on-source, so a typo (e.g. `dsit`) would make CI believe it tests the dist
entry point while actually running src. Throw on any value other than
'dist'/'src'/unset (the safe tsx default is preserved for unset/''/'src', so it
still never selects dist without an explicit opt-in). Flip the unknown-mode unit
test to assert the throw and add the missing {mode:undefined, distExists:true}
case. Only ci-tests.yml sets the var (=dist), so no existing suite is affected.

Addresses PR #2394 tri-review findings minor-a/b.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(ci): run cli-entry.test.ts on the cross-platform matrix (#2394)

cli-entry.test.ts resolves CLI_SPAWN_PREFIX from a real path, and its last
assertion (cli[/\\]index) has a Windows backslash branch that only Ubuntu
exercised. Register it in PLATFORM_LOGIC so it runs on the Windows/macOS matrix
too. (shard-arg.test.ts stays out — pure string logic, OS-independent.) List
grows 73 -> 74; the generated shard matrix keeps coverage complete.

Addresses PR #2394 tri-review finding minor-c.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(test): share tsxLoaderUrl(), dedup the last tsx-loader boilerplate (#2394)

bridge-cache-reopen.test.ts carried its own copy of the tsx-loader-resolution
boilerplate (createRequire -> resolve('tsx/package.json') -> pathToFileURL) —
the one site the PR's CLI_SPAWN_PREFIX migration didn't cover (it spawns a seed
script, not the CLI). Export the existing tsxLoaderUrl() from cli-entry.ts and
reuse it here; the resolved loader URL is byte-identical.

Addresses PR #2394 tri-review finding minor-d.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(test): make skipUnlessFtsAvailable install FTS on miss so shards are self-sufficient (#2394)

Sharding the platform-sensitive suite into 3 exposed a latent test-isolation
bug: load-only FTS primitives (test/integration/lbug-core-adapter.test.ts) only
passed because a sibling installer test happened to co-locate in the same shard
and install FTS into the shared ~/.lbdb first. At 3 shards, lbug-core-adapter
landed in a shard with no installer sibling, so its load-only loadFTSExtension()
failed deterministically on macOS+Windows shard 2/3 under GITNEXUS_REQUIRE_FTS=1.

Make the gate self-sufficient: on a load-only miss under REQUIRE_FTS, install
FTS with `auto` (LOAD-first, then one bounded network INSTALL) before treating
it as a hard failure — mirroring withTestIndexedDB. A pre-installed extension
still costs no network (auto is LOAD-first); offline/local runs (no env var)
still skip gracefully. Verified: with a fresh HOME (no pre-installed FTS) +
REQUIRE_FTS=1, lbug-core-adapter now passes 15/15 (previously threw).

Addresses the 3-shard CI failure surfaced while validating PR #2394's F1 fix.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* ci: warm-cache the LadybugDB FTS extension across platform shards (#2394)

Follow-up to the FTS self-install fix: cache ~/.lbdb/extension per OS + lockfile
so a warm run skips the network install entirely and the parallel shards share
one download across runs. Pure reliability/speed — on a cache miss the tests
still self-install FTS on demand (test/helpers/fts-availability.ts), so this is
never a correctness dependency, just a way to cut the network-install surface
that made the sharded FTS tests flaky. Keyed by lockfile hash (a LadybugDB
version bump re-installs); per-OS since the extension is a native binary.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(ci): pass shard via env to clear zizmor template-injection (#2394)

Interpolating ${{ matrix.shard }} (now sourced from the shard-plan job output)
directly into the run: shell tripped zizmor's template-injection audit
(code-scanning alert #824, ci-tests.yml:147). Move the value into a SHARD env
var — assigned via ${{ }} but referenced as "$SHARD" in the shell, which is not
an injection sink — and set shell: bash so the expansion is uniform across the
windows + macOS matrix (the default run shell is pwsh on Windows, where $SHARD
would be empty and trip the new malformed-shard fail-loud). Verified locally
with zizmor: the :147 template-injection finding is gone.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* ci: shard the ubuntu coverage job and merge blobs before the threshold gate (#2394)

The coverage job ran the full suite unsharded (~16 min). Shard it like the
cross-platform matrix, then merge the per-shard coverage before enforcing the
threshold gate:

- shard-plan now also single-sources the coverage shard count (cov_total /
  cov_shards), so the coverage matrix + /N denominator can't drift.
- The `tests` job becomes a coverage shard matrix: each shard runs
  `vitest run --shard --coverage --reporter=blob` with thresholds forced to 0
  (a single shard's partial coverage can never meet the gate) and uploads its
  blob. FTS self-installs per shard, so sharding the full suite is safe.
- New `coverage-merge` job (needs: tests) reduces the blobs with
  `vitest --mergeReports`, enforcing the REAL config thresholds on the combined
  ('new') coverage — this is the gate. It also emits the merged test-results.json
  and runs the unsharded web + docker suites, so the `test-reports` artifact
  keeps the exact shape ci-report.yml consumes for its base-branch ('baseline')
  vs new coverage delta.

The shard arg goes through a SHARD env var + shell: bash (no template-injection).
Validated locally: shard blobs write and merge into a coverage-summary.json +
merged test-results.json; the merge enforces thresholds on the union. CI Gate
still aggregates the coverage-merge result via the reusable-workflow call.

Note: the coverage check names change (ubuntu / coverage 1/3 … + merge) — update
any pinned branch-protection required checks.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(ci): include hidden files when uploading the coverage blob (#2394)

The coverage shards write their blob to gitnexus/.vitest-reports/ (a dotdir).
actions/upload-artifact excludes hidden files by default, so the coverage-blob-*
artifacts uploaded empty — the merge job then downloaded 0 artifacts and
vitest --mergeReports failed with ENOENT scandir '.vitest-reports'. Set
include-hidden-files: true on the blob upload so the blobs actually ship.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(ci): group shard-plan GITHUB_OUTPUT writes to satisfy shellcheck SC2129 (#2394)

Adding the coverage shard outputs (cov_shards/cov_total) made the shard-plan gen
step write four individual `>> "$GITHUB_OUTPUT"` redirects, which shellcheck
(run by the actionlint check) flags as SC2129. Group the echoes into a single
`{ …; } >> "$GITHUB_OUTPUT"` block.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* perf(test): cost-balanced shard sequencer to cut CPU contention (#2394)

vitest's default --shard hashes file paths and splits by file COUNT, which
clustered the spawn-heavy suites onto one runner (Windows platform shard 1 ran
~4x the others). Add a custom sequence.sequencer that overrides only shard()
and balances by estimated WORK instead:

- specWeight() weights the fileParallelism:false spawn-heavy suites (cli-e2e,
  lbug-db — already isolated to run sequentially) far above the parallel default
  files, plus file size as a cheap finer signal. Deterministic per checkout.
- assignShards() does greedy longest-processing-time bin-packing (heaviest file
  into the currently-lightest shard). The partition stays complete and disjoint
  — verified: on the 74-file cross-platform set the three shards weigh
  7611/7610/8064 (the sequential-heavy files spread ~7/7/8) with zero overlap and
  no file dropped, vs the hash split's count-only balance.

sort() is left to the base sequencer so project groupOrder / duration-cache
ordering is untouched. Pure logic split into shard-balance.ts with a unit test
locking the disjoint+complete, balance, and determinism properties.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(ci): install + cache FTS up front on the coverage (and cross-platform) shards (#2394)

coverage 3/3 failed on extension-binary-real.test.ts: it uses the file-path FTS
gate (requireFtsResourceOrSkip), which resolves ~/.lbdb/extension at MODULE LOAD
and cannot self-install the way the load-path gate (skipUnlessFtsAvailable, U8)
does. The coverage job had no FTS cache and relied on an installer test running
first in the shard — the balancing sequencer reshuffled the shards and dropped
extension-binary-real into a shard with no installer, so FTS was absent.

Remove the ordering dependency: add scripts/ensure-fts.ts (init a throwaway lbug
db, loadFTSExtension with policy:auto → LOAD-first, INSTALL on miss) and run it
up front on every coverage AND cross-platform shard, after restoring the per-OS
FTS cache. The coverage job now shares that same cache key (it previously had
none — this is the "share the cached FTS with coverage" the failure pointed at).
Cold cache installs once; warm cache is a no-network load. Verified locally:
ensure-fts installs FTS into a fresh HOME and is a no-op when already present.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-08 09:09:11 +01:00

96 lines
4 KiB
TypeScript
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

/**
* How e2e tests launch the gitnexus CLI as a subprocess.
*
* Default — `node --import tsx src/cli/index.ts`: always reflects current source,
* so any local run (built or not) exercises your edits.
*
* CI opts into the built CLI by setting `GITNEXUS_E2E_CLI=dist` AFTER its build
* step (see `.github/workflows/ci-tests.yml`) — `node dist/cli/index.js`, which
* skips the per-spawn tsx transpile of the whole CLI source graph. The
* platform-sensitive suite spawns the CLI ~50 times and Windows is ~5× slower at
* process startup, so that transpile dominated the job and tripped its watchdog.
*
* We deliberately do NOT infer dist from a generic `CI` env var: CI-presence does
* not prove `dist/` is fresh, and an ambient `CI=1` (agent sandboxes, other tools)
* could otherwise silently spawn a STALE build. dist is used only when explicitly
* requested, so the entry point in effect is always knowable from the environment.
*
* Bonus: the CLI's `ensureHeap()` re-exec "just works" from dist; under tsx it drops
* the `--import` loader, which is why analyze-calling tests pre-set
* `--max-old-space-size` in their `cliEnv()`.
*/
import fs from 'node:fs';
import path from 'node:path';
import { createRequire } from 'node:module';
import { fileURLToPath, pathToFileURL } from 'node:url';
const require = createRequire(import.meta.url);
// test/helpers/cli-entry.ts → repo root is two levels up.
const repoRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..', '..');
const distEntry = path.join(repoRoot, 'dist', 'cli', 'index.js');
const srcEntry = path.join(repoRoot, 'src', 'cli', 'index.ts');
/**
* Pure resolver, exported for unit testing. `mode` is the raw `GITNEXUS_E2E_CLI`
* value: `'dist'` selects the built CLI (and requires it to exist); unset / `''` /
* `'src'` select tsx-on-source; any other value throws (a typo shouldn't silently
* degrade to tsx). Kept side-effect-free so the branch logic can be locked
* without env/filesystem gymnastics.
*/
export function computeSpawnPrefix(opts: {
mode: string | undefined;
distEntry: string;
srcEntry: string;
distExists: boolean;
tsxLoaderUrl: string;
}): string[] {
if (opts.mode === 'dist') {
if (!opts.distExists) {
throw new Error(
`GITNEXUS_E2E_CLI=dist but ${opts.distEntry} is missing — run \`npm run build\` first.`,
);
}
return [opts.distEntry];
}
// Fail loud on a typo instead of silently degrading to tsx: an unknown value
// (e.g. `dsit`) would otherwise make CI believe it tests dist while running
// src. Unset / '' / 'src' remain the safe tsx-on-source default.
if (opts.mode !== undefined && opts.mode !== '' && opts.mode !== 'src') {
throw new Error(
`Unknown GITNEXUS_E2E_CLI value '${opts.mode}' — use 'dist', 'src', or leave unset.`,
);
}
return ['--import', opts.tsxLoaderUrl, opts.srcEntry];
}
export function tsxLoaderUrl(): string {
// Absolute file:// URL to the tsx loader — a bare `tsx` specifier won't resolve
// when the CLI is spawned with a cwd outside the project tree. The subpath
// `tsx/dist/loader.mjs` isn't in tsx's `exports`, so resolve the package root
// then join. Only computed on the tsx path (never when dist is selected).
return pathToFileURL(
path.join(path.dirname(require.resolve('tsx/package.json')), 'dist', 'loader.mjs'),
).href;
}
function resolvePrefix(): string[] {
const mode = process.env.GITNEXUS_E2E_CLI;
return computeSpawnPrefix({
mode,
distEntry,
srcEntry,
distExists: mode === 'dist' && fs.existsSync(distEntry),
// Resolve the tsx loader lazily so the dist path pays nothing for it.
tsxLoaderUrl: mode === 'dist' ? '' : tsxLoaderUrl(),
});
}
/**
* `node` argv prefix that launches the gitnexus CLI (built dist when
* `GITNEXUS_E2E_CLI=dist`, else tsx-on-source). Spread it before the CLI's own
* arguments:
*
* spawnSync(process.execPath, [...CLI_SPAWN_PREFIX, 'analyze', repo], opts)
*/
export const CLI_SPAWN_PREFIX: readonly string[] = resolvePrefix();