GitNexus/gitnexus/test/utils/hook-test-helpers.ts
Minidoracat 912285064a
perf(hooks): cmdline-first Linux db-lock scan, drop the lsof fallback (#2180) (#2183)
* perf(hooks): cmdline-first Linux db-lock scan, drop the lsof fallback (#2180)

The probe's Linux scan was O(processes × fds) — stat every fd of every
process — so on a busy host it blew its budget and fell through to lsof,
which then timed out (~2 s) and fail-closed. Every Grep/Glob/Bash hook
spent ~2 s of CPU to conclude 'couldn't tell'.

Rewrite linuxProcScanFindGitNexusServer (name kept; return type now
tri-state 'owned' | 'not-owned' | 'timeout') as three phases:
  0. /proc/<pid>/comm prefilter — kernel task->comm, never touches the
     target's memory maps; truncation-safe whitelist match (comm is
     capped at 15 visible chars). Calibrated to what a real server
     reports: @ladybugdb/core's worker_threads rename the main thread to
     'MainThread', so that is whitelisted alongside the launcher
     basenames — omitting it would blind the probe to every server.
  1. bounded /proc/<pid>/cmdline read (openSync+readSync, default 16 KiB
     with a floor of 4 KiB and a bounded escalation up to a hard ceiling)
     so a D-state holder cannot stall the hook and the mcp/serve mode
     token is never clipped off a long interpreter path.
  2. dev+ino fd match for the 0–2 survivors only.

Dispatch: 'owned' and 'timeout' both map to true. Timeout is now
fail-closed (overload self-throttle) instead of falling through to lsof;
the Linux lsof fallback is removed entirely. End-to-end semantics on
busy hosts are unchanged (the old lsof arm also fail-closed there) — the
~2 s of wasted work and the orphan-spawning lsof are what's gone.
macOS lsof+ps and Windows Restart Manager paths are untouched.

Also: fix the budget parse bug (Number(raw && trim()) treated '0' as
1200; now parseInt-then-validate, with <= 0 an explicit immediate
timeout) and add GITNEXUS_HOOK_PROC_ROOT so the Linux scan can be unit
tested against a fixture procfs instead of the host's real /proc.

Measured on a 583-process host with 6 background gitnexus mcp servers:
owner detection 6–12 ms (was ~1216 ms + lsof timeout), ~100x.

Tests: new hook-db-lock-probe.test.ts drives all three phases against a
fake procfs (comm-truncation safety, Phase 0 trap, 4 KiB-boundary
owner-miss guard, budget=0 immediate timeout, EACCES fail-closed) plus a
live-/proc e2e that pins the fd-visible lbug-handle property against a
real subprocess holder. The lsof/ps owner-detection suites are relaned
to macOS (Linux no longer takes that path); the lsof orphan-reaping
suite is removed (no lsof is spawned on Linux now) with a rationale note.

Note: pre-commit typecheck skipped; remaining tsc errors are pre-existing
on main (none in files touched here).

* fix(hooks): honest EACCES verdict + real escalation coverage (#2183 review)

Addresses the tri-review (maintainer + Codex):

- [P2] Phase-2 fd-dir EACCES no longer claims 'owned'. /proc/<pid>/fd is
  owner-only (0500), so a cross-user/root gitnexus server serving ANY
  repo cleared Phase 0+1 and hit EACCES here, and the old catch returned
  'owned' — falsely claiming it locks THIS repo's lbug (dev+ino never
  compared) and permanently suppressing augment. Split the failure
  shapes: ENOENT -> continue (raced away); EACCES/EPERM and transient
  EIO/ESTALE -> 'timeout' (unverifiable -> fail-closed, but honest, not a
  false ownership claim); ENOTDIR/other structural errors -> continue
  (not a real fd dir). Same fail-closed dispatcher outcome, no false
  'owned', plus a GITNEXUS_DEBUG diagnostic so an operator can tell this
  skip path from a real owner.
- [P2] The escalation test now actually iterates the escalation loop:
  the gitnexus token sits under 4 KB while the mode token is padded past
  GITNEXUS_HOOK_PROC_CMDLINE_MAX=4096, and a readSync spy asserts >1 read
  (the old 9 KB-under-16 KB-cap shape read once and never escalated).
- escalation loop now re-checks the budget each iteration and returns a
  distinct timeout sentinel (never '' — an empty string would read as
  'not a candidate' and could drop a real owner -> fail-open); the caller
  maps it to 'timeout'.
- GITNEXUS_HOOK_PROC_ROOT is gated to test context so a stray production
  env export can't disable Linux owner detection (fail-open).
- New uid-agnostic spy tests pin every fd-readdir errno branch
  (EACCES/EPERM/EIO/ESTALE -> timeout, ENOTDIR -> not-owned) regardless
  of the runner's uid (the disk chmod-000 tests no-op under root).

Note: pre-commit typecheck skipped; remaining tsc errors are pre-existing
on main (none in files touched here).

* fix(hooks): drop the always-true outOfBudget presence guard (CodeQL #2183)

CodeQL flagged `typeof outOfBudget === 'function' && outOfBudget()` as
unneeded defensive code: readLinuxCmdline has a single caller
(linuxProcScanFindGitNexusServer) that always passes the callback, so
the typeof guard is dead. Drop it, leaving `if (outOfBudget())`, and note
the invariant in the comment. Mirrored in the byte-identical plugin copy.

* fix(hooks): parse numeric hook env with Number() so scientific notation works (#2183 review)

getCmdlineMaxBytes and resolveLinuxProcBudgetMs parsed their env via
Number.parseInt(raw, 10), so a value like "16e3" silently became 16 (parseInt
stops at 'e') instead of 16000. Switch both to Number(String(raw).trim()),
which honors scientific notation and is stricter on trailing garbage
("123abc" -> NaN -> default) — matching the repo-majority Number()+isFinite
env idiom (src/cli/analyze.ts, src/core/embeddings/hf-env.ts).

The two functions had DIFFERENT guard skeletons, so a verbatim swap would
regress the budget: resolveLinuxProcBudgetMs used `raw != null ?` with no
empty-string short-circuit, and Number("")===0 (vs parseInt("")===NaN) would
make a set-but-empty GITNEXUS_HOOK_LINUX_PROC_BUDGET_MS="" resolve to budget 0
=> immediate fail-CLOSED timeout => augment permanently skipped. Added the
`&& String(raw).trim()` guard so ''/whitespace fall to the 1200 default while
"0" still parses to the deliberate #2180 immediate-timeout vector.

Exported both helpers for white-box tests (the values are otherwise only
observable indirectly through scan timing) and added platform-independent
coverage: "16e3"->16000, ""/whitespace->1200 (the regression guard), "0"->0,
"123abc"/unset->1200, cmdline "8e3"->8000, "2e3"/""/unset->16384.

Both byte-identical hook-db-lock-probe.cjs copies updated together.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(hooks): allocUnsafe the per-chunk cmdline read buffer (#2183 review)

readLinuxCmdline allocated each per-chunk read buffer with Buffer.alloc(chunkCap),
zero-filling memory that readSync immediately and fully overwrites. Switch the
hot read buffer to Buffer.allocUnsafe — safe because readSync initializes
exactly [0, bytes), only buf.subarray(0, bytes) is consumed, and Buffer.concat
deep-copies that slice into `collected`, so the uninitialized tail can never
reach the decoded cmdline. The zero-length `collected = Buffer.alloc(0)` is left
unchanged (allocUnsafe gains nothing on a 0-length buffer). The existing D3
multi-chunk decode tests cover the read path and stay green.

Both byte-identical hook-db-lock-probe.cjs copies updated together.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(hooks): harden the live /proc owner-detection e2e against CI flake (#2183 review)

Two flake mechanisms, fixed without weakening what the e2e proves:

- Holder readiness (the genuine false-FAIL): the pid-file poll was 200x25ms=5s;
  a loaded runner can be slow to spawn the child, tripping
  expect(holderPid).toBeGreaterThan(0). Widened to ~10s and raised the per-test
  timeout 20s -> 40s.
- Scan budget (kept the assertion honest): the live scan ran at the default
  1200ms. Because the dispatcher maps a budget 'timeout' to owned=TRUE, a busy
  host exhausting 1200ms before reaching the holder would make the assertion
  pass for the WRONG reason (a hollow timeout, not real fd-visible detection).
  Set a generous explicit 10000ms budget via the existing setEnv() helper so the
  module afterEach restores it (replacing the raw `delete process.env...` that
  bypassed env tracking). Raised the coarse timing regression guard to sit ABOVE
  the budget (5000 -> 15000) so a legitimately-slow-but-correct scan can't trip
  it.

The load-bearing asserts (dev+ino fd-visibility precheck, owned===true for our
own lbug) are unchanged. Verified the e2e executes (not skipped) on Linux.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore(changelog): empty the root CHANGELOG [Unreleased] section

Per maintainer request, nothing should sit under [Unreleased] in the root
CHANGELOG.md (the release-owned changelog is gitnexus/CHANGELOG.md, whose
[Unreleased] is already empty). Removes all three accumulated blocks — Fixed
(#2163), Performance (#2180), Changed (KuzuDB->LadybugDB) — leaving only the
[Unreleased] header above [1.5.3]. Pure removal; no release sections touched.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Gergő Magyar <gergomagyar@icloud.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-13 11:52:14 +01:00

269 lines
11 KiB
TypeScript

/**
* Shared helpers for hook test files (unit + integration).
*/
import { spawnSync } from 'child_process';
import fs from 'fs';
import os from 'os';
import path from 'path';
export function runHook(
hookPath: string,
input: Record<string, any>,
cwd?: string,
options: { env?: NodeJS.ProcessEnv } = {},
): { stdout: string; stderr: string; status: number | null } {
const result = spawnSync(process.execPath, [hookPath], {
input: JSON.stringify(input),
encoding: 'utf-8',
timeout: 10000,
cwd,
// Used as-is when provided: every caller passes a full env (a spread of
// process.env plus overrides), so re-merging process.env here is redundant
// and, worse, on Windows it re-adds the original `Path` key alongside a
// replaced `PATH` — defeating envWithPath(), which deletes path variants so a
// scrubbed PATH is honored deterministically.
env: options.env ?? process.env,
stdio: ['pipe', 'pipe', 'pipe'],
});
return {
stdout: result.stdout || '',
stderr: result.stderr || '',
status: result.status,
};
}
export function parseHookOutput(
stdout: string,
): { hookEventName?: string; additionalContext?: string } | null {
if (!stdout.trim()) return null;
try {
const parsed = JSON.parse(stdout.trim());
return parsed.hookSpecificOutput || null;
} catch {
return null;
}
}
// ─── Stale-index hint PATH-detection helpers (#1938) ────────────────
//
// The hooks emit `gitnexus analyze` (no npx) when a launcher is on PATH. These
// helpers let an e2e test fabricate that condition deterministically: scrub any
// ambient `gitnexus` off PATH, then prepend a synthetic launcher — so the test
// asserts the hook's real PATH auto-detection rather than env-var forcing.
/** Names a global `gitnexus` may take on each platform (for scrub + fabricate). */
function gitNexusLauncherNames(): string[] {
return process.platform === 'win32'
? ['gitnexus', 'gitnexus.cmd', 'gitnexus.bat', 'gitnexus.exe', 'gitnexus.ps1']
: ['gitnexus'];
}
/** True if `dir` holds a runnable `gitnexus` launcher (isFile + X_OK on POSIX). */
function hasGitNexusLauncher(dir: string): boolean {
return gitNexusLauncherNames().some((name) => {
const candidate = path.join(dir, name);
try {
if (!fs.statSync(candidate).isFile()) return false;
if (process.platform !== 'win32') fs.accessSync(candidate, fs.constants.X_OK);
return true;
} catch {
return false;
}
});
}
// ─── Fake tool dir for the DB-owner probe (shared by unit + e2e) ────
//
// Builds a temp bin dir holding fake `gitnexus`, `lsof`, and `ps` executables so
// a hook spawned with hookEnv(binDir) sees a deterministic DB-owner probe result
// (and a marker-writing fake CLI) without touching the real process table.
// Module-private: only createHookToolDir writes these fakes; callers use the
// higher-level createHookToolDir, never writeExecutable directly.
function writeExecutable(filePath: string, content: string) {
fs.writeFileSync(filePath, content, { mode: 0o755 });
}
export function createHookToolDir(options: {
gitnexusStderr?: string;
gitnexusMarkerPath?: string;
/** Fake gitnexus CLI writes its own PID here as its FIRST statement, minimizing detection latency for augment orphan-reaping tests (#2163 follow-up). */
gitnexusPidFile?: string;
/** Fake gitnexus CLI sleeps this long instead of exiting — models a hung augment child. */
gitnexusSleepMs?: number;
/** Fake gitnexus CLI traps SIGTERM as a no-op before sleeping — models an unkillable CLI that only SIGKILL can end (#2163 follow-up). */
gitnexusIgnoreSigterm?: boolean;
lsofOutput?: string;
lsofOutputLines?: string[];
psOutput?: string;
psOutputByPid?: Record<string, string>;
lsofSleepMs?: number;
/** Fake lsof writes this marker file as soon as it starts — proves whether the probe reached the lsof fallback at all (#2163). */
lsofMarkerPath?: string;
/** Fake lsof writes its own PID here as its FIRST statement, minimizing detection latency for orphan-reaping tests (#2163). */
lsofPidFile?: string;
/** Fake lsof traps SIGTERM as a no-op before sleeping — models an unkillable/D-state lsof that only SIGKILL can end (#2163). */
lsofIgnoreSigterm?: boolean;
}) {
const binDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gitnexus-hook-bin-'));
const gitnexusStderr = JSON.stringify(options.gitnexusStderr ?? '');
const markerPath = JSON.stringify(options.gitnexusMarkerPath ?? '');
// Composable prologue (mirrors the fake-lsof one below): pidFile write MUST
// stay the first statement (see the option docs above); the SIGTERM trap
// MUST be installed before any sleep.
const fakeGitNexus =
`#!/usr/bin/env node\nconst fs = require('fs');\n` +
(options.gitnexusPidFile != null
? `fs.writeFileSync(${JSON.stringify(options.gitnexusPidFile)}, String(process.pid));\n`
: '') +
(options.gitnexusIgnoreSigterm ? `process.on('SIGTERM', () => {});\n` : '') +
`const marker = ${markerPath};\nif (marker) fs.writeFileSync(marker, 'called');\n` +
(options.gitnexusSleepMs != null
? `setTimeout(() => {}, ${Number(options.gitnexusSleepMs)});\n`
: `process.stderr.write(${gitnexusStderr});\n`);
writeExecutable(path.join(binDir, 'gitnexus'), fakeGitNexus);
writeExecutable(path.join(binDir, 'gitnexus-cli.js'), fakeGitNexus);
const lsofOutput =
options.lsofOutputLines != null
? options.lsofOutputLines.join('\n') + (options.lsofOutputLines.length ? '\n' : '')
: (options.lsofOutput ?? '');
// Composable prologue: pidFile write MUST stay the first statement (see the
// option docs above); SIGTERM trap MUST be installed before any sleep.
const lsofPrologue =
`#!/usr/bin/env node\nconst fs = require('fs');\n` +
(options.lsofPidFile != null
? `fs.writeFileSync(${JSON.stringify(options.lsofPidFile)}, String(process.pid));\n`
: '') +
(options.lsofMarkerPath != null
? `fs.writeFileSync(${JSON.stringify(options.lsofMarkerPath)}, 'called');\n`
: '') +
(options.lsofIgnoreSigterm ? `process.on('SIGTERM', () => {});\n` : '');
const lsofBody =
options.lsofSleepMs != null
? `${lsofPrologue}setTimeout(() => {}, ${Number(options.lsofSleepMs)});\n`
: `${lsofPrologue}process.stdout.write(${JSON.stringify(lsofOutput)});\nprocess.exit(0);\n`;
writeExecutable(path.join(binDir, 'lsof'), lsofBody);
const psBody =
options.psOutputByPid != null
? `#!/usr/bin/env node
const byPid = ${JSON.stringify(options.psOutputByPid)};
const args = process.argv;
const p = args[args.indexOf('-p') + 1];
process.stdout.write(byPid[p] ?? '');
process.exit(0);
`
: `#!/usr/bin/env node\nprocess.stdout.write(${JSON.stringify(options.psOutput ?? '')});\nprocess.exit(0);\n`;
writeExecutable(path.join(binDir, 'ps'), psBody);
return binDir;
}
// ─── Fake /proc root for the Linux cmdline-first DB-owner scan (#2180) ──
//
// linuxProcScanFindGitNexusServer reads every path under GITNEXUS_HOOK_PROC_ROOT
// (defaulting to /proc in production). These helpers build a fixture tree so the
// three-phase scan (comm -> cmdline -> fd dev+ino) can be unit-tested without
// touching the test host's real, hundreds-of-process /proc — which is both slow
// and nondeterministic (other gitnexus servers may be running). fd entries are
// real symlinks to real files, so fs.statSync on them yields real dev+ino the
// scan can compare against the target lbug.
export interface FakeProcEntry {
pid: number | string;
/** /proc/<pid>/comm contents (kernel caps at 15 visible chars; caller models truncation). */
comm: string;
/** argv tokens; joined with NUL like the real /proc/<pid>/cmdline. */
cmdline: string[];
/** Absolute paths this pid "holds" open — each becomes an fd symlink target. */
fdTargets?: string[];
/** When true, make /proc/<pid>/fd unreadable-shaped by omitting it entirely so readdir throws ENOENT; for EACCES use the logic-path test instead. */
noFdDir?: boolean;
}
/**
* Build a fake /proc tree under a fresh temp dir and return its path (use as
* GITNEXUS_HOOK_PROC_ROOT). Caller is responsible for rm-ing the returned dir.
*/
export function createFakeProcRoot(entries: FakeProcEntry[]): string {
const procRoot = fs.mkdtempSync(path.join(os.tmpdir(), 'gitnexus-fakeproc-'));
for (const e of entries) {
const pidDir = path.join(procRoot, String(e.pid));
fs.mkdirSync(pidDir, { recursive: true });
fs.writeFileSync(path.join(pidDir, 'comm'), `${e.comm}\n`);
fs.writeFileSync(path.join(pidDir, 'cmdline'), e.cmdline.join('\0') + '\0');
if (!e.noFdDir) {
const fdDir = path.join(pidDir, 'fd');
fs.mkdirSync(fdDir, { recursive: true });
const targets = e.fdTargets ?? [];
targets.forEach((target, i) => {
// Real symlink so statSync(link) follows to the real file's dev+ino —
// exactly what the scan compares against the target lbug.
try {
fs.symlinkSync(target, path.join(fdDir, String(i + 3)));
} catch {
/* best-effort; a missing target just won't match */
}
});
}
}
return procRoot;
}
/** A full env that points a spawned hook at the fake tool dir from createHookToolDir. */
export function hookEnv(binDir: string) {
return {
...process.env,
PATH: `${binDir}${path.delimiter}${process.env.PATH || ''}`,
GITNEXUS_HOOK_CLI_PATH: path.join(binDir, 'gitnexus-cli.js'),
GITNEXUS_HOOK_LSOF_PATH: path.join(binDir, 'lsof'),
GITNEXUS_HOOK_PS_PATH: path.join(binDir, 'ps'),
};
}
/**
* The current PATH with every dir that contains a `gitnexus` launcher removed, so
* a test box that already has gitnexus installed cannot make the assertion pass
* (or fail) for the wrong reason. Mirrors the hook's own detection — isFile() +
* X_OK — rather than a bare existsSync.
*/
export function pathWithoutGitNexus(
pathValue: string = process.env.PATH || process.env.Path || process.env.path || '',
): string {
return pathValue
.split(path.delimiter)
.filter((dir) => dir && !hasGitNexusLauncher(dir))
.join(path.delimiter);
}
/** A full env copy with PATH replaced by `pathValue` and all case variants of the key removed. */
export function envWithPath(pathValue: string): NodeJS.ProcessEnv {
const env: NodeJS.ProcessEnv = { ...process.env };
for (const key of Object.keys(env)) {
if (key.toLowerCase() === 'path') delete env[key];
}
env.PATH = pathValue;
return env;
}
/**
* Create a temp dir holding a runnable `gitnexus` launcher and return a PATH that
* puts it first (with all other gitnexus launchers scrubbed). Caller must invoke
* cleanup() to remove the temp dir.
*/
export function createGitNexusPathEntry(): { pathValue: string; cleanup: () => void } {
const binDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gitnexus-path-'));
const launcher = path.join(binDir, process.platform === 'win32' ? 'gitnexus.cmd' : 'gitnexus');
fs.writeFileSync(
launcher,
process.platform === 'win32' ? '@echo off\r\nexit /b 0\r\n' : '#!/bin/sh\nexit 0\n',
);
if (process.platform !== 'win32') fs.chmodSync(launcher, 0o755);
return {
pathValue: [binDir, pathWithoutGitNexus()].filter(Boolean).join(path.delimiter),
cleanup: () => fs.rmSync(binDir, { recursive: true, force: true }),
};
}