The worker SIGTERM handler unconditionally sent {type:'error','Analysis cancelled'}
and didn't coordinate with the message handler that sends complete, so a cancel near
the finish line could report a cancelled job complete, or a late SIGTERM could flip
an already-complete job to failed.
Add a single terminal-outcome claim (createTerminalClaim) shared by the message
handler and the SIGTERM handler: whoever claims it first reports its terminal
message; the other skips its terminal send. Single-threaded JS makes the
check-and-set atomic. The cleanup + process.exit still run regardless.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JBJomjoTdBV2eveDVq4JMm
The forked analyze worker reported {type:'complete'} straight after runFullAnalysis,
so a server/web analyze of a half-finalized repo (meta.json written but the global
registry entry missing — a prior collision-aborted run, or a wiped registry) was
reported successful while the repo stayed unregistered/invisible to list_repos. The
CLI already guards this with assertAnalysisFinalized; the worker did not.
Extract the run -> finalize -> report contract into a side-effect-free
analyze-worker-core seam (the entry module's top-level process.on handlers make it
untestable directly) and call assertAnalysisFinalized before sending complete — a
failure is reported as {type:'error'} instead of a false success. The seam is
dependency-injected and unit-tested with fakes; the entry module wires the real deps
and keeps owning process.exit.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JBJomjoTdBV2eveDVq4JMm
Root cause of the ubuntu-coverage-only failure (3/3 CI runs, passing locally):
analyzeCommand calls ensureHeap() (analyze.ts:715), which RE-EXECS the process —
spawning `node <heap-flags> <argv>` with vitest's argv — unless NODE_OPTIONS
already carries --max-old-space-size (analyze.ts:498). That re-exec killed the
forked vitest worker ("Worker exited unexpectedly" → the assertion never ran).
It only reproduced on the memory-constrained CI runner because locally a high V8
heap-size-limit also short-circuits ensureHeap (analyze.ts:501).
Reproduced locally with NODE_OPTIONS="--no-warnings" (no heap cap) → same failure;
fixed by pre-setting --max-old-space-size in beforeAll (restored in afterAll), the
same workaround cli-e2e uses. Verified: passes under the repro condition, normally,
and with --coverage.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JBJomjoTdBV2eveDVq4JMm
The test failed deterministically only on the ubuntu coverage lane (2/2 runs)
while passing locally and in isolation, incl. with --coverage. Cause: the
vi.resetModules() + per-test `await import('analyze.js')` re-instrumented the
ENTIRE analyze module graph on every test; under --coverage on the
memory-constrained CI runner that OOM/crashed the forked worker ("Worker exited
unexpectedly" → the assertion never ran).
Import analyzeCommand ONCE and drive the mocks per-test via mockReturnValue
(resetModules wasn't needed — the hoisted mocks are controllable per-test). Keeps
the whole-file process.exit spy + afterAll fatal-handler strip from the prior pass.
Behavior under test is unchanged; passes in isolation and with --coverage.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JBJomjoTdBV2eveDVq4JMm
cancelJob() / the 30-min timeout (analyze-job.ts) send SIGTERM to the forked
analyze worker, but its SIGTERM handler still did a full `await closeLbug()`
(native conn/db teardown) — even though normal completion now skips it via
skipNativeCloseOnExit. A cancelled or timed-out --pdg server analyze could
therefore still hit the LadybugDB ClientContext destructor double-free, or block
behind the in-flight COPY's connection lock before exiting.
Mirror the CLI SIGINT path: a best-effort CHECKPOINT with
closeLbug({ skipNativeClose: true }) bounded by a 2s Promise.race timeout, then
process.exit(0) (which reclaims the handles). A CHECKPOINT failure is reported to
the parent over IPC rather than swallowed; the exit is in .finally so it always
fires.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JBJomjoTdBV2eveDVq4JMm
The prior commit healed every up-to-date-but-unregistered repo by falling through
to register it — which broke the #1169 guard: a plain `analyze` of an up-to-date
repo whose registry entry is missing MUST fail loudly ("Analysis did not finalize")
rather than silently register a possibly half-finalized index.
Distinguish the two causes of "unregistered":
- collision-rejected + user re-runs with --allow-duplicate-name → explicit intent
to register, so fall through to the pipeline and register it (#829).
- plain analyze, registry missing/wiped → keep the #1169 fail-loud behavior.
So self-heal is gated on options.allowDuplicateName; isRepoRegistered is only read
on that opt-in branch, so the common fast path keeps its single-stat cost. Both
cli-e2e guards (#1169 fail-loud, #829 heal) now pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JBJomjoTdBV2eveDVq4JMm
A prior 'analyze --name X' that hit a registry name collision writes meta.json
(meta-save runs before registerRepo) but fails before registering — leaving the
index up-to-date but UNREGISTERED. A later 'analyze --name X --allow-duplicate-name'
then matched the up-to-date gate and early-returned WITHOUT registering, so the
repo stayed invisible to list_repos/MCP and the CLI's assertAnalysisFinalized
rejected it. --allow-duplicate-name could never heal it.
This was latent on main, masked by the very close-hang this PR fixes: the lingering
process pushed the cli-e2e #829 step-3 analyze past its 60s spawn timeout
(status===null → the test's vacuous early-return). With the hang gone the analyze
exits promptly, exit 1 surfaces, and the bug becomes deterministic on all platforms.
Fix: the up-to-date fast path now short-circuits only when the repo is actually
registered (new isRepoRegistered helper, sharing assertAnalysisFinalized's exact
canonical/case-folded membership check). An indexed-but-unregistered repo falls
through to the pipeline, which registers it honoring allowDuplicateName. Already
registered repos keep the fast path unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JBJomjoTdBV2eveDVq4JMm
analyzeCommand calls installFatalHandlers(), which registers global
unhandledRejection/uncaughtException handlers that call the REAL process.exit(1).
Across this file's vi.resetModules() reimports they accumulate on `process`, and
under CI timing a stray async rejection fired one while no process.exit spy was
active — killing the forked vitest worker ("Worker exited unexpectedly"), which
only surfaced once the full test lanes finished (they were pending at review time).
Keep process.exit spied for the whole file (beforeAll/afterAll) so a fatal handler
can never really exit mid-run, strip the handlers installFatalHandlers added in
afterAll (preserving vitest's own, snapshotted up front) before restoring the real
process.exit, and reset process.exitCode so the worker exits clean. Passes in
isolation and grouped; behavior under test is unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JBJomjoTdBV2eveDVq4JMm
The parse worker swallowed or only-locally-logged several caught errors, so they
never reached the pool: the per-language-group catch was an empty catch {} that
silently dropped the whole group on any throw (not just an unavailable grammar),
the per-file parse/query-execution catches only logger.warn'd (worker-thread
local), and the C++ template-constraint catch swallowed silently.
Route all work-path catches through a new reportWarning() helper that posts
{ type: 'warning', message } to the pool (which logs it on the main thread AND
resets the worker idle timer, so a worker grinding through failing files isn't
falsely idle-evicted), with a logger.warn fallback for the non-worker path. The
existing inline warning sites (query-compilation, the extractParsedFile callback,
CFG build) are migrated to the same helper.
The 4 optional-grammar module-load guards (Swift/Dart/Kotlin/C) stay silent: they
run before the 'ready' handshake and their absence is already surfaced via
result.skippedLanguages + the isLanguageAvailable gate. Fatal/group-aborting
errors continue to flow through the message handler's { type: 'error', errorStack }.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JBJomjoTdBV2eveDVq4JMm
The worker's send() swallowed IPC failures (and a prior pass logged them to
stderr). Per review, all worker errors must be reported back to the parent over
the existing IPC channel (send({ type: 'error' })) and nothing silently dropped.
- send() no longer catches: a dead channel (ERR_IPC_CHANNEL_CLOSED) throws
instead of being swallowed.
- Every handler (uncaughtException, unhandledRejection, SIGTERM, the analysis
message handler) reports its error via send() in try and schedules process.exit
in finally, so a throw from send() can no longer skip the exit and wedge the
worker — the P3 'schedule the exit so it always fires' fix, without a swallow.
- SIGTERM cleanup failures are now reported to the parent instead of an empty
catch {}.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JBJomjoTdBV2eveDVq4JMm
The SIGINT handler calls closeLbug({skipNativeClose:true}), whose flushWAL
CHECKPOINT queues behind the connection lock held by an in-flight COPY — so a
single Ctrl-C during a long --pdg COPY appeared hung until the COPY released.
Race the cleanup against a 2s timeout before process.exit(130); the WAL replays
on the next analyze. The double-Ctrl-C escape hatch is unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JBJomjoTdBV2eveDVq4JMm
The worker's send() used `process.send?.(msg)` — the `?.` guards an undefined
channel but not a throw from an already-closed one (ERR_IPC_CHANNEL_CLOSED). A
throw in the catch-branch send() would escape the message handler and skip the
scheduled `setTimeout(process.exit(0))`, stranding the worker (with skip-close
leaving native handles open, #2264). Wrap process.send in try/catch so the exit
always fires; a vanished child is a failure to the parent regardless.
Not unit-tested: send() is module-private and importing the worker registers
process signal handlers; the change is a defensive try/catch around one call.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JBJomjoTdBV2eveDVq4JMm
The relationship COPY passed the module-level `conn` to copyCsvWithRetry while the
node COPY uses the captured `writeConn`. Use `writeConn` for both — one captured
reference for the whole bulk load, removing the latent identity dependency. Same
object during analyze (`conn` is only reassigned at open/close under the session
lock), so the queryAndDrain `targetConn === conn` lock gate still engages.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JBJomjoTdBV2eveDVq4JMm
lbug-checkpoint.test.ts grepped the adapter SOURCE (comments included) for
conn.close()/db.close()/.query('CHECKPOINT') literals, coupling a passing test to
comment wording — a prior commit had to reword a comment just to keep it green.
Strip comments from the read source before the structural assertions so they
reflect code only; the invariant (exactly two CHECKPOINT sites; close calls only
in safeClose) is preserved and no longer breaks on a comment edit.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JBJomjoTdBV2eveDVq4JMm
loadCachedEmbeddings issued raw conn.query reads on the singleton connection
outside withConnLock — safe today only because it runs before the WAL-checkpoint
driver starts, an ordering invariant not enforced by code. Wrap the whole read in
withConnLock so a future reorder can't race a CHECKPOINT on the connection. Leaf
read; no nested wrapped helpers.
Adds a routing assertion to lbug-conn-serialization.test.ts.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JBJomjoTdBV2eveDVq4JMm
A real conn.close() on the error path after large --pdg writes can itself hit the
LadybugDB ClientContext destructor double-free → SIGABRT, degrading an actionable
exit-1 error into a raw native abort. Switch the error-path close to
skipNativeClose (mirroring the success path). Safe now that the CLI catch
force-exits when isLbugReady() (the prior commit): handles left open are reclaimed
by that guaranteed process.exit, so the process terminates without the abort and
without hanging. flushWAL keeps the partial index durable.
Depends on the prior commit (CLI force-exit guard).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JBJomjoTdBV2eveDVq4JMm
The full-analysis success path skip-closes LadybugDB (handles left open, reclaimed
by process.exit). If a post-finalize step (assertAnalysisFinalized) then throws,
the outer catch soft-returns (process.exitCode = 1) — and with native handles
open the event loop never drains, so the process HANGS instead of exiting 1.
Guard once at the analyzeCommand wrapper, after the try/finally: if isLbugReady()
(handles still open) the analyze actually ran and we must force the exit. The
success path never reaches here (analyzeCommandImpl process.exit(0)s itself);
early-validation errors and unit tests that mock runFullAnalysis never open the DB
(isLbugReady() false), so the soft return is preserved.
Adds analyze-finalize-failure-exits.test.ts (force-exits when handles open; does
NOT when they aren't). The analyze-*.test.ts that mock lbug-adapter now also mock
isLbugReady (vitest throws on accessing an undefined export of a mocked module).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JBJomjoTdBV2eveDVq4JMm
The forked server analyze worker runs runFullAnalysis then force-exits
(process.exit(0)). With a real native close inside runFullAnalysis, the LadybugDB
ClientContext destructor can double-free after --pdg writes and abort the worker
BEFORE it sends 'complete', failing the parent's analyze. Pass
skipNativeCloseOnExit: true so the worker checkpoints for durability and lets its
process.exit reclaim the handles — same about-to-exit contract as the CLI.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JBJomjoTdBV2eveDVq4JMm
The CLI error handler soft-returns (process.exitCode = 1) instead of forcing
exit, relying on the released native handles to let Node terminate. The earlier
commit made runFullAnalysis's error-path closeLbug skip the native close, leaving
live LadybugDB handles that keep the event loop alive forever — a post-init
analyze failure would hang. Only the SUCCESS path (which guarantees a following
process.exit) skips the native close; the error path now always closes for real.
A late-error close could still abort in the destructor, but that terminates the
process — it does not hang.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JBJomjoTdBV2eveDVq4JMm
The skipNativeClose comment in closeLbug contained the literal `conn.close()`/
`db.close()`, which the structural guard test (lbug-checkpoint.test.ts:52-53 —
"closeLbug must not inline conn.close()/db.close()") greps for and fails on.
Reword the comment to describe the native close without the literal tokens; the
code already delegates close exclusively to safeClose, so the guard's intent holds.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JBJomjoTdBV2eveDVq4JMm
THE actual fix for the `analyze --pdg` crash. gdb shows the abort is a double-free
inside LadybugDB's own destructor during conn.close():
"double free or corruption (out)" -> abort
lbug::main::ClientContext::~ClientContext()
lbug::main::Connection::~Connection()
NodeConnection::Close(...) <- conn.close() from safeClose
It reproduces with the WAL driver OFF and with serial load, so it is NOT the
checkpoint/COPY concurrency the rest of this branch serialized — it's a native
LadybugDB engine bug (@ladybugdb/core 0.17.1, latest stable) triggered by the
larger --pdg write set, firing during teardown AFTER a fully-written, checkpointed
index.
Fix: closeLbug({ skipNativeClose }) CHECKPOINTs for durability (flushWAL) then
skips conn.close()/db.close(), leaving the handles referenced so no GC finalizer
re-runs the destructor. The CLI analyze command (success, error, and SIGINT paths
all process.exit) opts in via skipNativeCloseOnExit; long-lived callers (MCP
server, tests) keep the real close. Mirrors the pool adapter's fire-and-forget
native close and the ONNX native-cleanup philosophy.
Validated end-to-end: `analyze --pdg --force` now exits 0 with a 193,876-node
index; re-opening it (no --force) reads clean and reports up-to-date, proving the
CHECKPOINT-only persistence is durable without db.close().
Workaround for an upstream LadybugDB bug (ClientContext destructor double-free).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JBJomjoTdBV2eveDVq4JMm
lbug-checkpoint.test.ts asserted exactly two CHECKPOINT sites by grepping the
literal `conn.query('CHECKPOINT')`. The connection-serialization refactor changed
flushWAL/tryFlushWAL to capture `const c = conn` and call `c.query('CHECKPOINT')`
inside withConnLock, so the literal grep found 0 and the test failed (expected 2).
Make the regex receiver-agnostic (`.query('CHECKPOINT')`) — preserves the guard's
intent (exactly two authorized CHECKPOINT sites; a third is a regression) while
tolerating the captured-receiver form.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JBJomjoTdBV2eveDVq4JMm
Match the repo's single-underscore test-seam convention (_initLockPathForTest).
Pure rename of the @internal export and its sole importer.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JBJomjoTdBV2eveDVq4JMm
A future withConnLock-wrapped helper calling another wrapped helper would await
its own holder's tail and hang silently. Add an AsyncLocalStorage-based re-entry
guard: withConnLock throws a clear error when invoked from within a holding fn's
async context. A boolean flag can't do this — a legitimately-queued top-level
caller also runs while the lock is held; only AsyncLocalStorage distinguishes a
true nested call from normal contention.
Tests: re-entry throws (not deadlocks); sequential and concurrent top-level calls
do NOT false-fire; the lock releases after a re-entry throw.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JBJomjoTdBV2eveDVq4JMm
deleteAllInterprocTaintPaths, deleteAllCallSummaries, and
deleteAllCommunitiesAndProcesses awaited conn.query(...DELETE...) but dropped the
returned QueryResult (only the count result was closed), leaking a native result
handle and violating the helpers' own "query + drain inside the lock" contract.
Close each delete result via closeQueryResults, matching the count handling.
Adds a seeded drain test (closeQueryResults fires for the DELETE result).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JBJomjoTdBV2eveDVq4JMm
The per-table count read used a raw targetConn.query while the sibling DETACH
DELETE already routed through the locked queryAndDrain — an asymmetry that left
the count racing the WAL-checkpoint driver during incremental --pdg writeback.
Gate the count through withConnLock when targetConn === conn (the singleton),
matching queryAndDrain; per-query/temp connections stay lock-free.
Test asserts the count loop takes the lock once per filePath-bearing node table.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JBJomjoTdBV2eveDVq4JMm
queryImporters issued a raw conn.query on the singleton connection inside the
importer-BFS loop of incremental --pdg writeback, while the WAL-checkpoint driver
could fire a concurrent CHECKPOINT — the same double-free class. Wrap the read
(query + getAll + drain) in withConnLock.
Extends lbug-conn-serialization.test.ts with a routing assertion.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JBJomjoTdBV2eveDVq4JMm
The count + DETACH DELETE ran raw conn.query on the singleton connection during
incremental --pdg writeback while the WAL-checkpoint driver was live — the same
concurrent CHECKPOINT-vs-write double-free this branch fixes elsewhere. Wrap the
body in withConnLock, mirroring the already-wrapped deleteAllInterprocTaintPaths.
Adds test/integration/lbug-conn-serialization.test.ts (call-through withConnLock
spy) asserting the helper now acquires the lock, wired into the lbug-db vitest
project (and excluded from the default project so it doesn't run twice).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JBJomjoTdBV2eveDVq4JMm
LadybugDB is single-writer and its Connection is NOT safe for concurrent
query execution. The WAL-checkpoint driver (5s setInterval) issued
`conn.query('CHECKPOINT')` on the same module-singleton `conn` the analyze
pipeline used for COPY. With --pdg the extra BasicBlock / REACHING_DEF / CDG /
POST_DOMINATE / TAINTED / CALL_SUMMARY / TAINT_PATH table COPYs outlast the
5s tick, so a checkpoint executed concurrently with an in-flight COPY on one
connection -> two libuv workers mutate shared native state -> heap corruption
("double free or corruption (out)" / SIGABRT, detected at the final
"Saving metadata..." free).
Fix: add conn-lock.ts (`withConnLock`, a promise-chain mutex) and run every
singleton-`conn` helper's full query + result-drain inside it: queryAndDrain
(when targetConn === conn), executePrepared, executeWithReusedStatement,
flushWAL, tryFlushWAL, getLbugStats, deleteAllInterprocTaintPaths,
deleteAllCallSummaries. Add an `if (inflight) return` reentrancy guard to the
driver tick so overdue ticks don't stack checkpoints. streamQuery is
intentionally NOT wrapped (read path, re-entrant per-row callback).
Reproduced the crash with concurrent queries on one raw Connection (serial =
stable); verified the fix drives the same overlap through the locked adapter
without crashing.
Tests: conn-lock serialization (no overlap / FIFO / throw-releases) and
driver reentrancy guard.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JBJomjoTdBV2eveDVq4JMm
Bump gitnexus, plugin.json, and marketplace.json to 1.6.8 and add the
CHANGELOG section. Headline: opt-in PDG-backed impact analysis plus the
full PDG/taint substrate (CFG → reaching-defs → intra/inter-procedural
taint → control dependence) across the language matrix, multi-branch
indexing, private GitHub PAT + Azure DevOps support, and MCP trace/HTTP.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(impact-pdg): run mutation oracle's analyze child from built dist, not tsx-over-src
The nightly Impact PDG Mutation Report workflow failed at the first fixture with
ERR_MODULE_NOT_FOUND for src/cli/lazy-action.js. The harness shelled the real CLI
out as `node --import tsx src/cli/index.ts analyze …`; on the CI runner's Node
22.22.3, native TypeScript type-stripping is enabled by default and handles the
.ts entry instead of tsx, and native stripping does NOT remap the `./lazy-action.js`
import specifier to lazy-action.ts the way tsx does — so CLI startup crashes
before analyze even runs.
The workflow already builds dist/ (build: 'true'). Prefer the shipped
dist/cli/index.js (plain compiled JS — no tsx, no strip-types, and the parse
workers it spawns also resolve from dist/) for the analyze child, falling back to
tsx's own CLI over src only for build-free local runs. Production-faithful and
version-agnostic across the engines range (node >=22.0).
Verified on a real Node 22.22.3: the dist child starts cleanly with no
lazy-action resolution error; the full `--mutation --only=inter-dispatcher-thin`
run scores realized recall 1.0 and gate-mutation-recall passes. Workers are
independently confirmed green on 22.22.3 in CI (run 27874383902).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(impact-pdg): declare the mutation oracle's @babel/* deps
`bench/impact-pdg/mutation-oracle.mjs` imports @babel/parser, @babel/traverse,
@babel/generator and @babel/types to instrument + value-diff the fixture AST,
but none were declared in package.json. @babel/parser and @babel/types happen to
be hoisted into gitnexus/node_modules transitively, but @babel/traverse and
@babel/generator are only present at the monorepo root — so a fresh `npm ci` in
gitnexus/ (CI) can't resolve them and the oracle dies at module load with
`Cannot find package '@babel/traverse'` right after analyze succeeds.
Declare all four as devDependencies (they're already lazily imported only on the
--mutation path, so they stay out of the unit-test module graph). Verified the
oracle resolves them from gitnexus/node_modules and scores recall 1.0.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(impact-pdg): gate only recall-gated mutation checks (honor recallGated)
The recall gate filtered checks by `typeof c.recall === 'number'`, which includes
the UPSTREAM fixtures. The mutation oracle is a FORWARD value-diff: it mutates the
criterion line and observes which downstream lines' values change, so its
behavioral AIS can never intersect a reverse (upstream) PDG slice — recall is 0
by construction. measure.mjs already marks these `recallGated: false` (alongside
id-discrimination corroboration cases) and excludes them from its own internal
gate; the standalone gate just didn't honor that flag, so `intra-control-loop`
(direction: upstream, recall 0) tripped the floor even though the oracle ran the
full suite cleanly (mean recall 0.923).
Filter on `c.recallGated === true` so the floor applies only to the downstream
cases the forward oracle can fairly validate. Verified locally: an
upstream+downstream report now scores 1 of 2 and the gate passes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(impact-pdg): fail the mutation gate when it has no recall signal + fix README drift
Tri-review hardening of this PR's own changes:
- gate-mutation-recall.mjs: the floor check passed vacuously when `scored` was
empty (`min === null` short-circuits `min !== null && min < floor`). Narrowing
the filter to `recallGated === true` made an empty `scored` set reachable in
more inputs (a degenerate corpus, or a harvest that silently emptied every
behavioral AIS). Now fail loudly when checks exist but none are recall-gated,
so a hollow gate is red rather than a green "scored cases: 0 of N". A genuinely
empty report (0 checks) still passes — it's not a degenerate-corpus signal.
- README.md: the harness substrate section still documented the old
`node --import tsx src/cli/index.ts …` child invocation this PR replaced;
update it to the dist-preferred form to match `cliChildArgs`.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(routes): persist HTTP method on Route nodes
Part 1 of 2 for issue #2138 (skip redundant HTTP provider source-scan).
The ingestion routes phase already knows each route's HTTP verb —
`ExtractedRoute.httpMethod` (Spring/Laravel framework routes) and
`ExtractedDecoratorRoute.httpMethod` (decorator routes) — but dropped it
when creating the Route graph node. As a result `HttpRouteExtractor`'s
graph-assisted path could not recover the verb for `framework-route`
sources (whose edge `reason` is undecodable by `methodFromRouteReason`)
and had to fall back to re-scanning the handler source.
Changes:
- routes phase: carry `httpMethod` into `RouteEntry` and persist it as
`Route.method` (filesystem-derived Next.js/Expo/PHP routes have no
structural verb, so they stay method-less).
- HttpRouteExtractor: HANDLES_ROUTE query now returns `route.method`;
`extractProvidersGraph` prefers it and falls back to the edge reason
for older indexes / method-less routes (fail-open, fully backward
compatible).
- tests: graph-method precedence, multi-verb handler disambiguation via
the persisted verb, case normalization, and old-index fallback.
This change is intentionally NOT a performance optimization on its own:
the graph path still parses handler files to recover the handler *name*.
Eliminating that parse (and thus the redundant source-scan #2138 targets)
requires linking HANDLES_ROUTE to the handler symbol, which lands in
Part 2. This PR is the data-completeness groundwork for that.
Refs #2138
* test: account for new Route.method in blade route-registry assertion
The routes phase now persists httpMethod onto RouteEntry/Route nodes, so
the strict toEqual on the framework-route registry entry must include the
new method field.
* fix(routes): persist Route.method end-to-end + real-lbug round-trip test
Addresses review on #2234 (magyargergo + tri-review): the prior commit
read `route.method` in HANDLES_ROUTE_QUERY but never added the column to
the schema/persistence path, so against a real LadybugDB the query failed
to bind (`Cannot find property method for r.`) and the `catch { return [] }`
silently swallowed it — regressing the graph-assisted HTTP provider path.
- schema: add `method STRING` to ROUTE_SCHEMA.
- csv-generator: write `method` in the Route CSV row (header + row, column
order aligned with the COPY statement).
- lbug-adapter: add `method` to getCopyQuery('Route').
- routes phase: normalizeRouteMethod() canonicalizes the verb to upper-case
and skips non-verbs — Laravel resource/apiResource carry httpMethod
values like `resource`/`apiResource`, which must not land a junk method.
- http-route-extractor: log at debug when the HANDLES_ROUTE / FETCHES graph
query throws, so a total graph-provider outage is observable instead of
silently swallowed. Export HANDLES_ROUTE_QUERY for the round-trip test.
- tests: add a real-lbug round-trip (graph -> CSV -> COPY -> HANDLES_ROUTE_QUERY)
asserting the verb persists and reads back; update the blade registry
assertion for the normalized (upper-case) method.
Refs #2138
* fix(csv): coerce Route.method to string for escapeCSVField typecheck
node.properties.method is typed unknown (not a declared property), so
`x || ''` stayed unknown and failed tsc against escapeCSVField's
string|number param. Coerce explicitly with String(... ?? '').
* test(bench): regenerate emit-persistence fingerprint for Route.method column
Adding the method column to route.csv changes the byte-identity
fingerprint of the emit-persistence benchmark (the synthetic graph's
route.csv header now includes 'method'). scaling_ratio unchanged (~0.9,
linear); this is the documented regenerate-on-legitimate-emit-change
path. Streaming baseline (BasicBlock/PDG) is unaffected.
---------
Co-authored-by: henry <zhangwei2017@unipus.cn>
Co-authored-by: Gergő Magyar <gergomagyar@icloud.com>
* fix(server): resolve clone/upload/mapping roots from GITNEXUS_HOME
CLONE_ROOT (git-clone.ts), UPLOAD_ROOT (upload-paths.ts), and the
server-mapping file (embeddings/server-mapping.ts) computed their root
from os.homedir() directly, ignoring GITNEXUS_HOME. The Docker image
sets GITNEXUS_HOME=/data/gitnexus (the persistent volume that also holds
the registry and indexes), so cloned/uploaded repos and their .gitnexus
indexes instead landed in the container's ephemeral ~/.gitnexus and were
lost on container recreation, while registry.json (which honors
GITNEXUS_HOME) kept pointing at the now-dead path. That also defeated
incremental re-analysis: a recreated container re-clones from scratch and
full-rebuilds instead of git pull + incremental update.
Source all three from the existing getGlobalDir() helper — the same
GITNEXUS_HOME-aware primitive the registry and groups already use.
Behavior is unchanged when GITNEXUS_HOME is unset (CLI / local installs):
it falls back to ~/.gitnexus exactly as before. No signatures change and
no UPLOAD_ROOT consumers are touched.
Adds test/unit/gitnexus-home-roots.test.ts covering both the
GITNEXUS_HOME-set and unset paths for all three roots.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(server): make clone-root assertions GITNEXUS_HOME-aware; cover server-mapping fallback
Addresses PR review (#2229):
- git-clone.test.ts hardcoded path.join(os.homedir(), '.gitnexus', 'repos')
for the clone root, so the "direct child of the clone root" and containment
assertions failed when GITNEXUS_HOME was set in the ambient env (e.g. a CI
runner). CLONE_ROOT now derives from getGlobalDir(); mirror that derivation
via EXPECTED_CLONE_ROOT (GITNEXUS_HOME || ~/.gitnexus), computed at module
load — the same point CLONE_ROOT is frozen — so the two always agree.
- The GITNEXUS_HOME-unset fallback test covered clone + upload roots but not
server-mapping (one of the three changed modules). Add a fallback case for
readServerMapping. It redirects HOME/USERPROFILE to a tmp dir (os.homedir()
honors them) so it exercises the real ~/.gitnexus fallback without writing
into the developer's actual ~/.gitnexus/server-mapping.json.
Both files pass with and without GITNEXUS_HOME set.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(server): use mkdtemp for GITNEXUS_HOME path-root test temp dirs
CodeQL js/insecure-temporary-file flagged the two writes that used a fixed
os.tmpdir() name (gitnexus-home-roots-test, gitnexus-fallback-home): a
co-located process could pre-create or symlink the predictable path before
the test write lands. Switch both to fs.mkdtemp(), which atomically creates a
uniquely-named directory — the canonical sanitizer this repo already uses
(see core/group/storage.ts). Behavior is unchanged; tests still pass with and
without GITNEXUS_HOME set.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(embeddings): resolve server-mapping path for parity with clone/upload roots
MAPPING_FILE now wraps path.resolve() like CLONE_ROOT (git-clone.ts) and
UPLOAD_ROOT (upload-paths.ts), so a relative GITNEXUS_HOME yields an absolute
path. No-op for the supported absolute-GITNEXUS_HOME config (Docker) and for
readServerMapping's only caller (run-analyze.ts).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(server): derive EXPECTED_CLONE_ROOT from getGlobalDir() to avoid drift
The test mirror now imports getGlobalDir() and computes the expected clone root
the same way production CLONE_ROOT does, instead of re-deriving the
GITNEXUS_HOME || ~/.gitnexus fallback by hand. Byte-identical today; future-proof
if getGlobalDir() grows a branch. (os import retained — still used by os.tmpdir().)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(server): rename shadowed savedHome in server-mapping fallback test
The inner savedHome (saves process.env.HOME) shadowed the describe-scope
savedHome (saves process.env.GITNEXUS_HOME). Renamed the inner one to
savedProcessHome at its declaration and both restore sites in the finally
block. Pure rename, no behavior change.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(server): scope customHome temp dir to the GITNEXUS_HOME-set cases
beforeEach created a customHome temp dir for all five tests, but the two
fallback cases never use it (one manages its own fakeHome, the other needs
none). Moved the mkdtemp/rm into a nested describe('with GITNEXUS_HOME set')
wrapping the three set-cases; the shared outer afterEach still restores
GITNEXUS_HOME and resets modules for all five.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Gergő Magyar <gergomagyar@icloud.com>
* test(lbug): lock PARALLEL=false as a tested correctness invariant (#2203)
The parallel CSV reader (Kuzu-derived, default PARALLEL=true) cannot parse
quoted fields with embedded newlines (kuzudb/kuzu#5778); our content/text
columns hold source code, so PARALLEL=false is mandatory for correctness.
Add a live-DB multiline-quoted round-trip that fails if it is ever flipped,
plus a static guard on the generated COPY queries. Export COPY_CSV_OPTS /
getCopyQuery for the static assertion; document the invariant at the source.
* feat(lbug): expose node/rel phase boundary via onNodePhaseComplete hook (#2203)
streamAllCSVsToDisk now fires an optional onNodePhaseComplete(nodeFiles)
callback right after node CSVs are flushed and before the relationship pass
writes any rel_*.csv — the boundary the COPY-overlap leg needs. The node-file
manifest construction is hoisted above the rel pass and reused in the return,
so output is byte-for-byte identical when no callback is supplied (verified by
the emit bench fingerprint and the splitRelCsvByLabelPair differential oracle).
The callback is not awaited, so the rel pass runs concurrently with the
caller's node COPY.
* perf(lbug): overlap node COPY with relationship emit (#2203)
The deferred parallelism leg of #2203. LadybugDB is single-writer and its
parallel CSV reader is unsafe for our multiline content (kuzudb/kuzu#5778), so
the only safe parallelism is pipeline-overlap: node COPY (uses conn, never the
rel files) runs concurrently with the relationship emit pass (writes rel_*.csv,
never conn). Node COPY now starts at streamAllCSVsToDisk's onNodePhaseComplete
boundary while the rel pass keeps writing; the relationship COPY still waits for
node COPY (FK precondition), so DB load order and content are unchanged.
- Extract copyNodeCSVs; start it in the hook (overlap) or after emit (serial).
- GITNEXUS_SERIAL_LBUG_LOAD=1 forces the legacy strictly-sequential path
(operator escape hatch + differential-test oracle).
- Settle the in-flight node-COPY promise on emit failure (no unhandled
rejection); rethrow node-COPY errors at the FK barrier.
- Preserve the PDG manifest merge + collision guards (node merge at the hook,
rel merge before rel COPY) and all retry/fallback/cleanup behavior.
- PROF_LBUG_LOAD gains mode=overlap|serial; copy-nodes becomes the residual
node-COPY time after emit (trends to 0 as overlap hides it).
* test(lbug): differential gate — overlap load === serial load (#2203)
Loads one fixture (multiple node tables, multiple edge pairs, multiline File
content + BasicBlock text) into two fresh DBs — once via the default node-COPY
‖ rel-emit overlap, once via GITNEXUS_SERIAL_LBUG_LOAD=1 — and asserts the two
databases are content-equivalent: identical per-table node counts, per-type
edge counts, byte-for-byte multiline content/text, and identical
insertedRels/skippedRels/warnings. This is the issue's byte-identical-content
acceptance gate for the parallelism leg.
* fix(review): apply autofix feedback
- csv-generator: onNodePhaseComplete doc-contract now matches reality (a sync
throw is allowed and is how loadGraphToLbug surfaces the manifest collision
guard) — drops the inaccurate 'must not throw synchronously' line.
- lbug-adapter: copyNodeCSVs totalSteps is the node-table count (drop the +1
rel-step holdover; the rel COPY has its own progress line).
- lbug-adapter: on emit+node-COPY double-failure, log the swallowed node-COPY
error before rethrowing the emit error (diagnosability).
- lbug-load-prof test: assert mode=overlap on the default path.
* fix(test): use mkdtemp for secure temp dirs (CodeQL js/insecure-temporary-file)
CodeQL flagged lbug-load-overlap.test.ts writing a file into a predictable
os.tmpdir() path. Create the base temp dir with fs.mkdtemp (atomic, random
suffix) in both new live-DB tests, and switch to the gitnexus-lbug- prefix that
TEST_FIXTURE_PREFIXES recognizes so the Windows stale-sidecar sweep covers
these fixtures.
* fix(lbug): check PDG manifest rel-pair collision before node COPY (#2203)
Found by Codex in tri-review. The manifest rel-pair collision guard ran after
the FK barrier (after node COPY committed), so on that should-never-happen
error branch the overlap path left orphan node rows AND the
GITNEXUS_SERIAL_LBUG_LOAD escape hatch diverged from the legacy 'validate
manifest before any COPY' behavior. Move the rel merge + collision check ahead
of beginNodeCopy/the barrier: the serial path now detects a collision before
committing any node rows (legacy parity restored — the escape hatch is a
faithful oracle again), and the overlap path detects it as early as csvResult
is available. The node-collision guard already ran before node COPY (in the
hook).
* test(lbug): cover rel-emit failure with node COPY in flight (#2203)
Resolves a P1 review gap on PR #2226: the overlap's catch(emitErr) branch
(settle the in-flight node-COPY promise, then rethrow the emit error) was
untested. Fault-injects via a vi.mock of streamAllCSVsToDisk that fires
onNodePhaseComplete (starting a real node COPY on a live DB) then throws,
asserting loadGraphToLbug rejects with the emit error and no unhandled
rejection leaks. Also covers the both-fail case (node COPY error is logged,
emit error still wins). Listener removed in finally; macrotask queue flushed
before the assertion so it can't pass vacuously.
* test(lbug): cover node-COPY hard-failure rethrow at the FK barrier (#2203)
Resolves the second P1 review gap on PR #2226. Mocks emit to fire
onNodePhaseComplete with a nodeFiles entry pointing at a missing CSV (a
bind-time COPY error that IGNORE_ERRORS does not suppress) and otherwise
succeed, so copyNodeCSVs throws, the error is captured in nodeCopyError, and
loadGraphToLbug rethrows it at the FK barrier — asserted via rejects /COPY
failed for File/.
* test(lbug): cover PDG manifest rel-pair collision in overlap + serial (#2203)
Resolves the P2 gap behind the Codex tri-review finding: the manifest rel-pair
collision guard (moved ahead of node COPY in ad195582) had no test. A leaky
graph with a structural BasicBlock->BasicBlock edge (routed by id-prefix, no
BasicBlock nodes — isolating the rel-pair clash from the node-CSV one) plus a
PdgEmitSink manifest declaring the same pair makes loadGraphToLbug reject with
the rel-pair collision error, asserted on both the overlap (default) and serial
(GITNEXUS_SERIAL_LBUG_LOAD=1) paths.
* perf(lbug): yield the event loop periodically during relationship emit (#2203)
Resolves a P2 review finding on PR #2226: the relationship-emit loop ran long
synchronous stretches between write-stream drain awaits, which could starve the
overlapped node-COPY callbacks on fast I/O and erode the node-COPY-||-rel-emit
overlap. Yield via setImmediate every REL_YIELD_EVERY (5000) edges so the node
COPY and drains get scheduling time. Scheduling-only — emit bench fingerprint
unchanged (byte-identical), csv-pipeline determinism + overlap differential
green.
* refactor(lbug): extract shared copyCsvWithRetry helper (#2203)
Resolves a P2 maintainability finding on PR #2226: the COPY + IGNORE_ERRORS
retry block was duplicated in copyNodeCSVs and the inline relationship-COPY
loop. Extract copyCsvWithRetry(conn, query, onError); the callback receives the
RAW retry error so each site keeps its own message shape + slice length (node
throws, slices 200; relationship warns + records the failed pair, slices 80).
Behavior-preserving — guarded by the live-DB round-trips plus the new
node-COPY-failure and overlap error-path tests.
* docs(lbug): document loadGraphToLbug non-transactionality (#2203)
Resolves the advisory review finding on PR #2226: loadGraphToLbug runs
independent COPYs with no surrounding transaction, so a mid-load failure leaves
a partial DB and recovery is a --force re-analyze. Make that contract explicit
on the function so callers don't assume atomicity.
* perf(lbug): add PROF_LBUG_LOAD persistence-path timing breakdown (#2203 U1)
loadGraphToLbug is un-timed today; the analyze 'emit' number is the
scope-resolution emit bucket, not the CSV->COPY persistence path. Add a
zero-cost-when-off per-stage breakdown (csv-emit/copy-nodes/rel-split/
copy-rels/fallback/total + node/rel counts) gated by PROF_LBUG_LOAD=1,
mirroring the PROF_SCOPE_RESOLUTION pattern. Document the flag in README.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* perf(lbug): route relationships to per-pair CSVs in the emit pass (#2203 U2)
Relationships were written once to a monolithic relations.csv, then re-read
line-by-line (regex per edge) and re-split into per-FROM->TO-label-pair files
before COPY — writing and reading the entire ~1M-edge set twice. Route each
edge to its pair file directly during the single emit pass via a shared
RelPairRouter, eliminating the monolithic write + re-read + per-edge regex.
The router applies the SAME getNodeLabel + validTables filter as the legacy
splitRelCsvByLabelPair, which is retained as a differential oracle. A new
differential test asserts the direct-emit per-pair files are byte-for-byte
identical to the oracle's, with identical skip/total accounting. The prof
line (U1) drops its rel-split stage (routing now folds into csv-emit).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* perf(lbug): skip per-row microtask tick in BufferedCSVWriter (#2203 U3)
addRow awaited an already-resolved promise on every buffered row, scheduling
a microtask per node even when nothing flushed (millions at scale). It now
returns a promise ONLY when it flushes; the node-emit loop awaits once per
iteration after the switch. Flush/drain semantics are unchanged, so
backpressure on the rows that actually write is preserved and the emitted
CSV bytes are byte-identical (covered by the determinism + differential tests).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* bench(lbug): emit throughput + byte-identity gate for the persistence path (#2203 U4)
Build-free bench (bench/emit-persistence/measure.mjs) times streamAllCSVsToDisk
on a synthetic graph at two scales and gates: (1) an order-independent sha256
fingerprint over every emitted CSV line — the byte-identity guard for the U2/U3
emit optimisations — and (2) a scaling-ratio budget catching an O(n^2) emit
re-regression. Wired into ci-tests.yml alongside the cfg/scope-capture benches.
The LadybugDB COPY half needs a real DB, so its timing stays in PROF_LBUG_LOAD
+ the integration round-trip tests (documented in the bench README, with the
deferred COPY-parallelism follow-up).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(review): apply autofix feedback (#2203)
- P1: router backpressure drain-await rejected with a generic AbortError,
masking the real EMFILE/disk-full error. Expose RelPairRouter.lastError and
rethrow it in the emit catch — mirrors the oracle's throw streamError ?? err.
- P1: cover RelPairRouter error + backpressure + teardown paths with a new
unit test (test/unit/rel-pair-routing.test.ts) using an injected mock stream.
- P2: wrap streamAllCSVsToDisk body in try/finally so the setMaxListeners bump
is always restored (the U2 rel-routing throw path could leak it).
- P2: dedup WriteStreamFactory — re-export the canonical type from
rel-pair-routing instead of a second identical declaration.
- P2: annotate splitRelCsvByLabelPair @internal as the retained differential
oracle so a future dead-code sweep doesn't delete the byte-identity guard.
- P3: differential test now covers the proc_ prefix + clears
GITNEXUS_SORT_GRAPH_OUTPUT to prevent env-leak desync.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(lbug): scope byte-identity to quote-free ids + lock the quote-in-id divergence (#2215 review)
The 'byte-identical' claim was unconditional, but the router derives labels from
the raw id while the retained splitRelCsvByLabelPair oracle re-derives them via a
regex over the escaped row — so for an id containing a double-quote they diverge
(the router is the more-correct path). Soften the wording in rel-pair-routing.ts,
the bench README, and the differential-test comment to document the exception,
and add a differential test asserting the intended divergence (router routes the
quote-in-id edge; oracle drops it) so a future change can't silently revert to
the buggy regex semantics.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* bench(lbug): per-file fingerprint so the gate catches pair-file mis-routing (#2215 review)
fingerprintEmit flattened every line of every per-pair file into one array,
sorted globally, and hashed — losing file boundaries, so a row routed to the
WRONG pair file produced an identical fingerprint. Hash a per-file digest
(filename + sha256(file bytes)) and combine the sorted entry list, so mis-routing
(and within-file row reordering) now changes the fingerprint. Baseline
regenerated; the new scheme yields a different hash on byte-identical emit,
confirming it is sensitive to file structure the old flatten ignored.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* bench(lbug): add absolute large-scale wall-time backstop to the emit gate (#2215 review)
The scaling-ratio gate only compares large/small, so a uniform Nx slowdown at
both scales passes with ratio ~1.0. Add an opt-in max_ms_large ceiling (1000ms
vs observed ~200ms — generous, host-noise-tolerant) that --check enforces
alongside the ratio, catching a gross absolute regression the ratio misses.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(lbug): cover the sorted-output path in the byte-identity differential (#2215 review)
The differential test only exercised the default insertion-order emit path. Add
a case under GITNEXUS_SORT_GRAPH_OUTPUT=1 that feeds the oracle the same
id-sorted order orderedRelationships() uses and asserts per-pair byte-identity,
so within-pair row reordering on the sorted path can't slip past the gate.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(lbug): cover the invalid-TO-label skip branch (#2215 review)
Only an invalid-FROM label was exercised; the validTables skip is an OR over
both endpoints, so the invalid-TO branch was untested (an inverted && would
have slipped through). Add a valid-FROM/invalid-TO edge to the differential
test and the router unit test, asserting it's skipped identically by router and
oracle.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(lbug): exercise the BufferedCSVWriter FLUSH_EVERY boundary in vitest (#2215 review)
The U3 addRow change (returns a flush promise only on flush; undefined when
buffered) and the loop's `if (pending) await pending` were only crossed by the
bench, never vitest (all fixtures are <500 nodes). Add a 600-node graph through
streamAllCSVsToDisk asserting all rows land exactly once across the 500-row
flush boundary — no drops, dups, or corruption.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(lbug): drop redundant step cast in buildRelRow (#2215 review)
GraphRelationship.step is already typed number?, so (rel as { step?: number }).step
was a no-op structural cast that obscured the shared-type coupling. Use rel.step
directly. Byte-identical — bench fingerprint unchanged, differential test green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(lbug): make the unknown-label node drop explicit (#2215 review)
With the U3 `let pending` switch idiom, a node whose label matches neither
codeWriterMap nor multiLangWriters left `pending` undefined and was silently
dropped — a footgun for a future node type. Add an explicit else with a comment
documenting that unknown labels are intentionally not persisted and that a new
type must be wired into a writer map. No behavior change (byte-identity + tests
unchanged).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(lbug): drop the unused WriteStreamFactory re-export (#2215 review)
The type was re-exported from lbug-adapter 'to preserve this module's surface,'
but no external code imports it by name from here (the only test reference is a
comment). Keep the import from rel-pair-routing.ts (its canonical home, still
used by splitRelCsvByLabelPair's signature) and drop the dead re-export.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(cfg): retain dense reaching-defs as differential oracle + fuzz harness (#2201 U1)
* refactor(cfg): extract shared harvest/adjacency/sweep + swappable in-set computer (#2201 U2)
* perf(cfg): sparse change-driven reaching-defs solver + canonical truncation (#2201 U3,U4)
* perf(cfg): switch production reaching-defs to the sparse solver (#2201 U5)
* perf(cfg): true SSA-sparse reaching-defs solver with auto-dispatch (#2201 U3)
Replace the per-variable worklist (correct but no faster — it still walks
pass-through blocks per binding) with Cytron SSA: CHK dominators + dominance
frontiers + phi-placement + stack renaming over a synthetic entry, answering
block-entry reaching queries by walking the SSA def-use graph (SCC-condensed,
cycle-safe). Pass-through blocks carry the dominating def via the rename stack
and phi-nodes statically capture loop merges, so dense-bindings drops from
O(n^2) to O(n) (5-23x faster, asymptotic) and deep nests are depth-independent.
The sweep now queries a lazy reachingAt accessor with a sparse intra-block
overlay (no full per-block lattice copy). Production auto-dispatches: SSA for
looping functions >=16 blocks (where it pays off, incl. the deep nests the
dense ceiling used to truncate -> ceiling stops firing), dense elsewhere (small
/ loop-free functions, 1.0x — no regression). Throw-edge and unreachable-block
functions fall back to dense (byte-identical). Held byte-identical to the dense
oracle across a 300k-CFG (~1.2M-comparison) differential fuzz.
* test(cfg): R5 contrast — dense ceiling fires, SSA solver converges (#2201 U6)
* bench(cfg): deep-nest scenario + tighten dense-bindings rd budget 10->2 (#2201 U7)
dense-bindings rd_scaling drops 5.2->0.86 (SSA linear); budget tightened to 2.0.
New deep-nest scenario (N nested loops, one carried var) measures rd under the
production blocks×64 ceiling and asserts the SSA solver still COMPUTES full
facts (facts_large_min) where the dense worklist would truncate — the
ceiling-stops-firing acceptance. CFG fingerprints unchanged.
* docs(cfg): document SSA-sparse solver + resolve the WTO no-go note (#2201 U8)
* fix(review): apply autofix feedback (#2201)
- Close the production SSA-dispatcher fuzz-coverage gap: the generator's
maxBlocks=14 was below SSA_MIN_BLOCKS=16, so the auto-dispatcher's SSA branch
was never differentially fuzzed. Raise to 36, add a hadLargeLoop coverage
assertion + a back-edge-into-entry canonical CFG. Validated byte-identical on
100k random CFGs incl. >=16-block looping shapes via both entry points.
- Correct stale function JSDocs + @internal annotations (dispatch/fallback roles).
- Add an independent rd_all_computed bench gate (catches partial truncation).
- maxBlockVisits comment, SSA_MIN_BLOCKS calibration note, nx->next rename.
* fix(cfg): gate out-of-range binding indices to the dense fallback (#2201 review)
Tri-review (adversarial lane, reproduced) found the SSA path less tolerant than
the dense oracle it replaced: an out-of-range binding index in defs/uses/mayDefs
(a corrupted/stale durable store) crashed the nBindings-sized arrays
(defBlocks[v]/stacks[u]), where dense tolerated it as a Map key. The throw
escaped the unguarded taint/harvest call sites and lost a whole file's taint
layer. Add a malformed-input gate that falls back to the dense solver (which
handles any index), preserving byte-identity AND the graceful per-function
degradation. Add an OOB canonical CFG to the differential fuzz + a production-
entry no-throw unit test (the generator only ever emitted in-range indices, so
this divergent input was structurally invisible).
* perf(cfg): bound the SSA value-graph, fall back to dense when oversized (#2201 review R1)
maxFacts bounds fact materialization in sweepFacts, but nothing bounded the
SSA-sparse solver's φ/value-graph construction. A high-binding-density deep
loop routed to SSA (≥16 blocks + a reachable loop) builds an O(blocks×bindings)
value graph the dense path would have truncated at its maxBlockVisits ceiling
(~1.5 GB measured on a 3000-block × 300-binding function).
Cap the value graph: after φ-placement (where nodeKeys.length == the φ count,
the input-superlinear term) plus a 2×Σgen bound on the renaming nodes, fall
back to computeInSetsDense before paying for renaming + Tarjan SCC. The fallback
is byte-identical (dense is the equivalence oracle) and bounded (dense honors
maxBlockVisits). Mirrors the existing throw/unreachable/OOB-binding gates.
The ceiling is DEFAULT_MAX_SSA_VALUE_GRAPH_NODES (1e6 — far above any real or
benchmarked function; dense-bindings/deep-nest build <1e4), overridable per call
via ReachingDefsLimits.maxSsaValueGraphNodes. The new unit test makes the
otherwise-invisible routing flip observable by pairing the cap with a tight
maxBlockVisits (dense truncates, SSA computes). Equivalence fuzz unchanged
(byte-identical, 20k CFGs green); tsc clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* perf(cfg): alias single-source SCC reaching-sets in reachByScc (#2201 review R2)
The SCC-condensation pass built a fresh Set for every SCC and copied each
cross-SCC operand's reaching-set element-by-element — O(defs²) at wide-fan-in φ
merges (a φ over many predecessors, each carrying a large reaching-set).
Add an alias fast path: an SCC with no own leaf keys whose cross-SCC operands
all resolve to ONE source SCC has exactly that source's reaching-set, so share
it by reference instead of copying. This is the common shape (pass-through φ /
single-operand value node). The full union is still built when an SCC has own
keys or genuinely merges ≥2 distinct sources.
Safe to share: reachByScc sets are read-only after construction (operand SCCs
are numbered before s in Tarjan's reverse-topological order and are only
iterated), and contents are identical — set iteration order is irrelevant
because sweepFacts sorts each use's keys before emission (KTD6). Byte-identical
to the dense oracle (30k-CFG fuzz green); tsc clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* perf(cfg): fold the SSA reachability gate into the RPO pass (#2201 review R8)
computeInSetsSparse ran a standalone reachability BFS to gate unreachable-block
functions to the dense oracle, then immediately computed a reverse-post-order
over the synthetic-entry graph — two traversals of the same successor structure.
reversePostOrder now returns the reachability bitmap its DFS already builds, and
the sparse path reuses it for the unreachable-block gate (S→entry is S's only
edge, so reachX[b] for b<n is exactly "reachable from entry" — identical to the
removed BFS). One traversal instead of two on every SSA-dispatched function.
The dispatcher's hasReachableLoop pass is left in place: it decides SSA-vs-dense
BEFORE the solver is entered, and computeInSetsSparse must stay self-contained
(the equivalence fuzz drives it directly, bypassing the dispatcher), so the two
cannot share a traversal without coupling the InSetsComputer contract.
Routing and facts unchanged — byte-identical to the dense oracle (30k-CFG fuzz,
including unreachable-block shapes, green); tsc clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* perf(cfg): trim per-statement/per-use/per-block allocations (#2201 review R9)
Three transient allocations in the hot paths, all behavior-preserving:
- sweepFacts: replace the per-statement `new Set([...defs, ...mayDefs])` with a
direct `includes()` scan over the (1–3 element) def/mayDef arrays, guarded by a
cheap hasSelfDefs flag that short-circuits pure-use statements.
- sweepFacts: reuse a single scratch array for each use's reaching def-keys
instead of spreading a fresh array per use. The KTD6 pre-sort still runs in
place (load-bearing for truncated byte-identity).
- computeInSetsSparse: build dPredsX by skipping consecutive-equal `from` values
(preds[b] is pre-sorted by buildAdjacency, so duplicates are adjacent) instead
of a per-block Set + spread + sort; the synthetic entry S = n exceeds every
block index so it appends in order.
The sweep is shared with the dense oracle, so these stay byte-identical on both
paths — 50k-CFG fuzz (incl. maxFacts truncation, the order-sensitive case)
green; tsc clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(cfg): correct the sweepFacts truncation byte-identity mechanism (#2201 review R6)
The outer sweepFacts JSDoc attributed a truncated result's cross-solver
byte-identity to the two solvers producing "identical inSets — insertion order
included". That is wrong: the dense (RPO fixpoint) and SSA (renaming/SCC)
solvers deliberately build a loop-carried use's reaching set in DIFFERENT
insertion orders — same set, different order. The actual mechanism is the KTD6
per-use sort that canonicalizes each use's keys by defKey BEFORE the maxFacts
cutoff (already documented correctly on the inner comment). Rewrite the outer
doc to say so. Documentation only.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(cfg): extract pure graph sub-stages to reaching-defs-graph.ts (#2201 review R4)
reaching-defs.ts had grown to ~1190 lines with the #2201 SSA rewrite. Move the
self-contained, pure (plain-array) algorithms into a sibling module:
- reversePostOrder
- buildDominators (Cooper-Harvey-Kennedy)
- buildDominanceFrontiers (Cytron)
- tarjanScc + condenseReachingSets (SCC condensation, alias fast path)
- hasReachableLoop (dispatcher loop check)
- unionSets / latticeEquals (def-set / lattice primitives)
The new module has a STRICT one-way dependency (it imports nothing from
reaching-defs.ts — every helper is parameterized over plain arrays/Sets), so
there is no import cycle and each stage is independently testable. reaching-defs.ts
now holds the orchestrator, the two solver bodies, harvest, adjacency, the
statement sweep, and the dispatcher: 1190 → 988 lines.
Pure mechanical extraction — behavior is preserved by the differential
equivalence fuzz (40k CFGs byte-identical) + the reaching-defs unit/snapshot
suites; tsc clean. The helpers are @internal (kept out of the shipped .d.ts by
the stripInternal change).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(pdg): stamp the reaching-defs solver identity for incremental re-analysis (#2201 review R3)
The SSA-sparse rewrite computes full REACHING_DEF facts for deep-loop functions
the old dense worklist truncated to empty at the blocks×64 ceiling. But an
existing `--pdg` index carries those stale-truncated rows, and nothing forced a
re-analysis: RepoMeta.pdg had no solver-identity key, so an upgraded run over an
unchanged file kept the incremental fast path and never recomputed.
Add a constant `reachingDefSolver: 'ssa-sparse-v1'` to the resolved pdg stamp
(and to the RepoMeta['pdg'] type). It rides the existing key-union
pdgModeMismatch comparator: a pre-#2201 stamp lacks the key, so
'ssa-sparse-v1' !== undefined trips one full writeback that recomputes the
fuller coverage — no `--force` needed — exactly like the M2 REACHING_DEF cap and
M5 CDG cap upgrade paths. A matching post-#2201 stamp compares equal, so there
is no spurious re-analysis churn on steady-state re-runs.
Tests: new pre-#2201→SSA upgrade block in pdg-mode-flip.test.ts (stamp present,
absent-key mismatch, identical-stamp no-churn) + the persisted-stamp shape
assertions and resolvePdgConfig DEFAULTS updated for the new key. tsc clean;
pdg-mode-flip + run-analyze suites green (55/55).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* build(ts): stripInternal so @internal test-only exports stay out of the shipped .d.ts (#2201 review R5)
computeReachingDefsDense/computeReachingDefsSparse are exported only for the
equivalence fuzz and tagged @internal, but `declaration: true` emitted them into
the public dist/**/*.d.ts. stripInternal removes any @internal-tagged export from
the declaration output.
This is repo-wide, which is the intended behavior: the same applies to every
other test-only @internal export (hf-env's withDownloadTimeout etc., worker-pool's
buildDispatchMessage/crashSignature, parse-impl's handleWorkerStartupFailure, the
logger/safe-parse test resets, and the new reaching-defs-graph SSA helpers) — all
of which are documented as not-public.
Verified:
- declaration emit succeeds with no TS4094/TS9006 ("cannot be named") errors;
- the @internal functions are gone from the emitted .d.ts (reaching-defs-graph.d.ts
is now `export {};`), while public symbols (computeReachingDefs) remain;
- gitnexus-web — the only cross-package consumer — typechecks clean and imports
only from gitnexus-shared, never from gitnexus internals;
- runtime .js and the vitest/tsx tests are source-based, so unaffected.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(bench): add wide-merge scenario + tighten deep-nest facts floor (#2201 review R7)
wide-merge: N bindings, each assigned in a 3-way branch (a wide multi-operand φ
per binding) inside a loop, then all used after the merge. Unlike dense-bindings
(one chained redef per `if`), every binding fans into its own wide φ, so the
scenario exercises φ-placement + renaming + the reachByScc condensation across
many independent wide merges. N bindings × constant arms ⇒ O(N) facts, so the
gate is rd_scaling LINEARITY (measured ~1.07; budget 2.0 catches a regression to
the per-binding-rescan O(N²) class the reachByScc alias path guards against). It
runs the production SSA path (10007 blocks + a loop) and computes all facts under
the blocks×64 budget (facts_large_min 24000 of a measured 26008 + the
rd_all_computed gate).
deep-nest: tighten facts_large_min 100 → 150 (measured 164) so a partial-
truncation regression that still cleared 100 — but lost facts — now fails, with
~9% headroom for noise.
bench --check PASS (9 scenarios) under --expose-gc; all existing CFG fingerprints
unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* style(cfg): drop trailing blank line in reaching-defs.ts (prettier)
Whitespace-only — a stray trailing newline left by the U4 extraction. `prettier
--check` (the root format CI gate) now passes on every changed file. No behavior
change.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
