mirror of https://github.com/abhigyanpatwari/GitNexus.git synced 2026-08-28 05:25:25 +00:00

Mirror of https://github.com/abhigyanpatwari/GitNexus.git

Find a file

Copilot c34c36036f Some checks are pending CodeQL / Analyze (javascript-typescript) (push) Waiting to run Details CodeQL / Analyze (python) (push) Waiting to run Details Gitleaks / gitleaks (push) Waiting to run Details Publish / Classify release event (push) Waiting to run Details Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions Details Publish / ci (push) Blocked by required conditions Details Publish / Publish to npm (push) Blocked by required conditions Details Publish / Build & Push RC Docker images (push) Blocked by required conditions Details Scorecard / Scorecard analysis (push) Waiting to run Details Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run Details Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run Details fix(workers): resilient + zero-copy ingestion worker pool — prevent analyze hangs on TS-root-scale loads (#1693 ) * Initial plan * fix: skip worker-timeout files in sequential fallback and optimize TS capture node lookup Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/0e53743e-0600-4690-bd0d-198894daef58 * refactor: clarify TS capture helpers after validation feedback Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/0e53743e-0600-4690-bd0d-198894daef58 * fix(workers): exclude in-flight file on worker error/exit, not just singleton timeout WorkerPoolDispatchError previously surfaced the stalled path only for the singleton-timeout final-fail branch. Worker `error` and `exit` events (and the msg-channel `error` reply) fell back to plain `Error`, so the sequential fallback re-attempted every file in the active job — re-hanging on the same pathological file when the worker crashed mid-parse. Lift the in-flight-file inference into `inFlightExcludePath(job, lastProgress)` and wire it into the three remaining in-pool failure sites. `lastProgress` is already in `runWorker` scope, so `items[lastProgress]` (the next file the worker was about to acknowledge) is the best single guess at the culprit; earlier files are still re-tried sequentially. Returns `[]` when no path is determinable (`lastProgress >= items.length`, or path missing/non-string) so sequential retries the whole job. Replacement-worker startup failures stay plain `Error` (no job context); the result-before-flush protocol bug stays plain `Error` (code fault, not file). Tests cover the three new exclusion paths plus a negative test confirming non-WorkerPoolDispatchError throws fall through to full sequential retry. * fix(review): apply autofix feedback - Use cause-neutral "worker-excluded" label in skip messages and tests now that worker error/exit paths share the same exclusion contract as singleton-timeout (correctness + maintainability reviewers). - Add JSDoc to findSelfOrAncestorOfType{s} explaining the parent-walk short-circuit vs root-DFS fallback (maintainability reviewer). * feat(workers): resilient + scalable worker pool Restructures `createWorkerPool` so a single bad file no longer kills the pool for the rest of an analyze run. Five interlocking layers: 1. Auto-respawn on error/exit — worker death triggers `replaceWorker` on the same slot, bounded by `maxRespawnsPerSlot` (default 3). The slot is dropped from rotation when the budget is exhausted; other slots keep running. 2. Circuit breaker — replaces the permanent `poolBroken=true` with a consecutive-failure counter. The pool only trips after `consecutiveFailureThreshold` deaths (default `max(3, poolSize)`) with no successful job in between. A successful job resets the counter so transient bursts of bad files don't escalate. 3. Session-scoped file quarantine — paths identified as the in-flight file at the moment of a worker death are added to a `Set<string>` on the pool. `dispatch()` filters quarantined items up front (they never reach a worker again this pool lifetime). Exposed via the new `WorkerPool.getQuarantinedPaths()` so callers can log/route them. `processParsing` surfaces the per-chunk quarantine summary alongside the existing fallback-exclusion log. 4. Authoritative in-flight tracking — `parse-worker.ts` emits `{type:'starting-file', path}` before each file. The pool tracks this per slot and uses it for crash attribution, falling back to the `items[lastProgress]` heuristic only when no starting-file has been observed (very-early crash, older worker build). Closes the reorder/race concerns raised by reviewers C1 and R3 in the earlier review run. 5. Per-job cumulative timeout budget — each `WorkerJob` tracks the total wall time spent across attempts/splits/retries. When the budget is exhausted (default 5x `subBatchIdleTimeoutMs`), the pool surfaces the in-flight path instead of letting exponential backoff balloon into multi-hour stalls. Cross-layer wiring: a new `wakeIdleSlots` helper kicks any non-busy live slot when items are requeued (after a death or split-retry), so a dropped slot doesn't strand work in the queue. `recoverAndResume` consolidates the per-job teardown shared by the three in-pool death sites (`error`, `exit`, msg-channel `error`). New env knobs: `GITNEXUS_WORKER_MAX_RESPAWNS_PER_SLOT`, `GITNEXUS_WORKER_MAX_CUMULATIVE_TIMEOUT_MS`, `GITNEXUS_WORKER_CONSECUTIVE_FAILURE_THRESHOLD`. New `WorkerPoolOptions.workerFactory` injection point for unit tests. Tests: 12 new unit tests using a FakeWorker mock cover quarantine seeding, slot-respawn, slot-drop after budget, breaker trip + reset, and quarantine filtering. Plus option-resolution tests for the three new env vars. All 19 worker-pool/-fallback/-options tests pass; full unit suite 6040 passed / 30 skipped / 0 failed. * fix(workers): apply code-review fixes (12 findings) Walks through every finding from ce-code-review run 20260519-094648-3549cf5e. All 12 picked Apply. Critical: - F1 — Layer 5 cumulative-timeout exhaustion no longer silently drops the rest of the job. `requeueRemainder` is now invoked before `handleWorkerDeath` in both Layer 5 and singleton-final-fail give-up paths so non-quarantined items get re-tried by another worker. - F2 — idle-timer recovery overhaul. `!shouldContinue` branch no longer calls `replaceWorker` (double-spawn race with the `handleWorkerDeath` inside `requeueAfterTimeout`). `shouldContinue` branch now enforces `maxRespawnsPerSlot` before respawning, closing the budget-bypass for the timeout-retry path. Also fixes premature `maybeDone` by simplifying the bookkeeping. - F3 — `requeueRemainder` no longer pre-charges `cumulativeTimeoutMs` by `job.timeoutMs`. The death itself consumed no budget, so the next `requeueAfterTimeout` was double-billing the first attempt. - F4 — `WorkerPool.getQuarantinedPaths` is now optional on the interface, matching the defensive `?.()` call site and the existing mocks. Removes the contract-vs-callsite contradiction. - F5 — per-job unattributed-death tracking. When a worker dies with no exclusion attribution, `requeueRemainder` tracks death count per `startIndex`. First time: re-queue intact. Second time: quarantine items[0] as best guess, or drop the job entirely when items lack paths. Bounds the death loop the original design admitted to. - F6 — per-slot consecutive-failure counter. Replaces the pool-wide scalar so a chronically-failing slot trips the breaker on its own streak instead of being masked by another slot's successes. Smaller: - F7 — exhaustiveness `never` check on `WorkerOutgoingMessage` union. - F8 — recursive `runWorker` on fully-quarantined jobs converted to a while-loop. - F9 — `tripBreaker` calls `reject(err)` BEFORE awaiting `worker.terminate()`. A stuck terminate no longer blocks the caller. - F10 — `parsing-processor.ts` quarantine log de-duplicates per pool instance via a `WeakMap`. Only newly-quarantined paths are logged in each chunk; the per-chunk count still surfaces via progress. - F11 — extract `firstPath` local in `requeueAfterTimeout`; eliminates double `itemPath` call and the `unknown as string` cast. Tests (F12, 6 new): - crash-error event path (errorHandler). - F5 drop-branch coverage via items without `.path`. - Common-case unattributable crash falling back to items[0] heuristic. - `replaceWorker` startup failure (workerFactory emits 'exit' before 'online'). - All-slots-dropped breaker trip. - `GITNEXUS_WORKER_MAX_CUMULATIVE_TIMEOUT_MS` env override. Residual gap (deferred): no unit test exercises the Layer 5 cumulative-budget runtime path — requires fake-timer interleaving with FakeWorker that's too brittle for this iteration. Tracked. Unit suite: 257 files / 6056 passed / 30 skipped / 0 failed. * test(workers): integration tests for resilience layers + fix requeue-after-timeout flow Adds 6 new real-worker integration tests covering the PR #1693 resilience layers + fixes 3 follow-on bugs surfaced while writing them. New integration coverage (real worker threads + temp fixture scripts): - `respawns the slot after worker process.exit and finishes the work on the replacement` — exercises Layer 1 auto-respawn + Layer 3 quarantine through real IPC. - `attributes exactly via authoritative starting-file message on worker crash` — Layer 4 end-to-end: starting-file message → exact quarantine attribution (not the items[0] heuristic). - `quarantine filters subsequent dispatches without sending to a worker` — second dispatch's sub-batch payload audited via filesystem; the quarantined path is never sent across the message channel. - `drops a slot after maxRespawnsPerSlot and continues on the survivor` — 2-slot pool, slot dies twice past budget, survivor finishes re-queued remainder. - `trips the circuit breaker on cascading per-slot consecutive failures` — single-slot pool, dies on every job, breaker trips after consecutiveFailureThreshold with WorkerPoolDispatchError carrying the cumulative quarantine. - `survives a worker error event (uncaught throw) the same as a process.exit` — validates recoverAndResume on the errorHandler path via a real worker `throw` (not just process.exit). Bug fixes uncovered while writing these tests: 1. Stack-overflow recursion in runWorker's no-worker branch — `if (!worker) { ...; wakeIdleSlots(); maybeDone(); }` recursed indefinitely when multiple slots were mid-respawn simultaneously (wakeIdleSlots → runWorker → no worker → wakeIdleSlots → …). Removed the wakeIdleSlots call: the slot's own respawn IIFE owns runWorker post-respawn, and other slots will pick up work via finishJob's runWorker. 2. requeueAfterTimeout dispatched work before respawn completed — the F2 fix had `requeueAfterTimeout` `void`-discarding `handleWorkerDeath`, so the `!shouldContinue` IIFE had no way to know when the respawn finished. New design: `requeueAfterTimeout` returns a `TimeoutDecision` discriminated union; the IIFE owns the death-and-respawn-and-dispatch orchestration in an async closure so it can `await handleWorkerDeath` and then call `runWorker` deterministically. 3. Stalled-singleton + protocol-error + replacement-startup-crash tests had stale contracts predating the resilience refactor. The stalled-singleton no longer rejects (it quarantines + resolves `[]`); the protocol-error rejection message now mentions "circuit breaker tripped"; the replacement-startup-crash test documents the known `waitForWorkerOnline` race (online fires before the worker's main script runs, so a top-level throw looks like a successful spawn) — the test asserts the file is quarantined via the second-idle-timeout give-up path. Full suite: 334 files / 8982 passed / 43 skipped / 0 failed (second run; first run had a Vitest-reported flake from an uncaught worker exception bleeding into the test report — repeated runs are clean). * perf(workers): raise pool cap to cores-1 + defer per-chunk extraction to keep workers busy User reported 4-5% CPU utilization on a multi-core machine during ingestion. Two structural reasons: 1. Pool cap. `createWorkerPool` resolved size as `Math.min(8, max(1, os.cpus().length - 1))` — a 16-core box got 8 workers (50% theoretical max). U1 lifts the default to `min(16, max(1, cores - 1))`, exposes `GITNEXUS_WORKER_POOL_SIZE` env override, and adds `--workers <N>` CLI flag (`0` disables the pool for sequential fallback). 2. Per-chunk extraction serialized the loop. Per chunk: dispatch → await workers → main-thread `processImportsFromExtracted` + `processHeritageFromExtracted` + `processRoutesFromExtracted` + `synthesizeWildcardImportBindings` + `seedCrossFileReceiverTypes` → next chunk dispatch. Workers sat idle through every extraction block. U2 (revised from the plan's pipelined-chunks design) defers these passes to a single end-of-loop batch. Chunk loop becomes parse + merge + accumulate. Resolution sees strictly-more-info (full repo graph) so cross-chunk import/heritage targets resolve at least as well as before. Memory cost: `deferredWorkerImports` accumulates across chunks; bounded by total file count, acceptable. Plan deviation note: the plan called for an in-flight chunk pipeline (N concurrent dispatches with bounded memory). That design needed either a `processParsing` API refactor or duplicating its catch-block fallback in `parse-impl`. The deferred-extraction approach delivers the same "workers stay busy" outcome with much smaller surface area and zero changes to `processParsing`. The `GITNEXUS_PARSE_CHUNK_CONCURRENCY` env var documented in U2 of the plan is therefore not implemented in this commit; if memory growth from `deferredWorkerImports` becomes a problem at very-large-repo scale, a bounded sliding-window variant can land as a follow-up. Tests: - New `test/unit/analyze-worker-pool-size.test.ts` covers --workers validation (5 invalid inputs rejected with exit code 1 + clear error; valid integers set the env var; `--workers 0` routes to sequential). - Extended `worker-pool-resilience.test.ts` with `resolveAutoPoolSize` scenarios: env override, env=0, env above cap, invalid env fallback, auto-formula match, integer return type. - Full unit suite: 6097 / 6127 passed / 30 skipped / 0 failed. - Full integration suite (second run): 77 / 78 passed / 1 skipped / 0 failed. First run had a known cosmetic flake from an uncaught worker exception bleeding into the test reporter. Resilience contract from PR #1693 preserved: per-slot respawn budget, circuit breaker, quarantine, authoritative in-flight tracking, cumulative timeout budget — all unchanged. New env vars surfaced in --help: GITNEXUS_WORKER_POOL_SIZE, GITNEXUS_PARSE_CHUNK_CONCURRENCY (reserved for future bounded pipelining). * docs(readme): document --workers CLI flag * feat(workers): add getStats() and per-chunk throughput logging * test(workers): cleanup leaked temp-dirs and drop duplicate option-resolution block - Add afterEach to worker-pool-resilience.test.ts cleaning up the per-test temp directory created by beforeEach (~25 stale dirs per CI run previously). - Delete the duplicated describe('worker pool option resolution', ...) block. Verified the first block (lines 490-532) is a strict superset (includes the GITNEXUS_WORKER_MAX_CUMULATIVE_TIMEOUT_MS env test the second block omitted), so deletion loses no test coverage. Addresses PR #1693 review findings L2 (temp-dir leak) and L3 (duplicate block). * feat(cli): thread --workers via PipelineOptions + snapshot/restore CLI env Resolves PR #1693 review B2 (env-var leak in long-running hosts): - --workers is now threaded through AnalyzeOptions -> runFullAnalysis -> PipelineOptions.workerPoolSize -> createWorkerPool's explicit poolSize arg, bypassing the GITNEXUS_WORKER_POOL_SIZE env channel. The env var remains as a back-compat fallback inside resolveAutoPoolSize for operators who set it directly. - analyzeCommand and wikiCommand snapshot the GITNEXUS_* env vars they mutate at function entry and restore them in finally. Inner Impl extraction keeps the diff surgical (no body re-indent). process.exit(0) on the CLI success path still terminates the process; restoration matters for programmatic callers (tests, long-running hosts) reaching early-return paths or the alreadyUpToDate fast path. - Tests updated to assert the new behavior: analyze-worker-pool-size.test.ts: workerPoolSize flows through runFullAnalysis options; env is not mutated; back-to-back calls see their own values, not the previous call's leak. analyze-worker-timeout.test.ts: env IS set during the runFullAnalysis call (captured via mockImplementation) and restored after, proving the timeout reaches downstream while the leak fix holds. - Also addresses L4: afterEach NODE_OPTIONS restore so back-to-back test runs don't accumulate --max-old-space-size=8192 tokens. Addresses PR #1693 review B2 (blocker) and L4 (test polish). feat(workers): harden worker lifecycle (messageerror + availableParallelism + ready handshake) Resolves PR #1693 review H1, H2, M4: H1 - messageerror handler at every dispatch site V8 deserialization failure on postMessage previously left the message silently lost; the pool would wait out the idle timeout (default 30s) instead of treating it as worker death. The dispatch loop now wires worker.once('messageerror', ...) alongside error/exit and routes through recoverAndResume so the existing per-slot respawn budget, in-flight file attribution, and circuit-breaker layers fire as designed. H2 - resolveAutoPoolSize uses os.availableParallelism() Mirrors the pattern at capabilities.ts:85 (defaultEmbeddingThreads). os.cpus().length returns the host CPU count, which over-sizes the pool on cgroup-limited containers, taskset-restricted runtimes, and CI runners with explicit CPU quotas. Falls back to os.cpus().length on Node < 18.14. M4 - worker-side ready handshake replaces online-trust parse-worker.ts now emits {type: 'ready'} after all top-of-script initialization completes, BEFORE the message handler is attached. The pool's renamed waitForWorkerReady listens for this message under a bounded WORKER_READY_TIMEOUT_MS (5s) budget instead of trusting Node's online event - which fires when the worker thread starts, BEFORE the script body runs, letting init crashes slip past pool startup. ready is added to WorkerOutgoingMessage with an exhaustiveness-checked no-op branch in the dispatch handler (defensive: the message is consumed by waitForWorkerReady before dispatch handlers attach). messageerror is wired into waitForWorkerReady the same way. Test scaffolding: - FakeWorker emits {type: 'ready'} in addition to 'online' so replacement workers in unit tests don't hit the 5s budget. - Integration test ad-hoc worker scripts go through a writeReadyWorker helper that prepends the ready handshake. Tests intending to script "crash BEFORE ready" can bypass the helper. 61/61 worker-pool unit tests pass; 28/28 integration tests pass. * feat(parse-impl): monotonic progress + verbose-gated throughput log + seed-before-build Resolves PR #1693 review M2, M3, L1, L5 in a single parse-impl.ts pass: M2 - Monotonic progress through deferred phase (no more "stuck at 82%") Previously the deferred resolution stages (imports, heritage, routes, calls) all emitted percent: 82 — the UI looked frozen for the duration of the deferred work, which on large repos is several seconds to minutes and visually identical to the hang PR #1693 set out to fix. Redistributed: parse phase: 20-70 (was 20-82) imports: 70-75 heritage: 75-80 routes: 80-85 calls: 85-95 Each deferred stage now advances through its own band via the existing per-batch progress callback. Skipped stages (zero deferred input) leave their band as a no-op jump - the next stage still starts at its own band, preserving strict monotonicity. The "no parseable files" early return now jumps to 95 (was 82), and the duplicate "Parsing N files..." announcement is suppressed when totalParseable === 0 to avoid a non-monotonic 95 -> 20 regression that pre-existed (uncovered by the new monotonic test). M3 - Throughput log gated on `--verbose`, not just NODE_ENV=development The per-chunk files/s log was gated on `isDev`, so operators running `gitnexus analyze --verbose` in a production install never saw it. Now fires when (isDev \|\| isVerboseIngestionEnabled()) — matches the documented promise that `--verbose` shows tuning observability. L1 - Typo rename: `chunkChunkStartMs` -> `chunkStartMs` L5 - `buildExportedTypeMapFromGraph` runs BEFORE `seedCrossFileReceiverTypes` Previously the seeding branch was reached with `exportedTypeMap.size === 0` in the worker path (the map was only built far below, AFTER the seeding branch), so the seed dead-coded itself silently and call resolution never got the cross-file receiver-type enrichment. Now the map is populated from the in-progress graph before the seed call; the post-parse builder remains as a defensive sequential-path fallback, guarded by `size === 0` so we don't pay the cost twice on the worker path. Net win: cross-file CALLS edges that previously had no receiver type now get enriched. New test: parse-impl-progress-monotonic.test.ts Asserts the emitted percent stream is strictly non-decreasing across the parse + deferred phases, and that the deferred band (>=70) is actually reached. Also pins the "no parseable files" path to exactly [95] so the 95 -> 20 regression we just fixed can't re-emerge. * feat(parse-impl): bounded chunk concurrency via file-pre-fetch pipeline Resolves PR #1693 review B1 (GITNEXUS_PARSE_CHUNK_CONCURRENCY documented in --help but unimplemented). The chunk loop now pre-fetches chunk file contents up to `parseChunkConcurrency` chunks ahead of the worker-dispatch cursor so disk I/O overlaps with worker compute. Worker dispatch itself stays serial because WorkerPool.dispatch is not reentrant — concurrent calls would race on the shared per-slot busy/in-flight state, regressing the hang/resilience work this PR is built on. The pre-fetch path is the honest interpretation of "concurrent in-flight parse chunks" that the help text advertises: I/O overlap, not parallel worker dispatch. Concurrency value resolution: 1. PipelineOptions.parseChunkConcurrency (threaded from CLI) 2. GITNEXUS_PARSE_CHUNK_CONCURRENCY env var 3. Default 2 (matches the help text) F4 (wildcard-synthesis ordering) is preserved: deferred-state aggregation runs in chunkIdx order because the for-loop iterates sequentially after awaiting each chunk's pre-fetched contents. Cross-chunk processors (processImportsFromExtracted, synthesizeWildcardImportBindings, etc.) still run only after all chunks complete — they see deterministic input regardless of file-read completion order. Concurrency=1 produces behavior identical to the pure-serial loop; that's the regression baseline. New test: parse-impl-chunk-concurrency.test.ts - Asserts graph output is identical (nodeCount + relationshipCount) between parseChunkConcurrency=1 and =2 — the critical correctness invariant. Exact .toBe(N) comparisons per DoD §2.7 (the second run's counts must equal the first run's exactly). - Pins specific fixture symbols (foo/bar/Baz) under both parseChunkConcurrency=1 and the env-fallback (3) path. - Env-fallback test confirms GITNEXUS_PARSE_CHUNK_CONCURRENCY is honored when the option is undefined. * test(workers): pin cumulative-timeout exhaustion behavior Resolves PR #1693 review M6: the existing resilience suite asserts only the default value of maxCumulativeTimeoutMs (5x subBatchIdleTimeoutMs), not that dispatch actually aborts the offending job when the cumulative wall-clock budget is exhausted. Without this test, a future refactor could remove the exhaustion branch in requeueAfterTimeout and the suite would stay green while the pool sat in retry loops for an hour on a real production stall. Scenario: subBatchIdleTimeoutMs = 100ms timeoutBackoffFactor = 10 maxCumulativeTimeoutMs = 300ms Single file, HangingWorker that never responds. First attempt times out at 100ms (cumulative=100). The next backoff (1000ms, cumulative 1100ms) exceeds the 300ms cap, so requeueAfterTimeout returns give-up on the first timeout retry and the file goes to the session quarantine. Asserts: - pool.getQuarantinedPaths() includes 'src/stuck.ts' after dispatch - if dispatch rejected, the error is a WorkerPoolDispatchError (the typed surface that routes to sequential fallback) Uses a local minimal HangingWorker double rather than the full action-scripted FakeWorker from worker-pool-resilience.test.ts — the inverse pattern (always hang) doesn't need the scripted-action machinery and keeps the test file focused on the one behavior. * docs(readme): add environment-variables reference table Resolves PR #1693 review L6: operator-facing env vars were either mentioned inline (GITNEXUS_WORKER_SUB_BATCH_TIMEOUT_MS) or only documented via `gitnexus --help`, with no single place to look up the full set. The new "Environment variables" subsection under the Quick Start CLI block lists every operator-facing knob with default, effect, and tuning guidance, matching the names in cli/index.ts addHelpText post-U2 / U1. Covers: GITNEXUS_WORKER_POOL_SIZE (--workers) GITNEXUS_PARSE_CHUNK_CONCURRENCY (newly real per U1) GITNEXUS_VERBOSE (--verbose) GITNEXUS_MAX_FILE_SIZE (--max-file-size) GITNEXUS_WORKER_SUB_BATCH_TIMEOUT_MS (--worker-timeout × 1000) GITNEXUS_WORKER_SUB_BATCH_MAX_BYTES GITNEXUS_CHUNK_BYTE_BUDGET GITNEXUS_NO_GITIGNORE GITNEXUS_SKIP_OPTIONAL_GRAMMARS CLI flag vs env-var precedence is stated explicitly (CLI > env > default) so operators running long-lived hosts (MCP server, eval-server) know which channel wins. * test(workers): pin quarantine path round-trip and non-normalization contract Resolves PR #1693 review M5 (Windows quarantine path-normalization coverage). worker-pool.ts quarantines paths via a Set<string> keyed by exact string equality. The existing suite never asserted this contract, which lets a future "helpfully normalizing" refactor on one side of the pipeline (caller, worker, or pool) silently break quarantine filtering on Windows. This file pins the contract from both directions: 1. Round-trip: a path the caller dispatches with backslashes (src\bad.ts) flows through starting-file -> death -> quarantine -> next-dispatch filter verbatim. The replacement worker never sees the re-dispatched bad path because the pool's pre-dispatch filter short-circuits it. 2. Non-normalization: quarantining src\poison.ts does NOT filter src/poison.ts. Whoever changes that contract has to update this test alongside (the load-bearing assertion catches accidental path.normalize() calls in the quarantine path). Runs on every platform — the path strings are test-injected, so the test exercises the same code path regardless of the host's path.sep. Used a self-contained FakeWorker that emits {type:'ready'} for U3's waitForWorkerReady handshake, so the test doesn't depend on the larger worker-pool-resilience.test.ts harness. * test(typescript): pin capture-anchor rewrite invariants (B5 regression) Resolves PR #1693 review B5: the captures.ts ancestor-walk rewrite (findSelfOrAncestorOfType[s] + pickFirstNode replacing the prior findNodeAtRange-from-root path) was semantically equivalent to its predecessor per Lane 4 of the production-readiness review, but the existing typescript-captures.test.ts didn't pin the specific sharp edges where an over-aggressive walk would silently break captures. This file does. Each test exercises a capture class whose anchor type is one the rewrite explicitly handles: - member call obj.foo() -> @reference.call.member (call_expression anchor walks to self) - dynamic import import("./helper") -> raw @import.dynamic gets decomposed by splitImportStatement into @import.statement with @import.kind=dynamic + @import.source stripped of quotes - JSX <Foo /> in .tsx -> @reference.call.free emitted (TSX query pattern, query.ts:899-905) but @declaration.parameter-count is NOT synthesized because findSelfOrAncestorOfType('call_expression') returns null on a jsx_self_closing_element anchor. Pre-rewrite the range lookup also returned null. Pinning this contract catches accidental "walk JSX -> outer call" refactors. - constructor `new Foo(1,2)` -> @reference.call.constructor (new_expression anchor walks to self) - named/namespace import + re-export -> @import.statement (one each) - class method override -> @declaration.method per class, no collapse - member read obj.foo (no call) -> @reference.read.member All assertions use exact .toBe(N) per DoD §2.7. * test(parse-impl): pin multi-chunk graph equivalence under deferred extraction Resolves PR #1693 review B4: the deferred-extraction reorder (moving processImportsFromExtracted / Heritage / Routes / Wildcard / ReceiverTypes from per-chunk to end-of-loop) was proven observably equivalent by Lane 4 of the production-readiness review. Until now, the existing suite never asserted cross-chunk graph equivalence, which lets a future refactor that accidentally tightens the per-chunk vs end-of-loop coupling silently break cross-chunk resolution. This test forces multi-chunk parsing on a small fixture by setting GITNEXUS_CHUNK_BYTE_BUDGET=64 BEFORE the parse-impl module loads (the budget is captured at module load via vi.resetModules — a future move to function-scope env reads is U14 in Phase 2). Then runs the same fixture under a 10MB budget (single chunk) and asserts the two graphs are byte-identical: same nodeCount, same relationshipCount, exact .toBe(N) per DoD §2.7. Fixture: 3-file class hierarchy with cross-file inheritance — Animal (a.ts) -> Dog extends Animal (b.ts) -> makeDog returns Dog (c.ts). Forces the resolver to chain imports + heritage across chunks. A second test pins specific symbol names (Animal, Dog, makeDog, speak, bark) in the multi-chunk graph so a regression in chunk-boundary resolution surfaces as a missing-symbol failure with a specific diagnostic instead of a bare count mismatch. * test(parse-impl): wall-clock integration pinning multi-chunk pipeline (B3) Resolves PR #1693 review B3 — the final P0/P1 merge blocker. With this test, all five doc-review blockers (B1-B5) are pinned by regression coverage. The PR's headline claim is "analyze no longer hangs on TS-root-shaped loads". The existing suite pins each resilience layer (worker-pool- resilience.test.ts), the deferred-extraction equivalence (U7), and the chunk-concurrency contract (U1). What was missing: a single end-to-end run that exercises the full chunked parse-and-resolve path on a multi-chunk fixture, BOUNDED by a wall-clock budget so a regression that re-introduces the hang fails this test loudly via timeout rather than slipping past as a count drift. Implementation: - 17-file synthetic fixture: 15 small modules (one function each), one "realistic dense" complex.ts (30 functions + class + interface), and an index.ts re-exporting them. Forces cross-chunk import chains. - GITNEXUS_CHUNK_BYTE_BUDGET=64 via vi.resetModules forces multi-chunk parsing on the small fixture. - Promise.race with 30s timeout: a hang fails as "exceeded WALL_CLOCK_BUDGET_MS — likely the hang B3 was meant to prevent", not as a bounds-only inequality (DoD §2.7 distinction — hang-detector via exception, not regression-mask via inequality). - Exact .toBe(true) assertions on specific expected symbols (fn0..fn14, Service, Config, configure, describe, complex0/15/29) so a silent mid-chunk crash that exits 0 without producing graph data also fails this test, not just the hang case. Scope: runs the sequential-fallback path (skipWorkers: true) because the full real-worker scenario requires a built dist/parse-worker.js and ~60s wall-clock per run — appropriate for a CI-integration job, not vitest. The load-bearing invariants pinned here catch the bulk of B3's concern; the dist-worker swap is a Phase 2 follow-up documented in the file header. * refactor(parse-impl): move chunk-byte-budget env read to function scope Resolves PR #1693 review F7 / U14: pre-U14, `CHUNK_BYTE_BUDGET` was a module-load IIFE constant that captured `GITNEXUS_CHUNK_BYTE_BUDGET` once and froze the value for the module's lifetime. That defeated per-call option threading (a future `PipelineOptions.chunkByteBudget` was silently no-op'd because the function body read the frozen module-level constant) AND forced tests to use `vi.resetModules` to vary chunk layout. The U7 deferred-extraction test and the U6 multi-chunk integration test both used the workaround. After this change: - `DEFAULT_CHUNK_BYTE_BUDGET = 2 * 1024 * 1024` stays as a module-level constant — purely a default, no env access. - `resolveChunkByteBudget(options)` runs per call: option wins, then env, then default. Same options-first/env-fallback/default pattern as resolveAutoPoolSize and the U1 parseChunkConcurrency resolver — keeps the ingestion code's configuration model uniform. - `PipelineOptions.chunkByteBudget?` added with documentation that threading through options lets long-running hosts (eval-server, MCP daemon) size per-call without leaking process.env state across analyze invocations. New test (parse-impl-env-reads.test.ts) pins all four behaviors: 1. option-first: option present + env present -> option wins 2. env-fallback: option absent + env present -> env wins 3. default-fallback: both absent -> 2 MB default 4. per-call: two back-to-back runs in the same vitest worker with different chunkByteBudget option values observe their OWN values, proving the module-load freeze is gone (no vi.resetModules in this test — that's the invariant being verified). All four assertions use exact `.toBe(N)` per DoD §2.7. The chunk count is observed by parsing the `Parsing chunk X/Y` progress message stream — a stable proxy that doesn't require exposing internal parse-impl counter state. Note: U7 and U6 tests still use `vi.resetModules` because they were written before this change. A follow-up cleanup could simplify those tests (drop the resetModules dance, pass chunkByteBudget via options), but they pass as-is so this commit doesn't touch them. * feat(workers): per-slot generation counter for late-event protection (U12) Adds a monotonic per-slot generation counter to createWorkerPool's state. Each successful worker replacement (replaceWorker) bumps the slot's counter exactly once — atomically with the workers[slotIndex] swap, so observers (getStats) see the new (worker, generation) pair consistently. Handler closures in the dispatch loop capture the slot's generation at attach time and short-circuit when they fire on a stale generation. In the current implementation, cleanup() synchronously removes listeners on a Worker instance the moment a death is observed, so no listener naturally fires on a stale generation — the guard is a defensive layer protecting against any future refactor that loosens cleanup() ordering or re-attaches handlers across the swap. The load-bearing observable is the slotGenerations[] array exposed via WorkerPoolStats so operators (and tests) can confirm a slot was actually replaced and not just the same worker recycled. Implementation: - const slotGenerations: number[] = new Array(size).fill(0) in createWorkerPool's per-pool state, alongside respawnCount and consecutiveFailuresPerSlot. - replaceWorker: slotGenerations[workerIndex]++ AFTER the workers[workerIndex] = replacement swap (only on the success branch — drop-slot paths leave the counter unchanged). - runWorker dispatch loop: const slotGen = slotGenerations[workerIndex] captured before handler attachment; every handler (handler / errorHandler / exitHandler / messageErrorHandler) starts with `if (slotGenerations[workerIndex] !== slotGen) return`. - WorkerPoolStats gains `readonly slotGenerations: readonly number[]`. - getStats() returns slotGenerations.slice() so callers can't mutate pool state by writing to the returned array. Two existing toEqual snapshots in worker-pool-resilience.test.ts extended with the new slotGenerations field (both expect all-zeros — neither test scenario triggers a respawn). New test file (worker-pool-slot-generation.test.ts, 4 tests): 1. Fresh pool: every slot at generation 0. 2. Successful crash + respawn: generation bumps to 1 exactly once. 3. Crash that drops the slot (maxRespawnsPerSlot:0): generation stays at 0 because no successful respawn happened. The dispatch rejection on breaker trip is the expected outcome here; the load-bearing assertion is the post-rejection stats. 4. Multi-slot independence: one slot crashing bumps only that slot's generation, not the other. Order-independent via sort() because the round-robin assignment isn't pinned by contract. All assertions exact .toEqual / .toBe per DoD §2.7. * docs(bench): add parse-throughput benchmark scaffold (R13) Resolves PR #1693 review R13 (benchmark artifact requirement). Creates `gitnexus/bench/parse-throughput.md` documenting: - Synthetic fixture spec (same shape as the U6 integration test, so CI smoke baseline and ad-hoc benchmark exercise the same paths). - What to measure (wall-clock, peak heap, chunk count, getStats snapshot) and the hardware-shape metadata to record alongside. - Harness recipe — vitest + env-var overrides to exercise sequential fallback vs worker-pool paths. - Latest-measurement table with placeholder rows for the three paths (sequential, workers+concurrency, workers single-threaded) and an explicit "Status: scaffold — fill in before merging" callout. The U6 test's observed ~6 s wall-clock is captured as a smoke-baseline. - Operator-tuning quick reference cross-linked to the README env-var section (U11) so the doc is actionable without re-reading the PR. - "What this benchmark does NOT measure" section explicitly scoping the artifact's limits (synthetic ≠ real-repo, throughput-only ≠ resilience-tested, Phase 3 IPC repack row reserved for U16-U17). Mitigates the doc-review SG5 "static doc drift" concern via: 1. Explicit "regenerate this file before merging" callout at the top. 2. Self-contained methodology so anyone can re-run the numbers. 3. Cross-links to the U6 integration test that already bounds the wall-clock as part of the CI suite — so "is it still completing?" is regression-tested even if the numbers in this doc drift. The standalone harness script (`bench/scripts/parse-throughput.ts`) remains a stretch goal per the original plan. The U6 vitest with verbose ingestion logs covers the primary observability gap until the standalone harness lands. * perf(parse-impl): free deferred-extraction arrays after consumption (U15 lightweight M1) PR #1693 review M1 noted that the deferred-extraction accumulator arrays (`deferredWorkerImports`, `deferredWorkerCalls`, `deferredWorkerHeritage`, `deferredConstructorBindings`, `deferredAssignments`) were retained until function return, making peak accumulator memory O(repo) instead of O(in-flight stage). This commit implements the LIGHTWEIGHT version: free each array immediately after its last consumer drains/reads it, dropping peak accumulator memory progressively through the deferred-extraction stages. The structural per-chunk streaming variant (the original U15 framing) is deliberately deferred — the doc-review's adversarial reviewer (A4) flagged it as defending unmeasured memory pressure, and the simpler array-clearing captures the bulk of the benefit without committing to a scheduling-strategy decision (microtask vs parallel extractor task vs worker-side) that profile data should inform. Clears added: 1. After `processImportsFromExtracted` (the sole consumer of `deferredWorkerImports`): clear the imports array before the heavier heritage/calls stages run. 2. After `buildHeritageMap` (the LAST consumer of the raw `deferredWorkerHeritage` records — processCallsFromExtracted reads from the derived `fullWorkerHeritageMap` instead): clear the heritage array before the call-resolution stage. 3. After `processAssignmentsFromExtracted` (the joint last consumer with processCallsFromExtracted for the calls/ bindings/assignments triple): clear all three before downstream graph-build / scope-resolution uses its own working memory. Arrays returned in the function result object (allFetchCalls, allExtractedRoutes, allDecoratorRoutes, allToolDefs, allORMQueries, allParsedFiles) intentionally stay live — downstream consumers need them. Graph-output equivalence is preserved (U7 multi-chunk equivalence test passes — the clears happen AFTER each array's last consumer has copied data into the graph or derived structures). * feat(workers): introduce protocol.ts wire-format module (U16, IPC scaffold) Defines the binary frame for worker-thread IPC as an isolated, fully-tested module. Production wiring is deferred to U17 — shipping the wire-format contract first de-risks the migration by establishing a single source of truth for the byte layout. Resolves the scaffold half of PR #1693 review R12. Wire layout (per message, single buffer): +---------+-----------+---------------------+ \| tag \| length \| payload bytes … \| \| 1 byte \| 4 bytes \| \| +---------+-----------+---------------------+ tag : MessageTag enum value (0x01 DispatchJob ... 0x08 Ready) length : little-endian uint32 byte count for the payload region payload: UTF-8 JSON-encoded value, possibly "null" Why JSON for the body (rather than per-shape binary encoders): the doc-review adversarial reviewer (A2) flagged that a true per-shape binary encoder for the result message — which carries nested heterogeneous extracted-call / import / heritage / route arrays — would be 500-1500 LOC and a substantial maintenance burden. The honest perf win the IPC repack targets is moving file CONTENTS via ArrayBuffer transferList (zero-copy ownership transfer for the largest single piece of state in any message). That win is captured by U17 layering transferList over the bulk file-content payload while keeping this module's framing for the surrounding metadata. If U18 benchmark data shows the JSON body is itself a bottleneck after U17 lands, a follow-up unit can swap to per-shape binary encoding behind the same encodeMessage / decodeMessage surface without changing the frame. API: - MessageTag (const object): stable byte tags 0x01..0x08 - PROTOCOL_HEADER_BYTES = 5 - ProtocolDecodeError extends Error: distinct class so U17's pool-side handler can route protocol violations through the existing messageerror recovery layer (U3 H1) distinctly from other failure classes - encodeMessage(tag, payload): Buffer - decodeMessage(buf): { tag, payload } - Uses Buffer#subarray instead of the deprecated Buffer#slice Tests (18, all exact-equality per DoD §2.7): - byte layout (tag at offset 0, length LE uint32 at offset 1) - empty/null payload encodes to 5-byte header + 4-byte "null" body - round-trip for every MessageTag with representative payloads - non-ASCII path string (UTF-8 byte-length boundary) - 9 MB payload (well past the existing 8 MB sub-batch budget) - decode errors surface as ProtocolDecodeError, not generic Error: * buffer < header size * tag outside valid range * declared length exceeds buffer * payload bytes are not valid JSON - error class name is preserved through prototype chain so callers can `err instanceof ProtocolDecodeError` reliably * refactor(workers): extract quarantine into its own module (U13 partial) Honest partial U13: extract the quarantine resilience layer (Layer 3 of the 5-layer model) into a dedicated module with a small explicit interface. The full 5-module split that the original plan named was flagged by doc-review A10 as abstraction-without-multi-consumer-demand ("Each has exactly one consumer: worker-pool.ts. None of these layers is imported elsewhere in the codebase pre-extraction, and the plan doesn't identify any future consumer.") This commit ships the smallest self-contained layer as a named module to validate the factory + interface pattern with minimal risk. The remaining four layers (respawn-budget, cumulative-timeout, circuit-breaker, slot-attribution) stay inline until a real second consumer emerges (e.g., a non-parse worker pool that reuses the same resilience layers). Module shape (`workers/quarantine.ts`, ~30 LOC): interface Quarantine { add(path: string): void; has(path: string): boolean; snapshot(): string[]; // defensive copy readonly size: number; // getter, reflects state at access time } function createQuarantine(): Quarantine Replaces in `worker-pool.ts`: - `const quarantined: Set<string> = new Set()` -> `createQuarantine()` - `quarantined.has(p)` -> `quarantine.has(p)` (2 sites) - `quarantined.add(p)` -> `quarantine.add(p)` (2 sites) - `quarantined.size` -> `quarantine.size` (2 sites) - `Array.from(quarantined)` -> `quarantine.snapshot()` (6 sites) Public worker-pool.ts API is unchanged — `getQuarantinedPaths()` still returns the same defensive `string[]` copy. The behavioral contract is preserved: paths are quarantined as opaque strings (the U9 / M5 non-normalization contract still holds — see the new dedicated test). Tests: - 8 isolated unit tests for the quarantine module — pins the interface contract (empty start, add/has/size, dedup on repeated add, no separator normalization, snapshot defensive copy + freshness, size-getter live behavior). - All 86 existing worker-pool tests pass unchanged — they exercise the quarantine through the pool and act as the regression net for behavior preservation. Why not the full 5-module extraction in this commit: doc-review A10's concern is real — a single-consumer abstraction adds module-boundary overhead (5 sets of imports, 5 dedicated test files, 5 interfaces to keep in sync with worker-pool) without any structural benefit until a second consumer materializes. Extracting one validates the pattern; the remaining four can be moved on demand. * feat(workers): wire protocol.ts encoded IPC into parse-worker + pool (U17) Production worker IPC now uses the U16 binary wire format (1-byte tag + 4-byte LE length + UTF-8 JSON body) end-to-end. The pool encodes every outgoing `sub-batch` / `flush` dispatch via `encodeMessage`; the worker decodes incoming frames via `decodeMessage` and encodes its `ready`, `starting-file`, `progress`, `sub-batch-done`, `result`, `warning`, and `error` outputs the same way. The load-bearing correctness fix is making `decodeMessage` accept `Uint8Array` rather than only `Buffer`: Node's `worker_threads` `postMessage` structured-clones the payload, which strips the `Buffer` prototype on the receive side. A frame sent as `Buffer` arrives as a plain `Uint8Array`, and `Buffer.isBuffer(raw)` returns false — so the first attempt at U17 (gating decode on `Buffer.isBuffer`) silently treated every incoming frame as POJO and the worker never responded. The fix adopts the underlying memory zero-copy via `Buffer.from(view.buffer, view.byteOffset, view.byteLength)` and uses `raw instanceof Uint8Array` at every call site (parse-worker decode, pool dispatch handler, pool ready-handshake handler, FakeWorker test mocks, and the integration-test worker preamble). The pool stays tolerant of POJO incoming so unit-test FakeWorkers don't need rewriting — only the new outgoing encoded dispatches require the test scaffolding to decode on receive, which the test FakeWorkers and the integration test's inline `parentPort.on` wrapper now do. The slot-drop integration test was rewritten from a shared-counter-file race (which pre-U17 timing happened to land on the assertion-friendly counter==2 endpoint, but post-U17 protocol decoding latency shifted to counter==1 and produced 3 quarantines instead of 2) to a deterministic path-based crash trigger: slot 0 crashes on a.ts, respawns, crashes on the requeued b.ts, slot is dropped after budget exhausted; slot 1 handles [c.ts, d.ts] normally. Outcome no longer depends on inter-worker file-write ordering. Protocol coverage adds two regression tests pinning the Uint8Array decode path: structured-clone-stripped frames decode identically to their Buffer originals, and Uint8Array views with non-zero byteOffset into a wider ArrayBuffer also decode correctly (catches `Buffer.from(uint8)` copying semantics if a future refactor loses the zero-copy adoption). All 94 worker-pool tests (9 files, unit + integration) pass; the full unit suite (6128 tests across 268 files) passes unchanged. * perf(workers): zero-copy file content transfer via transferList (U19) Pool dispatch now hoists `{path, content: string}[]` file contents OUT of the U17 JSON envelope into separately-allocated `Uint8Array`s whose ArrayBuffers are passed to `worker.postMessage`'s `transferList` for zero-copy ownership transfer. The envelope itself carries only lightweight metadata (`{path, byteLength}` per file) and is structure- cloned the same as before. What this saves vs U17 baseline: - JSON.stringify of file contents on main thread drops to zero — the envelope is now O(paths + sizes), not O(total bytes). For a 200- file sub-batch of 10 KB TS files, that's ~2 MB of escape processing per dispatch that disappears. JSON.stringify's per-character branch on quotes/backslashes/control chars is roughly 2x slower than UTF-8 transcode in TextEncoder, so the replacement is a CPU win even though it adds a single TextEncoder.encode per file. - Structured-clone memcpy of file contents drops to zero — the contents' backing ArrayBuffers are ownership-transferred, not copied into the worker's heap. The envelope's struct-clone cost is now proportional to metadata size only. - JSON.parse on worker thread likewise no longer scales with content size. Worker decodes each `Uint8Array` to string via `TextDecoder` lazily at the parse boundary — runs on the worker thread, parallel with continued main-thread work, vs U17's sequential JSON.parse blocking the worker before processBatch can start. Pipelining: TextEncoder.encode (main) and TextDecoder.decode (worker) can both run while the OTHER side is doing useful work. Under U17, struct-clone was a synchronous main-thread blocker. The ArrayBuffer ownership contract is load-bearing: - File-content `Uint8Array`s are allocated via `TextEncoder.encode`, NOT `Buffer.from(str, 'utf8')`. TextEncoder produces a dedicated ArrayBuffer per call; `Buffer.from(str)` carves from Node's shared `Buffer.poolSize` slab for small strings, so transferring one pool-backed Buffer's ArrayBuffer would detach every other Buffer that shares that slab — silent data corruption. - The envelope itself is NOT transferred. It MAY be pool-backed by `encodeMessage`, and at ~30-80 bytes/file the struct-clone cost is negligible. Not transferring avoids the same detach-collateral risk the contents path is careful to dodge. Detection is strict: every input element must have both `path: string` and `content: string`. A single non-conforming element disqualifies the whole batch from the transfer path and falls back to the legacy single-Uint8Array `encodeMessage` envelope. Safer than partial transfer (which would split a sub-batch into mixed-shape messages the worker can't reassemble). `parse-worker.ts` `decodeIncomingMessage` recognizes the hybrid `{envelope, contents}` shape, decodes the envelope, zips metadata positionally with the contents array, decodes UTF-8 → string per file, and hands the reassembled `ParseWorkerInput[]` to the existing `processBatch`. Identical downstream behavior to U17 — the IPC optimization is invisible above this line. Test scaffolding (3 FakeWorkers + 1 integration-test preamble) gain a `decodeDispatchedMessage` helper that tolerates BOTH shapes (legacy single-frame Uint8Array AND the new hybrid envelope+contents) so the in-process unit mocks keep their existing action-scripting API and the 9 ad-hoc integration test workers keep their `msg.type === 'sub-batch'` handlers unchanged. `buildDispatchMessage` is now exported from worker-pool.ts so its contract can be tested in isolation. A new `test/unit/worker-pool-transferlist.test.ts` pins: - hybrid shape produced for parse-worker inputs - transferList carries one ArrayBuffer per file in input order - envelope decodes to metadata only (no `content` field) - content bytes round-trip byte-for-byte through UTF-8 (ASCII, multi-byte, surrogate-pair emoji) - each content's ArrayBuffer is independently allocated (no pool sharing) — the load-bearing transfer-safety invariant - non-parse shapes, empty arrays, and mixed-conformance arrays all fall back to the legacy single-frame path All 271 test files (6166 unit + integration tests) pass. * fix(workers,tests,docs): apply ce-code-review findings (16 items) Walks the full set of findings from a multi-agent code review (11 reviewers, 1 maintainability dispatch lost to tool-permission denial) of the PR #1693 branch. All 16 actionable findings — 4 P1, 4 P2, 8 P3 — applied in a single pass against a consistent tree. Tests pass (269/269 unit files, 29/29 integration). P1 — bounds-only / disguised-bounds assertions across 4 test files (per user-memory DoD §2.7): - worker-pool.test.ts: 5 sites — `nodes.length > 0` dropped (redundant after `.toContain('validateInput')`); `files.length >= 4` pinned to `.toBe(7)` (mini-repo/src has exactly 7 .ts files); `results.length > 0` pinned to `.toHaveLength(1)` (default sub-batch absorbs all 7); `result.fileCount >= 0` pinned to `.toBe(1)` (empty file is still "processed"); `warnRecords.length > 0` replaced with content- predicate `/respawn\|dropping\|replacement\|did not report ready/` (catches silenced warnings); `fallbackExcludePaths.length > 0` pinned to exact `['one.ts', 'two.ts']` (deterministic given the single-slot pool + 2 items + per-item starting-file). - parse-impl-fallback.test.ts: 3 sites — `astCacheClearCalls >= 1` pinned to exact 4 (per-chunk × 2 + finally × 2); the two error-path delta checks pinned to exact +2 and +3 (verified empirically). - parse-impl-progress-monotonic.test.ts: `percents.length > 0` → `.not.toEqual([])`; per-element `Math.max(prev, cur)` tautology replaced with direct `if (cur < prev) throw`; final-percent `Math.min(last, 95)` tautology pinned to exact `.toBe(70)` (3-file skipWorkers fixture's deferred band lands at the band start). - parse-impl-large-fixture.test.ts: `Math.min(elapsedMs, BUDGET)` tautology removed; Promise.race rejection is the load-bearing wall-clock check. P1 — terminate() lacks `.catch` mask: - worker-pool.ts terminate() now matches the `.catch(() => undefined)` pattern used at every other internal terminate site. Prevents a hung/OOM worker's terminate rejection from masking the original pipeline error when called from parse-impl.ts's finally block, and guarantees `workers.length = 0` / `activeSlots.clear()` always run. P1 — hybrid envelope length-mismatch + null-payload silent data loss: - parse-worker.ts decodeIncomingMessage: explicit non-null-and-typed check before `.type` access (decodeMessage permits null payloads per encodeMessage contract); explicit length-equality assertion between `decoded.files` and `contents` before zipping. Without these, `TextDecoder.decode(undefined)` silently returns "" and produces empty-content graph nodes — a contract violation that used to be undetectable. Both throws route through the outer try/catch → worker `error` reply → pool's recoverAndResume. P1 — unsafe casts at the IPC boundary: - buildDispatchMessage now uses a properly-typed `isParseWorkerItemArray` type guard. The narrowed branch accesses `item.path` and `item.content` as statically-typed strings — a future rename of `ParseWorkerInput.content` would fail to compile inside the branch instead of silently mismatching at runtime. The remaining decodeMessage payload casts are bounded by the F3/F6 runtime guards. P2 — idle-timeout retry bypasses circuit breaker: - worker-pool.ts timeout-retry IIFE now increments `consecutiveFailuresPerSlot[workerIndex]` alongside `respawnCount`. A slot that consistently times out (vs crashes) now trips the per-slot breaker, instead of consuming its full respawn budget over potentially tens of minutes without the breaker firing. P2 — null/non-object worker message crashes pool handler: - Dispatch handler in worker-pool.ts now guards `null / non-object / no string type discriminant` before `msg.type` access and routes through recoverAndResume on violation. Previously a legitimate `null` payload would throw TypeError out of the EventEmitter listener → uncaughtException on main, crashing the analyze. P2 — workerPoolSize === 0 creates unusable pool: - parse-impl.ts now treats `workerPoolSize === 0` as `skipWorkers` at the gate. Matches the PipelineOptions docstring contract ("0 disables the pool entirely — equivalent to skipWorkers"); avoids constructing a pool that rejects every dispatch and logs "Worker pool parsing stopped" per chunk. P2 — encodeMessage 2-buffer allocation per frame: - protocol.ts encodeMessage coalesced to a single `Buffer.allocUnsafe + writeUInt8 + writeUInt32LE + buf.write (string, offset, 'utf8')`. Drops the intermediate `Buffer.from(JSON.stringify(...), 'utf8')` allocation + memcpy. Length pre-check via `Buffer.byteLength(string, 'utf8')` surfaces the uint32 cap before any allocation. P3 — slotGenerations made optional on WorkerPoolStats so external implementations of getStats() that predate U12 don't compile-break; in-repo callers already use optional chaining. P3 — buildDispatchMessage marked `@internal` so it isn't surfaced as public API by typedoc / api-extractor (it's a test-only export). P3 — verboseThroughputLog hoisted above the chunk loop (env vars can't change mid-run; one O(env-read) per analyze, not per chunk). P3 — corrected the messageerror routing comment in worker-pool.ts dispatch handler. `ProtocolDecodeError` is caught by the surrounding try/catch — distinct from `messageerror`, which fires for V8 structured-clone failures before the message body would reach the handler. P3 — initial pool spawn now uses a `Promise.allSettled` ready-handshake gate symmetric with `replaceWorker`. Dispatch awaits this gate before selecting slots, so an init-crashing initial worker is dropped from `activeSlots` and a downstream OOM/missing-native-binding failure surfaces in seconds (bounded by WORKER_READY_TIMEOUT_MS) rather than waiting for the first idle timeout (30s default). P3 — `GITNEXUS_WORKER_MAX_RESPAWNS_PER_SLOT`, `GITNEXUS_WORKER_MAX_CUMULATIVE_TIMEOUT_MS`, `GITNEXUS_WORKER_CONSECUTIVE_FAILURE_THRESHOLD` added to: - CLI `--help` text in src/cli/index.ts - Root README env-var table - gitnexus/README troubleshooting section (new "Worker pool resilience tuning" subsection) P3 — CLI `catch (e: any)` / `catch (err: any)` in analyze.ts replaced with `catch (err: unknown)` + narrowed access; matches modern TS best practice and the codebase pattern at other catch sites. P3 — `WorkerPoolStats.terminated: boolean` field added (optional, for backward compatibility). `terminate()` sets it true; `getStats()` surfaces it. Distinguishes graceful shutdown from a circuit-breaker trip in observability surfaces. Coverage / advisory items not addressed in this commit (kept in the report only): - maintainability reviewer failed (Read/Bash denied) — god-module audit on worker-pool.ts (~1400 LOC) carried as residual risk - quarantine case-sensitivity contract unpinned (adversarial #8) - WORKER_READY_TIMEOUT_MS env-configurability (adversarial #2) - chunk-byte-budget × parseChunkConcurrency memory multiplier doc (adversarial #5) - MCP discoverability gaps for env vars / verbose (agent-native W1/W2) - bench/parse-throughput.md scaffold-with-TBD-rows (PS RR-003) * fix(parsing): sequential gap-fill for worker-quarantined chunk files (U20.U1) When the worker pool's Layer 3 quarantine filters one or more files out of a chunk's dispatch, the worker results returned to processParsing are silently narrower than the input chunk. Without this reparse, the graph for this run would be missing every quarantined file's symbols/imports/calls/heritage with no failure signal. After the existing per-chunk quarantine log emits in processParsing's worker-path try-block, run processParsingSequential on JUST the quarantined-in-chunk files. The sequential path writes directly to the graph, so symbols for those files land alongside worker output for the surviving files. Mirrors the WorkerPoolDispatchError catch-block's processParsingSequential call shape — same signature, same args, same scopeTreeCache wiring. Emits a structured warn naming `reparsedPaths` so operators can observe the sequential fall-through. This fixes the in-run side of the corruption Codex's adversarial review of PR #1693 flagged. The cross-run side (chunk-cache poisoning) is closed by U20.U2 in a follow-up commit. References plan: docs/plans/2026-05-20-002-fix-chunk-cache-corruption-on-worker-quarantine-plan.md * fix(parse-impl): suppress chunk-cache write when any chunk file was quarantined (U20.U2) The chunk hash at parse-impl.ts:424-428 is computed from every file in the chunk. The worker pool's Layer 3 quarantine (worker-pool.ts createQuarantine) filters quarantined files out of dispatch, so `rawResults` reflects only the surviving files. Before this commit, the write at line 500-507 stored that partial result under the full-coverage chunk hash — and on the next analyze with unchanged content, the cache HIT branch (line 439-464) silently replayed the incomplete result. Symbols from the quarantined file were missing from the graph for as long as the cache survived. Codex's adversarial review of PR #1693 flagged this as a silent- corruption class because there's no failure signal: no warn log during the replay, no graph-equivalence check, no exit code change. The corruption only surfaces if an operator notices a missing symbol in `gitnexus_query` output. Guard the write with `chunkFiles.some(f => quarantineSet.has(f.path))`. When any chunk file is in the worker pool's cumulative quarantine snapshot, skip the `parseCache.entries.set` call. Emits a verbose- only info log so operators investigating "why aren't my chunks caching" have a diagnostic trail. Skipping the write means the next analyze gets a cache miss for this chunk and re-dispatches it. Quarantine is session-scoped (a fresh createWorkerPool starts with an empty quarantine), so the new pool gives the quarantined file another chance. If quarantine fires again, U20.U1's sequential gap-fill still produces a complete graph for that run; the cache stays empty for the chunk until a fully-clean dispatch lands. The cache-hit replay branch at parse-impl.ts:439-464 is unchanged. Its contract strengthens: "cache entries are complete" becomes true post-fix, but the replay code doesn't need to know that. Closes the cross-run side of the Codex finding. U20.U3 adds the regression test. References plan: docs/plans/2026-05-20-002-fix-chunk-cache-corruption-on-worker-quarantine-plan.md * test(parse-impl): integration regression for quarantine + chunk-cache (U20.U3) Pins the U20 fix end-to-end via REAL `worker_threads` + `createWorkerPool`. Mirrors the writeReadyWorker pattern from `test/integration/worker-pool.test.ts` — inline READY_PREAMBLE + custom test worker script that: 1. Decodes the U17/U19 IPC protocol (Buffer frame OR hybrid envelope/ contents shape) the same way the production parse-worker does. 2. Emits a `{type:'ready'}` handshake so the pool's `waitForWorkerReady` resolves promptly. 3. On a sub-batch containing `poison.ts`, emits starting-file + `process.exit(134)`. The pool attributes the death to `poison.ts` via the in-flight signal and adds it to the session-scoped quarantine. 4. On a sub-batch without poison, synthesizes a minimal valid `ParseWorkerResult` with one `Function` node per file (no tree-sitter dep in the test worker — the synthesized nodes give `mergeChunkResults` deterministic content for the graph). Assertions exercise both fix layers: - U1 (sequential gap-fill in processParsing): the graph contains a `Function` node named `poison` AFTER the run. The custom worker never emits anything for `poison.ts`, so the only path for that symbol to reach the graph is `processParsing`'s sequential reparse of the quarantined-in-chunk file using the real tree-sitter parser against the actual source. - U2 (cache-write suppression in runChunkedParseAndResolve): `parseCache.entries` does NOT contain the chunk hash after the run; `parseCache.usedKeys` DOES contain it (chunk processed, cache write specifically skipped). - Cross-run: a second pass over the same fixture with the same parseCache and a fresh worker pool re-dispatches the chunk (cache empty), the worker crashes again, sequential gap-fill runs again, and the cache stays empty. Pins the round-trip contract. Adds `workerUrlForTest?: URL` to PipelineOptions — same `@internal` test-only injection precedent as `workerThresholdsForTest` (already in PipelineOptions for thresholds). When set, parse-impl uses the provided URL instead of the src/ → dist/ resolution dance. Production call sites never set this field; the only consumer today is this integration test. Why integration over unit: - The fix lives at the boundary between parsing-processor.ts and parse-impl.ts under a real WorkerPool. Unit-mocking the worker-pool module bypasses the structured-clone boundary, the dispatch lifecycle, and the actual quarantine flow — it verifies the test setup rather than the contract. The real worker thread executing through the U17/U19 IPC protocol IS the load-bearing surface. - User-explicit preference (saved as feedback_integration_over_vimock.md memory). For worker-pool / parse-impl / IPC-touching code: write integration tests under test/integration/ using writeReadyWorker patterns; avoid vi.mock on worker-pool.js. Test wall-clock: under 2s; both `it` blocks together complete in ~1.8s under the existing CI conditions. References plan: docs/plans/2026-05-20-002-fix-chunk-cache-corruption-on-worker-quarantine-plan.md * refactor(parsing): remove sequential-parser fallback (U20 design pivot) The worker pool's resilience layers — respawn budget, circuit breaker, quarantine, slot-attribution, cumulative timeout — are now the SOLE contract for handling worker failures. Two sequential-reparse paths are removed from processParsing: 1. U20.U1 sequential gap-fill for quarantined chunk files (just added in commit `7dd489e9`, now reverted). The pre-emptive rescue would re-run processParsingSequential on the file that ALREADY killed a worker — which for the most common quarantine cause (tree-sitter native SIGSEGV on a pathological file) re-triggers the same native crash on the main thread, killing the entire analyze. The "rescue" turned silent missing-symbols into a louder analyze-wide crash. Drop the rescue; accept the per-run gap. 2. Pre-existing WorkerPoolDispatchError catch-block sequential fallback (in production since PR #1693's resilience layer landed). Same risk class — when the pool exhausts its respawn budget / trips the circuit breaker, the failing files are precisely the ones likely to crash a sequential parser too. The "graceful degradation" hid pool failures behind degraded-but- completing analyze runs, making operational issues harder to surface and diagnose. Drop the catch-block; WorkerPoolDispatchError propagates to the analyze entry point where the user sees a clear hard signal. What stays: - The `skipWorkers: true` / small-repo path that uses `processParsingSequential` as the EXPLICIT primary path (not a fallback). Caller-driven opt-out and tiny-repo perf optimization are different intents. - U2's chunk-cache write suppression in parse-impl.ts (commit `7c9c9556`). When quarantine fires, the chunk stays uncached so the next analyze with a fresh pool retries the file cleanly. That's the cross-run correctness Codex's adversarial review actually asked for. - The per-chunk quarantine warn log (parsing-processor.ts) — operators see which files were skipped, both immediately and across runs. What changed: - `processParsing` worker-path try-block: unwrapped. The `processParsingWithWorkers` call is now direct (no try/catch wrapping); errors propagate to the chunk-loop caller. - `parsing-worker-fallback.test.ts` rewritten: the previous 5 tests asserted graceful sequential-fallback behavior. Replaced with 3 tests pinning the new contract — raw Error propagates, WorkerPool- DispatchError propagates with fallbackExcludePaths intact, normal quarantine signal does NOT throw and surfaces via progress detail. - `parse-impl-quarantine-cache-skip.test.ts` (U20 integration test) updated: poison.ts is NOT in the post-run graph; surviving files are; chunk-cache stays empty; second pass re-dispatches and leaves cache empty. - Plan doc updated to mark R1 as dropped and explain the U20 pivot in the Summary. User decision: explicit directive ("let's remove the sequential fallback entirely we must rely on entirely that the parallel process is resilient enough to work itself through the code base"). The pool's resilience layers are designed for this — respawn budget, circuit breaker, quarantine, slot-generation, cumulative-timeout cap — and adding a layer below them was redundant insurance with real downside. Tests: 269/269 unit files (6135 tests) green. 31/31 worker-pool + parse-impl integration tests green. The 2 reported "errors" in the integration run are the pre-existing intentional-process.exit unhandled- exception leaks from test workers — unchanged by U20. References plan: docs/plans/2026-05-20-002-fix-chunk-cache-corruption-on-worker-quarantine-plan.md * fix(workers,tests,docs): address ce-ultrareview findings F1/F2/F3/F4 Multi-lane review run on the PR #1693 branch surfaced four addressable items beyond the blocking three. F1 (minor, CodeQL): unused `findMatch` helper in test/unit/scope-resolution/typescript/typescript-captures-anchor.test.ts:28 removed. `countMatchesTsx` flagged by the same CodeQL pass is a false positive — it's called at line 88 by the JSX-anchor regression tests so the rewrite case actually fires under TSX, not just TS. F2 (medium, docs): bench/parse-throughput.md retitled as "(scaffold)" with an explicit "no measurement data has been collected yet" note above the table. The self-contradictory "Regenerate this file before merging any PR that touches the ingestion pipeline" instruction is dropped — the file ships intentionally without numbers; the load-bearing perf-regression protection lives in test/integration/parse-impl-large-fixture.test.ts (U6, 30s Promise.race wall-clock budget). The Latest measurement section now preserves the ~6s sequential observation as a smoke reference, not as a regression target. F3 (low, API hygiene): `WorkerPoolDispatchError.fallbackExcludePaths` renamed to `quarantinedPaths`. The "fallback" terminology was load-bearing under the pre-U20 design when `processParsing`'s sequential-fallback catch-block consumed it to filter the fallback file list. After commit `be1f65c` removed that catch-block, no production code reads the field — but it stays populated by the pool because the snapshot is genuinely useful operator diagnostics when the breaker trips. The rename clarifies the field's actual semantics (here are the files the pool quarantined before it tripped) without changing wire behavior. Definition + the lone surviving in-pool comment reference + both test assertions updated. F4 (low → real fix, reliability): timeout-retry IIFE in worker-pool.ts now consults `consecutiveFailureThreshold` and trips the circuit breaker when the per-slot consecutive-failure count crosses it. Closes a gap left by ce-code-review's REL-02 patch — that fix added the `consecutiveFailuresPerSlot[workerIndex]++` increment in the timeout-retry path but did NOT add the corresponding threshold-check + tripBreaker call. Result: chronic pure-timeout deaths accumulated counts that never tripped the breaker until the slot also hit `respawnCount > maxRespawnsPerSlot`. Now timeouts and crashes are structurally treated the same way by the breaker, which is what the REL-02 increment was meant to enable. Test coverage: worker-pool-resilience.test.ts already exercises the breaker via the shared handleWorkerDeath path; this new branch traces the same trip semantics with a different entry point, so the breaker-tripped state is observable via the same `getStats().poolBroken` and `WorkerPoolDispatchError.quarantinedPaths` surface. Out of scope here (caller actions or future PRs): - F5 (info): cumulative-quarantine cache check is safe in practice because chunks are alphabetically deterministic; no action. - F6 (low): exit-code-0 quarantine exemption — pre-existing P2 residual, bounded by quarantine + respawn budget; deferred. - F7 (info): dispatch non-reentrancy contract documented but not enforced; no production caller violates it; deferred. - PR title `[WIP]` removal — happens on GitHub side. Tests: 274/274 test files (6185 passing, 30 skipped). The single "error" in the integration runner is the pre-existing intentional- process.exit unhandled-exception leak from the deliberate startup- crash test worker, unchanged by these fixes. * fix(workers): swap protocol body from JSON to V8 serialize/deserialize CI scope-parity tests on Ubuntu surfaced silent data loss in the worker IPC: `Phase 'scopeResolution' failed: scope.typeBindings is not iterable` (Python, Go) and `importerModule.typeBindings.has is not a function` (Python). Plus three #1066 large-file regression tests (Python / C# / TypeScript) failed because call relationships weren't resolving from the worker output. Root cause: U17 introduced `JSON.stringify`/`JSON.parse` as the protocol body codec. JSON has no representation for `Map`, `Set`, `Date`, `RegExp`, `BigInt`, `TypedArray`, `undefined` values, or circular refs — `JSON.stringify(someMap)` returns `"{}"`. Production scope-resolution code keys data structures on Maps throughout (`ParsedFile.scopes[].typeBindings: ReadonlyMap<string, TypeRef>`, plus `bindings`, `bySourceScope`, `byTargetDef`, the finalize-algorithm edge indexes, etc.). The JSON round-trip silently turned every Map into an empty object, manifesting downstream as iteration / `.has` calls failing on the decoded payload. Fix:* replace the JSON body with `node:v8`'s `serialize` / `deserialize`. That's the same structured-clone algorithm Node's `worker.postMessage` uses natively — bit-for-bit compatible with the pre-U17 implicit-clone path. Full type fidelity for Map, Set, Date, RegExp, BigInt, TypedArray, undefined values, and circular refs. No external dependency. A previous iteration of this fix attempted to bolt a Map/Set replacer+reviver onto the JSON path. Rejected in favor of V8 serialization because: - the JSON tag-marker approach requires per-type registration (Map, Set; then Date, RegExp, BigInt would each need their own sentinels); V8 handles them all uniformly - keys to JSON-encode would still need handling for nested types (and the marker approach doesn't survive nested Maps-in-Maps cleanly without recursive replacer logic) - V8 is faster than JSON for object-heavy payloads anyway (binary format, no string escaping pass) - the user-explicit ask was "a much more generic solution that will work for everything" — V8 serialization IS the generic solution Trade-offs documented in the module header: - body bytes are opaque (binary, not human-readable) — debugging requires `v8.deserialize` ad-hoc; protocol.test.ts exercises every supported MessageTag including the new type-fidelity cases as a regression net. - format is tied to the running Node major. Pool always spawns workers on the same Node instance the main thread runs, so this is moot in production. Would matter if frames ever persisted to disk (nothing does today). Protocol test file rewritten: - drops the JSON-specific byte-layout assertions (e.g. `body must equal "null" string`) — replaced with V8-derived expected lengths - adds a "structured-clone type fidelity" describe block that pins Map, nested Map, Set, Date, RegExp, BigInt, TypedArray, undefined values, and circular-ref round-trips. These are the load-bearing regression tests preventing a future "optimize" PR from quietly swapping V8 back to JSON. - the bad-body decode-error test now uses arbitrary non-V8 bytes instead of `{not-json}` — same intent. Integration test READY_PREAMBLEs (worker-pool.test.ts and parse-impl-quarantine-cache-skip.test.ts) update their inline decoders to use `v8.deserialize` matching the production codec. Both files have a standalone CJS worker preamble that can't import dist/protocol.js by relative path, so the V8 dependency is required via `node:v8` directly. Tests: 271/271 unit files (6163 tests + 30 skipped). 28/28 worker-pool integration. 3/3 parse-impl integration. 791/791 scope-parity tests (the four CI-failing files: python.test.ts, go.test.ts, typescript.test.ts, csharp.test.ts) all green again. References plan: docs/plans/2026-05-20-002-fix-chunk-cache-corruption-on-worker-quarantine-plan.md * refactor(workers): drop protocol.ts; use native postMessage + transferList The protocol.ts framing layer was redundant — Node's `worker.postMessage` already runs V8 structured-clone internally, the same algorithm that backed `v8.serialize`. Wrapping V8.serialize → Buffer → postMessage(struct-clone-Buffer) was a double-walk: one full structured-clone pass to produce the Buffer, then another pass when postMessage cloned that Buffer across threads. This commit cuts the wrapper layer; workers and pool exchange POJO directly via `worker.postMessage(value, transferList)`, with file-content `ArrayBuffer`s in `transferList` for zero-copy ownership transfer. What changes: - Deleted `src/core/ingestion/workers/protocol.ts` (~180 LOC) + `test/unit/workers/protocol.test.ts` (~250 LOC). The MessageTag enum / ProtocolDecodeError / encodeMessage / decodeMessage surface is gone. Tag-based routing is replaced by the `msg.type` discriminant that every receive site already checks. Protocol-decode errors map to Node's `messageerror` event (V8 deserialization failures during postMessage), which the pool already wires to `recoverAndResume`. - `worker-pool.ts`: `decodeIncomingWorkerMessage` removed; handlers receive POJO directly. `buildDispatchMessage` now returns `{message: {type:'sub-batch', files: [{path, content: Uint8Array}]}, transferList: ArrayBuffer[]}`. The Uint8Array-per-content allocation via `TextEncoder.encode` is preserved (it's the load-bearing transfer-safety contract that keeps content out of Node's shared `Buffer.poolSize` slab). Flush dispatch is now plain `worker.postMessage({type:'flush'})`. - `parse-worker.ts`: `decodeIncomingMessage` removed. The message handler receives POJO directly; the only conversion is `Uint8Array → string` for sub-batch file contents at the `decodeSubBatchFiles` boundary, before handing to `processBatch`. Outgoing messages are emitted as POJO via plain `parentPort.postMessage({type:'starting-file', ...})` etc. The `sharedHybridDecoder` is now `sharedContentDecoder` (same intent, clearer name for the simpler shape). - Test scaffolding: FakeWorkers in `worker-pool-resilience`, `worker-pool-windows-quarantine`, and `worker-pool-slot-generation` drop their `decodeMessage` import + `decodeDispatchedMessage` helper. The helpers stay (still convert `files[i].content` Uint8Array → string for test-action introspection) but no longer touch any protocol framing — just shape-check for sub-batch. - Integration READY_PREAMBLEs (worker-pool.test.ts and parse-impl-quarantine-cache-skip.test.ts): drop the inline v8.deserialize + envelope-unzip logic; the preamble is now just the ready handshake + a `parentPort.on` wrapper that converts `files[i].content` Uint8Array → string for the ad-hoc test worker scripts. - `worker-pool-transferlist.test.ts`: contract tests updated for the new buildDispatchMessage shape — no `envelope` field anymore; `message.files[i].content` is Uint8Array; transferList holds each content.buffer in input order. Pool-slab independence still pinned. What stays the same: - Zero-copy file-content transfer via transferList — every file's ArrayBuffer is ownership-transferred to the worker (no copy). - Full structured-clone type fidelity — Map / Set / Date / RegExp / BigInt / TypedArray / undefined / circular refs all preserved by Node's native postMessage. The V8 fix from commit 06f6957e is inherent in this path; there's no JSON layer to lose them. - TextEncoder-per-content allocation — keeps content buffers out of the shared `Buffer.poolSize` slab so transferring one cannot detach another. - The pool's resilience layers (respawn, breaker, quarantine, starting-file attribution, cumulative timeout, ready handshake, slot-generation guard) — unchanged. - U20 chunk-cache write suppression on quarantine — unchanged. Net: ~430 LOC removed (protocol.ts + tests + inline decoders + helpers), ~120 LOC simplified in worker-pool.ts and parse-worker.ts. One less serialization pass per message on the hot path. Tests: 270/270 unit files (6133 + 30 skipped). 822/822 integration tests including the four CI-failing scope-parity files (Python, Go, TypeScript, C#) — the V8-fidelity contract holds via native postMessage with no explicit serializer. The single "error" reported in worker-pool.test.ts is the pre-existing intentional process.exit unhandled-exception artifact from the deliberate startup-crash test, unchanged by this commit. * refactor(parse-worker): drop legacy single-message dispatch mode The `parentPort.on('message', ...)` handler had an `Array.isArray(msg)` branch left over from a pre-sub-batch dispatch shape — the pool used to send the items array directly, before the worker pool added sub-batching and the `{type:'sub-batch', files: ...}` envelope. No production caller has dispatched that shape since the sub-batching refactor landed; verified by grepping the repo for `postMessage([` patterns (zero matches). The `ParseWorkerInput[]` arm in the `WorkerIncomingMessage` discriminated union also blocked exhaustiveness narrowing — flagged by the kieran-typescript code review (RR-01) as "if a future unit removes the legacy array path, this arm should be dropped." Dropping it now. What changes: - Remove the `Array.isArray(msg)` branch from the message handler. - Drop `ParseWorkerInput[]` from the `WorkerIncomingMessage` union; it's now a clean `{type:'sub-batch'} \| {type:'flush'}` discriminated union, so the dispatch switch is exhaustive over `msg.type`. Tests: 71/71 worker-pool unit + integration tests green (resilience, slot-generation, windows-quarantine, transferlist, parsing-worker- fallback, worker-pool integration, parse-impl-quarantine-cache-skip). --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: Gergő Magyar <gergomagyar@icloud.com>		2026-05-20 20:39:35 +01:00
.claude/skills/gitnexus	feat(javascript): migrate JavaScript to scope-based resolution (RFC #909 Ring 3, issue #928 ) (#1640 )	2026-05-19 06:23:13 +01:00
.claude-plugin	fix(plugin): hook logic bug and version mismatch	2026-02-26 18:08:32 +05:30
.cursor	feat: add docker support (#848 )	2026-04-18 08:39:18 +01:00
.github	fix(ci): bump publish job to Node 24 for npm OIDC support (#1628 )	2026-05-16 09:18:08 +01:00
.history/gitnexus	fix(test): add --repo to CLI e2e tool tests for multi-repo environment	2026-03-18 08:12:25 +00:00
.husky	feat: configure prettier with pre-commit hook (#563 )	2026-03-28 14:58:04 +00:00
.sisyphus/drafts	fixed constructor to method relation not getting stored in kuzu issue	2026-01-26 22:58:15 +05:30
deploy/kubernetes	ci(docker): mirror signed images to Docker Hub alongside GHCR (#1029 )	2026-04-23 18:59:26 +01:00
docs	feat(ingestion): add thrift contracts impl (#1234 )	2026-05-06 09:19:10 +01:00
eslint-rules	fix(windows): 32767-char tree-sitter crash + VECTOR extension SIGSEGV (#1433 )	2026-05-10 16:00:36 +01:00
eval	fix(eval-server): localhost now doesn't normalize into IPv4 instead lets OS decide which to bind (#1722 )	2026-05-20 16:14:13 +01:00
gitnexus	fix(workers): resilient + zero-copy ingestion worker pool — prevent analyze hangs on TS-root-scale loads (#1693 )	2026-05-20 20:39:35 +01:00
gitnexus-claude-plugin	feat(wiki): added --lang <lang> flags to gitnexus wiki for multilanguage wiki generation support (#1613 )	2026-05-17 19:54:02 +01:00
gitnexus-cursor-integration	fix(hooks): cap concurrent augment subprocesses (#1486 ) (#1510 )	2026-05-13 08:56:27 +01:00
gitnexus-shared	feat(cpp): Expand type_traits constraint registry (#1648 )	2026-05-18 21:10:18 +01:00
gitnexus-test-setup	feat: merge gitnexus-mcp into gitnexus package - unified CLI+MCP	2026-02-04 01:12:41 +05:30
gitnexus-web	chore(deps)(deps): bump zod from 3.25.76 to 4.3.6 in /gitnexus-web (#1464 )	2026-05-19 06:56:04 +01:00
.cursorrules	docs: agent development framework, GitHub templates, eval refactor (#479 )	2026-03-25 06:48:41 +00:00
.dockerignore	fix(ci): Change docker base image from alpine to debian (#1014 )	2026-04-21 21:31:58 +01:00
.env.example	ci(docker): mirror signed images to Docker Hub alongside GHCR (#1029 )	2026-04-23 18:59:26 +01:00
.git-blame-ignore-revs	feat: configure eslint with unused import removal (#564 )	2026-03-28 15:28:09 +00:00
.gitattributes	feat: configure prettier with pre-commit hook (#563 )	2026-03-28 14:58:04 +00:00
.gitignore	fix(cli): `--skip-git` treats cwd as index root instead of walking up to parent git repo (#1245 )	2026-05-01 09:49:48 +01:00
.mcp.json	fix: use cross-platform npx command in .mcp.json	2026-02-22 17:46:04 +00:00
.prettierignore	chore(quality): exclude test/fixtures from CodeQL, ESLint, and Prettier (#1313 )	2026-05-04 09:35:34 +01:00
.prettierrc	feat: configure prettier with pre-commit hook (#563 )	2026-03-28 14:58:04 +00:00
.windsurfrules	resources implemented and agents.md and skills updated to use it	2026-02-05 05:13:48 +05:30
AGENTS.md	feat(javascript): migrate JavaScript to scope-based resolution (RFC #909 Ring 3, issue #928 ) (#1640 )	2026-05-19 06:23:13 +01:00
ARCHITECTURE.md	feat(ingestion): TypeScript registry-primary scope resolution (Ring 3) (#1050 )	2026-04-26 08:23:08 +01:00
CHANGELOG.md	chore: release v1.5.3 — update CHANGELOG and package-lock	2026-04-01 21:35:24 +05:30
CLAUDE.md	feat(javascript): migrate JavaScript to scope-based resolution (RFC #909 Ring 3, issue #928 ) (#1640 )	2026-05-19 06:23:13 +01:00
compound-engineering.local.md	feat: Phase 7 type resolution — return-aware loop inference & PHP class-property iterables (#341 )	2026-03-18 08:39:38 +00:00
CONTRIBUTING.md	refactor(ci): unify release pipeline under publish.yml (#1610 )	2026-05-16 07:46:56 +01:00
docker-compose.yaml	fix(docker): add dedicated health endpoint for container healthcheck (#1147 ) (#1355 )	2026-05-05 21:40:32 +01:00
docker-server.mjs	fix(server): close js/path-injection cluster — /api/file + docker-server.mjs (U2) (#1322 )	2026-05-04 12:28:02 +01:00
docker-server.test.mjs	fix(server): close js/path-injection cluster — /api/file + docker-server.mjs (U2) (#1322 )	2026-05-04 12:28:02 +01:00
Dockerfile.cli	fix(docker): symlink gitnexus binary onto $PATH in runtime image (#1551 )	2026-05-13 17:14:52 +01:00
Dockerfile.web	fix(security): Pin Docker Node base images, remove runtime package-manager CVE surface, verify Trivy on PRs, and harden Dependabot policy (#1455 )	2026-05-09 16:55:31 +01:00
DoD.md	docs: add DoD.md repo-wide Definition of Done (#1032 )	2026-04-23 08:00:37 +01:00
eslint.config.mjs	fix(windows): 32767-char tree-sitter crash + VECTOR extension SIGSEGV (#1433 )	2026-05-10 16:00:36 +01:00
GUARDRAILS.md	fix: shard parse cache persistence on large repos (#1580 )	2026-05-16 07:19:06 +01:00
LICENSE	docs: update license copyright holder	2026-02-03 22:54:01 +05:30
llms.txt	docs: agent development framework, GitHub templates, eval refactor (#479 )	2026-03-25 06:48:41 +00:00
MIGRATION.md	feat(mcp): rank context/impact disambiguation candidates and expose kind/file_path hints (#888 )	2026-04-18 12:52:42 +01:00
package-lock.json	feat: configure eslint with unused import removal (#564 )	2026-03-28 15:28:09 +00:00
package.json	feat(package): add gitnexus commands for analysis	2026-04-29 16:37:06 +03:00
README.md	fix(workers): resilient + zero-copy ingestion worker pool — prevent analyze hangs on TS-root-scale loads (#1693 )	2026-05-20 20:39:35 +01:00
RUNBOOK.md	docs: agent development framework, GitHub templates, eval refactor (#479 )	2026-03-25 06:48:41 +00:00
SECURITY.md	ci(security): add automated security and vulnerability scans (#1297 )	2026-05-04 08:21:53 +01:00
skills.mdm	FEAT: Added support for optional skill generation based on KuzuDB after initial repo analysis (npx gitnexus analyze --skills) (#171 )	2026-03-13 08:29:13 +00:00
swift-ingestion-gaps.md	docs: add macro declarations to Swift ingestion gaps	2026-03-23 14:21:32 +01:00
TESTING.md	docs: optimize context files for LLM accuracy and token efficiency (#857 )	2026-04-16 08:43:11 +01:00
type-resolution-roadmap.md	feat: implement cross-file binding propagation for multiple languages	2026-03-21 07:47:04 +00:00
type-resolution-system.md	feat(ingestion): TypeScript registry-primary scope resolution (Ring 3) (#1050 )	2026-04-26 08:23:08 +01:00

README.md

GitNexus

⚠️ Important Notice: GitNexus has NO official cryptocurrency, token, or coin. Any token/coin using the GitNexus name on Pump.fun or any other platform is not affiliated with, endorsed by, or created by this project or its maintainers. Do not purchase any cryptocurrency claiming association with GitNexus.

Join the official Discord to discuss ideas, issues etc!

Enterprise (SaaS & Self-hosted) - akonlabs.com

Building nervous system for agent context.

Indexes any codebase into a knowledge graph — every dependency, call chain, cluster, and execution flow — then exposes it through smart tools so AI agents never miss code.

https://github.com/user-attachments/assets/172685ba-8e54-4ea7-9ad1-e31a3398da72

Like DeepWiki, but deeper. DeepWiki helps you understand code. GitNexus lets you analyze it — because a knowledge graph tracks every relationship, not just descriptions.

TL;DR: The Web UI is a quick way to chat with any repo. The CLI + MCP is how you make your AI agent actually reliable — it gives Cursor, Claude Code, Codex, and friends a deep architectural view of your codebase so they stop missing dependencies, breaking call chains, and shipping blind edits. Even smaller models get full architectural clarity, making it compete with Goliath models.

Star History

Two Ways to Use GitNexus

	CLI + MCP	Web UI
What	Index repos locally, connect AI agents via MCP	Visual graph explorer + AI chat in browser
For	Daily development with Cursor, Claude Code, Codex, Windsurf, OpenCode	Quick exploration, demos, one-off analysis
Scale	Full repos, any size	Limited by browser memory (~5k files), or unlimited via backend mode
Install	`npm install -g gitnexus`	No install — gitnexus.vercel.app
Storage	LadybugDB native (fast, persistent)	LadybugDB WASM (in-memory, per session)
Parsing	Tree-sitter native bindings	Tree-sitter WASM
Privacy	Everything local, no network	Everything in-browser, no server

Bridge mode: gitnexus serve connects the two — the web UI auto-detects the local server and can browse all your CLI-indexed repos without re-uploading or re-indexing.

Enterprise

GitNexus is available as an enterprise offering - either as a fully managed SaaS or a self-hosted deployment. Also available for commercial use of the OSS version with proper licensing.

Enterprise includes:

PR Review - automated blast radius analysis on pull requests
Auto-updating Code Wiki - always up-to-date documentation (Code Wiki is also available in OSS)
Auto-reindexing - knowledge graph stays fresh automatically
Multi-repo support - unified graph across repositories
OCaml support - additional language coverage
Priority feature/language support - request new languages or features

Upcoming:

Auto regression forensics
End-to-end test generation

👉 Learn more at akonlabs.com

💬 For commercial licensing or enterprise inquiries, ping us on Discord or drop an email at founders@akonlabs.com

Development

ARCHITECTURE.md — packages, index → graph → MCP flow, where to change code
RUNBOOK.md — analyze, embeddings, stale index, MCP recovery, CI snippets
GUARDRAILS.md — safety rules and operational “Signs” for contributors and agents
CONTRIBUTING.md — license, setup, commits, and pull requests
TESTING.md — test commands for gitnexus and gitnexus-web

CLI + MCP (recommended)

The CLI indexes your repository and runs an MCP server that gives AI agents deep codebase awareness.

Quick Start

# Index your repo (run from repo root)
npx gitnexus analyze

That's it. This indexes the codebase, installs agent skills, registers Claude Code hooks, and creates AGENTS.md / CLAUDE.md context files — all in one command.

To configure MCP for your editor, run npx gitnexus setup once — or set it up manually below.

Faster install (no C++ toolchain needed): set GITNEXUS_SKIP_OPTIONAL_GRAMMARS=1 before npm install -g gitnexus to skip the native tree-sitter-dart and tree-sitter-proto builds. Dart/Proto files won't be parsed, but install completes in seconds without python3/make/g++. Strict =1 only — any other value falls through to the rebuild.

MCP Setup

gitnexus setup auto-detects your editors and writes the correct global MCP config. You only need to run it once.

Editor Support

Editor	MCP	Skills	Hooks (auto-augment)	Support
Claude Code	Yes	Yes	Yes (PreToolUse + PostToolUse)	Full
Cursor	Yes	Yes	Yes (postToolUse, manual install)	Full
Codex	Yes	Yes	—	MCP + Skills
Windsurf	Yes	—	—	MCP
OpenCode	Yes	Yes	—	MCP + Skills

Claude Code gets the deepest integration: MCP tools + agent skills + PreToolUse hooks that enrich searches with graph context + PostToolUse hooks that detect a stale index after commits and prompt the agent to reindex.

Community Integrations

Built by the community — not officially maintained, but worth checking out.

Project	Author	Description
pi-gitnexus	@tintinweb	GitNexus plugin for pi — `pi install npm:pi-gitnexus`
gitnexus-stable-ops	@ShunsukeHayashi	Stable ops & deployment workflows (Miyabi ecosystem)

Have a project built on GitNexus? Open a PR to add it here!

If you prefer manual configuration:

Recommended for fastest startup: install gitnexus globally (npm i -g gitnexus) and run gitnexus setup — this writes an absolute-path MCP config that bypasses npx entirely. The pinned-npx snippets below are a quickstart fallback; on a cold cache the npx install can exceed Claude Code's MCP_TIMEOUT default (~30s).

Claude Code (full support — MCP + skills + hooks):

# macOS / Linux
claude mcp add gitnexus -- npx -y gitnexus@latest mcp

# Windows
claude mcp add gitnexus -- cmd /c npx -y gitnexus@latest mcp

Codex (full support — MCP + skills):

codex mcp add gitnexus -- npx -y gitnexus@latest mcp

Cursor (~/.cursor/mcp.json — global, works for all projects):

{
  "mcpServers": {
    "gitnexus": {
      "command": "npx",
      "args": ["-y", "gitnexus@latest", "mcp"]
    }
  }
}

OpenCode (~/.config/opencode/config.json):

{
  "mcp": {
    "gitnexus": {
      "type": "local",
      "command": ["gitnexus", "mcp"]
    }
  }
}

Codex (~/.codex/config.toml for system scope, or .codex/config.toml for project scope):

[mcp_servers.gitnexus]
command = "npx"
args = ["-y", "gitnexus@latest", "mcp"]

CLI Commands

gitnexus setup                   # Configure MCP for your editors (one-time)
gitnexus analyze [path]          # Index a repository (or update stale index)
gitnexus analyze --repair-fts    # Fast path: rebuild/verify only FTS indexes on existing index data
gitnexus analyze --force         # Full rebuild: re-parse + graph rebuild + FTS rebuild
gitnexus analyze --skills        # Generate repo-specific skill files from detected communities
gitnexus analyze --skip-embeddings  # Skip embedding generation (faster)
gitnexus analyze --skip-agents-md  # Preserve custom AGENTS.md/CLAUDE.md gitnexus section edits
gitnexus analyze --skip-git        # Index folders that are not Git repositories
gitnexus analyze --embeddings    # Enable embedding generation (slower, better search)
gitnexus analyze --verbose       # Log skipped files when parsers are unavailable
gitnexus analyze --worker-timeout 60  # Increase worker idle timeout for slow parses
gitnexus analyze --workers <n>        # Parse worker pool size (default: cores-1, capped at 16; 0 = sequential)
gitnexus mcp                     # Start MCP server (stdio) — serves all indexed repos
gitnexus serve                   # Start local HTTP server (multi-repo) for web UI connection
gitnexus list                    # List all indexed repositories
gitnexus status                  # Show index status for current repo
gitnexus clean                   # Delete index for current repo
gitnexus clean --all --force     # Delete all indexes
gitnexus wiki [path]             # Generate repository wiki from knowledge graph
gitnexus wiki --model <model>    # Wiki with custom LLM model (default: gpt-4o-mini)
gitnexus wiki --base-url <url>   # Wiki with custom LLM API base URL
gitnexus publish                 # Notify the understand-quickly registry (opt-in, see below)

# Repository groups (multi-repo / monorepo service tracking)
gitnexus group create <name>                                   # Create a repository group
gitnexus group add <group> <groupPath> <registryName>          # Add a repo to a group. <groupPath> is a hierarchy path (e.g. hr/hiring/backend); <registryName> is the repo's name from the registry (see `gitnexus list`)
gitnexus group remove <group> <groupPath>                      # Remove a repo from a group by its hierarchy path
gitnexus group list [name]                                     # List groups, or show one group's config
gitnexus group sync <name>                                     # Extract contracts and match across repos/services
gitnexus group contracts <name>  # Inspect extracted contracts and cross-links
gitnexus group query <name> <q>  # Search execution flows across all repos in a group
gitnexus group status <name>     # Check staleness of repos in a group

If analyze reports a worker parse timeout on a large or unusual repository, it keeps running and falls back safely. To give slow worker jobs more time, use gitnexus analyze --worker-timeout 60 or set GITNEXUS_WORKER_SUB_BATCH_TIMEOUT_MS=60000. For very large files, GITNEXUS_WORKER_SUB_BATCH_MAX_BYTES controls the worker job byte budget.

Environment variables

Most analyze knobs are also CLI flags (--workers, --worker-timeout, --max-file-size, --verbose). Use the env-var form when you'd otherwise repeat the same flag every run, or when invoking GitNexus from a long-running host (MCP server, eval-server, CI shell) that already manages its own environment. CLI flags take precedence over env vars; env vars take precedence over built-in defaults.

Variable	Default	Effect	Tune when…
`GITNEXUS_WORKER_POOL_SIZE`	`cores - 1`, capped at 16	Parse worker pool size. `0` disables the pool (sequential fallback). Equivalent to `--workers <n>`.	Constrained containers (cgroup CPU limits), CI runners with explicit quotas, or debugging a worker-only crash via `0`.
`GITNEXUS_PARSE_CHUNK_CONCURRENCY`	`2`	Number of chunks whose file contents may be read into memory in parallel while the pool dispatches the current chunk. Worker dispatch itself stays serial.	Repos large enough to chunk (multi-MB total source) where disk I/O is a measurable fraction of analyze wall-clock.
`GITNEXUS_VERBOSE`	unset	When `1`, enables verbose ingestion logs (skipped-file warnings, per-chunk throughput, parse-cache stats). Equivalent to `--verbose`.	Debugging an analyze that "completed" but seems to have missed files; tuning `--workers` / chunk concurrency against observable throughput.
`GITNEXUS_MAX_FILE_SIZE`	`512` (KB)	Walker skip threshold in KB. Hard cap is `32768` (tree-sitter buffer ceiling). Equivalent to `--max-file-size <kb>`.	Indexing repos with intentionally-large source files (generated parsers, vendored bundles) that should still be parsed.
`GITNEXUS_WORKER_SUB_BATCH_TIMEOUT_MS`	`30000`	Worker idle timeout in milliseconds before retry/fallback. Equivalent to `--worker-timeout <seconds>` × 1000.	Slow-parsing files (large minified JS, deeply-nested TS types) that legitimately need more than 30s.
`GITNEXUS_WORKER_SUB_BATCH_MAX_BYTES`	`8388608` (8 MB)	Per-job byte budget the pool will send to a worker in one `postMessage`.	Very large individual files; mostly diagnostic — bumping past 8 MB risks structured-clone memory pressure.
`GITNEXUS_WORKER_MAX_RESPAWNS_PER_SLOT`	`3`	Max replacement spawns per worker slot before the slot is dropped from the active rotation. Bounds respawn loops on a chronically-crashing slot.	Hosts where a flaky worker should retry more (raise) or fail-fast (lower) before the slot is dropped.
`GITNEXUS_WORKER_MAX_CUMULATIVE_TIMEOUT_MS`	`5 × subBatchTimeoutMs`	Total retry wall-time budget per job before quarantining. Combined with `timeoutBackoffFactor`, prevents exponentially-growing retries from stalling for hours.	Slow files that legitimately need long total retry windows; lower to fail-fast on stalls.
`GITNEXUS_WORKER_CONSECUTIVE_FAILURE_THRESHOLD`	`max(3, poolSize)`	Per-slot consecutive deaths before the pool's circuit breaker trips. After tripping, every subsequent dispatch rejects until a fresh pool is created.	Hosts where a SIGSEGV-prone native grammar should trip the breaker sooner; CI runners that should fail loudly.
`GITNEXUS_CHUNK_BYTE_BUDGET`	`2097152` (2 MB)	Chunk boundary used for cache-key composition and dispatch. Smaller = finer-grained cache hits but more dispatch overhead.	Tuning incremental-analyze cache behavior on monorepos.
`GITNEXUS_NO_GITIGNORE`	unset	When set, skips `.gitignore` parsing. `.gitnexusignore` is still honored.	Indexing a repo whose `.gitignore` excludes files you actually want indexed (e.g., generated code committed for cross-repo lookup).
`GITNEXUS_SKIP_OPTIONAL_GRAMMARS`	unset	When `=1` strictly, skips native builds for `tree-sitter-dart` / `tree-sitter-proto` at install time.	Installing on a host without a C++ toolchain; you're willing to skip Dart/Proto parsing.

Publishing to understand-quickly (opt-in)

looptech-ai/understand-quickly is a public registry of code-knowledge graphs that lists gitnexus@1 as a first-class format. After registering your repo once (npx @understand-quickly/cli add or the wizard), gitnexus publish fires a single repository_dispatch event so the registry resyncs your entry on demand instead of waiting for the nightly job.

It is opt-in and a no-op without UNDERSTAND_QUICKLY_TOKEN — a fine-grained GitHub PAT with Repository dispatches: write on the registry repo. Nothing else happens; no graph file is uploaded. See the protocol spec for the full contract.

What Your AI Agent Gets

16 tools exposed via MCP (11 per-repo + 5 group):

Tool	What It Does	`repo` Param
`list_repos`	Discover all indexed repositories	—
`query`	Process-grouped hybrid search (BM25 + semantic + RRF)	Optional
`context`	360-degree symbol view — categorized refs, process participation	Optional
`impact`	Blast radius analysis with depth grouping and confidence	Optional
`detect_changes`	Git-diff impact — maps changed lines to affected processes	Optional
`rename`	Multi-file coordinated rename with graph + text search	Optional
`cypher`	Raw Cypher graph queries	Optional
`group_list`	List configured repository groups	—
`group_sync`	Extract contracts and match across repos/services	—
`group_contracts`	Inspect extracted contracts and cross-links	—
`group_query`	Search execution flows across all repos in a group	—
`group_status`	Check staleness of repos in a group	—

When only one repo is indexed, the repo parameter is optional. With multiple repos, specify which one: query({query: "auth", repo: "my-app"}).

Resources for instant context:

Resource	Purpose
`gitnexus://repos`	List all indexed repositories (read this first)
`gitnexus://repo/{name}/context`	Codebase stats, staleness check, and available tools
`gitnexus://repo/{name}/clusters`	All functional clusters with cohesion scores
`gitnexus://repo/{name}/cluster/{name}`	Cluster members and details
`gitnexus://repo/{name}/processes`	All execution flows
`gitnexus://repo/{name}/process/{name}`	Full process trace with steps
`gitnexus://repo/{name}/schema`	Graph schema for Cypher queries

2 MCP prompts for guided workflows:

Prompt	What It Does
`detect_impact`	Pre-commit change analysis — scope, affected processes, risk level
`generate_map`	Architecture documentation from the knowledge graph with mermaid diagrams

4 agent skills installed to .claude/skills/ automatically:

Exploring — Navigate unfamiliar code using the knowledge graph
Debugging — Trace bugs through call chains
Impact Analysis — Analyze blast radius before changes
Refactoring — Plan safe refactors using dependency mapping

Repo-specific skills generated with --skills:

When you run gitnexus analyze --skills, GitNexus detects the functional areas of your codebase (via Leiden community detection) and generates a SKILL.md file for each one under .claude/skills/generated/. Each skill describes a module's key files, entry points, execution flows, and cross-area connections — so your AI agent gets targeted context for the exact area of code you're working in. Skills are regenerated on each --skills run to stay current with the codebase.

Multi-Repo MCP Architecture

GitNexus uses a global registry so one MCP server can serve multiple indexed repos. No per-project MCP config needed — set it up once and it works everywhere.

flowchart TD
    subgraph CLI [CLI Commands]
        Setup["gitnexus setup"]
        Analyze["gitnexus analyze"]
        Clean["gitnexus clean"]
        List["gitnexus list"]
    end

    subgraph Registry ["~/.gitnexus/"]
        RegFile["registry.json"]
    end

    subgraph Repos [Project Repos]
        RepoA[".gitnexus/ in repo A"]
        RepoB[".gitnexus/ in repo B"]
    end

    subgraph MCP [MCP Server]
        Server["server.ts"]
        Backend["LocalBackend"]
        Pool["Connection Pool"]
        ConnA["LadybugDB conn A"]
        ConnB["LadybugDB conn B"]
    end

    Setup -->|"writes global MCP config"| CursorConfig["~/.cursor/mcp.json"]
    Analyze -->|"registers repo"| RegFile
    Analyze -->|"stores index"| RepoA
    Clean -->|"unregisters repo"| RegFile
    List -->|"reads"| RegFile
    Server -->|"reads registry"| RegFile
    Server --> Backend
    Backend --> Pool
    Pool -->|"lazy open"| ConnA
    Pool -->|"lazy open"| ConnB
    ConnA -->|"queries"| RepoA
    ConnB -->|"queries"| RepoB

How it works: Each gitnexus analyze stores the index in .gitnexus/ inside the repo (portable, gitignored) and registers a pointer in ~/.gitnexus/registry.json. When an AI agent starts, the MCP server reads the registry and can serve any indexed repo. LadybugDB connections are opened lazily on first query and evicted after 5 minutes of inactivity (max 5 concurrent). If only one repo is indexed, the repo parameter is optional on all tools — agents don't need to change anything.

Web UI (browser-based)

A client-side graph explorer and AI chat — your code never leaves your machine.

Try it now: gitnexus.vercel.app — run npx gitnexus@latest serve locally and the page auto-connects to your local backend.

Or run the frontend locally:

git clone https://github.com/abhigyanpatwari/gitnexus.git
cd gitnexus/gitnexus-shared && npm install && npm run build
cd ../gitnexus-web && npm install
npm run dev
# Then in another terminal, start the backend the frontend connects to:
npx gitnexus@latest serve

Docker

The official Docker setup ships two signed images orchestrated by docker-compose.yaml. Each image is published to both GitHub Container Registry (GHCR) and Docker Hub — same build, same digest, same Cosign signature — so pick whichever registry you prefer:

Purpose	GHCR (default in `docker-compose.yaml`)	Docker Hub mirror
CLI / `gitnexus serve` backend (HTTP API on port `4747`, MCP, indexer)	`ghcr.io/abhigyanpatwari/gitnexus:latest`	`akonlabs/gitnexus:latest`
Static web UI (port `4173`)	`ghcr.io/abhigyanpatwari/gitnexus-web:latest`	`akonlabs/gitnexus-web:latest`

Heads-up — image rename. Earlier releases published the web UI under ghcr.io/abhigyanpatwari/gitnexus. Starting with the introduction of the bundled backend, that slug now hosts the CLI/server image and the UI moved to ghcr.io/abhigyanpatwari/gitnexus-web. The previous tags remain available for pulling, but new versions are only published under the new slugs. Update your docker run / compose files accordingly (or just adopt the bundled compose).

One-command setup

docker compose up -d

This starts the server on http://localhost:4747 and the web UI on http://localhost:4173. The UI auto-detects the server because the browser runs on the host and reaches the container via the mapped port.

A named volume (gitnexus-data) persists the global registry, indexes, and cloned repos at /data/gitnexus inside the server container. To make repos on your host machine indexable, set WORKSPACE_DIR before bringing the stack up:

WORKSPACE_DIR=$HOME/code docker compose up -d
# Inside the server container the directory is mounted read-only at /workspace.
docker compose exec gitnexus-server gitnexus index /workspace/my-repo

Direct `docker run`

# Server
docker run --rm -d \
  --name gitnexus-server \
  -p 4747:4747 \
  -v gitnexus-data:/data/gitnexus \
  ghcr.io/abhigyanpatwari/gitnexus:latest

# Web UI
docker run --rm -d \
  --name gitnexus-web \
  -p 4173:4173 \
  ghcr.io/abhigyanpatwari/gitnexus-web:latest

Optional env file (override image tags, container names, ports, workspace dir):

cp .env.example .env
docker compose --env-file .env up -d

Versioning & supply-chain protection

The Docker images are version-locked to the npm package:

Stable images are only published from vX.Y.Z git tags (via docker.yml triggered directly by the tag push), and the workflow refuses to build unless the tag exactly matches gitnexus/package.json's version. So ghcr.io/abhigyanpatwari/gitnexus:1.6.2 (and its Docker Hub mirror akonlabs/gitnexus:1.6.2) is byte-for-byte the same release as npm install gitnexus@1.6.2 — no drift, no floating builds from main. Both registries receive the same digest from a single build step, so you can pull from either and the signature verifies identically.
Release-candidate images (e.g. :1.7.0-rc.1) are published alongside each RC npm release. They are built by publish.yml calling docker.yml as a reusable workflow after the RC tag is created and pushed.
:latest is auto-promoted only from non-prerelease tags by the Docker metadata action, so it always points at a real, npm-published version.

Both images are signed with Cosign keyless signing using the workflow's GitHub OIDC identity, and shipped with build provenance and SBOM attestations. This is your protection against supply-chain attacks: even if an attacker republishes a same-named image elsewhere (or somehow pushes to a typo-squatted registry), they cannot forge a Cosign signature tied to abhigyanpatwari/GitNexus's docker.yml. Always verify before pulling into sensitive environments:

Stable releases — signed from the v* tag ref:

cosign verify ghcr.io/abhigyanpatwari/gitnexus:1.6.2 \
  --certificate-identity-regexp '^https://github\.com/abhigyanpatwari/GitNexus/\.github/workflows/docker\.yml@refs/tags/v[0-9]+\.[0-9]+\.[0-9]+(-[a-zA-Z0-9.]+)?$' \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com

# Same signature verifies the Docker Hub mirror (identical digest):
cosign verify docker.io/akonlabs/gitnexus:1.6.2 \
  --certificate-identity-regexp '^https://github\.com/abhigyanpatwari/GitNexus/\.github/workflows/docker\.yml@refs/tags/v[0-9]+\.[0-9]+\.[0-9]+(-[a-zA-Z0-9.]+)?$' \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com

The regex pins the certificate identity to this repo's docker.yml workflow run from a v* tag — rejecting unsigned images, images signed by other workflows, and images signed from unprotected refs. It is identical for both registries because both sets of tags were signed at the same digest in one workflow run.

Release candidates — signed from refs/heads/main (the caller's ref when publish.yml invokes docker.yml as a reusable workflow):

cosign verify ghcr.io/abhigyanpatwari/gitnexus:1.7.0-rc.1 \
  --certificate-identity 'https://github.com/abhigyanpatwari/GitNexus/.github/workflows/docker.yml@refs/heads/main' \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com

You can also inspect the build provenance and SBOM:

cosign download attestation ghcr.io/abhigyanpatwari/gitnexus:1.6.2 \
  --predicate-type https://slsa.dev/provenance/v1

Kubernetes: enforce signatures at admission

For Kubernetes deployments, ship the bundled ClusterImagePolicy so the Sigstore policy-controller rejects any GitNexus pod whose image is not signed by this repo's docker.yml running from a vX.Y.Z tag — the same identity the cosign verify snippet above pins.

# 1. Install the controller (one-time, cluster-wide)
helm repo add sigstore https://sigstore.github.io/helm-charts && helm repo update
helm install policy-controller -n cosign-system --create-namespace \
  sigstore/policy-controller

# 2. Opt your namespace in
kubectl label namespace <your-ns> policy.sigstore.dev/include=true

# 3. Apply the policy
kubectl apply -f deploy/kubernetes/cluster-image-policy.yaml

After this, attempting to deploy an unsigned image — or one signed by anything other than abhigyanpatwari/GitNexus's docker.yml at a v* tag — fails the admission webhook before a pod is ever created. This turns the verifiable signature into an enforced policy, which is the supply-chain control most clusters actually need.

Files

Dockerfile.web — builds gitnexus-shared and gitnexus-web, then serves the production frontend.
Dockerfile.cli — builds the CLI/server (with its native deps) and runs gitnexus serve --host 0.0.0.0.
docker-compose.yaml — starts both signed images side by side.
.env.example — overrides for image names, container names, ports, and the workspace mount.

The web UI uses the same indexing pipeline as the CLI but runs entirely in WebAssembly (Tree-sitter WASM, LadybugDB WASM, in-browser embeddings). It's great for quick exploration but limited by browser memory for larger repos.

Local Backend Mode: Run gitnexus serve and open the web UI locally — it auto-detects the server and shows all your indexed repos, with full AI chat support. No need to re-upload or re-index. The agent's tools (Cypher queries, search, code navigation) route through the backend HTTP API automatically.

The Problem GitNexus Solves

Tools like Cursor, Claude Code, Codex, Cline, Roo Code, and Windsurf are powerful — but they don't truly know your codebase structure.

What happens:

AI edits UserService.validate()
Doesn't know 47 functions depend on its return type
Breaking changes ship

Traditional Graph RAG vs GitNexus

Traditional approaches give the LLM raw graph edges and hope it explores enough. GitNexus precomputes structure at index time — clustering, tracing, scoring — so tools return complete context in one call:

flowchart TB
    subgraph Traditional["Traditional Graph RAG"]
        direction TB
        U1["User: What depends on UserService?"]
        U1 --> LLM1["LLM receives raw graph"]
        LLM1 --> Q1["Query 1: Find callers"]
        Q1 --> Q2["Query 2: What files?"]
        Q2 --> Q3["Query 3: Filter tests?"]
        Q3 --> Q4["Query 4: High-risk?"]
        Q4 --> OUT1["Answer after 4+ queries"]
    end

    subgraph GN["GitNexus Smart Tools"]
        direction TB
        U2["User: What depends on UserService?"]
        U2 --> TOOL["impact UserService upstream"]
        TOOL --> PRECOMP["Pre-structured response:
        8 callers, 3 clusters, all 90%+ confidence"]
        PRECOMP --> OUT2["Complete answer, 1 query"]
    end

Core innovation: Precomputed Relational Intelligence

Reliability — LLM can't miss context, it's already in the tool response
Token efficiency — No 10-query chains to understand one function
Model democratization — Smaller LLMs work because tools do the heavy lifting

How It Works

GitNexus builds a complete knowledge graph of your codebase through a multi-phase indexing pipeline:

Structure — Walks the file tree and maps folder/file relationships
Parsing — Extracts functions, classes, methods, and interfaces using Tree-sitter ASTs
Resolution — Resolves imports, function calls, heritage, constructor inference, and self/this receiver types across files with language-aware logic
Clustering — Groups related symbols into functional communities
Processes — Traces execution flows from entry points through call chains
Search — Builds hybrid search indexes for fast retrieval

Supported Languages

Language	Imports	Named Bindings	Exports	Heritage	Type Annotations	Constructor Inference	Config	Frameworks	Entry Points
TypeScript	✓	✓	✓	✓	✓	✓	✓	✓	✓
JavaScript	✓	✓	✓	✓	—	✓	✓	✓	✓
Python	✓	✓	✓	✓	✓	✓	✓	✓	✓
Java	✓	✓	✓	✓	✓	✓	—	✓	✓
Kotlin	✓	✓	✓	✓	✓	✓	—	✓	✓
C#	✓	✓	✓	✓	✓	✓	✓	✓	✓
Go	✓	—	✓	✓	✓	✓	✓	✓	✓
Rust	✓	✓	✓	✓	✓	✓	—	✓	✓
PHP	✓	✓	✓	—	✓	✓	✓	✓	✓
Ruby	✓	—	✓	✓	—	✓	—	✓	✓
Swift	—	—	✓	✓	✓	✓	✓	✓	✓
C	—	—	✓	—	✓	✓	—	✓	✓
C++	—	—	✓	✓	✓	✓	—	✓	✓
Dart	✓	—	✓	✓	✓	✓	—	✓	✓

Imports — cross-file import resolution · Named Bindings — import { X as Y } / re-export tracking · Exports — public/exported symbol detection · Heritage — class inheritance, interfaces, mixins · Type Annotations — explicit type extraction for receiver resolution · Constructor Inference — infer receiver type from constructor calls (self/this resolution included for all languages) · Config — language toolchain config parsing (tsconfig, go.mod, etc.) · Frameworks — AST-based framework pattern detection · Entry Points — entry point scoring heuristics

Tool Examples

Impact Analysis

impact({target: "UserService", direction: "upstream", minConfidence: 0.8})

TARGET: Class UserService (src/services/user.ts)

UPSTREAM (what depends on this):
  Depth 1 (WILL BREAK):
    handleLogin [CALLS 90%] -> src/api/auth.ts:45
    handleRegister [CALLS 90%] -> src/api/auth.ts:78
    UserController [CALLS 85%] -> src/controllers/user.ts:12
  Depth 2 (LIKELY AFFECTED):
    authRouter [IMPORTS] -> src/routes/auth.ts

Options: maxDepth, minConfidence, relationTypes (CALLS, IMPORTS, EXTENDS, IMPLEMENTS), includeTests

Process-Grouped Search

query({query: "authentication middleware"})

processes:
  - summary: "LoginFlow"
    priority: 0.042
    symbol_count: 4
    process_type: cross_community
    step_count: 7

process_symbols:
  - name: validateUser
    type: Function
    filePath: src/auth/validate.ts
    process_id: proc_login
    step_index: 2

definitions:
  - name: AuthConfig
    type: Interface
    filePath: src/types/auth.ts

Context (360-degree Symbol View)

context({name: "validateUser"})

symbol:
  uid: "Function:validateUser"
  kind: Function
  filePath: src/auth/validate.ts
  startLine: 15

incoming:
  calls: [handleLogin, handleRegister, UserController]
  imports: [authRouter]

outgoing:
  calls: [checkPassword, createSession]

processes:
  - name: LoginFlow (step 2/7)
  - name: RegistrationFlow (step 3/5)

Detect Changes (Pre-Commit)

detect_changes({scope: "all"})

summary:
  changed_count: 12
  affected_count: 3
  changed_files: 4
  risk_level: medium

changed_symbols: [validateUser, AuthService, ...]
affected_processes: [LoginFlow, RegistrationFlow, ...]

Rename (Multi-File)

rename({symbol_name: "validateUser", new_name: "verifyUser", dry_run: true})

status: success
files_affected: 5
total_edits: 8
graph_edits: 6     (high confidence)
text_search_edits: 2  (review carefully)
changes: [...]

Cypher Queries

-- Find what calls auth functions with high confidence
MATCH (c:Community {heuristicLabel: 'Authentication'})<-[:CodeRelation {type: 'MEMBER_OF'}]-(fn)
MATCH (caller)-[r:CodeRelation {type: 'CALLS'}]->(fn)
WHERE r.confidence > 0.8
RETURN caller.name, fn.name, r.confidence
ORDER BY r.confidence DESC

Wiki Generation

Generate LLM-powered documentation from your knowledge graph:

# Requires an LLM API key (OPENAI_API_KEY, etc.)
gitnexus wiki

# Use a custom model or provider
gitnexus wiki --model gpt-4o
gitnexus wiki --base-url https://api.anthropic.com/v1

# Force full regeneration
gitnexus wiki --force


# Increase the timeout or retries for large codebase or slow LLM providers
gitnexus wiki --timeout <seconds> # LLM request timeout in seconds (default: disabled)
gitnexus wiki --retries <n>      # Max LLM retry attempts per request (default: 3)

# Change the language generation for wiki
gitnexus wiki --lang <lang>  # Output language for generated documentation (e.g. english, chinese, spanish, japanese)

The wiki generator reads the indexed graph structure, groups files into modules via LLM, generates per-module documentation pages, and creates an overview page — all with cross-references to the knowledge graph.

Tech Stack

Layer	CLI	Web
Runtime	Node.js (native)	Browser (WASM)
Parsing	Tree-sitter native bindings	Tree-sitter WASM
Database	LadybugDB native	LadybugDB WASM
Embeddings	HuggingFace transformers.js (GPU/CPU)	transformers.js (WebGPU/WASM)
Search	BM25 + semantic + RRF	BM25 + semantic + RRF
Agent Interface	MCP (stdio)	LangChain ReAct agent
Visualization	—	Sigma.js + Graphology (WebGL)
Frontend	—	React 18, TypeScript, Vite, Tailwind v4
Clustering	Graphology	Graphology
Concurrency	Worker threads + async	Web Workers + Comlink

Roadmap

Actively Building

LLM Cluster Enrichment — Semantic cluster names via LLM API
AST Decorator Detection — Parse @Controller, @Get, etc.
Incremental Indexing — Only re-index changed files

Recently Completed

Constructor-Inferred Type Resolution, self/this Receiver Mapping
Wiki Generation, Multi-File Rename, Git-Diff Impact Analysis
Process-Grouped Search, 360-Degree Context, Claude Code Hooks
Multi-Repo MCP, Zero-Config Setup, 14 Language Support
Community Detection, Process Detection, Confidence Scoring
Hybrid Search, Vector Index

Security & Privacy

CLI: Everything runs locally on your machine. No network calls. Index stored in .gitnexus/ (gitignored). Global registry at ~/.gitnexus/ stores only paths and metadata.
Web: Everything runs in your browser. No code uploaded to any server. API keys stored in localStorage only.
Open source — audit the code yourself.

Acknowledgments

Tree-sitter — AST parsing
LadybugDB — Embedded graph database with vector support (formerly KuzuDB)
Sigma.js — WebGL graph rendering
transformers.js — Browser ML
Graphology — Graph data structures
MCP — Model Context Protocol

README.md Unescape Escape