mirror of https://github.com/abhigyanpatwari/GitNexus.git synced 2026-10-06 02:49:56 +00:00

History

Gergő Magyar b1d87c1f33 fix(eval): sweep evidence handling and measurement health, with guarded comparator reuse (#3207 ) * fix(eval): cut skill-evolution wall clock without shrinking the gate Reuse matching incumbent/CE cells, sanitize each SHA once, and default dispatch workers to 3 so weekly review generations finish inside the EventBridge window. Cap the sweep from leftover instance uptime so a Friday dispatch still uploads evidence. Co-authored-by: Cursor <cursoragent@cursor.com> Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * perf(eval): pipeline graph setup and correct the wall-clock cost model The evolution sweep paid `sanitize` + `analyze --pdg --index-only` for every unique task SHA on the critical path, one at a time, with nothing overlapping. `_run_sweep` now starts the next unpaid SHA's clone template and graph snapshot on a prefetch thread as soon as the current task's cells are dispatched, so every SHA but the first hides behind a paid session wave. The thread is joined before that SHA is used and before the trees tempdir is torn down, and a prefetch failure is recorded against the SHA exactly as an inline failure is. Tasks whose cells are all reusable comparator rows are not prefetched: they never build a graph, so priming one would be pure cost. Adds `measure_evolution_cost.py`, the cost model behind these numbers. It reads the review corpus, the evolve defaults, and the workflow's workers default — it does not start a session. Its first version charged `copy_isolated_tree` once per paid cell, serially. `run_cell` clones inside its own pool worker, so the clones in a wave overlap and only one is on the critical path per wave; the model now charges `ceil(cells / workers)` waves. Estimated review generation at workers=3: cold 21570s, weekly 7710s. Wall clock is quantised by `ceil(cells_per_task / workers)`. A cold review task is 9 cells, so workers=4 buys the wall clock of workers=3 and pays host contention for it. Documented in the workflow's rollout checklist. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * perf(eval): price the benchmark against measured cell durations The cost model assumed every cell runs the 1140s mean. Cells are not uniform: the 41 rows in Actions run 33912693948's artifact are 826s at the median, 1262s at the mean, 2976s at p90, with two pinned at the 5400s session ceiling. A wave waits for its slowest cell, so a mean understates every concurrent schedule — the previous model called workers=3 cold 5.99h when the same schedule against real durations is 10.33h. session_durations.json carries the sample in submission order with its provenance and its caveat: every cell in that run returned unusable evidence, so the durations are real but a clean run may sit lower. It is the only live artifact; the 2026-07-22 green run's has expired. The model now simulates the schedule cell by cell rather than multiplying a mean by a wave count, averaged over all 41 rotations of the sample so no single alignment between sample order and cell index decides the answer. It prices today's barrier (wave_makespan) against a continuously fed pool (fed_makespan) and reports both, and it charges the proposer session — one per generation, measured at 344.7s — which it had been omitting entirely. Measurement only; no runtime behaviour changes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * perf(eval): price arms separately and stop inventing setup constants Two errors in the model, both found by auditing it against the artifact it claims to describe. The arms are not interchangeable. `candidate_review` runs 1416s at the mean against `review`'s 1204s and `ce_review`'s 1176s, and the weekly lane pays the candidate arm and nothing else — reuse skips both incumbents. Pricing weekly from a pooled sample charged it for arms it never runs: weekly is 4.59h, not the 3.65h a pooled sample reported. Cells are also submitted run-major and arm-minor, so at workers=3 every wave holds one cell of each arm and the slowest arm sets the wave; the model now builds cells in that order. The setup constants were invented. GRAPH_ANALYZE_SECONDS=600 and TEMPLATE_SANITIZE_SECONDS=180 charged 3900s of per-SHA setup for a cold run — more than the entire non-session time of the source run, which was 2541s for 41 cells and 5 SHAs. `duration_s` is the sum of a cell's Claude sessions (runner_sessions.py), so that 2541s residual is every clone, graph build, sandbox and teardown the sweep paid. The model now charges the measured residual per cell, 62.0s, and no longer credits clone templates or graph prefetch: both landed after that run and there is no measurement of them yet. The residual bounds what they can be worth. Cold 37452s (10.40h), weekly 16541s (4.59h), against a fed pool at 31683s and 16541s. Measurement only; no runtime behaviour changes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * perf(eval): charge sweep overhead where more workers cannot dissolve it Three defects, found by auditing the model against the artifact again. The overhead was charged inside the schedule. session_durations.json claimed the residual was charged "per cell and serially - the pessimistic reading", but task_cells folded it into each cell's duration, where the pool then divided it by the worker count. The residual mixes per-cell work the pool really does divide with per-SHA graph setup it cannot, and the artifact cannot separate them, so it now sits outside the schedule: cold 11.09h, not 10.40h. Alignment averaging weighted the shortest sample twice. The arm samples are 13, 14 and 14 long and the average ran over max()=14 offsets, so candidate_review's first cell was counted twice and its last never. Averaging over lcm()=182 offsets weights every arm's sample evenly. The wall assumed all 54 cells run. Replaying the sample's own error_kind sequence through today's systemic_outage_streak trips the outage breaker at cell 5 of 41. The source run executed all 41, so its runner did not break on that sequence, but the current one would: these numbers price a HEALTHY sweep, and a sweep with the sample's failure profile never reaches them. Stated on generation_seconds and recorded next to the sample it qualifies. Measurement only; no runtime behaviour changes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(eval): give the review agent somewhere it can actually write Every review cell in the last recorded generation returned unusable evidence. Not some — all 41, across all three arms and all six tasks, at $3653 for the run. The transcripts say why, 127 times across 35 of 35 sessions: EROFS: read-only file system, open '/workspace/review-output.json.tmp.2.90a76e583b0c' The review arm mounted the artifact as a writable FILE at /workspace/review-output.json while binding /workspace read-only. The Write tool writes atomically: it creates `<target>.tmp.<n>.<hex>` beside the target and renames it. The parent was read-only, so the temp create failed and the artifact was never written. A writable file inside a read-only directory is not writable to anything that writes atomically. Agents tried /proc/self/root/workspace/... and /proc/1/root/workspace/... to get around it; all 41 artifacts came back 0 bytes. The artifact now lives in its own writable directory bound at /review-output, outside the workspace. That is what a rename needs, and it lets the workspace get stricter rather than looser: the review phase may now change nothing there at all (enforce_phase_workspace gained allowed_artifact=None), where before it was entitled to one path inside it. The file is no longer pre-created — the agent writes it, and absence is now meaningful evidence. parse_review_output reported every one of these as "review output is not valid UTF-8 JSON". The file was empty, and its except folded OSError, UnicodeError and JSONDecodeError into that one string, so a sandbox that made writing impossible was indistinguishable from an encoding fault. That is why this read as an agent-quality problem for fifteen consecutive non-green runs. Each cause now names itself: never written, empty, not valid UTF-8, not valid JSON with the decoder's position. run_arm also keeps the FIRST error_detail, as it already did for error_kind, so a phase-boundary violation is no longer buried under the parse failure it causes. The test double conflated sandbox.private_root with the clone, which put the artifact directory inside the workspace and would have hidden the stricter check. Regression tests pin the mount shape in the generated bwrap argv, the contract path in the prompt, the four parse diagnostics, and the untouched-workspace contract. Verified by unit tests only: this container has unprivileged user namespaces disabled, so bwrap cannot run here and the mount was not exercised end to end. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(review): close the artifact path in every layer that gates it Code review of this branch found the relocated review artifact was fixed in the bwrap mount and nowhere else. Four independent layers decide whether the agent can write it, and three still named the old location. Claude Code applies its own filesystem policy to its own tools, and build_claude_settings listed only /workspace, /tmp and /home/agent under allowWrite with denyRead ["/"]. The artifact used to live under /workspace, so this list was correct until it moved. SANDBOX_REVIEW_OUTPUT is now in allowWrite and allowRead; without it the bwrap bind grants a write the CLI then refuses. The task corpus still ran `test -s review-output.json` from the workspace, in a separate sandbox invocation that never sees the artifact mount. Every review cell would have been stamped verify-failed with resolved=False no matter how good the review was, which also made those rows permanently unreusable and so silently disabled this branch's own comparator reuse for review arms. The verify and hidden-oracle commands now read the location from GITNEXUS_BENCH_REVIEW_OUTPUT and get the directory bound read-only, mirroring the mount-plus-env-var shape _run_hidden_oracle already used. host_text and host_path did not translate the new path, so the host-unsafe backend told the agent to write somewhere that exists on neither backend. Adding the mapping exposed a second defect: host_text substituted every occurrence of a target, and "/review-output" appears twice in "/review-output/review-output.json" - once as the directory and once inside the filename. Matching is now anchored to a path boundary. Comparator reuse had three ways to accept evidence it should have rejected. A row with no runtime_digest passed the drift lock because the guard only compared when both sides were bound, and the branch's own test asserted that as correct; absence is now a mismatch and the test states the rule. materialize_reused_row overwrote recorded_at with the copy time while the age check read that field, so a row copied forward each generation refreshed its own clock and never aged out; the first measurement time is now preserved and aged against. A future-dated stamp passed a one-sided bound and is now rejected as corrupt. RUNTIME_DIGEST never reached the runner at all: runner_environment builds a fixed dict and process_control replaces the child environment wholesale, so the digest the workflow exports was dropped and the lock it feeds was inert. The instance-window deadline was also checked only after run_proposer returned, buying a proposal the generation had no room to benchmark. The graph prefetch thread was started without copy_context, so it never saw the cancellation ContextVar the rest of the sweep shares, and the outage breaker returned without setting cancel_event - together, a tripped breaker would block on joining a prefetch that was never told to stop. Both fixed, with outage checked before cancellation at the two exits so an outage keeps exit 1 instead of becoming a Ctrl-C's 130. Both bwrap canaries that actually execute a write still bound the pre-fix shape against a file this branch no longer creates, so they would have errored rather than caught anything. They now bind the directory and write atomically - temp file beside the target, then rename - which is the exact operation that failed with EROFS. A source-text assertion over inspect.getsource(run_arm) was replaced with one that inspects the real mount, and a wall-clock assertion was pinned to a fixed monotonic clock. Not applied, and why: binding task-asset and dependency digests into comparator reuse needs asset snapshots prepared before the reuse decision rather than inside the per-task loop, and shipping the comparison without that would add a guard that silently never fires. Forcing a paid canary cell per incumbent arm and folding reused rows into the outage streak are behaviour decisions, not fixes. Clone-template reuse still has no test. The cost model's per-cell residual still shrinks with arm count, overstating weekly savings by at most the 2541s residual; the docstring now says so rather than inventing a split. 585 eval tests pass, ruff clean, 29 workflow contract tests pass. The two test_model_gateway.py failures are pre-existing and fail on main. * fix(review): bind reuse to its environment and keep the health canary real Applies the five findings the previous review round left open. Comparator reuse ignored the environment a row was measured in. TaskReuseBinding carried the task and oracle identity but not the task-asset or sandbox-dependency digests, and this branch itself changes sandbox_dependencies in the review corpus - so a reused comparator could be measured against one dependency set and compared against a candidate built on another, handing the gate a false comparison. Closing it needed the digests to exist before the reuse decision, so asset snapshots are now prepared for every task up front instead of lazily inside the per-task loop. That also removes the concurrent TaskAssetCache.prepare the prefetch thread could otherwise race, which the file's own "plain dict, read-then-write race" comment warned about. Both digests fail closed on either side, matching the runtime digest. The broken-incumbent canary could not fire when reuse was working. It read `resolved`, which counts reused rows, so an arm whose cells were all reused always looked healthy - in precisely the run where a broken environment would go unnoticed. aggregate now also reports `resolved_fresh` and the canary reads it. That count would be vacuous if an arm were reused end to end, so the sweep keeps one paid cell per incumbent arm and says which one it kept. Reused rows did not participate in the outage streak, so a run of failures could carry across them and trip on stale history. A reused success now resets the streak the way a paid success does. The cost model charged sweep overhead per cell, which credited a weekly generation for shrinking work it still performs: it pays one arm instead of three but builds exactly the same graphs. Overhead is charged per SHA now. Weekly is 5.20h rather than the 4.80h the per-cell rate reported; cold is 10.86h. The residual still cannot be split between per-SHA and per-cell work from one artifact, so session_durations.json records that assumption and the direction it errs in, rather than leaving a number nobody can trace. Clone-template reuse - the branch's core speedup, taken on essentially every multi-cell sweep - now has a test that builds a real sanitized template, asserts the cell runs against the copy with the template's HEAD, and fails if run_cell re-clones. A second test asserting only on a namespace built inside the test was written and deleted: it exercised nothing, which is the failure this review round penalised elsewhere. 589 eval tests pass, ruff clean, 29 workflow contract tests pass. The two test_model_gateway.py failures are pre-existing and fail on main. * refactor(eval): consolidate duplicated harness logic after the review round Simplification pass over the branch. Behavior-preserving throughout; three reviewers, nine findings applied, two skipped. The review-artifact block in _run_hidden_oracle was unreachable. That function runs only in run_arm's non-review branch, while the directory it probes for is created only in the review branch, and each sandbox serves exactly one arm - so `review_artifact.parent.is_dir()` could never be true. It was added an hour earlier to make the hidden oracle resolve the moved artifact; the oracle never runs for review tasks, so the guard was dead on arrival. Deleting it also removes the duplication it had with the verify-command wiring. EXCLUDED_ERROR_KINDS is now one definition. runner.py and comparator_reuse.py each carried the same six-member frozenset, kept in sync by a comment. Only one direction is possible: runner already imports from comparator_reuse, so the reverse import fails at module-init with a circular-import error. That is now stated where the alias lives, so nobody tries it the other way. ensure_task_graph and prefetch_next_graph shared ten keyword parameters, passed through two call sites and forwarded whole between them. They now take a GraphBuildEnv, mirroring TaskCellContext, which already bundles per-cell state in this file. Its ready_keys() replaces an inline four-set union at the call site. Smaller consolidations: _sha256_file's hand-rolled chunk loop becomes hashlib.file_digest (3.11+, already used in runner_artifacts); _copy_owner_only reuses task_assets._write_all and COPY_CHUNK_BYTES instead of repeating the short-write retry; its stat-then-open existence check becomes the O_EXCL failure it was already relying on, which is atomic rather than merely narrow; and runner_environment reads the digest through comparator_reuse.current_runtime_digest instead of re-parsing the environment variable. Three test docstrings summarised the branch's own history ("the branch's core speedup", "the regression that produced fifteen runs") rather than the invariant under test. Rewritten to state the constraint, which is what survives the merge. Repaired the indentation left behind by the outage-streak edit and flattened the prefetch dispatch from three nested conditionals to one. Skipped: consolidating comparator_reuse._real_directory onto proposer_sandbox's same-named helper - they differ, the sandbox one rejects any symlink in the resolved path while this one checks only the leaf, so sharing it would tighten behavior rather than preserve it. That needs a decision about which policy the reuse path wants, not a simplification. 589 eval tests pass, ruff clean, 29 workflow contract tests pass. Unrelated and pre-existing: two test_model_gateway.py failures, and test_process_control.py::test_timeout_kills_term_ignoring_descendants_before_they_write, which is a TERM-to-KILL timing flake (passes 2 of 3 in isolation) in a file this branch does not touch. * refactor(eval): name the reuse directory check for the promise it makes The simplification pass left one finding open: comparator_reuse and proposer_sandbox both defined `_real_directory`, same name and same shape, with different guarantees. The sandbox one rejects every symlink hop in the path; the reuse one checks only the leaf and resolves through parents. Sharing the name invites a consolidation that would silently tighten one of them. They should not be merged, so the name stops claiming they could be. proposer_sandbox guards a mount root, where a symlink hop changes what an untrusted session is handed. comparator_reuse guards a data directory whose contents are already validated one file at a time - reads go through _regular_file, which lstats and rejects symlinks, and writes through O_NOFOLLOW. A symlinked parent therefore grants nothing those guards do not already cover, while refusing one would reject a symlinked artifacts directory or macOS's /var for no gain. Renamed to _resolved_directory, with the reasoning recorded at the definition, and a test that pins both halves: a symlinked parent is accepted and resolved, a symlinked leaf is still refused. Behavior is unchanged. 591 eval tests pass, ruff clean. The two test_model_gateway.py failures are pre-existing and fail on main. * test(eval): measure the sweep scheduler instead of modelling it measure_evolution_cost predicts wall clock from a model of what sweep_task_cells does. This runs the real thing - real threads, the real wave barrier, the real outage breaker - with only the paid agent session replaced by a sleep, and times it. Durations are the measured per-arm samples divided by 5000, so a 1416s cell takes ~0.28s. The shape is kept on purpose: the median cell is 826s against a 5400s ceiling, and that spread is the entire reason a barrier costs anything. Uniform random sleeps would erase the effect under test. All schedulers consume one identical seeded plan, so a comparison cannot be an artifact of one of them drawing luckier cells. The model survives contact: it tracks real execution within about 10%, and workers=1 - which runs without a pool at all - sits at 0.95, so the residual above 1.0 at higher worker counts is per-wave thread overhead rather than a modelling error. Two structural claims that were arithmetic are now observed. Weekly is flat from workers=3: 3.59, 3.59, 3.59, 3.60, 3.59, 3.59 across w=3..8. workers=4 buys nothing over workers=3 on cold, 7.68 against 7.78. Two prototype schedulers are measured beside it, deliberately before any production code exists. A continuously fed pool per task is worth more than the model claimed on cold, -27.3% against a predicted -17.9%, and exactly nothing on weekly, +0.0%, because a weekly task is one wave with nothing to feed. One pool across all tasks beats both: -40.7% weekly and -42.9% cold at workers=3, rising to -65.7% and -63.9% at workers=8. It also subsumes the fed pool, since packing across tasks is a fed pool. That reorders the backlog. Cross-task packing moves from second to first: it dominates on both profiles, and it is the only thing that moves weekly at all. Raising the worker count is worth nothing until it lands - under the barrier weekly does not improve from w=3 to w=8, and speedup against serial is 1.58x for three workers and only 2.40x for eight. The bound on all of it: sleeping threads do not contend. Real sandboxed sessions compete for CPU, page cache and disk, and the duration sample was itself measured at workers=1, so it carries no contention either. These speedups are upper bounds. The ordering is trustworthy because the schedulers were compared under identical conditions; the magnitudes are not. The packed prototype is also a bare ThreadPoolExecutor with no breaker folding, no per-task graph lifecycle and no reuse binding - which is the actual cost of building it, and is not measured here. * test(eval): carry the sweep invariants into the packed prototype The first packed prototype was a bare ThreadPoolExecutor. It reported -43% and none of the invariants the shipped scheduler holds, so it priced an idea nobody could ship. This one carries them: a global submission order continued across task boundaries, in-order folding, the real outage breaker, and per-task graph readiness gating behind a serial builder. The fidelity check first reported the two schedulers tripping on different cells, 17 against 16. That was my instrumentation, not a divergence - sweep_task_cells folds an entire wave before it evaluates the breaker, so the last cell folded is not the cell that tripped. With the harness mirroring the breaker's own evaluation the two agree exactly, across failures starting at cell 0, 4 and 12, with overrun inside the workers-1 bound the wave docstring promises. Two results worth the exercise. Head-of-line blocking, not the barrier, is what a naive in-order design pays. Holding submission to `workers` cells beyond the fold pointer leaves the faithful scheduler at -8.1% cold and -2.7% weekly: one slow cell stalls the pointer, the window cannot slide, and it reproduces the wave almost exactly. That is the number to quote if anyone proposes the obvious implementation. But the overrun bound turns out to be set by the worker count, not the window. Only `workers` cells can be running when the breaker trips; everything queued behind them short-circuits on the halt flag. Overrun is 3 at an unbounded window exactly as at 6, and the trip cell never moves off 16. So H2 does not have to trade breaker fidelity for speed - a wide window takes -42% with the semantics intact. The tension I assumed was there is not, and window=12 already captures 97% of it. Still an upper bound: sleeping threads do not contend, and the sample was measured at workers=1. What this establishes is that the invariants are affordable, which was the thing blocking H2. Not built here: the trees tempdir lifecycle, reuse-row binding, and the cancel_event path. 591 eval tests pass, ruff clean. * test(eval): put the scheduler comparison under real CPU contention Every Phase 2 number so far came from sleeping threads, which contend for nothing, against a duration sample measured at workers=1, which contains no contention either. That was the standing caveat on the whole result, so this measures it. A cell now waits for its API share and then burns a fixed number of sha256 rounds in a subprocess. Work-bounded rather than wall-clock bounded, so it takes longer when cores are busy - that is the effect under test. A subprocess because Python threads burning Python would measure the GIL rather than the machine. Calibrated at 519k rounds/s, stable within 2% across three probes. The first run of this was worthless and is recorded as such: on a 24-core host with 3 to 6 workers nothing ever contends, since cpu_fraction 0.5 at 6 workers is about 3 cores of demand out of 24. It measured an absence. Re-run pinned with taskset to 4 and 2 cores. The packing advantage survives. It holds between -40% and -47% across every host size and CPU fraction tested, including a genuinely oversubscribed 2-core box at cpu_fraction 0.5 with 6 workers. But contention erodes packing more than it erodes waves, for a structural reason: packing is what creates the concurrency. Moving from 24 cores to 2 at cpu 0.5 and 6 workers, the faithful scheduler slows 13% while the wave slows 3.7%, and the gain narrows from 45.0% to 39.8%. Packing and a higher worker count are therefore not independent wins - packing spends the contention headroom first, so raising workers has to be re-argued after it lands rather than added to it. Three things this still does not measure, and they bound the result. The real CPU fraction of a benchmark cell is a guess informed by roughly 180 tool calls per session; nobody has profiled one. The evolution runner's core count decides which column applies and is unknown here. And the burn is sha256, pure CPU, while real cells run vitest and analyze, which are memory and IO heavy - so this is a floor on contention, not a ceiling. 591 eval tests pass, ruff clean. * perf(eval): add a packed sweep scheduler, and correct the bound I claimed for it sweep_task_cells finishes one task before starting the next and drains a wave before refilling it, so a task with fewer cells than workers leaves workers idle and one slow cell stalls its whole wave. sweep_packed_cells feeds every task's cells through a single pool instead. Measured against the review corpus it is worth about 40% of a cold sweep, and it is the only change that moves a seeded weekly run at all - there a task is three cells and a wave is never full. The breaker keeps its exact meaning. Cells carry a total submission order continued across task boundaries, a folder walks results in that order, and consecutive systemic failures are counted there, so a doomed run aborts on the same cell it would have under waves. Verified at three failure positions. This commit also corrects a finding from the Phase 2 prototype. I claimed the overrun bound was set by the worker count rather than the submission window, and that packing therefore cost nothing in breaker fidelity. That was derived from a window sweep that only ever injected failures at one position. Driving the real function at other positions shows the halt flag does not bound overrun at all: the folder walks in order, so a slow early cell lets workers race ahead and the trip is detected after those cells have already paid. An unbounded queue overran by 11 cells where waves overrun by 2. So the window is load-bearing and the trade is real, measured at workers=3 with failures injected at four positions: window 3 -> -8% wall, overrun 2 (the wave scheduler's own bound) window 6 -> -27% wall, overrun 4 window 12 -> -42% wall, overrun 9 window 54 -> -44% wall, overrun 11 Overrun is wasted paid sessions at roughly $70 each. The default multiplier is 2, keeping the worst case within twice the wave bound while taking most of the gain; the curve is in the constant's comment so raising it is an informed decision rather than a guess. Not wired in yet: _run_sweep still calls sweep_task_cells per task. Moving the per-task graph, trees tempdir and reuse binding out of that loop behind await_ready is the larger and riskier half, and it belongs in its own change. 595 eval tests pass, ruff clean. * fix(eval): judge harness health on execution, not on how many tasks resolved broken_incumbent_arms infers "the environment is broken" from an arm resolving zero tasks. That inference does not hold: a reviewer can be wrong about every task in a hard corpus while every process, mount and capture worked perfectly. Actions run 33962002890 is exactly that shape - 51 cells, all resolved=False with error_kind=oracle-failed, median score 0.212, and a healthy harness. Someone already knew this, and patched it by excluding review arms at the call site. That leaves the unsound inference in place for workflow and workflow_direct, and leaves review arms with no health check at all - so the run that genuinely was broken, 33912693948, where the mount made an atomic write impossible and all 41 artifacts came back empty, could not have been caught here either. So this replaces the inference rather than adding another exemption. aggregate now classifies fresh rows into execution failures (the process or its tooling did not complete), evidence failures (it completed but produced nothing trustworthy or scoreable), and admissible measurements. An arm is unhealthy only when it has fresh attempts, zero admissible measurements, and at least one execution or evidence failure. Resolution count is no longer consulted. Arms with only reused rows report current health as UNKNOWN rather than good. With the inference corrected, review arms are checked again, which is what lets the empty-artifact case be caught at all. Deliberately unchanged: comparator reuse eligibility, quality denominators, promotion thresholds, model settings, skill prompts and scheduler behaviour. Failures that stop being called infrastructure failures still surface in the counts and reasons - an agent-originated failure must not vanish from reporting because it was reclassified. broken_incumbent_arms and its tests are left in place; deleting behaviour belongs in its own change. Seven regression tests, built from both runs' shapes and labelled as reconstructed from logged observations, since 33962002890's results.jsonl did not survive the instance shutdown. They pin: a badly-scoring reviewer is healthy; an all-zero score is still a valid negative; empty artifacts are unhealthy; one admissible cell keeps an arm healthy while its failures stay visible; reused rows alone leave health unknown; reused successes do not mask fresh failures; and a parseable artifact does not excuse a failed session. 602 eval tests pass, ruff clean. * fix(eval): pin the health guard below the breaker, and stop calling mixed runs healthy Two corrections to the health-classification patch. The regression I wrote could not have proved what it claimed. A fixture of 41 empty artifacts aborts through the outage breaker long before finalization: review-evidence-invalid is in SYSTEMIC_ERROR_KINDS and the limit is 5, so it trips at cell 5 through the pre-existing path. It demonstrated failure detection, not the new guard. The decisive test now uses ONE fresh unusable cell, asserts the streak stays under the breaker threshold, and only then requires finalization to abort - leaving the new check as the only thing that can catch it. Removing the call makes that test fail; restoring it passes. The accurate defect statement is narrower than the last message claimed. Review arms were excluded from the final incumbent-health check while the consecutive- failure breaker gave them separate, partial coverage. They were not unguarded. Second: "one admissible cell plus two execution failures" was asserted as healthy. That converts "not wholly unusable" into "ran reliably", which is how a partly-broken sweep passes review. Arms now report UNKNOWN, OBSERVED_OK, DEGRADED or UNUSABLE. Only UNUSABLE is fatal, so eligibility and promotion are untouched - this changes what is reported, not what is allowed. The guard is extracted as enforce_measurement_health so it can be driven directly, and it now reports a status line per arm. It names no cause: an empty artifact establishes that evidence is unusable, not that a mount rejected the write, so it prints cause=undetermined rather than guessing EROFS. It still runs after report.md and promotion.json are written, so a failing sweep leaves its evidence behind. ce_review is named explicitly at the call site. It is a comparator rather than a candidate, so it is absent from CANDIDATE_ARMS.values(), and dropping the review exclusion alone would have left it unclassified. The wiring test reads _run_sweep's compiled code object for the referenced global rather than matching source text. It is honest about its limit: it proves the call exists and would catch its removal, but no test here drives _run_sweep end to end, which needs bwrap and a sandbox. broken_incumbent_arms is marked LEGACY and NON-AUTHORITATIVE with removal tracked. It has no production caller. 608 eval tests pass, ruff clean. The two test_model_gateway.py failures are test_locked_litellm_translates_messages_to_offline_responses and test_openai_gateway_never_leaves_proxy_output_on_an_undrained_pipe; both fail identically on origin/main in this environment, checked directly rather than carried forward as an inherited label. * fix(eval): review artifact path, evidence classification, comparator reuse Extracted from the combined skill-evolution branch. This is the runtime change set: everything that alters how a sweep executes and what it records. The packed scheduler and its measurement harness were separated onto perf/skill-evolution-packed-scheduler, which is purely additive. Correctness. The review artifact was mounted as a writable FILE inside a read-only workspace while the agent's Write tool writes atomically - temp file beside the target, then rename - so the temp create failed EROFS and the artifact was never written. Four layers gate that path and three named the old location: the CLI's own allowWrite/allowRead policy, the task corpus verify command run in its own sandbox invocation, and host_text/host_path for the host-unsafe backend. Fixing the translator exposed a second defect, since "/review-output" appears twice in "/review-output/review-output.json"; matching is now anchored to a path boundary. parse_review_output folded OSError, UnicodeError and JSONDecodeError into one message, so an artifact that was never written looked like an encoding fault; each cause now names itself. Health classification. broken_incumbent_arms inferred a broken environment from an arm resolving zero tasks, which a reviewer facing a hard corpus falsifies - Actions run 33962002890 is exactly that shape. Arms are now classified from fresh execution and evidence outcomes as UNKNOWN, OBSERVED_OK, DEGRADED or UNUSABLE, and only UNUSABLE aborts. Resolution count is not consulted. The guard names no cause: an empty artifact establishes unusable evidence, not that a mount rejected the write. Comparator reuse. Reuse accepted evidence it should have rejected: a row without a runtime_digest passed the drift lock, recorded_at was overwritten with the copy time so a row could outlive its own max_age, and the binding ignored task-asset and dependency digests although this change alters sandbox_dependencies in the review corpus. Closing the last one required preparing asset snapshots before the reuse decision, which also removes the concurrent TaskAssetCache.prepare the prefetch thread could race. These three concerns share aggregate() and _run_sweep, which is why they ship together: separating them further would mean hunk-level surgery on a function all three modify, and the risk of a silent omission outweighs the reviewability gain. 592 eval tests pass at this base. The two test_model_gateway.py failures, test_locked_litellm_translates_messages_to_offline_responses and test_openai_gateway_never_leaves_proxy_output_on_an_undrained_pipe, fail identically on origin/main in this environment. Known gap, and the reason this is not ready to merge: no test drives _run_sweep end to end. enforce_measurement_health is unit-tested including the below-breaker unusable case, and the caller wiring is pinned structurally by reading _run_sweep's compiled code object, but interruption semantics, exit precedence and persisted artifacts are not exercised through the real path. * fix(eval): address PR review feedback (#3207) - aggregate: count admissible rows directly instead of subtracting the execution and evidence counters, which double-charged a row that is both a session error and invalid review evidence and could report UNUSABLE for an arm holding real measurements. - run_proposer: bound the session timeout by what is left of --max-runtime-seconds, so clearing the sweep minimum cannot start a full-length session past the instance window. - comparator reuse: hold one O_NOFOLLOW descriptor for the size check, digest and copy, and prove it is the inode that was checked, closing the swap window a concurrent writer of the reuse directory had. - Drive the review-artifact mount assertion through run_arm and the clone-template assertion through run_cell, instead of rebuilding the expected values in the tests (also removes the CodeQL unnecessary lambda). - Assert the workflow invokes run-evolution.sh rather than that its YAML mentions --max-runtime-seconds, which only appears in a comment. - Correct the parse_review_output failure-mode claim: the fold was empty artifacts reported as "not valid UTF-8 JSON"; a never-created file raised FileNotFoundError. - prettier: wrap the over-long readFileSync call flagged by PR autofix. Note: pre-existing failure in tests/test_model_gateway.py::test_locked_litellm_translates_messages_to_offline_responses (local LiteLLM proxy never becomes ready in this environment) not addressed by this PR. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(eval): address the second round of PR review feedback (#3207) - Refuse a symlinked `transcripts` component on both sides of comparator reuse. O_NOFOLLOW guards the leaf only, so a link there redirected the read or the copy out of the results directory; checked per component as evolution._require_directory_chain does. - Base the paid incumbent canary on the cells this sweep PLANS. Reuse selection accepts any prior run index, so a results directory produced with more runs left extra keys, the equality never held, and the canary stopped firing. Extracted as drop_canary_reuse_key and unit-tested. - Start the runtime clock in main(). --max-runtime-seconds is measured from /proc/uptime before exec, so parsing, task I/O, preflight and gateway setup were being handed back to the sweep out of the upload reserve. - Do not fall back to shutil.copytree when the managed clone copy was cancelled or timed out; that fallback is for a filesystem that cannot reflink, and copytree cannot be cancelled. - Assert the review session's writable mount, not only the verifier's read-only one: the EROFS bug is about the agent's write. - Exercise ref isolation in the copy_isolated_tree test rather than comparing an initial HEAD a shared namespace would also match. - Point the stale-symlink fixture at the sentinel via os.path.relpath, and skip the reuse symlink tests where symlink creation needs privilege. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(eval): close the runtime-cap gap and pin the reuse directory Both were left open on #3207 as approach decisions rather than nits. Runtime cap: run-evolution.sh computed the budget in its own `uv run python -c` and passed a number, so the script's remaining provenance work and the CLI's own startup were spent by nobody and charged to the sweep — out of the upload reserve the cap exists to protect. The script now passes --max-runtime-from-instance-window and evolve reads /proc/uptime itself, on the line after it starts the clock the budget is measured against, so no interval exists to lose. Also removes an interpreter start from the script and lets --dry-run print the real argv. Reuse directory: _real_child_directory lstat-checked `transcripts` and returned its pathname, so a concurrent writer could rename the directory and leave a symlink before the name was used again — O_NOFOLLOW guards only the leaf. Every artifact is now resolved against a held descriptor: _open_real_directory opens with O_DIRECTORY\|O_NOFOLLOW (check and open in one syscall), and _open_regular / _copy_owner_only take dir_fd. The reuse path is therefore POSIX-only; _require_openat says so and fails closed, which the runner already treats as "run a paid cell". _resolved_directory still tolerates a symlinked reuse root, unchanged and still tested. evolution._require_directory_chain is still lstat-per-component. It guards a different surface (candidate overlay reads) that neither review raised, so it is left alone rather than widened into here. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(eval): sample the proposer budget where it is spent, digest what is copied Four findings against `f0cdc9e7`, all of them mine. - run_proposer took a precomputed remaining_seconds, but it clones, sanitizes and builds a sandbox before the session starts, so the caller's reading was already stale. It takes started_monotonic now and samples the budget on the last line before run_claude. The caller's earlier reading still decides whether to start at all — it just no longer decides how long to allow. - _copy_transcript_artifact hashed the source and then read it again to copy it. A held descriptor stops the pathname being substituted, not the inode being rewritten, so the row could record the expected digest while the destination held other bytes. _copy_owner_only now digests the same buffers it writes and returns (digest, bytes); a mismatch unlinks the destination and raises. One read instead of two. - Explain the empty except in _open_real_directory: an existing directory is the ordinary case, and the O_DIRECTORY\|O_NOFOLLOW open below is what proves what it is (CodeQL). - Drop the unused uptime fixture from the runtime-cap test, and correct the cross-file contract described in the workflow-contract test and the workflow YAML: neither names --max-runtime-from-instance-window, the script does. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(eval): close the reuse producer/consumer contract through the real run_cell Every comparator-reuse test built its rows by hand. That proves the predicate's logic and nothing about the producer: a fixture can satisfy eligibility while a row the runner actually emits never does, and a key-name inventory cannot tell the difference. I checked that inventory first - all 25 keys the predicate reads have a producer - which is exactly why it was not sufficient evidence. This carries one record through the production path instead: real run_cell -> production JSONL writer -> load_result_rows -> row_is_reusable_comparator Only the expensive dependencies are replaced: the model session, sandbox launch, repository acquisition, graph preparation, git plumbing. The digest fields the reuse binding compares are assembled by run_cell itself from its TaskCellContext, so they stay real - they are the subject, not scaffolding. Expectations are built from the sweep's own configuration rather than copied out of the emitted row, since copying them back would make producer and consumer agree because the test arranged it. Two cases: a production-emitted row with matching bindings is eligible, and the same evidence with one dependency binding changed is rejected. A test-controlled digest set keeps both deterministic. Mutation-checked. Replacing run_cell's task_asset_manifest_digest assignment with None makes the positive case fail; restoring it passes. That is evidence the inventory audit could not produce. Building it also documented what a real review row must carry, which no fixture had recorded: a scored review with review_weighted_f1 present, and at least one transcript artifact whose source is PARENT_EVENT_STREAM_SOURCE. An empty artifact list is not reusable. Those are production contracts my first double got wrong, and the reader rejected it each time. No production code changed. 607 eval tests pass, ruff clean. * test(eval): drive the real sweep to its finalization decision enforce_measurement_health was unit-tested and its call site pinned by reading _run_sweep's compiled code object. Neither showed the guard running inside a sweep. These drive the real _run_sweep with cell execution scripted and everything downstream left alone: folding, aggregation, the artifact writers, the health guard and the exit selection. The below-breaker case is the decisive one. A fixture of many unusable cells aborts through the pre-existing outage breaker instead - review-evidence-invalid is systemic with a limit of five - and would pass whether or not the finalization guard exists. One fresh unusable cell stays under that threshold, so only the guard can catch it. The test asserts the arm and status the guard reports, and that no systemic-outage line was printed, rather than accepting any SystemExit: an unrelated setup failure must not satisfy it. Mutation-checked at the runtime path. Removing the guard invocation fails the below-breaker test because the expected finalization behaviour disappears, not because a name went missing from a code object. A zero score is covered separately. review_weighted_f1 = 0.0 must stay a present valid negative measurement; a truthiness check would read it as absent and turn a quality result into an execution-health failure. The third test asserts the evidence survives - results.jsonl carries the scored row and report.md exists - so the persisted artifacts tell the same story as the exit. Only expensive setup is replaced: cell execution, graph preparation, asset snapshots, and task-binding resolution, which clones the repository and verifies the ref. Building the fixture also documented that the report renders the whole review metric set, so an incomplete row fails in formatting rather than logic. Still open on this track: interruption semantics and exit-reason precedence (outage 1 before cancellation 130) are not yet exercised, and fold order and the real sandbox mount contract remain on separate tracks. No production code changed. 610 eval tests pass, ruff clean. The two test_model_gateway.py failures reproduce on origin/main in this environment. * fix(eval): an interrupted sweep is interrupted, not aborted Driving the real _run_sweep to its exit revealed that cancellation without an outage exits 1 and writes "Sweep aborted: partial evidence", where the contract is 130 and "Sweep cancelled". sweep_task_cells returns (streak, tripped), and its caller assigns that flag to outage_tripped and turns it into exit 1. Both cancellation paths returned True for it. The breaker's own return is also True, so the two became indistinguishable one frame up and cancellation inherited the outage's exit and wording. The fix is to return False from the cancellation paths: the flag means the breaker tripped, and the caller already tests cancel_event itself for the stop decision, so nothing stops running any later than before. Reproduced before the fix, through the real caller, not from reading. The failing assertion was the report line; the exit code was 1. Two runtime tests cover it. Each begins with one admissible cell so the arm classifies DEGRADED rather than UNUSABLE - otherwise enforce_measurement_health supplies exit 1 first and a precedence test passes without ever reaching exit selection. Cancellation is set from a completed cell rather than a sleep or a real signal, so the interruption point is deterministic. The precedence test is mutation-checked: moving the cancel_event check ahead of the outage check fails it with "assert 130 == 1" - it observes the wrong exit code, not merely some failure. My first attempt at that mutation silently matched nothing and the suite passed; a no-op mutation proves nothing, so the edit now asserts its own anchor. test_process_control read the same flag as "stopped" and asserted it after a cancellation. It now reports the event and the flag separately, which is what it was really asserting: cancelled, and not an outage. The scored-row shape moved into tests/bench_fixtures.py with zero values written out, so the finalization and interruption tests share one definition of what a real review row carries. Impact analysis returned risk UNKNOWN - the index predates this branch and does not carry the eval harness - so the callers were confirmed by text search as the rules require for UNKNOWN: one production caller and three test sites, all updated or verified. detect_changes reports zero for the same reason; that zero is unseen, not unaffected. 612 eval tests pass, ruff clean. The two test_model_gateway.py failures reproduce on origin/main in this environment. * test(eval): prove the review artifact mount under a real sandbox The orchestration tests replace the sandbox, so they say nothing about isolation. This covers the filesystem contract the EROFS defect actually broke, through the production configuration: the same command_prefix_for call run_arm makes for a review cell, with read_only_workspace=True and the review-output directory as the writable mount - not a hand-built mount tuple that merely looks right. A deterministic writer stands in for the agent. It writes a temp file beside the destination and renames it into place, which is the operation that failed: an atomic write needs a WRITABLE PARENT DIRECTORY, and binding the file itself left nowhere to put the temp file. It then attempts a workspace write and must be refused, and the production parse_review_output reads the bytes the sandbox left behind. No model session and no credentials. Placed in test_proposer_sandbox.py, which the "eval / containment (ubuntu)" job already runs with GITNEXUS_REQUIRE_BWRAP_CANARY=1. That gate is the point: where the variable is set, a missing namespace capability FAILS the job instead of skipping into a green tick. Verified here - forcing the variable on this machine fails with "bwrap: No permissions to create new namespace" rather than skipping. What this does not establish: the assertions have never executed. This machine cannot create user namespaces, so the test stops at the preflight. Imports, signatures and the payload were checked statically instead - parse_review_output returns a tuple rather than an object, and it rejects a body without "verdict", so both of my first attempts were wrong and are fixed. The first CI run on a namespace-capable runner is what will actually confirm it. Scope is the filesystem and process contract only. A stand-in writer does not establish that a particular agent CLI's own file-access policy permits the same operation; that is a second, independent gate. 40 passed, 10 skipped locally; ruff clean. * fix(eval): read the review artifact before the sandbox deletes it The containment (ubuntu) job has been red since before the mount canary was added, and both failures have the same cause. This branch moved the review artifact out of the clone and into the session's private root, which is the point of the change: the agent writes atomically, so the artifact needs a writable parent DIRECTORY outside the read-only workspace. prepare_sandbox removes that private root in a finally on scope exit. Two tests read the artifact AFTER the with block, so they were asserting against a directory the sandbox had already deleted - FileNotFoundError, reported as "review output was never written". Production is not affected, and this is the reason: run_arm holds the live session and reads the artifact inside that scope, both to score it and to mount it read-only into the verifier's separate sandbox invocation. The tests were the only readers outside it. Both now read inside the scope. The pre-existing canary (test_read_only_review_workspace_exposes_only_one_writable_artifact) predates this PR and passes on main, where the artifact still lived in the clone and survived teardown; the relocation is what broke it, so the fix belongs here. The new canary found this independently and agreed on the cause, which is what it was written for - though only after CI ran it, since this machine cannot create user namespaces. 40 passed, 10 skipped locally; ruff clean. The bwrap-gated tests still skip here and remain unverified until the ubuntu job runs them. * test(eval): pin what an interrupted sweep persists and refuses to promote The completed-run persistence test could not show either of these: it never interrupts, so it would pass even if the writers only ran on the clean path. Evidence already paid for survives cancellation. The cell that completed before the interruption is still in results.jsonl with its measurement intact, and report.md is still written. Losing those rows would mean paying for evidence the sweep then discards. An interrupted run emits nothing that authorizes promotion. Asserted as the semantic condition rather than the absence of a file: promotion.json IS still written for an aborted run - it is the record of why nothing was promoted - so the test requires run_status "aborted" and every decision reduced to insufficient_evidence with a partial-evidence reason, rather than requiring the artifact to disappear. Mutation-checked. Forcing complete=True at the promotion_evidence call site fails the test with "assert 'complete' == 'aborted'" - it observes an aborted run claiming completeness, not merely some failure. These were the last two unasserted items on this PR's finalization checklist. 614 eval tests pass, ruff clean. The two test_model_gateway.py failures are environmental (litellm[proxy] console script absent) and predate this branch. * Address PR review feedback (#3207) An exhausted runtime cap no longer buys a second. remaining_runtime_seconds floors at 0 and the proposer call site wrapped it in max(1, ...), so a cap fully spent by the clone, the sanitize pass and the sandbox build started a paid session with a one-second allowance instead of stopping before the upload reserve the cap exists to protect. It now returns a not-ok record, which the caller already treats as end-of-run. Pinned by a test driving the real run_proposer with setup that consumes the whole budget; reverting to max(1, ...) fails it. Reuse copies are bounded before their size is validated, not after. The source is a prior sweep directory this module already treats as concurrently writable, so a transcript appended to after its metadata was recorded was streamed to EOF and only then compared against its declared size - filling the destination, or never reaching EOF, long before the drift check could reject it. _copy_owner_only now takes max_bytes and stops one byte past the ceiling, which keeps the drift comparison meaningful. Review and patch artifacts carry no recorded size, but "no recorded size" is not "no limit": they get MAX_TRANSCRIPT_BYTES, the ceiling the capture path already enforces. A reused review row must carry its artifact. The predicate accepted a row on its score and transcript metadata while materialize_reused_row copied the review artifact only when the row named one, so a scored review could be carried forward with nothing for a proposer to read. The shared row fixture was the thing out of step here, not the requirement - production sets review_artifact whenever the review source exists - so it now carries one too. Test fixes, all against code this PR added: unusable_review_row could not be overridden at all. It passed explicit keywords beside *overrides, and Python rejects the duplicate in the call expression before scored_review_row can apply its update, so unusable_review_row(error_kind=...) raised TypeError. Merged into one mapping. The finalization helper monkeypatched runner.prepare_ce_plugin_snapshot with raising=False. No such symbol exists - the real one is staged_ce_plugin_snapshot - so it silently added an attribute nothing reads. Removed. The promotion test named candidate_review in candidate_arms but _run_sweep builds cells only from args.arms, so no candidate cell ran and insufficient_evidence could hold because nothing executed rather than because partial evidence is barred. The candidate arm now runs, cancellation fires after its cell, and the test asserts the candidate actually produced a row so it cannot silently return to being vacuous. The reuse round trip serialized with json.dumps and write_text while claiming to cover the production writer, which applies redact_text over the row's own bytes. It now writes the way the sweep does. Its fixture also writes the review artifact, because run_cell records review_artifact only when the review source exists and the emitted row was otherwise one production never emits. Not addressing the duplicate-match nit on proposer_sandbox.py: the claim is that "/review-output" occurs once in "/review-output/review-output.json". It occurs twice, at offsets 0 and 14 - the separator before the basename forms it again - so the boundary the comment describes is real. Verified with re.finditer. The workspace-snapshot finding is parked for a human: excluding bootstrap noise is a documented deliberate choice, and tightening it is a genuine tradeoff. 634 eval tests pass, ruff clean. The two test_model_gateway.py failures are environmental (litellm[proxy] console script absent) and predate this branch. Address PR review feedback (#3207), round 2 Carry review_f1 in the scored-row fixture. score_review emits "f1" (review_scoring.py:317), which the runner folds in as review_f1, so a real scored row has it and the fixture did not - the same fidelity gap as the metrics already added there. aggregate now reports 0.5 for it instead of None. The reported failure mode was not real, and the distinction matters for anyone reading the thread later. aggregate filters on record.get(metric) is not None BEFORE indexing record[metric], so a missing key is skipped rather than raising KeyError; the finalization tests were passing throughout. Verified by running aggregate against the old fixture. Fixed because the fixture should match what production emits, not because anything was crashing. Drop the unused row parameter from the round-trip _expectation helper. It never read the argument - deliberately, since the docstring says the bindings must be derived from sweep configuration rather than copied out of the emitted row - but passing the row anyway was dead plumbing that suggested the opposite. The reuse-root TOCTOU finding is parked for a human rather than fixed: the lstat-then-resolve window is real, but _resolved_directory documents the weaker promise as deliberate and says not to merge it with proposer_sandbox's helper "without first deciding which promise the reuse path should make". That decision is the fix, and it is the same class as the reuse TOCTOU already parked on this PR. 634 eval tests pass, ruff clean. test_process_control's TERM-ignoring-descendant test flaked once under full-suite load and passes 3/3 in isolation; it is a timing test and neither file changed here touches it. The two test_model_gateway.py failures remain environmental. * fix(eval): pin the reuse roots to the directory that was checked Settles the reuse TOCTOU that has been parked twice on this PR. The docstring asked to decide which promise the reuse path makes before merging its helper with proposer_sandbox's, and the answer is that two separate things were being conflated. The symlink POLICY stays exactly as it was: parent hops remain allowed, so a symlinked artifacts directory or macOS's /var still works, and a symlinked leaf is still refused. Rejecting hops would break ordinary setups for no gain, which is what the docstring argued and it is still right. What is closed is the other thing - the gap between checking a name and using it. _resolved_directory lstats a name and the later open re-walks that same name, so a prior sweep that renames its results root and leaves something else behind is opened somewhere else entirely, and O_NOFOLLOW cannot see a link that resolve() already followed. _resolved_directory now returns the checked directory's identity alongside its path, and _open_pinned_root fstats the descriptor it opened and refuses a mismatch. The two are separable, so there was no trade to make: comparing identity rejects nothing that holds still, since a stable directory always matches itself. Both cases are pinned - a replacement by a different REAL directory is refused (the leaf-symlink rule does not cover that one), and an ordinary unchanged root opens normally. Why it matters here rather than as a general hardening: the failure is silent. Rows would be copied out of some other directory and folded into a comparator baseline as though they were this sweep's own evidence, which is the one thing the reuse path exists to get right - a corrupted baseline decides promotions. 636 eval tests pass, ruff clean. The two test_model_gateway.py failures remain environmental. * refactor(eval): one prompt digest, and drop two pieces of dead scaffolding Compute task_prompt_digest once. row_is_reusable_comparator compares the value a prior row stored against the value this sweep derives, as exact strings, and the two inline copies of that hash had already drifted apart - the expectation side had picked up a str() cast the row side lacked. They agree today only because tasks validate "prompt" as a string (runner_tasks.required_strings), so nothing was broken; a later divergence on one side would have silently stopped rows matching, with no test to catch it. Verified the extracted helper is byte-identical to what it replaced. Two comments named broken_incumbent_arms as the thing that would read stale health. This branch replaced that call with the measurement-health path, so the reasoning still holds but the name no longer does; they now name arm_health. The function itself stays: origin/main still calls it at runner.py:2098, and retiring it is its own change rather than a cleanup pass. Removed instance_window_budget_from_proc and its one test. It composes two helpers main() deliberately calls separately - there is a comment there explaining why the read and the budget calculation stay decoupled - and nothing outside its own test ever called it. Introduced on this branch, absent from main, so it was never deployed, public, or consumed elsewhere. Dropped an unused tmp_path parameter from a comparator-reuse test that does no filesystem work. Skipped three findings. Hoisting _assert_self_contained_git_objects to a once-per-template check is a real saving - measured at 5.66s over 1551 objects - but it verifies the OUTPUT of each copy operation, and git clone from a local path hardlinks objects by default, which is exactly what its st_nlink check catches. Checking a predecessor instead of the clone each cell runs against thins an isolation guarantee for 0.7% of a median cell. Deleting broken_incumbent_arms outright would remove a symbol main still calls. And a fourth copy of the test-only _git helper extends a pattern that already exists in three other test modules; centralising it means editing conftest.py, outside this scope. 635 eval tests pass, ruff clean. The two test_model_gateway.py failures are the environmental ones. --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>		2026-09-08 09:25:45 +00:00
..
.claude	feat(cpp): sfinae filter (#1623 )	2026-05-16 20:23:13 +01:00
bench	docs(bench): close FTS as an optimization target with measured evidence (#3209 )	2026-09-07 19:22:24 +01:00
hooks	fix(cli): stop churning the committed agent guides, and nudge --index-only (#2907 ) (#2927 )	2026-08-11 13:30:14 +01:00
scripts	fix(zig): vendor tree-sitter-zig so npm i -g no longer warns on peers (#3180 )	2026-09-05 10:35:33 +01:00
skills	feat(ingestion): mint Destination nodes from AsyncAPI 3.x documents (#3140 )	2026-09-02 22:01:15 +00:00
src	docs(bench): close FTS as an optimization target with measured evidence (#3209 )	2026-09-07 19:22:24 +01:00
test	fix(eval): sweep evidence handling and measurement health, with guarded comparator reuse (#3207 )	2026-09-08 09:25:45 +00:00
vendor	fix(zig): vendor tree-sitter-zig so npm i -g no longer warns on peers (#3180 )	2026-09-05 10:35:33 +01:00
.env.example	feat(serve): validate and port-scope the origin/proxy configuration surface (#2820 )	2026-08-05 06:52:39 +01:00
.npmignore	feat(install): toolchain-free tree-sitter via vendored prebuilds (#2113 )	2026-06-09 18:16:24 +01:00
CHANGELOG.md	chore: release v1.6.11 (#3177 )	2026-09-04 20:02:37 +01:00
Dockerfile.test	fix(security): Pin Docker Node base images, remove runtime package-manager CVE surface, verify Trivy on PRs, and harden Dependabot policy (#1455 )	2026-05-09 16:55:31 +01:00
package-lock.json	chore(deps)(deps): bump express-rate-limit in /gitnexus (#3214 )	2026-09-08 06:23:22 +01:00
package.json	fix(zig): vendor tree-sitter-zig so npm i -g no longer warns on peers (#3180 )	2026-09-05 10:35:33 +01:00
README.md	fix(zig): vendor tree-sitter-zig so npm i -g no longer warns on peers (#3180 )	2026-09-05 10:35:33 +01:00
tsconfig.json	perf(cfg): SSA-sparse reaching-defs to replace the dense-set worklist (#2201 ) (#2212 )	2026-06-15 19:16:53 +01:00
tsconfig.test.json	test: add test suite with vitest (unit + integration + fixtures)	2026-03-01 20:07:02 +05:30
vitest.config.ts	feat(ingestion): resolve Spring messaging destinations into Destination nodes (#3132 )	2026-09-02 11:46:50 +01:00

README.md

GitNexus

Graph-powered code intelligence for AI agents. Index any codebase into a knowledge graph, then query it via MCP or CLI.

Works with Cursor, Claude Code, Antigravity (Google), Codex, Windsurf, Cline, OpenCode, CodeBuddy (Tencent), Qoder (Alibaba), and any MCP-compatible tool.

Why?

AI coding tools don't understand your codebase structure. They edit a function without knowing 47 other functions depend on it. GitNexus fixes this by precomputing every dependency, call chain, and relationship into a queryable graph.

Three commands to give your AI agent full codebase awareness.

Quick Start

# Index your repo (run from repo root)
npx gitnexus analyze

That's it. This indexes the codebase, installs agent skills, registers Claude Code hooks, and creates AGENTS.md / CLAUDE.md context files — all in one command.

On npm 11.x? npx can crash during install (Cannot destructure property 'package' of 'node.target'). Use the pnpm form instead:
pnpm --allow-build=@ladybugdb/core --allow-build=gitnexus --allow-build=tree-sitter dlx gitnexus@latest analyze
See Troubleshooting → npx gitnexus crashes with node.target is null (npm 11) for the full matrix (global install, npm downgrade).

To configure MCP for your editor, run npx gitnexus setup once — or set it up manually below.

gitnexus setup auto-detects your editors and writes the correct global MCP config. You only need to run it once. To configure only selected integrations, pass --coding-agent/-c with a comma-separated list or repeat the option, for example gitnexus setup -c cursor,codex.

Editor Support

Editor	MCP	Skills	Hooks (auto-augment)	Support
Claude Code	Yes	Yes	Yes (PreToolUse + PostToolUse)	Full
Cursor	Yes	Yes	Yes (postToolUse, manual install)	Full
Antigravity (Google)	Yes	Yes	Yes (AfterTool, Gemini CLI hooks schema)	Full
Codex	Yes	Yes	Yes (PreToolUse + PostToolUse, Codex hooks)	Full
OpenCode	Yes	Yes	—	MCP + Skills
CodeBuddy (Tencent)	Yes	Yes	—	MCP + Skills
Qoder (Alibaba)	Yes	Yes	—	MCP + Skills
Windsurf	Yes	—	—	MCP

Claude Code and Codex get the deepest integration: MCP tools + agent skills + PreToolUse hooks that automatically enrich grep/glob/bash calls with knowledge graph context + PostToolUse hooks that detect a stale index after commits and prompt the agent to reindex.

Community Integrations

Agent	Install	Source
pi	`pi install npm:pi-gitnexus`	pi-gitnexus

MCP Setup (manual)

If you prefer to configure manually instead of using gitnexus setup:

Claude Code (full support — MCP + skills + hooks)

# macOS / Linux
claude mcp add gitnexus -- npx -y gitnexus@latest mcp

# Windows
claude mcp add gitnexus -- cmd /c npx -y gitnexus@latest mcp

Codex (full support — MCP + skills + hooks)

codex mcp add gitnexus -- npx -y gitnexus@latest mcp

Codex hooks (PreToolUse graph enrichment + PostToolUse stale-index detection in ~/.codex/hooks.json, same schema as Claude Code) need the bundled adapter script, so they are installed by gitnexus setup -c codex rather than manually.

Alternatively, install everything as a Codex plugin (MCP + skills + hooks in one step):

codex plugin marketplace add abhigyanpatwari/GitNexus
# then inside Codex: /plugins → install "GitNexus"

Codex notes: SessionStart is intentionally not registered — Codex reads AGENTS.md natively, which already carries the GitNexus context block. Newly installed hooks need a one-time approval in Codex via /hooks before they run. Pick one install route (gitnexus setup -c codex or the plugin): plugin hooks load alongside ~/.codex/hooks.json, so installing both can fire duplicate hooks per tool call.

Cursor / Windsurf

Add to ~/.cursor/mcp.json (global — works for all projects):

{
  "mcpServers": {
    "gitnexus": {
      "command": "npx",
      "args": ["-y", "gitnexus@latest", "mcp"]
    }
  }
}

OpenCode

Add to ~/.config/opencode/config.json:

{
  "mcp": {
    "gitnexus": {
      "command": "npx",
      "args": ["-y", "gitnexus@latest", "mcp"]
    }
  }
}

CodeBuddy

CodeBuddy reads only the first existing file in its config priority chain: ~/.codebuddy/.mcp.json (recommended) → ~/.codebuddy/mcp.json (deprecated) → ~/.codebuddy.json (legacy). Edit the first non-empty file that exists — creating a higher-priority file would hide the servers in the ones below it. If none exist, create ~/.codebuddy/.mcp.json:

{
  "mcpServers": {
    "gitnexus": {
      "command": "npx",
      "args": ["-y", "gitnexus@latest", "mcp"]
    }
  }
}

Qoder

Add to ~/.qoder.json:

{
  "mcpServers": {
    "gitnexus": {
      "command": "npx",
      "args": ["-y", "gitnexus@latest", "mcp"]
    }
  }
}

How It Works

GitNexus builds a complete knowledge graph of your codebase through a multi-phase indexing pipeline:

Structure — Walks the file tree and maps folder/file relationships
Parsing — Extracts functions, classes, methods, and interfaces using Tree-sitter ASTs
Resolution — Resolves imports and function calls across files with language-aware logic
- Field & Property Type Resolution — Tracks field types across classes and interfaces for deep chain resolution (e.g., user.address.city.getName())
- Return-Type-Aware Variable Binding — Infers variable types from function return types, enabling accurate call-result binding
Clustering — Groups related symbols into functional communities
Processes — Traces execution flows from entry points through call chains
Search — Builds hybrid search indexes for fast retrieval

The result is a LadybugDB graph database stored locally in .gitnexus/ with full-text search and semantic embeddings.

Experimental community detection engine

Experimental — not supported for production indexes. The Icebug engine is a research path for #2337. It carries no stability guarantee, may change or be removed without a major version, and partitions differently from the default, so switching engines changes community IDs and any generated context keyed on them. Reindex with graphology before relying on the output.

Community detection uses the bundled Graphology Leiden implementation by default. To try the #2337 Icebug path without changing default analyze behavior, install the optional native package alongside GitNexus and set the engine:

npm i @ladybugmem/icebug
GITNEXUS_COMMUNITY_ENGINE=icebug npx gitnexus analyze

Supported values are graphology, icebug, and auto. Today auto is behaviorally identical to icebug: both try Icebug and fall back to Graphology, while graphology skips Icebug entirely.

Icebug is not a declared dependency — its prebuilds link against system Arrow 24 (libarrow.so.2400), OpenMP, and glibc ≥ 2.38, none of which GitNexus can assume. Analyze falls back to Graphology and reports the reason in progress output when the module is missing, fails to load, or predates the setNumberOfThreads / setSeed controls that reproducible community IDs require (present at icebug-nodejs HEAD, absent from the published 12.8.0 tarball — so the fallback is what you will see today). The engine is pinned to threads: 1, randomize: false for determinism.

Note that the bundled Graphology path is no longer the slow option it once was: #2337 removed an accidental O(communities × N) copy in the vendored Leiden. On a synthetic 200k-node / 800k-edge benchmark graph it went from exceeding the 60s timeout to finishing in ~15s. Real projections vary with their degree distribution, so treat that as a direction, not a guarantee.

MCP Tools

Your AI agent gets 17 tools (15 per-repo + 2 group) automatically:

Tool	What It Does
`list_repos`	Discover all indexed repositories (paginated — `limit`/`offset`)
`query`	Process-grouped hybrid search (BM25 + semantic + RRF)
`context`	360-degree symbol view — categorized refs, process participation
`impact`	Blast radius analysis with depth grouping and confidence
`trace`	Shortest directed path between two symbols (call + class-member edges)
`detect_changes`	Git-diff impact — maps changed lines to affected processes
`check`	Read-only structural checks against the indexed graph
`rename`	Multi-file coordinated rename with graph + text search
`cypher`	Raw Cypher graph queries
`route_map`	API route map — which components fetch which endpoints, and handlers
`tool_map`	MCP/RPC tool definitions — where they're defined and handled
`shape_check`	Validate API response shapes against consumers' property accesses
`api_impact`	Pre-change impact report for an API route handler
`explain`	Explain persisted taint findings (source→sink flows, `--pdg` indexes)
`pdg_query`	Query control/data dependence at statement level (`--pdg` indexes)
`group_list`	List configured repository groups
`group_sync`	Rebuild a group's Contract Registry and cross-repo links

Read-only tools can omit repo when one repo is indexed, an MCP default is configured, or the GitNexus process cwd is inside a registered path without crossing into an unindexed nested Git checkout. Otherwise—and for mutating tools with multiple indexed repos and no MCP default—specify it explicitly: query({search_query: "auth", repo: "my-app"}). Per-repo tools also take an optional branch for indexes pinned with gitnexus analyze --branch; omitting it queries the workspace index, which follows your checked-out working tree. explain and pdg_query need an index built with gitnexus analyze --pdg.

MCP Resources

Resource	Purpose
`gitnexus://repos`	List all indexed repositories (read first)
`gitnexus://setup`	Setup and usage guidance for agents
`gitnexus://repo/{name}/context`	Codebase stats, staleness check, and available tools
`gitnexus://repo/{name}/clusters`	All functional clusters with cohesion scores
`gitnexus://repo/{name}/cluster/{name}`	Cluster members and details
`gitnexus://repo/{name}/processes`	All execution flows
`gitnexus://repo/{name}/process/{name}`	Full process trace with steps
`gitnexus://repo/{name}/schema`	Graph schema for Cypher queries
`gitnexus://group/{name}/contracts`	A group's extracted contracts and cross-links
`gitnexus://group/{name}/status`	Staleness of repos in a group

MCP Prompts

Prompt	What It Does
`detect_impact`	Pre-commit change analysis — scope, affected processes, risk level
`generate_map`	Architecture documentation from the knowledge graph with mermaid diagrams

CLI Commands

gitnexus setup                   # Configure MCP for detected editors (one-time; use -c to select)
gitnexus uninstall               # Preview removal of GitNexus MCP/skills/hooks (add --force to apply)
gitnexus analyze [path]          # Index a repository (or update stale index)
gitnexus analyze [path] --watch  # Watch local files and serialize incremental refreshes
gitnexus analyze --repair-fts    # Fast path: rebuild/verify only FTS indexes on existing index data
gitnexus analyze --force         # Rebuild graph + FTS; may reuse unchanged parser output
gitnexus analyze --no-parse-cache # Re-parse every source file, then rebuild graph + FTS
gitnexus analyze --embeddings    # Enable embedding generation (slower, better search)
gitnexus embeddings install      # Fetch the optional local embedding stack on demand (--cuda, --force)
gitnexus analyze --skills        # Generate repo-specific skill files from detected communities
gitnexus analyze --skip-agents-md  # Preserve custom AGENTS.md/CLAUDE.md gitnexus section edits (does not skip standard skills; use --skip-skills; community --skills files are unaffected)
gitnexus analyze --skip-skills   # Skip installing standard .claude/skills/gitnexus-* skill files
gitnexus analyze --skip-git      # Index folders that are not Git repositories
gitnexus analyze --workers <n>   # Parse worker pool size (>=1; default: cores-1, capped at 16)
gitnexus analyze --spring-actuator ./actuator  # Enrich with local Spring Boot Actuator JSON snapshots
gitnexus analyze --verbose       # Log skipped files when parsers are unavailable
gitnexus analyze --max-file-size 1024  # Skip files larger than N KB (default: 512, cap: 32768)
gitnexus analyze --worker-timeout 60  # Increase worker idle timeout for slow parses
gitnexus analyze --wal-checkpoint-threshold 67108864  # 64 MiB. Control LadybugDB WAL auto-checkpoint threshold (default: 67108864 = 64 MiB; -1 keeps Ladybug stock ~16 MiB)
gitnexus auto-sync [init|start|restart|stop|status|reset]  # Scheduled remote clone/pull + analyze from GITNEXUS_HOME/watch_config.yml
gitnexus mcp                     # Start MCP server (stdio) — serves all indexed repos
gitnexus serve                   # Start local HTTP server (multi-repo) for web UI
gitnexus index                   # Register an existing .gitnexus/ folder into the global registry
gitnexus list                    # List all indexed repositories
gitnexus status                  # Show index status for current repo
gitnexus clean                   # Delete index for current repo
gitnexus clean --all --force     # Delete all indexes
gitnexus wiki [path]             # Generate LLM-powered docs from knowledge graph
gitnexus wiki --model <model>    # Wiki with custom LLM model (default: minimax/minimax-m2.5)
gitnexus wiki --provider grok    # Local Grok Build CLI (uses `grok login`, no API key)
gitnexus wiki --base-url http://llama-box.local:8080/v1 --allow-insecure-connection llama-box.local
                                  # Allow an exact LAN/self-hosted HTTP LLM host; env: GITNEXUS_ALLOW_INSECURE_CONNECTION
gitnexus doctor                  # Show runtime platform capabilities and embedding configuration
gitnexus update                  # Install the latest published GitNexus (`npm i -g gitnexus@<x.y.z>`)

# Direct graph queries — the same tools the MCP server exposes, no MCP daemon needed
gitnexus query "<concept>"                                    # Process-grouped hybrid search
gitnexus context <symbol> [--uid <uid> | --file <path>]       # 360° symbol view; flags disambiguate a shared name
gitnexus impact <symbol> [--uid <uid> | --file <path> | --kind <kind>]  # Blast radius; flags disambiguate a shared name
gitnexus trace <from> <to>       # Shortest directed path between two symbols
gitnexus detect-changes          # Map the working-tree diff to affected symbols and execution flows
gitnexus check                   # Read-only structural checks against the indexed graph
gitnexus cypher "<query>"        # Run a raw Cypher query against the knowledge graph

# Repository groups (multi-repo / monorepo service tracking)
gitnexus group create <name>                                   # Create a repository group
gitnexus group add <group> <groupPath> <registryName>          # Add a repo to a group. <groupPath> is a hierarchy path (e.g. hr/hiring/backend); <registryName> is the repo's name from the registry (see `gitnexus list`)
gitnexus group remove <group> <groupPath>                      # Remove a repo from a group by its hierarchy path
gitnexus group list [name]                                     # List groups, or show one group's config
gitnexus group sync <name>                                     # Extract contracts and match across repos/services
gitnexus group contracts <name>  # Inspect extracted contracts and cross-links
gitnexus group query <name> <q>  # Search execution flows across all repos in a group
gitnexus group status <name>     # Check staleness of repos in a group
gitnexus group impact <name> --target <symbol> --repo <groupPath>  # Cross-repo blast radius

gitnexus analyze --watch requires a Git repository. It performs an initial analysis and then debounces scanner-admitted working-tree changes for 300 ms by default into serialized incremental refreshes. Events arriving during a run remain queued, and retryable failures retain the same batch with bounded backoff. Invalid .gitnexusrc or ignore-file reloads pause ordinary refreshes until the control file is fixed. Watch refreshes update only the graph: they intentionally skip AGENTS.md / CLAUDE.md injection and standard skill installation. Run a one-shot gitnexus analyze when those generated files need updating. Stop watch mode with Ctrl+C.

Watch mode accepts --debounce, --workers, --worker-timeout, --max-file-size, --branch, --pdg, --name, --allow-duplicate-name, and --verbose. Explicit one-shot options such as --force, --repair-fts, embedding flags, --skills, --default-branch, --skip-agents-md, --skip-skills, --no-stats, --self-commit, --index-only, and --skip-git are rejected. Unsupported defaults from .gitnexusrc are ignored with a warning.

POSIX requests clone-first copy-and-swap publication when the live index has no orphan sidecars. Windows and sidecar fallback runs update in place: failures known to occur before writes are retried, while a failure that may have mutated the live index stops the watcher. Watch mode does not pull remotes. Running MCP and serve processes periodically check for a newly published index and reopen it without a restart. MCP checks are throttled to once every five seconds, so a tool call before the next check can briefly use the previous index.

`gitnexus auto-sync`

gitnexus auto-sync is a different product from gitnexus analyze --watch. It is the explicit long-running auto-sync entrypoint that clones or pulls configured remotes. gitnexus watch is reserved and does not start either job: it prints this split. GITNEXUS_HOME defaults to ~/.gitnexus; gitnexus auto-sync init creates its default $GITNEXUS_HOME/watch_config.yml. Bare gitnexus auto-sync is the same as gitnexus auto-sync start; restart, stop, status, and reset manage the same GITNEXUS_HOME instance. reset removes only the derived analysis state and commit snapshot; clones, indexes, and registry entries are untouched. start runs in the foreground, reads the configuration once at startup, runs once immediately, then repeats on sync_interval_minutes; restart it after changing the configuration. Watch runtime artifacts live under $GITNEXUS_HOME/watch/: project_commit_info.txt is the human-readable per-loop snapshot, auto-sync-state.json is the machine state used for commit skipping and analyze failure thresholds, watch.mutex prevents multiple auto-sync processes for one home, watch.owner.json records ownership metadata, watch.pid plus watch.status.json expose process state, watch.stop.<ownerId>.json is a temporary owner-fenced stop request, and quarantine/ stores partial clone output before entries are removed after 14 days, keeping at most the five newest entries per repository regardless of age. Mutexes with verified dead owners are reclaimed automatically after an abnormal exit. Invalid or legacy mutexes fail closed; confirm no auto-sync process is running before manually removing watch.mutex and stale watch.pid / watch.owner.json.

sync_interval_minutes: 10
max_concurrency: 1
repo_git_timeout: 10s
analyze_timeout: 5m
analyze_failure_threshold: 3
projects:
  - local_path: /abs/path/to/repos
    branches: [master, main]
    overwrite_local_changes: false
    remote_urls:
      - git@github.com:owner/repo.git
      - git@gitlab.com:group/repo.git
      - git@gitee.com:owner/repo.git

sync_interval_minutes must be an integer of at least 5. local_path must be an absolute path without traversal; each remote is cloned below it as host/namespace/repo, preventing same-basename repositories from colliding. remote_urls must use SSH SCP form for github.com, gitlab.com, or gitee.com. repo_git_timeout applies to each repo clone/pull and defaults to 10s; a bare number such as 10 is interpreted as seconds, while 10000ms, 10s, and 1m keep their explicit units. It must not exceed one hour or sync_interval_minutes, whichever is smaller — so a bare 600000 is rejected, because it means 600000 seconds rather than milliseconds. analyze_timeout applies to each isolated analysis worker, defaults to half of sync_interval_minutes, and cannot exceed that value; this keeps it within Node's timer range. Timeout and auto-sync stop request safe cancellation; a worker already in native work exits after it returns to a JS-visible safe point. While waiting, auto-sync reports cancelling or stopping and keeps its ownership files so another auto-sync cannot take over. The parent waits up to 5 seconds for the worker to exit; after that it stops waiting, releases its ownership files, and leaves the worker to finish and exit on its own rather than killing it mid-write. auto-sync stop uses this same control path on macOS and Windows. overwrite_local_changes defaults to false; a dirty local clone is skipped with an error log, while true allows branch fallback to replace local changes and additionally discards untracked files and directories in the clone after checkout — ignored paths, including GitNexus's own .gitnexus/ storage, are preserved. max_concurrency defaults to 1 and is capped at runtime by floor(availableMemoryGB / 2) with a minimum of 1; the effective value is printed at the start of each loop. Each analysis worker's heap cap is the machine-wide cap divided by the number of repositories analyzed in parallel, so concurrent workers share one memory budget instead of each claiming the whole machine. analyze_failure_threshold defaults to 3, must be at least 2, and pauses repeated failures only for the same repo branch and commit; a new commit or gitnexus auto-sync reset clears the block and allows analysis again. Repositories are registered and added to groups by their full remote identity (host/namespace/repo), so repositories with the same basename remain distinct. Use branches to try branches in order; legacy branch remains supported, but the two fields cannot be set together. If all branches are unavailable or time out, watch logs an error, records the repo status, and skips that repo for the loop. Leave group_name empty or omit it to skip group add/sync for that project; otherwise create the group first with gitnexus group create <name>. $GITNEXUS_HOME/watch/project_commit_info.txt is for inspection only; GitNexus stores machine state separately in $GITNEXUS_HOME/watch/auto-sync-state.json.

GraphQL contract matching is opt-in in the group's group.yaml:

detect:
  graphql: true

The initial exact-only slice matches methods and properties on top-level NestJS @Resolver classes using imported @Query, @Mutation, and @Subscription decorators. Named .graphql/.gql operations are anchored by generated <OperationName>Document declarations; object, static gql template, and TypedDocumentString initializers must prove the operation name and root fields. Dynamic decorator names, anonymous operations, and ambiguous or missing graph anchors are deliberately omitted. Add common infrastructure fields such as /health to matching.exclude_links_paths to keep those GraphQL contracts visible without cross-linking them.

--spring-actuator is explicitly opt-in. The path may be a JSON bundle keyed by mappings, beans, conditions, configprops, and/or env, or a directory containing endpoint-named JSON files. Runtime mappings and beans confirm matching static nodes; conditions and configuration property keys enrich existing evidence, with conservative runtime-only nodes added when no match exists. The configured input is excluded from source scanning; only normalized repository-relative exclusions are retained for future scans, never absolute paths. Env/configprops values, origins, condition messages, and source names are never persisted or printed. Enabled runs always rebuild because runtime snapshots are external to git freshness; omitting the option later rebuilds once to remove runtime evidence. Project config can set the same path with springActuator in .gitnexusrc.

--asyncapi-spec is explicitly opt-in and accepts a directory of AsyncAPI documents or a single document; the path is resolved against the repository root, so a committed docs/asyncapi and an absolute cache written by something else both work. Each operations[] entry of an AsyncAPI 3.x document can contribute a Destination node keyed by broker and address, with action: send emitting PUBLISHES_TO and action: receive emitting CONSUMES_FROM, so a document and source code that name one address on one broker land on the same node. Edges start at the document, not at a callable — a document states that the service talks to an address, not which method does — and no address a document names is ever attached to an unresolved source site.

An operation must name a protocol, either through its own bindings or through the servers[].protocol of the servers its channel resolves to (a channel that lists no servers resolves to all of them); operations that name none are refused, as are operations whose two readings name different brokers, and channels that inherit a multi-protocol server set without choosing. HTTP and WebSocket documents are refused for destination minting: there the host rather than the address names the place, and an HTTP endpoint is already modelled as a Route. A parameterized address — a channel declaring parameters, or an address containing { — is refused rather than keyed: two services publishing {env}.orders share a pattern, not a queue. AsyncAPI 2.x is refused under its own counted reason and never mapped, because its publish/subscribe are inverted relative to 3.x send/receive and a naive mapping would reverse the async graph while leaving it connected. Every refusal is counted, and a configured path that yields nothing is reported rather than passed over in silence.

Like Actuator snapshots, documents are external to git freshness — replacing one moves no commit and dirties no file — so an enabled run always rebuilds, and the first later run without the option rebuilds once to remove document-derived evidence. There is no glob-based auto-discovery, and the option is unsupported with --watch.

gitnexus uninstall reverses gitnexus setup — it removes the GitNexus MCP entries, hooks, and skill directories it added to each detected editor. Skill directories are identified by bundled gitnexus skill name (e.g. gitnexus-cli/), so if you customized files inside an installed skill directory, back them up first. It is a dry-run preview by default and prints the exact paths it would remove; pass --force to apply. Per-repo indexes (gitnexus clean --all) and the global npm package (npm uninstall -g gitnexus) are left for you to remove.

Remote Embeddings

Set these env vars to use a remote OpenAI-compatible /v1/embeddings endpoint instead of the local model:

export GITNEXUS_EMBEDDING_URL=http://your-server:8080/v1
export GITNEXUS_EMBEDDING_MODEL=BAAI/bge-large-en-v1.5
export GITNEXUS_EMBEDDING_DIMS=1024          # optional, default 384
export GITNEXUS_EMBEDDING_REQUEST_DIMS=omit  # optional: omit "dimensions", or an integer to override it
export GITNEXUS_EMBEDDING_API_KEY=your-key   # optional, default: "unused"
export GITNEXUS_EMBEDDING_MAX_ATTEMPTS=3     # optional, total attempts (1-20)
export GITNEXUS_EMBEDDING_RETRY_CAP_MS=5000  # optional, maximum retry delay
export GITNEXUS_EMBEDDING_MIN_INTERVAL_MS=0  # optional, minimum request spacing
export GITNEXUS_EMBEDDING_HTTP_TIMEOUT_MS=180000 # optional, per-request timeout (max 300000)
gitnexus analyze . --embeddings

GITNEXUS_EMBEDDING_REQUEST_DIMS controls only the dimensions field sent in the request body, independently of GITNEXUS_EMBEDDING_DIMS (which still validates the returned vector's length):

omit (or none, off, false, 0) — do not send dimensions at all, for strict backends that return the right vector size but reject the field.
a positive integer — send that value instead of GITNEXUS_EMBEDDING_DIMS.
unset — send GITNEXUS_EMBEDDING_DIMS (the previous behavior).

Works with Infinity, vLLM, TEI, llama.cpp, Ollama, LM Studio, or OpenAI. Retry and pacing settings are provider-neutral; provider-specific limits should be supplied through configuration. When unset, local embeddings are used unchanged.

JVM Package Sibling Injection

Java and Kotlin files in the same package receive implicit sibling class bindings to resolve same-package references. By default, GitNexus injects at most 200 siblings per module scope, nearest first by path. Set GITNEXUS_MAX_INJECTED_SIBLINGS=0 to remove that per-file limit; this can substantially increase indexing work for large packages.

export GITNEXUS_MAX_INJECTED_SIBLINGS=200
gitnexus analyze .

When the limit truncates a file's sibling set, that file is marked visibility-incomplete: same-package references still resolve through the injected siblings, but wildcard-import attribution (used by the Spring bean/DI/config passes) is disabled for it rather than resolved against a partial view. Analyze logs a sibling injection truncated warning naming how many files were affected.

Packages with more than 500 files are a separate, fixed limit: they are skipped entirely (logged as skipping package with N files) and every file in them is marked visibility-incomplete. GITNEXUS_MAX_INJECTED_SIBLINGS does not lift that skip — including at 0.

Multi-Repo Support

GitNexus supports indexing multiple repositories. Each gitnexus analyze registers the repo in a global registry (~/.gitnexus/registry.json). The MCP server serves all indexed repos automatically.

Supported Languages

TypeScript, JavaScript, Python, Java, C, C++, C#, Go, Rust, PHP, Kotlin, Swift, Ruby, Dart, Zig

Language Feature Matrix

Language	Imports	Named Bindings	Exports	Heritage	Type Annotations	Constructor Inference	Config	Frameworks	Entry Points
TypeScript	✓	✓	✓	✓	✓	✓	✓	✓	✓
JavaScript	✓	✓	✓	✓	—	✓	✓	✓	✓
Python	✓	✓	✓	✓	✓	✓	✓	✓	✓
Java	✓	✓	✓	✓	✓	✓	—	✓	✓
Kotlin	✓	✓	✓	✓	✓	✓	—	✓	✓
C#	✓	✓	✓	✓	✓	✓	✓	✓	✓
Go	✓	—	✓	✓	✓	✓	✓	✓	✓
Rust	✓	✓	✓	✓	✓	✓	—	✓	✓
PHP	✓	✓	✓	—	✓	✓	✓	✓	✓
Ruby	✓	—	✓	✓	—	✓	—	✓	✓
Swift	—	—	✓	✓	✓	✓	✓	✓	✓
C	—	—	✓	—	✓	✓	—	✓	✓
C++	—	—	✓	✓	✓	✓	—	✓	✓
Dart	✓	—	✓	✓	✓	✓	—	✓	✓
Zig	✓	—	✓	—	✓	✓	✓	—	✓

Imports — cross-file import resolution · Named Bindings — import { X as Y } / re-export tracking · Exports — public/exported symbol detection · Heritage — class inheritance, interfaces, mixins · Type Annotations — explicit type extraction for receiver resolution · Constructor Inference — infer receiver type from constructor calls (self/this resolution included for all languages) · Config — language toolchain config parsing (tsconfig, go.mod, etc.) · Frameworks — AST-based framework pattern detection · Entry Points — entry point scoring heuristics

Agent Skills

GitNexus ships with skill files that teach AI agents how to use the tools effectively:

Exploring — Navigate unfamiliar code using the knowledge graph
Debugging — Trace bugs through call chains
Impact Analysis — Analyze blast radius before changes
Refactoring — Plan safe refactors using dependency mapping
Guide — GitNexus tool/resource/schema reference for the agent
CLI — Run analyze/status/clean/wiki commands on request
PDG Query — Statement-level control/data dependence queries (--pdg index)
Taint Analysis — Source→sink data-flow findings (--pdg index)
Plan / Work / Review / LFG — The engineering family: implementation-ready plans, gated plan execution, graph-backed change review with taint + expert lenses, and the end-to-end pipeline

Installed automatically by both gitnexus analyze (per-repo) and gitnexus setup (global). Run gitnexus analyze --skills to additionally generate each detected functional area as a direct project skill under .claude/skills/gitnexus-area-<name>/.

Requirements

Node.js >= 22
Git repository (uses git for commit tracking)
Linux: glibc 2.34 or newer (Ubuntu 22.04+, RHEL/Rocky/Alma 9+, Debian 12+, Fedora 35+). The LadybugDB native binary ships as a prebuild against that floor, so on an older host it cannot load and reinstalling does not help — see Linux: GLIBC_2.34' not found.
Windows, for full-text search: the Microsoft Visual C++ 2015-2022 Redistributable (x64) and OpenSSL 3 (libssl-3-x64.dll, libcrypto-3-x64.dll) resolvable on PATH — see Windows: full-text search unavailable.

Release candidates

Stable releases publish to the default latest dist-tag. When a pull request with non-documentation changes merges into main, an automated workflow also publishes a prerelease build under the rc dist-tag, so early adopters can try in-flight fixes without waiting for the next stable cut. (Docs-only merges are skipped.)

# Try the latest release candidate (pre-stable — may change at any time)
npm install -g gitnexus@rc
# — or —
npx gitnexus@rc analyze

Release-candidate versions follow the standard semver prerelease format X.Y.Z-rc.N, where X.Y.Z is the next stable target (bumped from the current latest by patch by default; minor or major when kicking off a bigger cycle) and N increments per published rc. Example sequence: 1.6.2-rc.1, 1.6.2-rc.2, …, then once 1.6.2 ships stable, 1.6.3-rc.1. See the Releases page for the full list; stable latest is unaffected.

Update notifications

GitNexus checks the npm registry's latest dist-tag at most once every 24 hours per installation and tells you when a newer stable version exists. The result is cached under $GITNEXUS_HOME (~/.gitnexus by default), so the check never runs on the command's hot path and never blocks output. Where the notice appears:

CLI — one line on stderr when you run a command interactively (never on stdout, so gitnexus query … | jq and other piped output stay clean), a line in gitnexus doctor when an update is known. Automatic notices never install. gitnexus update checks even when notices are opted out, then runs npm i -g gitnexus@<x.y.z> (same idea as claude update / codex update).
MCP server — one structured log record on the server's stderr per process per version (visible in your host's MCP log panel). Tool results, resources, prompts, and server instructions never carry update text.
Web UI — a dismissible banner when the server reports a newer version; dismissal persists per version.

The check is skipped entirely (no network request, no output) when CI is truthy, when the install is not an npm global/local install (npx cache, dev checkout, Docker image — the Docker CLI image sets the opt-out itself), or when opted out:

Variable	Effect
`GITNEXUS_NO_UPDATE_NOTIFIER`	Truthy (`1`, `true`, …) disables the update check on every surface.
`NO_UPDATE_NOTIFIER`	Cross-tool convention; honored the same way.
`npm_config_registry`	The check reads the `latest` dist-tag from this registry instead of `https://registry.npmjs.org`. Credentials are never sent, and registries that require authentication are not supported (the check silently skips).

Eval harnesses running a global install can set GITNEXUS_NO_UPDATE_NOTIFIER for a quiet registry.

Troubleshooting

`Cannot destructure property 'package' of 'node.target' as it is null`

This error comes from npm 11.x's arborist while installing gitnexus (often via npx), before gitnexus code runs. It is triggered by platform-filtered optionalDependencies in native packages such as onnxruntime-node / @huggingface/transformers (used when indexing with --embeddings). GitNexus cannot catch it at runtime — use one of these workarounds:

pnpm --allow-build=@ladybugdb/core --allow-build=gitnexus --allow-build=tree-sitter dlx gitnexus@latest analyze       # auto-selected when pnpm + npm 11+
npm install -g gitnexus@latest         # global install avoids per-run npx reify
gitnexus analyze                       # if already installed globally

On pnpm 10+, lifecycle scripts are blocked unless explicitly allowed — the resolver adds --allow-build for @ladybugdb/core, gitnexus, and tree-sitter automatically when it picks pnpm dlx.

If you must stay on npm 11.x without pnpm, downgrade npm toolchain-wide (last resort):

npm install -g npm@10.9.0

See #1939 and the original #819 thread. An older variant of this crash (tree-sitter-dart tarball URL) was fixed in gitnexus v1.6.2+ (#820); if you still see install failures after upgrading, clear cache:

npm cache clean --force
npx gitnexus@latest analyze

`ERR_DLOPEN_FAILED` / `lbugjs.node` missing (pnpm dlx, pnpx)

GitNexus depends on @ladybugdb/core, whose native database addon (lbugjs.node) is placed by a postinstall script. pnpm dlx, pnpx, and any install run with --ignore-scripts skip lifecycle scripts, so the addon is never put in place and the runtime crashes with ERR_DLOPEN_FAILED:

Error: dlopen(.../@ladybugdb/core/lbugjs.node, ...): tried: '...' (no such file)
  code: 'ERR_DLOPEN_FAILED'

Options that run install scripts:

# pnpm dlx with explicit build permission (one-off, no global install required)
pnpm --allow-build=@ladybugdb/core --allow-build=gitnexus --allow-build=tree-sitter \
  dlx gitnexus@latest serve

# npm: global install (recommended on npm 11+; bare npx may crash — see section above)
npm install -g gitnexus@latest
gitnexus serve

# npx (npm < 11, or after upgrading npm)
npx gitnexus@latest serve

# pnpm: global install with build scripts allowed (pnpm 10.2+; no approve-builds -g on pnpm 11+)
pnpm add -g --allow-build=@ladybugdb/core --allow-build=gitnexus --allow-build=tree-sitter gitnexus
gitnexus serve

Linux: `GLIBC_2.34' not found`

LadybugDB native binary (lbugjs.node) exists but failed to load:
  /lib64/libc.so.6: version `GLIBC_2.34' not found (required by .../lbugjs.node)

The LadybugDB addon ships as a prebuilt binary compiled against glibc 2.34. If your distribution is older (CentOS/RHEL 8 has 2.28, Ubuntu 20.04 has 2.31, Debian 11 has 2.31), the dynamic loader cannot resolve its symbols.

Reinstalling does not help — every download delivers the same prebuilt binary. The fix is a newer C library:

Run GitNexus on a distribution with glibc 2.34 or newer — Ubuntu 22.04+, RHEL/Rocky/Alma 9+, Debian 12+, Fedora 35+.
Or run it in the container image, which bundles a current glibc (see Docker).

gitnexus doctor reports the required and detected glibc versions when this happens (#2672).

Windows: full-text search unavailable

analyze completes, but keyword search is degraded and doctor shows the FTS extension failing with Windows error 126 (The specified module could not be found). The extension needs two runtime dependencies Windows does not ship by default:

Microsoft Visual C++ 2015-2022 Redistributable (x64) — https://aka.ms/vs/17/release/vc_redist.x64.exe
OpenSSL 3 — libssl-3-x64.dll and libcrypto-3-x64.dll, resolvable on PATH

The redistributable alone is not sufficient. If Git for Windows is installed you already have the OpenSSL DLLs — run gitnexus from Git Bash, or prepend the directory to PATH in the shell you use:

$env:PATH = "C:\Program Files\Git\mingw64\bin;$env:PATH"
gitnexus analyze --repair-fts

Without them the index is still built, but without search tables, so query returns empty keyword results until you re-run gitnexus analyze --repair-fts from a shell where the DLLs resolve (#2669).

Installation fails with native module errors

Some optional language grammars (Dart, Proto, Swift, Kotlin, Zig) ship vendored native prebuilds. If a prebuild is missing and a source build is not possible, GitNexus still works — those languages will be skipped. To skip them intentionally (no C++ toolchain needed), set GITNEXUS_SKIP_OPTIONAL_GRAMMARS=1 before installing.

If npm install -g gitnexus fails on native modules:

# Ensure build tools are available (Linux/macOS)
# Ubuntu/Debian: sudo apt install python3 make g++
# macOS: xcode-select --install

# Retry installation
npm install -g gitnexus

Installation fails behind an HTTP proxy (`onnxruntime-node` postinstall)

onnxruntime-node's postinstall downloads optional CUDA GPU binaries from api.nuget.org — outside the npm registry, so registry mirrors don't cover it, and its proxy layer (global-agent) ignores the standard HTTP_PROXY/HTTPS_PROXY variables and rejects 302 redirects (#2370).

Since the packages are optional dependencies, a failed download no longer breaks npm install -g gitnexus — npm skips the embedding stack and everything else works. The stack then self-heals on demand: the first gitnexus analyze --embeddings (or an explicit gitnexus embeddings install) fetches it through your configured npm registry — mirrors and proxies apply, no NuGet download involved — into ~/.gitnexus/embedding-runtime.

# heal a proxy-degraded install manually (CPU embeddings; registry-only)
gitnexus embeddings install

# reinstall into the prefix even when the stack already resolves
gitnexus embeddings install --force

# CUDA GPU hosts: also fetch GPU binaries (NuGet; set the proxy global-agent reads)
GLOBAL_AGENT_HTTPS_PROXY=<proxy-url> gitnexus embeddings install --cuda

The prefix defaults to ~/.gitnexus/embedding-runtime; set GITNEXUS_EMBEDDING_RUNTIME_DIR to install it elsewhere (e.g. a writable path in a container).

Node requirement for the on-demand prefix: the self-heal loads the prefixed packages via module.registerHooks, available on Node ≥ 22.15 (on the 22.x line) or ≥ 23.5 (on the 23.x line). On an older Node the packages install but can't be loaded from the prefix — reinstall them into the install itself instead (works on every supported Node): ONNXRUNTIME_NODE_INSTALL=skip npm install -g gitnexus (Windows: set ONNXRUNTIME_NODE_INSTALL=skip && npm install -g gitnexus). Skipping only the CUDA download keeps full CPU embeddings (CPU embeddings don't need it). Check the result any time with gitnexus doctor (Embeddings → Support line).

Analyze warns about unavailable FTS or VECTOR extensions

GitNexus uses optional DuckDB extensions for BM25 and vector search. The gitnexus serve and MCP read paths only ever try to LOAD the extensions — they never block on a network install. The analyze command, by default, attempts one bounded out-of-process install if LOAD fails (a plain INSTALL to download a missing extension, escalating to FORCE INSTALL only when the LOAD error shows the existing file is broken or truncated, so a permanent non-file failure does not re-download on every run) and proceeds even when that install times out, so the index is always written to disk; BM25/vector search degrade gracefully until the extensions become available.

Configure the behavior with these environment variables:

Variable	Values	Default	Effect
`GITNEXUS_LBUG_EXTENSION_INSTALL`	`auto`, `load-only`, `never`	`auto`	`auto` runs one bounded install if LOAD fails — a plain `INSTALL`, escalating to `FORCE INSTALL` only when the LOAD error shows the present extension file is broken. `load-only` only uses already-installed extensions (recommended for offline / firewalled environments). `never` skips optional extensions entirely.
`GITNEXUS_LBUG_EXTENSION_INSTALL_TIMEOUT_MS`	positive integer	`15000`	Wall-clock budget for the out-of-process extension-install child before it is killed.
`GITNEXUS_FTS_STEMMER`	supported LadybugDB stemmer	`porter`	Stemmer used when rebuilding BM25/FTS indexes. Use `none` for CJK-heavy repositories, or a language stemmer such as `german`, `french`, or `spanish` when that better matches repository comments and identifiers. Re-run `gitnexus analyze --repair-fts` after changing it.
`GITNEXUS_FTS_CJK_SEGMENTATION`	`none`, `bigram`	`none`	`bigram` inserts overlapping character-bigram boundaries into Chinese/Japanese Han-ideograph spans in `content`/`description` before FTS indexing, so LadybugDB's space-only tokenizer can see sub-phrase word boundaries. Scoped to CJK Unified Ideographs only — Japanese Hiragana/Katakana and Korean Hangul are not currently segmented. Unlike `GITNEXUS_FTS_STEMMER`, this rewrites stored text — enabling it on an already-indexed repo requires a full `gitnexus analyze --force`; neither `--repair-fts` nor a plain incremental `analyze` applies it to previously-indexed files. Set the same value wherever `analyze` and search-serving processes (CLI query, MCP server, web server) run.
`GITNEXUS_STREAM_GRAPH_EMIT`	`0`, `1`	`1` (on)	On by default on a full rebuild (`--force`); incremental runs ignore it. Holds structural relationships (CALLS, IMPORTS, ACCESSES, CONTAINS, ...) as CSV-on-disk plus compact in-memory columns instead of as objects in three overlapping indexes, cutting peak in-memory graph heap by ~1.4x at no measurable CPU cost (measured A/B on a synthetic 400k-node / 1.08M-edge graph: 819 MB -> 584 MB, iteration at parity, scaling verified linear from 100k to 800k nodes, with every edge still visible through the graph interface; no end-to-end measurement on a real repository yet). Nothing is traded away — community detection, process extraction, PDG taint summaries and the local-symbol pruner all read a complete relationship set and behave identically. Set to `0` only to bisect a suspected streaming-related fault.
`GITNEXUS_COMMUNITY_ENGINE`	`graphology`, `icebug`, `auto`	`graphology`	Community-detection engine used during analyze. `graphology` is the supported default. `icebug` and `auto` are experimental and currently behave identically: both try the optional `@ladybugmem/icebug` native Leiden over a CSR export and fall back to Graphology if it is not installed, cannot load, or lacks the deterministic thread/seed controls. Experimental engines partition differently, so community IDs are not comparable across engines.
`GITNEXUS_WAL_CHECKPOINT_THRESHOLD`	integer `>= -1`	`67108864` (64 MiB)	LadybugDB WAL auto-checkpoint threshold during analyze (bytes). Auto-checkpoint remains enabled; `-1` keeps Ladybug's stock ~16 MiB. Larger thresholds reduce checkpoint frequency but increase the WAL size at rotation time — choose a smaller value on disk-constrained environments.
`GITNEXUS_LBUG_BUFFER_POOL_SIZE`	integer `>= 0` (bytes)	min(2 GiB, 80% RAM)	LadybugDB buffer-pool ceiling for every GitNexus database (analyze, MCP server, serve, group bridges). Bounded so a long-lived `gitnexus mcp` process or a large incremental `analyze` cannot grow toward LadybugDB's native 80%-of-RAM default and OOM the host (#2557). `0` restores that native unbounded default; invalid values warn and fall back to the default. During `analyze` the pool is right-sized to the graph and, on non-4 KiB-page hosts (Apple Silicon 16 KiB, Ascend/aarch64 64 KiB), scaled by the page-size granule ratio up to min(2 GiB × pageSize/4 KiB, 80% RAM) (#2631); this env var overrides all of that as an absolute value.
`GITNEXUS_LBUG_MAX_DB_SIZE`	positive integer (bytes)	`17179869184` (16 GiB)	Upper bound for a single LadybugDB database file. This is an mmap/disk-address-space ceiling, not a memory limit — it does not constrain the buffer pool (use `GITNEXUS_LBUG_BUFFER_POOL_SIZE` for that). Raise it when indexing genuinely huge monorepos; invalid values silently fall back to the default.

# Offline/airgapped: never reach the network for extensions
GITNEXUS_LBUG_EXTENSION_INSTALL=load-only npx gitnexus analyze

# Slow network: give extension downloads more time
GITNEXUS_LBUG_EXTENSION_INSTALL_TIMEOUT_MS=30000 npx gitnexus analyze

# CJK-heavy codebase: rebuild keyword indexes without English stemming
GITNEXUS_FTS_STEMMER=none npx gitnexus analyze --repair-fts

# CJK-heavy codebase: enable sub-phrase search over Chinese/Japanese Han text.
# On an already-indexed repo, the first run after enabling this MUST be --force —
# --repair-fts and plain incremental `analyze` both leave old files un-segmented.
GITNEXUS_FTS_CJK_SEGMENTATION=bigram npx gitnexus analyze --force

Analysis runs out of memory

Memory management is automatic: analyze sizes its heap to the machine (always below physical RAM), caps each parse worker, and — rather than grinding into a GC death spiral or crash — stops early with a message telling you the one thing to do. Repeated Replacement worker did not report ready within 5000ms warnings on a large repository are part of the same picture: memory pressure starving healthy workers, not a worker bug (#2649).

If analyze says the repository doesn't fit, do what the message says:

The machine has more memory to give (a NODE_OPTIONS --max-old-space-size pin from your environment is holding analyze back): re-run without the pin — no flags needed.
The machine is the ceiling: shrink the scope (exclude generated or vendored directories, below) or use a machine with more RAM.

Escape hatches (GITNEXUS_MEMORY=off to decline the autopilot, GITNEXUS_WORKER_HEAP_MB to size workers yourself) are listed in the environment-variable table below — most users never need them.

For very large repositories:

# Increase Node.js heap size
NODE_OPTIONS="--max-old-space-size=16384" npx gitnexus analyze

# Exclude large directories (this repo only)
echo "vendor/" >> .gitnexusignore
echo "dist/" >> .gitnexusignore

# Exclude a directory across every repo you index, without touching each
# repo's own .gitnexusignore or needing push/commit access to it. GitNexus
# reads the same sources `git` itself does: core.excludesFile (all repos)
# and $GIT_DIR/info/exclude (this repo only, untracked). A repo's own
# .gitignore/.gitnexusignore can still override either with a `!pattern`
# negation. Skip both entirely with GITNEXUS_NO_GLOBAL_IGNORE=1.
git config --global core.excludesFile ~/.gitignore_global   # applies to every repo
echo "docs/" >> ~/.gitignore_global
echo "build/" >> .git/info/exclude                          # this repo only, untracked

Large files are being skipped

By default the walker skips files larger than 512 KB (see log line Skipped N large files (>512KB)). Raise the threshold via either the CLI flag or the environment variable — both accept a value in KB:

# CLI flag (takes precedence over the env var)
npx gitnexus analyze --max-file-size 2048     # skip only files > 2 MB

# Environment variable (persists across commands)
export GITNEXUS_MAX_FILE_SIZE=2048
npx gitnexus analyze

Values above 32768 KB (32 MB) are clamped to the tree-sitter parser ceiling; invalid values fall back to the 512 KB default with a one-time warning. When an override is active, analyze prints the effective threshold in its startup banner (e.g. GITNEXUS_MAX_FILE_SIZE: effective threshold 2048KB (default 512KB)).

Analyze reports a worker timeout

Worker parse timeouts are recoverable. GitNexus retries stalled worker jobs with backoff, splits large jobs to isolate slow files, and quarantines a file that repeatedly crashes its worker (respawning the slot so the pool keeps going). If a large repository needs more time per worker job, use either:

# CLI flag, in seconds
npx gitnexus analyze --worker-timeout 60

# Environment variable, in milliseconds
export GITNEXUS_WORKER_SUB_BATCH_TIMEOUT_MS=60000
npx gitnexus analyze

For repositories with very large source files, GITNEXUS_WORKER_SUB_BATCH_MAX_BYTES controls the worker job byte budget. The default is 8388608 bytes (8 MB).

Worker pool resilience tuning

Four env vars expose the pool's resilience layers (respawn budget, cumulative-timeout cap, circuit breaker, startup handshake). Defaults are tuned for typical repos; bump them when an analyze legitimately needs more retries, or lower them to fail-fast on a known-bad shape.

Variable	Default	Effect
`GITNEXUS_WORKER_MAX_RESPAWNS_PER_SLOT`	`3`	Max replacement spawns per slot before the slot is dropped from the active rotation.
`GITNEXUS_WORKER_MAX_CUMULATIVE_TIMEOUT_MS`	`5 × subBatchTimeoutMs`	Total retry wall-time budget per job before quarantining. Bounds exponentially-growing retry waits.
`GITNEXUS_WORKER_CONSECUTIVE_FAILURE_THRESHOLD`	`max(3, poolSize)`	Per-slot consecutive deaths before the pool's circuit breaker trips. After tripping, dispatches require a fresh pool.
`GITNEXUS_WORKER_SHUTDOWN_DRAIN_MS`	`30000`	Max wait at pool shutdown for a retired worker still inside native code — terminated at its next JS-safe point instead of mid-native-call, which would abort the process (`Napi::Error`, #2432).
`GITNEXUS_WORKER_READY_TIMEOUT_MS`	`5000`	Startup budget for a parse worker to load its grammar bindings and report `{type:'ready'}`. Slots that miss it are treated as startup crashes. Raise it on a slow or heavily loaded host where a full pool cold-starting concurrently needs more than 5s.
`GITNEXUS_MEMORY`	`off`	unset (autopilot on)	`off` declines GitNexus's memory autopilot: analyze will neither re-run itself with a RAM-aware heap cap nor abort the parse before V8 enters its ineffective-mark-compact death spiral. Use it when you want to drive memory manually; to simply pin a heap size, pass Node's own `--max-old-space-size`, which is already honoured as your decision.
`GITNEXUS_WORKER_HEAP_MB`	`clamp(512, RAM/2/poolSize, 4096)`	Per-worker V8 old-generation heap cap (#2649). Bounds pool RSS on large repos; a worker exceeding it dies with a real heap error handled by quarantine/respawn.
`GITNEXUS_SERVER_ANALYZE_HEAP_MB`	`min(8192, auto cap)`	Heap for the web/MCP server's forked analyze worker (#2649). Defaults to the historical 8192 MB bounded by the machine/container's RAM-aware auto cap; set an absolute MB value to override.
`GITNEXUS_CPP_CAPTURE_BUDGET_MS`	`20000`	Per-file wall-clock budget for C++ capture extraction; on breach the file keeps partial captures with a warning (#2432). `0` expires immediately.

Graph cleanup tuning

After scope resolution, analyze prunes inert block-local value symbols (a function-local const/let/var that ends up with only its structural File→DEFINES edge) to keep the graph focused on cross-symbol relationships. Module/file-scope symbols, class members, and any local with a real edge are always kept.

Variable	Default	Effect
`GITNEXUS_KEEP_LOCAL_VALUE_SYMBOLS`	unset	Set to `1`/`true` to keep inert block-local value symbols instead of pruning them.

Programmatic callers can pass keepLocalValueSymbols: true in PipelineOptions instead of setting the env var.

Scope-resolution property-key dispatch cap

During scope resolution GitNexus synthesizes CALLS edges through property-key dispatch — call sites like hooks.emitScopeCaptures() where a property key is registered by multiple definitions across the codebase. To keep this fan-in bounded, each property key is capped at 32 registrations: a key registered by more than 32 distinct functions is skipped entirely (no CALLS are synthesized through it), and the dropped key names are surfaced in the analyze log for operator visibility. The cap is calibrated at 2× this repo's own provider table (16 legitimate registrations, one per language provider).

Variable	Default	Effect
`GITNEXUS_MAX_PROPERTY_DISPATCH_FANOUT`	`32`	Per-property-key registration cap in the property-dispatch scope-resolution pass. Set to a positive integer to raise it for repositories whose provider/hook tables exceed the default and lose CALLS coverage on a legitimate key; non-integer or `< 1` values fall back to `32`. Lowering it tightens the overflow budget.

# A property key registered by 40 functions overflows the default 32 and drops
# all CALLS through it — raise the cap for that repo and rebuild so scope
# resolution reruns.
export GITNEXUS_MAX_PROPERTY_DISPATCH_FANOUT=64
npx gitnexus analyze --force

Scope-resolution dispatch-target cap

During scope resolution GitNexus resolves calls that flow through callable values — function/method references bound to variables, passed as arguments, or stored in maps/tables. To keep that inclusion-based resolution finite, each callable site is capped at 32 dispatch targets. When a site gathers more candidates than the cap it is treated as overflowed and all of its call edges are dropped — a cliff, not a tail, so a repository with a legitimately wide dispatch table (a single callable site resolving to 33+ targets) loses that site's whole call chain. In that case analyze logs callable-value-flow: candidate set exceeded the cap; no partial CALLS emitted alongside a warning carrying the language, the overflowing context, the candidate count, and the cap (32).

Raise the cap for such repositories:

Variable	Default	Effect
`GITNEXUS_MAX_CALLABLE_VALUE_TARGETS`	`32`	Per-callable-site dispatch-target cap in the callable-value-flow scope-resolution pass. Set to a positive integer to raise it for repositories whose wide dispatch tables overflow the default and lose a whole call chain; non-integer or `< 1` values fall back to `32`. Lowering it tightens the overflow budget.

# A callable site resolving to 48 targets overflows the default 32 and drops
# the chain — raise the cap for that repo and rebuild so scope resolution reruns.
export GITNEXUS_MAX_CALLABLE_VALUE_TARGETS=64
npx gitnexus analyze --force

Hook augmentation and skip diagnostics

The Claude Code / Antigravity hooks keep their stderr silent on normal skip paths so strict hook runners (e.g. Codex PreToolUse) never see unexpected diagnostic output.

When a GitNexus process holds the repo DB write lock (the common case — the MCP server is running, or the DB-lock probe timed out and failed closed), the local CLI augment can't run (LadybugDB is single-writer). Rather than drop the augmentation, the hook hands the agent a short, conditional MCP-query hint on stdout (the sanctioned additionalContext channel) — "if the GitNexus MCP tools are live in this session, call query …" — so an agent that has the tools can still fetch graph-ranked context. The hint is throttled to at most once per repo per window (GITNEXUS_MCP_HINT_THROTTLE_MS, default 10 min; 0 disables), so an owner-locked session isn't nudged on every search. A stale-index reminder, or an already-current index, stays silent.

To see why a hook skipped the CLI augment, set GITNEXUS_DEBUG=1 and re-run the action — the hook writes the reason (e.g. [GitNexus] augment skipped: MCP server owns DB) and the stale-index hint to its stderr:

GITNEXUS_DEBUG=1 <your command>   # surfaces hook skip/diagnostic reasons on stderr

Only GITNEXUS_DEBUG=1 and GITNEXUS_DEBUG=true enable diagnostics; every other value (including 0 and false) is treated as off. Diagnostics go to stderr only — the hook's structured stdout (the JSON the agent consumes) is unaffected.

Privacy

All processing happens locally on your machine
No code is sent to any server
Index stored in .gitnexus/ inside your repo (gitignored)
Global registry at ~/.gitnexus/ stores only paths and metadata

Web UI

GitNexus also has a browser-based UI at gitnexus.vercel.app — 100% client-side, your code never leaves the browser.

Local Backend Mode: Run gitnexus serve and open the web UI locally — it auto-detects the server and shows all your indexed repos, with full AI chat support. No need to re-upload or re-index. The agent's tools (Cypher queries, search, code navigation) route through the backend HTTP API automatically.

License

PolyForm Noncommercial 1.0.0

Free for non-commercial use. Contact for commercial licensing.

README.md Unescape Escape