mirror of https://github.com/abhigyanpatwari/GitNexus.git synced 2026-10-28 06:22:22 +00:00

History

Gergő Magyar abada2ccd1 perf(import-resolvers): index every scanning resolver, consolidate the memo, gate every registered language (#2911 ) * perf(import-resolvers): build buildSuffixIndex's dirMap lazily (#2903) `buildSuffixIndex` eagerly built three maps. `dirMap` is the array-valued one — one entry per directory suffix per file, so O(files x depth) in entries and array churn — and only four call sites ever read it, all via `getFilesInDir`: `import-resolvers/{php,csharp,jvm}.ts` and `import-resolvers/configs/python.ts`. Ruby (through workspace-file-index), the TypeScript scope resolver, Vue's import-target and the include-extractor never ask a directory question, and built it anyway. Since #2880 these indexes are retained for a whole resolution pass rather than rebuilt per import, so that waste is now resident memory. Deferring it to the first `getFilesInDir` call is behaviour-identical — same key, same descending-suffix order, same per-bucket push order, same `substring(lastIndexOf('.'))` extension clamp. The builder assigns the MAP on completion, so a repeated miss cannot rebuild it. Measured on `buildSuffixIndex` alone, 32k paths, index built and `getFilesInDir` never called: C# layout, 13 segments 79,018,680 -> 66,580,488 B -15.74% Ruby layout, 11 segments 60,752,792 -> 48,656,856 B -19.91% and on the whole retained WorkspaceFileIndex the bench measures: csharp 32k 73.62 -> 61.76 MiB ruby 32k 55.26 -> 43.69 MiB When `getFilesInDir` IS called the footprint is unchanged, so the deferral is never a loss. No new retention: all five construction sites already hold both input arrays alive beside the index. The laziness is pinned structurally rather than by timing. The test's corpus is a `string[]` whose elements are accessor properties, so an indexed read is observable and the read count IS the pass count: 14 after construction, still 14 after any number of get/getInsensitive, 28 after the first `getFilesInDir`, 28 after five more. Memoizing the decision instead of the map would read 42. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Co58j4au9JLf8dmJpwdF9B * perf(php): resolve imports from a per-run index, not a scan per import (#2901) PHP was the last language whose import resolution scanned the workspace per import. Both `resolvePhpImportTarget` and `resolvePhpImportTargetInternal` materialized two full arrays from the Set on every call, then passed `undefined` as the `index` argument — so `resolvePhpImportInternal` fell through to `suffixResolve`'s linear `findIndex`, once per extension per path part. Measured at 20,000 files: 96.40 ms per import. Handing it the shared SuffixIndex would have moved IMPORTS edges. All three index-fed sites answer a different question than the scan they short-circuit, each found by differential with a concrete witness: 1. `getInsensitive` — the scan leg is `allFiles.has(path)`, exact whole-path with no case-insensitive counterpart; the shared index answers a ci SUFFIX probe. 2. `getFilesInDir` — the scan is root-anchored `startsWith(nsDir + '/')`; `dirMap` is keyed on every directory SUFFIX, so a vendor copy can win. 3. `suffixResolve` — the scan's `endsWith('/' + S)` matches only a PROPER suffix; `buildSuffixIndex` indexes j=0, so a root-level `Foo.php` starts resolving `use Foo` where it returned null. 3b. the scan's `endsWith(p) \|\| lower.endsWith(lower(p))` has a second disjunct that subsumes the first, so it is purely first-in-Set-order and case-insensitive; `get(S) \|\| getInsensitive(S)` lets a case-exact hit anywhere beat an earlier ci hit. So this is not Ruby's #2880 shape. Both sites take `getWorkspaceFileIndex` for the memoized arrays and hand the internal resolver a PARITY `SuffixIndex` memoized on the same Set identity: `getInsensitive` disabled, `get` implementing the scan's real rule via the shared ci lookup plus one O(files) whole-path correction map, `getFilesInDir` root-anchored in Set order. no composer.json 96.40 -> 0.036 ms/import steady state with composer.json 100.19 -> 0.068 ms/import steady state Also closes PHP's last per-import traversal, in `import-resolvers/php.ts`: its namespace-directory scan ran whenever `getFilesInDir` came back EMPTY, not merely when no index was supplied — despite the comment above it claiming "only when SuffixIndex unavailable". An empty bucket is already the answer, so the scan could only confirm it, at one full pass per import whose namespace matches a PSR-4 prefix but whose directory has no direct `.php` child (measured 11 traversals for 10 imports; now 1). Moving it into the `else` is safe because the bucket is a SUPERSET of what the scan finds — a root-anchored direct child `nsDir/<x>.php` has its directory exactly equal to `nsDir`, and a directory is always one of its own suffixes, so both index shapes contain it. Nine mutations of the new code are caught, including M1 "pass the raw shared index" (the naive fix) at 23 arms. The adapter guard reads 600 instead of 1 under a defensive `new Set(allFilePaths)` — the #1918 P1 hazard the unit differential is structurally blind to. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Co58j4au9JLf8dmJpwdF9B * perf(java): index import resolution instead of scanning per import (#2908) Java scanned the whole workspace twice per import: once for the three-tier direct match, and again INSIDE the progressive prefix-stripping loop — so a single unresolvable import cost one full pass per stripped segment. No WeakMap, no index, and it is registered in `SCOPE_RESOLVERS`, so it ran in production. This is byte-for-byte the C# shape #2878 fixed, so Java now reads the same machinery: `getWorkspaceFileIndex` for `normToRaw` + the segment-suffix index, and a Java-owned `PackageDirIndex` WeakMap over `buildPackageDirIndex(_, n => n.endsWith('.java'))` read through `firstFileDirectlyInPkgDir`. Structure mirrors C#'s `narrowContext` / `resolveDirectMatch` / `resolveByProgressiveStripping`. 20k files, 256 imports, 7-in-8 unresolvable: 8.05 -> 0.62 ms/import steady state once the index is built: 0.0036 ms/import Tie-breaks preserved, and Java's are NOT identical to C#'s: - tier 1 `break`s on the exact match, so an exact whole-path hit wins even when a suffix or directory-child hit came earlier in iteration order — hence `normToRaw.get` before `index.get`, which conflates them; - the stripping loop instead returns at the FIRST hit of `f === tailFile \|\| f.endsWith('/' + tailFile)` and only yields its directory child after the scan completes, so the conflated `index.get` is the correct lookup THERE. Applying tier 1's exact-wins rule inside the loop is a real behaviour change (mutation M6); - `.` wildcard stripping stays ahead of everything; - `firstFileDirectlyInPkgDir` reproduces Java's at-root/at-nested predicate exactly, including the first-`indexOf` rule — proved algebraically rather than assumed: the `atRoot` branch matches iff `dir === pathLike`, which is `D.indexOf(P) === 0 === D.length - P.length`, and the `atNested` branch's first occurrence in `f` is the first occurrence in `D` shifted by one. Six mutations are caught; a seventh (swapping the two index builds) is a true equivalence and is recorded as such. Hand-derivation also corrected four cases where the legacy code resolves and I had predicted null — including `java.util.List` reaching a local `util/List.java`, because Java has no in-repo-namespace gate like C#'s #1881. That is preserved here and filed separately as #2910; the parity test pins it so the fix is visible. The adapter guard reads 800 instead of 2 under a defensive `new Set(allFilePaths)`. Two traversals is correct: the workspace index and the package-dir index are separate WeakMaps and each iterates the Set once, the same accounting as C#. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Co58j4au9JLf8dmJpwdF9B perf(cobol): index COPY resolution instead of two scans per statement (#2908) `cobolScopeResolver.resolveImportTarget` ran two full workspace scans per `COPY`, each calling `path.extname` + `path.basename` + `.toUpperCase()` on every entry: tier 1 over `.cpy`/`.copybook`, tier 2 over `.cbl`/`.cob`/ `.cobol`. No WeakMap, no index, and registered in `SCOPE_RESOLVERS`. Two uppercased-basename maps, one per tier, filled in a SINGLE pass over the Set and memoized on Set identity. Lookup is `copybooks.get(upper) ?? sources.get(upper) ?? null`. 20k files, 500 COPY operands: 3879-4082 -> 10.5-11.7 us/import (~350-369x) steady state once built: 0.253 us/import Tie-breaks preserved: - TIER ORDER. A `.cpy` match beats a `.cbl` match even when the source file appears EARLIER in Set-iteration order. This is the one a naive single-map rewrite silently breaks, so it gets its own fixture. - Within a tier, first in Set-iteration order wins (`if (!tier.has(...))`, mirroring the scans' first-match return). - The key is built with the identical call sequence, `basename(fp, extname(fp).toLowerCase()).toUpperCase()`, so `Foo.CPY` still keys under `FOO.CPY` rather than `FOO`. - `path` stays in the loop rather than hand-rolled `/`-slicing, so backslash handling is unchanged on every platform — pinned by a `dir\sub\BOOK.cpy` case. All six mutations are caught: collapsing the tiers, within-tier last-wins, dropping the target uppercase, dropping the extension lowercase, hand-rolled slicing, and the adapter's defensive copy. The first five are caught by the differential and are invisible to the adapter guard; the sixth is the reverse, which is the layering working as intended — the guard reads 600 instead of 1. `COBOL_SOURCE_EXTENSIONS` was being re-allocated on every call; hoisted to module scope beside `COPYBOOK_EXTENSIONS`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Co58j4au9JLf8dmJpwdF9B * perf(csharp): index the csproj leg's namespace-directory scan (#2902) #2878 moved C#'s no-csproj leg onto memoized indexes; the csproj leg kept a per-import full scan in `resolveCSharpImportInternal` step 3, measured at ~1.10 ms per import at 50,000 `.cs` files. The fix the issue proposed would have moved edges. It suggested skipping the fallback when an exhaustive index is available, on the assumption that step 2's `getFilesInDir` answers the same question. It does not: step 2's `dirMap` is keyed on segment-aligned directory suffixes, while step 3's `normalized.indexOf(dirPrefix + '/')` is an UNANCHORED substring match, so step 3 finds a strict superset — and it runs only when step 2 came back empty, so those extra hits are observable, not shadowed: dirPrefix 'ubModels' step 2 [] step 3 ['src/SubModels/Widget.cs'] dirPrefix 'rc/Models' step 2 [] step 3 src/Models/* AND vendor/mysrc/Models/* So the predicate is kept byte-for-byte and made fast instead. It depends only on the file's directory (the needle ends with `/`, so every occurrence lies wholly inside `D + '/'`), which reduces to the `package-dir-index` formula minus the anchoring leading slash. `PackageDirIndex` itself cannot be reused for the same reason — its matcher is anchored. The index is memoized on the `normalizedFileList` array identity and built lazily at the point step 3 is first reached, so BCL usings — which `continue` out at the root-namespace gate — never pay for it. Candidates come from an exact last-segment bucket when `dirPrefix` contains a slash, a last-segment key sweep when it does not, and `singleSegmentDirs` when it is empty. Positions rather than paths, merged and sorted when several directories match, so file-list order survives. App.Missing @ {App, src} 1103.0 -> 7.6 us (145x, and flat in file count: 7.3 @10k, 7.6 @50k, 8.4 @200k) App.Missing @ {App, ''} 626.7 -> 108.5 us App @ {App, ''} 1077.9 -> 2.0 us (539x) App.Ns8 @ {App, src} 0.6 -> 0.6 us (step-2 hit, untouched) `relative === ''` is preserved exactly, including the no-`projectDir` case where the needle is a bare `/` and the answer is "every `.cs` whose directory has no slash of its own" — `getFilesInDir('', '.cs')` cannot answer that over repo-relative paths, so it has its own arm. 13 of 14 mutations are caught, including M1, the naive skip-when-indexed cleanup, at 9 arms. The survivor drops the empty-prefix fast path and is a true equivalence. M9 initially survived and exposed a real corpus gap — no non-`.cs` file lived inside a directory — now covered. The remaining non-constant term is the slash-free sweep, O(distinct last segments): 456 us at 200k files on a unique-name layout, but 7.9 us on a `SrcN/Models` layout, which is how C# repos are actually laid out. Closing the unique-name case needs a character-suffix map over segments — the O(files x depth) memory shape `package-dir-index.ts` cites #2649 to avoid — so it is documented in the code as a design change rather than tuned here. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Co58j4au9JLf8dmJpwdF9B * test(scope-resolution): assert index reuse for every registered language (#2909) Index reuse was asserted by nine hand-written per-language files, so the guarantee existed exactly for the languages someone remembered — and #2908 is the proof that is not good enough: Java and COBOL were registered, quadratic and unguarded until this branch. `resolveImportTarget` is a required member of `ScopeResolver` with one signature and 16 registrations, so "calling it N times against a stable `allFilePaths` must not traverse the set N times" is a property of the CONTRACT. `import-target-index-reuse.contract.test.ts` drives every entry of `SCOPE_RESOLVERS`, modelled on `construction-syntax-wiring.test.ts` — the established shape here for a property plus a justified inventory. Measured counts, all memoized: c 1 cobol 1 cpp 1 csharp 2 dart 1 go 1 java 2 javascript 2 kotlin 1 php 1 python 1 ruby 1 rust 0 swift 1 typescript 2 vue 2 `KNOWN_UNINDEXED` is empty. The audit that produced it also cleared C, C++, Rust, Swift, TypeScript, Vue and JavaScript by hand — Rust's memo lives in `qualified-call.ts::moduleIndexFor`, C's and Swift's loops are inside their WeakMap builders. The empty map stays as a mechanism: a 17th language cannot opt out silently, and the inventory arm fails when a registered resolver has no fixture. Two things the assertion had to get right: - it is `scans(200) === scans(2)`, not `scans === 1`. Per-language counts legitimately differ (C# and Java build two indexes), and comparing two counts needs no per-language expected value. - Rust legitimately scans ZERO times — it answers every leg with `allFilePaths.has(candidate)` probes — so the floor is a per-language `minimumScans`, 1 for fifteen languages and 0 for Rust with the reason on the interface. Paired with a `hitTarget` that must resolve non-null, so the property cannot pass vacuously on a resolver that stopped answering. Miss targets are distinct per import, which defeats the TS/JS/Vue per-target `resolveCache`. Also unifies the instrument. Kotlin and Python counted index BUILDS from production; the other seven count traversals of a `CountingSet`. The build counter is strictly weaker — a scan added BESIDE a reused index moves no build count, which is exactly the mutation `baselines.json` `_blind_spot` records as invisible to every timing arm — and it costs two production modules that ship in the bundle purely for tests, holding module-global state every test must `reset()`. Both guards migrate to `CountingSet`, and `languages/{kotlin,python}/index-stats.ts` plus both call sites are gone, for -59 lines of shipped source. (Mechanical note: the two `index-stats.ts` file deletions appear in the #2901 commit rather than this one. They were staged with `git rm` while a concurrent commit swept the index. The final tree is correct; only that attribution is off, and rewriting a sibling commit to move them was not worth the risk.) Coverage went up in the swap: Kotlin's old "rebuilds when the file set is a different object" arm (3 sets, 3 builds) would have PASSED under a defensive adapter copy. Its replacement fails, as do all six arms across the two files. Verified by mutation: `new Set(allFilePaths)` inserted into the kotlin, python and go adapters fails exactly those three and no others — `python: 200 imports cost 201 traversals, 2 cost 3`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Co58j4au9JLf8dmJpwdF9B * test(import-target): gate the four newly-indexed resolvers, retighten heap The bench covered go/csharp/dart/ruby/kotlin. The four resolvers indexed on this branch shipped unmeasured, and #2903's memory win was not locked in. php, java and cobol join the shared corpus, each with the two load-bearing properties the header requires: imports scale with file count, and most imports MISS so the full cascade runs (resolve rates php 36.0%, java 34.4%, cobol 36.0%). Java's miss families were measured rather than assumed, since it has no in-repo-namespace gate (#2910): `java.` 1041 imports and `com.google.` 1006, both resolving 0. COBOL's collide layout repeats a bookname across BOTH extension tiers, so it reaches the copybook-over-source tie-break rather than only the basename map. `csharp_csproj` is a sixth LANGS entry, not a new arm dimension — an entry needs five small additions and inherits all five arms and all seven gates, where a context axis would have to be threaded through `buildRepo`, `resolveAll`, `identityPass`, the report shape and every gate. `buildFiles` aliases it to `csharp`, so the two share one corpus by construction and cannot drift. Two configs (`{App, 'src'}`, `{Lib, ''}`) produce all three `dirPrefix` shapes — slashed, slash-free and empty — in five arms instead of ten: App.Ns{d} 30.6% src/Ns{d} step 2 hit App.Missing{n} 25.5% src/Missing{n} step 3, last-segment bucket Lib 14.0% (empty) step 3, singleSegmentDirs Lib.Missing{n} 12.0% Missing{n} step 3, KEY SWEEP — the one non-constant path BCL / Ghost 12.4% — root-namespace-gate control 2221 of 3200 imports reach the indexed leg, only 12.4% `continue` out. What that arm pins is stated plainly rather than overclaimed: step 3 answers null for all 2221 here (the hits land at step 2), so it gates that leg's COST and its null answers; its positive tie-breaks stay pinned by the unit parity test. Heap ceilings retightened. #2903 dropped the measured figures, leaving the 1.5x ceilings at ~1.9x — a straight revert to the old size would have passed: csharp 116,000,000 -> 98,000,000 B (measured 61.76 MiB) ruby 87,000,000 -> 69,000,000 B (measured 43.69 MiB) php new 106,000,000 B (measured 67.29 MiB) java new 154,000,000 B (measured 97.32 MiB, the largest in the file — Maven layout is 18 segments) php and java are gated because both retained NOTHING across imports at BASE and now retain the O(files x depth) suffix index — the same argument that gates C#. cobol is not: two `Map<basename, path>`, O(files) with no depth term, and its retained delta does not clear measurement noise, so a ceiling would gate nothing. `csharp_csproj` is not: same corpus, same index, a duplicate number — its one distinguishing footprint, the lazily-built `dirMap` its `getFilesInDir` forces back, is measured at +20.8% and recorded as a residual instead, because gating it would licence eager-dirMap everywhere. csharp's `depth_ratio` also fell 3.318 -> 2.31 (the no-csproj leg never asks a directory question, so the deep arm stopped paying an eager dirMap build). Budget 5 -> 3.5, restoring the file's 1.5x convention — and `_arms_note` says plainly that 3.5 does NOT lock that win in, because locking it needs ~2.9, which is 1.25x over a 1.05x spread and the kind of tightening `_triage` warns buys flake rather than signal. All five pre-existing languages are byte-identical: 25 cells x 5 fields = 125 values, 0 mismatches. The new arms were proven live by a doctored baseline (cobol ceiling 0.01, php heap 1000 B, java resolved 999) producing three correctly-worded failures and exit 1. Wall-clock 10.9 -> 26.1 s, php and csharp_csproj ~11 s of it — both cascades end in `suffixResolve`'s ~50-extension probe, and both gate the two largest wins on this branch, so neither is a candidate to drop. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Co58j4au9JLf8dmJpwdF9B * perf(javascript): build the suffix index JS resolution never had JavaScript's `PassCache` was TypeScript's minus one field: `index`. So JS called the shared `resolveTsTarget` with `ctx.index === undefined`, and `import-resolvers/standard.ts` fell through to `suffixResolve`'s linear `findIndex` — scanning the materialized path list once per extension (~39) per path part, per import. 2000 files 6448.9 -> 28.5 us/import (TypeScript: 25.0) 8000 files 25972.6 -> 27.4 us/import (TypeScript: 27.0) Per-import scaling over 4x the files: 4.12x -> 1.09x. Every instrument on this branch was blind to it. `CountingSet` counts traversals of the Set; this walked the array the adapter had already materialized — the blind spot `counting-file-set.ts` documents in its own header and `baselines.json` records under `_blind_spot`. Under mutation M1, which drops `index` and reproduces the shipped defect exactly, the sixteen- language contract test stays GREEN for javascript, because the pass cache is still reused and `files.scans` reads 2 either way. Two new arms do catch it: a `suffixResolve` linear-branch counter that runs the legacy adapter first as its control (135 entries legacy, 0 now), and a mock-free behavioural assertion that a repo-root module resolves by bare specifier. Adding an index moves output, exactly as it did for PHP in #2901, so it was characterized rather than assumed — 211,200 pairs (400 corpora x 3 importers x 176 targets) plus 184 hand cases. Two classes move and there is no third: A null -> repo-root file (108) `require('config')` with root `config.js`. The scan tests `endsWith('/' + suffix)`, so a path with no slash has no proper suffix and was unreachable through that leg — while `./config` from the root already resolved via the exact `Set.has` branch. JS was internally inconsistent. B file -> different file (5679) `import 'app/main'` was resolving to `node_modules/dep0/lib/main.js`; the scan skipped the whole-path candidate at the 2-segment suffix and fell through to the 1-segment `/main.js`, taking the first such file in Set order. C hit -> null ZERO, and impossible: proper-suffix keys are a subset of the index's keys. Both moved classes are JS being wrong. JS-new agrees with TypeScript on all 211,200 pairs and every corpus case, 0 disagreements — which is the intended design, since JS delegates to the TS resolver and differed only by this field. Also swaps the single-slot `let cached: PassCache \| null` in JS, TS and Vue for a module-level `WeakMap`, matching every other language. Two alternating file sets rebuilt everything on every call: 12.0 -> 1438.2 ms at 4000 files x 400 imports (120x); after, 11.0 -> 15.7 ms. This is LATENT, not live — `pipeline/run.ts:673` builds one Set per provider pass and the three are separate providers — but it is why these were the only languages that could not carry the standard distinct-set guard. They can now: the arm fails on HEAD for all three (`expected 42 to be 2`) and passes after. Six mutations caught, including a global `resolveCache` (M5), which needed a new arm — `expectDistinctFileSetsGetOwnIndex` builds two IDENTICAL corpora, so a stale answer carried between them is also the right answer. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Co58j4au9JLf8dmJpwdF9B * refactor(ingestion): one per-file-set memo primitive, twenty-one call sites Every language that indexes its import resolution hand-rolled the same memo: declare a module-level `WeakMap` keyed on the file-set object, `get`, `if undefined` build and `set`, return. One concept, written twenty-one times, and this branch had just added five more. `import-resolvers/per-file-set.ts` exports it once: perFileSet<K extends object, T extends object>(build: (key: K) => T): (key: K) => T Two decisions, both recorded in the file. `T extends object` rather than `has`-then-`get`: `WeakMap.get` returning `undefined` cannot distinguish "not built" from "built as undefined", and the `has` form needs a cast or a non-null assertion, both banned here — the constraint makes the ambiguous case unrepresentable instead, and a future caller wanting `string \| null` gets a compile error pointing at the decision. A throwing build stores nothing and runs again next call, so failures are not memoized and a half-filled index is never published — inert for these pure builders, and the safer direction. `K extends object` rather than `ReadonlySet<string>` is what lets C#'s `readonly string[]`-keyed cache share the helper. Twenty-one sites migrated across `import-resolvers/` and fifteen languages. Every existing doc comment was re-homed onto the new call rather than deleted — several record real invariants (the Set-identity contract, the #1918 pass-through rule, why Rust's memo lives on a different hook). TypeScript, JavaScript and Vue additionally had byte-identical `PassCache` interfaces and builders. `import-resolvers/pass-cache.ts` now holds the one builder, taking a single argument — every difference the three have lives in the CONSUMER (`tsconfigPaths`, the extension list), not the builder. The builder is shared, the memo deliberately is not: each adapter keeps its own `perFileSet`, hence its own index and its own `resolveCache`, because the three disagree about what a specifier resolves to and one shared cache would hand a language another language's answers. It buys no runtime reuse and the module says so — each provider pass builds its own `allFilePaths` Set, so the three are always different keys. C and C++'s `augmentedFilePaths` was a two-LEVEL memo, and needed no new abstraction: the outer memo's value is a function and a function is an object, so `perFileSet(perFileSet(...))` composes. The two instances stay one per file, and the reason is now in BOTH doc comments rather than only C++'s — cpp delegates to `resolveCImportTarget`, whose `suffixIndex` is keyed on the augmented set, so a shared memo would cross the two languages' indexes. Two sites are deliberately NOT migrated, each with the reason written at the declaration so the next sweep does not re-litigate them: - `configs/swift.ts` is a two-input memo keyed on one. `targets` is not derivable from the key; re-keying on `ctx` would force a banned non-null assertion or an unreachable fallback inside a memo builder. - `rust/qualified-call.ts` `MODULE_SCOPE_CACHE` is three inputs keyed on one, and sits ten lines below a `perFileSet` in the same file — the likeliest thing to be "fixed" by mistake. The other ten remaining `WeakMap`s are different concerns and stay: AST-node caches, worker-pool runtime state, graph metadata, mutable lazily-filled accumulators, and the C++ ADL / inline-namespace indexes, which are reassigned by explicit clear functions and epoch-stamped on read — validity rules beyond key identity that a closure over a private cache cannot express. Net −20 lines of code, +22 of the two "why not" notes. The primitive's own doc is where the cost sits: the Set-identity contract and the two design decisions are written once instead of being twenty-one implicit facts. Pure refactor: 1764 unit tests, 42 guard tests, all sixteen contract-test traversal counts unchanged (c 1, cobol 1, cpp 1, csharp 2, dart 1, go 1, java 2, javascript 2, kotlin 1, php 1, python 1, ruby 1, rust 0, swift 1, typescript 2, vue 2), 647 C/C++ tests, and every bench fingerprint unchanged. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Co58j4au9JLf8dmJpwdF9B * test(import-target): gate every registered language, not nine of sixteen The bench pinned output fingerprints and scaling for 9 of the 16 languages in `SCOPE_RESOLVERS`. The other seven — c, cpp, javascript, python, rust, swift, typescript, vue — resolve imports in production with nothing pinning their output or their cost. JavaScript was the sharpest case: the 25,972 us/import defect fixed earlier on this branch was gated by unit tests alone. All 16 are now gated, plus the `csharp_csproj` variant: 17 entries. The nine existing languages are byte-identical — 234 committed values (9 x 5 arms x 5 fields, plus 9 top-level fingerprints), 0 changed, and no pre-existing budget touched. Measured both before and after the memo consolidation in `e6f15274e`, so it doubles as an independent check that the refactor preserved behaviour. Corpora keep both load-bearing rules — most imports MISS, and import count scales with file count — at resolve rates of 26-36%. C and C++ follow the `csharp_csproj` precedent: a `LANGS` entry carrying its own context (header paths through `resolutionConfig`) over an aliased corpus, since cpp delegates into C's `resolveCImportTarget`. Vue threads `tsconfigPaths` so its alias branch actually runs; ts/js use bare specifiers only, because relative ones never reach `suffixResolve`. Two corrections to my own profiling, both verified rather than assumed: Swift's `byModule` IS depth-scaled (one bucket entry per interior segment, not O(files)), and Python's index is depth-free while its RESOLVER is quadratic in depth — `hasRepoCandidate` and `resolveAbsoluteFromFiles` each rebuild one ancestor prefix per importer directory component, per import. That is why python's `depth_budget` is 11 against a 3.5 next-highest; the arm is pinning a real defect rather than a comfortable number, and it is filed separately. Rust's collide arm was redesigned rather than budgeted away: it is flat on file count by construction, so a shared-leaf arm would have asserted nothing. Its collide corpus varies `::` segment count — the axis its cost actually has — and the linear 1.8 budget asserts the file-count flatness. Heap: all 8 measured, 3 gated. javascript (44.07 MiB, retained nothing before its fix), python (7.27 MiB), c (9.55 MiB). Five skipped with their numbers in `_arms_note` rather than silently: rust 16 B (no index on this hook), swift reads 3x SMALLER on a 4x corpus so it is below its own noise floor, typescript 288 B on 46 MB, vue +5.4%, cpp 0.04% from c. Every gate type was proven able to fail: one run with 10 doctored values fired 10 correctly-worded failures across all 8 new languages, covering per-scale fingerprint, shape/resolved, shape/distinct_outcomes on a non-small arm, depth, collide scaling, absolute small ms, absolute collide ms, top-level fingerprint and heap bytes. That proof found two wrong messages, now fixed: the heap failure claimed a `buildSuffixIndex` cause that is false for python and c, and the fingerprint failure pointed at a parity harness covering none of the eight. Wall clock 26 -> 46 s. The ts/js/vue family is 14.6 s of the 18.8 s added, because `suffixResolve` probes ~39 extensions per path part on a miss — the real resolver, not something the bench can tune. Per language the bench got cheaper (2.7 s vs 3.0 s). If it must shrink, `_arms_note` and the CI comment record the one cut that removes duplicate work rather than coverage — drop collide for typescript and vue only, -3.9 s, since all three share `resolveTsTarget` and javascript keeps the arm covering their common axis. Explicitly NOT `REPS`: it is 15 because `depth_ratio` flaked 1-in-20 at 5, and lowering it would re-open that for all 17 languages. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Co58j4au9JLf8dmJpwdF9B * perf(import-resolvers): stop building half of every suffix index Applies the findings of a four-lane quality review over this branch. Half of `buildSuffixIndex` was dead weight for most of its consumers. Commit `b6ee577e0` on this branch made the THIRD map (`dirMap`) lazy for exactly this reason and left the two larger ones eager. Tracing every reader: Java and no-csproj C# call `get` and never `getInsensitive`; PHP calls `getInsensitive` and never `get`. Measured dead weight at 32k paths: Java 49.98 MiB of a 100.82 MiB index, PHP 34.49 of 69.85. All three maps are now built on first use, and `lowerMap` is DERIVED from `exactMap`'s insertion order rather than re-traversed — measured 330 ms against 389 ms today, so it is cheaper even for the two-map consumers. `pass-cache.ts` hands the builder an already-lowercased list, so for TypeScript, JavaScript and Vue the derivation is the identity and `getInsensitive` aliases the one map. java 80.26 -> 25.61 MiB retained (-68%) csharp no-csproj 57.15 -> 21.52 (-62%) javascript 44.07 -> 22.65 (-49%) php 60.86 -> 32.09 (-47%) build @32k 562.1 -> 119.6 ms (get-only), 329.5 ms (both) The derivation is proven, not asserted: keys, values AND insertion order byte-equal over 968,418 entries across case-colliding, Unicode-adversarial and pathological corpora, plus 400 seeded-fuzz rounds. Order matters because it is what makes `getInsensitive` return the first match in file order. PHP additionally defers `filesByRawDirectory` (statically unreachable unless a composer.json parses) and `firstProperSuffixMatch` (0 entries and 35.6 ms on the bench corpus) to the branches that read them. One suggested micro-optimisation was REJECTED with a counterexample rather than taken: hoisting `suffixResolve`'s lowercase out of the extension loop assumes `(s + ext).toLowerCase() === s.toLowerCase() + ext`, which is false for a segment ending in Greek capital sigma — `("ΑΣ" + ".ts").toLowerCase()` is `"ασ.ts"`, not `"ας.ts"`, because Final_Sigma is context-sensitive and `.` is case-ignorable. A file named `ΑΣ.ts` would have stopped resolving. 16 mismatches in 2,171,190 checks, for 8.7%. The heap arms had become ceilings over nothing. `retainedIndexBytes` read only `index.all.length`, so once the maps went lazy it built none of them and reported ~0 B — passing every ceiling. All heap arms now route through `retainedPassBytes`, resolving a real missing import through the real resolver, so the maps measured are the maps production forces. Two further measurement defects surfaced while fixing it: PHP reaches the index through a second memo, so the ephemeron chain needs four GC cycles and was reporting 249,208 B for a 9.3 MB index; and `bytes_large` carried an ~11% rope-flattening bias that made every ratio read 0.85-0.96 for structures that are linear (now 0.998-1.017). A `heap_floor_fraction` arm was added — a ceiling can only say "not too big" — and proven by simulating the exact regression: `16 B at 32000 files < floor 17325000 B — this arm has almost certainly stopped MEASURING`. `csharp_csproj` is now gated too: its old exclusion as "a duplicate of csharp" held at +20.8% and is false at 2.47x. Three silent-coverage holes in the bench. `LANGS` was a hand-written literal claiming to mirror `SCOPE_RESOLVERS` while never importing it — the seam that let JavaScript ship ungated; it is now derived, with an inventory arm reconciling both directions. Four per-language budget lookups compared against a possibly-`undefined` value, so deleting a key deleted the gate. Five dispatchers ended in bare fallthroughs meaning "ruby" and "csharp", so a mistyped language would have been benchmarked as Ruby's corpus under C#'s resolver, forever green. REPS is now chosen per language (15 below 5 ms, else `clamp(ceil(150/ms),7,15)`) rather than globally by the noisiest cell: timing phase 39.8 -> 28.7 s, with the six reduced-N languages showing peak-to-peak 1.008-1.071, no worse than the eleven that kept 15. Worst headroom across all 85 cells is 0.71 of budget. `depth_budget` for csharp 3.5 -> 2.2 and java 3.4 -> 2.2: their ratios fell to 1.438/1.402 because the lazy maps stop the deep arm paying for a map it never reads. The file's own note said 3.5 did not lock that win in; 2.2 does. Also fixes a raw NUL byte that made `suffix-index-lazy-dir-map.test.ts` BINARY to git — all 395 lines were invisible to diff, blame and grep. The repo documents this exact hazard in `route-extractors/dispatch-guard.ts`. That file now also carries the guard the refactor lacked: eight arms pinning one-map-per consumer and zero-extra-pass derivation, each proven against four mutations, including a fused-eager rebuild that moves no total and is caught solely by the at-construction count. All 17 bench fingerprints and all 85 per-scale tuples unchanged. 1772 unit tests, 12 adapter guards, tsc clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Co58j4au9JLf8dmJpwdF9B * perf(python): memoize the importer's ancestor chain per directory (#2913) Python's file index was always depth-free; the resolver was not. `hasRepoCandidate` and `resolveAbsoluteFromFiles` each rebuilt one ancestor prefix per directory component of the importer on EVERY import, and the index's own `dirPrefixes` build inserted one entry per component per file. So an import from `a/b/c/d/e/f/mod.py` did ~6x the prefix work of one from `a/mod.py` regardless of corpus size — `depth_ratio` 7.239 where the next worst language sat at 3.446. The prefixes are a pure function of the importer's DIRECTORY, so they are memoized per directory inside `getPythonFileIndex` (`ancestorsByDir`), which is itself already per-file-set. Three smaller cuts came out of profiling the same delta: the leading segment is rejected up front against a set of nested directory names, the module and package buckets are consulted before the walk instead of inside it, and the `dirPrefixes` build stops at the first ancestor already stored. Measured over 6 serial runs: depth_ratio 1.748-1.872 against 7.239, and at a fixed 400 files the per-import cost at 18 directory components drops 6.761 -> 1.065 us. All five python fingerprints are byte-identical, so this is a hoist; the budget retightening lands in the following commit, because `_arms_note` is a single JSON line that also carries the heap-gate rewrite. Also memoizes `pythonFileExportsName`'s `parsedFiles.find`, which was O(files) for every import whose package probe resolved — the same shape #2901 removed, keyed on `parsedFiles` rather than on `allFilePaths`. The new gate is a count, not a timing: `ancestorsByDir.size` after N imports from D directories must equal D, paired with a reference-identity assertion so a memo that rebuilds AND re-stores still fails. `CountingSet` cannot see this defect — the chain derives from the `fromFile` string and a rebuilt prefix traverses the file set zero extra times. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NtGZN6YSNU8chALSnKYHBp * fix(import-target): close the eleven findings from the #2911 review Seven P2s and four P3s. Every one is a gate that could not fail or a comment that had become false; no shipped behaviour defect was found, and all 85 per-language fingerprints are unchanged. GATES THAT COULD NOT FAIL - The C# namespace-dir memo was keyed on a materialized array, so a one-character `[...normalized]` copy at the adapter boundary minted a fresh WeakMap key per import while traversing the file set zero extra times: 67 tests stayed green and only a timing ratio caught it. `resolveCSharpImportInternal` now takes the Set and derives both arrays from `getWorkspaceFileIndex`, so there is ONE key shape and ONE instrument. Copying the Set now turns three arms red. Established first that `configs/csharp.ts` is test-only (`buildImportTargetWorkspace` has no production caller) and that both derivations are byte-identical — otherwise the rekey would have been a behaviour change, not a hoist. - The contract test called `resolveImportTarget` with four arguments where `pipeline/run.ts:682` passes five, so everything behind `context` was ungated for all 16 languages: defeating PHP's `filesByDirectory` memo cost 197.0 -> 9,976.2 us/import (50.6x) with 248/248 tests green. `CountingSet` provably cannot see it — the builder iterates the `parsedFiles` array and touches the Set zero times — so the new gate counts own-index reads on `parsedFiles` through a Proxy. Only PHP and Python have a context leg; the other fourteen carry the floor anyway. - Three heap budgets were read with no presence check. `ceiling * undefined` is NaN and `bytes < NaN` is false, so deleting `heap_floor_fraction` disabled the floor for all eight arms; deleting `heap_ratio_budget` did the same; and iterating the baseline's keys dropped a language whose ceiling key was deleted out of the gate entirely. All three now fail closed with a message naming the broken comparison. - `HEAP_PROBE_TARGET` decided what each heap arm measured and was compared to nothing: repointing csharp_csproj at a non-matching namespace dropped it 73.70 -> 59.92 MB with `--check` still exiting 0. The four corpus fields are now asserted through the loop the timing scales already use, and the floor derives from a recorded reading rather than from a ceiling that is itself 1.5x the measurement. - About 35 of the 86 PHP parity arms were structurally unable to fail: both sides called the same production helper, so deleting the `..` guard left them green. Every hand case now pins an absolute literal as well as the differential. Eight of those literals pin a bug or a documented limitation and say so rather than blessing the value. - The registry inventory arm was weighed and KEPT, against the review's suggestion, on a structural number rather than a timing: the benchmarks job runs 9m23s against a 12m58s critical path, so its seconds buy no merge latency, and moving the arm to vitest would put the registry load ON that path while weakening what it reconciles. The "7.3 s" and "~46 -> ~42 s" figures it was justified with are corrected, including stating that only report mode got faster. - python's `depth_budget` drops 11 -> 2.6 now that #2913 is in. 1.39x the measured maximum rather than the file's usual 1.5x, deliberately: at 2.8 a revert of the nested-name rejection (2.734) would pass. The two parts of that fix this arm cannot gate are named, with the count-based arms that do gate them. COMMENTS THAT HAD BECOME FALSE - `pass-cache.ts` said it deduplicated "three byte-identical copies". JavaScript's had five fields and never called `buildSuffixIndex` — that missing field IS this PR's headline defect. - The per-language census said nine where it is twelve, three of them added by this PR. Replaced in seven places with the mechanism that enforces it, which cannot go stale. - `getFilesInDir` handed out the index's live bucket. Now `readonly string[]`, so mutation is a compile error; `.slice()` was rejected because `configs/python.ts` reads only `.length` and a per-import copy would reintroduce the term this PR removes. - #2910 is the Java in-repo-namespace gap, not the JavaScript index defect. 13 references corrected, the one correct Java use left in place. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NtGZN6YSNU8chALSnKYHBp * perf(python,bench): flatten the bare-import walk, measure the context leg Two follow-ups the #2911 review surfaced but left open. BARE IMPORTS (`import os`) still walked every ancestor of the importer. #2913 fixed the dotted tier; this tier lives in `import-resolvers/python.ts` and no bench arm can reach it, because every python arm here spells its imports with a dot and returns at the `pathLike.includes('/')` guard. It also ran TWICE per `from x import y`: `resolvePythonImportTarget` probed the package with `targetIncludesImportedName: true`, and on null — the expensive case, having already walked to the workspace root — fell through to a byte-identical call. Established that the two cannot differ before collapsing them: the flag's only effect is to skip `pythonImportedSubmoduleTarget`, so the recursion re-runs the outer frame's entire tail on the same three references, and reaching the fallthrough means that tail already returned null. The walk itself is now a memoized chain plus an O(1) proof of absence against the index's basename buckets. Its chain is NOT the one #2913 memoized and the difference is semantic, not accidental — no `filter(Boolean)`, self excluded, workspace root included — so under an absolute-path workspace the unfiltered chain probes `/abs/a/` where a filtered one would probe `abs/a/`, a prefix of nothing. Two negative arms pin that in both directions. The shared index moved to `import-resolvers/python-file-index.ts` rather than being reached across a cycle, which also collapsed a standalone memo into the one per-file-set. 12 / 24 / 72 Set probes at depth 1 / 4 / 16 become a flat 2. At 18 path components, 11.615 -> 0.740 us/import (15.7x) and the depth curve is gone: 7.843 -> 0.925. Gated by probe COUNT, not timing. THE BENCH CALLED `resolveImportTarget` WITH THREE ARGUMENTS where `pipeline/run.ts:682` passes five, so no timing arm entered the `context` leg for any language. Arity checked against the registry rather than the comment: php and python declare five, every other hook three or four. `parsedFiles` is built first and `allFilePaths` derived from it, matching `run.ts`; fresh per pass, because the memos behind that leg key on the array identity and `fastest()` takes a min. Python's `parsedFiles` was structurally unreadable, not merely unread: the arm passed a `namespace` spelling, which makes `pythonImportedSubmoduleTarget` return null before the context is consulted. The import KIND had to change too. No fingerprint moved anywhere — on this corpus PHP's leg returns the same file the cascade already did — which is exactly why the new `context` arm asserts with-context against without-context instead. Defeating PHP's `filesByDirectory` memo now costs 1003.7 ms against a 148 ms budget; before this the bench could not see it at all. Re-recorded on a quiet box, maxima over 5 serial runs: php small 27.762 -> 34.023 and heap 37.6 -> 49.6 MB (`filesByDirectory` is now retained for the pass), python small 1.76 -> 4.358. `depth_budget.python` moves 2.6 -> 2.2, because the added work is depth-FLAT: absolute cost doubled while the ratio FELL to 1.563, so the old budget had gone slack. Both lock-in figures were re-measured under the new call shape rather than carried over — reverting the ancestor memo scores 2.524, reverting the nested-name rejection 2.553, so each fails at 2.2 with 13% to spare. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NtGZN6YSNU8chALSnKYHBp * refactor(import-target): make the key-shape rule a type, drop three censuses Cleanup pass over the #2911 review-fix commits. No behaviour change: all 85 per-language fingerprints, every `resolved` and every `distinct_outcomes` are byte-identical, and the targeted suite is 1851/1851. MEASURED — `byBasename` was 71% empty array slots `byBasename` holds roughly one bucket per file, and building each with `[]` followed by `push` makes V8 grow the backing store to its 16-slot minimum, so every single-file bucket retained 15 empty pointer slots. Constructing the one-element bucket directly is byte-identical in contents and 5.50 -> 1.60 MiB at 32000 `.py` paths. The bench arm reads 10543848 -> 6360936 B (-39.7%); `heap_reading_bytes.python` and its ceiling are re-recorded. The same edit shares one `{ raw, norm }` between both maps instead of allocating a second literal for every `__init__.py`. THE RULE THAT COST A TIMING RATIO TO FIND IS NOW A COMPILE ERROR `perFileSet`'s key is narrowed from `object` to `ReadonlySet<string> \| readonly ParsedFile[]`. Reintroducing the #2911 defect shape — a memo keyed on an array materialized from the file set — now fails with TS2345 instead of silently minting a fresh `WeakMap` key per import while traversing the Set zero extra times, which every scan-counting guard reads as green at its correct value. That also retires the header's hand-maintained roster of `ParsedFile[]`-keyed call sites, which listed three — this PR added a fourth in `395c707d4` and did not update it. A census inside a comment warning that censuses go stale, stale inside one commit. The header now names shapes; the compiler names sites. Two more claims that had drifted from their code: - `per-file-set.ts` asserted "No index derived from the file set is keyed on an ARRAY materialized from it". `configs/swift.ts` is, deliberately, with its reasons written down. Two files in one directory disagreeing is worse than either; the rule now states what the type rejects and names the exception. - `SuffixIndex.getFilesInDir`'s doc explained that it returns the index's own bucket by reference. True of `buildSuffixIndex`; the other implementation of that interface, in `languages/php/import-target.ts`, returns a filtered copy. The interface now carries only the caller-facing contract (`readonly`, do not mutate) and the sharing rationale moved onto the implementation it describes. - The contract test still described Python as having "NO memo on this key". `parsedFileByPath` landed in `395c707d4`; the floor of 1 is now its single build rather than a per-import scan. DEDUP `importerDirOf` replaces four copies of `replace / lastIndexOf / slice` — two in production, where one was a memo KEY and the other a memo's query argument, so the two per-directory memos in one index agreed only by inspection. The tests keep their own verbatim derivation on purpose: importing production's would make the key lookup agree by construction and hide a regression. `buildParsedFiles` maps through `probeFile` instead of repeating its 7-field literal 900 lines away; `requireNumericBudget` and `expectNoOrphanKeys` replace three and three copies, with every per-arm `why` kept per-arm. The two Python memo guards collapse onto shared arms in `test/helpers/counting-file-set.ts` — 1847 tests before and after, and both still go red under mutation. SKIPPED, with reasons: dropping `normSet` for bucket scans (trades O(1) probes on the hot path for ~1.6 MB against a 6.4 MB reading); measuring heap for all 17 languages (+9 s and a design decision, not a cleanup); `readonly` on the five sibling resolvers' array parameters and the `getDirMap` slice/join rewrite (both correct, both outside this diff). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NtGZN6YSNU8chALSnKYHBp * perf(import-target): rewrite the dirMap build, gate heap for every language The three items the /simplify pass deferred, plus what measuring them found. `getDirMap` BUILD — 226.9 ms -> 173.1 ms at 32 000 paths It built every key with `dirParts.slice(j).join('/')`: one parts array, one slice array and one joined string per file per directory component, in the map its own doc calls "by far the most expensive" of the three. Now a `lastIndexOf` walk slicing substrings out of the original string — the same rewrite `getExactMap` already records at 357.4 -> 264.5 ms. The key set is identical, not merely equivalent: 272 956 keys over a 32 000 path corpus carrying absolute paths, leading/interior/trailing doubled separators, Windows separators, extensionless files, dotfiles, dotted directories and colons, run both slash-normalized and raw. Zero differences in keys, in key INSERTION ORDER, in bucket contents, in bucket ORDER, or across 767 732 probes through the real index. Bucket order matters because `php.ts` reads `[0]`. READONLY on the per-pass shared arrays `WorkspaceFileIndex.normalized`/`.all` and the `normalizedFileList`/ `allFileList` parameters of jvm, php, ruby, go and standard are now `readonly string[]`. This PR already made that argument for one bucket accessor; these are the two biggest arrays held for a whole pass, and the blast radius of an in-place sort is larger. Types only — no cast, no copy — and it let two pre-existing `as string[]` casts in `languages/typescript/import-target.ts` be deleted rather than added to. HEAP IS NOW MEASURED FOR ALL SEVENTEEN LANGUAGES, AND THE PROSE WAS WRONG Nine were excluded on measurements taken once and never re-checked, with the re-entry condition stated in a comment and watched by nothing. Measuring them: - go, dart and kotlin had NO stated reason at all — the header said "six of seventeen" against a list of eight. kotlin retains 45.85 MiB, the second-largest reading in this file, larger than ruby's and java's; - swift and cobol were recorded as below-noise (0.29 MB, 0 B). They read 3.29 MB and 2.21 MB and grow the right way. The arm changed under them — #2903's real-import probe, then corpus flattening — and nobody re-took it; - the header quoted javascript at two different values four paragraphs apart. Only rust's exclusion survived: 16 B at both scales, identical over five runs. Six of the nine are now FULLY budgeted rather than merely bounded — ceiling, floor and ratio — because each grows linearly (0.996-1.004 against a 1.25 budget). cobol, swift and rust keep an upper bound and no floor, deliberately: a floor over a reading at or below its own noise gates the noise. Proven live: restating kotlin's reading so its floor clears the real measurement fails with "this arm has almost certainly stopped MEASURING rather than started saving" — the failure that once left four arms at 0 B under passing ceilings. Cost: +1.37 s in the heap phase, measured per language rather than asserted. `normSet` was NOT removed, and the reason is now in the code. It is derivable from the two buckets, but `byBasename` is keyed on BASENAME: on a 9 000-file service tree `utils.py` and `models.py` hold 1 000 entries each, so `import utils` would scan every `utils.py` in the workspace per import — the exact defect class #2901/#2902/#2908 removed. ~1.6 MB against a 6.4 MB reading buys both probes staying O(1). All 85 per-language fingerprints unchanged; 1854 tests pass. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NtGZN6YSNU8chALSnKYHBp * test(php): drop the impossible undefined comparison from the parity copy CodeQL (js/comparison-between-incompatible-types, alert 945) flags the `ctx === undefined` arm of the legacy adapter copy: `WorkspaceIndex` is an object type at that position, so the comparison can never be true. Optional chaining expresses the same guard without the type-level clash — an undefined index still fails the `typeof` test and returns null — so the copy remains behaviourally verbatim against the shipped adapter, which is the only property this harness relies on. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017oEY2i74d1HLa5FuGVuPiT --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>		2026-08-10 17:22:51 +01:00
..
.claude	feat(cpp): sfinae filter (#1623 )	2026-05-16 20:23:13 +01:00
bench	perf(import-resolvers): index every scanning resolver, consolidate the memo, gate every registered language (#2911 )	2026-08-10 17:22:51 +01:00
hooks	feat(cli): add a bunx lane so bun-only machines can run gitnexus (#2765 )	2026-08-06 08:40:36 +00:00
scripts	fix(analyze): gate FTS-indexed DML before the incremental writeback (#2841 ) (#2854 )	2026-08-07 09:44:44 +01:00
skills	feat: close reported graph blind spots in reference resolution, analyze and storage (#2856 )	2026-08-08 09:58:14 +01:00
src	perf(import-resolvers): index every scanning resolver, consolidate the memo, gate every registered language (#2911 )	2026-08-10 17:22:51 +01:00
test	perf(import-resolvers): index every scanning resolver, consolidate the memo, gate every registered language (#2911 )	2026-08-10 17:22:51 +01:00
vendor	perf(communities): fix the O(communities x N) copy in vendored Leiden, wire Icebug to its real API (#2337 ) (#2692 )	2026-07-25 13:23:17 +01:00
.env.example	feat(serve): validate and port-scope the origin/proxy configuration surface (#2820 )	2026-08-05 06:52:39 +01:00
.npmignore	feat(install): toolchain-free tree-sitter via vendored prebuilds (#2113 )	2026-06-09 18:16:24 +01:00
CHANGELOG.md	chore: release v1.6.9 (#2367 )	2026-07-04 07:53:06 +01:00
Dockerfile.test	fix(security): Pin Docker Node base images, remove runtime package-manager CVE surface, verify Trivy on PRs, and harden Dependabot policy (#1455 )	2026-05-09 16:55:31 +01:00
package-lock.json	chore(deps)(deps-dev): bump tsx from 4.23.4 to 4.23.5 in /gitnexus (#2862 )	2026-08-08 08:17:11 +01:00
package.json	chore(deps)(deps): bump @ladybugdb/core in /gitnexus (#2863 )	2026-08-08 08:17:00 +01:00
README.md	feat: make MAX_CALLABLE_VALUE_TARGETS configurable via env (#2725 )	2026-08-01 12:42:36 +01:00
tsconfig.json	perf(cfg): SSA-sparse reaching-defs to replace the dense-set worklist (#2201 ) (#2212 )	2026-06-15 19:16:53 +01:00
tsconfig.test.json	test: add test suite with vitest (unit + integration + fixtures)	2026-03-01 20:07:02 +05:30
vitest.config.ts	fix(analyze): gate FTS-indexed DML before the incremental writeback (#2841 ) (#2854 )	2026-08-07 09:44:44 +01:00

README.md

GitNexus

Graph-powered code intelligence for AI agents. Index any codebase into a knowledge graph, then query it via MCP or CLI.

Works with Cursor, Claude Code, Antigravity (Google), Codex, Windsurf, Cline, OpenCode, CodeBuddy (Tencent), Qoder (Alibaba), and any MCP-compatible tool.

Why?

AI coding tools don't understand your codebase structure. They edit a function without knowing 47 other functions depend on it. GitNexus fixes this by precomputing every dependency, call chain, and relationship into a queryable graph.

Three commands to give your AI agent full codebase awareness.

Quick Start

# Index your repo (run from repo root)
npx gitnexus analyze

That's it. This indexes the codebase, installs agent skills, registers Claude Code hooks, and creates AGENTS.md / CLAUDE.md context files — all in one command.

On npm 11.x? npx can crash during install (Cannot destructure property 'package' of 'node.target'). Use the pnpm form instead:
pnpm --allow-build=@ladybugdb/core --allow-build=gitnexus --allow-build=tree-sitter dlx gitnexus@latest analyze
See Troubleshooting → npx gitnexus crashes with node.target is null (npm 11) for the full matrix (global install, npm downgrade).

To configure MCP for your editor, run npx gitnexus setup once — or set it up manually below.

gitnexus setup auto-detects your editors and writes the correct global MCP config. You only need to run it once. To configure only selected integrations, pass --coding-agent/-c with a comma-separated list or repeat the option, for example gitnexus setup -c cursor,codex.

Editor Support

Editor	MCP	Skills	Hooks (auto-augment)	Support
Claude Code	Yes	Yes	Yes (PreToolUse + PostToolUse)	Full
Cursor	Yes	Yes	Yes (postToolUse, manual install)	Full
Antigravity (Google)	Yes	Yes	Yes (AfterTool, Gemini CLI hooks schema)	Full
Codex	Yes	Yes	Yes (PreToolUse + PostToolUse, Codex hooks)	Full
OpenCode	Yes	Yes	—	MCP + Skills
CodeBuddy (Tencent)	Yes	Yes	—	MCP + Skills
Qoder (Alibaba)	Yes	Yes	—	MCP + Skills
Windsurf	Yes	—	—	MCP

Claude Code and Codex get the deepest integration: MCP tools + agent skills + PreToolUse hooks that automatically enrich grep/glob/bash calls with knowledge graph context + PostToolUse hooks that detect a stale index after commits and prompt the agent to reindex.

Community Integrations

Agent	Install	Source
pi	`pi install npm:pi-gitnexus`	pi-gitnexus

MCP Setup (manual)

If you prefer to configure manually instead of using gitnexus setup:

Claude Code (full support — MCP + skills + hooks)

# macOS / Linux
claude mcp add gitnexus -- npx -y gitnexus@latest mcp

# Windows
claude mcp add gitnexus -- cmd /c npx -y gitnexus@latest mcp

Codex (full support — MCP + skills + hooks)

codex mcp add gitnexus -- npx -y gitnexus@latest mcp

Codex hooks (PreToolUse graph enrichment + PostToolUse stale-index detection in ~/.codex/hooks.json, same schema as Claude Code) need the bundled adapter script, so they are installed by gitnexus setup -c codex rather than manually.

Alternatively, install everything as a Codex plugin (MCP + skills + hooks in one step):

codex plugin marketplace add abhigyanpatwari/GitNexus
# then inside Codex: /plugins → install "GitNexus"

Codex notes: SessionStart is intentionally not registered — Codex reads AGENTS.md natively, which already carries the GitNexus context block. Newly installed hooks need a one-time approval in Codex via /hooks before they run. Pick one install route (gitnexus setup -c codex or the plugin): plugin hooks load alongside ~/.codex/hooks.json, so installing both can fire duplicate hooks per tool call.

Cursor / Windsurf

Add to ~/.cursor/mcp.json (global — works for all projects):

{
  "mcpServers": {
    "gitnexus": {
      "command": "npx",
      "args": ["-y", "gitnexus@latest", "mcp"]
    }
  }
}

OpenCode

Add to ~/.config/opencode/config.json:

{
  "mcp": {
    "gitnexus": {
      "command": "npx",
      "args": ["-y", "gitnexus@latest", "mcp"]
    }
  }
}

CodeBuddy

CodeBuddy reads only the first existing file in its config priority chain: ~/.codebuddy/.mcp.json (recommended) → ~/.codebuddy/mcp.json (deprecated) → ~/.codebuddy.json (legacy). Edit the first non-empty file that exists — creating a higher-priority file would hide the servers in the ones below it. If none exist, create ~/.codebuddy/.mcp.json:

{
  "mcpServers": {
    "gitnexus": {
      "command": "npx",
      "args": ["-y", "gitnexus@latest", "mcp"]
    }
  }
}

Qoder

Add to ~/.qoder.json:

{
  "mcpServers": {
    "gitnexus": {
      "command": "npx",
      "args": ["-y", "gitnexus@latest", "mcp"]
    }
  }
}

How It Works

GitNexus builds a complete knowledge graph of your codebase through a multi-phase indexing pipeline:

Structure — Walks the file tree and maps folder/file relationships
Parsing — Extracts functions, classes, methods, and interfaces using Tree-sitter ASTs
Resolution — Resolves imports and function calls across files with language-aware logic
- Field & Property Type Resolution — Tracks field types across classes and interfaces for deep chain resolution (e.g., user.address.city.getName())
- Return-Type-Aware Variable Binding — Infers variable types from function return types, enabling accurate call-result binding
Clustering — Groups related symbols into functional communities
Processes — Traces execution flows from entry points through call chains
Search — Builds hybrid search indexes for fast retrieval

The result is a LadybugDB graph database stored locally in .gitnexus/ with full-text search and semantic embeddings.

Experimental community detection engine

Experimental — not supported for production indexes. The Icebug engine is a research path for #2337. It carries no stability guarantee, may change or be removed without a major version, and partitions differently from the default, so switching engines changes community IDs and any generated context keyed on them. Reindex with graphology before relying on the output.

Community detection uses the bundled Graphology Leiden implementation by default. To try the #2337 Icebug path without changing default analyze behavior, install the optional native package alongside GitNexus and set the engine:

npm i @ladybugmem/icebug
GITNEXUS_COMMUNITY_ENGINE=icebug npx gitnexus analyze

Supported values are graphology, icebug, and auto. Today auto is behaviorally identical to icebug: both try Icebug and fall back to Graphology, while graphology skips Icebug entirely.

Icebug is not a declared dependency — its prebuilds link against system Arrow 24 (libarrow.so.2400), OpenMP, and glibc ≥ 2.38, none of which GitNexus can assume. Analyze falls back to Graphology and reports the reason in progress output when the module is missing, fails to load, or predates the setNumberOfThreads / setSeed controls that reproducible community IDs require (present at icebug-nodejs HEAD, absent from the published 12.8.0 tarball — so the fallback is what you will see today). The engine is pinned to threads: 1, randomize: false for determinism.

Note that the bundled Graphology path is no longer the slow option it once was: #2337 removed an accidental O(communities × N) copy in the vendored Leiden. On a synthetic 200k-node / 800k-edge benchmark graph it went from exceeding the 60s timeout to finishing in ~15s. Real projections vary with their degree distribution, so treat that as a direction, not a guarantee.

MCP Tools

Your AI agent gets 17 tools (15 per-repo + 2 group) automatically:

Tool	What It Does
`list_repos`	Discover all indexed repositories (paginated — `limit`/`offset`)
`query`	Process-grouped hybrid search (BM25 + semantic + RRF)
`context`	360-degree symbol view — categorized refs, process participation
`impact`	Blast radius analysis with depth grouping and confidence
`trace`	Shortest directed path between two symbols (call + class-member edges)
`detect_changes`	Git-diff impact — maps changed lines to affected processes
`check`	Read-only structural checks against the indexed graph
`rename`	Multi-file coordinated rename with graph + text search
`cypher`	Raw Cypher graph queries
`route_map`	API route map — which components fetch which endpoints, and handlers
`tool_map`	MCP/RPC tool definitions — where they're defined and handled
`shape_check`	Validate API response shapes against consumers' property accesses
`api_impact`	Pre-change impact report for an API route handler
`explain`	Explain persisted taint findings (source→sink flows, `--pdg` indexes)
`pdg_query`	Query control/data dependence at statement level (`--pdg` indexes)
`group_list`	List configured repository groups
`group_sync`	Rebuild a group's Contract Registry and cross-repo links

With one indexed repo, the repo param is optional. With multiple, specify which: query({search_query: "auth", repo: "my-app"}). Per-repo tools also take an optional branch for indexes pinned with gitnexus analyze --branch; omitting it queries the workspace index, which follows your checked-out working tree. explain and pdg_query need an index built with gitnexus analyze --pdg.

MCP Resources

Resource	Purpose
`gitnexus://repos`	List all indexed repositories (read first)
`gitnexus://setup`	Setup and usage guidance for agents
`gitnexus://repo/{name}/context`	Codebase stats, staleness check, and available tools
`gitnexus://repo/{name}/clusters`	All functional clusters with cohesion scores
`gitnexus://repo/{name}/cluster/{name}`	Cluster members and details
`gitnexus://repo/{name}/processes`	All execution flows
`gitnexus://repo/{name}/process/{name}`	Full process trace with steps
`gitnexus://repo/{name}/schema`	Graph schema for Cypher queries
`gitnexus://group/{name}/contracts`	A group's extracted contracts and cross-links
`gitnexus://group/{name}/status`	Staleness of repos in a group

MCP Prompts

Prompt	What It Does
`detect_impact`	Pre-commit change analysis — scope, affected processes, risk level
`generate_map`	Architecture documentation from the knowledge graph with mermaid diagrams

CLI Commands

gitnexus setup                   # Configure MCP for detected editors (one-time; use -c to select)
gitnexus uninstall               # Preview removal of GitNexus MCP/skills/hooks (add --force to apply)
gitnexus analyze [path]          # Index a repository (or update stale index)
gitnexus analyze --repair-fts    # Fast path: rebuild/verify only FTS indexes on existing index data
gitnexus analyze --force         # Full rebuild: re-parse + graph rebuild + FTS rebuild
gitnexus analyze --embeddings    # Enable embedding generation (slower, better search)
gitnexus embeddings install      # Fetch the optional local embedding stack on demand (--cuda, --force)
gitnexus analyze --skills        # Generate repo-specific skill files from detected communities
gitnexus analyze --skip-agents-md  # Preserve custom AGENTS.md/CLAUDE.md gitnexus section edits
gitnexus analyze --skip-skills   # Skip installing standard .claude/skills/gitnexus-* skill files
gitnexus analyze --skip-git      # Index folders that are not Git repositories
gitnexus analyze --workers <n>   # Parse worker pool size (>=1; default: cores-1, capped at 16)
gitnexus analyze --verbose       # Log skipped files when parsers are unavailable
gitnexus analyze --max-file-size 1024  # Skip files larger than N KB (default: 512, cap: 32768)
gitnexus analyze --worker-timeout 60  # Increase worker idle timeout for slow parses
gitnexus analyze --wal-checkpoint-threshold 67108864  # 64 MiB. Control LadybugDB WAL auto-checkpoint threshold (default: 67108864 = 64 MiB; -1 keeps Ladybug stock ~16 MiB)
gitnexus mcp                     # Start MCP server (stdio) — serves all indexed repos
gitnexus serve                   # Start local HTTP server (multi-repo) for web UI
gitnexus index                   # Register an existing .gitnexus/ folder into the global registry
gitnexus list                    # List all indexed repositories
gitnexus status                  # Show index status for current repo
gitnexus clean                   # Delete index for current repo
gitnexus clean --all --force     # Delete all indexes
gitnexus wiki [path]             # Generate LLM-powered docs from knowledge graph
gitnexus wiki --model <model>    # Wiki with custom LLM model (default: minimax/minimax-m2.5)
gitnexus wiki --base-url http://llama-box.local:8080/v1 --allow-insecure-connection llama-box.local
                                  # Allow an exact LAN/self-hosted HTTP LLM host; env: GITNEXUS_ALLOW_INSECURE_CONNECTION
gitnexus doctor                  # Show runtime platform capabilities and embedding configuration

# Direct graph queries — the same tools the MCP server exposes, no MCP daemon needed
gitnexus query "<concept>"                                    # Process-grouped hybrid search
gitnexus context <symbol> [--uid <uid> | --file <path>]       # 360° symbol view; flags disambiguate a shared name
gitnexus impact <symbol> [--uid <uid> | --file <path> | --kind <kind>]  # Blast radius; flags disambiguate a shared name
gitnexus trace <from> <to>       # Shortest directed path between two symbols
gitnexus detect-changes          # Map the working-tree diff to affected symbols and execution flows
gitnexus check                   # Read-only structural checks against the indexed graph
gitnexus cypher "<query>"        # Run a raw Cypher query against the knowledge graph

# Repository groups (multi-repo / monorepo service tracking)
gitnexus group create <name>                                   # Create a repository group
gitnexus group add <group> <groupPath> <registryName>          # Add a repo to a group. <groupPath> is a hierarchy path (e.g. hr/hiring/backend); <registryName> is the repo's name from the registry (see `gitnexus list`)
gitnexus group remove <group> <groupPath>                      # Remove a repo from a group by its hierarchy path
gitnexus group list [name]                                     # List groups, or show one group's config
gitnexus group sync <name>                                     # Extract contracts and match across repos/services
gitnexus group contracts <name>  # Inspect extracted contracts and cross-links
gitnexus group query <name> <q>  # Search execution flows across all repos in a group
gitnexus group status <name>     # Check staleness of repos in a group
gitnexus group impact <name> --target <symbol> --repo <groupPath>  # Cross-repo blast radius

gitnexus uninstall reverses gitnexus setup — it removes the GitNexus MCP entries, hooks, and skill directories it added to each detected editor. Skill directories are identified by bundled gitnexus skill name (e.g. gitnexus-cli/), so if you customized files inside an installed skill directory, back them up first. It is a dry-run preview by default and prints the exact paths it would remove; pass --force to apply. Per-repo indexes (gitnexus clean --all) and the global npm package (npm uninstall -g gitnexus) are left for you to remove.

Remote Embeddings

Set these env vars to use a remote OpenAI-compatible /v1/embeddings endpoint instead of the local model:

export GITNEXUS_EMBEDDING_URL=http://your-server:8080/v1
export GITNEXUS_EMBEDDING_MODEL=BAAI/bge-large-en-v1.5
export GITNEXUS_EMBEDDING_DIMS=1024          # optional, default 384
export GITNEXUS_EMBEDDING_REQUEST_DIMS=omit  # optional: omit "dimensions", or an integer to override it
export GITNEXUS_EMBEDDING_API_KEY=your-key   # optional, default: "unused"
export GITNEXUS_EMBEDDING_MAX_ATTEMPTS=3     # optional, total attempts (1-20)
export GITNEXUS_EMBEDDING_RETRY_CAP_MS=5000  # optional, maximum retry delay
export GITNEXUS_EMBEDDING_MIN_INTERVAL_MS=0  # optional, minimum request spacing
export GITNEXUS_EMBEDDING_HTTP_TIMEOUT_MS=180000 # optional, per-request timeout (max 300000)
gitnexus analyze . --embeddings

GITNEXUS_EMBEDDING_REQUEST_DIMS controls only the dimensions field sent in the request body, independently of GITNEXUS_EMBEDDING_DIMS (which still validates the returned vector's length):

omit (or none, off, false, 0) — do not send dimensions at all, for strict backends that return the right vector size but reject the field.
a positive integer — send that value instead of GITNEXUS_EMBEDDING_DIMS.
unset — send GITNEXUS_EMBEDDING_DIMS (the previous behavior).

Works with Infinity, vLLM, TEI, llama.cpp, Ollama, LM Studio, or OpenAI. Retry and pacing settings are provider-neutral; provider-specific limits should be supplied through configuration. When unset, local embeddings are used unchanged.

JVM Package Sibling Injection

Java and Kotlin files in the same package receive implicit sibling class bindings to resolve same-package references. By default, GitNexus injects at most 200 siblings per module scope, nearest first by path. Set GITNEXUS_MAX_INJECTED_SIBLINGS=0 to remove that per-file limit; this can substantially increase indexing work for large packages.

export GITNEXUS_MAX_INJECTED_SIBLINGS=200
gitnexus analyze .

When the limit truncates a file's sibling set, that file is marked visibility-incomplete: same-package references still resolve through the injected siblings, but wildcard-import attribution (used by the Spring bean/DI/config passes) is disabled for it rather than resolved against a partial view. Analyze logs a sibling injection truncated warning naming how many files were affected.

Packages with more than 500 files are a separate, fixed limit: they are skipped entirely (logged as skipping package with N files) and every file in them is marked visibility-incomplete. GITNEXUS_MAX_INJECTED_SIBLINGS does not lift that skip — including at 0.

Multi-Repo Support

GitNexus supports indexing multiple repositories. Each gitnexus analyze registers the repo in a global registry (~/.gitnexus/registry.json). The MCP server serves all indexed repos automatically.

Supported Languages

TypeScript, JavaScript, Python, Java, C, C++, C#, Go, Rust, PHP, Kotlin, Swift, Ruby, Dart

Language Feature Matrix

Language	Imports	Named Bindings	Exports	Heritage	Type Annotations	Constructor Inference	Config	Frameworks	Entry Points
TypeScript	✓	✓	✓	✓	✓	✓	✓	✓	✓
JavaScript	✓	✓	✓	✓	—	✓	✓	✓	✓
Python	✓	✓	✓	✓	✓	✓	✓	✓	✓
Java	✓	✓	✓	✓	✓	✓	—	✓	✓
Kotlin	✓	✓	✓	✓	✓	✓	—	✓	✓
C#	✓	✓	✓	✓	✓	✓	✓	✓	✓
Go	✓	—	✓	✓	✓	✓	✓	✓	✓
Rust	✓	✓	✓	✓	✓	✓	—	✓	✓
PHP	✓	✓	✓	—	✓	✓	✓	✓	✓
Ruby	✓	—	✓	✓	—	✓	—	✓	✓
Swift	—	—	✓	✓	✓	✓	✓	✓	✓
C	—	—	✓	—	✓	✓	—	✓	✓
C++	—	—	✓	✓	✓	✓	—	✓	✓
Dart	✓	—	✓	✓	✓	✓	—	✓	✓

Imports — cross-file import resolution · Named Bindings — import { X as Y } / re-export tracking · Exports — public/exported symbol detection · Heritage — class inheritance, interfaces, mixins · Type Annotations — explicit type extraction for receiver resolution · Constructor Inference — infer receiver type from constructor calls (self/this resolution included for all languages) · Config — language toolchain config parsing (tsconfig, go.mod, etc.) · Frameworks — AST-based framework pattern detection · Entry Points — entry point scoring heuristics

Agent Skills

GitNexus ships with skill files that teach AI agents how to use the tools effectively:

Exploring — Navigate unfamiliar code using the knowledge graph
Debugging — Trace bugs through call chains
Impact Analysis — Analyze blast radius before changes
Refactoring — Plan safe refactors using dependency mapping
Guide — GitNexus tool/resource/schema reference for the agent
CLI — Run analyze/status/clean/wiki commands on request
PDG Query — Statement-level control/data dependence queries (--pdg index)
Taint Analysis — Source→sink data-flow findings (--pdg index)
Plan / Work / Review / LFG — The engineering family: implementation-ready plans, gated plan execution, graph-backed change review with taint + expert lenses, and the end-to-end pipeline

Installed automatically by both gitnexus analyze (per-repo) and gitnexus setup (global). Run gitnexus analyze --skills to additionally generate each detected functional area as a direct project skill under .claude/skills/gitnexus-area-<name>/.

Requirements

Node.js >= 22
Git repository (uses git for commit tracking)
Linux: glibc 2.34 or newer (Ubuntu 22.04+, RHEL/Rocky/Alma 9+, Debian 12+, Fedora 35+). The LadybugDB native binary ships as a prebuild against that floor, so on an older host it cannot load and reinstalling does not help — see Linux: GLIBC_2.34' not found.
Windows, for full-text search: the Microsoft Visual C++ 2015-2022 Redistributable (x64) and OpenSSL 3 (libssl-3-x64.dll, libcrypto-3-x64.dll) resolvable on PATH — see Windows: full-text search unavailable.

Release candidates

Stable releases publish to the default latest dist-tag. When a pull request with non-documentation changes merges into main, an automated workflow also publishes a prerelease build under the rc dist-tag, so early adopters can try in-flight fixes without waiting for the next stable cut. (Docs-only merges are skipped.)

# Try the latest release candidate (pre-stable — may change at any time)
npm install -g gitnexus@rc
# — or —
npx gitnexus@rc analyze

Release-candidate versions follow the standard semver prerelease format X.Y.Z-rc.N, where X.Y.Z is the next stable target (bumped from the current latest by patch by default; minor or major when kicking off a bigger cycle) and N increments per published rc. Example sequence: 1.6.2-rc.1, 1.6.2-rc.2, …, then once 1.6.2 ships stable, 1.6.3-rc.1. See the Releases page for the full list; stable latest is unaffected.

Troubleshooting

`Cannot destructure property 'package' of 'node.target' as it is null`

This error comes from npm 11.x's arborist while installing gitnexus (often via npx), before gitnexus code runs. It is triggered by platform-filtered optionalDependencies in native packages such as onnxruntime-node / @huggingface/transformers (used when indexing with --embeddings). GitNexus cannot catch it at runtime — use one of these workarounds:

pnpm --allow-build=@ladybugdb/core --allow-build=gitnexus --allow-build=tree-sitter dlx gitnexus@latest analyze       # auto-selected when pnpm + npm 11+
npm install -g gitnexus@latest         # global install avoids per-run npx reify
gitnexus analyze                       # if already installed globally

On pnpm 10+, lifecycle scripts are blocked unless explicitly allowed — the resolver adds --allow-build for @ladybugdb/core, gitnexus, and tree-sitter automatically when it picks pnpm dlx.

If you must stay on npm 11.x without pnpm, downgrade npm toolchain-wide (last resort):

npm install -g npm@10.9.0

See #1939 and the original #819 thread. An older variant of this crash (tree-sitter-dart tarball URL) was fixed in gitnexus v1.6.2+ (#820); if you still see install failures after upgrading, clear cache:

npm cache clean --force
npx gitnexus@latest analyze

`ERR_DLOPEN_FAILED` / `lbugjs.node` missing (pnpm dlx, pnpx)

GitNexus depends on @ladybugdb/core, whose native database addon (lbugjs.node) is placed by a postinstall script. pnpm dlx, pnpx, and any install run with --ignore-scripts skip lifecycle scripts, so the addon is never put in place and the runtime crashes with ERR_DLOPEN_FAILED:

Error: dlopen(.../@ladybugdb/core/lbugjs.node, ...): tried: '...' (no such file)
  code: 'ERR_DLOPEN_FAILED'

Options that run install scripts:

# pnpm dlx with explicit build permission (one-off, no global install required)
pnpm --allow-build=@ladybugdb/core --allow-build=gitnexus --allow-build=tree-sitter \
  dlx gitnexus@latest serve

# npm: global install (recommended on npm 11+; bare npx may crash — see section above)
npm install -g gitnexus@latest
gitnexus serve

# npx (npm < 11, or after upgrading npm)
npx gitnexus@latest serve

# pnpm: global install with build scripts allowed (pnpm 10.2+; no approve-builds -g on pnpm 11+)
pnpm add -g --allow-build=@ladybugdb/core --allow-build=gitnexus --allow-build=tree-sitter gitnexus
gitnexus serve

Linux: `GLIBC_2.34' not found`

LadybugDB native binary (lbugjs.node) exists but failed to load:
  /lib64/libc.so.6: version `GLIBC_2.34' not found (required by .../lbugjs.node)

The LadybugDB addon ships as a prebuilt binary compiled against glibc 2.34. If your distribution is older (CentOS/RHEL 8 has 2.28, Ubuntu 20.04 has 2.31, Debian 11 has 2.31), the dynamic loader cannot resolve its symbols.

Reinstalling does not help — every download delivers the same prebuilt binary. The fix is a newer C library:

Run GitNexus on a distribution with glibc 2.34 or newer — Ubuntu 22.04+, RHEL/Rocky/Alma 9+, Debian 12+, Fedora 35+.
Or run it in the container image, which bundles a current glibc (see Docker).

gitnexus doctor reports the required and detected glibc versions when this happens (#2672).

Windows: full-text search unavailable

analyze completes, but keyword search is degraded and doctor shows the FTS extension failing with Windows error 126 (The specified module could not be found). The extension needs two runtime dependencies Windows does not ship by default:

Microsoft Visual C++ 2015-2022 Redistributable (x64) — https://aka.ms/vs/17/release/vc_redist.x64.exe
OpenSSL 3 — libssl-3-x64.dll and libcrypto-3-x64.dll, resolvable on PATH

The redistributable alone is not sufficient. If Git for Windows is installed you already have the OpenSSL DLLs — run gitnexus from Git Bash, or prepend the directory to PATH in the shell you use:

$env:PATH = "C:\Program Files\Git\mingw64\bin;$env:PATH"
gitnexus analyze --repair-fts

Without them the index is still built, but without search tables, so query returns empty keyword results until you re-run gitnexus analyze --repair-fts from a shell where the DLLs resolve (#2669).

Installation fails with native module errors

Some optional language grammars (Dart, Proto, Swift, Kotlin) require native compilation. If they fail, GitNexus still works — those languages will be skipped. To skip them intentionally (no C++ toolchain needed), set GITNEXUS_SKIP_OPTIONAL_GRAMMARS=1 before installing.

If npm install -g gitnexus fails on native modules:

# Ensure build tools are available (Linux/macOS)
# Ubuntu/Debian: sudo apt install python3 make g++
# macOS: xcode-select --install

# Retry installation
npm install -g gitnexus

Installation fails behind an HTTP proxy (`onnxruntime-node` postinstall)

onnxruntime-node's postinstall downloads optional CUDA GPU binaries from api.nuget.org — outside the npm registry, so registry mirrors don't cover it, and its proxy layer (global-agent) ignores the standard HTTP_PROXY/HTTPS_PROXY variables and rejects 302 redirects (#2370).

Since the packages are optional dependencies, a failed download no longer breaks npm install -g gitnexus — npm skips the embedding stack and everything else works. The stack then self-heals on demand: the first gitnexus analyze --embeddings (or an explicit gitnexus embeddings install) fetches it through your configured npm registry — mirrors and proxies apply, no NuGet download involved — into ~/.gitnexus/embedding-runtime.

# heal a proxy-degraded install manually (CPU embeddings; registry-only)
gitnexus embeddings install

# reinstall into the prefix even when the stack already resolves
gitnexus embeddings install --force

# CUDA GPU hosts: also fetch GPU binaries (NuGet; set the proxy global-agent reads)
GLOBAL_AGENT_HTTPS_PROXY=<proxy-url> gitnexus embeddings install --cuda

The prefix defaults to ~/.gitnexus/embedding-runtime; set GITNEXUS_EMBEDDING_RUNTIME_DIR to install it elsewhere (e.g. a writable path in a container).

Node requirement for the on-demand prefix: the self-heal loads the prefixed packages via module.registerHooks, available on Node ≥ 22.15 (on the 22.x line) or ≥ 23.5 (on the 23.x line). On an older Node the packages install but can't be loaded from the prefix — reinstall them into the install itself instead (works on every supported Node): ONNXRUNTIME_NODE_INSTALL=skip npm install -g gitnexus (Windows: set ONNXRUNTIME_NODE_INSTALL=skip && npm install -g gitnexus). Skipping only the CUDA download keeps full CPU embeddings (CPU embeddings don't need it). Check the result any time with gitnexus doctor (Embeddings → Support line).

Analyze warns about unavailable FTS or VECTOR extensions

GitNexus uses optional DuckDB extensions for BM25 and vector search. The gitnexus serve and MCP read paths only ever try to LOAD the extensions — they never block on a network install. The analyze command, by default, attempts one bounded out-of-process install if LOAD fails (a plain INSTALL to download a missing extension, escalating to FORCE INSTALL only when the LOAD error shows the existing file is broken or truncated, so a permanent non-file failure does not re-download on every run) and proceeds even when that install times out, so the index is always written to disk; BM25/vector search degrade gracefully until the extensions become available.

Configure the behavior with these environment variables:

Variable	Values	Default	Effect
`GITNEXUS_LBUG_EXTENSION_INSTALL`	`auto`, `load-only`, `never`	`auto`	`auto` runs one bounded install if LOAD fails — a plain `INSTALL`, escalating to `FORCE INSTALL` only when the LOAD error shows the present extension file is broken. `load-only` only uses already-installed extensions (recommended for offline / firewalled environments). `never` skips optional extensions entirely.
`GITNEXUS_LBUG_EXTENSION_INSTALL_TIMEOUT_MS`	positive integer	`15000`	Wall-clock budget for the out-of-process extension-install child before it is killed.
`GITNEXUS_FTS_STEMMER`	supported LadybugDB stemmer	`porter`	Stemmer used when rebuilding BM25/FTS indexes. Use `none` for CJK-heavy repositories, or a language stemmer such as `german`, `french`, or `spanish` when that better matches repository comments and identifiers. Re-run `gitnexus analyze --repair-fts` after changing it.
`GITNEXUS_FTS_CJK_SEGMENTATION`	`none`, `bigram`	`none`	`bigram` inserts overlapping character-bigram boundaries into Chinese/Japanese Han-ideograph spans in `content`/`description` before FTS indexing, so LadybugDB's space-only tokenizer can see sub-phrase word boundaries. Scoped to CJK Unified Ideographs only — Japanese Hiragana/Katakana and Korean Hangul are not currently segmented. Unlike `GITNEXUS_FTS_STEMMER`, this rewrites stored text — enabling it on an already-indexed repo requires a full `gitnexus analyze --force`; neither `--repair-fts` nor a plain incremental `analyze` applies it to previously-indexed files. Set the same value wherever `analyze` and search-serving processes (CLI query, MCP server, web server) run.
`GITNEXUS_STREAM_GRAPH_EMIT`	`0`, `1`	`1` (on)	On by default on a full rebuild (`--force`); incremental runs ignore it. Holds structural relationships (CALLS, IMPORTS, ACCESSES, CONTAINS, ...) as CSV-on-disk plus compact in-memory columns instead of as objects in three overlapping indexes, cutting peak in-memory graph heap by ~1.4x at no measurable CPU cost (measured A/B on a synthetic 400k-node / 1.08M-edge graph: 819 MB -> 584 MB, iteration at parity, scaling verified linear from 100k to 800k nodes, with every edge still visible through the graph interface; no end-to-end measurement on a real repository yet). Nothing is traded away — community detection, process extraction, PDG taint summaries and the local-symbol pruner all read a complete relationship set and behave identically. Set to `0` only to bisect a suspected streaming-related fault.
`GITNEXUS_COMMUNITY_ENGINE`	`graphology`, `icebug`, `auto`	`graphology`	Community-detection engine used during analyze. `graphology` is the supported default. `icebug` and `auto` are experimental and currently behave identically: both try the optional `@ladybugmem/icebug` native Leiden over a CSR export and fall back to Graphology if it is not installed, cannot load, or lacks the deterministic thread/seed controls. Experimental engines partition differently, so community IDs are not comparable across engines.
`GITNEXUS_WAL_CHECKPOINT_THRESHOLD`	integer `>= -1`	`67108864` (64 MiB)	LadybugDB WAL auto-checkpoint threshold during analyze (bytes). Auto-checkpoint remains enabled; `-1` keeps Ladybug's stock ~16 MiB. Larger thresholds reduce checkpoint frequency but increase the WAL size at rotation time — choose a smaller value on disk-constrained environments.
`GITNEXUS_LBUG_BUFFER_POOL_SIZE`	integer `>= 0` (bytes)	min(2 GiB, 80% RAM)	LadybugDB buffer-pool ceiling for every GitNexus database (analyze, MCP server, serve, group bridges). Bounded so a long-lived `gitnexus mcp` process or a large incremental `analyze` cannot grow toward LadybugDB's native 80%-of-RAM default and OOM the host (#2557). `0` restores that native unbounded default; invalid values warn and fall back to the default. During `analyze` the pool is right-sized to the graph and, on non-4 KiB-page hosts (Apple Silicon 16 KiB, Ascend/aarch64 64 KiB), scaled by the page-size granule ratio up to min(2 GiB × pageSize/4 KiB, 80% RAM) (#2631); this env var overrides all of that as an absolute value.
`GITNEXUS_LBUG_MAX_DB_SIZE`	positive integer (bytes)	`17179869184` (16 GiB)	Upper bound for a single LadybugDB database file. This is an mmap/disk-address-space ceiling, not a memory limit — it does not constrain the buffer pool (use `GITNEXUS_LBUG_BUFFER_POOL_SIZE` for that). Raise it when indexing genuinely huge monorepos; invalid values silently fall back to the default.

# Offline/airgapped: never reach the network for extensions
GITNEXUS_LBUG_EXTENSION_INSTALL=load-only npx gitnexus analyze

# Slow network: give extension downloads more time
GITNEXUS_LBUG_EXTENSION_INSTALL_TIMEOUT_MS=30000 npx gitnexus analyze

# CJK-heavy codebase: rebuild keyword indexes without English stemming
GITNEXUS_FTS_STEMMER=none npx gitnexus analyze --repair-fts

# CJK-heavy codebase: enable sub-phrase search over Chinese/Japanese Han text.
# On an already-indexed repo, the first run after enabling this MUST be --force —
# --repair-fts and plain incremental `analyze` both leave old files un-segmented.
GITNEXUS_FTS_CJK_SEGMENTATION=bigram npx gitnexus analyze --force

Analysis runs out of memory

Memory management is automatic: analyze sizes its heap to the machine (always below physical RAM), caps each parse worker, and — rather than grinding into a GC death spiral or crash — stops early with a message telling you the one thing to do. Repeated Replacement worker did not report ready within 5000ms warnings on a large repository are part of the same picture: memory pressure starving healthy workers, not a worker bug (#2649).

If analyze says the repository doesn't fit, do what the message says:

The machine has more memory to give (a NODE_OPTIONS --max-old-space-size pin from your environment is holding analyze back): re-run without the pin — no flags needed.
The machine is the ceiling: shrink the scope (exclude generated or vendored directories, below) or use a machine with more RAM.

Escape hatches (GITNEXUS_MEMORY=off to decline the autopilot, GITNEXUS_WORKER_HEAP_MB to size workers yourself) are listed in the environment-variable table below — most users never need them.

For very large repositories:

# Increase Node.js heap size
NODE_OPTIONS="--max-old-space-size=16384" npx gitnexus analyze

# Exclude large directories (this repo only)
echo "vendor/" >> .gitnexusignore
echo "dist/" >> .gitnexusignore

# Exclude a directory across every repo you index, without touching each
# repo's own .gitnexusignore or needing push/commit access to it. GitNexus
# reads the same sources `git` itself does: core.excludesFile (all repos)
# and $GIT_DIR/info/exclude (this repo only, untracked). A repo's own
# .gitignore/.gitnexusignore can still override either with a `!pattern`
# negation. Skip both entirely with GITNEXUS_NO_GLOBAL_IGNORE=1.
git config --global core.excludesFile ~/.gitignore_global   # applies to every repo
echo "docs/" >> ~/.gitignore_global
echo "build/" >> .git/info/exclude                          # this repo only, untracked

Large files are being skipped

By default the walker skips files larger than 512 KB (see log line Skipped N large files (>512KB)). Raise the threshold via either the CLI flag or the environment variable — both accept a value in KB:

# CLI flag (takes precedence over the env var)
npx gitnexus analyze --max-file-size 2048     # skip only files > 2 MB

# Environment variable (persists across commands)
export GITNEXUS_MAX_FILE_SIZE=2048
npx gitnexus analyze

Values above 32768 KB (32 MB) are clamped to the tree-sitter parser ceiling; invalid values fall back to the 512 KB default with a one-time warning. When an override is active, analyze prints the effective threshold in its startup banner (e.g. GITNEXUS_MAX_FILE_SIZE: effective threshold 2048KB (default 512KB)).

Analyze reports a worker timeout

Worker parse timeouts are recoverable. GitNexus retries stalled worker jobs with backoff, splits large jobs to isolate slow files, and quarantines a file that repeatedly crashes its worker (respawning the slot so the pool keeps going). If a large repository needs more time per worker job, use either:

# CLI flag, in seconds
npx gitnexus analyze --worker-timeout 60

# Environment variable, in milliseconds
export GITNEXUS_WORKER_SUB_BATCH_TIMEOUT_MS=60000
npx gitnexus analyze

For repositories with very large source files, GITNEXUS_WORKER_SUB_BATCH_MAX_BYTES controls the worker job byte budget. The default is 8388608 bytes (8 MB).

Worker pool resilience tuning

Four env vars expose the pool's resilience layers (respawn budget, cumulative-timeout cap, circuit breaker, startup handshake). Defaults are tuned for typical repos; bump them when an analyze legitimately needs more retries, or lower them to fail-fast on a known-bad shape.

Variable	Default	Effect
`GITNEXUS_WORKER_MAX_RESPAWNS_PER_SLOT`	`3`	Max replacement spawns per slot before the slot is dropped from the active rotation.
`GITNEXUS_WORKER_MAX_CUMULATIVE_TIMEOUT_MS`	`5 × subBatchTimeoutMs`	Total retry wall-time budget per job before quarantining. Bounds exponentially-growing retry waits.
`GITNEXUS_WORKER_CONSECUTIVE_FAILURE_THRESHOLD`	`max(3, poolSize)`	Per-slot consecutive deaths before the pool's circuit breaker trips. After tripping, dispatches require a fresh pool.
`GITNEXUS_WORKER_SHUTDOWN_DRAIN_MS`	`30000`	Max wait at pool shutdown for a retired worker still inside native code — terminated at its next JS-safe point instead of mid-native-call, which would abort the process (`Napi::Error`, #2432).
`GITNEXUS_WORKER_READY_TIMEOUT_MS`	`5000`	Startup budget for a parse worker to load its grammar bindings and report `{type:'ready'}`. Slots that miss it are treated as startup crashes. Raise it on a slow or heavily loaded host where a full pool cold-starting concurrently needs more than 5s.
`GITNEXUS_MEMORY`	`off`	unset (autopilot on)
`GITNEXUS_WORKER_HEAP_MB`	`clamp(512, RAM/2/poolSize, 4096)`	Per-worker V8 old-generation heap cap (#2649). Bounds pool RSS on large repos; a worker exceeding it dies with a real heap error handled by quarantine/respawn.
`GITNEXUS_SERVER_ANALYZE_HEAP_MB`	`min(8192, auto cap)`	Heap for the web/MCP server's forked analyze worker (#2649). Defaults to the historical 8192 MB bounded by the machine/container's RAM-aware auto cap; set an absolute MB value to override.
`GITNEXUS_CPP_CAPTURE_BUDGET_MS`	`20000`	Per-file wall-clock budget for C++ capture extraction; on breach the file keeps partial captures with a warning (#2432). `0` expires immediately.

Graph cleanup tuning

After scope resolution, analyze prunes inert block-local value symbols (a function-local const/let/var that ends up with only its structural File→DEFINES edge) to keep the graph focused on cross-symbol relationships. Module/file-scope symbols, class members, and any local with a real edge are always kept.

Variable	Default	Effect
`GITNEXUS_KEEP_LOCAL_VALUE_SYMBOLS`	unset	Set to `1`/`true` to keep inert block-local value symbols instead of pruning them.

Programmatic callers can pass keepLocalValueSymbols: true in PipelineOptions instead of setting the env var.

Scope-resolution property-key dispatch cap

During scope resolution GitNexus synthesizes CALLS edges through property-key dispatch — call sites like hooks.emitScopeCaptures() where a property key is registered by multiple definitions across the codebase. To keep this fan-in bounded, each property key is capped at 32 registrations: a key registered by more than 32 distinct functions is skipped entirely (no CALLS are synthesized through it), and the dropped key names are surfaced in the analyze log for operator visibility. The cap is calibrated at 2× this repo's own provider table (16 legitimate registrations, one per language provider).

Variable	Default	Effect
`GITNEXUS_MAX_PROPERTY_DISPATCH_FANOUT`	`32`	Per-property-key registration cap in the property-dispatch scope-resolution pass. Set to a positive integer to raise it for repositories whose provider/hook tables exceed the default and lose CALLS coverage on a legitimate key; non-integer or `< 1` values fall back to `32`. Lowering it tightens the overflow budget.

# A property key registered by 40 functions overflows the default 32 and drops
# all CALLS through it — raise the cap for that repo and rebuild so scope
# resolution reruns.
export GITNEXUS_MAX_PROPERTY_DISPATCH_FANOUT=64
npx gitnexus analyze --force

Scope-resolution dispatch-target cap

During scope resolution GitNexus resolves calls that flow through callable values — function/method references bound to variables, passed as arguments, or stored in maps/tables. To keep that inclusion-based resolution finite, each callable site is capped at 32 dispatch targets. When a site gathers more candidates than the cap it is treated as overflowed and all of its call edges are dropped — a cliff, not a tail, so a repository with a legitimately wide dispatch table (a single callable site resolving to 33+ targets) loses that site's whole call chain. In that case analyze logs callable-value-flow: candidate set exceeded the cap; no partial CALLS emitted alongside a warning carrying the language, the overflowing context, the candidate count, and the cap (32).

Raise the cap for such repositories:

Variable	Default	Effect
`GITNEXUS_MAX_CALLABLE_VALUE_TARGETS`	`32`	Per-callable-site dispatch-target cap in the callable-value-flow scope-resolution pass. Set to a positive integer to raise it for repositories whose wide dispatch tables overflow the default and lose a whole call chain; non-integer or `< 1` values fall back to `32`. Lowering it tightens the overflow budget.

# A callable site resolving to 48 targets overflows the default 32 and drops
# the chain — raise the cap for that repo and rebuild so scope resolution reruns.
export GITNEXUS_MAX_CALLABLE_VALUE_TARGETS=64
npx gitnexus analyze --force

Hook augmentation and skip diagnostics

The Claude Code / Antigravity hooks keep their stderr silent on normal skip paths so strict hook runners (e.g. Codex PreToolUse) never see unexpected diagnostic output.

When a GitNexus process holds the repo DB write lock (the common case — the MCP server is running, or the DB-lock probe timed out and failed closed), the local CLI augment can't run (LadybugDB is single-writer). Rather than drop the augmentation, the hook hands the agent a short, conditional MCP-query hint on stdout (the sanctioned additionalContext channel) — "if the GitNexus MCP tools are live in this session, call query …" — so an agent that has the tools can still fetch graph-ranked context. The hint is throttled to at most once per repo per window (GITNEXUS_MCP_HINT_THROTTLE_MS, default 10 min; 0 disables), so an owner-locked session isn't nudged on every search. A stale-index reminder, or an already-current index, stays silent.

To see why a hook skipped the CLI augment, set GITNEXUS_DEBUG=1 and re-run the action — the hook writes the reason (e.g. [GitNexus] augment skipped: MCP server owns DB) and the stale-index hint to its stderr:

GITNEXUS_DEBUG=1 <your command>   # surfaces hook skip/diagnostic reasons on stderr

Only GITNEXUS_DEBUG=1 and GITNEXUS_DEBUG=true enable diagnostics; every other value (including 0 and false) is treated as off. Diagnostics go to stderr only — the hook's structured stdout (the JSON the agent consumes) is unaffected.

Privacy

All processing happens locally on your machine
No code is sent to any server
Index stored in .gitnexus/ inside your repo (gitignored)
Global registry at ~/.gitnexus/ stores only paths and metadata

Web UI

GitNexus also has a browser-based UI at gitnexus.vercel.app — 100% client-side, your code never leaves the browser.

Local Backend Mode: Run gitnexus serve and open the web UI locally — it auto-detects the server and shows all your indexed repos, with full AI chat support. No need to re-upload or re-index. The agent's tools (Cypher queries, search, code navigation) route through the backend HTTP API automatically.

License

PolyForm Noncommercial 1.0.0

Free for non-commercial use. Contact for commercial licensing.

README.md Unescape Escape