c was excluded from the update monitor, so an upstream c update went unnoticed.
Include it, but as report-only via a `hold`: c is ABI-pinned at 0.21.4
(#1242/#858) and must not auto-bump without a tree-sitter runtime upgrade, so an
available c update is detected + surfaced (notice + job summary) but never
auto-PR'd — even if it were ABI-13/14. `--apply c` refuses defensively. (Live:
upstream c is 0.24.1 / ABI 15 today, so c is doubly held — reported, not applied.)
Adds a weekly (+ dispatchable) workflow that checks each vendored grammar against
its source-of-origin (npm for swift/kotlin, the GitHub default branch for
dart/proto; c is excluded — held at 0.21.4 for ABI safety) and opens a PR
re-vendoring any update that is ABI-COMPATIBLE with the pinned tree-sitter@0.21.1
(LANGUAGE_VERSION 13-14).
ABI awareness is the point: most upstreams have moved to ABI 15 (newer
tree-sitter), so a blind "bump to latest" would open PRs that can't build. The
monitor fetches the candidate source, reads its parser.c LANGUAGE_VERSION, and
only re-vendors 13/14 — incompatible updates are reported (notice + job summary),
never applied. (Confirmed live: dart/proto upstreams are ABI 15 today and are
correctly held; swift/kotlin are current.)
The re-vendor refreshes only the source-build inputs + runtime entrypoints,
preserving the GitNexus-hardened binding.gyp / README / prebuilds; the version
bump then triggers build-tree-sitter-prebuilds.yml, whose ABI-validation is the
final safety net so a subtly-wrong re-vendor can't silently ship. PR creation is
gated on the RELEASE_APP secret (skips with a notice if absent), mirroring the
build aggregate. Unit test locks the ABI gate; the script is import-safe.
The npm-pack-based guard timed out in CI: `npm pack`'s prepare/prepack lifecycle
is not skipped by `--ignore-scripts` (flag or env config) on the CI npm, so the
inner pack ran the full build (~20s+) — fine for the slow smoke job, but it blew
past vitest's 30s test timeout in the coverage job (and risked re-entering this
prepack guard).
Replace it with a deterministic, fast (~0.1s) check that needs no subprocess:
since `files: ["vendor"]` OVERRIDES `.npmignore` for the vendored subtree (so
`.npmignore` can never drop vendored source — verified), the ONLY lever that can
exclude source is narrowing the package.json `files` field. The guard now reads
`files` directly: a grammar "ships source" iff `files` includes the vendor
subtree AND the grammar carries a buildable source set on disk. A lean publish
that narrows `files` while a grammar lacks 6/6 prebuilds still fails the gate.
The prepack guard shelled out to `npm pack --dry-run --ignore-scripts --json`,
but the `--ignore-scripts` flag is not reliably honored by npm pack's
prepare/prepack lifecycle on the CI npm — so build.js ran, polluted the --json
stdout with `[build] …`, and the guard's JSON.parse threw. That broke every
`npm pack` (packaged-install-smoke on ubuntu+windows) and failed the guard's own
real-repo unit test (the only coverage-job failure). Force script-skipping via
the reliable `npm_config_ignore_scripts` env config (also removes the prepack
re-entry/recursion risk) and parse defensively from the JSON-array start.
If renameSync(partial, dest) failed AND the rollback renameSync(backup, dest)
also failed, the grammar was left unmaterialized (node_modules/<name> missing)
with only a generic "could not materialize" warning — the recoverable backup at
<dest>.materialize-bak was unmentioned. Emit a CRITICAL warning naming the
backup path and the recovery command on that double-failure, and document that
the fail-soft catch removes only the scratch `partial`, never the `backup`
(which may be the sole recoverable copy). Never-throw / exit-0 contract intact.
AGENTS.md still said postinstall "patches tree-sitter-swift, builds
tree-sitter-proto" and that only kotlin/swift are "optional". Update to the
vendored-uniform model: postinstall materializes the vendored grammars and
prefers a committed prebuild (source-build only when none matches); c is
required while dart/proto/swift/kotlin are optional + skippable via
GITNEXUS_SKIP_OPTIONAL_GRAMMARS=1, with non-fatal warnings only on a
toolchain-less host with no matching prebuild.
c's binding.gyp used an unconditional `cflags_c: ["-std=c11"]`, while
kotlin/swift gate MSVC flags behind an `OS=='win'` condition (/std:c11 /utf-8).
Inert today (no non-ASCII bytes in c's parser.c, and node-gyp ignores cflags_c
on MSVC anyway), but align the three so a future source-build fallback on
Windows behaves consistently.
Every vendored grammar's index.js does `require("node-gyp-build")` at runtime
to load even a prebuilt .node, so node-gyp-build is runtime-load-critical (and
node-addon-api is needed for the source-build fallback). They were
optionalDependencies, surviving `--omit=optional` only via the required
tree-sitter's transitive edge — correct today but fragile. Promote both to
regular dependencies so the contract is explicit (optionalDependencies is now
empty and removed). Lock the contract with a cli-commands assertion.
The strict 6/6 prebuild assertion was dormant whenever a grammar vendors source
(binding.gyp) — which is every grammar — so a dropped prebuild passed CI
silently. Add a FULLY_PREBUILT allowlist of grammars GitNexus has committed 6/6
for (today: swift); those must keep all six even with a source fallback, so
losing one now fails CI. Grammars graduate into the set as the
build-tree-sitter-prebuilds workflow lands their binaries. (The static-import
degradation smoke is covered by the registry-import-closure regression test
extended in the C lazy-load commit.)
The publish guard inferred "is source shipped?" from a single .npmignore toggle
line, which a partial/out-of-order edit could defeat (exclude binding.gyp but
leave parser.c → unbuildable yet "source-shipping"). It now inspects the
EFFECTIVE tarball via `npm pack --dry-run --ignore-scripts --json` (the
--ignore-scripts avoids re-entering this guard through prepack): a grammar
"ships source" only when EVERY on-disk source-build input (binding.gyp +
binding.cc + parser.c + scanner.c when present + a tree_sitter header) is
actually in the packed file list.
This also surfaced that the gated lean-publish .npmignore block was inert:
package.json's `files: ["vendor"]` allow-list overrides .npmignore for the
vendored subtree, so those exclusion lines never dropped anything. Replace the
dead toggle with documentation of the real mechanism (narrow the `files` field)
and note the guard enforces safety on the effective pack regardless of how the
slim is done.
`inputs.open_pr` is null on pull_request events, and the prior
`inputs.open_pr != false` leg relied on GHA's direction-ambiguous null
coercion (Codex F4) to decide whether to open the prebuild PR. Gate
explicitly on the event: a non-fork pull_request that bumped a grammar
version opens the prebuild PR (the documented flow), and `open_pr` is only
consulted on workflow_dispatch — so a manual run with open_pr=false stays
artifacts-only and no event's behavior rests on coercion.
`gitnexus/package-lock.json` changes on nearly every dependency PR, so it
fired the prebuild workflow's guard job on unrelated churn (the matrix stayed
correctly skipped — `gitnexus/package.json` already covers the transition-window
pin, so removing the lock only drops guard noise). Also bump the native build
job timeout 30 -> 45 min for headroom compiling the 23 MB kotlin / 18 MB swift
parser.c, especially under arm emulation.
The aggregate job mints a GitHub App token as its first step; with
RELEASE_APP_ID/RELEASE_APP_PRIVATE_KEY unset it hard-failed AFTER a full
(up-to-6-runner) native build. Since the `secrets` context isn't available in
a job-level `if:`, the guard job now computes a `release_app` boolean output
(a step can read secrets) and emits an actionable `::notice::`; aggregate
gates on it and skips cleanly, while the build job's artifacts still upload
(run with open_pr=false for artifacts-only).
The workflow pinned actions/attest-build-provenance@bd77c077… commented
`# v2.4.0`, but v2.4.0 is e8998f94… (verified via the GitHub API); bd77c077…
is an untagged mid-stream commit, so the SLSA-attestation step ran unvetted
action code and the comment misrepresented what runs. Repin to the real
v2.4.0 commit and drop the `# PLACEHOLDER-PIN` markers on both this line and
the setup-python pin (a26af69b… is already the correct v5.6.0 — only its
comment was stale). Update the header NOTE accordingly.
tree-sitter-c is now vendored prebuild-only (#2116) with 0/6 committed
prebuilds, so on a toolchain-less or `--ignore-scripts` install C has no native
binding. Three modules loaded it via a hard top-level `import C from
'tree-sitter-c'`, which throws ERR_MODULE_NOT_FOUND at module-load — crashing
`analyze` before parser-loader's optional/severity:error degradation can run.
This is the #2091/#2093 bug class (previously fixed for swift/dart/kotlin); C was
left static because it used to be an always-present npm dependency.
- languages/c/query.ts: load via the lazy guarded getLanguageGrammar(C), mirroring
swift/query.ts; the main-thread isLanguageAvailable filter ensures the getters
are reached only when C is present.
- workers/parse-worker.ts: guarded `_require('tree-sitter-c')` + conditional
languageMap spread, like swift/dart/kotlin.
- group/extractors/include-extractor.ts: guarded `_require`; getLanguageForFile
returns null for .c/.h when absent, so C include-extraction degrades to a no-op
(C++ unaffected).
- extend the registry-import-closure regression test (#2091/#2093) to assert C
also loads lazily at registry static-import time.
The per-grammar activation scripts (c/dart/proto/swift/kotlin) were ~95%
identical — same prefer-prebuild → source-build → never-fail flow, differing only
in name, target_name, required-vs-optional, and the display label in warnings.
- scripts/build-tree-sitter-grammars.cjs: one registry-driven script. Bare call
builds all (postinstall); `... <name>` builds only the named grammars (so the
probe test can isolate one). c is `required: true` (ignores the opt-out gate);
the rest honor GITNEXUS_SKIP_OPTIONAL_GRAMMARS. Per-grammar try/catch + a final
process.exit(0) preserve the postinstall never-exit-non-zero invariant.
- package.json: postinstall is now `materialize && build-tree-sitter-grammars.cjs`
(was five chained `build-tree-sitter-<name>.cjs` calls).
- tests: replace the two near-identical *-probe.test.ts files with one
parameterized build-tree-sitter-grammars-probe.test.ts that also covers the
required-vs-optional opt-out split and an unknown-grammar arg.
- update cli-commands.test.ts postinstall assertions + the vendor c/kotlin/swift
README + swift provenance to reference the consolidated script.
Behavior is preserved (warnings normalized to one consistent format). Removes 5
scripts + 1 test file; adds 1 script + 1 test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Vendoring grammar source (parser.c) alongside the prebuilds means the npm
tarball now carries ~50 MB of generated source it almost never compiles (every
supported platform-arch has a prebuild). Prepare to drop it from the published
package once all prebuilds exist — safely.
- .npmignore: add a GATED, commented-out "lean publish" block that excludes the
source-build inputs (parser.c/scanner.c/tree_sitter/binding.gyp/binding.cc) but
keeps prebuilds/ + the runtime files. Uncommenting ships prebuilds-only.
- scripts/assert-publish-grammar-coverage.cjs: a prepack guard that refuses to
pack/publish if the source exclusion is active while any vendored grammar still
lacks 6/6 prebuilds (which would ship a grammar with no loadable binding). Wired
into `prepack` (runs on npm pack + publish, incl. the publish.yml dry-run) and
exposed as `npm run assert-publish-coverage`.
- test: pure-core decision cases + a real-repo publish-safety check that fails CI
if .npmignore is activated prematurely.
Net: the prebuilds already publish today (files: ["vendor"]); this makes the
future switch to a prebuilds-only tarball a one-line uncomment that can't ship a
dead grammar. The guard currently reports "source + prebuilds" (only swift has
6/6 prebuilds so far) and passes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Swift was the last grammar handled differently — it shipped only upstream
prebuilds, while c/dart/proto/kotlin vendor their grammar source and use a
prefer-prebuild -> source-build-fallback activation script. Vendor swift's
source so all five are handled identically (one uniform build path).
- vendor/tree-sitter-swift: add binding.gyp (win-hardened), bindings/node/
binding.cc, src/parser.c (ABI-14 default, ~18 MB), src/scanner.c, and
src/tree_sitter/ headers. The 6/6 prebuilds are retained. The legacy
parser_abi13.c alternate is intentionally not vendored.
- build-tree-sitter-swift.cjs: rewrite the prebuild probe into the dart-style
prefer-prebuild then source-build fallback (keeps the GITNEXUS_SKIP gate and
the never-exit-non-zero postinstall invariant).
- build-tree-sitter-prebuilds.yml: register swift (kind 'vendored'); add its
package.json to the version-gated pull_request paths and a validate snippet.
- prebuild-coverage guard auto-moves swift into the source-fallback cohort
(binding.gyp now present); refresh the stale "swift is prebuild-only" comments.
- tests: add build-tree-sitter-swift-probe.test.ts; fix the pre-existing
build-tree-sitter-kotlin-probe.test.ts breakage (it still asserted the old
probe strings after kotlin's dart-style conversion); assert swift's vendored
source in cli-commands.test.ts.
- docs: README / .devcontainer / kotlin vendor README — swift's prebuilds are
now GitNexus-cross-built from vendored source like the rest, not upstream-only.
Verified: swift source-builds against node-addon-api@8 -> N-API binary -> loads
against the pinned tree-sitter@0.21.1 (ABI 14) -> parses cleanly.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`npm prune --omit=dev` in the gitnexus CLI image drops anything not in
package.json's dependency tree — including the VENDORED tree-sitter grammars
(materialized by postinstall, not declared deps) and their built bindings. The
`serve` image analyzes/parses repos at runtime, so re-run the grammar postinstall
after the prune (in the toolchain-equipped builder) to restore them. Load-bearing
for tree-sitter-c, a core REQUIRED grammar now vendored (#2116): as a former
dependency it survived prune; vendored, it would not. Also restores
swift/dart/proto/kotlin, which were silently pruned from the image before.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The vendored prebuild-only grammars (c, kotlin) had empty prebuilds/ until the
build-tree-sitter-prebuilds workflow runs, so they could not load in CI — and
C is hard-required by cross-platform tests (tree-sitter-languages/parsing on
ubuntu+macos+windows), which I cannot pre-build for macos/windows locally. The
robust fix is a source-build fallback that works on every CI runner (all have a
toolchain), mirroring dart/proto:
- Vendor the grammar source (binding.gyp + src/) for c and kotlin; their build
scripts now PREFER a committed prebuild (toolchain-free) and fall back to
`node-gyp rebuild` from the vendored source when no prebuild matches. Verified
both compile against the hoisted node-addon-api@^8 and the runtime loads.
- prebuild-coverage guard is now bootstrap-tolerant: a grammar that vendors its
source (binding.gyp) may have an incomplete prebuild set (the workflow fills
it); a prebuild-only grammar (swift) still must ship all six. Any present
prebuild must still be N-API. Guard goes green; it re-tightens per-grammar as
the workflow populates prebuilds.
- actionlint: silence a false-positive SC2016 (JS template literals inside the
single-quoted `node -e` validate block).
Note: kotlin's generated parser.c is large (~23 MB on disk; compresses heavily
in git). Once the workflow populates all six kotlin prebuilds, the source serves
only as the fallback and could be slimmed if desired.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
tree-sitter-c is the one grammar dependency upstream ships incomplete prebuilds
for (4/6 — no linux-arm64/win32-arm64), AND it is a REQUIRED grammar: its own
`install` (node-gyp-build) compiles from source when no prebuild matches and
exits non-zero, so on a toolchain-less ARM host `npm install gitnexus` HARD-FAILS
at the c step — during npm's dependency phase, before any GitNexus postinstall
runs (so a postinstall "supplement" can't help).
Fix: vendor c prebuild-only at the pinned 0.21.4 (Kotlin pattern), with all six
prebuilds GitNexus-cross-built, and drop it from `dependencies`:
- vendor/tree-sitter-c/ (bindings + node-types + manifest + prebuilds); build
probe scripts/build-tree-sitter-c.cjs; added to the build workflow registry
(kind 'npm' — built from c@0.21.4 source).
- materialize-vendor-grammars.cjs: c is REQUIRED, so it is always materialized,
even under GITNEXUS_SKIP_OPTIONAL_GRAMMARS (it needs no toolchain).
- Removed from package.json dependencies + lockfile (nothing else needs npm c —
tree-sitter-cpp's dep on c is dev-only and not installed). Preserves the #1242
ABI pin: vendoring 0.21.4 keeps the good ABI while closing the ARM gap.
- parser-loader note + the prebuild-coverage guard + a cli-commands assertion
updated; c moves from the npm-gap allow-list into the vendored 6/6 cohort.
Verified: tsc clean, 31 unit tests pass, c loads/parses; the guard is RED for
c/dart/proto/kotlin until the workflow populates prebuilds (the must-fill gate).
Closes the operational risk in #2116.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(cli): add `gitnexus uninstall` to reverse setup (#2060)
`gitnexus uninstall` was documented in #168 but never implemented, so the
CLI rejected it with "error: unknown command 'uninstall'" (#2060).
Add an `uninstall` command that reverses `gitnexus setup` target-by-target:
removes the GitNexus MCP server entries (Cursor, Claude Code, Antigravity,
OpenCode, Codex), the installed skill directories, and the Claude Code /
Antigravity hook entries plus their bundled hook scripts. Edits are surgical
and idempotent — only gitnexus-owned keys/entries/dirs are touched, and JSONC
comments/indentation are preserved. Defaults to a dry-run preview; `--force`
applies. Per-repo indexes and the global npm package are left alone with
printed hints, since both are destructive in ways setup never caused.
Adds i18n entries (en + zh-CN), help wiring, README/CHANGELOG docs, and unit
tests covering MCP/hook/skill/Codex-TOML removal, dry-run, corrupt-file
safety, and the no-op case.
* changelog changes
* changelog changes
* fix(cli): harden uninstall against data-loss edge cases (review #2062)
Address review findings on the uninstall command:
- Empty derived skill name no longer wipes the whole skills dir: a bare
'.md' source file would make basename() return '', resolving to the
skills dir itself. Skip empty names in derivation and reject
empty/'.'/'..'/separator names in removeSkillsFrom.
- Corrupt settings.json no longer orphans the hook: gate the hook-script
dir removal on status !== 'corrupt' so we don't delete a script while a
still-registered entry points at it (Claude + Antigravity blocks).
- Hook removal is now element-granular: delete only the gitnexus command
inside an entry's hooks[], removing the whole entry only when it becomes
empty. Preserves a user command co-located in the same entry.
- Fallback TOML stripper: also remove descendant sub-tables
([mcp_servers.gitnexus.env]), track multiline strings so a bracketed
line inside a value isn't treated as a header, and stop reflowing
unrelated blank lines.
- Set process.exitCode=1 on partial failure; add a 10s timeout to
'codex mcp remove'.
Tests expanded 7 -> 17: empty-skill guard, corrupt-settings hook
preservation, shared-entry hook removal, OpenCode MCP keyPath,
Antigravity MCP + AfterTool hooks, codex-remove success path, TOML
sub-table + multiline-string cases, dry-run for hooks/skills, and the
directory-layout skill branch.
* refactor(cli): share setup/uninstall target map + harden TOML fallback (review #2062)
Maintainer review follow-ups:
- Extract editor target identities into editor-targets.ts (MCP paths/keyPaths,
Codex TOML section, skill dirs, hook settings/events/needles/script dirs,
shared detectIndentation). Both setup.ts and uninstall.ts consume it, so a
target change updates both sides — killing the silent drift hazard.
- Add a setup -> uninstall round-trip integration test that iterates
getEditorTargets(): setup writes every target, uninstall removes all of them,
and a co-located user MCP server + user hook survive. Drift tripwire in both
directions.
- Preview now prints the exact paths it would remove; command output + README
state skills are matched by bundled gitnexus skill name. (Provenance marker
deferred to a tracked follow-up.)
Hardening of the hand-rolled Codex TOML fallback (found in code review):
- Strip a section header that has a trailing inline comment (was matched as a
header but failed the exact classify check -> section left behind while
reported removed).
- Preserve CRLF line endings instead of rewriting the whole file to LF.
- Fix multiline-string scan: a line with an odd count of BOTH """ and '''
no longer mis-picks the delimiter and desyncs the scanner (left->right scan).
- removeSkillsFrom guard also rejects absolute names.
Regression tests added for each. Full setup/uninstall suite green.
---------
Co-authored-by: Gergő Magyar <gergomagyar@icloud.com>
Regression guard so a toolchain-less install can never silently lose a tree-sitter
language on a supported platform-arch:
- Vendored grammars (vendor/tree-sitter-*): every one MUST ship a loadable N-API
prebuild for all 6 tuples {linux,darwin,win32}-{x64,arm64}. Asserts the
napi_register_module_v1 entry symbol in each .node (cross-platform, no need to
run the binary). Currently RED for dart/proto/kotlin until the
build-tree-sitter-prebuilds workflow populates their prebuilds/ — this is the
must-fill-before-merge gate (swift already passes 6/6).
- npm-dependency grammars: asserts upstream ships 6/6 N-API too, catching a
future platform drop. tree-sitter-c is allow-listed at 4/6 (missing
linux-arm64/win32-arm64) pending #2116; the guard also fails if that gap is
silently closed (prompting allow-list removal).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Eliminate the C/C++-toolchain requirement at install for the at-risk grammars
(dart, proto, kotlin) by generating + vendoring native prebuilds, mirroring the
existing vendored tree-sitter-swift. The 10 grammars that already ship 6 upstream
prebuilds stay npm dependencies (toolchain-free AND dependency-review-tracked).
- .github/workflows/build-tree-sitter-prebuilds.yml: a registry-parameterized
workflow that builds {dart,proto,kotlin} x {linux,darwin,win32}-{x64,arm64}
prebuilds natively, validates each loads + parses on its arch, and opens a PR
vendoring them. A `guard` job gates the heavy matrix to run ONLY on dispatch
or a real grammar-version change — ordinary code PRs cost zero matrix minutes.
- dart/proto: prefer a committed prebuild; fall back to today's source build
when none matches (no behavior change until prebuilds are vendored).
- kotlin: vendor it (Swift parity) instead of compiling the third-party
optionalDependency from source at the user's install — supersedes #2110's
optionalDependency mechanism. The ~23 MB parser.c is NOT vendored (the
workflow builds from the published package); only node-types + bindings +
prebuilds are. Removed from optionalDependencies; lock regenerated; probe,
parser-loader note, README/.devcontainer docs, and the #2110 tests updated.
DO NOT MERGE until vendor/tree-sitter-kotlin/prebuilds/ is populated by the
build-tree-sitter-prebuilds workflow: until then Kotlin is unavailable (vendored
with no source-build fallback). dart/proto remain fully functional throughout.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(install): document Kotlin optional-grammar toolchain behavior + graceful install probe
tree-sitter-kotlin is a third-party npm optionalDependency that ships
source-only (no upstream prebuilds) and compiles its native binding via
node-gyp at install. It was the only optional grammar without a GitNexus
install-time probe, and the README's GITNEXUS_SKIP_OPTIONAL_GRAMMARS
"no toolchain needed" note omitted Kotlin entirely. This adds a fail-soft
probe (mirroring the Swift one) that warns clearly and always exits 0 so
install never breaks, wires it into postinstall, and corrects the
optional-grammar docs in README.md and .devcontainer/README.md. Shipping
prebuilt .node binaries (the literal request) needs an upstream/CI build
matrix and is intentionally left as follow-up.
Refs #2107
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix: address PR #2110 tri-review findings (Kotlin optional-grammar install)
Addresses the four P2 findings from the PR #2110 tri-review:
- F1: docs no longer imply GITNEXUS_SKIP_OPTIONAL_GRAMMARS=1 skips Kotlin's
toolchain. npm compiles tree-sitter-kotlin via its own node-gyp-build step
regardless of that variable; point to `npm install --omit=optional` as the
real lever (README.md + .devcontainer/README.md).
- F2: the install probe now surfaces its "Kotlin unavailable" guidance on the
dir-absent branch — the dominant toolchain-less case, where npm prunes the
failed optional dependency so the package dir is gone at postinstall. Gated on
npm_config_omit so a deliberate `--omit=optional` stays silent. Still never
throws or exits non-zero.
- F3: add a behavioral test that executes the probe across its skip /
dir-absent-warn / dir-absent-omit-silent paths and asserts exit code 0
(guards the postinstall "never exit non-zero" invariant a static assertion
cannot).
- F4: reframe prebuilt Kotlin as deferred Swift-parity follow-up — GitNexus
already vendors its own self-built Swift prebuilds and could do the same for
Kotlin — tracked in #2107, not an upstream-only blocker.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* perf(query): batch per-symbol process/cohesion/content lookups (N+1 -> 2-3)
Port of the local-backend query-batching from gitnexus-enterprise PR #222
into the OSS local MCP backend. The query tool traced each matched symbol
to its processes + cohesion (+ content) with up to 3N sequential pool
round-trips; batch them into 2-3 'WHERE n.id IN $nodeIds' queries keyed
back to each symbol by a prepended 'n.id AS nodeId' column. Output is
identical: the aggregation loop is unchanged, iterates merged in the same
order, and reads pre-fetched maps instead of issuing a query per symbol.
Adaptations over a blind cherry-pick (would otherwise change output):
- per-nodeId first-row community pick replaces the per-symbol LIMIT 1, so
each symbol keeps its own community (not one for the whole batch);
- batched rows regrouped to the originating merged item by nodeId so the
JS-side RRF item.score still drives process ranking;
- positional fallbacks shift +1 (process row[1..6], cohesion [1]/[2],
content [1]); CodeRelation{type:...} relation form kept; IN-list chunked
at 100 like the impact path.
Adds a regression test asserting per-node community/content association
(func:login keeps comm:auth; func:validate inherits no community).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(docker): bake LadybugDB FTS extension into the CLI/serve image
The container runs `serve` under the default `load-only` extension policy
(the read pool pins {policy:'load-only'}), so a runtime LOAD EXTENSION fts
never INSTALLs. Dockerfile.cli copied the extension installer but never ran
it, so the runtime user's HOME had no FTS extension: keyword search
silently degraded (no FTS indexes written, ranking falls back to
vector-only with only a warning field). Same class of footgun fixed for
the Hub image in gitnexus-enterprise PR #222.
Run install-duckdb-extension.mjs as the `node` user with the runtime HOME
so INSTALL fts materializes the extension under $HOME/.lbdb/extension where
the runtime LOAD resolves it offline. Pin ENV HOME=/home/node because
Docker does not derive HOME from USER — without it the build-install and
runtime-load would resolve different paths. Verified locally: INSTALL lands
in $HOME/.lbdb/extension/0.17.0 and a fresh offline load-only
`LOAD EXTENSION fts` resolves it. Dockerfile.web is unaffected (static
frontend, no @ladybugdb backend).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(lbug): FTS evict->reload RSS repro + inert pool RSS tracing
Settles the gitnexus-enterprise PR #222 root-cause hypothesis for OSS:
does re-running LOAD EXTENSION fts on every pool evict->reload strand the
native FTS arena (unbounded RSS growth in long-lived MCP serve), or does
db.close() reclaim it (bounded by MAX_POOL_SIZE)? Static read could not
decide — the native lbugjs.node binary documents no close->extension-unload
contract.
Adds gitnexus/scripts/bench/fts-evict-reload-rss.mjs: a NATIVE mode that
reproduces the exact native sequence doInitLbug()+closeOne() perform
(open Database -> Connection -> LOAD EXTENSION fts -> QUERY_FTS_INDEX ->
close) across K self-built FTS fixtures, and a --via-pool mode that drives
the real compiled pool (initLbug/executeParameterized/closeLbug) against an
existing analyzed repo. Plus a behavior-neutral GITNEXUS_POOL_RSS_TRACE=1
stderr trace on pool init/close (stdout reserved for MCP JSON-RPC; single
env read when disabled).
RESULT (native, 24 and 40 cycles x 6 fixtures, --expose-gc): PLATEAU. RSS
warms up to ~400 MB then flattens (40-cycle: +36 MB over cycles 1-10, +3 MB
over 30-40; decelerating), not the linear climb a per-reload arena leak
would produce (240 reloads x stranded arena = multi-GB). db.close()
reclaims the FTS arena. The unbounded-leak hypothesis is NOT reproduced for
the OSS path: the pool's LRU eviction + close-on-evict BOUNDS the footprint,
which is exactly the protection the enterprise Hub supervisor lacked (it
opened bridge DBs in-process without eviction -> 15 GB). => plan U4
(worker/process isolation) is NOT justified by this evidence; U1 + U2 are
the only OSS-shared changes. Caveat: small fixtures + awaited close; a
--via-pool run against a large analyzed repo over a long session is the
production-faithful follow-up (instrumentation is in place for it).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(review): apply ce-code-review autofix feedback (#222 migration)
Adversarial review found the U3 bench PLATEAU->no-leak conclusion was
over-claimed from a 600-row fixture: a size-proportional FTS-arena leak
would be sub-threshold at that scale. Strengthen the bench and make its
verdict honest:
- scale the fixture (--rows, UNWIND batch insert), probe ALL 5 FTS indexes
in --via-pool (not 2 of 5), add a --no-await-close variant (the pool
fire-and-forget close shape), and replace the absolute-delta gate with a
SLOPE-DECELERATION 3-way verdict (PLATEAU / CLIMB / INCONCLUSIVE) plus
step-discontinuity detection. At production-representative scale the
synthetic runs are noisy/INCONCLUSIVE (deceleration argues against an
UNBOUNDED leak but does not prove bounded), so plan U4 stays GATED on a
--via-pool run against a real large analyzed repo -- not closed.
- Dockerfile.cli: source the scratch-DB size from ENV GITNEXUS_LBUG_MAX_DB_SIZE
(single source of truth) and add a build-time verify-only LOAD gate
that fails the build on a HOME/extension-dir mismatch instead of silently
degrading runtime keyword search.
- install-duckdb-extension.mjs: additive verify-only mode (LOAD-only in a
fresh process) + robust size parse; back-compatible with the runtime
positional-size caller (validated).
- tests: wire func:validate into a second process (proc:beta-flow) so the
batched STEP_IN_PROCESS row[1..6] positional shift is exercised by a
genuine multi-process symbol, and assert process ranking. No blast radius
(75 seed-consuming tests pass).
- pool-adapter.ts: trim the traceRss narrated-code comment (DoD 2.3).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(bench): classify a sustained sub-floor RSS slope as INCONCLUSIVE, not PLATEAU
Tri-review P2: the FTS evict->reload verdict short-circuited to PLATEAU
whenever secondHalfSlope < SUSTAIN_FLOOR, BEFORE the deceleration check —
so a sustained (non-decelerating) linear leak below 0.5 MB/cycle was
labeled PLATEAU ("no leak"), the label that would wrongly close plan U4.
Extract median/slopeMbPerCycle/classifyVerdict into a pure, side-effect-free
fts-rss-verdict.mjs (zero imports) so it is unit-testable without loading the
native addon or running the bench, and fix the classifier:
- epsilon-first gate: a truly flat tail (< 0.1 MB/cycle) is PLATEAU regardless
of decelRatio (guards against over-correcting a real negative into
INCONCLUSIVE);
- a sustained sub-floor positive slope (>= epsilon, < floor, decelRatio >= 0.6)
is INCONCLUSIVE — a slow creep RSS cannot distinguish from noise at this
scale, so the honest label is "not resolved", never a clean PLATEAU;
- the noise floor now scales with the WORKING-SET growth (peak-baseline), not
the pre-DB baseline RSS (which is interpreter/addon overhead, larger in
--via-pool mode, and would inflate the floor and HIDE leaks).
Reconcile the stale "per-row-relative delta floor" docstring; add floor +
decelRatio to the MACHINE line. New fts-rss-verdict.test.ts pins all label
boundaries (flat->PLATEAU, sustained-sub-floor->INCONCLUSIVE,
decelerated->PLATEAU, sustained-linear->CLIMB, step->INCONCLUSIVE,
working-set floor, no import side effects). U1 does NOT add detection power
for sub-floor leaks (RSS cannot attribute that magnitude) — it stops the
false PLATEAU and routes that regime to the --via-pool run.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(query): signal partial/warning on a real enrichment failure (not benign missing-table)
Tri-review P2: when a batched enrichment query (process/cohesion/content)
threw, it was caught + logged and the chunk's symbols silently fell back to
`definitions` with no signal — the caller could not tell "genuinely
standalone" from "enrichment failed".
Track an `enrichmentDegraded` flag in the three enrichment catch blocks and,
at response build, compose a single `warning` (FTS-missing and/or the
enrichment message, so neither overwrites the other) plus `partial: true`.
Both fields are omitted on the clean path, so the success-path response shape
is byte-identical.
Crucially, the flag fires ONLY for a REAL failure (timeout / lock / native
fault), NOT the benign "no Process/Community table" prepare error — a repo
analyzed without processes/communities is a normal config, and firing
`partial` on every such query would desensitize callers
(isBenignMissingTableError gates it).
New unit test test/unit/query-degraded-signal.test.ts (vi.mock pool-adapter,
override hybrid search to feed one matched symbol, route STEP_IN_PROCESS ->
throw): real failure -> warning+partial+symbol still returned; benign
missing-table -> no signal; FTS-missing + enrichment failure -> both messages
in one warning. Plus a success-path no-warning/no-partial assertion in the
calltool integration test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(c): skip computed #include MACRO instead of emitting a garbage import source (F5)
* fix(cpp): emit a Variable per name for structured-binding declarations (F9)
* fix(dart): extract static const/final class fields (F26)
* fix(dart): capture old-style function typedefs (F28)
* fix(dart): read real top-level variable shape instead of a dead type field (F29)
* fix(kotlin): capture callable references (F47)
* fix(kotlin): anchor infix-call capture to the operator only (F49)
* fix(kotlin): extract secondary constructors as members (F48)
* fix(kotlin): capture destructuring declarations (F51)
* fix(kotlin): index companion-object properties as fields (F52)
* test(kotlin): assert callable-reference coverage runs on the worker path (F47)
* fix(swift): extract protocol property requirements (F75)
* fix(swift): recognize enum_class_body as a method body node (F79)
* test(ingestion): rebaseline swift captures-golden + scope-capture fingerprints (#1919)
* fix(kotlin): attribute secondary-constructor body calls to the Constructor node (#1919 review CF1)
A Kotlin secondary constructor's body executes statements like a method body,
but the registry-primary scope-resolution path had no Function scope or
Constructor def for it. A call inside the body resolved its caller anchor up to
the enclosing Class scope, mis-attributing the CALLS edge to the class rather
than the Constructor.
Add `(secondary_constructor) @scope.function` to the Kotlin scope query so the
body becomes its own scope, and synthesize a `@declaration.constructor` (named
`constructor`, qualified `<Class>.constructor`, with parameter metadata) so the
scope owns a Constructor def that bridges to the structure-phase Constructor node.
Also add an arity-disambiguating lookup key for overloadable callables: two
same-name secondary constructors of different arity (e.g. a zero-arg vs a 2-arg)
share the qualified key whose first-write-wins assignment is source-order-
dependent — so a zero-arg overload could resolve to a sibling. The structure
node id encodes `#<arity>`; mirror that in the bridge keyspace and match by the
def's parameterCount. Same-arity overloads collapse onto one arity key exactly
as before, so no regression there.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(kotlin): do not own function-local property bindings under the enclosing class (#1919 review CF3)
Kotlin emits destructuring / loop bindings (`val (a,b) = pair`,
`for ((k,v) in m)`) as `@definition.property` to dodge the block-scope
local-symbol pruner. When such a binding sits inside a method body of a class,
the structure-phase owner walk found the enclosing class and emitted a spurious
HAS_PROPERTY edge (e.g. `C -> k`), treating a function-local as a class member.
Guard the Property owner resolution: if a function-like ancestor is reached
before any class container, the property is function-local and gets no owner
edge (it falls back to a File DEFINES edge). Language-agnostic — genuine class
fields sit directly in the class body with no intervening function, so they
keep their HAS_PROPERTY owner edge.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(kotlin): guard non-companion property isStatic=false (#1919 review CF4)
Add a field-extraction case for a plain non-companion class
`class C { val x: Int = 1 }` asserting the property `x` has isStatic=false,
guarding the `isInsideKotlinCompanion` walk against false-positives.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(kotlin): dedup type_identifier lookup in extractOwnerName (#1919 review CF5)
The `node.namedChildren.find(c => c.type === 'type_identifier')?.text` lookup was
duplicated across the companion and non-companion branches of the Kotlin
field-extractor's extractOwnerName. Hoist it into a single local, preserving the
existing behavior (anonymous companion falls back to "Companion"; other nodes
prefer the `name` field, else the type_identifier text, else undefined).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(dart): capture generic old-style function typedefs (#1919 review CF2)
* test(dart): guard multi-name field count and top-level-var labels (#1919 review CF4)
* docs(swift): correct isStatic comment re multi-modifier hasKeyword (#1919 review CF5)
* test(ingestion): rebaseline dart+kotlin scope-capture fingerprints after review remediation (#1919)
* fix(ingestion): correct CF3 owner-strip boundary set for accessor/init bodies and Dart signatures (#1919 review)
The CF3 property-ownership guard used FUNCTION_NODE_TYPES, which (a) includes
Dart bare signatures (function_signature/method_signature) — over-stripping
every Dart class getter/setter's HAS_PROPERTY owner — and (b) omits Kotlin
anonymous_initializer/getter/setter and Swift computed accessors — under-
stripping destructuring/locals inside init{} and accessor bodies, emitting
spurious Class->local HAS_PROPERTY edges. Introduces a guard-specific
LOCAL_SCOPE_BODY_NODE_TYPES set (signatures excluded, accessor/init bodies
included). Adds Dart accessor-ownership + Kotlin init/accessor destructuring
regression fixtures. Both confirmed on the worker pipeline; no cross-language
regression (1597 cross-language tests green).
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Ring 4 retires the legacy call-resolution DAG. With the legacy resolver
gone (RING4-1 #942, RING4-2 #943), shadow mode has nothing to dual-run
against, so the remaining shadow-mode artifacts are dead code.
- Delete gitnexus-shared/src/scope-resolution/shadow/{diff,aggregate}.ts
(pure parity comparison logic) and its gitnexus-shared barrel exports.
- Delete the static parity dashboard (gitnexus/shadow-parity-dashboard/),
which also removes the last GITNEXUS_SHADOW_MODE reference in the repo.
- Delete the shadow-mode unit tests (gitnexus/test/unit/shadow/).
- Scrub stale doc comments referencing the shadow harness / parity
dashboard / removed legacy run (csharp/php/python/typescript index.ts,
evidence.ts, module-scope-index.ts).
Already removed by RING4-1/-2 (verified): the shadow harness source and
GITNEXUS_SHADOW_MODE env handling; no CI job published dashboard artifacts.
Historical parity records preserved per acceptance: the CHANGELOG entry
(#918, #923, #951, #972) and the ci.yml RING4-1 note remain. Last
documented parity state is that historical coverage — no live
.gitnexus/shadow-parity/ run data exists in-tree (runtime output only).
Closes#944.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(ingestion): reduce parse-phase memory for huge repos (#1983)
Stop retaining full parse-cache chunks in RAM alongside the merged graph,
slim on-disk shards, defer worker ParsedFile emission for scope-resolver
languages, and add GITNEXUS_DEBUG_HEAP probes for OOM diagnosis.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(ingestion): address #2038 tri-review findings (parse-phase memory)
Resolves the confirmed review findings on PR #2038:
- P1: thread exportedTypeMap through the sequential parse path
(processParsingSequential) so a no-worker run over a partially-warm
cache no longer silently drops the sequential-miss files' exported
types. Cache hits made exportedTypeMap.size > 0, suppressing the
end-of-loop buildExportedTypeMapFromGraph rebuild, but the sequential
path never populated the map. Regression test added (fails on the
pre-fix tree, passes after) plus a fully-sequential differential oracle.
- P2: saveParseCache builds its on-disk index from hashes actually
written/copied (writtenKeys), never a usedKeys hash whose shard write
or copy was skipped — no more phantom index entries.
- P2: add a unit test asserting SCOPE_RESOLUTION_LANGUAGES stays in sync
with SCOPE_RESOLVERS (asymmetric drift would lose a language's ParsedFile).
- Backfill cache coverage: loadParseCacheChunk missing/corrupt -> undefined,
pruneCache onDiskKeys branch, slim preserves nodes, saveParseCache
copy-evicted-shard round-trip.
- Cleanups: single-source heap-probe gating via isDebugHeapEnabled();
hoist the per-chunk mkdir in persistParseCacheChunk behind a
process-scoped Set; gate COBOL's unused worker-side ParsedFile
extraction (graph nodes still come from cobolPhase) while keeping
fileCount/progress unconditional.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(ingestion): remove dead worker-side ParsedFile extraction
After #2038 gated worker `ParsedFile` emission behind `!isScopeResolutionLanguage(language)`, and with all 16 SupportedLanguages registered in SCOPE_RESOLVERS, that gate was structurally always true — the worker already produced no ParsedFiles and scope-resolution re-extracts each file from source on the main thread (run.ts). Remove the now-dead machinery:
- Drop both worker `extractParsedFile` call-sites (tree-sitter processFileGroup + the standalone-provider branch) and the `result.parsedFiles.push`. The standalone branch keeps fileCount/onFileProcessed per file. `result.parsedFiles` stays declared but empty (field removal deferred).
- Remove the now-orphaned `scopeSourceKind` var + `ScopeCaptureSourceKind`/`extractParsedFile`/`isScopeResolutionLanguage` imports.
- Delete the consumerless `migrated-languages.ts` (isScopeResolutionLanguage + SCOPE_RESOLUTION_LANGUAGES) and its drift-guard test — parse-worker was their only importer. Also improves AGENTS.md "shared ingestion code must not name languages" compliance.
`extractParsedFile` and the scope-extractor-bridge stay (scope-resolution/run.ts + Vue resolver use them). Behavior-preserving: worker-sequential-parity passes before and after; tsc/eslint clean; no baseline/golden drift.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(ingestion): worker-pool-only parsing; remove sequential parser (#1983)
Completes the #1983 huge-repo parse-OOM effort by making the worker pool
GitNexus's sole parse path.
Parallel serialization (the perf core): workers serialize their ParsedFiles to
a disk store in parallel and stream them back to scope-resolution, so the main
thread no longer re-parses every file (the tree-sitter native-memory leak that
caused the OOM). Adds chunk merge-pipelining + work-proportional chunk sizing so
the pool stays saturated.
Remove the sequential parser: `--workers 0`, `GITNEXUS_WORKER_POOL_SIZE=0`, and
`skipWorkers` now hard-error (no silent degrade — #1741); the small-repo
threshold no longer selects an in-process path; pool creation stays lazy /
cache-miss-gated so warm all-hit runs never spawn workers.
Worker-path parity fixes — removing sequential surfaced two pre-existing gaps
that tiny-fixture tests had masked by running below the worker threshold, both
fixed by carrying per-file metadata as DATA across the worker boundary (never
re-parsing on the main thread, preserving the OOM fix):
- C++: templateConstraints wired into worker node identity (SFINAE overload
disambiguation) + ADL / inline-namespace capture side-channel serialized
onto the ParsedFile.
- Kotlin: companion-scope side-channel serialized the same way (companion /
static dispatch).
Validation: tsc + build clean; full suite green (10,190 pass — the only
deterministic failures were the now-fixed C++/Kotlin worker-path gaps; the 2
remaining full-run failures are pre-existing load flakiness, green in
isolation); cpp-pipeline benchmark stays linear on a 1-worker pool.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(ingestion): wire C static-linkage side-channel + ADL O(1) collect + tri-review cleanups (#1983)
Follow-up to the worker-pool-only refactor, from a tri-review of the parse path.
- C static-linkage side-channel (P1): cProvider had no collect/applyCaptureSideChannel,
so on the now-sole worker path C `static` file-local marks were lost across the worker
boundary -> false cross-file CALLS edges + over-broad #include wildcard visibility on
every C analysis (the Linux kernel is C). Mirror the C++/Kotlin wiring: serialize
`staticNames` per file onto ParsedFile.captureSideChannel and restore it on the main
thread (no re-parse). + a worker-path regression test (the existing c-static-isolation
fixture passed vacuously — its collision resolves via #include before the global
free-call fallback ever consults static-linkage).
- captureSideChannel `kind` discriminant: add `kind:'cpp'`/`kind:'c'` tags + guards
(Kotlin already had one) now that C/C++/Kotlin share the single generic field.
- Perf: collectCppAdlSideChannel scanned the whole argInfoBySite/noAdlSites maps per file
(O(F^2) per sub-batch, ~100M parseSiteKey calls at kernel scale). Add per-filePath
lockstep indexes -> O(1) collect; serialized snapshot byte-identical.
- Cleanups: inline the one-line processParsingWithWorkers wrapper into processParsing;
drop the always-empty WorkerExtractedData.calls/assignments/constructorBindings fields;
remove the voided astCache param from processParsing; refresh stale "sequential
fallback" JSDoc.
Validation: tsc + build clean; cpp 297/297, c 8/8 (incl. the new worker-path
static-linkage guard), typescript + parsedfile-store green; cpp ADL benchmark stays linear.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* perf(scope-resolution): index C/C++ #include resolution in finalize (O(n²)→O(n))
Kernel-scale C/C++ analysis ground in finalizeScopeModel because three
per-#include operations each did a full O(F) scan with no index — the
finalize O(n²) that surfaced once the #1983 parse-phase OOM was fixed:
- expand{C,Cpp}WildcardNames: parsedFiles.find() per wildcard edge → O(R·F)
- resolveImportTarget: new Set(allFilePaths) rebuilt per #include
- resolveCImportTarget: suffix-match scanned all workspace paths
Each is replaced with a WeakMap-per-pass index keyed on the stable
parsedFiles/allFilePaths references that scope-resolution run.ts passes
once per pass:
- Map<ScopeId,ParsedFile> for wildcard expansion (c/static-linkage.ts +
cpp/file-local-linkage.ts)
- memoized augmented header set (c/scope-resolver.ts + cpp/scope-resolver.ts)
- basename-bucketed suffix index in resolveCImportTarget (c/import-target.ts),
shared by C and C++ since resolveCppImportTarget delegates to it
Collapses the C/C++ finalize from O(R·F) to O(R+F). Pure-perf, byte-identical
edge output: 962 targeted tests green (490 C + 472 C/C++ scope-resolution);
the basename index preserves the exact endsWith('/'+target) match and the
fewest-path-components-then-lexicographic tie-break.
The kernel's ~25-30k .h headers are classified C++, so both providers must
be fixed. Proven on the Linux kernel: the C finalize completed
(sr-post-finalize lang=c → sr-end lang=c), which the pre-fix run never
reached in 16+ min of grinding.
Build-independent follow-ups (separate from this finalize fix), documented
for later: emitFreeCallFallback same-name buckets (emit phase),
buildGraphNodeLookup + precount global setup, the ParsedFile store-load,
the dart/go/ruby expand-wildcards .find siblings, and the ~26GB
scope-resolution memory floor (full kernel completion needs >~40GB RAM).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(bench): regenerate C scope-capture baseline for the #1983 c-static-linkage-worker fixture
bench/scope-capture/measure.mjs fingerprints emitCScopeCaptures over the
lang-resolution/c-* fixture corpus. The #1983 PR added the
c-static-linkage-worker fixture (caller.c/lib.c/lib.h/local.c — the
worker-path static-linkage side-channel test) but did not regenerate the C
baseline, so `--check` has been red on this branch (main, lacking the
fixture, still matches 0de009b).
Pure fixture-corpus drift — no c/captures.ts or query change branch-vs-main,
existing fixtures' captures byte-identical (c-captures.test.ts 45/45),
scaling stays linear (~0.97). Regenerated: 0de009b -> 39f3a83. Bench now
PASS (14 languages). Unrelated to the finalize O(n²) fix.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* perf(scope-resolution): lower kernel-scale resident memory floor + setup cost
Reduce the scope-resolution resident-memory floor and setup throughput on
huge repos (Linux kernel), the wall that remains after #1983 (parse OOM) and
the finalize O(n^2) fix (b71c77b8). Five units; all preserve byte-identical
edge output (C fixture 177n/255e + c/cpp/cross-file/php/static-linkage suites
green, 619 tests).
U1 (src/cli/analyze.ts): RAM-aware auto heap-cap. Replace the hardcoded
16384MB cap with computeHeapCapMb = max(16384, floor(0.75*effectiveRAM)),
where effectiveRAM = min(os.totalmem(), process.constrainedMemory()) with the
unconstrained-sentinel guard. Add --max-semi-space-size=128 on the respawn.
A user-supplied NODE_OPTIONS heap still wins (no re-exec). Verified: 23973MB
on a 31964MB box, 16384 floor on small machines, cgroup-aware, sentinel safe.
U2 (src/storage/parsedfile-store.ts, .../pipeline/phase.ts): export forceGc()
and call it at the per-language eviction boundary, so a finished language's
ParsedFiles are reclaimed before the next language's store-load instead of
collected lazily under the next pass's allocation pressure (which at cap>=RAM
degrades into swap-thrash). Measured on a real drivers/net/ethernet run:
C 2113->894MB and C++ 1754->1057MB reclaimed at the boundary (no fragmentation
defeat). Answers the plan's Open Question 1.
U3 (src/storage/parsedfile-store.ts): intern def objects by nodeId in the load
reviver so a SymbolDefinition's three serialized copies (localDefs /
scope.ownedDefs / scope.bindings[].def) collapse to one shared object on load.
Per-shard def pool (a def's copies are shard-local). Measured ~42% off the
def-object retained heap (3->1; 1.8M->600k distinct objects on 600k defs).
U4 (.../passes/free-call-fallback.ts): memoize pickUniqueGlobalCallable's
post-filter candidate list per (name, callerFilePath), only when no per-caller
visibility filter applies (the list is then a pure function of name+file), so
repeated free calls of one name from a file reuse the same-name-bucket scan
instead of re-walking a potentially huge bucket per site. The cached array is
read-only-consumed by the .filter()-based arity/overload narrowers. Exported
pickUniqueGlobalCallable + buildGlobalCallableIndex and added an equivalence
test (memoized == un-memoized reference for every (name, file, arity),
including warm-cache repeats and cross-file file-local exclusion).
U5 (.../pipeline/phase.ts): replace the O(L*F) per-language precount + repeated
scannedFiles.filter() with a single O(F) partition-by-language pass; bracket
buildGraphNodeLookup with scope-setup-nodeLookup heap probes so the long setup
is no longer silent.
Plan: docs/plans/2026-06-06-001-perf-kernel-scope-resolution-memory-plan.md
(U6 out-of-core global index deferred). Note: the kernel's full C++ pass floor
(~20k headers + the 8.8GB graph) likely still exceeds 24GB by itself, which is
why U6 remains the only unit that clears the wall.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(test): match OOM-guidance e2e assertions to the U1 reworded hint
The analyze-heap-oom-e2e real-child-OOM test still asserted the pre-U1
wording ('...out of memory.' + a hardcoded 24576 cap). U1 reworded the hint
to mention the auto heap-cap and use a <MB> placeholder, so the three
toContain substrings no longer matched (the assertion at line 62 failed on
all platforms). Update them to the current message. The unit twin
(analyze-heap-respawn) was already updated in 85bfc216; this integration
test was missed by the targeted local run.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* perf(lbug): U6a — deterministic id-sorted graph output behind GITNEXUS_SORT_GRAPH_OUTPUT
First increment of U6 (out-of-core scope-resolution). Adds an optional
deterministic ordering of node + relationship CSV rows by their unique graph
id, behind GITNEXUS_SORT_GRAPH_OUTPUT (default OFF = today's graph-insertion
order, byte-identical — the iterator is returned untouched). With the flag ON
the CSV becomes a pure function of the node/edge SET rather than of emit order.
This is the structural enabler for the windowed/out-of-core resolve (U6b-U6d):
csv-generator.ts:518 currently iterates graph.iterRelationships() in insertion
order with NO terminal sort, so any deviation from parsedFiles-order emit would
change bytes. With U6a on, a windowed emit need only reproduce the same edge
SET, not the global insertion order — removing the single largest byte-identical
hazard from every later windowing step.
Verified: default off keeps the existing csv-pipeline suite byte-identical; on,
node rows are id-sorted and output is independent of graph insertion order
(set-build) with the same node/edge set.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* perf(storage): U6d foundation — disk-backed scope store + lazy ScopeTree
Adds scope-index-store.ts: persistScopeShards (per-file scope shards via the
proven mapReplacer + def-interning reviver) + DiskBackedScopeTree, a lazy
ScopeTree that serves getScope from a bounded LRU of decoded shards plus a small
resident skeleton (scopeId -> {shard, childIds, parent}). Exports
makeInterningReviver from parsedfile-store for reuse.
This is the contained, highest-risk mechanism of U6d (out-of-core scope
resolution): the emit passes reach the heavy per-Scope binding payload
(~17-20GB on the kernel) ONLY through scopeTree.getScope (a point lookup) and
getChildren — they never read parsed.scopes directly — so moving that payload to
disk behind getScope is transparent. Every consumer reads a Scope BY VALUE, so a
value-faithful disk round-trip is byte-identical to resolution.
Proven in isolation: DiskBackedScopeTree is value-identical to buildScopeTree
for getScope/getChildren/getParent/getAncestors/has/size across multiple files
and after LRU eviction, and preserves the def-identity collapse (ownedDefs[i]
=== binding.def). Nothing wires it yet (the resolution-pipeline integration is
the next increment) — zero production impact; default off.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* perf(scope-resolution): U6d integration — seal scopeTree to disk before emit (GITNEXUS_DISK_SCOPE_INDEX)
Wires the U6d out-of-core scope index into the live pipeline behind
GITNEXUS_DISK_SCOPE_INDEX (default OFF = byte-identical). When on:
- finalize-orchestrator builds a TransitionalScopeTree (validated, fully
resident) instead of buildScopeTree, so finalize/propagate/resolve are
unchanged.
- After resolve, before emit, run.ts seals it: persists the scopes to a
file-sharded scope-index-store, swaps the model's scopeTree to disk-backed
serving from the inside (the frozen bundle can't be reassigned, but the
wrapper nulls its own resident backing), and drops the heavy Scope.bindings
payload from all THREE holders — the model's tree (seal), the caller's
preExtractedParsedFiles, and run.ts's own parsedFiles (scope-stripped copies
for emit). Emit reads scopes only via scopeTree.getScope (a point lookup,
now disk-backed + LRU) — verified it never reads parsed.scopes.
Purpose: lower the per-language resident PEAK (kernel C pass ~20→~12 GB by
moving the ~8-9 GB scope payload to disk) so the analysis fits on smaller-RAM
machines. At >=24 GB the full kernel already fits with U1-U5 (U2's 8.7 GB
inter-language forceGc reclaim keeps each pass under cap) — empirically
confirmed — so this is the sub-24 GB lever, not needed at 24 GB.
Byte-identical evidence: DiskBackedScopeTree/TransitionalScopeTree return
value-identical scopes vs buildScopeTree (getScope/getChildren/getParent/
getAncestors, across files + after LRU eviction + post-seal); emit reads only
getScope + referenceSites; flag-off (394 tests) and flag-on-resident (91 tests)
resolver suites stay green; an end-to-end A/B on a 212-file C+cpp+rust subset
produced identical 17,444 nodes / 31,343 edges with the seal firing per language
(c: 410→141 MB reclaimed). Kernel-scale peak-drop measurement pending the
in-flight verdict run freeing memory.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* perf(scope-resolution): U6d — id-back workspaceIndex so the disk seal can reclaim scopes
The kernel run revealed the contained scopeTree seal didn't lower the heap:
WorkspaceResolutionIndex held Scope OBJECTS (classScopeByDefId / moduleScopeByFile),
built from every ParsedFile and live through emit, so the ~28k module + class
scopes stayed pinned past the seal (sr-seal-pre 17,583 -> sr-seal-post 17,771 MB,
no drop). It was the sole residual Scope-object holder (SemanticModel holds none).
Fix: classScopeByDefId / moduleScopeByFile become id-backed ScopeByKeyView
instances — a ReadonlyMap<K, Scope> facade over a K->ScopeId map + the scopeTree,
whose .get fetches via scopeTree.getScope(id). The index now pins only ids, so
once the tree seals to disk the scopes become collectible. Byte-identical: the
view returns the same Scope the resident tree holds (or a value-identical revived
one in disk mode), and iteration keeps the old insertion order. buildWorkspace
ResolutionIndex takes an optional scopeTree (live pipeline passes it); without it
(unit tests) the legacy direct Scope-object maps are returned unchanged.
Verified byte-identical: 733 tests across workspace-index / imported-return-types
/ c / cpp / cross-file / go / java. Kernel peak-drop re-measurement to follow.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* perf(scope-resolution): U6d — precompute exportedCallableByName (fix disk-getScope thrash)
The workspaceIndex id-backing freed the kernel scopes but exposed a throughput
collapse: findExportedDefByName's workspace fallback (walkers.ts:1019) scanned
EVERY module scope's bindings per unresolved free call, and under the U6d
disk-backed scopeTree each module-scope access faulted a shard in from disk —
lib ON went ~1min -> ~7.5min.
Fix: precompute the fallback result once into
WorkspaceResolutionIndex.exportedCallableByName (simpleName -> first module-local
callable def, first-file-wins — the exact semantics the scan returned), built
from the resident module-scope bindings at index-build time. findExportedDefByName
now does an O(1) lookup with zero disk reads.
Result: lib ON ~7.5min -> 21s (cache-warm), byte-identical 17,444/31,343; 758
tests green across workspace-index + c/cpp/cross-file/go/python.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: rename cryptic U-unit codes to descriptive names in comments
The plan-unit shorthand (U3/U4/U6a/U6d/...) was meaningless in the code.
Renamed in comments + test descriptions (no behavior change, byte-identical):
out-of-core scope index (was U6)
deterministic output (was U6a)
disk-backed scope seal (was U6d)
def-object interning (was U3)
free-call candidate cache (was U4)
Also renamed throughout the PR title/summary. Pushed commit messages keep
their original U-codes as historical record.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(ingestion): durable ParsedFile shards for warm-cache coverage (#2038)
On a warm re-analyze where every chunk is a parse-cache HIT, no parse worker
runs, the run-scoped ParsedFile store is cleared at parse start, and the cached
ParseWorkerResult carries no ParsedFiles (the worker writes them to the store
and empties them from the message). Scope-resolution then found an empty store
and fell back to main-thread extractParsedFile — re-opening the #1983
tree-sitter native-leak OOM the disk store closes (abhigyanpatwari review on
parse-cache.ts).
Fix: workers ALSO write their ParsedFiles to a durable, content-addressed store
(parsedfile-cache/) keyed by chunk hash, mirroring the parse cache's lifecycle
(version-gated by PARSE_CACHE_VERSION, pruned in lockstep to the surviving
keys). On a warm hit the chunk's durable shards are byte-COPIED into the
run-scoped store (no re-parse, no re-serialize -> byte-identical), so
scope-resolution streams them exactly as on a cold run. A coherence gate
re-dispatches the worker whenever a cached chunk's durable shards are missing
(migration / pruned / version-stale) -- never the main-thread extract.
- worker-pool/parse-worker: thread chunkHash through dispatch->job->flush
(incl. split/requeue) so the worker tags its durable shard by content
- parsedfile-store: durable persist / restore / index / prune API (sibling
dir, never cleared per run); content-addressing makes stale reuse impossible
- parse-impl: load durable index, gate the cache hit on durable coverage,
restore on hit, dispatch chunkHash on miss
- run-analyze: prune+save the durable store to the parse cache's surviving keys
- saveParseCache returns its written keys (the durable keepKeys)
Verified on linux/lib: warm preExtractedHits = full coverage (520/207/1, zero
main-thread re-parse), byte-identical cold==warm (17,456n/31,353e), warm 8.5x
faster. New two-run + mixed-mode + coherence-gate regression test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(ingestion): clear stale scope-index-store shards on each seal (#2038)
The disk-backed scope index writes sequential s<n>.json shards into a shared
<storagePath>/scope-index-store/ dir, with the index resetting per
persistScopeShards call. A seal that writes fewer shards than a previous one
(a later language with fewer files, or a re-run of a shrunken repo) left stale
tail shards on disk indefinitely -- never read by the disk-backed tree, but
multi-GB on kernel-scale repos.
Add clearScopeIndexStore() and clear at the start of persistScopeShards: the
previously sealed language has finished emit and been released before the next
seal runs, so its DiskBackedScopeTree never reads those shards again. Unit
tests: a stale prior-run shard is removed, a fewer-files re-seal leaves no tail
shards, and the helper is idempotent.
Addresses abhigyanpatwari review on run.ts (disk hygiene for the
GITNEXUS_DISK_SCOPE_INDEX path).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(rust): F70 — replace struct_expression name:(_) with 3 specific patterns
* fix(rust): F70 — cover scoped+turbofish struct literals (foo::Bar::<T> {})
The three patterns enumerate struct_expression.name as type_identifier /
scoped_type_identifier / generic_type_with_turbofish, but
generic_type_with_turbofish.type can itself be a scoped_identifier
(e.g. foo::Bar::<i32> {}), which the turbofish pattern — requiring
type:(type_identifier) — did not match. That dropped the constructor
reference entirely (verified: emitRustScopeCaptures returns 0 ctors for
foo::Bar::<i32> {} and a:🅱️:Bar::<i32> {}).
Add a fourth pattern that captures the trailing identifier of the scoped
turbofish path (scoped_identifier.name is an identifier, not a
type_identifier), and correct the comment that claimed all cases were
covered.
Strengthen rust-f70.test.ts: assert exactly one constructor per case, add
negative assertions guarding against the old full-path capture, and add
the scoped+turbofish and crate:: cases.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Gergo Magyar <gergomagyar@icloud.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(mcp): prevent orphan processes by handling stdin close/end and startup race condition
Three gaps in stdin EOF handling:
1. Startup race: parent can die before `process.stdin.on("end", ...)` is
registered, so the event is missed entirely.
2. Missing "close" event: when pipe is forcibly closed (parent SIGKILL),
"close" fires without "end" on some platforms.
3. Transport layer did not propagate stdin termination to its onclose
callback.
Fixes:
- Check readableEnded/destroyed in start() before registering listeners.
- Register stdin end+close listeners in CompatibleStdioServerTransport.
- Add _closed guard for idempotent close().
- Throw if start() is called after close().
- Add process.stdin.on("close") in server.ts alongside existing handlers.
- Add 5 regression tests.
* fix(mcp): register stdin shutdown before server connect
* fix(csharp): bind qualified constructor names, capture : base/: this, fix generic strip
Mirrors the Java #1928 parsing-layer fixes for the C# scope-resolution path —
the same three defect classes exist verbatim in C#:
- Qualified / qualified-generic / alias-qualified constructor calls
(`new Ns.Foo()`, `new A.B.Foo()`, `new Ns.Box<int>()`, `new MyAlias::Foo()`,
`new global::Foo()`) bound only `@reference.call.constructor.qualified` with no
`@reference.name`, so the central extractor fell back to the whole-expression
anchor and the reference name became the raw `new Ns.Foo()` text (never
resolved). Derive the simple-name tail via the existing `terminalTypeNameNode`
helper (handles qualified_name, generic tail, and alias_qualified_name), and
add a query arm for the top-level `alias_qualified_name` shape that was not
captured at all.
- `: base(...)` / `: this(...)` explicit constructor initializers, modeled by
tree-sitter as `constructor_initializer` and never matched by the scope query,
dropped the chained-constructor CALLS edges. Synthesize them: `this` → enclosing
type name; `base` → the base type's bare name (first base-list entry, which C#
requires to be the base class). Arity attached for overload disambiguation.
- `interpretCsharpTypeBinding`'s qualifier strip used `lastIndexOf('.')` over the
whole string, cutting inside a qualified generic type ARGUMENT
(`Dictionary<string, Ns.User>` → `User>`). Make stripQualifier generic-aware:
reduce only the segment before the first `<`, re-attaching the generic suffix —
multi-arg generics stay intact so the `.Values`/`.Keys` collection-accessor
unwrap keeps working.
Tests: capture-level unit tests for every constructor shape (incl. alias-qualified,
double-match guard) and `: base`/`: this` (incl. struct/record/mixed-base);
interpretCsharpTypeBinding unit tests (the corruption case + nullable/nested/
unknown-generic edges); end-to-end resolver tests with new fixtures. The
csharp-captures golden was regenerated — drift is purely additive (only the new
fixtures; zero existing-fixture digests changed).
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(csharp): enhance constructor resolution and namespace qualification
- Implemented qualified constructor name binding to resolve collisions between types in different namespaces.
- Added support for `: base(...)` and `: this(...)` constructor initializers to ensure correct edge emission in the scope resolution.
- Improved generic argument stripping to prevent incorrect parsing of qualified types.
- Introduced tests for new features, including handling of interface-only base classes and qualified constructor calls.
This update addresses issues related to constructor resolution and namespace qualification, ensuring accurate type references in C# code. Tests have been added to validate these changes.
* fix(csharp): implement namespace prefix tagging for file-level type definitions
- Updated the C# ingestion process to tag file-level type definitions with their enclosing namespace path using a new `namespacePrefix` field, without altering the `qualifiedName`.
- Enhanced the scope resolver to utilize the `namespacePrefix` for resolving same-tail collisions in constructor calls, improving accuracy in type resolution.
- Added unit tests to validate the new functionality, ensuring that namespace prefixes are correctly applied to both block-scoped and file-scoped types, while leaving namespace-free types untagged.
This change addresses issues related to namespace qualification and constructor resolution in C# code, facilitating better handling of type references.
* refactor(scope-resolution): share isOverloadableCallable via util
Extract the ctor/function/method overload predicate into
callable-labels.ts so graph-bridge registration and lookup stay aligned
without duplicated private copies in ids.ts and node-lookup.ts.
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Gergő Magyar <gergomagyar@icloud.com>
* fix(java): close parsing-layer coverage gaps F35/F38/F41 (#1928)
Registry-primary scope-resolution path (the live one post-#942/#943):
- F35 [HIGH]: qualified / qualified-generic constructor calls. `new pkg.Foo()`
parses as a `scoped_type_identifier` that the query bound only as
`@reference.call.constructor.qualified` with no `@reference.name`, so the
scope extractor fell back to the whole-expression anchor and the reference
name became the raw `new pkg.Foo()` text (never resolved). Bind the simple
-name tail (end-anchored last child) and add an arm for the previously
uncaptured `new pkg.Box<String>()` (qualified + generic) shape.
- F38 [MEDIUM]: `super(...)` / `this(...)` explicit constructor invocations,
modeled as `explicit_constructor_invocation` and never matched by the scope
query, dropped the chained-constructor CALLS edges. Synthesize them with the
target resolved structurally (this -> enclosing type name; super -> superclass
tail via the shared javaBaseLookupNameNode, skipping implicit Object) plus
arity for overload disambiguation.
- F41 [LOW]: interpretJavaTypeBinding stripped the qualifier before generics, so
a qualified generic type arg (`Map<String, com.example.User>`) was cut inside
the generic into `User>`. Strip generics first, then the qualifier; make the
erasure fallback qualifier-tolerant.
F36/F37 already landed upstream (#1940/#1956); F39/F40 are legacy-bank remnants
that are no longer consumed (legacy @import skipped in parse-worker; legacy
@call never read in parse-impl) so they are intentionally left untouched.
Tests: low-level capture unit tests (constructor shapes incl. double-match
guard; super/this/enum/implicit-Object), interpretJavaTypeBinding unit tests
(qualified generic args + the corruption case), and end-to-end resolver tests
with new fixtures asserting the CALLS edges resolve to the correct constructors.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(scope-resolution): register Constructor overload keys so this()/super() chains don't self-loop (#1928 F38 review)
Review of #2045 caught two gaps; both confirmed by reproduction.
P2 — F38 this() emitted a self-loop. On the java-explicit-constructor fixture,
Child(int){ this(); } produced CALLS Child()#0 -> Child()#0 instead of
Child(int)#1 -> Child()#0. Root cause is the language-agnostic graph-bridge: the
parse phase mints distinct Constructor nodes (Child#0, Child#1) carrying
parameterTypes, but node-lookup.ts registered the parameter-types / shape
overload keys only for Function/Method, never Constructor, so both ctors
collapsed onto the first-wins qualified/simple key and the caller Child(int)
resolved to Child#0 (the this() target). Extend the overload keys to Constructor
in both node-lookup.ts (registration) and ids.ts (lookup) via a shared
isOverloadableCallable predicate. Verified the edge now connects distinct nodes
(Child#1 -> Child#0); super(1)->Base#1 still correct. No cross-language
regressions (the 9 worker-path failures reproduce identically on clean HEAD).
Also harden the integration test: it matched the this() edge on name only, which
a self-loop satisfies; now assert the endpoints are DISTINCT constructors.
P3 — F41 order-regression guard was inert (List<Map<String,User>> normalizes to
List under both strip orders). Add List<com.x.Foo<String>> -> List, which is
corrupted to Foo<String>> under the old order and only correct generics-first.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(java): update fingerprint and add notes for constructor query captures in baselines.json
Updated the fingerprint for the Java section and added detailed notes regarding the enhancements in constructor query captures, including qualified and qualified-generic constructor queries. This change reflects ongoing improvements in the parsing layer coverage and fixture updates.
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Gergő Magyar <gergomagyar@icloud.com>
