fix(grammars): load vendored tree-sitter grammars from vendor/ by absolute path (#2111) (#2144)

* fix(grammars): load vendored tree-sitter grammars from vendor/ by absolute path (#2111)

The recurring Windows `EPERM: operation not permitted, symlink` (errno -4048)
when adding the MCP server to Antigravity is NOT the #2101/#2110 module-load
crash — it is an install-time arborist failure during the `_npx` reify that the
MCP client triggers on every `npx gitnexus` launch.

Root cause: the `postinstall` materialize step copied each vendored grammar
(`vendor/tree-sitter-{c,dart,proto,swift,kotlin}`) into
`node_modules/gitnexus/node_modules/tree-sitter-*` as a real package so runtime
`require('tree-sitter-dart')` would resolve. Those packages are in no dependency
graph, so every subsequent npm/npx reify treats them as **extraneous** and
prunes/relocates them — on Windows the relocation goes through
`@npmcli/move-file`'s symlink path and throws EPERM (symlinks need Developer
Mode/admin), and on every OS the 2nd run silently deletes the grammars. This is
the same class as #1728, which the materialize step itself claimed to have
fixed.

Fix (the prebuildify + node-gyp-build ecosystem pattern): never copy grammars
into node_modules. Load each by absolute path from `vendor/<name>` via the new
`requireVendoredGrammar` helper — the grammar's own `bindings/node` runs
`node-gyp-build(<dir>)` and loads the committed `vendor/<name>/prebuilds/
<platform>-<arch>/…` directly (all 5 ship all 6 tuples). vendor/ is inside the
package but not a node_modules subtree, so arborist never sees the grammars and
the reify is idempotent — no EPERM, no silent deletion.

- new src/core/tree-sitter/vendored-grammars.ts (requireVendoredGrammar /
  vendoredGrammarDir / VENDORED_GRAMMAR_PACKAGES; VENDOR_ROOT stable in dev+dist)
- route all consumers through it: parser-loader, parse-worker, grpc proto,
  include-extractor (C), http-patterns kotlin, cli optional-grammars probe
- postinstall drops the materialize step; build-tree-sitter-grammars.cjs builds
  in-place under vendor/ (gitignored) and deletes materialize-vendor-grammars.cjs
- tests + grammar-introspection helper load grammars from vendor/ too (single
  source of truth); new vendored-grammars.test.ts guards against reintroducing a
  bare `require('tree-sitter-<vendored>')`

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(grammars): throw on a non-vendored name in requireVendoredGrammar

Drift guard (PR #2144 review, P3): validate the argument against
VENDORED_GRAMMAR_PACKAGES and fail loudly on an unknown name, so the three
grammar lists (package set / CLI probe / build registry) drifting out of sync
surfaces as a clear error instead of a confusing absolute-path require miss.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(grammars): prepack guard against stray vendor/<g>/build/ shadowing prebuilds

Publish hygiene (PR #2144 review, P2). Now that build-tree-sitter-grammars.cjs
source-builds into vendor/<name>/build/, a stray build dir would ship in the
tarball (files:["vendor"] overrides .gitignore/.npmignore) AND shadow the
committed prebuild — node-gyp-build resolves build/Release before prebuilds/.
assert-publish-grammar-coverage.cjs (prepack) now fails `npm pack` if any
vendor/*/build exists (findStrayBuildArtifacts), with a clear `rm -rf` fix hint.
Adds unit coverage for the new pure function.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(grammars): harden the #2111 no-bare-require regression guard

PR #2144 review (P2). The guard regex missed dynamic import(), side-effect
`import 'x'`, /subpath, and backtick loads, and only scanned src/. It now covers
every node_modules-forcing form (single/double/backtick quotes, optional
subpath), scans test/ too (excluding fixtures and the guard file itself), drops
the `//`-substring false-negative (leading-comment-only heuristic), and adds a
self-test asserting every load form is caught while prose mentions and
tree-sitter-cpp are ignored.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(grammars): correct stale vendored-grammar comments

PR #2144 review (P3). kotlin/query.ts called tree-sitter-kotlin an
"optionalDependency" — it is vendored and loaded from vendor/ by absolute path
(#2111). proto.ts now states its remaining `_require` is only for the real
`tree-sitter` dependency, not a vendored grammar (which goes through
requireVendoredGrammar). Comment-only; no behavior change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Gergő Magyar 2026-06-10 14:20:42 +01:00 committed by GitHub
parent 3d30b94c46
commit 2870aa6248
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
30 changed files with 417 additions and 197 deletions

View file

@ -49,7 +49,7 @@
"test:watch": "vitest",
"test:coverage": "vitest run --coverage",
"test:cross-platform": "tsx scripts/run-cross-platform.ts",
"postinstall": "node scripts/materialize-vendor-grammars.cjs && node scripts/build-tree-sitter-grammars.cjs",
"postinstall": "node scripts/build-tree-sitter-grammars.cjs",
"assert-publish-coverage": "node scripts/assert-publish-grammar-coverage.cjs",
"prepare": "node scripts/build.js",
"prepack": "node scripts/assert-publish-grammar-coverage.cjs && node scripts/build.js"

View file

@ -112,6 +112,24 @@ function findCoverageProblems({ grammars }) {
return problems;
}
/**
* Stray local source-build outputs under `vendor/<name>/build/`. These would
* ship in the tarball (`files: ["vendor"]` overrides .gitignore/.npmignore) AND
* shadow the committed prebuilds `node-gyp-build` resolves `build/Release`
* BEFORE `prebuilds/`, so a consumer on the publisher's platform would load the
* stray (possibly stale/wrong) binding instead of the curated prebuild. The
* build dir is gitignored and only appears if a maintainer source-built locally
* (e.g. on a no-prebuild platform); refuse to publish it. (#2144 review.)
*/
function findStrayBuildArtifacts(vendorDir) {
if (!fs.existsSync(vendorDir)) return [];
return fs
.readdirSync(vendorDir)
.filter((d) => /^tree-sitter-/.test(d))
.filter((d) => fs.existsSync(path.join(vendorDir, d, 'build')))
.map((d) => `vendor/${d}/build`);
}
function collectGrammars(vendorDir, shipsVendorSource) {
if (!fs.existsSync(vendorDir)) return [];
return fs
@ -141,6 +159,18 @@ function main() {
process.exit(1);
}
const stray = findStrayBuildArtifacts(vendorDir);
if (stray.length > 0) {
console.error(
'[publish-guard] Refusing to publish — stray source-build output under vendor/ would\n' +
'ship and shadow the committed prebuilds (node-gyp-build loads build/Release before\n' +
'prebuilds/):',
);
for (const s of stray) console.error(` - ${s}`);
console.error('\nFix: remove it before packing, e.g. `rm -rf gitnexus/vendor/*/build`.');
process.exit(1);
}
const problems = findCoverageProblems({ grammars });
if (problems.length > 0) {
console.error('[publish-guard] Refusing to publish — a vendored grammar would ship unusable:');
@ -163,6 +193,7 @@ if (require.main === module) main();
module.exports = {
findCoverageProblems,
findStrayBuildArtifacts,
filesShipsVendorSource,
isBuildableFromSource,
sourceBuildSet,

View file

@ -1,19 +1,25 @@
#!/usr/bin/env node
/**
* Activate the vendored tree-sitter native bindings after
* materialize-vendor-grammars.cjs. One registry-driven script replaces the
* former per-grammar build-tree-sitter-<name>.cjs files (they were ~95%
* identical).
* Activate the vendored tree-sitter native bindings IN PLACE under `vendor/`.
* One registry-driven script replaces the former per-grammar
* build-tree-sitter-<name>.cjs files (they were ~95% identical).
*
* The grammars (tree-sitter-c/dart/proto/swift/kotlin) are loaded from
* `vendor/<name>/` by absolute path at runtime (see
* src/core/tree-sitter/vendored-grammars.ts) and are NEVER copied into
* node_modules an undeclared package under node_modules is "extraneous" to
* every subsequent npm/npx reify, which prunes/relocates it (Windows
* `EPERM: …, symlink` + a silent grammar deletion on the 2nd run; #2111/#1728).
*
* For each grammar the resolution order is identical:
* 1. If the package isn't materialized (no binding.gyp) or the binding is
* 1. If the vendored source is absent (no binding.gyp) or the binding is
* already built, do nothing.
* 2. Prefer a committed prebuild for this platform-arch (toolchain-free) via
* node-gyp-build the goal once build-tree-sitter-prebuilds.yml has
* populated all six tuples.
* node-gyp-build `vendor/<name>/prebuilds/` ships all six tuples, so on a
* supported platform this returns immediately and writes nothing.
* 3. Otherwise source-build from the vendored grammar source (binding.gyp +
* src/) so parsing still works on any toolchain host e.g. CI, before the
* prebuilds land.
* src/) into `vendor/<name>/build/` (gitignored) so parsing still works on
* a toolchain host that lacks a matching prebuild.
*
* HARD INVARIANT: this runs in `gitnexus`'s postinstall, so it MUST NEVER throw
* or exit non-zero a failure for any single grammar must not break the install.
@ -53,7 +59,7 @@ function buildGrammar(short) {
return;
}
const dir = path.join(__dirname, '..', 'node_modules', `tree-sitter-${short}`);
const dir = path.join(__dirname, '..', 'vendor', `tree-sitter-${short}`);
const bindingGyp = path.join(dir, 'binding.gyp');
const bindingNode = path.join(dir, 'build', 'Release', `tree_sitter_${short}_binding.node`);

View file

@ -1,97 +0,0 @@
#!/usr/bin/env node
/**
* Copy vendored tree-sitter grammars into node_modules/ using real files (fs.cpSync).
*
* Published gitnexus used to declare these as optionalDependencies with
* `file:./vendor/...`, which makes npm symlink/junction vendor node_modules on
* install. Windows without Developer Mode often fails with EPERM (#1728).
*
* Vendor trees stay read-only in gitnexus/vendor/; build artifacts must only
* land under node_modules/ (see #836).
*/
const fs = require('fs');
const path = require('path');
const ROOT = path.join(__dirname, '..');
// tree-sitter-c is a REQUIRED grammar that we vendor prebuild-only purely to
// close upstream's ARM prebuild gap (#2116) — it needs no toolchain and is not a
// language the user opts out of, so it is always materialized, even under
// GITNEXUS_SKIP_OPTIONAL_GRAMMARS. The rest are optional (user-skippable, and
// Dart/Proto compile from source) and honor the skip flag.
const REQUIRED_VENDORED = ['tree-sitter-c'];
const OPTIONAL_VENDORED = [
'tree-sitter-dart',
'tree-sitter-proto',
'tree-sitter-swift',
'tree-sitter-kotlin',
];
const skipOptional = process.env.GITNEXUS_SKIP_OPTIONAL_GRAMMARS === '1';
if (skipOptional) {
console.warn(
'[gitnexus] GITNEXUS_SKIP_OPTIONAL_GRAMMARS=1: skipping optional Dart/Proto/Swift/Kotlin materialize (required C is still materialized).',
);
}
const VENDORED_GRAMMARS = skipOptional
? REQUIRED_VENDORED
: [...REQUIRED_VENDORED, ...OPTIONAL_VENDORED];
for (const name of VENDORED_GRAMMARS) {
const src = path.join(ROOT, 'vendor', name);
const dest = path.join(ROOT, 'node_modules', name);
if (!fs.existsSync(src)) {
console.warn(`[gitnexus] vendor/${name} missing; skipping materialize.`);
continue;
}
// Sequence: copy src → partial; rename dest → backup; rename partial → dest;
// remove backup. If any step fails, restore from backup so a previously-
// materialized grammar is never lost. Targets the #1728 EPERM scenario plus
// narrower failure modes (Windows AV scanner racing on rename, EBUSY mid-swap).
const partial = `${dest}.materialize-tmp`;
const backup = `${dest}.materialize-bak`;
try {
fs.mkdirSync(path.join(ROOT, 'node_modules'), { recursive: true });
fs.rmSync(partial, { recursive: true, force: true });
fs.rmSync(backup, { recursive: true, force: true });
fs.cpSync(src, partial, { recursive: true, verbatim: true });
if (fs.existsSync(dest)) {
fs.renameSync(dest, backup);
}
try {
fs.renameSync(partial, dest);
} catch (renameErr) {
// Best-effort rollback: restore the previous dest from backup.
let restored = false;
if (fs.existsSync(backup)) {
try {
fs.renameSync(backup, dest);
restored = true;
} catch {
// Rollback also failed — dest is now missing. Leave the backup in
// place (the catch below will NOT remove it) and surface where it is.
}
}
if (!restored && fs.existsSync(backup)) {
console.warn(
`[gitnexus] CRITICAL: could not materialize vendor/${name} AND could not restore the ` +
`previous node_modules/${name}. A recoverable copy remains at ${backup}` +
`restore it (e.g. \`mv ${backup} ${dest}\`) or reinstall to recover ${name}.`,
);
}
throw renameErr;
}
fs.rmSync(backup, { recursive: true, force: true });
} catch (err) {
// Fail-soft: a single locked/inaccessible file (common on Windows) must not
// abort the whole gitnexus install. Matches build-tree-sitter-*.cjs pattern.
// Only remove the scratch `partial`; never the `backup` (it may be the sole
// recoverable copy after a failed rollback above).
fs.rmSync(partial, { recursive: true, force: true });
console.warn(`[gitnexus] Could not materialize vendor/${name}: ${err.message}`);
console.warn(
`[gitnexus] ${name} parsing will be unavailable. Other functionality is unaffected.`,
);
}
}

View file

@ -1,15 +1,13 @@
/**
* Optional grammar availability check.
*
* tree-sitter-dart, tree-sitter-proto, and tree-sitter-swift are vendored
* under vendor/ and materialized into node_modules/ at postinstall. Dart
* and Proto are built from source with node-gyp; Swift ships platform
* prebuilds activated via node-gyp-build. tree-sitter-kotlin is a declared
* optionalDependency (not vendored). All can be skipped via
* tree-sitter-dart, -proto, -swift, and -kotlin are vendored under vendor/ and
* loaded from there by absolute path (NEVER copied into node_modules see
* core/tree-sitter/vendored-grammars.ts / #2111). Each ships committed platform
* prebuilds activated via node-gyp-build. All can be skipped via
* GITNEXUS_SKIP_OPTIONAL_GRAMMARS=1 (postinstall scripts), or can silently
* soft-fail when the toolchain is missing (Dart/Proto), when no prebuild
* matches the host platform (Swift), or when the optional install was
* skipped or its native build failed (Kotlin).
* soft-fail when no prebuild matches the host platform (and a source build was
* unavailable / not attempted).
*
* Either path produces the same observable: the .node binding is absent
* at runtime. This helper detects that condition and surfaces a single
@ -17,17 +15,15 @@
* support is unavailable instead of silently getting a degraded index.
*/
import { createRequire } from 'module';
import { SupportedLanguages } from 'gitnexus-shared';
import { isGrammarRuntimeSkipped } from '../core/tree-sitter/parser-loader.js';
import { requireVendoredGrammar } from '../core/tree-sitter/vendored-grammars.js';
import { cliWarn } from './cli-message.js';
const _require = createRequire(import.meta.url);
interface OptionalGrammar {
/** Display name in warnings */
name: string;
/** Module name to require.resolve */
/** Vendored grammar package name (directory under vendor/) */
pkg: string;
/** File extensions this grammar parses */
extensions: string[];
@ -109,7 +105,7 @@ export function detectMissingOptionalGrammars(): MissingGrammar[] {
continue;
}
try {
_require(g.pkg);
requireVendoredGrammar(g.pkg);
} catch (err) {
const code = (err as NodeJS.ErrnoException | undefined)?.code;
const msg = err instanceof Error ? err.message : String(err);

View file

@ -1,4 +1,5 @@
import { createRequire } from 'node:module';
import { requireVendoredGrammar } from '../../../tree-sitter/vendored-grammars.js';
import {
compilePatterns,
runCompiledPatterns,
@ -10,11 +11,11 @@ import type { GrpcDetection, GrpcLanguagePlugin } from './types.js';
/**
* Protobuf (.proto) tree-sitter plugin for gRPC contract extraction.
*
* Uses `tree-sitter-proto` (coder3101/tree-sitter-proto) as an
* optionalDependency if the grammar is not installed (e.g. native
* compilation failed on an unusual platform), the plugin exports
* `null` and the orchestrator falls back to the existing manual
* string-sanitizing parser.
* Uses `tree-sitter-proto` (coder3101/tree-sitter-proto), loaded from
* `vendor/` by absolute path (NEVER copied into node_modules see
* vendored-grammars.ts / #2111). If the grammar's binding cannot be loaded
* (e.g. no prebuild for an unusual platform), the plugin exports `null` and the
* orchestrator falls back to the existing manual string-sanitizing parser.
*
* The grammar is vendored in `vendor/tree-sitter-proto/` with
* parser.c regenerated against tree-sitter-cli 0.24 (ABI version 14)
@ -22,10 +23,13 @@ import type { GrpcDetection, GrpcLanguagePlugin } from './types.js';
* (which loads ABI 1314).
*/
// Only for `tree-sitter` (a real npm dependency) in the smoke-test below;
// the vendored grammar goes through requireVendoredGrammar (never a bare
// `_require('tree-sitter-proto')`, which would force a node_modules copy — #2111).
const _require = createRequire(import.meta.url);
let ProtoGrammar: unknown = null;
try {
ProtoGrammar = _require('tree-sitter-proto');
ProtoGrammar = requireVendoredGrammar('tree-sitter-proto');
} catch {
// Grammar not installed — PROTO_GRPC_PLUGIN will be null.
}

View file

@ -1,5 +1,5 @@
import Parser from 'tree-sitter';
import { createRequire } from 'node:module';
import { requireVendoredGrammar } from '../../../tree-sitter/vendored-grammars.js';
import {
compilePatterns,
runCompiledPatterns,
@ -60,17 +60,16 @@ import type { HttpDetection, HttpLanguagePlugin } from './types.js';
* value_argument
* string_literal the path
*
* tree-sitter-kotlin is an optional npm dependency when its native
* binding is unavailable the plugin gracefully exports `null` and
* `http-patterns/index.ts` skips registration for `.kt`/`.kts` files.
* tree-sitter-kotlin is a vendored grammar loaded from `vendor/` by absolute
* path (NEVER copied into node_modules see vendored-grammars.ts / #2111)
* when its native binding is unavailable the plugin gracefully exports `null`
* and `http-patterns/index.ts` skips registration for `.kt`/`.kts` files.
*/
const _require = createRequire(import.meta.url);
/** Loaded lazily; null when the grammar binding isn't installed. */
/** Loaded lazily; null when the grammar binding isn't available. */
let Kotlin: unknown | null = null;
try {
Kotlin = _require('tree-sitter-kotlin');
Kotlin = requireVendoredGrammar('tree-sitter-kotlin');
} catch {
Kotlin = null;
}

View file

@ -1,20 +1,20 @@
import * as path from 'node:path';
import * as fs from 'node:fs/promises';
import { createRequire } from 'node:module';
import { glob } from 'glob';
import Parser from 'tree-sitter';
import Cpp from 'tree-sitter-cpp';
import { requireVendoredGrammar } from '../../tree-sitter/vendored-grammars.js';
// `tree-sitter-c` is vendored prebuild-only (#2116) and may be absent on a
// toolchain-less / `--ignore-scripts` install. Load it via a guarded `_require`
// rather than a top-level `import C from 'tree-sitter-c'`, which would throw
// ERR_MODULE_NOT_FOUND at module-load and crash analyze (#2091/#2093). When the
// `tree-sitter-c` is vendored (#2116), loaded from `vendor/` by absolute path
// (NEVER copied into node_modules — see vendored-grammars.ts / #2111). Load it
// via a guarded call rather than a top-level `import C from 'tree-sitter-c'`,
// which would throw ERR_MODULE_NOT_FOUND at module-load and crash analyze
// (#2091/#2093). It may be absent on a platform without a prebuild; when the
// binding is absent, `getLanguageForFile` returns null for `.c`/`.h` so C
// include-extraction is skipped (C++ is unaffected — its binding always ships).
const _require = createRequire(import.meta.url);
let C: unknown = null;
try {
C = _require('tree-sitter-c');
C = requireVendoredGrammar('tree-sitter-c');
} catch {
/* C grammar unavailable — C include extraction degrades to a no-op. */
}

View file

@ -1,7 +1,8 @@
import Parser from 'tree-sitter';
import { SupportedLanguages } from 'gitnexus-shared';
// `tree-sitter-kotlin` is an optionalDependency that may be absent on a default
// install (or fail its native build). Loaded lazily + guarded via parser-loader
// `tree-sitter-kotlin` is a vendored grammar (loaded from vendor/ by absolute
// path, never node_modules — vendored-grammars.ts / #2111) that may be absent on
// a platform without a matching prebuild. Loaded lazily + guarded via parser-loader
// rather than statically imported: this module is pulled onto the main thread
// eagerly by the scope-resolution registry and the language-provider index, so
// a top-level `import Kotlin from 'tree-sitter-kotlin'` would throw

View file

@ -11,7 +11,7 @@ import Go from 'tree-sitter-go';
import Rust from 'tree-sitter-rust';
import PHP from 'tree-sitter-php';
import Ruby from 'tree-sitter-ruby';
import { createRequire } from 'node:module';
import { requireVendoredGrammar } from '../../tree-sitter/vendored-grammars.js';
import { SupportedLanguages } from 'gitnexus-shared';
import { getProvider } from '../languages/index.js';
import {
@ -39,8 +39,8 @@ import type {
type TreeSitterLanguage = Parameters<typeof Parser.prototype.setLanguage>[0];
// ── Worker grammar loading — enforcement boundary (#2091/#2093, #2101) ───────
// The worker maintains its own grammar table (the guarded `_require`s below +
// `languageMap`) and intentionally does NOT consult the runtime
// The worker maintains its own grammar table (the guarded vendored-grammar
// loads below + `languageMap`) and intentionally does NOT consult the runtime
// `GITNEXUS_SKIP_OPTIONAL_GRAMMARS` opt-out. It does not need to: the MAIN
// THREAD's `parseableScanned` filter (pipeline-phases/parse-impl.ts, gated on
// `parser-loader.isLanguageAvailable`, which honors the runtime opt-out and a
@ -51,33 +51,29 @@ type TreeSitterLanguage = Parameters<typeof Parser.prototype.setLanguage>[0];
// `isLanguageAvailable` must re-introduce the gate here. (The cleaner end-state
// — routing this table through `parser-loader.getLanguageGrammar` so there is
// one loader — is the deferred Tier-1 consolidation.)
// tree-sitter-swift is an optionalDependency — may not be installed
const _require = createRequire(import.meta.url);
// Swift/Dart/Kotlin/C are vendored grammars loaded from `vendor/` by absolute
// path (NEVER copied into node_modules — see vendored-grammars.ts / #2111). Each
// may be absent on a platform without a prebuild or a toolchain-less /
// `--ignore-scripts` install, so every load is guarded so a missing binding
// cannot crash the worker at module-load (#2091/#2093, #2116).
let Swift: TreeSitterLanguage | null = null;
try {
Swift = _require('tree-sitter-swift');
Swift = requireVendoredGrammar('tree-sitter-swift') as TreeSitterLanguage;
} catch {}
// tree-sitter-dart is an optionalDependency — may not be installed
let Dart: TreeSitterLanguage | null = null;
try {
Dart = _require('tree-sitter-dart');
Dart = requireVendoredGrammar('tree-sitter-dart') as TreeSitterLanguage;
} catch {}
// tree-sitter-kotlin is an optionalDependency — may not be installed
let Kotlin: TreeSitterLanguage | null = null;
try {
Kotlin = _require('tree-sitter-kotlin');
Kotlin = requireVendoredGrammar('tree-sitter-kotlin') as TreeSitterLanguage;
} catch {}
// tree-sitter-c is now vendored prebuild-only (#2116) and may be absent on a
// toolchain-less / `--ignore-scripts` install. Guard it like Swift/Dart/Kotlin so
// a missing binding cannot crash the worker at module-load (#2091/#2093); the
// main-thread `isLanguageAvailable` filter keeps C files from being dispatched
// here when the entry is absent.
let C: TreeSitterLanguage | null = null;
try {
C = _require('tree-sitter-c');
C = requireVendoredGrammar('tree-sitter-c') as TreeSitterLanguage;
} catch {}
import { getLanguageFromFilename } from 'gitnexus-shared';
import {

View file

@ -3,6 +3,7 @@ import { createRequire } from 'node:module';
import { SupportedLanguages } from 'gitnexus-shared';
import { logger } from '../logger.js';
import { requireVendoredGrammar } from './vendored-grammars.js';
const _require = createRequire(import.meta.url);
/**
@ -131,7 +132,7 @@ const SOURCES: Record<string, GrammarSource> = {
// user-opt-out grammar like Swift/Dart/Kotlin: a failure here is always an
// install/platform problem the user needs to see.
[SupportedLanguages.C]: {
load: () => _require('tree-sitter-c'),
load: () => requireVendoredGrammar('tree-sitter-c'),
optional: true,
severity: 'error',
unavailableNote:
@ -147,7 +148,7 @@ const SOURCES: Record<string, GrammarSource> = {
// optionalDependencies — may be absent on platforms without prebuilds
// or when users skip optional installs.
[SupportedLanguages.Swift]: {
load: () => _require('tree-sitter-swift'),
load: () => requireVendoredGrammar('tree-sitter-swift'),
optional: true,
userSkippable: true,
unavailableNote:
@ -157,7 +158,7 @@ const SOURCES: Record<string, GrammarSource> = {
`See ${ISSUES_URL}/1130.`,
},
[SupportedLanguages.Dart]: {
load: () => _require('tree-sitter-dart'),
load: () => requireVendoredGrammar('tree-sitter-dart'),
optional: true,
userSkippable: true,
unavailableNote:
@ -167,7 +168,7 @@ const SOURCES: Record<string, GrammarSource> = {
`See ${ISSUES_URL}/1125.`,
},
[SupportedLanguages.Kotlin]: {
load: () => _require('tree-sitter-kotlin'),
load: () => requireVendoredGrammar('tree-sitter-kotlin'),
optional: true,
userSkippable: true,
unavailableNote:

View file

@ -0,0 +1,71 @@
import { createRequire } from 'node:module';
import { fileURLToPath } from 'node:url';
import path from 'node:path';
const _require = createRequire(import.meta.url);
/**
* Absolute path to the vendored grammar tree (`<pkg>/vendor`).
*
* This module compiles to `<pkg>/dist/core/tree-sitter/vendored-grammars.js`
* and runs from `<pkg>/src/core/tree-sitter/...` under tsx in dev both sit
* three directories below the package root, and the build (`tsc`) never bundles,
* so `import.meta.url` resolves the same way in both. `vendor/` ships in the
* published package via package.json `files`.
*/
export const VENDOR_ROOT = path.resolve(
path.dirname(fileURLToPath(import.meta.url)),
'..',
'..',
'..',
'vendor',
);
/**
* The tree-sitter grammars GitNexus vendors inside its own package (NOT npm
* dependencies). Kept in one place so consumers (runtime loaders, the CLI
* availability probe, and the test grammar-introspection helper) agree on which
* grammars resolve from `vendor/` rather than `node_modules`.
*/
export const VENDORED_GRAMMAR_PACKAGES: ReadonlySet<string> = new Set([
'tree-sitter-c',
'tree-sitter-dart',
'tree-sitter-proto',
'tree-sitter-swift',
'tree-sitter-kotlin',
]);
/** Absolute directory of a vendored grammar package under `vendor/`. */
export const vendoredGrammarDir = (packageName: string): string =>
path.join(VENDOR_ROOT, packageName);
/**
* Load a vendored tree-sitter grammar by its absolute path under `vendor/`.
*
* GitNexus vendors five grammars (c/dart/proto/swift/kotlin) inside its own
* package under `vendor/`, each shipping committed per-platform prebuilds. They
* are deliberately NOT npm dependencies and must NEVER be copied into
* `node_modules`: an undeclared package under `node_modules` is "extraneous" to
* every subsequent `npm`/`npx` arborist reify, which prunes or relocates it.
* That is the root cause of #2111 / #1728 on Windows the relocation throws
* `EPERM: operation not permitted, symlink` (errno -4048) during the npx-cache
* reify Antigravity triggers when it launches the MCP server, and on every OS
* the second run silently deletes the materialized grammars.
*
* Resolving the grammar by absolute path runs its own `bindings/node` entry,
* which calls `node-gyp-build(<grammarDir>)` and loads
* `vendor/<name>/prebuilds/<platform>-<arch>/…` directly no build, no write,
* no `node_modules` copy. (`node-gyp-build` itself IS an npm dependency and
* resolves normally from the grammar directory.)
*/
export const requireVendoredGrammar = (packageName: string): unknown => {
// Fail loudly on a name that isn't actually vendored — a typo or a list that
// drifted out of sync (VENDORED_GRAMMAR_PACKAGES vs the CLI probe vs the build
// registry) would otherwise surface as a confusing absolute-path require miss.
if (!VENDORED_GRAMMAR_PACKAGES.has(packageName)) {
throw new Error(
`'${packageName}' is not a vendored grammar (expected one of: ${[...VENDORED_GRAMMAR_PACKAGES].join(', ')}).`,
);
}
return _require(vendoredGrammarDir(packageName));
};

View file

@ -28,6 +28,10 @@ import {
isLanguageAvailable,
resolveLanguageKey,
} from '../../src/core/tree-sitter/parser-loader.js';
import {
VENDORED_GRAMMAR_PACKAGES,
vendoredGrammarDir,
} from '../../src/core/tree-sitter/vendored-grammars.js';
const _require = createRequire(import.meta.url);
@ -101,6 +105,13 @@ interface NodeTypeEntry {
/** Resolve the on-disk directory of an installed package, or null if absent. */
function resolvePackageDir(pkg: string): string | null {
// Vendored grammars (c/dart/proto/swift/kotlin) are NOT in node_modules — they
// load from vendor/ by absolute path (vendored-grammars.ts / #2111), so resolve
// their node-types.json from there rather than via _require.resolve.
if (VENDORED_GRAMMAR_PACKAGES.has(pkg)) {
const dir = vendoredGrammarDir(pkg);
return existsSync(dir) ? dir : null;
}
try {
return dirname(_require.resolve(`${pkg}/package.json`));
} catch {

View file

@ -17,8 +17,11 @@
import { describe, it, expect, beforeAll } from 'vitest';
import path from 'path';
import Parser from 'tree-sitter';
import Kotlin from 'tree-sitter-kotlin';
import { requireVendoredGrammar } from '../../../src/core/tree-sitter/vendored-grammars.js';
import { emitKotlinScopeCaptures } from '../../../src/core/ingestion/languages/kotlin/captures.js';
// Vendored grammar — loaded from vendor/ by absolute path, never node_modules (#2111).
const Kotlin = requireVendoredGrammar('tree-sitter-kotlin');
import { KOTLIN_QUERIES } from '../../../src/core/ingestion/tree-sitter-queries.js';
import { FIXTURES, getRelationships, runPipelineFromRepo, type PipelineResult } from './helpers.js';
import type { CaptureMatch } from 'gitnexus-shared';

View file

@ -2,6 +2,9 @@ import { describe, it, expect } from 'vitest';
import { spawnSync } from 'node:child_process';
import { createRequire } from 'node:module';
import { fileURLToPath } from 'node:url';
import { mkdtempSync, mkdirSync, rmSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import path from 'node:path';
/**
* Coverage for the publish guard `scripts/assert-publish-grammar-coverage.cjs`.
@ -18,7 +21,8 @@ const requireCjs = createRequire(import.meta.url);
const SCRIPT = fileURLToPath(
new URL('../../scripts/assert-publish-grammar-coverage.cjs', import.meta.url),
);
const { findCoverageProblems, filesShipsVendorSource } = requireCjs(SCRIPT);
const { findCoverageProblems, filesShipsVendorSource, findStrayBuildArtifacts } =
requireCjs(SCRIPT);
describe('findCoverageProblems (pure decision core)', () => {
it('passes when source ships, even with incomplete prebuilds (transitional state)', () => {
@ -73,6 +77,43 @@ describe('filesShipsVendorSource', () => {
});
});
describe('findStrayBuildArtifacts (stray vendor build dirs that would ship + shadow prebuilds)', () => {
const mkVendor = (): string => mkdtempSync(path.join(tmpdir(), 'vguard-'));
it('returns [] when no grammar has a build/ dir', () => {
const dir = mkVendor();
try {
mkdirSync(path.join(dir, 'tree-sitter-y', 'prebuilds', 'linux-x64'), { recursive: true });
writeFileSync(path.join(dir, 'tree-sitter-y', 'prebuilds', 'linux-x64', 'y.node'), '');
expect(findStrayBuildArtifacts(dir)).toEqual([]);
} finally {
rmSync(dir, { recursive: true, force: true });
}
});
it('flags a grammar that carries a stray build/ output (would shadow the prebuild)', () => {
const dir = mkVendor();
try {
mkdirSync(path.join(dir, 'tree-sitter-x', 'build', 'Release'), { recursive: true });
mkdirSync(path.join(dir, 'tree-sitter-y', 'prebuilds'), { recursive: true });
expect(findStrayBuildArtifacts(dir)).toEqual(['vendor/tree-sitter-x/build']);
} finally {
rmSync(dir, { recursive: true, force: true });
}
});
it('ignores non-grammar dirs and a missing vendor dir', () => {
const dir = mkVendor();
try {
mkdirSync(path.join(dir, 'leiden', 'build'), { recursive: true }); // not tree-sitter-*
expect(findStrayBuildArtifacts(dir)).toEqual([]);
} finally {
rmSync(dir, { recursive: true, force: true });
}
expect(findStrayBuildArtifacts(path.join(tmpdir(), 'vguard-does-not-exist'))).toEqual([]);
});
});
describe('real repo publish-safety (guards against premature files narrowing)', () => {
it('the script exits 0 against the committed repo state', () => {
// Deterministic: reads package.json + walks vendor/ — no npm pack, fast.

View file

@ -18,7 +18,7 @@ import { fileURLToPath } from 'node:url';
* asserts exit code 0 every time, plus the required-vs-optional opt-out split.
*
* The script is copied into an isolated temp `scripts/` dir so its
* `__dirname`-relative `../node_modules/tree-sitter-<name>` resolves under our
* `__dirname`-relative `../vendor/tree-sitter-<name>` resolves under our
* control. The temp dir has no reachable `node-gyp-build` / `node-addon-api`, so
* the source-build path stops at the "hoisted build deps not resolvable" guard
* (still exit 0) instead of invoking a real compile.
@ -61,8 +61,9 @@ function runBuild(grammar: string, overrides: Record<string, string | undefined>
}
function materializeShell(grammar: string) {
// A package shell with a binding.gyp present but no prebuild / built binary.
const pkg = path.join(tmpRoot, 'node_modules', `tree-sitter-${grammar}`);
// A vendored grammar shell with a binding.gyp present but no prebuild / built
// binary — mirrors `vendor/tree-sitter-<name>/` (the script's build target).
const pkg = path.join(tmpRoot, 'vendor', `tree-sitter-${grammar}`);
mkdirSync(path.join(pkg, 'bindings', 'node'), { recursive: true });
writeFileSync(path.join(pkg, 'binding.gyp'), '{ "targets": [] }');
writeFileSync(path.join(pkg, 'bindings', 'node', 'index.js'), '');
@ -102,7 +103,7 @@ describe('build-tree-sitter-grammars.cjs consolidated activation', () => {
expect(r.stderr).toMatch(/hoisted build deps not resolvable|Could not build native binding/);
expect(r.stderr).not.toContain('built successfully');
} finally {
rmSync(path.join(tmpRoot, 'node_modules'), { recursive: true, force: true });
rmSync(path.join(tmpRoot, 'vendor'), { recursive: true, force: true });
}
});

View file

@ -10,7 +10,7 @@ import TypeScript from 'tree-sitter-typescript';
import Python from 'tree-sitter-python';
import Java from 'tree-sitter-java';
import CSharp from 'tree-sitter-c-sharp';
import Kotlin from 'tree-sitter-kotlin';
import { requireVendoredGrammar } from '../../src/core/tree-sitter/vendored-grammars.js';
import Go from 'tree-sitter-go';
import Rust from 'tree-sitter-rust';
import CPP from 'tree-sitter-cpp';
@ -18,6 +18,9 @@ import PHP from 'tree-sitter-php';
import { SupportedLanguages } from '../../src/config/supported-languages.js';
import { getProvider } from '../../src/core/ingestion/languages/index.js';
// Vendored grammar — loaded from vendor/ by absolute path, never node_modules (#2111).
const Kotlin = requireVendoredGrammar('tree-sitter-kotlin');
/**
* Helper: parse code, run the language query, and return all @call captures
* as { callNode, nameNode } pairs.

View file

@ -74,13 +74,19 @@ describe('CLI commands', () => {
});
describe('optional parser dependencies', () => {
it('materializes vendored grammars at postinstall instead of file: optionalDependencies (#1728)', async () => {
it('loads vendored grammars from vendor/ — never file: optionalDependencies (#1728) nor a node_modules copy (#2111)', async () => {
const pkg = await import('../../package.json', { with: { type: 'json' } });
const optional = pkg.default.optionalDependencies ?? {};
expect(optional['tree-sitter-dart']).toBeUndefined();
expect(optional['tree-sitter-proto']).toBeUndefined();
expect(optional['tree-sitter-swift']).toBeUndefined();
expect(pkg.default.scripts.postinstall).toContain('materialize-vendor-grammars.cjs');
// #2111: the grammars MUST NOT be copied into node_modules at install — an
// undeclared node_modules package is "extraneous" to every subsequent
// npm/npx reify, which prunes/relocates it (Windows EPERM symlink + silent
// deletion on the 2nd run). They are loaded from vendor/ by absolute path
// (vendored-grammars.ts), so postinstall no longer materializes anything.
expect(pkg.default.scripts.postinstall).not.toContain('materialize-vendor-grammars.cjs');
expect(pkg.default.scripts.postinstall).toContain('build-tree-sitter-grammars.cjs');
expect(pkg.default.files).toContain('vendor');
});
@ -138,7 +144,7 @@ describe('CLI commands', () => {
const optional = pkg.default.optionalDependencies ?? {};
// Kotlin is now VENDORED (like Swift/Dart/Proto), not a third-party npm
// optionalDependency. Its prebuilds are GitNexus-cross-built (upstream
// ships source only) and materialized into node_modules/ at postinstall.
// ships source only) and loaded from vendor/ by absolute path (#2111).
expect(optional['tree-sitter-kotlin']).toBeUndefined();
expect(pkg.default.scripts.postinstall).toContain('build-tree-sitter-grammars.cjs');
expect(kotlinPkg.default.version).toBe('0.3.8');

View file

@ -19,20 +19,23 @@ import Go from 'tree-sitter-go';
import Cpp from 'tree-sitter-cpp';
import Ruby from 'tree-sitter-ruby';
import CSharp from 'tree-sitter-c-sharp';
import Dart from 'tree-sitter-dart';
import { requireVendoredGrammar } from '../../src/core/tree-sitter/vendored-grammars.js';
// Vendored grammars — loaded from vendor/ by absolute path, never node_modules (#2111).
const Dart = requireVendoredGrammar('tree-sitter-dart');
let Kotlin: unknown;
try {
Kotlin = require('tree-sitter-kotlin');
Kotlin = requireVendoredGrammar('tree-sitter-kotlin');
} catch {
// Kotlin grammar may not be installed
// Kotlin grammar may not have a prebuild for this platform
}
let Swift: unknown;
try {
Swift = require('tree-sitter-swift');
Swift = requireVendoredGrammar('tree-sitter-swift');
} catch {
// Swift grammar is an optional dependency; may not be installed
// Swift grammar may not have a prebuild for this platform
}
import { csharpConfig as csharpFieldConfig } from '../../src/core/ingestion/field-extractors/configs/csharp.js';
import { SupportedLanguages } from '../../src/config/supported-languages.js';

View file

@ -16,10 +16,13 @@ import {
TREE_SITTER_MAX_BUFFER,
} from '../../src/core/ingestion/constants.js';
import Parser from 'tree-sitter';
import C from 'tree-sitter-c';
import CPP from 'tree-sitter-cpp';
import Python from 'tree-sitter-python';
import TypeScript from 'tree-sitter-typescript';
import { requireVendoredGrammar } from '../../src/core/tree-sitter/vendored-grammars.js';
// Vendored grammar — loaded from vendor/ by absolute path, never node_modules (#2111).
const C = requireVendoredGrammar('tree-sitter-c');
describe('getLanguageFromFilename', () => {
describe('TypeScript', () => {

View file

@ -30,17 +30,19 @@ import PHP from 'tree-sitter-php';
import Ruby from 'tree-sitter-ruby';
import Rust from 'tree-sitter-rust';
import { SupportedLanguages } from '../../src/config/supported-languages.js';
import { requireVendoredGrammar } from '../../src/core/tree-sitter/vendored-grammars.js';
// Vendored grammars — loaded from vendor/ by absolute path, never node_modules (#2111).
let Kotlin: unknown;
try {
Kotlin = require('tree-sitter-kotlin');
Kotlin = requireVendoredGrammar('tree-sitter-kotlin');
} catch {
// Kotlin grammar may not be installed
}
let Dart: unknown;
try {
Dart = require('tree-sitter-dart');
Dart = requireVendoredGrammar('tree-sitter-dart');
// Verify the grammar actually works with the installed tree-sitter version
const testParser = new Parser();
testParser.setLanguage(Dart as Parser.Language);
@ -50,7 +52,7 @@ try {
let Swift: unknown;
try {
Swift = require('tree-sitter-swift');
Swift = requireVendoredGrammar('tree-sitter-swift');
// Verify the grammar actually works with the installed tree-sitter version
const testParser = new Parser();
testParser.setLanguage(Swift as Parser.Language);

View file

@ -12,12 +12,14 @@ import Parser from 'tree-sitter';
import Ruby from 'tree-sitter-ruby';
import { findEnclosingClassInfo } from '../../src/core/ingestion/utils/ast-helpers.js';
import { rubyProvider } from '../../src/core/ingestion/languages/ruby.js';
import { requireVendoredGrammar } from '../../src/core/tree-sitter/vendored-grammars.js';
// Vendored grammar — loaded from vendor/ by absolute path, never node_modules (#2111).
let Kotlin: unknown;
try {
Kotlin = require('tree-sitter-kotlin');
Kotlin = requireVendoredGrammar('tree-sitter-kotlin');
} catch {
// Kotlin grammar may not be installed
// Kotlin grammar may not have a prebuild for this platform
}
const parser = new Parser();

View file

@ -18,13 +18,16 @@ import Go from 'tree-sitter-go';
import Rust from 'tree-sitter-rust';
import Python from 'tree-sitter-python';
import CPP from 'tree-sitter-cpp';
import Kotlin from 'tree-sitter-kotlin';
import PHP from 'tree-sitter-php';
import Ruby from 'tree-sitter-ruby';
import { requireVendoredGrammar } from '../../src/core/tree-sitter/vendored-grammars.js';
// Vendored grammars — loaded from vendor/ by absolute path, never node_modules (#2111).
const Kotlin = requireVendoredGrammar('tree-sitter-kotlin');
let Dart: unknown;
try {
Dart = require('tree-sitter-dart');
Dart = requireVendoredGrammar('tree-sitter-dart');
const testParser = new Parser();
testParser.setLanguage(Dart as Parser.Language);
} catch {
@ -33,7 +36,7 @@ try {
let Swift: unknown;
try {
Swift = require('tree-sitter-swift');
Swift = requireVendoredGrammar('tree-sitter-swift');
const testParser = new Parser();
testParser.setLanguage(Swift as Parser.Language);
} catch {

View file

@ -23,15 +23,18 @@ import Python from 'tree-sitter-python';
import Go from 'tree-sitter-go';
import Rust from 'tree-sitter-rust';
import Cpp from 'tree-sitter-cpp';
import C from 'tree-sitter-c';
import Ruby from 'tree-sitter-ruby';
import Dart from 'tree-sitter-dart';
import { requireVendoredGrammar } from '../../src/core/tree-sitter/vendored-grammars.js';
// Vendored grammars — loaded from vendor/ by absolute path, never node_modules (#2111).
const C = requireVendoredGrammar('tree-sitter-c');
const Dart = requireVendoredGrammar('tree-sitter-dart');
let Kotlin: unknown;
try {
Kotlin = require('tree-sitter-kotlin');
Kotlin = requireVendoredGrammar('tree-sitter-kotlin');
} catch {
// Kotlin grammar may not be installed
// Kotlin grammar may not have a prebuild for this platform
}
const parser = new Parser();

View file

@ -0,0 +1,131 @@
import { describe, it, expect } from 'vitest';
import { readdirSync, readFileSync, existsSync, writeFileSync, rmSync } from 'node:fs';
import path from 'node:path';
import { fileURLToPath } from 'node:url';
import {
VENDOR_ROOT,
VENDORED_GRAMMAR_PACKAGES,
vendoredGrammarDir,
requireVendoredGrammar,
} from '../../src/core/tree-sitter/vendored-grammars.js';
/**
* Regression guard for #2111 / #1728.
*
* The five vendored tree-sitter grammars (c/dart/proto/swift/kotlin) MUST load
* from `vendor/` by absolute path and MUST NEVER be copied into / required from
* `node_modules`. An undeclared package under node_modules is "extraneous" to
* every subsequent npm/npx arborist reify, which prunes/relocates it on
* Windows that threw `EPERM: operation not permitted, symlink` during the
* npx-cache reify an MCP client triggers, and on every OS it silently deleted
* the grammars on the 2nd run. These tests fail if anyone reintroduces a bare
* `require('tree-sitter-<vendored>')` / `import … from 'tree-sitter-<vendored>'`
* (which would force a node_modules copy back into existence).
*/
const SRC_ROOT = fileURLToPath(new URL('../../src', import.meta.url));
const TEST_ROOT = fileURLToPath(new URL('..', import.meta.url));
/**
* All `.ts` files we police: every file under src/, plus test/ EXCEPT
* test/fixtures/ (fixtures are arbitrary sample code to be analyzed, not our
* code). A reintroduced bare load can defeat the fix from test/ too, so the
* guard must cover it not just src/.
*/
function policedFiles(): string[] {
const self = fileURLToPath(import.meta.url);
const under = (root: string) =>
readdirSync(root, { recursive: true, encoding: 'utf8' })
.filter((p) => p.endsWith('.ts'))
.map((p) => path.join(root, p));
return [
...under(SRC_ROOT),
...under(TEST_ROOT).filter((p) => !p.includes(`${path.sep}fixtures${path.sep}`)),
// This guard file itself holds the bad-load patterns as regex-probe fixtures.
].filter((p) => p !== self);
}
/**
* A bare ESM/CJS load of a vendored grammar package in real code. Covers every
* node_modules-forcing form static `import … from`, side-effect `import 'x'`,
* dynamic `import('x')`, `require('x')`, `require.resolve('x')` with single,
* double, OR backtick quotes, and an optional `/subpath`. Skips matches inside a
* leading-`//` or `*` comment (several query.ts files mention the bad pattern in
* prose, e.g. "`import Dart from 'tree-sitter-dart'` would throw"). Biased toward
* over-matching: a missed real load defeats the guard, a flagged trailing-comment
* mention only costs a glance.
*/
function bareVendoredLoadLines(file: string): string[] {
const names = [...VENDORED_GRAMMAR_PACKAGES].join('|');
// prefix: `from`, `import(`, `import ` (side-effect), `require(`, `require.resolve(`
const re = new RegExp(
`(?:from|import|require\\(|require\\.resolve\\()\\s*\\(?\\s*['"\\\`](?:${names})(?:/[^'"\\\`]*)?['"\\\`]`,
);
const hits: string[] = [];
for (const raw of readFileSync(file, 'utf8').split('\n')) {
const m = re.exec(raw);
if (!m) continue;
const trimmed = raw.trimStart();
const isComment = trimmed.startsWith('//') || trimmed.startsWith('*');
if (!isComment) hits.push(`${path.basename(file)}: ${raw.trim()}`);
}
return hits;
}
describe('vendored grammars load from vendor/ (#2111)', () => {
it('resolves every vendored grammar to a real dir under vendor/, never node_modules', () => {
expect(VENDOR_ROOT.endsWith(`${path.sep}vendor`)).toBe(true);
for (const pkg of VENDORED_GRAMMAR_PACKAGES) {
const dir = vendoredGrammarDir(pkg);
expect(dir.startsWith(VENDOR_ROOT)).toBe(true);
expect(dir.includes(`${path.sep}node_modules${path.sep}`)).toBe(false);
expect(existsSync(dir), `${pkg} missing under vendor/`).toBe(true);
}
});
it('loads each vendored grammar by absolute path (committed prebuild, no node_modules copy)', () => {
for (const pkg of VENDORED_GRAMMAR_PACKAGES) {
const grammar = requireVendoredGrammar(pkg);
expect(grammar, `${pkg} failed to load from vendor/`).toBeTruthy();
}
});
it('no src/test file bare-imports/requires a vendored grammar (would force a node_modules copy back)', () => {
const offenders = policedFiles().flatMap(bareVendoredLoadLines);
expect(
offenders,
`Use requireVendoredGrammar(...) instead of a bare specifier:\n${offenders.join('\n')}`,
).toEqual([]);
});
it('guard regex catches every node_modules-forcing load form (and ignores prose mentions)', () => {
// Sanity-check the guard itself so the adversarial bypasses (#2144 review)
// stay closed: static/side-effect/dynamic/subpath/backtick all flagged,
// requireVendoredGrammar + leading-comment prose ignored.
const tmp = path.join(fileURLToPath(new URL('.', import.meta.url)), `__guard_probe__.ts.txt`);
const caught = [
`import C from 'tree-sitter-c';`,
`import 'tree-sitter-dart';`,
`await import('tree-sitter-kotlin');`,
`const x = require('tree-sitter-swift');`,
`require.resolve('tree-sitter-proto');`,
'const y = require(`tree-sitter-c`);',
`import Node from 'tree-sitter-c/bindings/node';`,
];
const ignored = [
`// import C from 'tree-sitter-c' would throw`,
` * mentions 'tree-sitter-dart' in a block comment`,
`requireVendoredGrammar('tree-sitter-c');`,
`import Cpp from 'tree-sitter-cpp';`, // not a vendored grammar
];
writeFileSync(tmp, [...caught, ...ignored].join('\n'));
try {
const flagged = bareVendoredLoadLines(tmp).map((l) => l.split(': ').slice(1).join(': '));
for (const c of caught) expect(flagged, `should flag: ${c}`).toContain(c);
for (const i of ignored) expect(flagged, `should NOT flag: ${i}`).not.toContain(i);
} finally {
rmSync(tmp, { force: true });
}
});
});

View file

@ -6,7 +6,7 @@
"license": "MIT",
"main": "bindings/node/index.js",
"types": "bindings/node/index.d.ts",
"_vendoredBy": "gitnexus - runtime package derived from tree-sitter-c@0.21.4 (tree-sitter/tree-sitter-c). HELD at 0.21.4 for ABI compatibility with the bundled tree-sitter@0.21.1 runtime (#1242/#858) — do not bump without the runtime upgrade. Vendored because upstream ships native prebuilds for only 4 of 6 platforms (no linux-arm64/win32-arm64, #2116), and tree-sitter-c is a REQUIRED grammar whose source build hard-fails `npm install` on a toolchain-less ARM host. GitNexus cross-builds all six prebuilds via .github/workflows/build-tree-sitter-prebuilds.yml; the C source (binding.gyp + src/) is ALSO vendored so build-tree-sitter-c.cjs can source-build the binding on a toolchain host when no prebuild matches (e.g. CI before prebuilds land). Copied to node_modules/ by materialize-vendor-grammars.cjs (no scripts.install here — #836/#1728).",
"_vendoredBy": "gitnexus - runtime package derived from tree-sitter-c@0.21.4 (tree-sitter/tree-sitter-c). HELD at 0.21.4 for ABI compatibility with the bundled tree-sitter@0.21.1 runtime (#1242/#858) — do not bump without the runtime upgrade. Vendored because upstream ships native prebuilds for only 4 of 6 platforms (no linux-arm64/win32-arm64, #2116), and tree-sitter-c is a REQUIRED grammar whose source build hard-fails `npm install` on a toolchain-less ARM host. GitNexus cross-builds all six prebuilds via .github/workflows/build-tree-sitter-prebuilds.yml; the C source (binding.gyp + src/) is ALSO vendored so build-tree-sitter-grammars.cjs can source-build the binding on a toolchain host when no prebuild matches (e.g. CI before prebuilds land). Loaded from vendor/ by absolute path at runtime (vendored-grammars.ts) — NEVER copied to node_modules (#2111) (no scripts.install here — #836/#1728).",
"peerDependencies": {
"tree-sitter": "^0.21.0"
},

View file

@ -6,7 +6,7 @@
"license": "ISC",
"main": "bindings/node",
"types": "bindings/node",
"_vendoredBy": "gitnexus - pinned to UserNobody14/tree-sitter-dart commit 80e23c07b64494f7e21090bb3450223ef0b192f4. Copied to node_modules/ by materialize-vendor-grammars.cjs; native build via build-tree-sitter-dart.cjs (#1728, #836).",
"_vendoredBy": "gitnexus - pinned to UserNobody14/tree-sitter-dart commit 80e23c07b64494f7e21090bb3450223ef0b192f4. Loaded from vendor/ by absolute path at runtime (vendored-grammars.ts) — NEVER copied to node_modules (#2111); native build via build-tree-sitter-grammars.cjs (#1728, #836).",
"peerDependencies": {
"tree-sitter": "^0.21.0"
},

View file

@ -6,7 +6,7 @@
"license": "MIT",
"main": "bindings/node/index.js",
"types": "bindings/node/index.d.ts",
"_vendoredBy": "gitnexus - runtime package derived from tree-sitter-kotlin@0.3.8 (fwcd). Unlike Swift's upstream-shipped prebuilds, upstream tree-sitter-kotlin ships SOURCE ONLY (no prebuilds/); the native prebuilds/ here are GitNexus-cross-built by .github/workflows/build-tree-sitter-prebuilds.yml. The grammar source (parser.c/scanner.c/binding.gyp + src/) is ALSO vendored so build-tree-sitter-kotlin.cjs can source-build the binding on a toolchain host when no prebuild matches (e.g. CI before prebuilds land). The generated parser.c is large (~23 MB on disk; it compresses heavily in git); once the prebuilds cover every platform-arch the source serves only as the fallback. Copied to node_modules/ by materialize-vendor-grammars.cjs (no scripts.install here — #836/#1728).",
"_vendoredBy": "gitnexus - runtime package derived from tree-sitter-kotlin@0.3.8 (fwcd). Unlike Swift's upstream-shipped prebuilds, upstream tree-sitter-kotlin ships SOURCE ONLY (no prebuilds/); the native prebuilds/ here are GitNexus-cross-built by .github/workflows/build-tree-sitter-prebuilds.yml. The grammar source (parser.c/scanner.c/binding.gyp + src/) is ALSO vendored so build-tree-sitter-grammars.cjs can source-build the binding on a toolchain host when no prebuild matches (e.g. CI before prebuilds land). The generated parser.c is large (~23 MB on disk; it compresses heavily in git); once the prebuilds cover every platform-arch the source serves only as the fallback. Loaded from vendor/ by absolute path at runtime (vendored-grammars.ts) — NEVER copied to node_modules (#2111) (no scripts.install here — #836/#1728).",
"peerDependencies": {
"tree-sitter": "^0.21.0"
},

View file

@ -5,7 +5,7 @@
"repository": "https://github.com/coder3101/tree-sitter-proto",
"license": "MIT",
"main": "bindings/node",
"_vendoredBy": "gitnexus — materialized to node_modules/ by materialize-vendor-grammars.cjs; native build via build-tree-sitter-proto.cjs. Do NOT re-add dependencies or an install script (#836, #1728).",
"_vendoredBy": "gitnexus — loaded from vendor/ by absolute path at runtime (vendored-grammars.ts) — NEVER copied to node_modules (#2111); native build via build-tree-sitter-grammars.cjs. Do NOT re-add dependencies or an install script (#836, #1728).",
"peerDependencies": {
"tree-sitter": ">=0.21.0"
}

View file

@ -9,7 +9,7 @@
"type": "git",
"url": "git+https://github.com/alex-pinkus/tree-sitter-swift.git"
},
"_vendoredBy": "gitnexus - runtime package derived from official tree-sitter-swift@0.7.1 (gitHead 88bfd19a89be9d0481b14566fb6160cccea2fe0a). Unified with Dart/Proto/Kotlin/C: the grammar source (parser.c/scanner.c/binding.gyp + src/) is ALSO vendored so build-tree-sitter-grammars.cjs can source-build the binding on a toolchain host when no prebuild matches (e.g. CI before prebuilds land); src/parser.c is the ABI-14 default (~18 MB on disk, compresses heavily in git — the upstream parser_abi13.c alternate is not vendored). The native prebuilds/ are GitNexus-cross-built by .github/workflows/build-tree-sitter-prebuilds.yml (originally upstream-shipped). Build activation runs via gitnexus/scripts/build-tree-sitter-grammars.cjs after materialize-vendor-grammars.cjs (no scripts.install here — avoids #836 / #1728).",
"_vendoredBy": "gitnexus - runtime package derived from official tree-sitter-swift@0.7.1 (gitHead 88bfd19a89be9d0481b14566fb6160cccea2fe0a). Unified with Dart/Proto/Kotlin/C: the grammar source (parser.c/scanner.c/binding.gyp + src/) is ALSO vendored so build-tree-sitter-grammars.cjs can source-build the binding on a toolchain host when no prebuild matches (e.g. CI before prebuilds land); src/parser.c is the ABI-14 default (~18 MB on disk, compresses heavily in git — the upstream parser_abi13.c alternate is not vendored). The native prebuilds/ are GitNexus-cross-built by .github/workflows/build-tree-sitter-prebuilds.yml (originally upstream-shipped). Build activation runs via gitnexus/scripts/build-tree-sitter-grammars.cjs (no scripts.install here — avoids #836 / #1728).",
"peerDependencies": {
"tree-sitter": "^0.21.1 || ^0.22.1"
},