GitNexus/gitnexus/test/unit/assert-publish-grammar-coverage.test.ts
Gergő Magyar 2870aa6248
fix(grammars): load vendored tree-sitter grammars from vendor/ by absolute path (#2111) (#2144)
* fix(grammars): load vendored tree-sitter grammars from vendor/ by absolute path (#2111)

The recurring Windows `EPERM: operation not permitted, symlink` (errno -4048)
when adding the MCP server to Antigravity is NOT the #2101/#2110 module-load
crash — it is an install-time arborist failure during the `_npx` reify that the
MCP client triggers on every `npx gitnexus` launch.

Root cause: the `postinstall` materialize step copied each vendored grammar
(`vendor/tree-sitter-{c,dart,proto,swift,kotlin}`) into
`node_modules/gitnexus/node_modules/tree-sitter-*` as a real package so runtime
`require('tree-sitter-dart')` would resolve. Those packages are in no dependency
graph, so every subsequent npm/npx reify treats them as **extraneous** and
prunes/relocates them — on Windows the relocation goes through
`@npmcli/move-file`'s symlink path and throws EPERM (symlinks need Developer
Mode/admin), and on every OS the 2nd run silently deletes the grammars. This is
the same class as #1728, which the materialize step itself claimed to have
fixed.

Fix (the prebuildify + node-gyp-build ecosystem pattern): never copy grammars
into node_modules. Load each by absolute path from `vendor/<name>` via the new
`requireVendoredGrammar` helper — the grammar's own `bindings/node` runs
`node-gyp-build(<dir>)` and loads the committed `vendor/<name>/prebuilds/
<platform>-<arch>/…` directly (all 5 ship all 6 tuples). vendor/ is inside the
package but not a node_modules subtree, so arborist never sees the grammars and
the reify is idempotent — no EPERM, no silent deletion.

- new src/core/tree-sitter/vendored-grammars.ts (requireVendoredGrammar /
  vendoredGrammarDir / VENDORED_GRAMMAR_PACKAGES; VENDOR_ROOT stable in dev+dist)
- route all consumers through it: parser-loader, parse-worker, grpc proto,
  include-extractor (C), http-patterns kotlin, cli optional-grammars probe
- postinstall drops the materialize step; build-tree-sitter-grammars.cjs builds
  in-place under vendor/ (gitignored) and deletes materialize-vendor-grammars.cjs
- tests + grammar-introspection helper load grammars from vendor/ too (single
  source of truth); new vendored-grammars.test.ts guards against reintroducing a
  bare `require('tree-sitter-<vendored>')`

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(grammars): throw on a non-vendored name in requireVendoredGrammar

Drift guard (PR #2144 review, P3): validate the argument against
VENDORED_GRAMMAR_PACKAGES and fail loudly on an unknown name, so the three
grammar lists (package set / CLI probe / build registry) drifting out of sync
surfaces as a clear error instead of a confusing absolute-path require miss.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(grammars): prepack guard against stray vendor/<g>/build/ shadowing prebuilds

Publish hygiene (PR #2144 review, P2). Now that build-tree-sitter-grammars.cjs
source-builds into vendor/<name>/build/, a stray build dir would ship in the
tarball (files:["vendor"] overrides .gitignore/.npmignore) AND shadow the
committed prebuild — node-gyp-build resolves build/Release before prebuilds/.
assert-publish-grammar-coverage.cjs (prepack) now fails `npm pack` if any
vendor/*/build exists (findStrayBuildArtifacts), with a clear `rm -rf` fix hint.
Adds unit coverage for the new pure function.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(grammars): harden the #2111 no-bare-require regression guard

PR #2144 review (P2). The guard regex missed dynamic import(), side-effect
`import 'x'`, /subpath, and backtick loads, and only scanned src/. It now covers
every node_modules-forcing form (single/double/backtick quotes, optional
subpath), scans test/ too (excluding fixtures and the guard file itself), drops
the `//`-substring false-negative (leading-comment-only heuristic), and adds a
self-test asserting every load form is caught while prose mentions and
tree-sitter-cpp are ignored.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(grammars): correct stale vendored-grammar comments

PR #2144 review (P3). kotlin/query.ts called tree-sitter-kotlin an
"optionalDependency" — it is vendored and loaded from vendor/ by absolute path
(#2111). proto.ts now states its remaining `_require` is only for the real
`tree-sitter` dependency, not a vendored grammar (which goes through
requireVendoredGrammar). Comment-only; no behavior change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-10 14:20:42 +01:00

124 lines
5.3 KiB
TypeScript

import { describe, it, expect } from 'vitest';
import { spawnSync } from 'node:child_process';
import { createRequire } from 'node:module';
import { fileURLToPath } from 'node:url';
import { mkdtempSync, mkdirSync, rmSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import path from 'node:path';
/**
* Coverage for the publish guard `scripts/assert-publish-grammar-coverage.cjs`.
*
* The guard refuses to pack/publish if a vendored grammar would ship with no
* loadable binding — i.e. the package.json `files` field was narrowed to drop the
* vendored source while a grammar still lacks 6/6 prebuilds. (`.npmignore` can't
* exclude the vendored subtree — `files` overrides it — so `files` is the only
* lever, and the guard reads it directly rather than shelling out to `npm pack`.)
* We test the pure decision core + the `files` check directly, and assert the real
* repo state is publish-safe (catching a premature narrowing in CI).
*/
const requireCjs = createRequire(import.meta.url);
const SCRIPT = fileURLToPath(
new URL('../../scripts/assert-publish-grammar-coverage.cjs', import.meta.url),
);
const { findCoverageProblems, filesShipsVendorSource, findStrayBuildArtifacts } =
requireCjs(SCRIPT);
describe('findCoverageProblems (pure decision core)', () => {
it('passes when source ships, even with incomplete prebuilds (transitional state)', () => {
const grammars = [{ name: 'tree-sitter-kotlin', prebuilt: 0, shipsSource: true }];
expect(findCoverageProblems({ grammars })).toEqual([]);
});
it('fails when source is not shipped and a grammar lacks 6/6 prebuilds', () => {
const grammars = [{ name: 'tree-sitter-kotlin', prebuilt: 4, shipsSource: false }];
const problems = findCoverageProblems({ grammars });
expect(problems).toHaveLength(1);
expect(problems[0]).toContain('tree-sitter-kotlin');
expect(problems[0]).toContain('not shipped');
expect(problems[0]).toContain('2 platform-arch tuple(s)');
});
it('passes when source is not shipped but every grammar has all 6 prebuilds', () => {
const grammars = [
{ name: 'tree-sitter-swift', prebuilt: 6, shipsSource: false },
{ name: 'tree-sitter-c', prebuilt: 6, shipsSource: false },
];
expect(findCoverageProblems({ grammars })).toEqual([]);
});
it('fails when a grammar has neither prebuilds nor shipped source', () => {
const grammars = [{ name: 'tree-sitter-x', prebuilt: 0, shipsSource: false }];
const problems = findCoverageProblems({ grammars });
expect(problems).toHaveLength(1);
expect(problems[0]).toContain('no loadable binding');
});
});
describe('filesShipsVendorSource', () => {
it('ships when a broad vendor entry is present', () => {
expect(filesShipsVendorSource(['dist', 'vendor', 'web'])).toBe(true);
expect(filesShipsVendorSource(['vendor/'])).toBe(true);
expect(filesShipsVendorSource(['vendor/**'])).toBe(true);
expect(filesShipsVendorSource(['vendor/*'])).toBe(true);
});
it('does NOT ship when files is narrowed to non-source subpaths (lean publish)', () => {
expect(
filesShipsVendorSource([
'dist',
'vendor/**/prebuilds/**',
'vendor/**/package.json',
'vendor/**/bindings/node/index.js',
]),
).toBe(false);
expect(filesShipsVendorSource([])).toBe(false);
expect(filesShipsVendorSource(undefined)).toBe(false);
});
});
describe('findStrayBuildArtifacts (stray vendor build dirs that would ship + shadow prebuilds)', () => {
const mkVendor = (): string => mkdtempSync(path.join(tmpdir(), 'vguard-'));
it('returns [] when no grammar has a build/ dir', () => {
const dir = mkVendor();
try {
mkdirSync(path.join(dir, 'tree-sitter-y', 'prebuilds', 'linux-x64'), { recursive: true });
writeFileSync(path.join(dir, 'tree-sitter-y', 'prebuilds', 'linux-x64', 'y.node'), '');
expect(findStrayBuildArtifacts(dir)).toEqual([]);
} finally {
rmSync(dir, { recursive: true, force: true });
}
});
it('flags a grammar that carries a stray build/ output (would shadow the prebuild)', () => {
const dir = mkVendor();
try {
mkdirSync(path.join(dir, 'tree-sitter-x', 'build', 'Release'), { recursive: true });
mkdirSync(path.join(dir, 'tree-sitter-y', 'prebuilds'), { recursive: true });
expect(findStrayBuildArtifacts(dir)).toEqual(['vendor/tree-sitter-x/build']);
} finally {
rmSync(dir, { recursive: true, force: true });
}
});
it('ignores non-grammar dirs and a missing vendor dir', () => {
const dir = mkVendor();
try {
mkdirSync(path.join(dir, 'leiden', 'build'), { recursive: true }); // not tree-sitter-*
expect(findStrayBuildArtifacts(dir)).toEqual([]);
} finally {
rmSync(dir, { recursive: true, force: true });
}
expect(findStrayBuildArtifacts(path.join(tmpdir(), 'vguard-does-not-exist'))).toEqual([]);
});
});
describe('real repo publish-safety (guards against premature files narrowing)', () => {
it('the script exits 0 against the committed repo state', () => {
// Deterministic: reads package.json + walks vendor/ — no npm pack, fast.
const r = spawnSync(process.execPath, [SCRIPT], { encoding: 'utf8', timeout: 20_000 });
expect(r.status, r.stderr).toBe(0);
expect(r.stdout).toContain('[publish-guard] OK');
});
});