mirror of
https://github.com/abhigyanpatwari/GitNexus.git
synced 2026-08-28 05:25:25 +00:00
* fix(deps): pin tree-sitter-c/cpp to fix Windows segfault (#1242) `tree-sitter-c@0.23.2` ships native prebuilds compiled against tree-sitter ABI 14 (tree-sitter-cli >=0.24), while GitNexus is pinned to the tree-sitter@0.21.1 JS runtime. On Windows the JS runtime hits `Cannot read properties of undefined (reading '161')` inside `unmarshalNode` and a native segfault in the parse-worker pipeline on real C codebases (e.g. STM32 headers from the issue reporter). Two coordinated registry pins fix the root cause without any override gymnastics or vendoring: - `tree-sitter-c` -> `0.21.4` (last release built against the tree-sitter@0.21 ABI; declared peer `^0.21.0`). - `tree-sitter-cpp` -> `0.23.2` (last 0.23.x release before tree-sitter-cpp added a runtime dep on the broken-ABI `tree-sitter-c@^0.23.1`; pinning here lets us drop the previous global override entirely). `npm ls tree-sitter-c` is now clean: single deduped 0.21.4, no `overridden` annotations, no nested copy. Parser loader collapsed to one declarative table: - One `SOURCES` map with `{ load, unavailableNote, optional? }` rows for every grammar including TSX. Adding/removing a grammar is one entry; `unavailableNote` is mandatory and the type checker enforces it, so failures are never silent and never generic. - Single `loadGrammar(key)` does lazy require + cache + per-failure classification. Required failures `console.error` the note and rethrow the original (preserves stack); optional failures `console.warn` and report the language as Unsupported. One warn-once `Set` deduplicates per language key. - The previous bespoke `warnCUnavailable` + `cWarningEmitted` state and 4 conditional spreads in the language map are gone. Per-grammar `unavailableNote` strings name the package, list the most likely failure mode for that grammar, and link the relevant tracking issue (#1013, #1125, #1130, #1242) where applicable. Tests: new `C parser ABI compatibility (#1242)` block under parser-loader.test.ts exercises the actual failure paths (non-trivial parse + tree walk + Query.captures + TreeCursor descent). The original report's `unmarshalNode` crash sits on exactly the traversal hot path these tests now cover. Validation: - npx tsc --noEmit: clean - npx vitest run test/unit: 4808 passed, 10 skipped - npx vitest run test/integration/resolvers/cpp.test.ts: 133/133 - minimal C parse + walk + query + cursor verified manually under tree-sitter@0.21.1 + tree-sitter-c@0.21.4 on Win11 x64 / Node 22 Closes #1242. Does not unblock the broader tree-sitter@0.25 upgrade tracked in #858. Made-with: Cursor * chore(ci): redesign tree-sitter upgrade-readiness report (#858) The daily script that owns the body of #858 used to dump one giant matrix and leave a human to figure out which grammars are actually ready to bump. After pinning `tree-sitter-c@0.21.4` and `tree-sitter-cpp@0.23.2` for #1242, several rows in that matrix now look like regressions when in fact they are deliberate. The report now classifies each grammar instead of just listing them. What changed in `check-tree-sitter-upgrade-readiness.py`: - New `INTENTIONAL_PINS` table documents grammars deliberately held below `npm latest`, with a one-line rationale and a tracking issue per row (#1242 for C and C++, #1013 for C#). The script reads pins straight from `gitnexus/package.json` so a future bump cannot drift away from this report. - New `_classify_grammar(...)` produces one primary disposition per grammar: Ready for 0.25 / Intentionally pinned / Waiting on upstream npm release / Blocked on upstream / Could not check. The dispositions drive the report layout. - New `vendored_drift_summary(...)` covers all three vendored parsers (`tree-sitter-proto`, `tree-sitter-dart`, `tree-sitter-swift`) uniformly: ABI from `parser.c` when present, upstream npm + GitHub status, and the rationale extracted from each vendor's `_vendoredBy` field. Prebuilt-only vendors (Swift today) report `ABI 'prebuilt'` instead of `None`. - Report layout: top-of-page TL;DR + counts, an actionable "What you can do today" section, then one section per disposition bucket, then a dedicated "Vendored parsers" section. The original raw matrix is preserved inside a collapsible `<details>` block so the row-diff bot that watches this issue still has stable input. - `sys.stdout.reconfigure(encoding="utf-8")` so the workflow no longer crashes on Windows when the report contains arrows or em-dashes. No workflow / cron changes; the daily job posts the new body the next time it runs. #858 itself was updated by hand in the meantime to keep the tracker readable. Made-with: Cursor * fix(parser-loader): log C grammar load failures at error severity (#1242) Addresses review feedback on #1243. `tree-sitter-c` is in `dependencies` (not `optionalDependencies`) so a load failure on a supported platform always indicates a real install problem the user needs to see — corrupted node_modules, unsupported Node version, or an ABI mismatch with the bundled runtime. Previously the optional-grammar machinery downgraded that to `console.warn`, which can be missed in long log streams and silently drops C analysis for an entire repo. Decouples log severity from throw behavior: - `GrammarSource.severity?: 'warn' | 'error'` is a new optional field that overrides the default log level for a load failure. Default is `error` for required grammars and `warn` for optional ones, matching the prior behavior for every existing row. - `LoadResult` carries the resolved severity through `loadGrammar` so `logFailure` no longer derives it from `fatal`. - `tree-sitter-c` row sets `optional: true, severity: 'error'`. The pipeline still degrades gracefully (callers see Unsupported instead of a thrown error), but the diagnostic is loud and the `unavailableNote` now spells out what to try first (`npm rebuild tree-sitter-c`, reinstall) and links the tracker. No test changes needed: `parser-loader.test.ts` exercises behavior on the success path and on optional-failure dispatch; severity is a display-only concern routed through `console.error` vs `console.warn`, which the existing tests don't assert on. Made-with: Cursor * fix(ci): treat intentional pins as 0.25 blockers in readiness report Addresses review feedback on #1243. `_classify_grammar` returned bucket `intentional` before checking `target_compat`, and the per-grammar status loop only added a row to `blockers` when npm-latest was incompatible with the target runtime. The combination meant: if every other grammar resolved tomorrow but we were still holding `tree-sitter-c@0.21.4` and `tree-sitter-cpp@0.23.2` (both incompatible with `tree-sitter@0.25.x`), the script would emit "**Ready** — all grammars are 0.25-compatible" and mislead maintainers into thinking the runtime upgrade was unblocked. Fix: - The status loop now adds an entry to `blockers` whenever a grammar is in `INTENTIONAL_PINS`, regardless of npm-latest's peer dep. The blocker message names the pinned spec, embeds the rationale from `INTENTIONAL_PINS`, and tells the reader the pin must be lifted before the target runtime upgrade. When the pin is removed (entry deleted from `INTENTIONAL_PINS`), the grammar resumes standard classification on the next run. - `bump_now` now excludes intentional pins so they never show up in the "What you can do today" section. Bumping an intentional pin requires a deliberate edit to both `INTENTIONAL_PINS` and `package.json`, not a one-line dependency bump. Verified locally: TL;DR now reports 8 blockers (6 upstream + 2 intentional) where it previously reported 6, and the verdict correctly remains **Blocked** even in the hypothetical future where all upstream blockers clear. Made-with: Cursor
169 lines
6 KiB
TypeScript
169 lines
6 KiB
TypeScript
import { describe, it, expect } from 'vitest';
|
|
import { loadParser, loadLanguage } from '../../src/core/tree-sitter/parser-loader.js';
|
|
import { SupportedLanguages } from '../../src/config/supported-languages.js';
|
|
|
|
describe('parser-loader', () => {
|
|
describe('loadParser', () => {
|
|
it('returns a Parser instance', async () => {
|
|
const parser = await loadParser();
|
|
expect(parser).toBeDefined();
|
|
expect(typeof parser.parse).toBe('function');
|
|
});
|
|
|
|
it('returns the same singleton instance', async () => {
|
|
const parser1 = await loadParser();
|
|
const parser2 = await loadParser();
|
|
expect(parser1).toBe(parser2);
|
|
});
|
|
});
|
|
|
|
describe('loadLanguage', () => {
|
|
it('loads TypeScript language', async () => {
|
|
await expect(loadLanguage(SupportedLanguages.TypeScript)).resolves.not.toThrow();
|
|
});
|
|
|
|
it('loads JavaScript language', async () => {
|
|
await expect(loadLanguage(SupportedLanguages.JavaScript)).resolves.not.toThrow();
|
|
});
|
|
|
|
it('loads Python language', async () => {
|
|
await expect(loadLanguage(SupportedLanguages.Python)).resolves.not.toThrow();
|
|
});
|
|
|
|
it('loads Java language', async () => {
|
|
await expect(loadLanguage(SupportedLanguages.Java)).resolves.not.toThrow();
|
|
});
|
|
|
|
it('loads C language', async () => {
|
|
await expect(loadLanguage(SupportedLanguages.C)).resolves.not.toThrow();
|
|
});
|
|
|
|
it('loads C++ language', async () => {
|
|
await expect(loadLanguage(SupportedLanguages.CPlusPlus)).resolves.not.toThrow();
|
|
});
|
|
|
|
it('loads C# language', async () => {
|
|
await expect(loadLanguage(SupportedLanguages.CSharp)).resolves.not.toThrow();
|
|
});
|
|
|
|
it('loads Go language', async () => {
|
|
await expect(loadLanguage(SupportedLanguages.Go)).resolves.not.toThrow();
|
|
});
|
|
|
|
it('loads Rust language', async () => {
|
|
await expect(loadLanguage(SupportedLanguages.Rust)).resolves.not.toThrow();
|
|
});
|
|
|
|
it('loads PHP language', async () => {
|
|
await expect(loadLanguage(SupportedLanguages.PHP)).resolves.not.toThrow();
|
|
});
|
|
|
|
it('loads TSX grammar for .tsx files', async () => {
|
|
// TSX uses a different grammar (TypeScript.tsx vs TypeScript.typescript)
|
|
await expect(
|
|
loadLanguage(SupportedLanguages.TypeScript, 'Component.tsx'),
|
|
).resolves.not.toThrow();
|
|
});
|
|
|
|
it('loads TS grammar for .ts files', async () => {
|
|
await expect(loadLanguage(SupportedLanguages.TypeScript, 'utils.ts')).resolves.not.toThrow();
|
|
});
|
|
|
|
it('loads Ruby language', async () => {
|
|
await expect(loadLanguage(SupportedLanguages.Ruby)).resolves.not.toThrow();
|
|
});
|
|
|
|
it('throws for unsupported language', async () => {
|
|
await expect(loadLanguage('erlang' as SupportedLanguages)).rejects.toThrow(
|
|
'Unsupported language',
|
|
);
|
|
});
|
|
});
|
|
|
|
// #1242: regression coverage for the Windows tree-sitter@0.21.1 + tree-sitter-c
|
|
// ABI mismatch. setLanguage alone could pass while the first non-trivial
|
|
// traversal/query produced "Cannot read properties of undefined (reading
|
|
// '161')" inside unmarshalNode (or a native segfault under the worker).
|
|
describe('C parser ABI compatibility (#1242)', () => {
|
|
const C_SOURCE = `#include <stdio.h>
|
|
struct Foo { int a; int b; };
|
|
typedef struct Foo Bar;
|
|
static int helper(int x) { return x * 2; }
|
|
int add(int a, int b) { return a + b; }
|
|
int main(void) {
|
|
Bar b = {1, 2};
|
|
return add(b.a, helper(b.b));
|
|
}
|
|
`;
|
|
|
|
it('parses a non-trivial C translation unit and walks the tree', async () => {
|
|
const parser = await loadParser();
|
|
await loadLanguage(SupportedLanguages.C);
|
|
const tree = parser.parse(C_SOURCE);
|
|
|
|
expect(tree.rootNode.type).toBe('translation_unit');
|
|
|
|
let nodeCount = 0;
|
|
const walk = (node: { type: string; children: any[] }): void => {
|
|
nodeCount += 1;
|
|
// Touching `.type` here is what triggered the original
|
|
// unmarshalNode crash on incompatible ABIs.
|
|
expect(typeof node.type).toBe('string');
|
|
for (const child of node.children) walk(child);
|
|
};
|
|
walk(tree.rootNode as any);
|
|
expect(nodeCount).toBeGreaterThan(20);
|
|
});
|
|
|
|
it('extracts function definitions and call expressions via a query', async () => {
|
|
const Parser = (await import('tree-sitter')).default;
|
|
const parser = await loadParser();
|
|
await loadLanguage(SupportedLanguages.C);
|
|
const tree = parser.parse(C_SOURCE);
|
|
const language = parser.getLanguage();
|
|
|
|
const query = new (Parser as any).Query(
|
|
language,
|
|
'(function_definition declarator: (function_declarator declarator: (identifier) @name)) ' +
|
|
'(call_expression function: (identifier) @callee)',
|
|
);
|
|
const captures = query.captures(tree.rootNode);
|
|
const names = captures.filter((c: any) => c.name === 'name').map((c: any) => c.node.text);
|
|
const callees = captures.filter((c: any) => c.name === 'callee').map((c: any) => c.node.text);
|
|
|
|
expect(names).toEqual(expect.arrayContaining(['helper', 'add', 'main']));
|
|
expect(callees).toEqual(expect.arrayContaining(['add', 'helper']));
|
|
});
|
|
|
|
it('walks a TreeCursor without throwing (catches unmarshalNode regressions)', async () => {
|
|
const parser = await loadParser();
|
|
await loadLanguage(SupportedLanguages.C);
|
|
const tree = parser.parse(C_SOURCE);
|
|
const cursor = tree.walk();
|
|
let visited = 0;
|
|
const descend = (): void => {
|
|
visited += 1;
|
|
if (cursor.gotoFirstChild()) {
|
|
do {
|
|
descend();
|
|
} while (cursor.gotoNextSibling());
|
|
cursor.gotoParent();
|
|
}
|
|
};
|
|
descend();
|
|
expect(visited).toBeGreaterThan(20);
|
|
});
|
|
});
|
|
|
|
describe('Swift optional dependency', () => {
|
|
it('loads Swift from the default optional dependency and parses source', async () => {
|
|
const parser = await loadParser();
|
|
await loadLanguage(SupportedLanguages.Swift);
|
|
|
|
const tree = parser.parse('class Foo { func bar() {} }');
|
|
|
|
expect(tree.rootNode.type).toBe('source_file');
|
|
expect(tree.rootNode.namedChildCount).toBe(1);
|
|
});
|
|
});
|
|
});
|