scriptoriumadmin/GitNexus
1
0
Fork 0
mirror of https://github.com/abhigyanpatwari/GitNexus.git synced 2026-10-11 03:38:07 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 7

fix(deps): upgrade native tree-sitter to 0.25.1 (#3531)

Browse source 
* fix(deps): upgrade native tree-sitter to 0.25.1

* fix(tree-sitter): preserve sync loading and update verified capture baselines

Route C# scope queries through the synchronous native grammar loader so the compiled analyze module remains require-compatible on Node 22.18. Refresh only the audited empty-Python-module and renamed-PHP-fixture captures, and pin the runtime assertion to 0.25.1.

Validated 479 targeted tests, all 16 scope-capture benchmark cases, Python capture/import guards, compiled C# worker parsing, build, typecheck and formatting. Performance budgets are unchanged.

* fix(tree-sitter): refresh upstream vendors and preserve parser semantics

* fix(ci): refresh prebuilds and support Windows ARM

Refresh macOS, Linux ARM, and Windows x64 grammar binaries from the successful native CI builds. Pin node-gyp 12.4.0 so the Windows ARM jobs recognize Visual Studio 2026.

Use local Git filtering for E2E to avoid PR-files API timeouts on generated parser diffs.

Validation: 143 focused tests, Actionlint, and all 16 scope-capture benchmark checks passed. Full npm test run started; native database suites still running.

* fix(ci): refresh Windows ARM grammar prebuilds

Import all seven Windows ARM grammar artifacts from successful native build run 37965761760. Artifact filenames and SHA-256 hashes verified; all 42 native build jobs passed.

Validation: 31 prebuild coverage tests passed.

* fix(bench): stabilize scope capture timing guard

Warm each input size seven times and use 21 median samples while retaining isolated timing blocks. Yield between languages so native finalizers release allocations outside the measured sections. Keep all fixture sizes, fingerprints, scaling formulas, and budgets unchanged.

Validation: all 16 language guards passed with a 2 GB heap limit (1.38 deep-chain ratio; 1518 MB peak RSS). Linear and quadratic controls passed their expected guard outcomes. Formatting and graph change analysis passed. Full npm test is running without failures so far.

* fix(parser): reduce TypeScript capture traversal cost

* fix(test): await parser worker readiness before teardown

* fix(parser): load C# synchronously in workers

* fix(grpc): resolve native proto detections against their own file

With the native proto grammar active, sibling .proto files that declare
the same short service name under different packages tied in the
directory-based resolveProtoConflict heuristic and both providers were
dropped. A proto-origin detection already knows its defining file, so
use that file's entry before falling back to the heuristic.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* ci(packaging): assert native grammars load from the installed tarball

The packaged-install smoke only ran `gitnexus --version`, so a grammar
that failed to load from the published package (bundled runtime, patched
peer manifests, vendored prebuilds, C# sync loader, Protobuf wrapper)
would ship silently; the dev-tree ABI smoke also accepts vendored
grammars that report unavailable. Parse a sample for every registered
grammar plus the native Protobuf plugin from the installed package, on
the main thread and in a worker thread.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* ci(tree-sitter): track compatibility workarounds instead of 0.25 readiness

The daily readiness report still targeted tree-sitter@0.25.0, so after the
upgrade it would keep reporting "Blocked" and never close its tracking issue.
Repurpose it to track what we carry for the bundled runtime: derive the target
from the package.json pin, list each pack-time peer patch from
prepare-tree-sitter-bundle.cjs with its drop condition (bump, wait for an
upstream release, transitive, or intentionally pinned), and reword the
buckets. The workflow adopts the issue opened under the old title.

Also update the grammar-update-monitor header and auto-PR text, which still
described the 0.21.1 runtime and a tree-sitter-c hold.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(dart): assert both nested labels target the loop; fix Zig call-shape doc

The nested-label CFG test only jumped to `outer` and checked reachability,
which still passes when a jump is retargeted to the inner loop (its exit
reaches the outer header). Jump to both labels and assert the exact
`continue` edges; dropping the inner label now fails the test.

Update the Zig callable-flow comment: call_expression now wraps actuals in a
required `arguments:` field.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Gergő Magyar 2026-10-10 13:07:59 +03:00 • committed by GitHub
parent d4e6c9422c
commit 999cc9f913
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
152 changed files with 1399580 additions and 1349795 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

8
.github/dependabot.yml vendored
View file

@ -52,11 +52,11 @@ updates:
- ci
# Gitnexus npm deps — tree-sitter grammars checked daily so we catch
# new releases that unblock the tree-sitter 0.25 upgrade ASAP. Grammars
# releases that let us drop a pack-time peer patch ASAP. Grammars
# are grouped so lockstep bumps produce a single PR. The tree-sitter
# RUNTIME is pinned — upgrade deliberately via the drift check workflow.
# See .github/scripts/check-tree-sitter-upgrade-readiness.py for
# the upgrade readiness tracker.
# RUNTIME is pinned — upgrade deliberately. See
# .github/scripts/check-tree-sitter-upgrade-readiness.py for the
# compatibility-workarounds tracker.
- package-ecosystem: npm
directory: /gitnexus
schedule:

126
.github/scripts/check-tree-sitter-upgrade-readiness.py vendored
View file

@ -1,13 +1,16 @@
#!/usr/bin/env python3
"""Monitor tree-sitter 0.25 upgrade readiness — two things Dependabot can't see:
"""Track the compatibility workarounds GitNexus carries for its bundled tree-sitter
runtime — things Dependabot can't see:
1. Peer-dep compatibility: when every grammar's *latest npm release* accepts
tree-sitter@0.25.0 (so we can upgrade without --legacy-peer-deps).
2. Vendored upstream drift: whether a vendored grammar's upstream parser.c moved.
1. Peer-dep patches: several grammars' npm peer ranges reject the bundled
runtime, so prepare-tree-sitter-bundle.cjs patches their manifests at pack
time (AUDITED_PEERS). The report flags when npm latest (or upstream main)
accepts the runtime, so a patch can be dropped by bumping the grammar.
2. Intentional pins and vendored holds, plus vendored upstream drift.
Invoked daily from tree-sitter-upgrade-readiness.yml; runs locally too. Outputs
Markdown to stdout; exit 1 when blockers remain (the workflow upserts a tracking
issue). stdlib-only — runs on any vanilla runner.
Markdown to stdout; exit 1 while any workaround remains (the workflow upserts a
tracking issue). stdlib-only — runs on any vanilla runner.
python3 .github/scripts/check-tree-sitter-upgrade-readiness.py [--offline | --assert-current]
"""
@ -31,10 +34,12 @@ GITNEXUS_DIR = REPO_ROOT / "gitnexus"
# vendored ABIs are still read from the repo. The read-path mirror of --assert-current.
OFFLINE = os.environ.get("GITNEXUS_TS_READINESS_OFFLINE", "") not in ("", "0", "false")
# ── Upgrade target ──────────────────────────────────────────────────────
# The runtime version we want to upgrade TO. Update this when the goal
# changes (e.g. once 0.25 lands and we target 0.26).
TARGET_RUNTIME = "0.25.0"
# ── Target runtime ──────────────────────────────────────────────────────
# The exact runtime GitNexus bundles, read from the package.json pin so the report
# can never go stale against an upgrade (a hardcoded target did after 0.25 landed).
TARGET_RUNTIME = json.loads((GITNEXUS_DIR / "package.json").read_text())["dependencies"][
"tree-sitter"
]
TARGET_RUNTIME_MAJOR_MINOR = ".".join(TARGET_RUNTIME.split(".")[:2])
# Tree-sitter runtime -> (min_abi, max_abi) it can load. Only the current
@ -168,6 +173,20 @@ def read_pinned_grammar_versions() -> dict[str, str]:
return pinned
def read_patched_peers() -> list[tuple[str, str, str]]:
"""Return the pack-time peer patches as (name, version, original peer range).
Parsed from AUDITED_PEERS in gitnexus/scripts/prepare-tree-sitter-bundle.cjs,
the single source of truth for which bundled manifests get the runtime added
to their tree-sitter peer range.
"""
script = GITNEXUS_DIR / "scripts" / "prepare-tree-sitter-bundle.cjs"
return re.findall(
r"\['(tree-sitter[\w-]*)@([\d.]+)',\s*'([^']+)'\]",
script.read_text(encoding="utf-8"),
)
def npm_view_json(pkg: str) -> dict | None:
"""Fetch package metadata from the npm registry via HTTPS.
@ -620,13 +639,13 @@ def _render_vendored_section(
elif v["upstream_abi"] and v["vendored_abi"] and v["upstream_abi"] > v["vendored_abi"]:
if v["upstream_abi"] <= target_abi_range[1]:
lines.append(
f" - **Action:** after upgrading to tree-sitter@{TARGET_RUNTIME}, "
f"regenerate `parser.c` from upstream `{v['upstream_sha']}`."
f" - **Action:** regenerate `parser.c` from upstream "
f"`{v['upstream_sha']}` (its ABI loads on tree-sitter@{TARGET_RUNTIME})."
)
else:
lines.append(
f" - **Action:** wait for a runtime supporting ABI "
f"{v['upstream_abi']}; current target ({TARGET_RUNTIME}) only "
f"{v['upstream_abi']}; tree-sitter@{TARGET_RUNTIME} only "
f"goes up to ABI {target_abi_range[1]}."
)
blockers[f"vendored-{v['name']}-abi"] = (
@ -647,7 +666,7 @@ def main() -> int:
# Label for npm/upstream values we couldn't determine: in --offline mode the
# fetch was deliberately skipped (not "failed"), so say so honestly.
miss_label = "offline" if OFFLINE else "fetch failed"
lines.append(md_h("Tree-sitter 0.25 upgrade readiness", 1))
lines.append(md_h("Tree-sitter compatibility workarounds", 1))
lines.append("")
if OFFLINE:
lines.append(
@ -663,8 +682,7 @@ def main() -> int:
pinned_versions = read_pinned_grammar_versions()
lines.append(
f"`tree-sitter@{current_runtime}.x` (ABI {current_abi_range[0]}–{current_abi_range[1]}) "
f"→ target `tree-sitter@{TARGET_RUNTIME}` "
f"Bundled runtime `tree-sitter@{TARGET_RUNTIME}` "
f"(ABI {target_abi_range[0]}–{target_abi_range[1]})."
)
lines.append("")
@ -674,7 +692,7 @@ def main() -> int:
# workflow's row-diff change-detection keeps working).
grammar_rows: list[dict] = []
raw_matrix: list[str] = [
"| Grammar | Pinned | npm latest | Peer dep | Satisfies 0.25? | ABI | Upstream ABI | Status |",
"| Grammar | Pinned | npm latest | Peer dep | Peer accepts runtime | ABI | Upstream ABI | Status |",
"|---|---|---|---|---|---|---|---|",
]
@ -710,8 +728,8 @@ def main() -> int:
f"vendored `{name}`: ABI {v['vendored_abi']} outside target range "
f"{target_abi_range[0]}..{target_abi_range[1]}"
)
# A held vendored grammar (e.g. tree-sitter-c, #1242/#858) is frozen below
# a runtime upgrade: in-range ABI or not, keep it a blocker until the hold
# A held vendored grammar (e.g. tree-sitter-swift) is a workaround we carry:
# in-range ABI or not, keep it a blocker until the hold
# (from the manifest) is lifted — same treatment as npm INTENTIONAL_PINS.
if hold:
v["target_compat"] = False
@ -802,19 +820,15 @@ def main() -> int:
# (entry removed from INTENTIONAL_PINS), then reclassified next run.
status = "Intentionally pinned"
blockers[name] = (
f"`{name}` intentionally pinned at `{pinned_spec}` "
f"({INTENTIONAL_PINS[name]}) — pin must be lifted "
f"before the {TARGET_RUNTIME} runtime upgrade"
f"`{name}` intentionally pinned at `{pinned_spec}` ({INTENTIONAL_PINS[name]})"
)
elif target_compat:
status = "Ready"
elif upstream_abi and upstream_abi >= 15:
status = "Unreleased (ABI 15 on main)"
upstream_progress = f"ABI 15 on `{upstream_repo}@{upstream_branch}` not yet published"
blockers[name] = f"`{name}`: ABI 15 on `{upstream_repo}` main but not published to npm"
else:
status = "Blocking"
blockers[name] = f"`{name}@{npm_version}`: peer `{peer_display}` incompatible with 0.25"
status = "Peer patch required"
blockers[name] = (
f"`{name}@{npm_version}`: peer `{peer_display}` rejects {TARGET_RUNTIME}"
)
# Also check upstream package.json for relaxed peer dep — beats
# the ABI-15 hint when both are true.
@ -874,11 +888,14 @@ def main() -> int:
vendored_ready = sum(1 for v in vendored_grammars if v["target_compat"])
if not blockers:
verdict = "**Ready** — all grammars are 0.25-compatible. The runtime upgrade can proceed."
verdict = (
f"**Clean** — every grammar runs on tree-sitter@{TARGET_RUNTIME} "
"without a workaround."
)
else:
moved = "no" if not by_bucket["waiting"] else f"yes — {len(by_bucket['waiting'])} grammars have unreleased fixes on main"
verdict = (
f"**Blocked** — {len(blockers)} grammars are not yet 0.25-compatible. "
f"**Workarounds** — {len(blockers)} grammars still need one. "
f"Upstream movement: {moved}."
)
@ -888,11 +905,11 @@ def main() -> int:
lines.append(f"- {ready_count}/{npm_count} npm-installed grammars already accept tree-sitter@{TARGET_RUNTIME}")
if vendored_count:
lines.append(
f"- {vendored_ready}/{vendored_count} vendored grammars at an ABI within the target runtime range"
f"- {vendored_ready}/{vendored_count} vendored grammars at an ABI within the runtime range"
)
lines.append(f"- {len(by_bucket['intentional'])} intentionally pinned (see below)")
lines.append(f"- {len(by_bucket['waiting'])} waiting on an upstream npm release")
lines.append(f"- {len(by_bucket['blocked'])} blocked on upstream (no fix even on main)")
lines.append(f"- {len(by_bucket['blocked'])} need a peer patch with no upstream fix (not even on main)")
if by_bucket['fetch_failed']:
why = "checks skipped in offline mode" if OFFLINE else "npm registry unreachable"
lines.append(f"- {len(by_bucket['fetch_failed'])} could not be checked ({why})")
@ -909,7 +926,8 @@ def main() -> int:
lines.append(
"These pins lag npm latest and the latest version's peer dep already "
"accepts our current `tree-sitter@" + current_runtime + ".x` runtime. "
"Bumping is independent of the 0.25 upgrade and should be a quick PR."
"When bumping, remove the old version's `AUDITED_PEERS` entry in "
"`gitnexus/scripts/prepare-tree-sitter-bundle.cjs` (prepack fails until you do)."
)
lines.append("")
for r in sorted(bump_now, key=lambda r: r["name"]):
@ -933,8 +951,9 @@ def main() -> int:
lines.append(md_h("Disposition", 2))
_emit_bucket(
"Ready for 0.25",
"These grammars' npm-latest peer dep already accepts the target runtime. No action needed for the upgrade.",
"Peer accepts runtime",
f"These grammars' npm-latest peer dep already accepts tree-sitter@{TARGET_RUNTIME}. "
"No workaround needed at npm latest.",
by_bucket["ready"],
lambda r: (
f"- `{r['name']}` — pinned `{r['pinned_spec']}`, npm latest `{r['npm_version_label']}`"
@ -960,7 +979,7 @@ def main() -> int:
_emit_bucket(
"Waiting on upstream npm release",
"Fixes are merged on the upstream main branch but not yet published to npm. "
"We can move forward as soon as upstream cuts a release.",
"The peer patch can be dropped as soon as upstream cuts a release.",
by_bucket["waiting"],
lambda r: (
f"- `{r['name']}@{r['npm_version_label']}` — peer `{r['peer_range'] or 'none'}`. "
@ -969,9 +988,9 @@ def main() -> int:
)
_emit_bucket(
"Blocked on upstream",
"Peer dep is too tight on both the latest npm release and on upstream main. "
"These need an upstream issue/PR before we can proceed.",
"Peer patch required",
"Peer dep rejects the runtime on both the latest npm release and upstream main. "
"The pack-time peer patch stays until upstream relaxes it.",
by_bucket["blocked"],
lambda r: f"- `{r['name']}@{r['npm_version_label']}` — peer `{r['peer_range'] or 'none'}`",
)
@ -988,6 +1007,35 @@ def main() -> int:
lambda r: f"- `{r['name']}` (pinned `{r['pinned_spec']}`)",
)
# ── Pack-time peer patches ──────────────────────────────────────
patched = read_patched_peers()
if patched:
rows_by_name = {r["name"]: r for r in grammar_rows}
patch_status = {
"ready": "can drop: npm latest accepts the runtime",
"intentional": "required while intentionally pinned",
"waiting": "drop once upstream publishes its relaxed peer",
"blocked": "required: no upstream fix yet",
"fetch_failed": f"unknown ({miss_label})",
}
lines.append(md_h(f"Pack-time peer patches ({len(patched)})", 2))
lines.append(
"`gitnexus/scripts/prepare-tree-sitter-bundle.cjs` adds "
f"`{TARGET_RUNTIME}` to these bundled manifests' tree-sitter peer range. "
"Removing an entry requires bumping (or dropping) the package that brings it in."
)
lines.append("")
for name, version, peer in sorted(patched):
row = rows_by_name.get(name)
if row is None or pinned_versions.get(name) != version:
status = "transitive: dropped when the grammar that depends on it is bumped"
elif row["bump_now"]:
status = f"can drop: bump to `{row['npm_version_label']}`"
else:
status = patch_status[row["bucket"]]
lines.append(f"- `{name}@{version}` (peer `{peer}`) — {status}")
lines.append("")
# ── Vendored parsers ────────────────────────────────────────────
lines.extend(_render_vendored_section(vendored_grammars, target_abi_range, blockers))

59
.github/scripts/test_check_tree_sitter_upgrade_readiness.py vendored
View file

@ -46,7 +46,7 @@ _ROW_DIFF_RE = re.compile(r"\| `(tree-sitter-[^`]+)` \|.*\| ([^|]+?) \|$", re.M)
_ISSUE_READY_RE = re.compile(
r"- (\d+)/(\d+) npm-installed grammars already accept tree-sitter@"
)
_ISSUE_BLOCKER_RE = re.compile(r"\*\*Blocked\*\* — (\d+) grammars? ")
_ISSUE_BLOCKER_RE = re.compile(r"\*\*Workarounds\*\* — (\d+) grammars? ")
def _physical_vendor_grammars() -> set[str]:
@ -118,8 +118,9 @@ class ManifestClassification(TestCase):
self.assertNotIn("tree-sitter-cpp", readiness.VENDORED_NAMES)
self.assertNotIn("tree-sitter-go", readiness.VENDORED_NAMES)
def test_c_carries_a_hold_cpp_does_not(self):
self.assertTrue(readiness.VENDORED["tree-sitter-c"]["hold"])
def test_swift_carries_a_hold_c_does_not(self):
self.assertTrue(readiness.VENDORED["tree-sitter-swift"]["hold"])
self.assertFalse(readiness.VENDORED["tree-sitter-c"]["hold"])
self.assertNotIn("tree-sitter-c", readiness.INTENTIONAL_PINS)
# cpp stays an npm intentional pin.
self.assertIn("tree-sitter-cpp", readiness.INTENTIONAL_PINS)
@ -278,9 +279,7 @@ class ReportRendering(TestCase):
cls.rows = dict(_ROW_DIFF_RE.findall(cls.report))
def test_no_bare_question_mark_anywhere(self):
# The only legitimate '?' is the "Satisfies 0.25?" column header.
sanitized = self.report.replace("Satisfies 0.25?", "Satisfies 0.25")
self.assertNotIn("?", sanitized, "report still contains a bare '?' placeholder")
self.assertNotIn("?", self.report, "report still contains a bare '?' placeholder")
def test_malformed_npm_version_renders_unknown_in_prose_not_bare_question(self):
# A successful (200) npm /latest response that omits `version` leaves
@ -304,8 +303,7 @@ class ReportRendering(TestCase):
contextlib.redirect_stdout(buf):
readiness.main()
report = buf.getvalue()
sanitized = report.replace("Satisfies 0.25?", "Satisfies 0.25")
self.assertNotIn("?", sanitized)
self.assertNotIn("?", report)
# The Ready bucket prose line for go shows the labeled 'unknown', not '?'.
self.assertRegex(report, r"`tree-sitter-go`.*npm latest `unknown`")
@ -328,14 +326,14 @@ class ReportRendering(TestCase):
self.assertNotIn("Could not check", self.report)
self.assertNotIn("fetch failed", self.report)
def test_held_c_renders_held_and_keeps_exit_nonzero(self):
def test_held_swift_renders_held_and_keeps_exit_nonzero(self):
# Status is the last matrix cell (the row-diff regex captures the whole
# tail, not just status, so read the cell directly).
cells = [c.strip() for c in self._matrix_row("tree-sitter-c").strip().strip("|").split("|")]
cells = [c.strip() for c in self._matrix_row("tree-sitter-swift").strip().strip("|").split("|")]
self.assertEqual(cells[-1], "Vendored — held")
self.assertIn("**Held:**", self.report)
# With every npm grammar mocked to "Ready", the ONLY remaining blocker is
# the held c — so a non-zero exit proves the hold is treated as a blocker.
# the held Swift — so a non-zero exit proves the hold is treated as a blocker.
self.assertEqual(self.code, 1)
def test_upstream_abi_miss_uses_labeled_sentinel(self):
@ -351,9 +349,9 @@ class ReportRendering(TestCase):
self.assertEqual(len(self.rows), len(readiness.GRAMMARS))
for name in readiness.VENDORED_NAMES:
self.assertIn(name, self.rows)
# group 2 is the Status cell — held c renders exactly "Vendored — held",
# group 2 is the Status cell — held Swift renders exactly "Vendored — held",
# and no captured status contains a pipe (proves cell-scoped capture).
self.assertEqual(self.rows["tree-sitter-c"], "Vendored — held")
self.assertEqual(self.rows["tree-sitter-swift"], "Vendored — held")
for status in self.rows.values():
self.assertNotIn("|", status)
@ -365,16 +363,30 @@ class ReportRendering(TestCase):
# Counts are derived from _render_report()'s mock corpus (all npm peer
# deps mocked permissive): of the 10 npm-installed grammars, 9 render
# Ready and 1 — tree-sitter-cpp — is the intentional pin (#1242), so it is
# not counted ready. The 4 blockers are that same pinned tree-sitter-cpp
# plus three held vendored grammars: ABI-held tree-sitter-c (#1242/#858),
# tree-sitter-kotlin (pinned to an unreleased fwcd main commit for `fun
# interface` support — ABI 14 is in range, but a hold counts as a blocker
# until it is lifted), and tree-sitter-objc. If a grammar is added/removed
# or a pin/hold changes,
# update _render_report()'s mock AND these expected counts together; a
# mismatch here means the report prose drifted, not the regex.
# not counted ready. The remaining workarounds are that intentional pin
# and Swift's declaration-recovery hold.
self.assertEqual(ready.groups(), ("9", "10"))
self.assertEqual(blockers.group(1), "4")
self.assertEqual(blockers.group(1), "2")
def test_lists_every_pack_time_peer_patch_with_its_drop_condition(self):
patched = readiness.read_patched_peers()
# Guards the AUDITED_PEERS regex: the bundle script patches 8 manifests today.
self.assertGreaterEqual(len(patched), 8)
self.assertIn(f"## Pack-time peer patches ({len(patched)})", self.report)
for name, version, _ in patched:
self.assertIn(f"- `{name}@{version}`", self.report)
line = lambda pkg: next(l for l in self.report.splitlines() if l.startswith(f"- `{pkg}`"))
# npm is mocked to a permissive 9.9.9, so a direct pin can be bumped away...
self.assertIn("can drop: bump to `9.9.9`", line("tree-sitter-java@0.23.5"))
# ...but not an intentional pin, nor a nested copy owned by another grammar.
self.assertIn("required while intentionally pinned", line("tree-sitter-cpp@0.23.4"))
self.assertIn("transitive", line("tree-sitter-javascript@0.23.1"))
self.assertIn("transitive", line("tree-sitter-c@0.23.6"))
def test_target_runtime_is_the_bundled_pin(self):
pkg = json.loads((_REPO_ROOT / "gitnexus" / "package.json").read_text())
self.assertEqual(readiness.TARGET_RUNTIME, pkg["dependencies"]["tree-sitter"])
self.assertIn(f"Bundled runtime `tree-sitter@{readiness.TARGET_RUNTIME}`", self.report)
def _matrix_row(self, name: str) -> str:
for line in self.report.splitlines():
@ -419,8 +431,7 @@ class OfflineMode(TestCase):
def test_offline_report_has_no_bare_question_mark(self):
report, _ = self._render_offline()
sanitized = report.replace("Satisfies 0.25?", "Satisfies 0.25")
self.assertNotIn("?", sanitized)
self.assertNotIn("?", report)
class VendoredAbiBranches(TestCase):

35
.github/scripts/update-vendored-grammars.mjs vendored
View file

@ -9,22 +9,16 @@
* validates the prebuilds — so even an imperfect re-vendor can never silently
* ship: its PR's CI goes red.
*
* ABI awareness is load-bearing. Every grammar is pinned to tree-sitter@0.21.1
* (LANGUAGE_VERSION 13–14, the #1922 gate). Most upstream grammar releases target
* a newer tree-sitter, so a blind "bump to latest" would pull an ABI-incompatible
* parser and open doomed PRs. This monitor fetches the candidate source, reads its
* parser.c `#define LANGUAGE_VERSION`, and only re-vendors when it is 13 or 14;
* incompatible updates are reported (and surfaced as a workflow notice), not
* applied.
* The runtime is tree-sitter@0.25.1 (LANGUAGE_VERSION 13–15). Read the
* candidate's generated ABI before updating; incompatible candidates are
* reported rather than applied. Semantic compatibility is verified by PR CI.
*
* Usage:
* node update-vendored-grammars.mjs # detect only → JSON report on stdout
* node update-vendored-grammars.mjs --apply X # re-vendor grammar X in place
*
* tree-sitter-c and tree-sitter-objc are MONITORED but report-only (`hold`): c is ABI-pinned at 0.21.4
* (#1242/#858) and must not auto-bump without a tree-sitter runtime upgrade, so an
* available c update is detected + reported but never auto-applied — even if it is
* ABI-13/14. A maintainer re-vendors it deliberately.
* Policy holds are recorded in the shared manifest. Swift is held at the
* newest tested release that preserves declaration ownership during recovery.
*/
import { execFileSync } from 'node:child_process';
import fs from 'node:fs';
@ -36,12 +30,12 @@ const __dirname = path.dirname(fileURLToPath(import.meta.url));
const REPO_ROOT = path.resolve(__dirname, '..', '..');
const VENDOR = path.join(REPO_ROOT, 'gitnexus', 'vendor');
const COMPATIBLE_ABI = new Set([13, 14]); // tree-sitter@0.21.1 LANGUAGE_VERSION range
const COMPATIBLE_ABI = new Set([13, 14, 15]); // tree-sitter@0.25.1 LANGUAGE_VERSION range
// Source-of-origin per grammar. npm grammars resolve `latest` via the registry;
// github grammars (no usable npm release) track the default branch HEAD. A `hold`
// reason makes a grammar report-only: updates are detected + surfaced but never
// auto-applied (c is ABI-pinned and must not move without a runtime upgrade).
// auto-applied (for example, when newer upstream error recovery regresses ownership).
//
// The vendored set lives in .github/vendored-grammars.json — the SHARED source of
// truth this monitor and .github/scripts/check-tree-sitter-upgrade-readiness.py both
@ -230,7 +224,7 @@ function detect(deps = {}) {
abiCompatible: abi == null ? null : COMPATIBLE_ABI.has(abi),
hold: g.hold || null,
// Auto-appliable only when there's an update, the ABI is known-compatible,
// AND the grammar is not on a policy hold (c).
// AND the grammar is not on a policy hold (e.g. swift).
applicable: newer && abi != null && COMPATIBLE_ABI.has(abi) && !g.hold,
});
}
@ -285,8 +279,8 @@ function apply(key, opts = {}) {
const abi = readAbiFn(srcRoot);
if (abi == null || !COMPATIBLE_ABI.has(abi))
throw new ApplyExit(
`${key}: candidate ${up.version} is ABI ${abi ?? 'unknown'} — not tree-sitter@0.21.1 ` +
`compatible (need 13/14); refusing to re-vendor. Handle manually.`,
`${key}: candidate ${up.version} is ABI ${abi ?? 'unknown'} — not tree-sitter@0.25.1 ` +
`compatible (need 13/14/15); refusing to re-vendor. Handle manually.`,
3,
);
@ -299,7 +293,8 @@ function apply(key, opts = {}) {
const dest = path.join(VENDOR, g.name);
// The source-build inputs + runtime entrypoints that change between versions.
// binding.gyp / README / LICENSE / prebuilds are intentionally NOT touched.
// Keep the synchronous CommonJS wrapper: upstream C now uses top-level await.
// binding.gyp / bindings/node/index.* / README / LICENSE / prebuilds are preserved.
for (const rel of [
'src/parser.c',
'src/scanner.c',
@ -308,8 +303,6 @@ function apply(key, opts = {}) {
'src/tree_sitter/array.h',
'src/tree_sitter/parser.h',
'bindings/node/binding.cc',
'bindings/node/index.js',
'bindings/node/index.d.ts',
]) {
copyFile(srcRoot, dest, rel);
}
@ -317,10 +310,12 @@ function apply(key, opts = {}) {
const pkgPath = path.join(dest, 'package.json');
const pkg = JSON.parse(fs.readFileSync(pkgPath, 'utf8'));
pkg.version = up.version;
if (up.kind === 'github') pkg._upstreamCommit = up.ref;
else delete pkg._upstreamCommit;
pkg._vendoredBy =
`gitnexus - re-vendored from ${g.npm ? `npm ${g.npm}@${up.version}` : `${g.github}@${up.ref}`} ` +
`by grammar-update-monitor on ABI ${abi}. Source-build inputs (parser.c/scanner.c/src/) refreshed; ` +
`the GitNexus-hardened binding.gyp + vendor README + prebuilds are preserved (prebuilds are ` +
`the GitNexus-hardened binding.gyp + synchronous loader + vendor README + prebuilds are preserved (prebuilds are ` +
`rebuilt by build-tree-sitter-prebuilds.yml on this version change). No scripts/dependencies here ` +
`(#836/#1728).`;
fs.writeFileSync(pkgPath, JSON.stringify(pkg, null, 2) + '\n');

32
.github/vendored-grammars.json vendored
View file

@ -3,34 +3,46 @@
"grammars": {
"c": {
"name": "tree-sitter-c",
"upstream": { "npm": "tree-sitter-c" },
"hold": "ABI-pinned at 0.21.4 (#1242/#858) — needs a tree-sitter runtime upgrade before bumping"
"upstream": {
"github": "tree-sitter/tree-sitter-c"
}
},
"objc": {
"name": "tree-sitter-objc",
"upstream": { "npm": "tree-sitter-objc" },
"hold": "Pinned at 3.0.2 for the Objective-C provider MVP; carries darwin/linux arm64+x64 prebuilds compatible with the current tree-sitter runtime (linux-arm64 built from vendored source because the upstream npm artifact is mislabeled)"
"upstream": {
"github": "tree-sitter-grammars/tree-sitter-objc"
}
},
"swift": {
"name": "tree-sitter-swift",
"upstream": { "npm": "tree-sitter-swift" }
"upstream": {
"npm": "tree-sitter-swift"
},
"hold": "Pinned to upstream 0.7.2-with-generated-files (7b7909f2): 0.7.3 and 0.7.4 re-parent a later top-level declaration after split conditional function headers. Keep until the declaration-ownership regression tests pass; see gitnexus/vendor/README.md."
},
"kotlin": {
"name": "tree-sitter-kotlin",
"upstream": { "npm": "tree-sitter-kotlin" },
"hold": "pinned to unreleased fwcd main commit c8ac3d26 for `fun interface` support (fwcd/tree-sitter-kotlin#169, closes #87) — npm latest (0.3.8) lacks the fix, so the monitor must NOT auto-revert (isNewer is strict-inequality: 0.3.8 != 0.4.0). Drop this hold and bump when upstream cuts a release that includes the fix"
"upstream": {
"github": "fwcd/tree-sitter-kotlin"
}
},
"dart": {
"name": "tree-sitter-dart",
"upstream": { "github": "UserNobody14/tree-sitter-dart" }
"upstream": {
"github": "UserNobody14/tree-sitter-dart"
}
},
"proto": {
"name": "tree-sitter-proto",
"upstream": { "github": "coder3101/tree-sitter-proto" }
"upstream": {
"github": "coder3101/tree-sitter-proto"
}
},
"zig": {
"name": "tree-sitter-zig",
"upstream": { "npm": "@tree-sitter-grammars/tree-sitter-zig" }
"upstream": {
"github": "tree-sitter-grammars/tree-sitter-zig"
}
}
}
}

28
.github/workflows/build-tree-sitter-prebuilds.yml vendored
View file

@ -9,9 +9,8 @@ name: Build tree-sitter prebuilds
# upstream prebuilds AND stays dependency-review-tracked, so it is left alone).
# All seven are vendored under gitnexus/vendor/; `kind` (below) only picks where
# the build job fetches the C source to compile:
# - tree-sitter-c (vendored prebuild-only; built from the published npm
# package — closes upstream's 4/6 ARM gap #2116 for a
# REQUIRED grammar)
# - tree-sitter-c (vendored source from upstream Git, including releases
# newer than npm; REQUIRED grammar)
# - tree-sitter-dart (vendored source; built from gitnexus/vendor/)
# - tree-sitter-proto (vendored source; built from gitnexus/vendor/)
# - tree-sitter-kotlin (vendored source; built from gitnexus/vendor/ — pinned to
@ -152,11 +151,11 @@ jobs:
// 'vendored' builds from gitnexus/vendor/<name> (which carries the C
// source + binding.gyp). Extend this list to cover a new grammar.
const REGISTRY = {
// c is vendored prebuild-only but BUILT from the published npm
// package (kind 'npm'), held at 0.21.4 — it closes upstream's 4/6
// c builds the exact upstream source snapshot in vendor/ so the
// prebuild matches its metadata — this also closes upstream's 4/6
// ARM gap (#2116) for a REQUIRED grammar that otherwise hard-fails
// install on toolchain-less ARM.
c: { name: 'tree-sitter-c', kind: 'npm' },
c: { name: 'tree-sitter-c', kind: 'vendored' },
dart: { name: 'tree-sitter-dart', kind: 'vendored' },
proto: { name: 'tree-sitter-proto', kind: 'vendored' },
// kotlin is vendored WITH its source (parser.c/scanner.c/binding.gyp),
@ -424,19 +423,20 @@ jobs:
# node-addon-api must match what the grammar's binding.cc expects.
# GitNexus hoists ^8 for the vendored grammars; npm grammars declare
# their own (do NOT pin it for npm grammars — let the dep resolve it).
# node-gyp >=12.1 recognizes Visual Studio 2026 on Windows ARM runners.
if [ "$KIND" = "vendored" ]; then
# Build from the vendored C source (carries parser.c + binding.gyp).
srcdir="$work/$NAME"
cp -R "$GITHUB_WORKSPACE/gitnexus/vendor/$NAME" "$srcdir"
rm -rf "$srcdir/prebuilds" "$srcdir/build" "$srcdir/node_modules"
npm install --no-audit --no-fund --ignore-scripts \
prebuildify@^6 node-gyp@^11 node-addon-api@^8
prebuildify@^6 node-gyp@12.4.0 node-addon-api@^8
pkgdir="$srcdir"
export npm_config_node_gyp="$work/node_modules/node-gyp/bin/node-gyp.js"
else
# Pull the published source-only package.
npm install --no-audit --no-fund --ignore-scripts \
"$NAME@${REF}" prebuildify@^6 node-gyp@^11
"$NAME@${REF}" prebuildify@^6 node-gyp@12.4.0
pkgdir="$work/node_modules/$NAME"
fi
@ -489,15 +489,11 @@ jobs:
cd "$probe"
# Pin tree-sitter to the repo's exact runtime peer so an ABI mismatch
# fails HERE, not in a user's install (mirrors the #1922 ABI gate).
# NOT --ignore-scripts: tree-sitter@0.21.1's tarball ships prebuilds for
# the common tuples but NOT linux-arm64 / win32-arm64, so on the arm64
# runners node-gyp-build must source-build the runtime — give it node-gyp
# + node-addon-api to do so. Where tree-sitter ships a prebuild (x64,
# darwin-arm64) node-gyp-build uses it and nothing compiles. The grammar
# .node we built is still loaded as a prebuild; only the runtime peer may
# compile. The grammar-vs-runtime ABI check still fires at setLanguage.
# Allow install scripts so node-gyp-build can compile the runtime if
# no matching runtime prebuild is available. The grammar built above
# is still loaded as a prebuild; setLanguage checks its runtime ABI.
npm install --no-audit --no-fund \
node-gyp-build@^4 node-gyp@^11 node-addon-api@^8 tree-sitter@0.21.1
node-gyp-build@^4 node-gyp@12.4.0 node-addon-api@^8 tree-sitter@0.25.1
# The node script is single-quoted on purpose — its ${...} are JS
# template literals read from the environment, not shell expansions.
# shellcheck disable=SC2016

3
.github/workflows/ci-e2e.yml vendored
View file

@ -20,6 +20,9 @@ jobs:
- uses: dorny/paths-filter@ceb8a2b8f2d89434be7ff52d3de7ec3738c5cc9d # v3
id: filter
with:
# Generated parser diffs can exceed GitHub's PR-files API limits.
# Use the action's local Git comparison against the PR base instead.
token: ''
filters: |
web:
- 'gitnexus-web/**'

8
.github/workflows/ci-tests.yml vendored
View file

@ -368,7 +368,7 @@ jobs:
# exemption is revoked here and an ABI-broken Zig binding fails the
# job instead of being accepted as a clean absence.
- name: Run parser-loader ABI load-smoke (dynamic)
run: npx vitest run test/unit/parser-loader-abi.test.ts
run: npx vitest run test/unit/parser-loader-abi.test.ts test/unit/group/grpc-extractor.test.ts
env:
GITNEXUS_REQUIRE_ZIG: '1'
working-directory: gitnexus
@ -376,7 +376,7 @@ jobs:
# End-to-end smoke test for the #1728 packaging fix: pack the published
# tarball, install it globally into a temp prefix, and assert no junction
# creation (the EPERM root cause) plus working CLI plus vendor cleanliness
# (#836). Runs on windows-latest because that is the platform the fix
# (#836) plus native grammar loading from the bundled runtime. Runs on windows-latest because that is the platform the fix
# targets; the in-repo `npm ci` job above only exercises the dev-tree path
# and skips the tarball reify step where the historical EPERM occurred.
packaged-install-smoke:
@ -456,6 +456,10 @@ jobs:
# installed artifact, not just the source workflow that produced it.
node "$INSTALLED/scripts/assert-web-assets.mjs" "$INSTALLED/web"
# Every bundled/vendored grammar must load and parse from the
# installed tree (main thread + worker), not just the dev checkout.
node "$INSTALLED/scripts/assert-native-parsers.mjs" "$INSTALLED"
# #836 invariant: no node_modules/ or build/ under any vendor/*.
BAD=$(find "$INSTALLED/vendor" \( -name node_modules -o -name build \) -print 2>/dev/null || true)
if [ -n "$BAD" ]; then

18
.github/workflows/grammar-update-monitor.yml vendored
View file

@ -2,19 +2,19 @@ name: Vendored grammar update monitor
# Periodically checks each vendored tree-sitter grammar against its
# source-of-origin and opens a PR re-vendoring any update that is ABI-COMPATIBLE
# with the pinned tree-sitter@0.21.1 (LANGUAGE_VERSION 13–14, #1922). The version
# with the bundled tree-sitter@0.25.1 (LANGUAGE_VERSION 13–15, #1922). The version
# bump then triggers build-tree-sitter-prebuilds.yml, which cross-builds + ABI-
# validates the prebuilds — so a re-vendor that is subtly wrong can never silently
# ship: its PR's CI goes red.
#
# ABI-INCOMPATIBLE updates (the common case — upstreams move to newer tree-sitter)
# are reported as a notice + job summary, NOT applied, so the monitor never opens
# doomed PRs. tree-sitter-c is MONITORED but report-only: it is ABI-pinned at
# 0.21.4 (#1242/#858), so an available c update is surfaced (notice + summary) but
# never auto-bumped — a maintainer re-vendors it deliberately after a runtime
# upgrade.
# ABI-INCOMPATIBLE updates (upstreams that move past the runtime's ABI range) are
# reported as a notice + job summary, NOT applied, so the monitor never opens
# doomed PRs. A grammar with a policy `hold` (today: tree-sitter-swift) is
# MONITORED but report-only: an available update is surfaced (notice + summary)
# but never auto-bumped — a maintainer re-vendors it deliberately once the hold's
# condition is met.
#
# The vendored set + per-grammar upstream coords + the tree-sitter-c hold live in
# The vendored set + per-grammar upstream coords + policy holds live in
# .github/vendored-grammars.json — the SHARED source of truth this monitor and
# tree-sitter-upgrade-readiness.yml both read, so the two workflows can never
# disagree about which grammars are vendored (#858). This monitor additionally
@ -125,7 +125,7 @@ jobs:
const body = [
`Automated re-vendor of **${name}** to \`${r.upstream}\` (from ${r.kind === 'npm' ? `npm \`${name}\`` : `\`${r.ref}\``}).`,
'',
`Verified ABI **${r.abi}** — compatible with the pinned \`tree-sitter@0.21.1\` (13–14).`,
`Verified ABI **${r.abi}** — compatible with the bundled \`tree-sitter@0.25.1\` (13–15).`,
'Source-build inputs refreshed; the GitNexus binding.gyp / README / prebuilds are preserved.',
'The version bump triggers `build-tree-sitter-prebuilds.yml` to rebuild + ABI-validate the',
'prebuilds — review its result before merging.',

35
.github/workflows/tree-sitter-upgrade-readiness.yml vendored
View file

@ -1,8 +1,9 @@
name: Tree-sitter Upgrade Readiness
name: Tree-sitter Compatibility Workarounds
# Monitors readiness for upgrading tree-sitter to 0.25.x. Tracks:
# 1. Peer-dep compatibility — can each NPM-installed grammar install cleanly
# with tree-sitter@0.25.0 without --legacy-peer-deps?
# Tracks the workarounds GitNexus carries for its bundled tree-sitter runtime:
# 1. Peer-dep patches — which npm grammars still reject the runtime, so
# prepare-tree-sitter-bundle.cjs must keep patching their manifests at pack
# time, and which patches a grammar bump would let us drop.
# 2. Vendored grammars — each grammar in .github/vendored-grammars.json
# (c/swift/kotlin/dart/proto/objc) is classified by its vendored ABI, read
# straight from gitnexus/vendor/<name>/src/parser.c (NOT node_modules,
@ -30,6 +31,7 @@ on:
- '.github/scripts/test_check_tree_sitter_upgrade_readiness.py'
- '.github/vendored-grammars.json'
- '.github/workflows/tree-sitter-upgrade-readiness.yml'
- 'gitnexus/scripts/prepare-tree-sitter-bundle.cjs'
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
@ -100,7 +102,7 @@ jobs:
- name: Annotate PR with readiness status
if: github.event_name == 'pull_request' && steps.readiness.outputs.exit_code != '0'
run: |
echo "::warning::Tree-sitter 0.25 upgrade has blockers. See job output for the full readiness report."
echo "::notice::Tree-sitter compatibility workarounds remain. See job output for the full report."
# Issue mutation is isolated here so `issues: write` is only ever granted on the
# scheduled run (never on PRs). Consumes the report + exit_code via job outputs.
@ -113,18 +115,20 @@ jobs:
permissions:
issues: write
steps:
- name: Upsert tracking issue on blockers
- name: Upsert tracking issue while workarounds remain
if: needs.report.outputs.exit_code != '0'
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
env:
REPORT: ${{ needs.report.outputs.report }}
with:
script: |
const title = 'Tree-sitter 0.25 upgrade readiness';
const title = 'Tree-sitter compatibility workarounds';
// Adopt the issue opened under the pre-0.25 title instead of opening a twin.
const titles = [title, 'Tree-sitter 0.25 upgrade readiness'];
const report = process.env.REPORT;
const body = report + '\n\n' +
'<sub>Generated daily by `.github/workflows/tree-sitter-upgrade-readiness.yml`. ' +
'Closes automatically when all blockers are resolved.</sub>';
'Closes automatically when no workarounds remain.</sub>';
const { data: open } = await github.rest.issues.listForRepo({
owner: context.repo.owner,
repo: context.repo.repo,
@ -132,7 +136,7 @@ jobs:
labels: 'tree-sitter-drift',
per_page: 10,
});
const existing = open.find(i => i.title === title);
const existing = open.find(i => titles.includes(i.title));
if (existing) {
// Extract ready/total count for the changelog comment.
// The two report.match() regexes below are mirrored as
@ -156,7 +160,7 @@ jobs:
);
const blockerMatch = requireMatch(
'blocker count',
report.match(/\*\*Blocked\*\* — (\d+) grammars? /),
report.match(/\*\*Workarounds\*\* — (\d+) grammars? /),
);
const ready = readyMatch[1];
const total = readyMatch[2];
@ -188,7 +192,7 @@ jobs:
}
const today = new Date().toISOString().slice(0, 10);
let comment = `**${today}:** ${ready}/${total} npm-installed ready. ${blockers} blocker(s) remaining.`;
let comment = `**${today}:** ${ready}/${total} npm-installed accept the runtime. ${blockers} workaround(s) remaining.`;
if (changes.length > 0) {
comment += '\n\nChanges:\n' + changes.map(c => `- ${c}`).join('\n');
} else {
@ -205,6 +209,7 @@ jobs:
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: existing.number,
title,
body,
});
core.info(`Updated existing issue #${existing.number}`);
@ -224,7 +229,9 @@ jobs:
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with:
script: |
const title = 'Tree-sitter 0.25 upgrade readiness';
const title = 'Tree-sitter compatibility workarounds';
// Adopt the issue opened under the pre-0.25 title instead of opening a twin.
const titles = [title, 'Tree-sitter 0.25 upgrade readiness'];
const { data: open } = await github.rest.issues.listForRepo({
owner: context.repo.owner,
repo: context.repo.repo,
@ -232,13 +239,13 @@ jobs:
labels: 'tree-sitter-drift',
per_page: 10,
});
const existing = open.find(i => i.title === title);
const existing = open.find(i => titles.includes(i.title));
if (existing) {
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: existing.number,
body: 'All grammars are now compatible with tree-sitter@0.25. Upgrade is ready! Closing automatically.',
body: 'No compatibility workarounds remain: every grammar runs on the bundled tree-sitter runtime as published. Closing automatically.',
});
await github.rest.issues.update({
owner: context.repo.owner,

19
CONTRIBUTING.md
View file

@ -156,12 +156,29 @@ Re-invoking `/autofix` after a successful apply is a safe no-op — the workflow
**Sensitive paths.** The apply workflow refuses any patch that touches `.github/` (workflow files, CODEOWNERS, dependabot config). A malicious PR could ship a custom prettier or ESLint config that reformats workflow YAML; if accepted, those edits would be pushed under `contents: write` without human review. Apply formatter changes to files under `.github/` manually in a normal commit so they get the same review every other workflow change gets.
### Native tree-sitter npm packages
The CLI pins and bundles the native `tree-sitter` runtime and its npm grammars.
Several compatible grammars still declare older runtime peer ranges. Root npm
`overrides` work in this checkout but do not propagate to consumers, so
`gitnexus/scripts/prepare-tree-sitter-bundle.cjs` runs during `prepack` to add the
tested runtime version to eight audited dependency manifests. It leaves grammar
sources and native binaries unchanged and rejects unexpected versions, peer
ranges, or dependency layouts before writing any manifests.
When upgrading these packages, update the exact pins, lockfile, and audited peer
list together. Run the parser, extraction, CFG, and worker tests, then use a real
`npm pack` (including `prepack`) to test a fresh consumer's `npm install`,
`npm ci`, `npm ls`, and a subsequent dependency install. Check native parser
loading on the supported Node.js versions and release platforms. A successful
install in this checkout alone does not verify the published bundle.
### Vendored tree-sitter grammars
`.github/vendored-grammars.json` is the **single source of truth** for the vendored tree-sitter grammar **set** and each grammar's policy `hold` (the ones shipped from `gitnexus/vendor/<name>` rather than installed from npm). It lists each grammar's name, upstream coords (`npm` or `github`), and any `hold`. The monitor resolves upstreams from it; the readiness report keeps its own upstream-drift coords and reads vendored ABIs from `gitnexus/vendor/`. Two workflows read it:
- `grammar-update-monitor.yml` (`.github/scripts/update-vendored-grammars.mjs`) — weekly; opens auto-PRs re-vendoring ABI-compatible upstream updates.
- `tree-sitter-upgrade-readiness.yml` (`.github/scripts/check-tree-sitter-upgrade-readiness.py`) — daily; renders the tree-sitter-0.25 readiness report (issue #858), reading each vendored grammar's ABI from `gitnexus/vendor/<name>/src/parser.c`.
- `tree-sitter-upgrade-readiness.yml` (`.github/scripts/check-tree-sitter-upgrade-readiness.py`) — daily; renders the tree-sitter compatibility-workarounds report (pack-time peer patches, intentional pins, vendored holds; issue #858), reading each vendored grammar's ABI from `gitnexus/vendor/<name>/src/parser.c`.
Sharing the manifest keeps the two aligned: a consistency-guard test asserts the manifest set equals the `gitnexus/vendor/tree-sitter-*` directories. **When you vendor a new grammar (or remove one), update `.github/vendored-grammars.json` in the same change** — otherwise that guard fails CI and the readiness report regresses to `?` placeholders.

2
gitnexus/bench/python-scope/baseline-fingerprint.txt
View file

@ -1 +1 @@
95a98cf485746489f3ecf32e9a6bc7ee40604fe156d4cdd3ce36bc40581f1629
390272fd1067ca18a91c5ec5cc1800fb38fab56d06671baf0d0a0e2bd0587658

3
gitnexus/bench/python-scope/measure.mjs
View file

@ -201,6 +201,9 @@ if (!CHECK) {
// the path is still linear (scaling ratio under budget). Re-baseline a
// legitimate capture change with `node --import tsx measure.mjs` (no --check)
// and commit the new baseline-fingerprint.txt deliberately.
// #3531: Tree-sitter 0.25 adds one zero-width module scope to each of the
// 15 empty __init__.py fixtures (3982 -> 3997 groups). All non-empty fixture
// captures and the synthetic DAO are unchanged; the golden test pins this.
const baselinePath = path.resolve(__dirname, 'baseline-fingerprint.txt');
const baseline = fs.readFileSync(baselinePath, 'utf8').trim();
const failures = [];

3
gitnexus/bench/scope-capture/baselines.json
View file

@ -99,7 +99,8 @@
"fixture_count": 202
},
"php": {
"fingerprint": "6e92ed6e50c2ba4b383b97b9f9b550a99d2d647a30a2867b53ad94f24114952d",
"fingerprint": "745700bcd75c3f7bccff2e8085205f129c9c965d38ad2e291bba42d56afcd5e6",
"_rebaselined_3531": "The php-typed-property-dedup fixture class Mixed was renamed MixedProperties because mixed is a reserved PHP type. Only that fixture's text/ranges change; all 142 fixture files emit identical captures under the old and new grammars when given the same source. Capture groups remain 2241 and the scaling budget is unchanged.",
"_rebaselined_3502_attributed_function_fixture": "#3502 extends the existing php-nested-function fixture with an attributed, comment-interposed named function and a long-whitespace closure control. This is fixture-content growth only: the capture implementation now classifies existing captures instead of scanning source text, so it cannot mint or remove matches. The fixture corpus stays at 143 files; captured groups move 2237 -> 2241, while synthetic scaling counts stay 4258/13608. Prior 0adf565ed79dc1d909c23a18a9e3a17ddaa6556fb73857081840cd7389953075 -> 6e92ed6e50c2ba4b383b97b9f9b550a99d2d647a30a2867b53ad94f24114952d.",
"_rebaselined_3502_nested_function_fixture": "#3502 adds the php-nested-function fixture used by the resolver regression. The capture implementation and query are unchanged; this is fixture-corpus growth only. The unified harness moves from 141 to 143 fixture files and from 2203 to 2237 captured groups, while the synthetic scaling counts remain 4258/13608 and every other language matches its baseline. Prior 421cbd318c8857f840aa60904a675cbde32c6bb57b01e6c0f970d10db572b99e -> 0adf565ed79dc1d909c23a18a9e3a17ddaa6556fb73857081840cd7389953075.",
"scaling_budget": 1.5,

17
gitnexus/bench/scope-capture/measure.mjs
View file

@ -366,7 +366,11 @@ function median(xs) {
}
function timeEmit(emit, src, file, reps) {
emit(src, `warmup-${file}`); // warm parser/query JIT (not counted)
// Stabilize parser/query JIT before collecting each size's samples.
// Keep sizes in separate batches so large allocations cannot bias small runs.
for (let i = 0; i < WARMUP_REPS; i++) {
emit(src, `warmup-${file}`);
}
const samples = [];
let count = 0;
for (let i = 0; i < reps; i++) {
@ -380,7 +384,8 @@ function timeEmit(emit, src, file, reps) {
const SMALL = 250;
const LARGE = 800;
const REPS = 7;
const WARMUP_REPS = 7;
const REPS = 21;
function measureLang(lang) {
// Correctness fingerprint over the fixture corpus + a fixed 20-entity source.
@ -423,7 +428,13 @@ function measureLang(lang) {
// ---- run ----
const CHECK = process.argv.includes('--check');
const results = LANGS.map(measureLang);
const results = [];
for (const lang of LANGS) {
results.push(measureLang(lang));
// Let native tree/node finalizers run between languages. Longer sampling
// otherwise retains their allocations for the entire synchronous batch.
await new Promise(setImmediate);
}
if (!CHECK) {
for (const r of results) process.stdout.write(JSON.stringify(r) + '\n');

171
gitnexus/package-lock.json generated
View file

@ -7,6 +7,19 @@
"": {
"name": "gitnexus",
"version": "1.6.12",
"bundleDependencies": [
"tree-sitter",
"tree-sitter-c-sharp",
"tree-sitter-cpp",
"tree-sitter-go",
"tree-sitter-java",
"tree-sitter-javascript",
"tree-sitter-php",
"tree-sitter-python",
"tree-sitter-ruby",
"tree-sitter-rust",
"tree-sitter-typescript"
],
"hasInstallScript": true,
"license": "PolyForm-Noncommercial-1.0.0",
"dependencies": {
@ -38,17 +51,17 @@
"pino-pretty": "^13.1.3",
"proxy-addr": "^2.0.7",
"smol-toml": "^1.8.0",
"tree-sitter": "0.21.1",
"tree-sitter-c-sharp": "0.23.1",
"tree-sitter-cpp": "0.23.2",
"tree-sitter-go": "^0.23.0",
"tree-sitter-java": "^0.23.5",
"tree-sitter-javascript": "^0.23.0",
"tree-sitter-php": "^0.23.0",
"tree-sitter-python": "0.23.4",
"tree-sitter-ruby": "^0.23.1",
"tree-sitter-rust": "0.23.1",
"tree-sitter-typescript": "^0.23.2",
"tree-sitter": "0.25.1",
"tree-sitter-c-sharp": "0.23.5",
"tree-sitter-cpp": "0.23.4",
"tree-sitter-go": "0.25.0",
"tree-sitter-java": "0.23.5",
"tree-sitter-javascript": "0.25.0",
"tree-sitter-php": "0.24.2",
"tree-sitter-python": "0.25.0",
"tree-sitter-ruby": "0.23.1",
"tree-sitter-rust": "0.24.0",
"tree-sitter-typescript": "0.23.2",
"uuid": "^14.0.0"
},
"bin": {
@ -3779,6 +3792,7 @@
"version": "8.9.2",
"resolved": "https://registry.npmjs.org/node-addon-api/-/node-addon-api-8.9.2.tgz",
"integrity": "sha512-VijLXbi3UACN69I0JVXJsX4tjACjNoQDgv2gTF6sx2wWEi8tkSg2eX8p5gSIFi8z2+DL3oHmY6OyKce38SDolg==",
"inBundle": true,
"license": "MIT",
"engines": {
"node": "^18 || ^20 || >= 21"
@ -3794,6 +3808,7 @@
"version": "4.8.4",
"resolved": "https://registry.npmjs.org/node-gyp-build/-/node-gyp-build-4.8.4.tgz",
"integrity": "sha512-LA4ZjwlnUblHVgq0oBF3Jl/6h/Nvs5fzBLwdEF4nuxnFdsfajde4WfxtJr3CaiH+F6ewcIB/q4jQ4UzPyid+CQ==",
"inBundle": true,
"license": "MIT",
"bin": {
"node-gyp-build": "bin.js",
@ -4700,28 +4715,50 @@
}
},
"node_modules/tree-sitter": {
"version": "0.21.1",
"resolved": "https://registry.npmjs.org/tree-sitter/-/tree-sitter-0.21.1.tgz",
"integrity": "sha512-7dxoA6kYvtgWw80265MyqJlkRl4yawIjO7S5MigytjELkX43fV2WsAXzsNfO7sBpPPCF5Gp0+XzHk0DwLCq3xQ==",
"version": "0.25.1",
"resolved": "https://registry.npmjs.org/tree-sitter/-/tree-sitter-0.25.1.tgz",
"integrity": "sha512-mrcEdkYtHfrK1A6fs3O6FxkBo0Qig5XUXqHhxUOQu0bmPo00QF4XaSx4edpazdHwxnSCjlGKGgIqWdaN4dvTLA==",
"hasInstallScript": true,
"inBundle": true,
"license": "MIT",
"dependencies": {
"node-addon-api": "^8.0.0",
"node-gyp-build": "^4.8.0"
"node-addon-api": "^8.5.0",
"node-gyp-build": "^4.8.4"
}
},
"node_modules/tree-sitter-c": {
"version": "0.23.6",
"resolved": "https://registry.npmjs.org/tree-sitter-c/-/tree-sitter-c-0.23.6.tgz",
"integrity": "sha512-0dxXKznVyUA0s6PjNolJNs2yF87O5aL538A/eR6njA5oqX3C3vH4vnx3QdOKwuUdpKEcFdHuiDpRKLLCA/tjvQ==",
"hasInstallScript": true,
"inBundle": true,
"license": "MIT",
"dependencies": {
"node-addon-api": "^8.3.0",
"node-gyp-build": "^4.8.4"
},
"peerDependencies": {
"tree-sitter": "^0.22.1"
},
"peerDependenciesMeta": {
"tree-sitter": {
"optional": true
}
}
},
"node_modules/tree-sitter-c-sharp": {
"version": "0.23.1",
"resolved": "https://registry.npmjs.org/tree-sitter-c-sharp/-/tree-sitter-c-sharp-0.23.1.tgz",
"integrity": "sha512-9zZ4FlcTRWWfRf6f4PgGhG8saPls6qOOt75tDfX7un9vQZJmARjPrAC6yBNCX2T/VKcCjIDbgq0evFaB3iGhQw==",
"version": "0.23.5",
"resolved": "https://registry.npmjs.org/tree-sitter-c-sharp/-/tree-sitter-c-sharp-0.23.5.tgz",
"integrity": "sha512-xJGOeXPMmld0nES5+080N/06yY6LQi+KWGWV4LfZaZe6srJPtUtfhIbRSN7EZN6IaauzW28v6W4QHFwmeUW6HQ==",
"hasInstallScript": true,
"inBundle": true,
"license": "MIT",
"dependencies": {
"node-addon-api": "^8.2.2",
"node-gyp-build": "^4.8.2"
"node-gyp-build": "^4.8.4"
},
"peerDependencies": {
"tree-sitter": "^0.21.1"
"tree-sitter": "^0.25.0"
},
"peerDependenciesMeta": {
"tree-sitter": {
@ -4730,14 +4767,16 @@
}
},
"node_modules/tree-sitter-cpp": {
"version": "0.23.2",
"resolved": "https://registry.npmjs.org/tree-sitter-cpp/-/tree-sitter-cpp-0.23.2.tgz",
"integrity": "sha512-GTa5Dx1O9ihzW70LvaUviTclh+wlBDRz6opR9Ij4NQIFmq/joeZ/k65UbLV4nLidR7xZ9eNNGT/SonCqAmjGVg==",
"version": "0.23.4",
"resolved": "https://registry.npmjs.org/tree-sitter-cpp/-/tree-sitter-cpp-0.23.4.tgz",
"integrity": "sha512-qR5qUDyhZ5jJ6V8/umiBxokRbe89bCGmcq/dk94wI4kN86qfdV8k0GHIUEKaqWgcu42wKal5E97LKpLeVW8sKw==",
"hasInstallScript": true,
"inBundle": true,
"license": "MIT",
"dependencies": {
"node-addon-api": "^8.2.1",
"node-gyp-build": "^4.8.2"
"node-gyp-build": "^4.8.2",
"tree-sitter-c": "^0.23.1"
},
"peerDependencies": {
"tree-sitter": "^0.21.1"
@ -4749,17 +4788,18 @@
}
},
"node_modules/tree-sitter-go": {
"version": "0.23.4",
"resolved": "https://registry.npmjs.org/tree-sitter-go/-/tree-sitter-go-0.23.4.tgz",
"integrity": "sha512-iQaHEs4yMa/hMo/ZCGqLfG61F0miinULU1fFh+GZreCRtKylFLtvn798ocCZjO2r/ungNZgAY1s1hPFyAwkc7w==",
"version": "0.25.0",
"resolved": "https://registry.npmjs.org/tree-sitter-go/-/tree-sitter-go-0.25.0.tgz",
"integrity": "sha512-APBc/Dq3xz/e35Xpkhb1blu5UgW+2E3RyGWawZSCNcbGwa7jhSQPS8KsUupuzBla8PCo8+lz9W/JDJjmfRa2tw==",
"hasInstallScript": true,
"inBundle": true,
"license": "MIT",
"dependencies": {
"node-addon-api": "^8.2.1",
"node-gyp-build": "^4.8.2"
"node-addon-api": "^8.3.1",
"node-gyp-build": "^4.8.4"
},
"peerDependencies": {
"tree-sitter": "^0.21.1"
"tree-sitter": "^0.25.0"
},
"peerDependenciesMeta": {
"tree-sitter": {
@ -4772,6 +4812,7 @@
"resolved": "https://registry.npmjs.org/tree-sitter-java/-/tree-sitter-java-0.23.5.tgz",
"integrity": "sha512-Yju7oQ0Xx7GcUT01mUglPP+bYfvqjNCGdxqigTnew9nLGoII42PNVP3bHrYeMxswiCRM0yubWmN5qk+zsg0zMA==",
"hasInstallScript": true,
"inBundle": true,
"license": "MIT",
"dependencies": {
"node-addon-api": "^8.2.2",
@ -4787,17 +4828,18 @@
}
},
"node_modules/tree-sitter-javascript": {
"version": "0.23.1",
"resolved": "https://registry.npmjs.org/tree-sitter-javascript/-/tree-sitter-javascript-0.23.1.tgz",
"integrity": "sha512-/bnhbrTD9frUYHQTiYnPcxyHORIw157ERBa6dqzaKxvR/x3PC4Yzd+D1pZIMS6zNg2v3a8BZ0oK7jHqsQo9fWA==",
"version": "0.25.0",
"resolved": "https://registry.npmjs.org/tree-sitter-javascript/-/tree-sitter-javascript-0.25.0.tgz",
"integrity": "sha512-1fCbmzAskZkxcZzN41sFZ2br2iqTYP3tKls1b/HKGNPQUVOpsUxpmGxdN/wMqAk3jYZnYBR1dd/y/0avMeU7dw==",
"hasInstallScript": true,
"inBundle": true,
"license": "MIT",
"dependencies": {
"node-addon-api": "^8.2.2",
"node-gyp-build": "^4.8.2"
"node-addon-api": "^8.3.1",
"node-gyp-build": "^4.8.4"
},
"peerDependencies": {
"tree-sitter": "^0.21.1"
"tree-sitter": "^0.25.0"
},
"peerDependenciesMeta": {
"tree-sitter": {
@ -4806,17 +4848,18 @@
}
},
"node_modules/tree-sitter-php": {
"version": "0.23.12",
"resolved": "https://registry.npmjs.org/tree-sitter-php/-/tree-sitter-php-0.23.12.tgz",
"integrity": "sha512-VwkBVOahhC2NYXK/Fuqq30NxuL/6c2hmbxEF4jrB7AyR5rLc7nT27mzF3qoi+pqx9Gy2AbXnGezF7h4MeM6YRA==",
"version": "0.24.2",
"resolved": "https://registry.npmjs.org/tree-sitter-php/-/tree-sitter-php-0.24.2.tgz",
"integrity": "sha512-zwgAePc/HozNaWOOfwRAA+3p8yhuehRw8Fb7vn5qd2XjiIc93uJPryDTMYTSjBRjVIUg/KY6pM3rRzs8dSwKfw==",
"hasInstallScript": true,
"inBundle": true,
"license": "MIT",
"dependencies": {
"node-addon-api": "^8.2.2",
"node-gyp-build": "^4.8.2"
},
"peerDependencies": {
"tree-sitter": "^0.21.1"
"tree-sitter": "^0.22.4"
},
"peerDependenciesMeta": {
"tree-sitter": {
@ -4825,17 +4868,18 @@
}
},
"node_modules/tree-sitter-python": {
"version": "0.23.4",
"resolved": "https://registry.npmjs.org/tree-sitter-python/-/tree-sitter-python-0.23.4.tgz",
"integrity": "sha512-MbmUAl7y5UCUWqHscHke7DdRDwQnVNMNKQYQc4Gq2p09j+fgPxaU8JVsuOI/0HD3BSEEe5k9j3xmdtIWbDtDgw==",
"version": "0.25.0",
"resolved": "https://registry.npmjs.org/tree-sitter-python/-/tree-sitter-python-0.25.0.tgz",
"integrity": "sha512-eCmJx6zQa35GxaCtQD+wXHOhYqBxEL+bp71W/s3fcDMu06MrtzkVXR437dRrCrbrDbyLuUDJpAgycs7ncngLXw==",
"hasInstallScript": true,
"inBundle": true,
"license": "MIT",
"dependencies": {
"node-addon-api": "^8.2.1",
"node-gyp-build": "^4.8.2"
"node-addon-api": "^8.5.0",
"node-gyp-build": "^4.8.4"
},
"peerDependencies": {
"tree-sitter": "^0.21.1"
"tree-sitter": "^0.25.0"
},
"peerDependenciesMeta": {
"tree-sitter": {
@ -4848,6 +4892,7 @@
"resolved": "https://registry.npmjs.org/tree-sitter-ruby/-/tree-sitter-ruby-0.23.1.tgz",
"integrity": "sha512-d9/RXgWjR6HanN7wTYhS5bpBQLz1VkH048Vm3CodPGyJVnamXMGb8oEhDypVCBq4QnHui9sTXuJBBP3WtCw5RA==",
"hasInstallScript": true,
"inBundle": true,
"license": "MIT",
"dependencies": {
"node-addon-api": "^8.2.2",
@ -4863,17 +4908,18 @@
}
},
"node_modules/tree-sitter-rust": {
"version": "0.23.1",
"resolved": "https://registry.npmjs.org/tree-sitter-rust/-/tree-sitter-rust-0.23.1.tgz",
"integrity": "sha512-wrMptzUAfbl3DbNrldZveyNM2CWmRw2VvEo2j/855qQbMMz4dlCF+TBwRN/1FL1S6cYvAEAJaCMesGqhocFJhQ==",
"version": "0.24.0",
"resolved": "https://registry.npmjs.org/tree-sitter-rust/-/tree-sitter-rust-0.24.0.tgz",
"integrity": "sha512-NWemUDf629Tfc90Y0Z55zuwPCAHkLxWnMf2RznYu4iBkkrQl2o/CHGB7Cr52TyN5F1DAx8FmUnDtCy9iUkXZEQ==",
"hasInstallScript": true,
"inBundle": true,
"license": "MIT",
"dependencies": {
"node-addon-api": "^8.2.2",
"node-gyp-build": "^4.8.2"
"node-gyp-build": "^4.8.4"
},
"peerDependencies": {
"tree-sitter": "^0.21.1"
"tree-sitter": "^0.22.1"
},
"peerDependenciesMeta": {
"tree-sitter": {
@ -4886,6 +4932,7 @@
"resolved": "https://registry.npmjs.org/tree-sitter-typescript/-/tree-sitter-typescript-0.23.2.tgz",
"integrity": "sha512-e04JUUKxTT53/x3Uq1zIL45DoYKVfHH4CZqwgZhPg5qYROl5nQjV+85ruFzFGZxu+QeFVbRTPDRnqL9UbU4VeA==",
"hasInstallScript": true,
"inBundle": true,
"license": "MIT",
"dependencies": {
"node-addon-api": "^8.2.2",
@ -4901,6 +4948,26 @@
}
}
},
"node_modules/tree-sitter-typescript/node_modules/tree-sitter-javascript": {
"version": "0.23.1",
"resolved": "https://registry.npmjs.org/tree-sitter-javascript/-/tree-sitter-javascript-0.23.1.tgz",
"integrity": "sha512-/bnhbrTD9frUYHQTiYnPcxyHORIw157ERBa6dqzaKxvR/x3PC4Yzd+D1pZIMS6zNg2v3a8BZ0oK7jHqsQo9fWA==",
"hasInstallScript": true,
"inBundle": true,
"license": "MIT",
"dependencies": {
"node-addon-api": "^8.2.2",
"node-gyp-build": "^4.8.2"
},
"peerDependencies": {
"tree-sitter": "^0.21.1"
},
"peerDependenciesMeta": {
"tree-sitter": {
"optional": true
}
}
},
"node_modules/tslib": {
"version": "2.8.1",
"resolved": "https://registry.npmjs.org/tslib/-/tslib-2.8.1.tgz",

42
gitnexus/package.json
View file

@ -63,7 +63,7 @@
"postinstall": "node scripts/build-tree-sitter-grammars.cjs",
"assert-publish-coverage": "node scripts/assert-publish-grammar-coverage.cjs",
"prepare": "node scripts/build.js",
"prepack": "node scripts/assert-publish-grammar-coverage.cjs && node scripts/assert-publish-fts-coverage.cjs && node scripts/build.js --web && node scripts/assert-web-assets.mjs web",
"prepack": "node scripts/prepare-tree-sitter-bundle.cjs && node scripts/assert-publish-grammar-coverage.cjs && node scripts/assert-publish-fts-coverage.cjs && node scripts/build.js --web && node scripts/assert-web-assets.mjs web",
"version": "node scripts/sync-plugin-manifests.mjs"
},
"dependencies": {
@ -95,17 +95,17 @@
"pino-pretty": "^13.1.3",
"proxy-addr": "^2.0.7",
"smol-toml": "^1.8.0",
"tree-sitter": "0.21.1",
"tree-sitter-c-sharp": "0.23.1",
"tree-sitter-cpp": "0.23.2",
"tree-sitter-go": "^0.23.0",
"tree-sitter-java": "^0.23.5",
"tree-sitter-javascript": "^0.23.0",
"tree-sitter-php": "^0.23.0",
"tree-sitter-python": "0.23.4",
"tree-sitter-ruby": "^0.23.1",
"tree-sitter-rust": "0.23.1",
"tree-sitter-typescript": "^0.23.2",
"tree-sitter": "0.25.1",
"tree-sitter-c-sharp": "0.23.5",
"tree-sitter-cpp": "0.23.4",
"tree-sitter-go": "0.25.0",
"tree-sitter-java": "0.23.5",
"tree-sitter-javascript": "0.25.0",
"tree-sitter-php": "0.24.2",
"tree-sitter-python": "0.25.0",
"tree-sitter-ruby": "0.23.1",
"tree-sitter-rust": "0.24.0",
"tree-sitter-typescript": "0.23.2",
"uuid": "^14.0.0"
},
"gitnexusEmbeddingStack": {
@ -136,9 +136,23 @@
},
"overrides": {
"adm-zip": ">=0.6.0",
"sharp": ">=0.35.0"
"sharp": ">=0.35.0",
"tree-sitter": "$tree-sitter"
},
"engines": {
"node": "^22.18.0 || >=24.11.0"
}
},
"bundleDependencies": [
"tree-sitter",
"tree-sitter-c-sharp",
"tree-sitter-cpp",
"tree-sitter-go",
"tree-sitter-java",
"tree-sitter-javascript",
"tree-sitter-php",
"tree-sitter-python",
"tree-sitter-ruby",
"tree-sitter-rust",
"tree-sitter-typescript"
]
}

97
gitnexus/scripts/assert-native-parsers.mjs Normal file
View file

@ -0,0 +1,97 @@
#!/usr/bin/env node
// Packaged-install smoke for the bundled tree-sitter runtime and grammars.
// Every registered grammar — including the vendored "optional" ones, which
// ship prebuilds for every supported platform — must load and parse a sample
// cleanly from the INSTALLED package, on the main thread and in a worker
// thread (the parse pipeline's real host). Usage:
// node scripts/assert-native-parsers.mjs <installed gitnexus dir>
import { createRequire } from 'node:module';
import path from 'node:path';
import { pathToFileURL } from 'node:url';
import { Worker, isMainThread, workerData, parentPort } from 'node:worker_threads';
// One sample per grammar key. A key without a sample fails the check, so a new
// grammar cannot ship without packaged coverage.
const SAMPLES = {
javascript: 'function f(a) { return a + 1; }\n',
typescript: 'class A { m(x: number): void {} }\n',
'typescript:tsx': 'const C = () => <div className="x">{1}</div>;\n',
python: 'def f(x):\n return x\n',
java: 'class A { void m() {} }\n',
csharp: 'class A { void M() {} }\n',
cpp: 'struct A { void m(); };\nint main() { return 0; }\n',
'objective-c': '@interface A : NSObject\n- (void)m;\n@end\n',
go: 'package main\nfunc main() {}\n',
rust: 'fn main() { let x = 1; }\n',
php: '<?php function f($a) { return $a; }\n',
ruby: 'def f(a)\n a\nend\n',
vue: 'export default { data() { return {}; } };\n',
c: 'int add(int a, int b) { return a + b; }\n',
swift: 'class A { func m() {} }\n',
dart: 'void main() { print(1); }\n',
kotlin: 'class A {\n fun m() {}\n}\n',
zig: 'pub fn main() void {}\n',
};
async function check(installedDir) {
const load = (rel) => import(pathToFileURL(path.join(installedDir, rel)).href);
const Parser = createRequire(path.join(installedDir, 'package.json'))('tree-sitter');
const { listGrammarSources, getLanguageGrammar } = await load(
'dist/core/tree-sitter/parser-loader.js',
);
const failures = [];
const keys = listGrammarSources().map((s) => s.key);
for (const key of keys) {
const sample = SAMPLES[key];
if (sample === undefined) {
failures.push(`${key}: no sample in assert-native-parsers.mjs`);
continue;
}
try {
const [language, variant] = key.split(':');
const parser = new Parser();
parser.setLanguage(getLanguageGrammar(language, variant ? `x.${variant}` : undefined));
const root = parser.parse(sample).rootNode;
if (root.hasError || root.namedChildCount === 0) {
failures.push(`${key}: parsed with errors: ${root.toString().slice(0, 200)}`);
}
} catch (err) {
failures.push(`${key}: ${err instanceof Error ? err.message : String(err)}`);
}
}
const { PROTO_GRPC_PLUGIN } = await load('dist/core/group/extractors/grpc-patterns/proto.js');
if (PROTO_GRPC_PLUGIN === null) {
failures.push('proto: native Protobuf grammar failed to initialize');
} else {
const parser = new Parser();
parser.setLanguage(PROTO_GRPC_PLUGIN.language);
const found = PROTO_GRPC_PLUGIN.scan(
parser.parse('syntax = "proto3";\nservice S { rpc M (R) returns (R); }\n'),
).map((d) => d.symbolName);
if (found.join() !== 'S.M') failures.push(`proto: expected [S.M], got [${found}]`);
}
return { checked: keys.length + 1, failures };
}
if (!isMainThread) {
parentPort.postMessage(await check(workerData));
} else {
const installedDir = path.resolve(process.argv[2] ?? '.');
const main = await check(installedDir);
const worker = await new Promise((resolve, reject) => {
const w = new Worker(new URL(import.meta.url), { workerData: installedDir });
w.once('message', resolve);
w.once('error', reject);
});
const failures = [
...main.failures.map((f) => `[main] ${f}`),
...worker.failures.map((f) => `[worker] ${f}`),
];
if (failures.length > 0) {
console.error(`[native-parsers] ${failures.length} failure(s):\n${failures.join('\n')}`);
process.exit(1);
}
console.log(
`[native-parsers] ${main.checked} grammar(s) parsed cleanly on the main thread and in a worker`,
);
}

95
gitnexus/scripts/prepare-tree-sitter-bundle.cjs Normal file
View file

@ -0,0 +1,95 @@
#!/usr/bin/env node
/**
* Prepare the tested native parser bundle for npm pack/publish.
*
* A dependency's npm overrides do not apply in a consumer project. Several
* ABI-compatible grammars still advertise older runtime peers, so bundling
* alone leaves an invalid dependency tree. Admit only the runtime version we
* tested, in only the audited manifests below. Sources and binaries are intact.
*/
const fs = require('node:fs');
const path = require('node:path');
const RUNTIME = '0.25.1';
const AUDITED_PEERS = new Map([
['tree-sitter-c@0.23.6', '^0.22.1'],
['tree-sitter-cpp@0.23.4', '^0.21.1'],
['tree-sitter-java@0.23.5', '^0.21.1'],
['tree-sitter-php@0.24.2', '^0.22.4'],
['tree-sitter-ruby@0.23.1', '^0.21.1'],
['tree-sitter-rust@0.24.0', '^0.22.1'],
['tree-sitter-javascript@0.23.1', '^0.21.1'],
['tree-sitter-typescript@0.23.2', '^0.21.0'],
]);
const readJson = (file) => JSON.parse(fs.readFileSync(file, 'utf8'));
function prepareTreeSitterBundle(packageRoot = path.resolve(__dirname, '..')) {
const modulesRoot = path.join(packageRoot, 'node_modules');
const manifest = readJson(path.join(packageRoot, 'package.json'));
if (manifest.dependencies['tree-sitter'] !== RUNTIME) {
throw new Error(`Tree-sitter bundle is only validated for runtime ${RUNTIME}`);
}
const grammars = Object.keys(manifest.dependencies).filter(
(name) => name === 'tree-sitter' || name.startsWith('tree-sitter-'),
);
for (const name of grammars) {
if (
!Array.isArray(manifest.bundleDependencies) ||
!manifest.bundleDependencies.includes(name)
) {
throw new Error(`${name} must be bundled; consumers do not inherit npm overrides`);
}
const installed = readJson(path.join(modulesRoot, name, 'package.json'));
if (installed.version !== manifest.dependencies[name]) {
throw new Error(`${name}: installed version differs from the exact package.json pin`);
}
}
const patches = [];
function visit(modules) {
if (!fs.existsSync(modules)) return;
for (const entry of fs.readdirSync(modules, { withFileTypes: true })) {
if (!entry.isDirectory() || entry.name.startsWith('.')) continue;
const dir = path.join(modules, entry.name);
if (entry.name.startsWith('@')) {
visit(dir);
continue;
}
const file = path.join(dir, 'package.json');
if (fs.existsSync(file)) {
const pkg = readJson(file);
const original = AUDITED_PEERS.get(`${pkg.name}@${pkg.version}`);
if (original) {
const peer = `${original} || ${RUNTIME}`;
if (![original, peer].includes(pkg.peerDependencies?.['tree-sitter'])) {
throw new Error(`Unexpected tree-sitter peer metadata for ${pkg.name}@${pkg.version}`);
}
pkg.peerDependencies['tree-sitter'] = peer;
patches.push({ file, pkg });
}
}
visit(path.join(dir, 'node_modules'));
}
}
visit(modulesRoot);
const identities = new Set(patches.map(({ pkg }) => `${pkg.name}@${pkg.version}`));
if (patches.length !== AUDITED_PEERS.size || identities.size !== AUDITED_PEERS.size) {
throw new Error('Tree-sitter dependency layout changed; revalidate the audited peer manifests');
}
// Validate the complete set before writing, so dependency drift cannot leave
// a partially prepared bundle behind. Repeated pack operations are idempotent.
for (const { file, pkg } of patches) {
fs.writeFileSync(file, JSON.stringify(pkg, null, 2) + '\n');
}
return patches.map(({ pkg }) => `${pkg.name}@${pkg.version}`);
}
if (require.main === module) {
const prepared = prepareTreeSitterBundle();
console.log(
`[tree-sitter-bundle] Validated runtime ${RUNTIME}; prepared ${prepared.length} peer manifests.`,
);
}
module.exports = { prepareTreeSitterBundle };

15
gitnexus/src/core/group/extractors/grpc-extractor.ts
View file

@ -135,9 +135,8 @@ function extractServiceBlocks(content: string): Array<{ name: string; body: stri
const results: Array<{ name: string; body: string }> = [];
// Sanitize comments and string literals so braces inside them don't
// throw off the depth counter. The sanitized copy has the same length
// and offsets as the original, so we use it ONLY to scan for service
// headers and braces; the service body we return is sliced from the
// ORIGINAL content to preserve exact source text for downstream use.
// and offsets as the original. Return sanitized bodies too, so downstream
// RPC matching cannot mistake commented-out declarations or strings for code.
const sanitized = stripProtoCommentsAndStrings(content);
const headerRe = /service\s+(\w+)\s*\{/g;
let headerMatch: RegExpExecArray | null;
@ -159,7 +158,7 @@ function extractServiceBlocks(content: string): Array<{ name: string; body: stri
if (depth !== 0) continue;
// body is between opening { (consumed by regex) and closing } (pos is one past it)
const body = content.slice(bodyStart, pos - 1);
const body = sanitized.slice(bodyStart, pos - 1);
results.push({ name: serviceName, body });
}
@ -655,7 +654,13 @@ export class GrpcExtractor implements ContractExtractor {
// Steps 4 + 5: legacy per-repo proto map resolution (no import).
const candidates = protoMap.get(d.serviceName) ?? [];
const proto = resolveProtoConflict(d.serviceName, filePath, candidates);
// A proto detection already knows its defining file; the directory
// heuristic is only for sources that name the service by short name.
const ownProto =
d.source === 'proto'
? candidates.find((c) => c.protoPath === normalizeProtoPath(filePath))
: undefined;
const proto = ownProto ?? resolveProtoConflict(d.serviceName, filePath, candidates);
// If there were proto candidates but resolution was ambiguous, skip
// contract emission rather than fabricating a package-qualified id from
// an arbitrary candidate. resolveProtoConflict already warned.

2
gitnexus/src/core/group/extractors/grpc-patterns/node.ts
View file

@ -123,7 +123,7 @@ interface NodeGrpcPatternBundle {
loadPackageDefinition: CompiledPatterns<Record<string, never>>;
}
function compileBundle(language: unknown, name: string): NodeGrpcPatternBundle {
function compileBundle(language: Parser.Language, name: string): NodeGrpcPatternBundle {
const mk = (
spec: PatternSpec<Record<string, never>> | PatternSpec<Record<string, never>>[],
suffix: string,

28
gitnexus/src/core/group/extractors/grpc-patterns/proto.ts
View file

@ -1,4 +1,5 @@
import { createRequire } from 'node:module';
import type Parser from 'tree-sitter';
import { requireVendoredGrammar } from '../../../tree-sitter/vendored-grammars.js';
import {
compilePatterns,
@ -17,19 +18,27 @@ import type { GrpcDetection, GrpcLanguagePlugin } from './types.js';
* (e.g. no prebuild for an unusual platform), the plugin exports `null` and the
* orchestrator falls back to the existing manual string-sanitizing parser.
*
* The grammar is vendored in `vendor/tree-sitter-proto/` with
* parser.c regenerated against tree-sitter-cli 0.24 (ABI version 14)
* so it is compatible with the project's tree-sitter 0.21.1 runtime
* (which loads ABI 13–14).
* The upstream-generated grammar is vendored in `vendor/tree-sitter-proto/`.
*/
// Only for `tree-sitter` (a real npm dependency) in the smoke-test below;
// the vendored grammar goes through requireVendoredGrammar (never a bare
// `_require('tree-sitter-proto')`, which would force a node_modules copy — #2111).
const _require = createRequire(import.meta.url);
let ProtoGrammar: unknown = null;
let ProtoGrammar: Parser.Language | null = null;
try {
ProtoGrammar = requireVendoredGrammar('tree-sitter-proto');
const grammar = requireVendoredGrammar('tree-sitter-proto');
// node-tree-sitter 0.25.1 derives JS subclass names from nodeTypeInfo.
// Protobuf's named `syntax` node becomes `class SyntaxNode extends
// SyntaxNode`, which throws in setLanguage. Omit only its subclass metadata
// on a private wrapper: the native node, queries, and childForFieldName()
// still work through the base SyntaxNode API. Do not alter upstream sources
// or the shared grammar's metadata. Remove when node-tree-sitter fixes the
// subclass-name collision.
ProtoGrammar = {
...grammar,
nodeTypeInfo: grammar.nodeTypeInfo.filter((node) => !(node.named && node.type === 'syntax')),
};
} catch {
// Grammar not installed — PROTO_GRPC_PLUGIN will be null.
}
@ -40,11 +49,8 @@ let SERVICE_PATTERNS: CompiledPatterns<Record<string, never>> | null = null;
if (ProtoGrammar) {
try {
// Validate that the grammar actually loads end-to-end: compile queries
// AND parse + walk a trivial proto file. tree-sitter's internal
// `initializeLanguageNodeClasses` can fail with a TDZ error in some
// test runners (vitest forks) when SyntaxNode isn't fully initialized
// yet. Catching that here ensures `PROTO_GRPC_PLUGIN` stays null and
// the orchestrator falls back to the manual parser.
// AND parse + walk a trivial proto file. Keep the manual fallback for
// platforms where the native grammar cannot initialize.
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const _Parser = _require('tree-sitter') as any;
// Smoke-test: parse + setLanguage to verify the grammar is

5
gitnexus/src/core/group/extractors/grpc-patterns/types.ts
View file

@ -56,11 +56,10 @@ export interface GrpcDetection {
* `GrpcDetection`s. The plugin is free to run multiple compiled query
* bundles and walk the AST to cross-reference captures.
*
* `language` is typed `unknown` for the same reason as in
* `tree-sitter-scanner.ts`.
* `language` is the concrete grammar selected from its package exports.
*/
export interface GrpcLanguagePlugin {
name: string;
language: unknown;
language: Parser.Language;
scan(tree: Parser.Tree): GrpcDetection[];
}

5
gitnexus/src/core/group/extractors/http-patterns/go.ts
View file

@ -400,7 +400,9 @@ function closureWritesName(stmt: Parser.SyntaxNode, name: string): boolean {
* when the block leaves `name` alone.
*/
function bareBlockWrite(block: Parser.SyntaxNode, name: string): Binding {
const stmts = codeChildren(block);
// tree-sitter-go 0.25 exposes a statement_list between a block and its
// statements; older grammars put those statements directly in the block.
const stmts = codeChildren(block.namedChildren.find((c) => c.type === 'statement_list') ?? block);
const declIndex = stmts.findIndex(
(s) =>
(s.type === 'short_var_declaration' || s.type === 'var_declaration') &&
@ -492,6 +494,7 @@ function lookupBinding(ident: Parser.SyntaxNode, from?: Parser.SyntaxNode | null
}
if (
node.type === 'block' ||
node.type === 'statement_list' ||
node.type === 'expression_case' ||
node.type === 'type_case' ||
node.type === 'communication_case' ||

4
gitnexus/src/core/group/extractors/http-patterns/kotlin.ts
View file

@ -125,7 +125,7 @@ import {
*/
/** Loaded lazily; null when the grammar binding isn't available. */
let Kotlin: unknown | null = null;
let Kotlin: Parser.Language | null = null;
try {
Kotlin = requireVendoredGrammar('tree-sitter-kotlin');
} catch {
@ -462,7 +462,7 @@ function enclosingAnnotationText(node: Parser.SyntaxNode): string {
* the queries against a null grammar would throw at module load time
* and abort the whole http-route-extractor module.
*/
function buildKotlinPlugin(language: unknown): HttpLanguagePlugin {
function buildKotlinPlugin(language: Parser.Language): HttpLanguagePlugin {
// ─── Provider: Spring class-level @RequestMapping prefix ──────────────
// Two patterns mirror the Java plugin's positional vs named split:
// @RequestMapping("/api") → value_argument has string_literal as its first named child

2
gitnexus/src/core/group/extractors/http-patterns/node.ts
View file

@ -195,7 +195,7 @@ interface NodePatternBundle {
requestObject: CompiledPatterns<Record<string, never>>;
}
function compileBundle(language: unknown, name: string): NodePatternBundle {
function compileBundle(language: Parser.Language, name: string): NodePatternBundle {
const mk = (spec: PatternSpec<Record<string, never>>, suffix: string) =>
compilePatterns({
name: `${name}-${suffix}`,

6
gitnexus/src/core/group/extractors/http-patterns/types.ts
View file

@ -93,9 +93,7 @@ export interface HttpFileDetections {
* compiled pattern bundles internally (see the shared scanner's
* `runCompiledPatterns` helper).
*
* `language` is typed as `unknown` for the same reason as
* `LanguagePatterns.language` in `tree-sitter-scanner.ts` — the
* grammar modules export different shapes.
* `language` is the concrete grammar selected from its package exports.
*/
/**
* Per-repo state a plugin can build during a `prepareRepo` pass before
@ -113,7 +111,7 @@ export interface HttpLanguagePlugin {
/** Human-readable plugin name for diagnostics. */
name: string;
/** tree-sitter grammar object (passed to the shared parser). */
language: unknown;
language: Parser.Language;
/**
* Whether ingestion is known to emit a `Route` graph node for EVERY
* provider route in this language (Spring/FastAPI/Laravel annotations are

4
gitnexus/src/core/group/extractors/include-extractor.ts
View file

@ -12,7 +12,7 @@ import { requireVendoredGrammar } from '../../tree-sitter/vendored-grammars.js';
// (#2091/#2093). It may be absent on a platform without a prebuild; when the
// binding is absent, `getLanguageForFile` returns null for `.c`/`.h` so C
// include-extraction is skipped (C++ is unaffected — its binding always ships).
let C: unknown = null;
let C: Parser.Language | null = null;
try {
C = requireVendoredGrammar('tree-sitter-c');
} catch {
@ -270,7 +270,7 @@ function isHeaderFile(filePath: string): boolean {
return HEADER_EXTENSIONS.has(path.extname(filePath).toLowerCase());
}
function getLanguageForFile(filePath: string): unknown | null {
function getLanguageForFile(filePath: string): Parser.Language | null {
const ext = path.extname(filePath).toLowerCase();
switch (ext) {
case '.c':

2
gitnexus/src/core/group/extractors/thrift-patterns/types.ts
View file

@ -15,6 +15,6 @@ export interface ThriftDetection {
export interface ThriftLanguagePlugin {
name: string;
language: unknown;
language: Parser.Language;
scan(tree: Parser.Tree): ThriftDetection[];
}

11
gitnexus/src/core/group/extractors/tree-sitter-scanner.ts
View file

@ -35,17 +35,14 @@ export interface PatternSpec<TMeta> {
* A set of patterns owned by one language plugin, bound to a specific
* tree-sitter grammar.
*
* `language` is typed as `unknown` because tree-sitter's TypeScript
* declarations use `any` for the grammar object, and the grammar modules
* export different shapes (plain grammar vs. namespace with `typescript`
* / `tsx` members). Callers pass the concrete grammar object; this
* module forwards it to `parser.setLanguage` / `new Parser.Query`.
* Callers select a concrete Parser.Language from each grammar package
* before passing it to `parser.setLanguage` / `new Parser.Query`.
*/
export interface LanguagePatterns<TMeta> {
/** Human-readable plugin name for diagnostics. */
name: string;
/** tree-sitter grammar object. */
language: unknown;
language: Parser.Language;
/** Patterns authored against `language`. */
patterns: PatternSpec<TMeta>[];
}
@ -58,7 +55,7 @@ export interface LanguagePatterns<TMeta> {
*/
export interface CompiledPatterns<TMeta> {
name: string;
language: unknown;
language: Parser.Language;
patterns: CompiledPattern<TMeta>[];
}

35
gitnexus/src/core/ingestion/cfg/visitors/dart.ts
View file

@ -52,10 +52,9 @@
* (`break [label] ;`), `continue_statement` (`continue [label] ;`),
* `throw_expression` (in an `expression_statement`), `rethrow_expression`
* (`rethrow ;`), `assert_statement` (`assert ( … ) ;` — may throw).
* - a labeled LOOP (`outer: for …`) parses as a stray `ERROR [identifier :]`
* SIBLING immediately before the `for_statement` (tree-sitter-dart does not
* model a statement label outside a switch); the visitor reads that ERROR
* sibling as a pending label, so `break outer` still resolves.
* - `labeled_statement` wraps a label and its statement. Nested labels name
* the same loop/switch frame, or a labeled block's exit. An ERROR-sibling
* recovery path remains for older or malformed label syntax.
*
* Edge-kind contract (matches the existing visitors — RD/CDG consume these):
* - if / else → `cond-true` / `cond-false`
@ -143,6 +142,7 @@ const CONTROL_FLOW_TYPES = new Set([
'break_statement',
'continue_statement',
'assert_statement',
'labeled_statement',
]);
/** Comment node types tree-sitter-dart surfaces. */
@ -282,6 +282,33 @@ class DartCfgWalk {
if (isThrowStatement(stmt)) return this.visitThrow(stmt);
if (isRethrowStatement(stmt)) return this.visitRethrow(stmt);
switch (stmt.type) {
case 'labeled_statement': {
// Upstream now wraps labels and their statement. Collect nested labels
// before dispatch so each one names the same loop/switch or block.
const labels = this.takeLabels();
let body: SyntaxNode | undefined = stmt;
while (body?.type === 'labeled_statement') {
const children = body.namedChildren;
const label = children.find((child) => child.type === 'identifier');
if (label) labels.push(label.text);
body = children.find((child) => child.id !== label?.id && !isComment(child));
}
if (!body) return null;
if (
['for_statement', 'while_statement', 'do_statement', 'switch_statement'].includes(
body.type,
)
) {
this.pendingLabels = labels;
return this.visitStmt(body);
}
const join = this.builder.newBlock(endLineOf(stmt), endLineOf(stmt), '');
this.cfc.pushLabeledBlock(join, labels);
const result = this.visitStmt(body);
this.cfc.pop();
if (result) this.builder.connect(result.exits, join, 'seq');
return { entry: result?.entry ?? join, exits: [join] };
}
case 'local_variable_declaration': {
// `var x = switch (v) { … }` (#2207): the value is a value-position
// branch — model it as control flow and bind the result on the rejoin.

9
gitnexus/src/core/ingestion/cfg/visitors/go.ts
View file

@ -185,11 +185,18 @@ class GoCfgWalk {
/** Wire a sequence of statements, coalescing straight-line runs into blocks. */
visitSeq(stmts: SyntaxNode[]): SeqResult {
return this.builder.withNesting(() => {
// Go 0.25 wraps block and case bodies in statement_list. Unwrap at the
// sequence boundary so control statements retain their own CFG edges.
const statements = stmts.flatMap((stmt) =>
stmt.type === 'statement_list'
? stmt.namedChildren.filter((child) => child.type !== 'comment')
: [stmt],
);
let entry: number | undefined;
let dangling: number[] = [];
let openSimple: number | undefined;
for (const stmt of stmts) {
for (const stmt of statements) {
if (CONTROL_FLOW_TYPES.has(stmt.type)) {
openSimple = undefined; // close any open straight-line block
const res = this.visitStmt(stmt);

6
gitnexus/src/core/ingestion/cfg/visitors/python.ts
View file

@ -33,8 +33,8 @@
* - `with_statement` field `body`; a `with_clause` of `with_item`s (field
* `value` = `as_pattern` or a bare expression). `__exit__` runs on normal AND
* exception exit — modeled as a finalizer (try/finally analogue).
* - `try_statement` field `body`; children `except_clause` /
* `except_group_clause` (each holds the exception expr/`as_pattern` + a
* - `try_statement` field `body`; children `except_clause` (including
* `except*`; each holds the exception expr/`as_pattern` + a
* `block`), `else_clause` (field `body`, runs if NO exception), and
* `finally_clause` (holds a `block`).
* - `match_statement` fields `subject`/`body`; the `body` `block` holds
@ -553,7 +553,7 @@ class PythonCfgWalk {
for (let i = 0; i < stmt.namedChildCount; i++) {
const c = stmt.namedChild(i);
if (!c) continue;
if (c.type === 'except_clause' || c.type === 'except_group_clause') exceptClauses.push(c);
if (c.type === 'except_clause') exceptClauses.push(c);
else if (c.type === 'else_clause') elseClause = c;
else if (c.type === 'finally_clause') finallyClause = c;
}

2
gitnexus/src/core/ingestion/language-provider.ts
View file

@ -300,7 +300,7 @@ interface LanguageProviderConfig {
readonly preprocessSource?: (sourceText: string, filePath: string) => string;
/** Select the native grammar for a pre-extracted embedded script. */
readonly selectEmbeddedGrammar?: (scriptLanguage: string) => object;
readonly selectEmbeddedGrammar?: (scriptLanguage: string) => Parser.Language;
/**
* Runtime/compiler identity reconciliation for framework metadata. The

12
gitnexus/src/core/ingestion/languages/csharp/query.ts
View file

@ -32,7 +32,8 @@
*/
import Parser from 'tree-sitter';
import CSharp from 'tree-sitter-c-sharp';
import { SupportedLanguages } from 'gitnexus-shared';
import { getLanguageGrammar } from '../../../tree-sitter/parser-loader.js';
const CSHARP_SCOPE_QUERY = `
;; Scopes
@ -545,15 +546,18 @@ let _query: Parser.Query | null = null;
export function getCsharpParser(): Parser {
if (_parser === null) {
_parser = new Parser();
_parser.setLanguage(CSharp as Parameters<Parser['setLanguage']>[0]);
const parser = new Parser();
// The upstream ESM wrapper uses top-level await. Keep the provider on the
// same synchronous native loader as workers and the compiled CLI.
parser.setLanguage(getLanguageGrammar(SupportedLanguages.CSharp));
_parser = parser;
}
return _parser;
}
export function getCsharpScopeQuery(): Parser.Query {
if (_query === null) {
_query = new Parser.Query(CSharp as Parameters<Parser['setLanguage']>[0], CSHARP_SCOPE_QUERY);
_query = new Parser.Query(getLanguageGrammar(SupportedLanguages.CSharp), CSHARP_SCOPE_QUERY);
}
return _query;
}

54
gitnexus/src/core/ingestion/languages/typescript/captures.ts
View file

@ -742,6 +742,23 @@ export function emitTsScopeCaptures(
return out;
}
/** Select matching nodes in the existing parent-first, right-to-left order. */
function relevantNodesRightFirst(root: SyntaxNode, types: readonly string[]): SyntaxNode[] {
const candidates = root.descendantsOfType([...types]).map((node, ordinal) => ({
node,
ordinal,
start: node.startIndex,
end: node.endIndex,
}));
candidates.sort((a, b) => {
if (a.start === b.start && a.end === b.end) return a.ordinal - b.ordinal;
if (a.start <= b.start && a.end >= b.end) return -1;
if (b.start <= a.start && b.end >= a.end) return 1;
return b.start - a.start;
});
return candidates.map(({ node }) => node);
}
/**
* Synthesize `@reference.inherits` captures from TypeScript class heritage so
* the registry-primary scope-resolution path emits EXTENDS / IMPLEMENTS edges
@ -778,14 +795,11 @@ export function emitTsScopeCaptures(
* `models.Base` → `Base`) so `findClassBindingInScope` resolves it.
*/
function synthesizeTsInheritanceReferences(root: SyntaxNode, out: CaptureMatch[]): void {
const stack: SyntaxNode[] = [root];
for (;;) {
const node = stack.pop();
if (node === undefined) break;
for (const child of node.namedChildren) {
if (child !== null) stack.push(child);
}
for (const node of relevantNodesRightFirst(root, [
'interface_declaration',
'class_declaration',
'abstract_class_declaration',
])) {
// `interface B extends A, C` hangs its bases off an `extends_type_clause`
// DIRECTLY on the interface — there is no `class_heritage` wrapper, so the
// class path below cannot reach them (#2842 review). The clause's `type`
@ -898,13 +912,7 @@ function terminalTsTypeNameNode(node: SyntaxNode): SyntaxNode | null {
* Left as a follow-up optimization.
*/
function synthesizeDestructuringBindings(root: SyntaxNode, out: CaptureMatch[]): void {
const stack: SyntaxNode[] = [root];
for (;;) {
const node = stack.pop();
if (node === undefined) break;
for (const child of node.namedChildren) {
if (child !== null) stack.push(child);
}
for (const node of relevantNodesRightFirst(root, ['variable_declarator'])) {
if (node.type !== 'variable_declarator') continue;
const nameNode = node.childForFieldName('name');
const valueNode = node.childForFieldName('value');
@ -962,13 +970,7 @@ function synthesizeDestructuringBindings(root: SyntaxNode, out: CaptureMatch[]):
* Uses sentinel `__MAP_TUPLE_i__:rhs` consumed by compound-receiver.
*/
function synthesizeForOfMapTupleBindings(root: SyntaxNode, out: CaptureMatch[]): void {
const stack: SyntaxNode[] = [root];
for (;;) {
const node = stack.pop();
if (node === undefined) break;
for (const child of node.namedChildren) {
if (child !== null) stack.push(child);
}
for (const node of relevantNodesRightFirst(root, ['for_in_statement'])) {
if (node.type !== 'for_in_statement') continue;
const left = node.childForFieldName('left');
const right = node.childForFieldName('right');
@ -1011,13 +1013,7 @@ function synthesizeForOfMapTupleBindings(root: SyntaxNode, out: CaptureMatch[]):
* declared types instead.
*/
function synthesizeInstanceofNarrowings(root: SyntaxNode, out: CaptureMatch[]): void {
const stack: SyntaxNode[] = [root];
for (;;) {
const node = stack.pop();
if (node === undefined) break;
for (const child of node.namedChildren) {
if (child !== null) stack.push(child);
}
for (const node of relevantNodesRightFirst(root, ['if_statement'])) {
if (node.type !== 'if_statement') continue;
const cond = node.childForFieldName('condition');
if (cond === null) continue;

15
gitnexus/src/core/ingestion/languages/zig/captures.ts
View file

@ -734,10 +734,10 @@ function rewriteZigThisAlias(
* `extractAssignment` pairs first-identifier → last-child positionally.
* `assignment_expression` (`self.f = target`) carries real `left`/`right`
* fields and is left to the shared path.
* - `call_expression` has NO argument-list wrapper: `invoke(second)` is
* `(call_expression function: (identifier) (identifier))`. Arguments are
* every named child other than the `function` field, hence
* `extractCallArguments`.
* - `call_expression` wraps its actuals in a required `arguments:` field:
* `invoke(second)` is `(call_expression function: (identifier)
* arguments: (arguments (identifier)))`. `extractCallArguments` reads that
* node's named children (minus comments).
*
* - Zig's receiver is an EXPLICIT first parameter named `self` (the same
* convention `interpretZigTypeBinding` keys receiver typing on), and a
@ -798,10 +798,9 @@ function zigCallableCaptureOptions(
);
},
extractCallArguments: (call: SyntaxNode) => {
const callee = call.childForFieldName('function');
const explicit = call.namedChildren.filter(
(child): child is SyntaxNode =>
child !== null && child.id !== callee?.id && child.type !== 'comment',
const argumentsNode = call.namedChildren.find((child) => child?.type === 'arguments');
const explicit = (argumentsNode?.namedChildren ?? []).filter(
(child): child is SyntaxNode => child !== null && child.type !== 'comment',
);
const receiver = zigImplicitReceiver(call, fnLocalNames, hostValueNames);
return receiver === undefined ? explicit : [receiver, ...explicit];

19
gitnexus/src/core/ingestion/languages/zig/query.ts
View file

@ -391,24 +391,21 @@ export const ZIG_SCOPE_QUERY = `
;; evaluation and is deliberately NOT modelled; \`impact\` reports the shortfall
;; as \`epistemic: "lower-bound"\` instead of pretending to certainty.
;; Call ARGUMENTS. In tree-sitter-zig arguments are direct children of
;; \`call_expression\`, NOT wrapped in an \`arguments\` node (only builtins have
;; one), so the callee has to be consumed explicitly by \`function:\` — without
;; that binding the same rule also matches the callee of \`foo(bar)\` and mints a
;; USES edge duplicating the call.
;; Call ARGUMENTS. The upstream grammar wraps actuals in \`arguments\`.
;; Match inside that wrapper so the callee cannot also become a value reference.
(call_expression
function: (_)
(identifier) @reference.name @reference.value-ref)
(arguments
(identifier) @reference.name @reference.value-ref))
;; Qualified argument — \`bridge.accessor(Element.getNamespaceUri, …)\`. The
;; RECEIVER is captured alongside the member so the site carries the owner it
;; was written with; \`@reference.name\` stays the member, which is the name the
;; scope walk resolves.
(call_expression
function: (_)
(field_expression
object: (_) @reference.receiver
member: (identifier) @reference.name) @reference.value-ref)
(arguments
(field_expression
object: (_) @reference.receiver
member: (identifier) @reference.name) @reference.value-ref))
;; Const binding initialiser — \`pub const defaultHandler = onReset;\`. Both
;; anchors are load-bearing: the leading \`.\` pins the bound name to the first

8
gitnexus/src/core/ingestion/workers/parse-worker.ts
View file

@ -11,8 +11,7 @@ import TypeScript from 'tree-sitter-typescript';
import Python from 'tree-sitter-python';
import Java from 'tree-sitter-java';
import CPP from 'tree-sitter-cpp';
// Explicit subpath import — see parser-loader.ts for rationale (#1013).
import CSharp from 'tree-sitter-c-sharp/bindings/node/index.js';
import { getLanguageGrammar } from '../../tree-sitter/parser-loader.js';
import Go from 'tree-sitter-go';
import Rust from 'tree-sitter-rust';
import PHP from 'tree-sitter-php';
@ -46,6 +45,11 @@ import type {
/** Language grammar type accepted by Parser.setLanguage(). */
type TreeSitterLanguage = Parameters<typeof Parser.prototype.setLanguage>[0];
// Avoid the upstream C# wrapper's top-level await: terminating a ready worker
// can race V8's async module completion and abort the process. Reuse the
// synchronous native loader and node-type metadata used by the main thread.
const CSharp = getLanguageGrammar(SupportedLanguages.CSharp);
// ── Worker grammar loading — enforcement boundary (#2091/#2093, #2101) ───────
// The worker maintains its own grammar table (the guarded vendored-grammar
// loads below + `languageMap`) and intentionally does NOT consult the runtime

29
gitnexus/src/core/tree-sitter/parser-loader.ts
View file

@ -1,5 +1,6 @@
import Parser from 'tree-sitter';
import { createRequire } from 'node:module';
import path from 'node:path';
import { SupportedLanguages } from 'gitnexus-shared';
import { logger } from '../logger.js';
@ -81,15 +82,18 @@ const SOURCES: Record<string, GrammarSource> = {
unavailableNote:
'Java parsing requires `tree-sitter-java`. Check the install and native binding.',
},
// tree-sitter-c-sharp declares `type: "module"` with `main: "bindings/node"`
// (no extension) and no `exports` field, which triggers Node 22's DEP0151
// deprecation warning on the bare-package import. The explicit subpath
// bypasses the deprecated ESM main-field resolution. (#1013)
// 0.23.5's ESM wrapper uses top-level await. Load the same native binding
// directly to preserve the synchronous grammar contract used by providers.
[SupportedLanguages.CSharp]: {
load: () => _require('tree-sitter-c-sharp/bindings/node/index.js'),
load: () => {
const root = path.dirname(_require.resolve('tree-sitter-c-sharp/package.json'));
const grammar = _require('node-gyp-build')(root);
grammar.nodeTypeInfo = _require(path.join(root, 'src/node-types.json'));
return grammar;
},
unavailableNote:
'C# parsing requires `tree-sitter-c-sharp/bindings/node/index.js`. ' +
`If the subpath is missing, see ${ISSUES_URL}/1013.`,
'C# parsing requires the `tree-sitter-c-sharp` native binding and node-type metadata. ' +
'Check the install and platform prebuild.',
},
[SupportedLanguages.CPlusPlus]: {
load: () => _require('tree-sitter-cpp'),
@ -105,7 +109,7 @@ const SOURCES: Record<string, GrammarSource> = {
'`gitnexus/vendor/tree-sitter-objc`) could not be loaded. GitNexus ships ' +
'prebuilt binaries for supported macOS/Linux runner architectures; this usually ' +
'indicates a corrupted install or native ABI mismatch with the bundled ' +
'tree-sitter@0.21.1 runtime.',
'tree-sitter@0.25.1 runtime.',
},
[SupportedLanguages.Go]: {
load: () => _require('tree-sitter-go'),
@ -151,7 +155,7 @@ const SOURCES: Record<string, GrammarSource> = {
'`gitnexus/vendor/tree-sitter-c`) could not be loaded. GitNexus ships ' +
'prebuilt binaries for all supported platforms (win32/darwin/linux ' +
'x64+arm64, N-API), so this usually indicates a corrupted install or a ' +
'native ABI mismatch with the bundled tree-sitter@0.21.1 runtime. ' +
'native ABI mismatch with the bundled tree-sitter@0.25.1 runtime. ' +
'Try reinstalling, then re-run analyze. ' +
`If the failure persists, file details at ${ISSUES_URL}/1242.`,
},
@ -376,10 +380,13 @@ export const isGrammarRuntimeSkipped = (
return source !== undefined && isRuntimeSkippedGrammar(key, source);
};
export const getLanguageGrammar = (language: SupportedLanguages, filePath?: string): unknown => {
export const getLanguageGrammar = (
language: SupportedLanguages,
filePath?: string,
): Parser.Language => {
const key = resolveLanguageKey(language, filePath);
const result = loadGrammar(key);
if (result.ok === true) return result.grammar;
if (result.ok === true) return result.grammar as Parser.Language;
// Fatal failures throw the original underlying error (preserving stack)
// after the note has been logged. Optional failures fall through to the
// standard "Unsupported language" message that callers already handle.

11
gitnexus/src/core/tree-sitter/safe-parse.ts
View file

@ -50,7 +50,7 @@ function resolveParseTimeoutMs(): number {
}
/**
* Minimal surface of the timeout knob we depend on. tree-sitter@0.21.x
* Minimal surface of the timeout knob we depend on. Native tree-sitter@0.25.1
* exposes `setTimeoutMicros(micros)`; the parse returns `null` once the
* budget is exceeded and the parser must be `reset()` before reuse.
*/
@ -60,16 +60,15 @@ interface TimeoutCapableParser {
}
/**
* Tiny shim around the runtime's parse-interruption knob so the future
* 0.25/0.26 swap (where `setTimeoutMicros` is removed in favour of
* `Parser.Options.progressCallback`) is a single-function change.
* Tiny shim around the native runtime's parse-interruption knob. Version
* 0.25.1 still exposes setTimeoutMicros, so the runtime upgrade preserves
* the existing timeout/reset contract.
*
* Returns `true` when a budget was armed (caller must clear it afterwards),
* `false` when the runtime offers no interruption mechanism (older/newer
* runtimes) so the caller can skip the reset/clear dance.
*
* Only the `setTimeoutMicros` branch is implemented today; add the
* `progressCallback` branch here when the runtime moves to 0.25+.
* If a future native binding replaces this API, adapt interruption here.
*/
function armParseBudget(parser: Parser, budgetMs: number): boolean {
if (budgetMs <= 0) return false;

3
gitnexus/src/core/tree-sitter/vendored-grammars.ts
View file

@ -1,5 +1,6 @@
import { createRequire } from 'node:module';
import path from 'node:path';
import type Parser from 'tree-sitter';
import { VENDOR_ROOT } from '../vendor-root.js';
const _require = createRequire(import.meta.url);
@ -45,7 +46,7 @@ export const vendoredGrammarDir = (packageName: string): string =>
* no `node_modules` copy. (`node-gyp-build` itself IS an npm dependency and
* resolves normally from the grammar directory.)
*/
export const requireVendoredGrammar = (packageName: string): unknown => {
export const requireVendoredGrammar = (packageName: string): Parser.Language => {
// Fail loudly on a name that isn't actually vendored — a typo or a list that
// drifted out of sync (VENDORED_GRAMMAR_PACKAGES vs the CLI probe vs the build
// registry) would otherwise surface as a confusing absolute-path require miss.

2
gitnexus/test/fixtures/lang-resolution/php-typed-property-dedup/app/Services/Mixed.php vendored
View file

@ -4,7 +4,7 @@ namespace App\Services;
use App\Models\UserRepo;
class Mixed
class MixedProperties
{
// Typed property: must emit exactly one Property def named `repo`,
// zero stray Variable defs.

2
gitnexus/test/fixtures/php-captures-golden/expected-captures.json vendored
View file

@ -517,7 +517,7 @@
},
"php-typed-property-dedup/app/Services/Mixed.php": {
"captureGroups": 15,
"digest": "25262e91a1f84e6b34c794c9bc4f61771f41325bc1c3d028901fbb8334284ca2"
"digest": "e6fb3d41d7229b638f4900953f3e46c3e63200bd95f22513e8ac2d5974563cbf"
},
"php-unresolved-receiver-arity/app/Models/Handler.php": {
"captureGroups": 28,

60
gitnexus/test/fixtures/python-captures-golden/expected-captures.json vendored
View file

@ -44,24 +44,24 @@
"digest": "89564bb524c12295365d65a304418c519b4910f29c53cfe2b755bc55a5d25474"
},
"python-ambiguous/models/__init__.py": {
"captureGroups": 0,
"digest": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
"captureGroups": 1,
"digest": "c218c1a1106c1dafd85e74c3ed12292e0958fc86a245e078afd896cf190d6407"
},
"python-ambiguous/models/handler.py": {
"captureGroups": 7,
"digest": "65f156895b52d0f2f221edb0ae34294b2a4ded348ed859f47b601aee4d7e516b"
},
"python-ambiguous/other/__init__.py": {
"captureGroups": 0,
"digest": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
"captureGroups": 1,
"digest": "c218c1a1106c1dafd85e74c3ed12292e0958fc86a245e078afd896cf190d6407"
},
"python-ambiguous/other/handler.py": {
"captureGroups": 7,
"digest": "37a41af1b2d5e8b2592e9e72ce2ca224a9a0ab4d9171819a80406931c16b9f86"
},
"python-ambiguous/services/__init__.py": {
"captureGroups": 0,
"digest": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
"captureGroups": 1,
"digest": "c218c1a1106c1dafd85e74c3ed12292e0958fc86a245e078afd896cf190d6407"
},
"python-ambiguous/services/user_handler.py": {
"captureGroups": 9,
@ -240,8 +240,8 @@
"digest": "6568834a7f228e78a11b08282196e138795980aed6c55b9953cc89e87c998e52"
},
"python-django-app-imports/accounts/__init__.py": {
"captureGroups": 0,
"digest": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
"captureGroups": 1,
"digest": "c218c1a1106c1dafd85e74c3ed12292e0958fc86a245e078afd896cf190d6407"
},
"python-django-app-imports/accounts/admin.py": {
"captureGroups": 2,
@ -252,8 +252,8 @@
"digest": "3171a6359a492381bf7cf8bc13ca6cc71e6042248253f4001d835eeb9e704e30"
},
"python-django-app-imports/accounts/migrations/__init__.py": {
"captureGroups": 0,
"digest": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
"captureGroups": 1,
"digest": "c218c1a1106c1dafd85e74c3ed12292e0958fc86a245e078afd896cf190d6407"
},
"python-django-app-imports/accounts/models.py": {
"captureGroups": 8,
@ -268,8 +268,8 @@
"digest": "1f5aa2e7edb841dbf237e03005da414d4ed8d3fdfac00f891d0dec589fc9d06c"
},
"python-django-app-imports/billing/__init__.py": {
"captureGroups": 0,
"digest": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
"captureGroups": 1,
"digest": "c218c1a1106c1dafd85e74c3ed12292e0958fc86a245e078afd896cf190d6407"
},
"python-django-app-imports/billing/admin.py": {
"captureGroups": 2,
@ -280,8 +280,8 @@
"digest": "d33f67a3e9e8664195e1398cd47481dfb73064b77b47ec1d727161e555e6958d"
},
"python-django-app-imports/billing/migrations/__init__.py": {
"captureGroups": 0,
"digest": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
"captureGroups": 1,
"digest": "c218c1a1106c1dafd85e74c3ed12292e0958fc86a245e078afd896cf190d6407"
},
"python-django-app-imports/billing/models.py": {
"captureGroups": 14,
@ -296,8 +296,8 @@
"digest": "1f5aa2e7edb841dbf237e03005da414d4ed8d3fdfac00f891d0dec589fc9d06c"
},
"python-django-app-imports/config/__init__.py": {
"captureGroups": 0,
"digest": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
"captureGroups": 1,
"digest": "c218c1a1106c1dafd85e74c3ed12292e0958fc86a245e078afd896cf190d6407"
},
"python-django-app-imports/config/asgi.py": {
"captureGroups": 7,
@ -364,8 +364,8 @@
"digest": "bc8f2332e9a4a683f3fdae966a0d8a8f534a310ddc438a7a4ebc040e97d642b8"
},
"python-from-module-alias/pkg/__init__.py": {
"captureGroups": 0,
"digest": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
"captureGroups": 1,
"digest": "c218c1a1106c1dafd85e74c3ed12292e0958fc86a245e078afd896cf190d6407"
},
"python-from-module-alias/pkg/app.py": {
"captureGroups": 23,
@ -404,8 +404,8 @@
"digest": "dda8b7578e22c44f2defdaa5d7bb76c0b04b03cd7017506ef7df6810d6116ade"
},
"python-grandparent-resolution/models/__init__.py": {
"captureGroups": 0,
"digest": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
"captureGroups": 1,
"digest": "c218c1a1106c1dafd85e74c3ed12292e0958fc86a245e078afd896cf190d6407"
},
"python-grandparent-resolution/models/a.py": {
"captureGroups": 10,
@ -628,8 +628,8 @@
"digest": "cb8c5013202cea31bd75d05907e2d45087060fa9b00c0983127701012c3dc6d2"
},
"python-multi-segment-ancestor-import/backend/routers/__init__.py": {
"captureGroups": 0,
"digest": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
"captureGroups": 1,
"digest": "c218c1a1106c1dafd85e74c3ed12292e0958fc86a245e078afd896cf190d6407"
},
"python-multi-segment-ancestor-import/backend/routers/alerts.py": {
"captureGroups": 3,
@ -640,8 +640,8 @@
"digest": "6e4851c201c7437c445a2600b8ad2a64f408069d8fb82973503261ded05602ca"
},
"python-multi-segment-ancestor-import/backend/services/__init__.py": {
"captureGroups": 0,
"digest": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
"captureGroups": 1,
"digest": "c218c1a1106c1dafd85e74c3ed12292e0958fc86a245e078afd896cf190d6407"
},
"python-multi-segment-ancestor-import/backend/services/alerts.py": {
"captureGroups": 4,
@ -724,8 +724,8 @@
"digest": "b661e8965b168f16124d4682906516527e79d4c9de6f36fa1e7cf7aff9211c28"
},
"python-parent-resolution/models/__init__.py": {
"captureGroups": 0,
"digest": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
"captureGroups": 1,
"digest": "c218c1a1106c1dafd85e74c3ed12292e0958fc86a245e078afd896cf190d6407"
},
"python-parent-resolution/models/base.py": {
"captureGroups": 8,
@ -768,8 +768,8 @@
"digest": "cec30bb7577edfb751bcd4f5026a0c9d77ed235071421a2a46f87580c7bcdf47"
},
"python-qualified-base/a/__init__.py": {
"captureGroups": 0,
"digest": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
"captureGroups": 1,
"digest": "c218c1a1106c1dafd85e74c3ed12292e0958fc86a245e078afd896cf190d6407"
},
"python-qualified-base/a/b.py": {
"captureGroups": 8,
@ -860,8 +860,8 @@
"digest": "0c734dc7f683469f9c1a9eb97fd8be2af68e29aedd20cd8c6f0c86c17f7ce13e"
},
"python-super-resolution/models/__init__.py": {
"captureGroups": 0,
"digest": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
"captureGroups": 1,
"digest": "c218c1a1106c1dafd85e74c3ed12292e0958fc86a245e078afd896cf190d6407"
},
"python-super-resolution/models/base.py": {
"captureGroups": 8,

4
gitnexus/test/integration/resolvers/php.test.ts
View file

@ -2124,8 +2124,8 @@ describe('PHP typed-property double-match dedup', () => {
result = await runPipelineFromRepo(path.join(FIXTURES, 'php-typed-property-dedup'), () => {});
}, 60000);
it('detects the Mixed class', () => {
expect(getNodesByLabel(result, 'Class')).toContain('Mixed');
it('detects the MixedProperties class', () => {
expect(getNodesByLabel(result, 'Class')).toContain('MixedProperties');
});
it('emits exactly one Property def for the typed property `$repo`', () => {

53
gitnexus/test/integration/worker-pool.test.ts
View file

@ -13,7 +13,7 @@ import {
WorkerPoolDispatchError,
} from '../../src/core/ingestion/workers/worker-pool.js';
import { pathToFileURL } from 'node:url';
import { spawn } from 'node:child_process';
import { spawn, spawnSync } from 'node:child_process';
import { createRequire } from 'node:module';
import path from 'node:path';
import fs from 'node:fs';
@ -98,12 +98,55 @@ describe('worker pool integration', () => {
}
});
it.skipIf(!hasDistWorker)('creates a worker pool from dist/ worker', () => {
it.skipIf(!hasDistWorker)('creates a worker pool from dist/ worker', async () => {
const workerUrl = pathToFileURL(DIST_WORKER) as URL;
pool = createWorkerPool(workerUrl, 1);
expect(pool.size).toBe(1);
// The empty dispatch waits for the real worker's ready handshake. Do not
// let teardown interrupt ESM/native grammar initialization (#2432).
await expect(pool.dispatch([])).resolves.toEqual([]);
expect(pool.getStats().activeSlots).toBe(1);
});
it.skipIf(!hasDistWorker)(
'terminates ready parser workers repeatedly without aborting the host',
() => {
// Run outside Vitest so a native/V8 abort becomes an assertion failure
// rather than killing the test runner and hiding the remaining cases.
const runner = `
const { Worker } = require('node:worker_threads');
const { pathToFileURL } = require('node:url');
(async () => {
const workerUrl = pathToFileURL(process.argv[1]);
for (let iteration = 0; iteration < 20; iteration++) {
const worker = new Worker(workerUrl);
await new Promise((resolve, reject) => {
worker.once('error', reject);
worker.once('exit', (code) => reject(new Error('worker exited before ready: ' + code)));
worker.on('message', (message) => {
if (message.type === 'ready') resolve();
});
});
await worker.terminate();
}
process.stdout.write('20 ready/terminate cycles completed');
})().catch((error) => {
console.error(error);
process.exitCode = 1;
});
`;
const child = spawnSync(process.execPath, ['--eval', runner, DIST_WORKER], {
encoding: 'utf8',
timeout: 60_000,
});
expect(child.error).toBeUndefined();
expect(child.signal, child.stderr).toBeNull();
expect(child.status, child.stderr).toBe(0);
expect(child.stdout).toContain('20 ready/terminate cycles completed');
},
65_000,
);
it.skipIf(!hasDistWorker)('dispatches an empty batch without error', async () => {
const workerUrl = pathToFileURL(DIST_WORKER) as URL;
pool = createWorkerPool(workerUrl, 1);
@ -290,6 +333,8 @@ describe('worker pool integration', () => {
it.skipIf(!hasDistWorker)('terminates cleanly', async () => {
const workerUrl = pathToFileURL(DIST_WORKER) as URL;
pool = createWorkerPool(workerUrl, 2);
await expect(pool.dispatch([])).resolves.toEqual([]);
expect(pool.getStats().activeSlots).toBe(2);
await pool.terminate();
pool = undefined; // already terminated
});
@ -308,6 +353,8 @@ describe('worker pool integration', () => {
const workerUrl = pathToFileURL(DIST_WORKER) as URL;
pool = createWorkerPool(workerUrl, 1);
const terminatedPool = pool;
await expect(terminatedPool.dispatch([])).resolves.toEqual([]);
expect(terminatedPool.getStats().activeSlots).toBe(1);
await terminatedPool.terminate();
pool = undefined; // already terminated — prevent afterEach double-terminate
@ -319,6 +366,8 @@ describe('worker pool integration', () => {
it.skipIf(!hasDistWorker)('double terminate does not throw', async () => {
const workerUrl = pathToFileURL(DIST_WORKER) as URL;
pool = createWorkerPool(workerUrl, 1);
await expect(pool.dispatch([])).resolves.toEqual([]);
expect(pool.getStats().activeSlots).toBe(1);
await pool.terminate();
await expect(pool.terminate()).resolves.toBeUndefined();
pool = undefined;

2
gitnexus/test/unit/call-extraction.test.ts
View file

@ -28,7 +28,7 @@ import Parser from 'tree-sitter';
import TypeScript from 'tree-sitter-typescript';
import Python from 'tree-sitter-python';
import Java from 'tree-sitter-java';
import CSharp from 'tree-sitter-c-sharp';
import CSharp from 'tree-sitter-c-sharp/bindings/node/index.js';
import Go from 'tree-sitter-go';
import Rust from 'tree-sitter-rust';
import CPP from 'tree-sitter-cpp';

2
gitnexus/test/unit/call-form.test.ts
View file

@ -9,7 +9,7 @@ import Parser from 'tree-sitter';
import TypeScript from 'tree-sitter-typescript';
import Python from 'tree-sitter-python';
import Java from 'tree-sitter-java';
import CSharp from 'tree-sitter-c-sharp';
import CSharp from 'tree-sitter-c-sharp/bindings/node/index.js';
import { requireVendoredGrammar } from '../../src/core/tree-sitter/vendored-grammars.js';
import Go from 'tree-sitter-go';
import Rust from 'tree-sitter-rust';

8
gitnexus/test/unit/cfg/csharp-visitor.test.ts
View file

@ -1,5 +1,5 @@
import { describe, it, expect, vi } from 'vitest';
import { createRequire } from 'node:module';
import csGrammar from 'tree-sitter-c-sharp/bindings/node/index.js';
import { createCsharpCfgVisitor } from '../../../src/core/ingestion/cfg/visitors/csharp.js';
import type { FunctionCfg } from '../../../src/core/ingestion/cfg/types.js';
import {
@ -22,12 +22,6 @@ import { computeControlDependence } from '../../../src/core/ingestion/cfg/contro
// done(), handle(e), …) lets us locate the block for a region by text and assert
// the control-flow topology around it.
// tree-sitter-c-sharp declares `main: "bindings/node"` (no extension) — load the
// explicit subpath, mirroring parser-loader.ts (#1013).
const csGrammar = createRequire(import.meta.url)(
'tree-sitter-c-sharp/bindings/node/index.js',
) as Parameters<typeof makeCfgHarness>[0];
const cs: CfgHarness = makeCfgHarness(csGrammar, createCsharpCfgVisitor(), 'fixture.cs');
const hasDef = (cfg: FunctionCfg, idx: number): boolean =>

33
gitnexus/test/unit/cfg/dart-visitor.test.ts
View file

@ -412,6 +412,39 @@ describe('Dart CfgVisitor — try/on/catch/finally', () => {
});
describe('Dart CfgVisitor — labeled break/continue', () => {
it('keeps both labels on a loop and continues to that loop from a nested loop', () => {
const cfg = dart.cfgOf(`void f(List xs, List ys, bool c) {
outer: inner: for (var i in xs) {
for (var j in ys) { if (c) { continue outer; } else { continue inner; } }
}
done();
}`);
// Exact edges, not reachability: a jump retargeted to the nested loop still
// reaches the outer header through that loop's exit.
const header = block(cfg, 'for (var i in xs)');
for (const jump of ['continue outer;', 'continue inner;']) {
expect(cfg.edges, jump).toContainEqual({
from: block(cfg, jump),
to: header,
kind: 'continue',
});
}
});
it('breaks out of a labeled block without attaching its label to a nested loop', () => {
const cfg = dart.cfgOf(`void f(List xs) {
outer: {
for (var i in xs) { break outer; }
skipped();
}
done();
}`);
const jump = block(cfg, 'break outer;');
expect(edgeKinds(cfg).has('break')).toBe(true);
expect(reaches(cfg, jump, block(cfg, 'done();'))).toBe(true);
expect(reaches(cfg, jump, block(cfg, 'skipped();'))).toBe(false);
});
it('labeled `break outer` escapes BOTH loops and reaches done()', () => {
const cfg = dart.cfgOf(`void f(List xs, List ys) {
outer: for (var i in xs) {

24
gitnexus/test/unit/cli-commands.test.ts
View file

@ -137,12 +137,12 @@ describe('CLI commands', () => {
const swiftPkg = await import('../../vendor/tree-sitter-swift/package.json', {
with: { type: 'json' },
});
// Exact pin (no caret) — #1922 holds the runtime at 0.21.1 so the ABI
// gate's assumptions (setTimeoutMicros semantics, ABI 13–14 grammar
// Exact pin (no caret) — hold the tested runtime at 0.25.1 so the ABI
// gate's assumptions (setTimeoutMicros semantics, ABI 13–15 grammar
// range) can't drift under a minor bump.
expect(pkg.default.dependencies['tree-sitter']).toBe('0.21.1');
expect(pkg.default.dependencies['tree-sitter']).toBe('0.25.1');
expect(pkg.default.scripts.postinstall).toContain('build-tree-sitter-grammars.cjs');
expect(swiftPkg.default.version).toBe('0.7.1');
expect(swiftPkg.default.version).toBe('0.7.2');
// No scripts.install / dependencies inside vendor/ (#836 / #1728 hygiene).
expect(
(swiftPkg.default as { scripts?: Record<string, string> }).scripts?.install,
@ -150,7 +150,7 @@ describe('CLI commands', () => {
expect(
(swiftPkg.default as { dependencies?: Record<string, string> }).dependencies,
).toBeUndefined();
expect(swiftPkg.default.peerDependencies['tree-sitter']).toContain('^0.21.1');
expect(swiftPkg.default.peerDependencies['tree-sitter']).toBe('0.25.1');
// Swift is now unified with Dart/Proto/Kotlin/C: the grammar SOURCE is
// vendored so build-tree-sitter-grammars.cjs can source-build the binding
// when no committed prebuild matches (e.g. CI before prebuilds land).
@ -178,7 +178,7 @@ describe('CLI commands', () => {
// ships source only) and loaded from vendor/ by absolute path (#2111).
expect(optional['tree-sitter-kotlin']).toBeUndefined();
expect(pkg.default.scripts.postinstall).toContain('build-tree-sitter-grammars.cjs');
expect(kotlinPkg.default.version).toBe('0.4.0');
expect(kotlinPkg.default.version).toBe('0.4.0-g1852ea1');
// No scripts.install / dependencies inside vendor/ (#836 / #1728 hygiene).
expect(
(kotlinPkg.default as { scripts?: Record<string, string> }).scripts?.install,
@ -186,7 +186,7 @@ describe('CLI commands', () => {
expect(
(kotlinPkg.default as { dependencies?: Record<string, string> }).dependencies,
).toBeUndefined();
expect(kotlinPkg.default.peerDependencies['tree-sitter']).toContain('^0.21');
expect(kotlinPkg.default.peerDependencies['tree-sitter']).toBe('0.25.1');
});
it('vendors tree-sitter-zig instead of an npm optionalDependency', async () => {
@ -202,27 +202,27 @@ describe('CLI commands', () => {
'@tree-sitter-grammars/tree-sitter-zig',
);
expect(pkg.default.scripts.postinstall).toContain('build-tree-sitter-grammars.cjs');
expect(zigPkg.default.version).toBe('1.1.2');
expect(zigPkg.default.version).toBe('1.1.2-g6479aa1');
expect(
(zigPkg.default as { scripts?: Record<string, string> }).scripts?.install,
).toBeUndefined();
expect(
(zigPkg.default as { dependencies?: Record<string, string> }).dependencies,
).toBeUndefined();
expect(zigPkg.default.peerDependencies['tree-sitter']).toContain('^0.21');
expect(zigPkg.default.peerDependencies['tree-sitter']).toBe('0.25.1');
});
it('vendors tree-sitter-c prebuild-only at the 0.21.4 ABI pin instead of an npm dependency (#2116/#1242)', async () => {
it('vendors tree-sitter-c at the tested upstream snapshot instead of an npm dependency (#2116/#1242)', async () => {
const pkg = await import('../../package.json', { with: { type: 'json' } });
const cPkg = await import('../../vendor/tree-sitter-c/package.json', {
with: { type: 'json' },
});
// c is a REQUIRED grammar that hard-fails install on toolchain-less ARM
// (upstream ships 4/6). Vendored with GitNexus-built prebuilds for all 6,
// held at 0.21.4 for ABI safety (#1242) — so it is NOT an npm dependency.
// sourced from the tested upstream snapshot rather than an npm dependency.
expect(pkg.default.dependencies['tree-sitter-c']).toBeUndefined();
expect(pkg.default.scripts.postinstall).toContain('build-tree-sitter-grammars.cjs');
expect(cPkg.default.version).toBe('0.21.4');
expect(cPkg.default.version).toBe('0.24.2-gb780e47');
expect(
(cPkg.default as { scripts?: Record<string, string> }).scripts?.install,
).toBeUndefined();

6
gitnexus/test/unit/field-extraction.test.ts
View file

@ -19,20 +19,20 @@ import Go from 'tree-sitter-go';
import Cpp from 'tree-sitter-cpp';
import Ruby from 'tree-sitter-ruby';
import Java from 'tree-sitter-java';
import CSharp from 'tree-sitter-c-sharp';
import CSharp from 'tree-sitter-c-sharp/bindings/node/index.js';
import { requireVendoredGrammar } from '../../src/core/tree-sitter/vendored-grammars.js';
// Vendored grammars — loaded from vendor/ by absolute path, never node_modules (#2111).
const Dart = requireVendoredGrammar('tree-sitter-dart');
let Kotlin: unknown;
let Kotlin: Parser.Language | null = null;
try {
Kotlin = requireVendoredGrammar('tree-sitter-kotlin');
} catch {
// Kotlin grammar may not have a prebuild for this platform
}
let Swift: unknown;
let Swift: Parser.Language | null = null;
try {
Swift = requireVendoredGrammar('tree-sitter-swift');
} catch {

61
gitnexus/test/unit/grammar-update-monitor.test.ts
View file

@ -7,8 +7,8 @@ import { fileURLToPath, pathToFileURL } from 'node:url';
/**
* Unit coverage for the ABI gate in the vendored-grammar update monitor
* (.github/scripts/update-vendored-grammars.mjs). The gate is load-bearing: every
* grammar is pinned to tree-sitter@0.21.1 (LANGUAGE_VERSION 13–14), so an update
* is only auto-applied when the candidate parser.c's ABI is 13 or 14 — otherwise
* grammar is pinned to tree-sitter@0.25.1 (LANGUAGE_VERSION 13–15), so an update
* is only auto-applied when the candidate parser.c's ABI is 13, 14 or 15 — otherwise
* the monitor would open PRs that can't build. We test the pure pieces (no
* network): reading the ABI from a parser.c and the compatibility set. The module
* is import-safe (its CLI is guarded behind an isMain check).
@ -55,7 +55,7 @@ describe('readAbi', () => {
it('reads LANGUAGE_VERSION 14 from src/parser.c', () => {
expect(mod.readAbi(fixture('#define LANGUAGE_VERSION 14'))).toBe(14);
});
it('reads LANGUAGE_VERSION 15 (an incompatible upstream)', () => {
it('reads LANGUAGE_VERSION 15 (a compatible upstream)', () => {
expect(mod.readAbi(fixture('#define LANGUAGE_VERSION 15'))).toBe(15);
});
it('returns null when parser.c is absent (generated-at-build-time grammars)', () => {
@ -64,11 +64,12 @@ describe('readAbi', () => {
});
describe('COMPATIBLE_ABI gate', () => {
it('accepts ABI 13 and 14, rejects 12 and 15', () => {
it('accepts ABI 13 through 15, rejects 12 and 16', () => {
expect(mod.COMPATIBLE_ABI.has(13)).toBe(true);
expect(mod.COMPATIBLE_ABI.has(14)).toBe(true);
expect(mod.COMPATIBLE_ABI.has(12)).toBe(false);
expect(mod.COMPATIBLE_ABI.has(15)).toBe(false);
expect(mod.COMPATIBLE_ABI.has(15)).toBe(true);
expect(mod.COMPATIBLE_ABI.has(16)).toBe(false);
});
});
@ -85,19 +86,15 @@ describe('GRAMMARS registry', () => {
]);
expect(mod.GRAMMARS.swift.npm).toBe('tree-sitter-swift');
expect(mod.GRAMMARS.dart.github).toContain('tree-sitter-dart');
expect(mod.GRAMMARS.zig.npm).toBe('@tree-sitter-grammars/tree-sitter-zig');
expect(mod.GRAMMARS.zig.github).toBe('tree-sitter-grammars/tree-sitter-zig');
});
it('marks c, kotlin, and objc report-only; swift/dart/proto/zig are auto-updatable', () => {
expect(mod.GRAMMARS.c.npm).toBe('tree-sitter-c');
expect(mod.GRAMMARS.c.hold).toBeTruthy(); // ABI-pinned: detected/reported, never auto-applied
expect(mod.GRAMMARS.objc.npm).toBe('tree-sitter-objc');
expect(mod.GRAMMARS.objc.hold).toBeTruthy();
// kotlin is pinned to an unreleased fwcd main commit for `fun interface`
// support (#169); npm latest (0.3.8) lacks it, so the strict-inequality
// isNewer would auto-revert the pin without this hold.
expect(mod.GRAMMARS.kotlin.hold).toBeTruthy();
for (const k of ['swift', 'dart', 'proto', 'zig']) {
it('holds Swift recovery regressions and tracks upstream source for the other vendors', () => {
expect(mod.GRAMMARS.c.github).toBe('tree-sitter/tree-sitter-c');
expect(mod.GRAMMARS.objc.github).toBe('tree-sitter-grammars/tree-sitter-objc');
expect(mod.GRAMMARS.kotlin.github).toBe('fwcd/tree-sitter-kotlin');
expect(mod.GRAMMARS.swift.hold).toContain('declaration-ownership');
for (const k of ['c', 'objc', 'kotlin', 'dart', 'proto', 'zig']) {
expect(mod.GRAMMARS[k].hold).toBeUndefined();
}
});
@ -182,10 +179,10 @@ describe('detect() classification (offline, injected deps)', () => {
? { version: '9.9.9', ref: '9.9.9', kind: 'npm' }
: { version: '1.0.0-gabc1234', ref: 'abc1234def0', kind: 'github' };
const deps: DetectDeps = {
vendoredVersion: (g) => (g.name === 'tree-sitter-kotlin' ? '9.9.9' : '0.0.0'),
vendoredVersion: (g) => (g.name === 'tree-sitter-kotlin' ? '1.0.0-gabc1234' : '0.0.0'),
resolveUpstream: baseResolveUpstream,
fetchSource: (g) => g.name, // pass the name through to the fake readAbi
readAbi: (name) => (name === 'tree-sitter-swift' ? 15 : 14),
readAbi: (name) => (name === 'tree-sitter-zig' ? 16 : 15),
};
let report: Array<Record<string, unknown>>;
const byKey = (k: string) =>
@ -203,30 +200,30 @@ describe('detect() classification (offline, injected deps)', () => {
});
it('does not flag a same-version grammar, and skips its ABI fetch', () => {
expect(byKey('kotlin').update).toBe(false); // vendored == upstream 9.9.9
expect(byKey('kotlin').update).toBe(false); // vendored == upstream provenance
expect(byKey('kotlin').abi).toBeNull();
expect(byKey('kotlin').applicable).toBe(false);
});
it('holds tree-sitter-c: update detected, ABI-compatible, but never applicable', () => {
const c = byKey('c');
it('holds Swift: update detected, ABI-compatible, but never applicable', () => {
const c = byKey('swift');
expect(c.update).toBe(true);
expect(c.abi).toBe(14);
expect(c.abi).toBe(15);
expect(c.abiCompatible).toBe(true);
expect(c.hold).toBeTruthy();
expect(c.applicable).toBe(false); // policy-hold gate
});
it('refuses an ABI-incompatible candidate (15) — not applicable', () => {
const s = byKey('swift');
expect(s.abi).toBe(15);
it('refuses an ABI-incompatible candidate (16) — not applicable', () => {
const s = byKey('zig');
expect(s.abi).toBe(16);
expect(s.abiCompatible).toBe(false);
expect(s.applicable).toBe(false); // ABI gate
});
it('marks a newer, un-held, ABI-14 grammar applicable', () => {
it('marks a newer, un-held, ABI-15 grammar applicable', () => {
const d = byKey('dart');
expect(d.abi).toBe(14);
expect(d.abi).toBe(15);
expect(d.applicable).toBe(true);
});
@ -306,7 +303,7 @@ describe('apply(--dry-run): resolves + validates but writes nothing', () => {
vendoredVersion: () => '0.0.0',
resolveUpstream: () => ({ version: '9.9.9', ref: '9.9.9abc', kind: 'github' }),
fetchSource: () => 'unused',
readAbi: () => 14,
readAbi: () => 15,
},
});
expect(version).toBe('9.9.9');
@ -334,18 +331,18 @@ describe('apply() error branches throw ApplyExit (CLI maps to exit codes)', () =
expect(codeOf(() => mod.apply('nope', { deps: {} }))).toBe(2);
});
it('held grammar (c) → exit code 3 (short-circuits before the newer check)', () => {
expect(codeOf(() => mod.apply('c', { deps: {} }))).toBe(3);
it('held grammar (swift) → exit code 3 (short-circuits before the newer check)', () => {
expect(codeOf(() => mod.apply('swift', { deps: {} }))).toBe(3);
});
it('ABI-incompatible candidate (15) → exit code 3', () => {
it('ABI-incompatible candidate (16) → exit code 3', () => {
const code = codeOf(() =>
mod.apply('dart', {
deps: {
vendoredVersion: () => '0.0.0', // newer than upstream → reaches the ABI gate
resolveUpstream: () => ({ version: '9.9.9', ref: '9.9.9abc', kind: 'github' }),
fetchSource: () => 'unused',
readAbi: () => 15,
readAbi: () => 16,
},
}),
);

61
gitnexus/test/unit/group/grpc-extractor.test.ts
View file

@ -3,6 +3,11 @@ import * as fs from 'node:fs';
import fsp from 'node:fs/promises';
import * as path from 'node:path';
import * as os from 'node:os';
import Parser from 'tree-sitter';
import {
PROTO_GRPC_PLUGIN,
extractPackageFromTree,
} from '../../../src/core/group/extractors/grpc-patterns/proto.js';
const { parseSourceSafeSpy } = vi.hoisted(() => ({ parseSourceSafeSpy: vi.fn() }));
@ -48,6 +53,49 @@ describe('GrpcExtractor', () => {
});
describe('proto file parsing', () => {
it('loads the native proto grammar and preserves syntax fields and service queries', () => {
expect(PROTO_GRPC_PLUGIN).not.toBeNull();
if (!PROTO_GRPC_PLUGIN) throw new Error('Vendored Protobuf grammar failed to initialize');
const plugin = PROTO_GRPC_PLUGIN;
const parser = new Parser();
parser.setLanguage(plugin.language);
const tree = parser.parse(`syntax = "proto3";
package native.v1;
service NativeService {
// rpc Removed (Request) returns (Response);
rpc Active (Request) returns (Response);
}`);
expect(tree.rootNode.hasError).toBe(false);
const syntax = tree.rootNode.namedChildren.find((node) => node.type === 'syntax');
expect(syntax?.childForFieldName('version')?.text).toBe('"proto3"');
expect(extractPackageFromTree(tree)).toBe('native.v1');
expect(plugin.scan(tree).map((detection) => detection.symbolName)).toEqual([
'NativeService.Active',
]);
});
it('excludes RPC declarations in comments and strings from the fallback map and contracts', async () => {
writeFile(
'api/commented-rpc.proto',
`syntax = "proto3";
package commented;
service Svc {
// rpc LineComment (Req) returns (Res);
/* rpc BlockComment (Req) returns (Res); */
option (description) = "rpc StringLiteral (Req) returns (Res);";
rpc Active (Req) returns (Res);
}`,
);
// buildProtoMap always uses the manual parser, even when the native
// grammar is available. Assert it independently of plugin selection.
const services = await buildProtoMap(tmpDir);
expect(services.get('Svc')?.map((service) => service.methods)).toEqual([['Active']]);
const contracts = await extractor.extract(null, tmpDir, makeRepo(tmpDir));
expect(contracts.map((contract) => contract.symbolName)).toEqual(['Svc.Active']);
});
it('test_extract_proto_service_single_rpc_returns_provider', async () => {
writeFile(
'proto/auth.proto',
@ -171,6 +219,19 @@ service ServiceB {
]);
});
it('keeps same-named services from sibling protos in different packages', async () => {
writeFile('api/a.proto', 'package a;\nservice Svc { rpc Get (R) returns (R); }');
writeFile('api/b.proto', 'package b;\nservice Svc { rpc Get (R) returns (R); }');
const contracts = await extractor.extract(null, tmpDir, makeRepo(tmpDir));
const ids = contracts
.filter((c) => c.role === 'provider')
.map((c) => `${c.symbolRef.filePath} ${c.contractId}`)
.sort();
expect(ids).toEqual(['api/a.proto grpc::a.Svc/Get', 'api/b.proto grpc::b.Svc/Get']);
});
it('test_extract_proto_with_nested_option_blocks_in_rpc', async () => {
writeFile(
'api/nested.proto',

2
gitnexus/test/unit/group/kotlin-const-route-fold.test.ts
View file

@ -49,7 +49,7 @@ import { KOTLIN_HTTP_PLUGIN } from '../../../src/core/group/extractors/http-patt
import type { HttpLanguagePlugin } from '../../../src/core/group/extractors/http-patterns/types.js';
// Vendored grammar — loaded from vendor/ by absolute path, never node_modules (#2111).
let Kotlin: unknown;
let Kotlin: Parser.Language | null = null;
try {
Kotlin = requireVendoredGrammar('tree-sitter-kotlin');
} catch {

2
gitnexus/test/unit/has-method.test.ts
View file

@ -5,7 +5,7 @@ import Python from 'tree-sitter-python';
import Java from 'tree-sitter-java';
import CPP from 'tree-sitter-cpp';
import Rust from 'tree-sitter-rust';
import CSharp from 'tree-sitter-c-sharp';
import CSharp from 'tree-sitter-c-sharp/bindings/node/index.js';
import Go from 'tree-sitter-go';
import {
findEnclosingClassId,

2
gitnexus/test/unit/kotlin-route-const-resolver.test.ts
View file

@ -43,7 +43,7 @@ import {
import { unquoteSpringLiteral } from '../../src/core/ingestion/route-extractors/spring-shared.js';
// Vendored grammar — loaded from vendor/ by absolute path, never node_modules (#2111).
let Kotlin: unknown;
let Kotlin: Parser.Language | null = null;
try {
Kotlin = requireVendoredGrammar('tree-sitter-kotlin');
} catch {

2
gitnexus/test/unit/kotlin-spring-route-ingestion.test.ts
View file

@ -16,7 +16,7 @@ import {
import { joinPath } from '../../src/core/ingestion/route-extractors/spring-shared.js';
import { KOTLIN_HTTP_PLUGIN } from '../../src/core/group/extractors/http-patterns/kotlin.js';
let Kotlin: unknown;
let Kotlin: Parser.Language | null = null;
try {
Kotlin = requireVendoredGrammar('tree-sitter-kotlin');
} catch {

2
gitnexus/test/unit/leading-doc-comment.test.ts
View file

@ -19,7 +19,7 @@ import {
// Vendored grammar — loaded from vendor/ by absolute path, never node_modules (#2111).
const Kotlin = requireVendoredGrammar('tree-sitter-kotlin');
function firstNode(language: unknown, src: string, type: string): SyntaxNode {
function firstNode(language: Parser.Language, src: string, type: string): SyntaxNode {
const parser = new Parser();
parser.setLanguage(language);
const node = parser.parse(src).rootNode.descendantsOfType(type)[0];

4
gitnexus/test/unit/leading-doc-description-all-languages.test.ts
View file

@ -17,7 +17,7 @@ import Parser from 'tree-sitter';
import TypeScript from 'tree-sitter-typescript';
import Go from 'tree-sitter-go';
import Rust from 'tree-sitter-rust';
import CSharp from 'tree-sitter-c-sharp';
import CSharp from 'tree-sitter-c-sharp/bindings/node/index.js';
import Ruby from 'tree-sitter-ruby';
import CPP from 'tree-sitter-cpp';
import PHP from 'tree-sitter-php';
@ -30,7 +30,7 @@ import type { NodeLabel } from 'gitnexus-shared';
function describeFromProvider(
language: SupportedLanguages,
grammar: unknown,
grammar: Parser.Language,
src: string,
nodeType: string,
captureKey: string,

8
gitnexus/test/unit/method-extraction.test.ts
View file

@ -25,7 +25,7 @@ import { methodInfoKey } from '../../src/core/ingestion/utils/method-props.js';
import Parser from 'tree-sitter';
import Java from 'tree-sitter-java';
import Go from 'tree-sitter-go';
import CSharp from 'tree-sitter-c-sharp';
import CSharp from 'tree-sitter-c-sharp/bindings/node/index.js';
import CPP from 'tree-sitter-cpp';
import TypeScript from 'tree-sitter-typescript';
import JavaScript from 'tree-sitter-javascript';
@ -37,14 +37,14 @@ import { SupportedLanguages } from '../../src/config/supported-languages.js';
import { requireVendoredGrammar } from '../../src/core/tree-sitter/vendored-grammars.js';
// Vendored grammars — loaded from vendor/ by absolute path, never node_modules (#2111).
let Kotlin: unknown;
let Kotlin: Parser.Language | null = null;
try {
Kotlin = requireVendoredGrammar('tree-sitter-kotlin');
} catch {
// Kotlin grammar may not be installed
}
let Dart: unknown;
let Dart: Parser.Language | null = null;
try {
Dart = requireVendoredGrammar('tree-sitter-dart');
// Verify the grammar actually works with the installed tree-sitter version
@ -54,7 +54,7 @@ try {
Dart = null;
}
let Swift: unknown;
let Swift: Parser.Language | null = null;
try {
Swift = requireVendoredGrammar('tree-sitter-swift');
// Verify the grammar actually works with the installed tree-sitter version

23
gitnexus/test/unit/parser-loader.test.ts
View file

@ -1,8 +1,29 @@
import { describe, it, expect } from 'vitest';
import { describe, it, expect, vi } from 'vitest';
import { loadParser, loadLanguage } from '../../src/core/tree-sitter/parser-loader.js';
import { SupportedLanguages } from '../../src/config/supported-languages.js';
describe('parser-loader', () => {
it('keeps reporting C# grammar initialization failures on repeated calls', async () => {
vi.resetModules();
const loader = await import('../../src/core/tree-sitter/parser-loader.js');
const failure = new Error('C# grammar failed to load');
const loadGrammar = vi.spyOn(loader, 'getLanguageGrammar').mockImplementation(() => {
throw failure;
});
try {
const { getCsharpParser } =
await import('../../src/core/ingestion/languages/csharp/query.js');
expect(() => getCsharpParser()).toThrow(failure);
expect(() => getCsharpParser()).toThrow(failure);
loadGrammar.mockRestore();
const parser = getCsharpParser();
expect(parser.parse('class Recovered {}').rootNode.hasError).toBe(false);
expect(getCsharpParser()).toBe(parser);
} finally {
loadGrammar.mockRestore();
}
});
describe('loadParser', () => {
it('returns a Parser instance', async () => {
const parser = await loadParser();

151
gitnexus/test/unit/prepare-tree-sitter-bundle.test.ts Normal file
View file

@ -0,0 +1,151 @@
import { afterEach, describe, expect, it } from 'vitest';
import { createRequire } from 'node:module';
import { mkdtempSync, mkdirSync, readFileSync, rmSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import path from 'node:path';
const require = createRequire(import.meta.url);
const { prepareTreeSitterBundle } = require('../../scripts/prepare-tree-sitter-bundle.cjs');
const roots: string[] = [];
const directVersions: Record<string, string> = {
'tree-sitter': '0.25.1',
'tree-sitter-c-sharp': '0.23.5',
'tree-sitter-cpp': '0.23.4',
'tree-sitter-go': '0.25.0',
'tree-sitter-java': '0.23.5',
'tree-sitter-javascript': '0.25.0',
'tree-sitter-php': '0.24.2',
'tree-sitter-python': '0.25.0',
'tree-sitter-ruby': '0.23.1',
'tree-sitter-rust': '0.24.0',
'tree-sitter-typescript': '0.23.2',
};
const peers: Record<string, string> = {
'tree-sitter-cpp': '^0.21.1',
'tree-sitter-java': '^0.21.1',
'tree-sitter-php': '^0.22.4',
'tree-sitter-ruby': '^0.21.1',
'tree-sitter-rust': '^0.22.1',
'tree-sitter-typescript': '^0.21.0',
};
const read = (file: string) => JSON.parse(readFileSync(file, 'utf8'));
const write = (file: string, value: unknown) => {
mkdirSync(path.dirname(file), { recursive: true });
writeFileSync(file, JSON.stringify(value, null, 2) + '\n');
};
function fixture() {
const root = mkdtempSync(path.join(tmpdir(), 'gitnexus-parser-bundle-'));
roots.push(root);
write(path.join(root, 'package.json'), {
dependencies: directVersions,
bundleDependencies: Object.keys(directVersions),
});
for (const [name, version] of Object.entries(directVersions)) {
write(path.join(root, 'node_modules', name, 'package.json'), {
name,
version,
peerDependencies: { 'tree-sitter': peers[name] ?? '^0.25.0' },
});
}
write(path.join(root, 'node_modules/tree-sitter-c/package.json'), {
name: 'tree-sitter-c',
version: '0.23.6',
peerDependencies: { 'tree-sitter': '^0.22.1' },
});
write(
path.join(
root,
'node_modules/tree-sitter-typescript/node_modules/tree-sitter-javascript/package.json',
),
{
name: 'tree-sitter-javascript',
version: '0.23.1',
peerDependencies: { 'tree-sitter': '^0.21.1' },
},
);
return root;
}
afterEach(() => {
for (const root of roots.splice(0)) rmSync(root, { recursive: true, force: true });
});
describe('native parser bundle preparation', () => {
it('admits the tested runtime for direct and transitive peers without changing binaries', () => {
const root = fixture();
const native = path.join(root, 'node_modules/tree-sitter-cpp/parser.node');
writeFileSync(native, Buffer.from([0, 1, 128, 255]));
const original = readFileSync(native);
expect(prepareTreeSitterBundle(root)).toHaveLength(8);
expect(
read(path.join(root, 'node_modules/tree-sitter-cpp/package.json')).peerDependencies[
'tree-sitter'
],
).toBe('^0.21.1 || 0.25.1');
expect(
read(
path.join(
root,
'node_modules/tree-sitter-typescript/node_modules/tree-sitter-javascript/package.json',
),
).peerDependencies['tree-sitter'],
).toBe('^0.21.1 || 0.25.1');
expect(
read(path.join(root, 'node_modules/tree-sitter-javascript/package.json')).peerDependencies[
'tree-sitter'
],
).toBe('^0.25.0');
expect(prepareTreeSitterBundle(root)).toHaveLength(8);
expect(readFileSync(native)).toEqual(original);
expect(read(path.join(root, 'package.json')).dependencies).toEqual(directVersions);
});
it('rejects changed peer metadata before writing any package', () => {
const root = fixture();
const first = path.join(root, 'node_modules/tree-sitter-c/package.json');
const before = readFileSync(first, 'utf8');
const last = path.join(root, 'node_modules/tree-sitter-typescript/package.json');
const pkg = read(last);
pkg.peerDependencies['tree-sitter'] = '^0.26.0';
write(last, pkg);
expect(() => prepareTreeSitterBundle(root)).toThrow('Unexpected tree-sitter peer metadata');
expect(readFileSync(first, 'utf8')).toBe(before);
});
it('rejects a missing transitive grammar instead of silently shipping stale peers', () => {
const root = fixture();
rmSync(path.join(root, 'node_modules/tree-sitter-typescript/node_modules'), {
recursive: true,
});
expect(() => prepareTreeSitterBundle(root)).toThrow('dependency layout changed');
});
it('refuses to rely on root overrides when a grammar is not bundled', () => {
const root = fixture();
const file = path.join(root, 'package.json');
const pkg = read(file);
pkg.bundleDependencies = ['tree-sitter'];
write(file, pkg);
expect(() => prepareTreeSitterBundle(root)).toThrow('must be bundled');
});
it('rejects a runtime upgrade until its compatibility has been revalidated', () => {
const root = fixture();
const file = path.join(root, 'package.json');
const pkg = read(file);
pkg.dependencies['tree-sitter'] = '0.26.0';
write(file, pkg);
expect(() => prepareTreeSitterBundle(root)).toThrow('only validated for runtime 0.25.1');
});
it('rejects installed versions that differ from the exact package pins', () => {
const root = fixture();
const file = path.join(root, 'node_modules/tree-sitter-go/package.json');
const pkg = read(file);
pkg.version = '0.26.0';
write(file, pkg);
expect(() => prepareTreeSitterBundle(root)).toThrow('installed version differs');
});
});

2
gitnexus/test/unit/resolve-enclosing-owner.test.ts
View file

@ -15,7 +15,7 @@ import { rubyProvider } from '../../src/core/ingestion/languages/ruby.js';
import { requireVendoredGrammar } from '../../src/core/tree-sitter/vendored-grammars.js';
// Vendored grammar — loaded from vendor/ by absolute path, never node_modules (#2111).
let Kotlin: unknown;
let Kotlin: Parser.Language | null = null;
try {
Kotlin = requireVendoredGrammar('tree-sitter-kotlin');
} catch {

18
gitnexus/test/unit/scope-resolution/python/python-captures-golden.test.ts
View file

@ -181,6 +181,24 @@ function formatGolden(snap: Snapshot): string {
}
describe('Python scope captures — golden parity', () => {
it('captures an empty package module without inventing declarations or references', () => {
// Tree-sitter 0.25 matches the zero-width module root. The 15 empty
// __init__.py fixtures therefore gain this scope (with its lexical
// decorations) and no other captures.
const range = { startLine: 1, startCol: 0, endLine: 1, endCol: 0 };
expect(emitPythonScopeCaptures('', '__init__.py')).toEqual([
{
'@scope.module': { name: '@scope.module', text: '', range },
'@scope.name-claims': { name: '@scope.name-claims', text: '[]', range },
'@scope.lookup-policy': {
name: '@scope.lookup-policy',
text: '{"callerScopeIsAuthoritative":true}',
range,
},
},
]);
});
it('keeps ordinary Python call arity out of the generic reference schema', () => {
const src = [
'zero()',

2
gitnexus/test/unit/scope-resolution/rust/rust-mod-scope-qualifier.test.ts
View file

@ -16,7 +16,7 @@ import {
import { couldNameAModule } from '../../../../src/core/ingestion/languages/rust/module-path.js';
const parser = new Parser();
parser.setLanguage(Rust as unknown);
parser.setLanguage(Rust);
/** First node of `type` in a post-order walk — the innermost such node. */
function findNode(root: SyntaxNode, type: string): SyntaxNode {

123
gitnexus/test/unit/scope-resolution/typescript/typescript-capture-order.test.ts Normal file
View file

@ -0,0 +1,123 @@
import { describe, expect, it } from 'vitest';
import { createHash } from 'node:crypto';
import { readFileSync, readdirSync } from 'node:fs';
import path from 'node:path';
import { fileURLToPath } from 'node:url';
import { emitTsScopeCaptures } from '../../../../src/core/ingestion/languages/typescript/captures.js';
function synthesizedCaptureFingerprint(): string {
const root = fileURLToPath(new URL('../../../fixtures/lang-resolution/', import.meta.url));
const files = readdirSync(root, { recursive: true, encoding: 'utf8' })
.filter((file) => file.startsWith('typescript-') && /\.tsx?$/.test(file))
.map((file) => file.split(path.sep).join('/'))
.sort();
const tags = [
'@reference.inherits',
'@type-binding.destructured',
'@type-binding.map-tuple-entry',
'@type-binding.instanceof-narrow',
];
expect(files.length).toBeGreaterThan(0);
const hash = createHash('sha256');
for (const file of files) {
const absolute = path.join(root, file);
const captures = emitTsScopeCaptures(readFileSync(absolute, 'utf8'), absolute).filter(
(capture) => tags.some((tag) => capture[tag]),
);
// Preserve match order: equal-strength bindings can depend on arrival order.
hash.update(JSON.stringify([file, captures])).update('\n');
}
return hash.digest('hex');
}
describe('TypeScript synthesized capture order', () => {
it('preserves the ordered synthesized capture stream across the TypeScript fixture corpus', () => {
// Recorded from the full-tree traversal before the native-selection change;
// re-recorded when #3532 added @reference.lookup-purpose (match order unchanged).
expect(synthesizedCaptureFingerprint()).toBe(
'2b848e344e6314b48277befdae8093cccadc3f7543fe71b0566756d5eb4b2ce0',
);
});
it('keeps parents before nested classes and visits sibling classes right to left', () => {
const captures = emitTsScopeCaptures(
`class First extends Left {}
abstract class Outer extends Parent {
method() {
class InnerLeft extends NestedLeft {}
class InnerRight extends NestedRight {}
}
}
interface Last extends Right, Extra {}`,
'inheritance-order.ts',
);
expect(
captures
.filter((capture) => capture['@reference.inherits'])
.map((capture) => capture['@reference.name']?.text),
).toEqual(['Right', 'Extra', 'Parent', 'NestedRight', 'NestedLeft', 'Left']);
});
it('visits destructuring declarations right to left and fields left to right', () => {
const captures = emitTsScopeCaptures(
`const { left } = first;
function nested() { const { middle, renamed: alias } = second; }
const { right } = third;`,
'destructuring-order.ts',
);
expect(
captures
.filter((capture) => capture['@type-binding.destructured'])
.map((capture) => [
capture['@type-binding.name']?.text,
capture['@type-binding.type']?.text,
]),
).toEqual([
['right', 'third.right'],
['middle', 'second.middle'],
['alias', 'second.renamed'],
['left', 'first.left'],
]);
});
it('keeps outer Map tuple bindings ahead of inner loops', () => {
const captures = emitTsScopeCaptures(
`for (const [leftKey, leftValue] of left) {}
for (const [outerKey, outerValue] of outer) {
for (const [innerKey, innerValue] of inner) {}
}
for (const [rightKey, rightValue] of right) {}`,
'map-order.ts',
);
expect(
captures
.filter((capture) => capture['@type-binding.map-tuple-entry'])
.map((capture) => capture['@type-binding.name']?.text),
).toEqual([
'rightKey',
'rightValue',
'outerKey',
'outerValue',
'innerKey',
'innerValue',
'leftKey',
'leftValue',
]);
});
it('keeps outer narrowings ahead of nested if statements and visits else first', () => {
const captures = emitTsScopeCaptures(
`if (left instanceof Left) {}
if (outer instanceof Outer) {
if (inner instanceof Inner) {}
} else if (alternative instanceof Alternative) {}
if (right instanceof Right) {}`,
'narrowing-order.ts',
);
expect(
captures
.filter((capture) => capture['@type-binding.instanceof-narrow'])
.map((capture) => capture['@type-binding.name']?.text),
).toEqual(['right', 'outer', 'alternative', 'inner', 'left']);
});
});

8
gitnexus/test/unit/type-env.test.ts
View file

@ -13,7 +13,7 @@ import {
import Parser from 'tree-sitter';
import TypeScript from 'tree-sitter-typescript';
import Java from 'tree-sitter-java';
import CSharp from 'tree-sitter-c-sharp';
import CSharp from 'tree-sitter-c-sharp/bindings/node/index.js';
import Go from 'tree-sitter-go';
import Rust from 'tree-sitter-rust';
import Python from 'tree-sitter-python';
@ -25,7 +25,7 @@ import { requireVendoredGrammar } from '../../src/core/tree-sitter/vendored-gram
// Vendored grammars — loaded from vendor/ by absolute path, never node_modules (#2111).
const Kotlin = requireVendoredGrammar('tree-sitter-kotlin');
let Dart: unknown;
let Dart: Parser.Language | null = null;
try {
Dart = requireVendoredGrammar('tree-sitter-dart');
const testParser = new Parser();
@ -34,7 +34,7 @@ try {
Dart = null;
}
let Swift: unknown;
let Swift: Parser.Language | null = null;
try {
Swift = requireVendoredGrammar('tree-sitter-swift');
const testParser = new Parser();
@ -45,7 +45,7 @@ try {
const parser = new Parser();
const parse = (code: string, lang: unknown) => {
const parse = (code: string, lang: Parser.Language) => {
parser.setLanguage(lang);
return parser.parse(code);
};

6
gitnexus/test/unit/variable-extraction.test.ts
View file

@ -30,7 +30,7 @@ import { requireVendoredGrammar } from '../../src/core/tree-sitter/vendored-gram
const C = requireVendoredGrammar('tree-sitter-c');
const Dart = requireVendoredGrammar('tree-sitter-dart');
let Kotlin: unknown;
let Kotlin: Parser.Language | null = null;
try {
Kotlin = requireVendoredGrammar('tree-sitter-kotlin');
} catch {
@ -664,10 +664,10 @@ describe('VariableExtractor — block-scoped declarations', () => {
};
parser.setLanguage(Go);
const tree = parser.parse('package main\nfunc foo() { x := 5 }');
// source_file > function_declaration > block > short_var_declaration
// Go 0.25 inserts statement_list between the block and its statements.
const funcDecl = tree.rootNode.namedChildren.find((c) => c.type === 'function_declaration')!;
const body = funcDecl.childForFieldName('body')!;
const shortVarDecl = body.namedChildren.find((c) => c.type === 'short_var_declaration');
const shortVarDecl = body.descendantsOfType('short_var_declaration')[0];
expect(shortVarDecl).toBeDefined();
const info = extractor.extract(shortVarDecl!, ctx);
expect(info).not.toBeNull();

47
gitnexus/vendor/README.md vendored Normal file
View file

@ -0,0 +1,47 @@
# Vendored Tree-sitter grammars
These grammars use the bundled Tree-sitter 0.25.1 runtime (ABI 13–15).
Generated C sources and scanners must be copied from upstream; never patch them
by hand. JavaScript/TypeScript compatibility adapters belong in GitNexus.
| Grammar | Version / snapshot | Upstream source commit |
| --- | --- | --- |
| c | `0.24.2-gb780e47` | [b780e47fc780](https://github.com/tree-sitter/tree-sitter-c/commit/b780e47fc780ddc8da13afa35a3f4ed5c157823d) |
| dart | `1.0.0-gbe07cf7` | [be07cf7118d3](https://github.com/UserNobody14/tree-sitter-dart/commit/be07cf7118d3dba06236a3f19541685a68209934) |
| proto | `0.6.0-gd7d1542` | [d7d15427321d](https://github.com/coder3101/tree-sitter-proto/commit/d7d15427321da271cb1c25c12735e180ef981473) |
| kotlin | `0.4.0-g1852ea1` | [1852ea17b7f6](https://github.com/fwcd/tree-sitter-kotlin/commit/1852ea17b7f60fb3f9d84e0b1555d56b46b39fb1) |
| objc | `3.0.2-g181a81b` | [181a81b8f23a](https://github.com/tree-sitter-grammars/tree-sitter-objc/commit/181a81b8f23a2d593e7ab4259981f50122909fda) |
| zig | `1.1.2-g6479aa1` | [6479aa13f32f](https://github.com/tree-sitter-grammars/tree-sitter-zig/commit/6479aa13f32f701c383083d8b28360ebd682fb7d) |
| swift | `0.7.2` | [7b7909f2f6b9](https://github.com/alex-pinkus/tree-sitter-swift/commit/7b7909f2f6b9414be0958275f4c8e5d69c3bca43) |
Swift 0.7.2 is the newest release that passes our declaration-ownership tests.
Both 0.7.3 and 0.7.4 move a later top-level struct inside an earlier class after
a conditional group splits a function header. That input is invalid Swift,
but preserving recovery for partial source is an existing GitNexus contract.
The hold in `.github/vendored-grammars.json` remains until the real-parser and
worker-pipeline tests in `tree-sitter-languages.test.ts` and
`swift-conditional-directive.test.ts` pass with a newer upstream grammar.
No generated source was patched to preserve this behavior.
To refresh a grammar:
1. Resolve an immutable upstream commit (Swift publishes generated sources on
`*-with-generated-files` tags). Copy `src/` runtime/build inputs and
`bindings/node/binding.cc` byte-for-byte. Keep the GitNexus CommonJS loader
and `binding.gyp`; upstream C's ESM loader uses top-level await and cannot be
substituted into the synchronous worker import path.
2. Update the vendor package version, `_upstreamCommit`, `_vendoredBy`, this
table and any hold in `.github/vendored-grammars.json`. GitHub snapshots use
`<upstream-version>-g<sha7>` so the monitor recognizes the installed commit.
3. Run semantic tests, including CFG and call/reference resolution. Successful
loading, parsing or definition captures alone do not prove compatibility.
4. Let `build-tree-sitter-prebuilds.yml` rebuild and execute the grammar on
all six `{linux,darwin,win32}-{x64,arm64}` hosts. Check the installed tarball
too; it must include the regenerated binaries and runtime metadata. Source
builds use the repository checkout, which also contains the build inputs.
Protobuf's upstream `syntax` node collides with node-tree-sitter 0.25.1's
JavaScript `SyntaxNode` subclass generator. Its gRPC plugin uses a private
metadata wrapper to retain the native node and field/query APIs without
changing upstream sources. Native loading and comment/string-safe fallback
extraction are covered in `test/unit/group/grpc-extractor.test.ts`.

48
gitnexus/vendor/tree-sitter-c/README.md vendored
View file

@ -1,41 +1,13 @@
## GitNexus vendor notice
This directory is a GitNexus-managed **runtime** package derived from
`tree-sitter-c@0.21.4` (tree-sitter/tree-sitter-c). It carries the runtime files
(`bindings/node/`, `src/node-types.json`, `LICENSE`), the native `prebuilds/`,
**and** the grammar source (`binding.gyp`, `src/parser.c`, `src/tree_sitter/`).
The prebuilds make C parsing toolchain-free; the source lets
`build-tree-sitter-grammars.cjs` compile the binding on a toolchain host when no
prebuild matches (e.g. CI before the prebuilds are vendored).
This runtime package uses upstream sources at [`b780e47fc780ddc8da13afa35a3f4ed5c157823d`](https://github.com/tree-sitter/tree-sitter-c/commit/b780e47fc780ddc8da13afa35a3f4ed5c157823d)
with Tree-sitter **0.25.1**. The copied C sources, scanner (where present),
headers, node metadata and native binding source are upstream-identical.
GitNexus preserves its synchronous CommonJS loader and hardened `binding.gyp`.
The exact version and source commit are recorded in `package.json`.
### Why this is vendored (unlike the other npm grammars)
`tree-sitter-c` is the one grammar dependency upstream ships **incomplete**
prebuilds for: only 4 of 6 platform-archs (no `linux-arm64` / `win32-arm64`,
[#2116](https://github.com/abhigyanpatwari/GitNexus/issues/2116)). And unlike
the optional grammars, `tree-sitter-c` is a **required** grammar whose own
`install` script (`node-gyp-build`) compiles from source when no prebuild
matches — which **hard-fails `npm install`** on a toolchain-less ARM host
(`node-gyp rebuild` exits non-zero for a required dependency). To make C parsing
toolchain-free on every platform, GitNexus builds all six prebuilds itself (via
the `build-tree-sitter-prebuilds` workflow) and vendors them; `node-gyp-build`
selects the right `.node` at require time.
### Held at 0.21.4 (do not bump here)
The version is pinned to **0.21.4** for ABI compatibility with the bundled
`tree-sitter@0.21.1` runtime — `tree-sitter-c@0.23.x` prebuilds segfault under
0.21.1 on Windows ([#1242](https://github.com/abhigyanpatwari/GitNexus/issues/1242),
[#858](https://github.com/abhigyanpatwari/GitNexus/issues/858)). Vendoring 0.21.4
*preserves* that pin while closing the ARM prebuild gap. Bump only as part of the
deliberate tree-sitter 0.21→0.23 runtime upgrade.
### Updating this vendor package
1. (Runtime upgrade only) bump `version` in `package.json` + refresh
`bindings/node/*` and `src/node-types.json` from the new `tree-sitter-c`
release, and refresh `_vendoredBy`.
2. Regenerate the six prebuilds by running the **`build-tree-sitter-prebuilds`**
workflow.
3. Verify the packed tarball can `require('tree-sitter-c')` and parse C on each
target platform-arch (the workflow's validate step does this in CI).
All six platform/architecture prebuilds are built from this vendored source by
`.github/workflows/build-tree-sitter-prebuilds.yml`. Load through
`requireVendoredGrammar`; do not copy the package into `node_modules`.
See [the vendor update policy](../README.md), including the Swift release hold.
Do not edit generated C files by hand.

2
gitnexus/vendor/tree-sitter-c/bindings/node/binding.cc vendored
View file

@ -6,7 +6,7 @@ extern "C" TSLanguage *tree_sitter_c();
// "tree-sitter", "language" hashed with BLAKE2
const napi_type_tag LANGUAGE_TYPE_TAG = {
0x8AF2E5212AD58ABF, 0xD5006CAD83ABBA16
0x8AF2E5212AD58ABF, 0xD5006CAD83ABBA16
};
Napi::Object Init(Napi::Env env, Napi::Object exports) {

9
gitnexus/vendor/tree-sitter-c/package.json vendored
View file

@ -1,18 +1,19 @@
{
"name": "tree-sitter-c",
"version": "0.21.4",
"version": "0.24.2-gb780e47",
"description": "C grammar for tree-sitter",
"repository": "https://github.com/tree-sitter/tree-sitter-c",
"license": "MIT",
"main": "bindings/node/index.js",
"types": "bindings/node/index.d.ts",
"_vendoredBy": "gitnexus - runtime package derived from tree-sitter-c@0.21.4 (tree-sitter/tree-sitter-c). HELD at 0.21.4 for ABI compatibility with the bundled tree-sitter@0.21.1 runtime (#1242/#858) — do not bump without the runtime upgrade. Vendored because upstream ships native prebuilds for only 4 of 6 platforms (no linux-arm64/win32-arm64, #2116), and tree-sitter-c is a REQUIRED grammar whose source build hard-fails `npm install` on a toolchain-less ARM host. GitNexus cross-builds all six prebuilds via .github/workflows/build-tree-sitter-prebuilds.yml; the C source (binding.gyp + src/) is ALSO vendored so build-tree-sitter-grammars.cjs can source-build the binding on a toolchain host when no prebuild matches (e.g. CI before prebuilds land). Loaded from vendor/ by absolute path at runtime (vendored-grammars.ts) — NEVER copied to node_modules (#2111) (no scripts.install here — #836/#1728).",
"_vendoredBy": "gitnexus — upstream generated src/ and bindings/node/binding.cc copied byte-for-byte from https://github.com/tree-sitter/tree-sitter-c@b780e47fc780ddc8da13afa35a3f4ed5c157823d. GitNexus keeps its synchronous CommonJS loader and hardened binding.gyp. Prebuilds are rebuilt from these sources by build-tree-sitter-prebuilds.yml. Do not edit generated C by hand.",
"peerDependencies": {
"tree-sitter": "^0.21.0"
"tree-sitter": "0.25.1"
},
"peerDependenciesMeta": {
"tree-sitter": {
"optional": true
}
}
},
"_upstreamCommit": "b780e47fc780ddc8da13afa35a3f4ed5c157823d"
}

BIN
gitnexus/vendor/tree-sitter-c/prebuilds/darwin-arm64/tree-sitter-c.node vendored
View file

Binary file not shown.

BIN
gitnexus/vendor/tree-sitter-c/prebuilds/darwin-x64/tree-sitter-c.node vendored
View file

Binary file not shown.

BIN
gitnexus/vendor/tree-sitter-c/prebuilds/linux-arm64/tree-sitter-c.node vendored
View file

Binary file not shown.

BIN
gitnexus/vendor/tree-sitter-c/prebuilds/linux-x64/tree-sitter-c.node vendored
View file

Binary file not shown.

BIN
gitnexus/vendor/tree-sitter-c/prebuilds/win32-arm64/tree-sitter-c.node vendored
View file

Binary file not shown.

BIN
gitnexus/vendor/tree-sitter-c/prebuilds/win32-x64/tree-sitter-c.node vendored
View file

Binary file not shown.

63
gitnexus/vendor/tree-sitter-c/src/node-types.json vendored
View file

@ -155,6 +155,10 @@
"type": "conditional_expression",
"named": true
},
{
"type": "extension_expression",
"named": true
},
{
"type": "false",
"named": true
@ -1528,6 +1532,21 @@
]
}
},
{
"type": "extension_expression",
"named": true,
"fields": {},
"children": {
"multiple": false,
"required": true,
"types": [
{
"type": "expression",
"named": true
}
]
}
},
{
"type": "field_declaration",
"named": true,
@ -2068,7 +2087,7 @@
"required": true,
"types": [
{
"type": "identifier",
"type": "expression",
"named": true
}
]
@ -2262,6 +2281,10 @@
"multiple": false,
"required": true,
"types": [
{
"type": "declaration",
"named": true
},
{
"type": "statement",
"named": true
@ -2491,6 +2514,10 @@
"multiple": true,
"required": false,
"types": [
{
"type": "compound_statement",
"named": true
},
{
"type": "identifier",
"named": true
@ -2545,6 +2572,10 @@
"type": "comma_expression",
"named": true
},
{
"type": "compound_statement",
"named": true
},
{
"type": "expression",
"named": true
@ -3395,6 +3426,16 @@
}
]
}
},
"children": {
"multiple": true,
"required": false,
"types": [
{
"type": "type_qualifier",
"named": true
}
]
}
},
{
@ -3583,6 +3624,7 @@
{
"type": "translation_unit",
"named": true,
"root": true,
"fields": {},
"children": {
"multiple": true,
@ -4185,6 +4227,10 @@
"type": "_Generic",
"named": false
},
{
"type": "_Nonnull",
"named": false
},
{
"type": "_Noreturn",
"named": false
@ -4197,10 +4243,18 @@
"type": "__alignof__",
"named": false
},
{
"type": "__asm",
"named": false
},
{
"type": "__asm__",
"named": false
},
{
"type": "__attribute",
"named": false
},
{
"type": "__attribute__",
"named": false
@ -4281,6 +4335,10 @@
"type": "__vectorcall",
"named": false
},
{
"type": "__volatile__",
"named": false
},
{
"type": "_alignof",
"named": false
@ -4319,7 +4377,8 @@
},
{
"type": "comment",
"named": true
"named": true,
"extra": true
},
{
"type": "const",

204806
gitnexus/vendor/tree-sitter-c/src/parser.c vendored
View file

File diff suppressed because it is too large Load diff

8
gitnexus/vendor/tree-sitter-c/src/tree_sitter/alloc.h vendored
View file

@ -12,10 +12,10 @@ extern "C" {
// Allow clients to override allocation functions
#ifdef TREE_SITTER_REUSE_ALLOCATOR
extern void *(*ts_current_malloc)(size_t);
extern void *(*ts_current_calloc)(size_t, size_t);
extern void *(*ts_current_realloc)(void *, size_t);
extern void (*ts_current_free)(void *);
extern void *(*ts_current_malloc)(size_t size);
extern void *(*ts_current_calloc)(size_t count, size_t size);
extern void *(*ts_current_realloc)(void *ptr, size_t size);
extern void (*ts_current_free)(void *ptr);
#ifndef ts_malloc
#define ts_malloc ts_current_malloc

184
gitnexus/vendor/tree-sitter-c/src/tree_sitter/array.h vendored
View file

@ -14,6 +14,7 @@ extern "C" {
#include <string.h>
#ifdef _MSC_VER
#pragma warning(push)
#pragma warning(disable : 4101)
#elif defined(__GNUC__) || defined(__clang__)
#pragma GCC diagnostic push
@ -51,67 +52,96 @@ extern "C" {
/// Reserve `new_capacity` elements of space in the array. If `new_capacity` is
/// less than the array's current capacity, this function has no effect.
#define array_reserve(self, new_capacity) \
_array__reserve((Array *)(self), array_elem_size(self), new_capacity)
#define array_reserve(self, new_capacity) \
((self)->contents = _array__reserve( \
(void *)(self)->contents, &(self)->capacity, \
array_elem_size(self), new_capacity) \
)
/// Free any memory allocated for this array. Note that this does not free any
/// memory allocated for the array's contents.
#define array_delete(self) _array__delete((Array *)(self))
#define array_delete(self) \
do { \
if ((self)->contents) ts_free((self)->contents); \
(self)->contents = NULL; \
(self)->size = 0; \
(self)->capacity = 0; \
} while (0)
/// Push a new `element` onto the end of the array.
#define array_push(self, element) \
(_array__grow((Array *)(self), 1, array_elem_size(self)), \
(self)->contents[(self)->size++] = (element))
#define array_push(self, element) \
do { \
(self)->contents = _array__grow( \
(void *)(self)->contents, (self)->size, &(self)->capacity, \
1, array_elem_size(self) \
); \
(self)->contents[(self)->size++] = (element); \
} while(0)
/// Increase the array's size by `count` elements.
/// New elements are zero-initialized.
#define array_grow_by(self, count) \
do { \
if ((count) == 0) break; \
_array__grow((Array *)(self), count, array_elem_size(self)); \
#define array_grow_by(self, count) \
do { \
if ((count) == 0) break; \
(self)->contents = _array__grow( \
(self)->contents, (self)->size, &(self)->capacity, \
count, array_elem_size(self) \
); \
memset((self)->contents + (self)->size, 0, (count) * array_elem_size(self)); \
(self)->size += (count); \
(self)->size += (count); \
} while (0)
/// Append all elements from one array to the end of another.
#define array_push_all(self, other) \
#define array_push_all(self, other) \
array_extend((self), (other)->size, (other)->contents)
/// Append `count` elements to the end of the array, reading their values from the
/// `contents` pointer.
#define array_extend(self, count, contents) \
_array__splice( \
(Array *)(self), array_elem_size(self), (self)->size, \
0, count, contents \
#define array_extend(self, count, other_contents) \
(self)->contents = _array__splice( \
(void*)(self)->contents, &(self)->size, &(self)->capacity, \
array_elem_size(self), (self)->size, 0, count, other_contents \
)
/// Remove `old_count` elements from the array starting at the given `index`. At
/// the same index, insert `new_count` new elements, reading their values from the
/// `new_contents` pointer.
#define array_splice(self, _index, old_count, new_count, new_contents) \
_array__splice( \
(Array *)(self), array_elem_size(self), _index, \
old_count, new_count, new_contents \
#define array_splice(self, _index, old_count, new_count, new_contents) \
(self)->contents = _array__splice( \
(void *)(self)->contents, &(self)->size, &(self)->capacity, \
array_elem_size(self), _index, old_count, new_count, new_contents \
)
/// Insert one `element` into the array at the given `index`.
#define array_insert(self, _index, element) \
_array__splice((Array *)(self), array_elem_size(self), _index, 0, 1, &(element))
#define array_insert(self, _index, element) \
(self)->contents = _array__splice( \
(void *)(self)->contents, &(self)->size, &(self)->capacity, \
array_elem_size(self), _index, 0, 1, &(element) \
)
/// Remove one element from the array at the given `index`.
#define array_erase(self, _index) \
_array__erase((Array *)(self), array_elem_size(self), _index)
_array__erase((void *)(self)->contents, &(self)->size, array_elem_size(self), _index)
/// Pop the last element off the array, returning the element by value.
#define array_pop(self) ((self)->contents[--(self)->size])
/// Assign the contents of one array to another, reallocating if necessary.
#define array_assign(self, other) \
_array__assign((Array *)(self), (const Array *)(other), array_elem_size(self))
#define array_assign(self, other) \
(self)->contents = _array__assign( \
(void *)(self)->contents, &(self)->size, &(self)->capacity, \
(const void *)(other)->contents, (other)->size, array_elem_size(self) \
)
/// Swap one array with another
#define array_swap(self, other) \
_array__swap((Array *)(self), (Array *)(other))
#define array_swap(self, other) \
do { \
void *_array_swap_tmp = (void *)(self)->contents; \
(self)->contents = (other)->contents; \
(other)->contents = _array_swap_tmp; \
_array__swap(&(self)->size, &(self)->capacity, \
&(other)->size, &(other)->capacity); \
} while (0)
/// Get the size of the array contents
#define array_elem_size(self) (sizeof *(self)->contents)
@ -156,82 +186,90 @@ extern "C" {
// Private
typedef Array(void) Array;
/// This is not what you're looking for, see `array_delete`.
static inline void _array__delete(Array *self) {
if (self->contents) {
ts_free(self->contents);
self->contents = NULL;
self->size = 0;
self->capacity = 0;
}
}
// Pointers to individual `Array` fields (rather than the entire `Array` itself)
// are passed to the various `_array__*` functions below to address strict aliasing
// violations that arises when the _entire_ `Array` struct is passed as `Array(void)*`.
//
// The `Array` type itself was not altered as a solution in order to avoid breakage
// with existing consumers (in particular, parsers with external scanners).
/// This is not what you're looking for, see `array_erase`.
static inline void _array__erase(Array *self, size_t element_size,
uint32_t index) {
assert(index < self->size);
char *contents = (char *)self->contents;
static inline void _array__erase(void* self_contents, uint32_t *size,
size_t element_size, uint32_t index) {
assert(index < *size);
char *contents = (char *)self_contents;
memmove(contents + index * element_size, contents + (index + 1) * element_size,
(self->size - index - 1) * element_size);
self->size--;
(*size - index - 1) * element_size);
(*size)--;
}
/// This is not what you're looking for, see `array_reserve`.
static inline void _array__reserve(Array *self, size_t element_size, uint32_t new_capacity) {
if (new_capacity > self->capacity) {
if (self->contents) {
self->contents = ts_realloc(self->contents, new_capacity * element_size);
static inline void *_array__reserve(void *contents, uint32_t *capacity,
size_t element_size, uint32_t new_capacity) {
void *new_contents = contents;
if (new_capacity > *capacity) {
if (contents) {
new_contents = ts_realloc(contents, new_capacity * element_size);
} else {
self->contents = ts_malloc(new_capacity * element_size);
new_contents = ts_malloc(new_capacity * element_size);
}
self->capacity = new_capacity;
*capacity = new_capacity;
}
return new_contents;
}
/// This is not what you're looking for, see `array_assign`.
static inline void _array__assign(Array *self, const Array *other, size_t element_size) {
_array__reserve(self, element_size, other->size);
self->size = other->size;
memcpy(self->contents, other->contents, self->size * element_size);
static inline void *_array__assign(void* self_contents, uint32_t *self_size, uint32_t *self_capacity,
const void *other_contents, uint32_t other_size, size_t element_size) {
void *new_contents = _array__reserve(self_contents, self_capacity, element_size, other_size);
*self_size = other_size;
memcpy(new_contents, other_contents, *self_size * element_size);
return new_contents;
}
/// This is not what you're looking for, see `array_swap`.
static inline void _array__swap(Array *self, Array *other) {
Array swap = *other;
*other = *self;
*self = swap;
static inline void _array__swap(uint32_t *self_size, uint32_t *self_capacity,
uint32_t *other_size, uint32_t *other_capacity) {
uint32_t tmp_size = *self_size;
uint32_t tmp_capacity = *self_capacity;
*self_size = *other_size;
*self_capacity = *other_capacity;
*other_size = tmp_size;
*other_capacity = tmp_capacity;
}
/// This is not what you're looking for, see `array_push` or `array_grow_by`.
static inline void _array__grow(Array *self, uint32_t count, size_t element_size) {
uint32_t new_size = self->size + count;
if (new_size > self->capacity) {
uint32_t new_capacity = self->capacity * 2;
static inline void *_array__grow(void *contents, uint32_t size, uint32_t *capacity,
uint32_t count, size_t element_size) {
void *new_contents = contents;
uint32_t new_size = size + count;
if (new_size > *capacity) {
uint32_t new_capacity = *capacity * 2;
if (new_capacity < 8) new_capacity = 8;
if (new_capacity < new_size) new_capacity = new_size;
_array__reserve(self, element_size, new_capacity);
new_contents = _array__reserve(contents, capacity, element_size, new_capacity);
}
return new_contents;
}
/// This is not what you're looking for, see `array_splice`.
static inline void _array__splice(Array *self, size_t element_size,
static inline void *_array__splice(void *self_contents, uint32_t *size, uint32_t *capacity,
size_t element_size,
uint32_t index, uint32_t old_count,
uint32_t new_count, const void *elements) {
uint32_t new_size = self->size + new_count - old_count;
uint32_t new_size = *size + new_count - old_count;
uint32_t old_end = index + old_count;
uint32_t new_end = index + new_count;
assert(old_end <= self->size);
assert(old_end <= *size);
_array__reserve(self, element_size, new_size);
void *new_contents = _array__reserve(self_contents, capacity, element_size, new_size);
char *contents = (char *)self->contents;
if (self->size > old_end) {
char *contents = (char *)new_contents;
if (*size > old_end) {
memmove(
contents + new_end * element_size,
contents + old_end * element_size,
(self->size - old_end) * element_size
(*size - old_end) * element_size
);
}
if (new_count > 0) {
@ -249,7 +287,9 @@ static inline void _array__splice(Array *self, size_t element_size,
);
}
}
self->size += new_count - old_count;
*size += new_count - old_count;
return new_contents;
}
/// A binary search routine, based on Rust's `std::slice::binary_search_by`.
@ -278,7 +318,7 @@ static inline void _array__splice(Array *self, size_t element_size,
#define _compare_int(a, b) ((int)*(a) - (int)(b))
#ifdef _MSC_VER
#pragma warning(default : 4101)
#pragma warning(pop)
#elif defined(__GNUC__) || defined(__clang__)
#pragma GCC diagnostic pop
#endif

35
gitnexus/vendor/tree-sitter-c/src/tree_sitter/parser.h vendored
View file

@ -18,6 +18,11 @@ typedef uint16_t TSStateId;
typedef uint16_t TSSymbol;
typedef uint16_t TSFieldId;
typedef struct TSLanguage TSLanguage;
typedef struct TSLanguageMetadata {
uint8_t major_version;
uint8_t minor_version;
uint8_t patch_version;
} TSLanguageMetadata;
#endif
typedef struct {
@ -26,10 +31,11 @@ typedef struct {
bool inherited;
} TSFieldMapEntry;
// Used to index the field and supertype maps.
typedef struct {
uint16_t index;
uint16_t length;
} TSFieldMapSlice;
} TSMapSlice;
typedef struct {
bool visible;
@ -47,6 +53,7 @@ struct TSLexer {
uint32_t (*get_column)(TSLexer *);
bool (*is_at_included_range_start)(const TSLexer *);
bool (*eof)(const TSLexer *);
void (*log)(const TSLexer *, const char *, ...);
};
typedef enum {
@ -78,6 +85,12 @@ typedef struct {
uint16_t external_lex_state;
} TSLexMode;
typedef struct {
uint16_t lex_state;
uint16_t external_lex_state;
uint16_t reserved_word_set_id;
} TSLexerMode;
typedef union {
TSParseAction action;
struct {
@ -92,7 +105,7 @@ typedef struct {
} TSCharacterRange;
struct TSLanguage {
uint32_t version;
uint32_t abi_version;
uint32_t symbol_count;
uint32_t alias_count;
uint32_t token_count;
@ -108,13 +121,13 @@ struct TSLanguage {
const TSParseActionEntry *parse_actions;
const char * const *symbol_names;
const char * const *field_names;
const TSFieldMapSlice *field_map_slices;
const TSMapSlice *field_map_slices;
const TSFieldMapEntry *field_map_entries;
const TSSymbolMetadata *symbol_metadata;
const TSSymbol *public_symbol_map;
const uint16_t *alias_map;
const TSSymbol *alias_sequences;
const TSLexMode *lex_modes;
const TSLexerMode *lex_modes;
bool (*lex_fn)(TSLexer *, TSStateId);
bool (*keyword_lex_fn)(TSLexer *, TSStateId);
TSSymbol keyword_capture_token;
@ -128,15 +141,23 @@ struct TSLanguage {
void (*deserialize)(void *, const char *, unsigned);
} external_scanner;
const TSStateId *primary_state_ids;
const char *name;
const TSSymbol *reserved_words;
uint16_t max_reserved_word_set_size;
uint32_t supertype_count;
const TSSymbol *supertype_symbols;
const TSMapSlice *supertype_map_slices;
const TSSymbol *supertype_map_entries;
TSLanguageMetadata metadata;
};
static inline bool set_contains(TSCharacterRange *ranges, uint32_t len, int32_t lookahead) {
static inline bool set_contains(const TSCharacterRange *ranges, uint32_t len, int32_t lookahead) {
uint32_t index = 0;
uint32_t size = len - index;
while (size > 1) {
uint32_t half_size = size / 2;
uint32_t mid_index = index + half_size;
TSCharacterRange *range = &ranges[mid_index];
const TSCharacterRange *range = &ranges[mid_index];
if (lookahead >= range->start && lookahead <= range->end) {
return true;
} else if (lookahead > range->end) {
@ -144,7 +165,7 @@ static inline bool set_contains(TSCharacterRange *ranges, uint32_t len, int32_t
}
size -= half_size;
}
TSCharacterRange *range = &ranges[index];
const TSCharacterRange *range = &ranges[index];
return (lookahead >= range->start && lookahead <= range->end);
}

11
gitnexus/vendor/tree-sitter-dart/package.json vendored
View file

@ -1,18 +1,19 @@
{
"name": "tree-sitter-dart",
"version": "1.0.0",
"version": "1.0.0-gbe07cf7",
"description": "Dart grammar attempt for tree-sitter",
"repository": "https://github.com/UserNobody14/tree-sitter-dart",
"license": "ISC",
"main": "bindings/node",
"types": "bindings/node",
"_vendoredBy": "gitnexus - pinned to UserNobody14/tree-sitter-dart commit 80e23c07b64494f7e21090bb3450223ef0b192f4. Loaded from vendor/ by absolute path at runtime (vendored-grammars.ts) — NEVER copied to node_modules (#2111); native build via build-tree-sitter-grammars.cjs (#1728, #836).",
"_vendoredBy": "gitnexus — upstream generated src/ and bindings/node/binding.cc copied byte-for-byte from https://github.com/UserNobody14/tree-sitter-dart@be07cf7118d3dba06236a3f19541685a68209934. GitNexus keeps its synchronous CommonJS loader and hardened binding.gyp. Prebuilds are rebuilt from these sources by build-tree-sitter-prebuilds.yml. Do not edit generated C by hand.",
"peerDependencies": {
"tree-sitter": "^0.21.0"
"tree-sitter": "0.25.1"
},
"peerDependenciesMeta": {
"tree_sitter": {
"tree-sitter": {
"optional": true
}
}
},
"_upstreamCommit": "be07cf7118d3dba06236a3f19541685a68209934"
}

BIN
gitnexus/vendor/tree-sitter-dart/prebuilds/darwin-arm64/tree-sitter-dart.node vendored
View file

Binary file not shown.

BIN
gitnexus/vendor/tree-sitter-dart/prebuilds/darwin-x64/tree-sitter-dart.node vendored
View file

Binary file not shown.

BIN
gitnexus/vendor/tree-sitter-dart/prebuilds/linux-arm64/tree-sitter-dart.node vendored
View file

Binary file not shown.

BIN
gitnexus/vendor/tree-sitter-dart/prebuilds/linux-x64/tree-sitter-dart.node vendored
View file

Binary file not shown.

BIN
gitnexus/vendor/tree-sitter-dart/prebuilds/win32-arm64/tree-sitter-dart.node vendored
View file

Binary file not shown.

BIN
gitnexus/vendor/tree-sitter-dart/prebuilds/win32-x64/tree-sitter-dart.node vendored
View file

Binary file not shown.

1999
gitnexus/vendor/tree-sitter-dart/src/grammar.json vendored
View file

File diff suppressed because it is too large Load diff

780
gitnexus/vendor/tree-sitter-dart/src/node-types.json vendored
View file

File diff suppressed because it is too large Load diff

366621
gitnexus/vendor/tree-sitter-dart/src/parser.c vendored
View file

File diff suppressed because it is too large Load diff

8
gitnexus/vendor/tree-sitter-dart/src/tree_sitter/alloc.h vendored
View file

@ -12,10 +12,10 @@ extern "C" {
// Allow clients to override allocation functions
#ifdef TREE_SITTER_REUSE_ALLOCATOR
extern void *(*ts_current_malloc)(size_t);
extern void *(*ts_current_calloc)(size_t, size_t);
extern void *(*ts_current_realloc)(void *, size_t);
extern void (*ts_current_free)(void *);
extern void *(*ts_current_malloc)(size_t size);
extern void *(*ts_current_calloc)(size_t count, size_t size);
extern void *(*ts_current_realloc)(void *ptr, size_t size);
extern void (*ts_current_free)(void *ptr);
#ifndef ts_malloc
#define ts_malloc ts_current_malloc

184
gitnexus/vendor/tree-sitter-dart/src/tree_sitter/array.h vendored
View file

@ -14,6 +14,7 @@ extern "C" {
#include <string.h>
#ifdef _MSC_VER
#pragma warning(push)
#pragma warning(disable : 4101)
#elif defined(__GNUC__) || defined(__clang__)
#pragma GCC diagnostic push
@ -51,67 +52,96 @@ extern "C" {
/// Reserve `new_capacity` elements of space in the array. If `new_capacity` is
/// less than the array's current capacity, this function has no effect.
#define array_reserve(self, new_capacity) \
_array__reserve((Array *)(self), array_elem_size(self), new_capacity)
#define array_reserve(self, new_capacity) \
((self)->contents = _array__reserve( \
(void *)(self)->contents, &(self)->capacity, \
array_elem_size(self), new_capacity) \
)
/// Free any memory allocated for this array. Note that this does not free any
/// memory allocated for the array's contents.
#define array_delete(self) _array__delete((Array *)(self))
#define array_delete(self) \
do { \
if ((self)->contents) ts_free((self)->contents); \
(self)->contents = NULL; \
(self)->size = 0; \
(self)->capacity = 0; \
} while (0)
/// Push a new `element` onto the end of the array.
#define array_push(self, element) \
(_array__grow((Array *)(self), 1, array_elem_size(self)), \
(self)->contents[(self)->size++] = (element))
#define array_push(self, element) \
do { \
(self)->contents = _array__grow( \
(void *)(self)->contents, (self)->size, &(self)->capacity, \
1, array_elem_size(self) \
); \
(self)->contents[(self)->size++] = (element); \
} while(0)
/// Increase the array's size by `count` elements.
/// New elements are zero-initialized.
#define array_grow_by(self, count) \
do { \
if ((count) == 0) break; \
_array__grow((Array *)(self), count, array_elem_size(self)); \
#define array_grow_by(self, count) \
do { \
if ((count) == 0) break; \
(self)->contents = _array__grow( \
(self)->contents, (self)->size, &(self)->capacity, \
count, array_elem_size(self) \
); \
memset((self)->contents + (self)->size, 0, (count) * array_elem_size(self)); \
(self)->size += (count); \
(self)->size += (count); \
} while (0)
/// Append all elements from one array to the end of another.
#define array_push_all(self, other) \
#define array_push_all(self, other) \
array_extend((self), (other)->size, (other)->contents)
/// Append `count` elements to the end of the array, reading their values from the
/// `contents` pointer.
#define array_extend(self, count, contents) \
_array__splice( \
(Array *)(self), array_elem_size(self), (self)->size, \
0, count, contents \
#define array_extend(self, count, other_contents) \
(self)->contents = _array__splice( \
(void*)(self)->contents, &(self)->size, &(self)->capacity, \
array_elem_size(self), (self)->size, 0, count, other_contents \
)
/// Remove `old_count` elements from the array starting at the given `index`. At
/// the same index, insert `new_count` new elements, reading their values from the
/// `new_contents` pointer.
#define array_splice(self, _index, old_count, new_count, new_contents) \
_array__splice( \
(Array *)(self), array_elem_size(self), _index, \
old_count, new_count, new_contents \
#define array_splice(self, _index, old_count, new_count, new_contents) \
(self)->contents = _array__splice( \
(void *)(self)->contents, &(self)->size, &(self)->capacity, \
array_elem_size(self), _index, old_count, new_count, new_contents \
)
/// Insert one `element` into the array at the given `index`.
#define array_insert(self, _index, element) \
_array__splice((Array *)(self), array_elem_size(self), _index, 0, 1, &(element))
#define array_insert(self, _index, element) \
(self)->contents = _array__splice( \
(void *)(self)->contents, &(self)->size, &(self)->capacity, \
array_elem_size(self), _index, 0, 1, &(element) \
)
/// Remove one element from the array at the given `index`.
#define array_erase(self, _index) \
_array__erase((Array *)(self), array_elem_size(self), _index)
_array__erase((void *)(self)->contents, &(self)->size, array_elem_size(self), _index)
/// Pop the last element off the array, returning the element by value.
#define array_pop(self) ((self)->contents[--(self)->size])
/// Assign the contents of one array to another, reallocating if necessary.
#define array_assign(self, other) \
_array__assign((Array *)(self), (const Array *)(other), array_elem_size(self))
#define array_assign(self, other) \
(self)->contents = _array__assign( \
(void *)(self)->contents, &(self)->size, &(self)->capacity, \
(const void *)(other)->contents, (other)->size, array_elem_size(self) \
)
/// Swap one array with another
#define array_swap(self, other) \
_array__swap((Array *)(self), (Array *)(other))
#define array_swap(self, other) \
do { \
void *_array_swap_tmp = (void *)(self)->contents; \
(self)->contents = (other)->contents; \
(other)->contents = _array_swap_tmp; \
_array__swap(&(self)->size, &(self)->capacity, \
&(other)->size, &(other)->capacity); \
} while (0)
/// Get the size of the array contents
#define array_elem_size(self) (sizeof *(self)->contents)
@ -156,82 +186,90 @@ extern "C" {
// Private
typedef Array(void) Array;
/// This is not what you're looking for, see `array_delete`.
static inline void _array__delete(Array *self) {
if (self->contents) {
ts_free(self->contents);
self->contents = NULL;
self->size = 0;
self->capacity = 0;
}
}
// Pointers to individual `Array` fields (rather than the entire `Array` itself)
// are passed to the various `_array__*` functions below to address strict aliasing
// violations that arises when the _entire_ `Array` struct is passed as `Array(void)*`.
//
// The `Array` type itself was not altered as a solution in order to avoid breakage
// with existing consumers (in particular, parsers with external scanners).
/// This is not what you're looking for, see `array_erase`.
static inline void _array__erase(Array *self, size_t element_size,
uint32_t index) {
assert(index < self->size);
char *contents = (char *)self->contents;
static inline void _array__erase(void* self_contents, uint32_t *size,
size_t element_size, uint32_t index) {
assert(index < *size);
char *contents = (char *)self_contents;
memmove(contents + index * element_size, contents + (index + 1) * element_size,
(self->size - index - 1) * element_size);
self->size--;
(*size - index - 1) * element_size);
(*size)--;
}
/// This is not what you're looking for, see `array_reserve`.
static inline void _array__reserve(Array *self, size_t element_size, uint32_t new_capacity) {
if (new_capacity > self->capacity) {
if (self->contents) {
self->contents = ts_realloc(self->contents, new_capacity * element_size);
static inline void *_array__reserve(void *contents, uint32_t *capacity,
size_t element_size, uint32_t new_capacity) {
void *new_contents = contents;
if (new_capacity > *capacity) {
if (contents) {
new_contents = ts_realloc(contents, new_capacity * element_size);
} else {
self->contents = ts_malloc(new_capacity * element_size);
new_contents = ts_malloc(new_capacity * element_size);
}
self->capacity = new_capacity;
*capacity = new_capacity;
}
return new_contents;
}
/// This is not what you're looking for, see `array_assign`.
static inline void _array__assign(Array *self, const Array *other, size_t element_size) {
_array__reserve(self, element_size, other->size);
self->size = other->size;
memcpy(self->contents, other->contents, self->size * element_size);
static inline void *_array__assign(void* self_contents, uint32_t *self_size, uint32_t *self_capacity,
const void *other_contents, uint32_t other_size, size_t element_size) {
void *new_contents = _array__reserve(self_contents, self_capacity, element_size, other_size);
*self_size = other_size;
memcpy(new_contents, other_contents, *self_size * element_size);
return new_contents;
}
/// This is not what you're looking for, see `array_swap`.
static inline void _array__swap(Array *self, Array *other) {
Array swap = *other;
*other = *self;
*self = swap;
static inline void _array__swap(uint32_t *self_size, uint32_t *self_capacity,
uint32_t *other_size, uint32_t *other_capacity) {
uint32_t tmp_size = *self_size;
uint32_t tmp_capacity = *self_capacity;
*self_size = *other_size;
*self_capacity = *other_capacity;
*other_size = tmp_size;
*other_capacity = tmp_capacity;
}
/// This is not what you're looking for, see `array_push` or `array_grow_by`.
static inline void _array__grow(Array *self, uint32_t count, size_t element_size) {
uint32_t new_size = self->size + count;
if (new_size > self->capacity) {
uint32_t new_capacity = self->capacity * 2;
static inline void *_array__grow(void *contents, uint32_t size, uint32_t *capacity,
uint32_t count, size_t element_size) {
void *new_contents = contents;
uint32_t new_size = size + count;
if (new_size > *capacity) {
uint32_t new_capacity = *capacity * 2;
if (new_capacity < 8) new_capacity = 8;
if (new_capacity < new_size) new_capacity = new_size;
_array__reserve(self, element_size, new_capacity);
new_contents = _array__reserve(contents, capacity, element_size, new_capacity);
}
return new_contents;
}
/// This is not what you're looking for, see `array_splice`.
static inline void _array__splice(Array *self, size_t element_size,
static inline void *_array__splice(void *self_contents, uint32_t *size, uint32_t *capacity,
size_t element_size,
uint32_t index, uint32_t old_count,
uint32_t new_count, const void *elements) {
uint32_t new_size = self->size + new_count - old_count;
uint32_t new_size = *size + new_count - old_count;
uint32_t old_end = index + old_count;
uint32_t new_end = index + new_count;
assert(old_end <= self->size);
assert(old_end <= *size);
_array__reserve(self, element_size, new_size);
void *new_contents = _array__reserve(self_contents, capacity, element_size, new_size);
char *contents = (char *)self->contents;
if (self->size > old_end) {
char *contents = (char *)new_contents;
if (*size > old_end) {
memmove(
contents + new_end * element_size,
contents + old_end * element_size,
(self->size - old_end) * element_size
(*size - old_end) * element_size
);
}
if (new_count > 0) {
@ -249,7 +287,9 @@ static inline void _array__splice(Array *self, size_t element_size,
);
}
}
self->size += new_count - old_count;
*size += new_count - old_count;
return new_contents;
}
/// A binary search routine, based on Rust's `std::slice::binary_search_by`.
@ -278,7 +318,7 @@ static inline void _array__splice(Array *self, size_t element_size,
#define _compare_int(a, b) ((int)*(a) - (int)(b))
#ifdef _MSC_VER
#pragma warning(default : 4101)
#pragma warning(pop)
#elif defined(__GNUC__) || defined(__clang__)
#pragma GCC diagnostic pop
#endif

35
gitnexus/vendor/tree-sitter-dart/src/tree_sitter/parser.h vendored
View file

@ -18,6 +18,11 @@ typedef uint16_t TSStateId;
typedef uint16_t TSSymbol;
typedef uint16_t TSFieldId;
typedef struct TSLanguage TSLanguage;
typedef struct TSLanguageMetadata {
uint8_t major_version;
uint8_t minor_version;
uint8_t patch_version;
} TSLanguageMetadata;
#endif
typedef struct {
@ -26,10 +31,11 @@ typedef struct {
bool inherited;
} TSFieldMapEntry;
// Used to index the field and supertype maps.
typedef struct {
uint16_t index;
uint16_t length;
} TSFieldMapSlice;
} TSMapSlice;
typedef struct {
bool visible;
@ -47,6 +53,7 @@ struct TSLexer {
uint32_t (*get_column)(TSLexer *);
bool (*is_at_included_range_start)(const TSLexer *);
bool (*eof)(const TSLexer *);
void (*log)(const TSLexer *, const char *, ...);
};
typedef enum {
@ -78,6 +85,12 @@ typedef struct {
uint16_t external_lex_state;
} TSLexMode;
typedef struct {
uint16_t lex_state;
uint16_t external_lex_state;
uint16_t reserved_word_set_id;
} TSLexerMode;
typedef union {
TSParseAction action;
struct {
@ -92,7 +105,7 @@ typedef struct {
} TSCharacterRange;
struct TSLanguage {
uint32_t version;
uint32_t abi_version;
uint32_t symbol_count;
uint32_t alias_count;
uint32_t token_count;
@ -108,13 +121,13 @@ struct TSLanguage {
const TSParseActionEntry *parse_actions;
const char * const *symbol_names;
const char * const *field_names;
const TSFieldMapSlice *field_map_slices;
const TSMapSlice *field_map_slices;
const TSFieldMapEntry *field_map_entries;
const TSSymbolMetadata *symbol_metadata;
const TSSymbol *public_symbol_map;
const uint16_t *alias_map;
const TSSymbol *alias_sequences;
const TSLexMode *lex_modes;
const TSLexerMode *lex_modes;
bool (*lex_fn)(TSLexer *, TSStateId);
bool (*keyword_lex_fn)(TSLexer *, TSStateId);
TSSymbol keyword_capture_token;
@ -128,15 +141,23 @@ struct TSLanguage {
void (*deserialize)(void *, const char *, unsigned);
} external_scanner;
const TSStateId *primary_state_ids;
const char *name;
const TSSymbol *reserved_words;
uint16_t max_reserved_word_set_size;
uint32_t supertype_count;
const TSSymbol *supertype_symbols;
const TSMapSlice *supertype_map_slices;
const TSSymbol *supertype_map_entries;
TSLanguageMetadata metadata;
};
static inline bool set_contains(TSCharacterRange *ranges, uint32_t len, int32_t lookahead) {
static inline bool set_contains(const TSCharacterRange *ranges, uint32_t len, int32_t lookahead) {
uint32_t index = 0;
uint32_t size = len - index;
while (size > 1) {
uint32_t half_size = size / 2;
uint32_t mid_index = index + half_size;
TSCharacterRange *range = &ranges[mid_index];
const TSCharacterRange *range = &ranges[mid_index];
if (lookahead >= range->start && lookahead <= range->end) {
return true;
} else if (lookahead > range->end) {
@ -144,7 +165,7 @@ static inline bool set_contains(TSCharacterRange *ranges, uint32_t len, int32_t
}
size -= half_size;
}
TSCharacterRange *range = &ranges[index];
const TSCharacterRange *range = &ranges[index];
return (lookahead >= range->start && lookahead <= range->end);
}

9
gitnexus/vendor/tree-sitter-kotlin/package.json vendored
View file

@ -1,18 +1,19 @@
{
"name": "tree-sitter-kotlin",
"version": "0.4.0",
"version": "0.4.0-g1852ea1",
"description": "Kotlin grammar for tree-sitter",
"repository": "https://github.com/fwcd/tree-sitter-kotlin",
"license": "MIT",
"main": "bindings/node/index.js",
"types": "bindings/node/index.d.ts",
"_vendoredBy": "gitnexus - runtime package derived from tree-sitter-kotlin (fwcd) at unreleased main commit c8ac3d2627240160b999a2c100de3babbdb8f419 (package.json version 0.4.0; latest npm/tag is still 0.3.8). Pinned to main rather than a release to pull in `fun interface` (functional/SAM interface) support — PR fwcd/tree-sitter-kotlin#169, fixing issue #87 — which 0.3.8 (tagged 2024-08-03) lacks: it parsed `fun interface Foo` as an ERROR node and dropped the declaration. Because the fix is unreleased on npm, the prebuild workflow builds kotlin from THIS vendored source (kind 'vendored', like swift), NOT from the npm package. The grammar source (parser.c/scanner.c/binding.gyp + src/) is vendored so build-tree-sitter-grammars.cjs can source-build the binding on a toolchain host when no prebuild matches; the native prebuilds/ are GitNexus-cross-built by .github/workflows/build-tree-sitter-prebuilds.yml (regenerated whenever this `version` changes). The generated parser.c is large (~33 MB on disk; it compresses heavily in git). Loaded from vendor/ by absolute path at runtime (vendored-grammars.ts) — NEVER copied to node_modules (#2111) (no scripts.install here — #836/#1728). To re-pin: bump `version`, refresh src/ + bindings/node/ from the new upstream ref, update this note, and let the prebuild workflow rebuild the binaries.",
"_vendoredBy": "gitnexus — upstream generated src/ and bindings/node/binding.cc copied byte-for-byte from https://github.com/fwcd/tree-sitter-kotlin@1852ea17b7f60fb3f9d84e0b1555d56b46b39fb1. GitNexus keeps its synchronous CommonJS loader and hardened binding.gyp. Prebuilds are rebuilt from these sources by build-tree-sitter-prebuilds.yml. Do not edit generated C by hand.",
"peerDependencies": {
"tree-sitter": "^0.21.0"
"tree-sitter": "0.25.1"
},
"peerDependenciesMeta": {
"tree-sitter": {
"optional": true
}
}
},
"_upstreamCommit": "1852ea17b7f60fb3f9d84e0b1555d56b46b39fb1"
}

BIN
gitnexus/vendor/tree-sitter-kotlin/prebuilds/darwin-arm64/tree-sitter-kotlin.node vendored
View file

Binary file not shown.

Some files were not shown because too many files have changed in this diff Show more