GitNexus/.github/workflows/tree-sitter-upgrade-readiness.yml
Gergő Magyar 999cc9f913
fix(deps): upgrade native tree-sitter to 0.25.1 (#3531)
* fix(deps): upgrade native tree-sitter to 0.25.1

* fix(tree-sitter): preserve sync loading and update verified capture baselines

Route C# scope queries through the synchronous native grammar loader so the compiled analyze module remains require-compatible on Node 22.18. Refresh only the audited empty-Python-module and renamed-PHP-fixture captures, and pin the runtime assertion to 0.25.1.

Validated 479 targeted tests, all 16 scope-capture benchmark cases, Python capture/import guards, compiled C# worker parsing, build, typecheck and formatting. Performance budgets are unchanged.

* fix(tree-sitter): refresh upstream vendors and preserve parser semantics

* fix(ci): refresh prebuilds and support Windows ARM

Refresh macOS, Linux ARM, and Windows x64 grammar binaries from the successful native CI builds. Pin node-gyp 12.4.0 so the Windows ARM jobs recognize Visual Studio 2026.

Use local Git filtering for E2E to avoid PR-files API timeouts on generated parser diffs.

Validation: 143 focused tests, Actionlint, and all 16 scope-capture benchmark checks passed. Full npm test run started; native database suites still running.

* fix(ci): refresh Windows ARM grammar prebuilds

Import all seven Windows ARM grammar artifacts from successful native build run 37965761760. Artifact filenames and SHA-256 hashes verified; all 42 native build jobs passed.

Validation: 31 prebuild coverage tests passed.

* fix(bench): stabilize scope capture timing guard

Warm each input size seven times and use 21 median samples while retaining isolated timing blocks. Yield between languages so native finalizers release allocations outside the measured sections. Keep all fixture sizes, fingerprints, scaling formulas, and budgets unchanged.

Validation: all 16 language guards passed with a 2 GB heap limit (1.38 deep-chain ratio; 1518 MB peak RSS). Linear and quadratic controls passed their expected guard outcomes. Formatting and graph change analysis passed. Full npm test is running without failures so far.

* fix(parser): reduce TypeScript capture traversal cost

* fix(test): await parser worker readiness before teardown

* fix(parser): load C# synchronously in workers

* fix(grpc): resolve native proto detections against their own file

With the native proto grammar active, sibling .proto files that declare
the same short service name under different packages tied in the
directory-based resolveProtoConflict heuristic and both providers were
dropped. A proto-origin detection already knows its defining file, so
use that file's entry before falling back to the heuristic.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* ci(packaging): assert native grammars load from the installed tarball

The packaged-install smoke only ran `gitnexus --version`, so a grammar
that failed to load from the published package (bundled runtime, patched
peer manifests, vendored prebuilds, C# sync loader, Protobuf wrapper)
would ship silently; the dev-tree ABI smoke also accepts vendored
grammars that report unavailable. Parse a sample for every registered
grammar plus the native Protobuf plugin from the installed package, on
the main thread and in a worker thread.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* ci(tree-sitter): track compatibility workarounds instead of 0.25 readiness

The daily readiness report still targeted tree-sitter@0.25.0, so after the
upgrade it would keep reporting "Blocked" and never close its tracking issue.
Repurpose it to track what we carry for the bundled runtime: derive the target
from the package.json pin, list each pack-time peer patch from
prepare-tree-sitter-bundle.cjs with its drop condition (bump, wait for an
upstream release, transitive, or intentionally pinned), and reword the
buckets. The workflow adopts the issue opened under the old title.

Also update the grammar-update-monitor header and auto-PR text, which still
described the 0.21.1 runtime and a tree-sitter-c hold.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(dart): assert both nested labels target the loop; fix Zig call-shape doc

The nested-label CFG test only jumped to `outer` and checked reachability,
which still passes when a jump is retargeted to the inner loop (its exit
reaches the outer header). Jump to both labels and assert the exact
`continue` edges; dropping the inner label now fails the test.

Update the Zig callable-flow comment: call_expression now wraps actuals in a
required `arguments:` field.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 13:07:59 +03:00

257 lines
11 KiB
YAML

name: Tree-sitter Compatibility Workarounds
# Tracks the workarounds GitNexus carries for its bundled tree-sitter runtime:
# 1. Peer-dep patches — which npm grammars still reject the runtime, so
# prepare-tree-sitter-bundle.cjs must keep patching their manifests at pack
# time, and which patches a grammar bump would let us drop.
# 2. Vendored grammars — each grammar in .github/vendored-grammars.json
# (c/swift/kotlin/dart/proto/objc) is classified by its vendored ABI, read
# straight from gitnexus/vendor/<name>/src/parser.c (NOT node_modules,
# which is never populated for vendored grammars — that mismatch is why
# the report used to render bare "?" placeholders, #858).
# See .github/scripts/check-tree-sitter-upgrade-readiness.py for the logic.
#
# .github/vendored-grammars.json is the SHARED source of truth for the vendored
# SET + policy holds: this readiness report and grammar-update-monitor.yml both
# read it, so the two workflows can never disagree about which grammars are
# vendored. (The monitor also resolves upstreams from it; this report keeps its
# own upstream-drift coords and reads vendored ABIs from gitnexus/vendor/.)
#
# Concurrency convention: see CONTRIBUTING.md → "GitHub Actions — Concurrency Convention".
on:
schedule:
# Daily at 09:00 UTC. Matches Dependabot's daily cadence so drift
# and dep PRs surface together.
- cron: '0 9 * * *'
workflow_dispatch:
pull_request:
paths:
- '.github/scripts/check-tree-sitter-upgrade-readiness.py'
- '.github/scripts/test_check_tree_sitter_upgrade_readiness.py'
- '.github/vendored-grammars.json'
- '.github/workflows/tree-sitter-upgrade-readiness.yml'
- 'gitnexus/scripts/prepare-tree-sitter-bundle.cjs'
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
permissions:
contents: read
jobs:
report:
name: Check upgrade readiness
runs-on: ubuntu-latest
timeout-minutes: 10
# Least privilege: rendering the report needs no write. The issue mutation
# lives in the schedule-only `upsert-issue` job below, so PR runs (incl. forks)
# never receive `issues: write` (#2187 review).
permissions:
contents: read
outputs:
report: ${{ steps.readiness.outputs.report }}
exit_code: ${{ steps.readiness.outputs.exit_code }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- uses: ./.github/actions/setup-gitnexus
with:
build: 'false'
# Guard the readiness script's logic (vendored classification, no bare "?",
# the manifest⇄vendor-dir consistency guard). Stdlib-only, so no extra deps;
# node_modules is populated by setup-gitnexus above, which the npm-path ABI
# reads need. Runs only on validation events (PR / manual), not the daily
# scheduled report.
- name: Run readiness script unit tests
if: github.event_name != 'schedule'
shell: bash
working-directory: .github/scripts
run: python3 -m unittest test_check_tree_sitter_upgrade_readiness -v
- name: Run upgrade readiness check
id: readiness
shell: bash
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set +e
python3 .github/scripts/check-tree-sitter-upgrade-readiness.py > drift-report.md
code=$?
set -e
echo "exit_code=$code" >> "$GITHUB_OUTPUT"
# Unguessable per-run heredoc delimiter: the report includes the manifest's
# `hold` field, which a fork PR can edit — a fixed delimiter (e.g. DRIFT_EOF)
# in a hold value could close the heredoc early and inject $GITHUB_OUTPUT keys.
# A random hex delimiter the report cannot contain neutralizes that.
DELIM="DRIFT_EOF_$(openssl rand -hex 16)"
{
echo "report<<${DELIM}"
cat drift-report.md
echo "${DELIM}"
} >> "$GITHUB_OUTPUT"
echo "=== Report ==="
cat drift-report.md
# On PR runs, the script validates that it runs correctly. Blockers
# are informational — the scheduled run opens a tracking issue.
- name: Annotate PR with readiness status
if: github.event_name == 'pull_request' && steps.readiness.outputs.exit_code != '0'
run: |
echo "::notice::Tree-sitter compatibility workarounds remain. See job output for the full report."
# Issue mutation is isolated here so `issues: write` is only ever granted on the
# scheduled run (never on PRs). Consumes the report + exit_code via job outputs.
upsert-issue:
name: Upsert tracking issue
needs: report
if: github.event_name == 'schedule'
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
issues: write
steps:
- name: Upsert tracking issue while workarounds remain
if: needs.report.outputs.exit_code != '0'
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
env:
REPORT: ${{ needs.report.outputs.report }}
with:
script: |
const title = 'Tree-sitter compatibility workarounds';
// Adopt the issue opened under the pre-0.25 title instead of opening a twin.
const titles = [title, 'Tree-sitter 0.25 upgrade readiness'];
const report = process.env.REPORT;
const body = report + '\n\n' +
'<sub>Generated daily by `.github/workflows/tree-sitter-upgrade-readiness.yml`. ' +
'Closes automatically when no workarounds remain.</sub>';
const { data: open } = await github.rest.issues.listForRepo({
owner: context.repo.owner,
repo: context.repo.repo,
state: 'open',
labels: 'tree-sitter-drift',
per_page: 10,
});
const existing = open.find(i => titles.includes(i.title));
if (existing) {
// Extract ready/total count for the changelog comment.
// The two report.match() regexes below are mirrored as
// _ISSUE_READY_RE / _ISSUE_BLOCKER_RE in
// .github/scripts/test_check_tree_sitter_upgrade_readiness.py, which is
// the ONLY place the contract is asserted against the rendered report.
// Keep all three in sync: changing the report prose means updating both
// these literals AND the test mirror, or requireMatch throws on the next
// scheduled run (the silent "?" fallback that used to hide drift is gone).
const requireMatch = (name, match) => {
if (!match) {
throw new Error(
`Could not extract ${name} from tree-sitter readiness report`,
);
}
return match;
};
const readyMatch = requireMatch(
'ready npm grammar count',
report.match(/- (\d+)\/(\d+) npm-installed grammars already accept tree-sitter@/),
);
const blockerMatch = requireMatch(
'blocker count',
report.match(/\*\*Workarounds\*\* — (\d+) grammars? /),
);
const ready = readyMatch[1];
const total = readyMatch[2];
const blockers = blockerMatch[1];
// Find grammars whose status changed by diffing the old and
// new table rows. Each row looks like:
// | `tree-sitter-foo` | ... | Ready |
// | `tree-sitter-foo` | ... | Blocking |
const parseRows = (md) => {
const map = {};
// Group 2 captures ONLY the Status cell ([^|]+? before the final
// `|$`), so change-detection fires on status transitions, not on
// unrelated cell drift (e.g. an upstream-ABI bump). Mirror this in
// _ROW_DIFF_RE in test_check_tree_sitter_upgrade_readiness.py.
for (const m of md.matchAll(/\| `(tree-sitter-[^`]+)` \|.*\| ([^|]+?) \|$/gm)) {
map[m[1]] = m[2].trim();
}
return map;
};
const oldRows = parseRows(existing.body || '');
const newRows = parseRows(report);
const changes = [];
for (const [name, newStatus] of Object.entries(newRows)) {
const oldStatus = oldRows[name];
if (oldStatus && oldStatus !== newStatus) {
changes.push(`\`${name}\`: ${oldStatus} → ${newStatus}`);
}
}
const today = new Date().toISOString().slice(0, 10);
let comment = `**${today}:** ${ready}/${total} npm-installed accept the runtime. ${blockers} workaround(s) remaining.`;
if (changes.length > 0) {
comment += '\n\nChanges:\n' + changes.map(c => `- ${c}`).join('\n');
} else {
comment += ' No changes from previous run.';
}
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: existing.number,
body: comment,
});
await github.rest.issues.update({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: existing.number,
title,
body,
});
core.info(`Updated existing issue #${existing.number}`);
} else {
const { data: created } = await github.rest.issues.create({
owner: context.repo.owner,
repo: context.repo.repo,
title,
body,
labels: ['tree-sitter-drift', 'dependencies'],
});
core.info(`Opened issue #${created.number}`);
}
- name: Close tracking issue on clean runs
if: needs.report.outputs.exit_code == '0'
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with:
script: |
const title = 'Tree-sitter compatibility workarounds';
// Adopt the issue opened under the pre-0.25 title instead of opening a twin.
const titles = [title, 'Tree-sitter 0.25 upgrade readiness'];
const { data: open } = await github.rest.issues.listForRepo({
owner: context.repo.owner,
repo: context.repo.repo,
state: 'open',
labels: 'tree-sitter-drift',
per_page: 10,
});
const existing = open.find(i => titles.includes(i.title));
if (existing) {
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: existing.number,
body: 'No compatibility workarounds remain: every grammar runs on the bundled tree-sitter runtime as published. Closing automatically.',
});
await github.rest.issues.update({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: existing.number,
state: 'closed',
});
core.info(`Closed issue #${existing.number}`);
}