Commit graph

13 commits

Author SHA1 Message Date
Gergo Magyar
4b9762577e perf(scope-resolution): O(1) workspace lookup index
Introduces `WorkspaceResolutionIndex` — a precomputed bundle of
lookup tables built ONCE per resolution run, after `populateOwners`
and after finalize, before any pass that needs to find members,
exported defs, or class scopes by id.

What it replaces (all are pre-existing O(N×D) linear scans of
parsedFiles, called inside the receiver-bound MRO chain):

- `findOwnedMember(ownerId, name, parsedFiles)` → `Map.get` via
  `index.memberByOwner.get(ownerId)?.get(name)`. Was the worst
  offender — receiver-bound dispatcher calls this O(sites × MRO
  depth) times.
- `findExportedDef(filePath, name, parsedFiles)` → `Map.get` via
  `index.defsByFileAndName`. Hot for namespace-receiver case.
- `findExportedDefByName` workspace-wide fallback scan → `Map.get`
  via `index.callablesBySimpleName`.
- `classScopeByDefId` (rebuilt inside `emitReceiverBoundCalls` on
  every invocation) — moved to one-shot build during finalize, read
  from `index.classScopeByDefId` everywhere.
- `moduleScopeByFile` (rebuilt inside `propagateImportedReturnTypes`
  on every invocation) — read from `index.moduleScopeByFile`.

Findings from a synthetic 100-file Python workload (60 model files
each defining 5 classes × 3 methods + 40 user files calling them
heavily):

  scope-resolution wall time: 764ms → 710ms (median, 5 iters)

That's a ~7% in-layer win. The smaller-than-expected gain was
informative: profiling the synthetic workload shows scope-resolution
breakdown is `extract=62% resolve=30% emit=4%`; the index touched
the 4% slice (emit + walker calls inside it). Larger O(D) per owner
classes will benefit more.

Profiling the FULL pipeline (49 fixtures × 3 iters) shows
scope-resolution accounts for ~1% of pipeline wall time — the
remaining 99% is parse (tree-sitter), heritage, ORM, MRO, processes,
and DB writes. So further optimization of this specific layer has
marginal pipeline impact; the next-biggest wins live in those
phases. Documented as the "double-parse" finding in the audit
(captures.ts re-parses each Python file even though the parse phase
already produced a tree-sitter Tree) — that's a separate plumbing
project across phase boundaries.

Bonus: opt-in PROF_SCOPE_RESOLUTION=1 env var prints a per-phase
ms breakdown to stderr, so future perf work can measure without
extra code changes.

Verification:
- REGISTRY_PRIMARY_PYTHON=0 (legacy): 191/191.
- REGISTRY_PRIMARY_PYTHON=1 (registry): 191/191.
- tsc --noEmit clean.
2026-04-20 15:21:28 +01:00
Gergo Magyar
6a1b4227eb ci(scope-resolution): automatic parity gate driven by MIGRATED_LANGUAGES
Adds the Ring 3 parity gate the RFC §6.4 requires: when a language's
scope-resolution migration is marked complete, CI runs its resolver
integration test twice on every PR (once with the legacy DAG, once with
the registry-primary path) and both must pass.

The "is this language migrated" signal is a single TypeScript constant:

  // gitnexus/src/core/ingestion/registry-primary-flag.ts
  export const MIGRATED_LANGUAGES: ReadonlySet<SupportedLanguages> =
    new Set([ /* SupportedLanguages.Python when ready */ ]);

Adding a language here has three simultaneous effects:

  1. `isRegistryPrimary(lang)` defaults to true for that language in
     production (env-var override still wins if set explicitly).
  2. `.github/workflows/ci-scope-parity.yml` auto-discovers the set via
     `npx tsx scripts/ci-list-migrated-languages.ts`, builds a parity
     matrix, and runs:
       - `REGISTRY_PRIMARY_<LANG>=0 npx vitest run resolvers/<slug>.test.ts`
       - `REGISTRY_PRIMARY_<LANG>=1 npx vitest run resolvers/<slug>.test.ts`
     Both legs must pass for the job to succeed.
  3. Legacy-path gating in call-processor.ts / import-processor.ts kicks
     in automatically through the same `isRegistryPrimary` lookup.

No JSON registry, no manual workflow edit, no second source of truth —
contributors update the Set and CI picks it up. Empty Set = parity job
is a skipped matrix (workflow still reports success).

The new `scope-parity` reusable workflow is added to ci.yml's `needs`
graph and ci-status gate. Its result must be `success` (skipped would
mean upstream discover job failed and should block).

Validation (with empty MIGRATED_LANGUAGES set):
- flag OFF: 191/191 pass (no behavior change)
- flag ON (manual REGISTRY_PRIMARY_PYTHON=1): 82 fails = baseline exact match
- `npx tsc --noEmit`: clean
- concurrency-convention script: pass
- tsx discovery script: emits `[]` correctly
2026-04-19 12:06:45 +01:00
Gergő Magyar
fec06b823c
fix: devendor tree-sitter-proto install lifecycle to prevent ENOTEMPTY on global upgrade (#846)
Some checks are pending
CI / quality (push) Waiting to run
CI / tests (push) Waiting to run
CI / e2e (push) Waiting to run
CI / global-upgrade (push) Waiting to run
CI / Save PR Metadata (push) Blocked by required conditions
CI / CI Gate (push) Blocked by required conditions
Release Candidate / Check if release candidate should run (push) Waiting to run
Release Candidate / ci (push) Blocked by required conditions
Release Candidate / Publish release candidate to npm (push) Blocked by required conditions
* fix: devendor tree-sitter-proto install lifecycle to fix ENOTEMPTY on global upgrade

PR #843's preinstall cleanup hook cannot address the reported bug because
it runs on the NEW package's staging tree, not the OLD install being
removed. Issue #836 still reproduces on 1.6.2-rc.8.

Root cause: vendor/tree-sitter-proto was declared as `file:` dep with its
own `dependencies` and `install` script, so npm created
`vendor/tree-sitter-proto/node_modules/node-addon-api/` at install time,
which blocked npm's rmdir on global upgrade.

Changes:
- Strip `dependencies` and `install` script from the vendored sub-package's
  package.json so npm no longer creates a nested node_modules or runs a
  lifecycle script under vendor/.
- Hoist `node-addon-api` and `node-gyp-build` into gitnexus
  optionalDependencies; npm resolves them at the consumer's top level.
- Add scripts/build-tree-sitter-proto.cjs modeled on patch-tree-sitter-swift.cjs.
  Runs at gitnexus postinstall, best-effort: skips cleanly on missing
  toolchain or --ignore-scripts so non-proto functionality keeps working.
- Remove scripts/preinstall-cleanup.cjs — dead code; cannot run against
  the old install being removed.
- Keep .npmignore entries from PR #843 (tarball hygiene, still correct).
- Add explicit .gitignore rules for gitnexus/vendor/**/build and
  gitnexus/vendor/**/node_modules (closes the repo-side hygiene gap).
- Add .github/workflows/ci-global-upgrade.yml: matrix smoke test that
  installs the previously-published rc globally, upgrades to the packed
  current branch, and verifies no vendor install-time artifacts survive.
  Runs on macOS (reporter's platform), Linux, and Windows. Also includes
  an --ignore-scripts degraded-mode lane. Wired into ci.yml gate.

Plan: docs/plans/2026-04-15-002-fix-tree-sitter-proto-vendor-deps-plan.md

Phase 1 (this commit) addresses the reported `node_modules/node-addon-api`
hazard. Phase 2 (follow-up) will migrate to prebuildify + prebuilt .node
binaries in the tarball — the 2026 canonical shape for tree-sitter
grammars, which eliminates the postinstall compile path entirely.

Refs #836

* fix(ci): ci-global-upgrade should be reusable-only and use setup-gitnexus

Three issues caught by CI on PR #846:

1. Concurrency linter rejected the `CIGU-` prefix (allowlist is
   `${{ github.workflow }}` or substring `CI-`). The literal-prefix
   guidance in ci.yml is specifically about disambiguating when
   reusable workflows run in nested contexts, and ci-global-upgrade
   doesn't need its own concurrency block at all — the caller
   (ci.yml) already governs concurrency for nested invocations.

2. `npm install` in gitnexus/ runs `prepare: node scripts/build.js`,
   which depends on gitnexus-shared/dist being built first. Other CI
   jobs handle this via the setup-gitnexus composite action. Use it
   here too (with build: 'false' — we only need the dep graph, then
   npm pack runs prepack which builds gitnexus itself).

3. Removed `pull_request` and `workflow_dispatch` triggers. The
   workflow is now pure `workflow_call` — invoked once from ci.yml
   via `uses:`. This avoids the duplicate-run problem where both the
   top-level pull_request trigger AND the nested workflow_call would
   fire on every PR.

* fix(ci): relax vendor build/ guard and use bash shell on Windows

Two fixes for ci-global-upgrade failures on PR #846:

1. The guard after the upgrade step was rejecting vendor/tree-sitter-proto/build/
   in the global install. That was too strict. The original #836 bug was
   about vendor/tree-sitter-proto/node_modules/ specifically, not build/.
   The build/ directory appears because node-gyp-build compiles through the
   symlink npm creates at node_modules/gitnexus/node_modules/tree-sitter-proto,
   and its contents are plain .node, .obj, .lib files that rmdir handles
   without trouble. We know this empirically because the test got past the
   upgrade step in the run where the old vendor/node_modules was present.
   The guard now only flags nested node_modules, which is what the fix
   actually removes.

2. The Windows --ignore-scripts lane failed with ENOENT when npm tried to
   open the tarball. The path was computed in a bash step using $(pwd),
   which on Windows returns /d/a/... form, but npm install ran in the
   default cmd shell and received a mangled Windows path. Adding
   shell: bash to the install steps keeps path handling consistent.
2026-04-15 17:38:47 +01:00
Copilot
7a5ab57bd3
fix: add preinstall cleanup to prevent ENOTEMPTY on global upgrade (#843)
* Initial plan

* fix: add preinstall cleanup for vendor/tree-sitter-proto to prevent ENOTEMPTY on upgrade

When upgrading gitnexus globally, npm may fail with ENOTEMPTY because it
cannot cleanly remove node_modules/ and build/ directories that a previous
installation's file: dependency resolution created inside
vendor/tree-sitter-proto/.

Add a preinstall script that removes those leftover directories before npm
resolves dependencies. Also add .npmignore entries for vendor build artifacts
as a belt-and-suspenders measure.

Fixes #836

Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/8b7c1fdd-0c20-4cf4-a64a-9e9d1c0b20ed

Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com>

* fix: log warnings in preinstall cleanup catch block

Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/8b7c1fdd-0c20-4cf4-a64a-9e9d1c0b20ed

Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com>
2026-04-15 14:10:49 +01:00
Dave Brophy
9364739fb4
fix: restore tree-sitter-swift postinstall patch for macOS ARM64 (#788)
* fix: restore tree-sitter-swift postinstall patch for macOS ARM64

PR #516 (77dcb06) deleted `scripts/patch-tree-sitter-swift.cjs` and
the `postinstall` script entry when bumping to `tree-sitter-swift@0.7.1`,
since 0.7.1 ships prebuilt darwin-arm64 binaries and no longer needs the
patch. PR #538 (01ddc3e) then had to revert `tree-sitter-swift` back to
`^0.6.0` (and `tree-sitter` back to `^0.21.1`) because `npm overrides`
doesn't apply when gitnexus is installed via `npx -y` (gitnexus isn't the
root project, so overrides are silently ignored, producing ERESOLVE errors).

PR #538 reverted the grammar package changes but did not restore the patch
script, leaving `tree-sitter-swift@0.6.0` unable to build its native
binding on macOS ARM64. The symptom is `gitnexus analyze` printing
"Skipping swift" or "swift parser not available".

`Dockerfile.test` still references `node scripts/patch-tree-sitter-swift.cjs`
(added in the same PR #516), confirming the regression — the test image
build is also broken.

This commit restores the patch script from commit `0c8ec95` (the last
revision before it was deleted) and re-adds the `postinstall` entry to
`package.json`. No logic changes — it is an exact restoration.

The TODO comment in the script ("Remove this script when tree-sitter is
upgraded to ^0.22.x") still applies.

* style: run prettier on patch-tree-sitter-swift.cjs
2026-04-11 18:24:02 +01:00
Abhigyan Patwari
dcf40da3a8 fix: ensure import rewrites survive npm publish lifecycle
npm runs `prepare` after `prepack` during publish, so the previous
`prepare: tsc` overwrote the rewritten imports before packing.
Both `prepare` and `prepack` now run the full build script so the
tarball always contains rewritten relative imports.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 17:28:14 +05:30
Abhigyan Patwari
71255a096c
fix: bundle gitnexus-shared into CLI dist (#613)
* fix(ci): build gitnexus-shared before publish, use CHANGELOG for release notes

The publish workflow was missing the gitnexus-shared build step that
the setup-gitnexus composite action provides. Since PR #536 unified
the ingestion pipeline, gitnexus imports types from gitnexus-shared,
so it must be built first.

Also replaces generate_release_notes with CHANGELOG.md extraction so
GitHub Releases use the reviewed changelog entry instead of a flat
PR title list.

Made-with: Cursor

* fix: bundle gitnexus-shared into CLI dist to fix module resolution

gitnexus-shared was declared as a file: dependency but never published
to npm, causing ERR_MODULE_NOT_FOUND for users installing gitnexus
globally. The build script now copies gitnexus-shared/dist into
dist/_shared/ and rewrites bare specifiers to relative paths.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: move gitnexus-shared to devDependencies, use tsc for prepare

gitnexus-shared must remain available for tsc to resolve imports during
development/CI, but is not needed at runtime since it's bundled into
dist/_shared/. Moving it to devDependencies keeps it out of production
installs while allowing compilation. The prepare script now runs plain
tsc (no shared bundling needed for local dev).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Abhigyan Patwari <abhigyan@Abhigyans-MacBook-Air.local>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 17:04:39 +05:30
Wojciech Guziak
77dcb06a8d
chore: upgrade tree-sitter to 0.25.0 and all grammar packages (#516) 2026-03-25 21:55:14 +00:00
marxo126
0c8ec952ee fix: handle trailing commas in tree-sitter-swift binding.gyp patch
The patch script fails to parse tree-sitter-swift@0.6.0's binding.gyp
because the file contains both Python-style # comments AND trailing
commas in JSON arrays. The existing regex strips # comments but leaves
trailing commas, causing JSON.parse() to fail with:

  "Unexpected token ']'"

This silently prevents tree-sitter-swift from building, which means
Swift files are skipped entirely during analysis.

Fix: add a second regex pass to strip trailing commas before ] or }
after comment removal.

Fixes #386, #406

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-22 11:46:34 +01:00
林 駿甫 (Shunsuke Hayashi)
3879490817
fix: add postinstall permission fix for CLI and hook scripts (#330) (#348) 2026-03-18 05:41:37 +00:00
jandyx
7a4bc9a260 docs: improve postinstall script comments with background and TODO 2026-02-27 11:39:14 +08:00
jandyx
f557716998 fix(swift): improve postinstall to auto-rebuild after patching binding.gyp
The script now detects missing native binding and runs node-gyp rebuild
after patching. This handles the case where tree-sitter-swift's own
postinstall fails during npm install — our postinstall picks up,
patches binding.gyp, and rebuilds successfully.
2026-02-27 11:14:45 +08:00
jandyx
ae8a76511d fix(swift): address review gaps — schema, call-processor, web package, postinstall
- Add init_declaration/deinit_declaration to call-processor FUNCTION_NODE_TYPES
  and findEnclosingFunction (syncs with parse-worker, avoids Dart PR #83 rejection)
- Add 7 missing CodeRelation FROM-TO pairs in schema.ts to eliminate analyze warnings
  (Function→Property, Constructor→Property/Typedef, Enum→Class/Interface,
   Struct→Interface, TypeAlias→Class)
- Mirror all Swift support to gitnexus-web: supported-languages, utils, queries,
  framework-detection, entry-point-scoring, parser-loader WASM path
- Add postinstall script to patch tree-sitter-swift binding.gyp actions array
2026-02-27 09:47:05 +08:00