* feat(extractors): detect jQuery $.ajax/$.get/$.post and axios object-form as HTTP consumers
The JS/TS HTTP consumer extractor currently recognises fetch() and
axios.<verb>() but misses three patterns extremely common in Laravel
and legacy frontends:
- jQuery shorthand: $.get(url), $.post(url, data)
- jQuery ajax form: $.ajax({ url, method }) / $.ajax({ url, type })
- axios object form: axios({ method, url })
Missing them means the frontend->backend cross-link disappears from
`group sync`, breaking impact analysis for whole classes of repos.
Implementation (node.ts):
- 3 new PatternSpecs alongside the existing FETCH_/AXIOS_ specs
- NodePatternBundle extended with jqueryShorthand / jqueryAjax /
axiosObject slots, compiled for JS / TS / TSX grammars
- readStringProp() helper walks object-literal `pair` children and
resolves `url` / `method` / `type` keys independent of order,
sidestepping the positional S-expression constraint on the
query form proposed in the issue
- 3 new scan loops in scanBundle() emit HttpDetection with
framework 'jquery' (new) or 'axios' (existing), confidence 0.7
to match the existing source-scan consumers, defaulting method
to GET when absent (matches both jQuery and axios runtime)
Tests (http-route-extractor.test.ts): 4 new cases -- 3 positive
(shorthand, ajax with method:/type: and default GET, object-form
with swapped key order and default GET) plus 1 negative control
that asserts unrelated \$.fn.extend / \$.each / non-axios helper
calls with {url, method} literals produce zero consumer contracts.
Closes#828
* test(extractors): cover jQuery $.ajax with template-literal URL
Extend the existing $.ajax fixture with `url: \`/api/orders/\${id}\``
and assert the consumer is emitted as http::GET::/api/orders/{param}.
This makes jQuery + template-URL explicit rather than implicit via the
axios object test (readStringProp already accepts template_string for
both; this is coverage, not new behaviour).
Addresses the single non-blocking finding on PR #887.
* Initial plan
* feat(ingestion): add variable extraction types, factory, configs, and wire into language providers
- Create variable-types.ts with VariableInfo, VariableExtractionConfig, VariableExtractor interfaces
- Create variable-extractors/generic.ts with createVariableExtractor() factory
- Add variableExtractor field to LanguageProvider interface
- Create per-language variable extraction configs for all 16 languages
- Wire variableExtractor into all language providers
- Add variable metadata enrichment to parse-worker for Const/Static/Variable labels
Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/3cb85c68-1792-473e-9a46-ea2588da0e5e
Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com>
* feat(ingestion): add variable extraction tests and fix Python/TS config issues
- Create test/unit/variable-extraction.test.ts with 29 tests covering
TypeScript, JavaScript, Python, Go, Rust, C, C++, Ruby, and factory behavior
- Fix isConst in generic factory to use config.isConst over node-type membership
(TS let/const both use lexical_declaration)
- Fix Python type extraction for annotated assignments at module scope
- Fix Python dunder name visibility (e.g., __name__ is public, not protected)
Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/3cb85c68-1792-473e-9a46-ea2588da0e5e
Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com>
* fix: address code review feedback — move imports, clarify scope comment, use shared test context
Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/3cb85c68-1792-473e-9a46-ea2588da0e5e
Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com>
* fix: address review comments, fix prettier formatting and lint errors
- Fix prettier formatting in 5 files (c-cpp, jvm, swift configs, test file)
- Remove unused SyntaxNode imports in php.ts and ruby.ts (lint errors)
- Remove unused constNodeSet/variableNodeSet variables in generic.ts (warnings)
- Remove semantically wrong `methodProps.isReadonly = varInfo.isConst` (review)
- Remove dead `nodeLabel === 'Variable'` guard in parse-worker (review)
- Fix test guard: replace `if (declNode)` with `expect(declNode).toBeDefined()` (review)
- Add comment about Python expression_statement broadness (review)
Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/040edbbf-65b5-40e1-80c8-e98f7c4bb54a
* feat(ingestion): add block-scoped variable extraction via tree-sitter queries
Add @definition.const and @definition.variable tree-sitter query patterns
for TypeScript, JavaScript, Python, Go, Java, C, C++, C#, PHP, Ruby, and
Dart. Add parse-worker dedup logic to avoid duplicate nodes when variable
captures overlap with existing function/property captures. Add 'Variable'
label support in getLabelFromCaptures and DEFINITION_CAPTURE_KEYS.
Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/9fa828c1-87b7-4482-8f26-d2079fb4c58a
Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com>
* test: add block-scoped variable extraction tests and query capture tests
Add 6 tests for block-scoped variable extraction (TypeScript, Go, Rust, C,
Python). Add 14 tests verifying @definition.const/@definition.variable
query patterns exist in all language query strings. Import RUBY_QUERIES
in test file.
Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/9fa828c1-87b7-4482-8f26-d2079fb4c58a
Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com>
* test: add Python non-assignment expression statement rejection test
Addresses code review feedback: verify that the Python variable extractor
returns null for expression_statement nodes that contain function calls
rather than assignments (e.g. `print("hello")`).
Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/9fa828c1-87b7-4482-8f26-d2079fb4c58a
Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com>
* fix: Dart query node type, add Variable schema, update schema counts
- Change `top_level_variable_declaration` → `declaration` in DART_QUERIES
(the former doesn't exist in tree-sitter-dart grammar, causing all
Dart integration tests to fail with TSQueryErrorNodeType)
- Add VARIABLE_SCHEMA to schema.ts and register in initLbug() so that
Variable-labeled nodes are persisted to LadybugDB (not silently dropped)
- Add 'Variable' to MULTI_LANG_TYPES in csv-generator.ts
- Update Dart variable config to remove invalid node type
- Update schema test counts (30→31 node schemas, 32→33 total)
Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/f79931d1-207f-4fbb-91da-259d44f7fd88
Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com>
* fix: address code review comment improvements
- Clarify processedDefinitionNodes tracks start indices, not nodes
- Improve Python variableNodeTypes comment wording
Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/f79931d1-207f-4fbb-91da-259d44f7fd88
Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com>
* fix: add Variable to NODE_TABLES, RELATION_SCHEMA, update golden snapshot
- Add 'Variable' to NODE_TABLES in gitnexus-shared so validTables.has('Variable')
returns true and Variable graph edges are not silently dropped
- Add FROM File TO Variable, FROM Variable TO Community, FROM Variable TO Process
to RELATION_SCHEMA so KuzuDB can represent edges connecting Variable nodes
- Update schema.test.ts: add Variable to multiLang list, fix count 30→31
- Regenerate pipeline-graph-golden snapshot for mini-repo fixture
Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/e3aad558-e7bb-40d1-b53f-0a2c0132ca96
Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com>
* fix: isolate golden test from cli-e2e fixture pollution
The pipeline-graph-golden test was non-deterministic because cli-e2e.test.ts
creates AGENTS.md, CLAUDE.md, .claude/skills/, and .gitignore in the shared
mini-repo fixture during analyze. These leftover files caused the golden test
to find 9 files instead of 7 when tests ran in parallel.
Fixes:
- Golden test now copies the fixture to a temp dir before running, making it
immune to concurrent test pollution
- cli-e2e afterAll cleanup now removes ALL generated files (AGENTS.md,
CLAUDE.md, .claude/, .gitignore) not just .git/ and .gitnexus/
- Golden snapshot regenerated from clean 7-file fixture
Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/bd378e73-6f37-49c6-aed6-7fabf4dc6183
Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com>
* Initial plan
* refactor(ingestion): move class extraction configs to configs/ subdirectory
Extract inline ClassExtractionConfig objects from 13 language provider files
into 11 config files under class-extractors/configs/, matching the pattern
established by method-extractors/configs/ and field-extractors/configs/.
Pure structural refactor — zero behavioral change. All class extraction
tests pass unchanged.
Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/8ca2bf18-46ea-41c3-9c7f-9eb3f5752ade
Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com>
* fix: resolve prettier formatting in c-cpp.ts import
Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/1b1e3c43-fc0e-444c-b109-cc0c56ffb470
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com>
* chore(deps): add tree-sitter aware Dependabot config and drift monitoring
Two things Dependabot cannot see on its own:
1. ABI consistency. The tree-sitter runtime supports a known range of
grammar ABIs. When a grammar bumps past that range, require() silently
fails and fallback paths mask the regression in test coverage.
2. Vendored upstream drift. vendor/tree-sitter-proto is a snapshot of
coder3101/tree-sitter-proto regenerated against a pinned cli version.
Upstream keeps moving. Nothing notices until a maintainer remembers to
look.
Dependabot configuration
- Added npm ecosystems for gitnexus, gitnexus-web, gitnexus-shared.
- Grouped all tree-sitter-* grammar bumps into one PR (ecosystem moves in
lockstep, one PR per grammar is noise).
- Pinned the tree-sitter runtime itself. Bumping 0.21 to 0.22+ changes
which grammar ABIs load and requires coordinated updates to the
vendored proto grammar. That stays a deliberate human decision.
- Pinned tree-sitter-cli for the same reason (it controls which ABI
vendor/tree-sitter-proto/src/parser.c emits when regenerated).
Drift check (.github/scripts/check-tree-sitter-drift.py)
- Reads the tree-sitter runtime version from gitnexus/package.json.
- Walks every installed tree-sitter-* grammar plus the vendored proto
and reports its LANGUAGE_VERSION against the runtime's supported ABI
range (table maintained in the script; extend when bumping runtime).
- Fetches coder3101/tree-sitter-proto main parser.c and compares byte
for byte to the vendored copy. Reports the upstream HEAD short SHA
and the upstream ABI so a maintainer can act.
- Prints a Markdown report; exits 0 when everything is in range and
matches upstream, 1 otherwise.
- Stdlib only, no external deps.
Drift workflow (.github/workflows/tree-sitter-drift-check.yml)
- Runs weekly (Mondays 09:00 UTC) to match Dependabot's cadence.
- Also runs on PRs that touch the script or workflow itself, where it
fails the PR check on drift so the drift gate cannot land broken.
- On scheduled runs with drift, opens or updates a single tracking
issue labeled tree-sitter-drift. On scheduled runs that come back
clean, closes the open tracking issue (if any) with a comment.
* refactor(deps): rewrite drift check as tree-sitter 0.25 upgrade readiness monitor
Replace the ABI drift pass/fail gate with a daily upgrade readiness
dashboard that tracks peer-dep compatibility of all 14 grammars with
tree-sitter@0.25.0 and reports which are ready, unreleased, or blocking.
Key changes:
- Rename drift-check → upgrade-readiness (script, workflow, job id)
- Fix P0: pass report via env var, not ${{ }} template interpolation
- Fix P1: npm fetch failure now adds a blocker instead of false-green
- Fix P1: pass GITHUB_TOKEN for authenticated GitHub API calls
- Switch Dependabot to daily for tree-sitter grammars
- Use dict for blockers (no prefix collision), derive TARGET_RUNTIME
constant, reuse GRAMMARS parser_path, normalize CRLF in comparisons
- Reduce per-call HTTP timeout from 15s to 8s for workflow budget
- PR runs warn on blockers instead of hard-failing
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore(ci): remove global-upgrade smoke test workflow
The ci-global-upgrade.yml workflow tested npm global install upgrades
over a specific release candidate (1.6.2-rc.8). That RC has shipped
and the workflow is no longer needed. Remove it and all references
from ci.yml (needs, env vars, gate check).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(ci): add changelog comments to upgrade readiness tracking issue
Each daily run now posts a comment summarizing what changed before
updating the issue body. Comments include the ready/blocker counts
and a diff of grammar status changes (e.g. tree-sitter-cpp:
Unreleased -> Ready). Gives a timeline of how the upgrade unblocks.
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
release-drafter v7 (merged in #852) removed the `disable-releaser`
input, causing the autolabel job to attempt creating a release and
fail with "Resource not accessible by integration". Replace with
`dry-run: true` which achieves the same label-only behavior.
Also update stale version comments for release-drafter and
action-semantic-pull-request to match the actual pinned versions.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: devendor tree-sitter-proto install lifecycle to fix ENOTEMPTY on global upgrade
PR #843's preinstall cleanup hook cannot address the reported bug because
it runs on the NEW package's staging tree, not the OLD install being
removed. Issue #836 still reproduces on 1.6.2-rc.8.
Root cause: vendor/tree-sitter-proto was declared as `file:` dep with its
own `dependencies` and `install` script, so npm created
`vendor/tree-sitter-proto/node_modules/node-addon-api/` at install time,
which blocked npm's rmdir on global upgrade.
Changes:
- Strip `dependencies` and `install` script from the vendored sub-package's
package.json so npm no longer creates a nested node_modules or runs a
lifecycle script under vendor/.
- Hoist `node-addon-api` and `node-gyp-build` into gitnexus
optionalDependencies; npm resolves them at the consumer's top level.
- Add scripts/build-tree-sitter-proto.cjs modeled on patch-tree-sitter-swift.cjs.
Runs at gitnexus postinstall, best-effort: skips cleanly on missing
toolchain or --ignore-scripts so non-proto functionality keeps working.
- Remove scripts/preinstall-cleanup.cjs — dead code; cannot run against
the old install being removed.
- Keep .npmignore entries from PR #843 (tarball hygiene, still correct).
- Add explicit .gitignore rules for gitnexus/vendor/**/build and
gitnexus/vendor/**/node_modules (closes the repo-side hygiene gap).
- Add .github/workflows/ci-global-upgrade.yml: matrix smoke test that
installs the previously-published rc globally, upgrades to the packed
current branch, and verifies no vendor install-time artifacts survive.
Runs on macOS (reporter's platform), Linux, and Windows. Also includes
an --ignore-scripts degraded-mode lane. Wired into ci.yml gate.
Plan: docs/plans/2026-04-15-002-fix-tree-sitter-proto-vendor-deps-plan.md
Phase 1 (this commit) addresses the reported `node_modules/node-addon-api`
hazard. Phase 2 (follow-up) will migrate to prebuildify + prebuilt .node
binaries in the tarball — the 2026 canonical shape for tree-sitter
grammars, which eliminates the postinstall compile path entirely.
Refs #836
* fix(ci): ci-global-upgrade should be reusable-only and use setup-gitnexus
Three issues caught by CI on PR #846:
1. Concurrency linter rejected the `CIGU-` prefix (allowlist is
`${{ github.workflow }}` or substring `CI-`). The literal-prefix
guidance in ci.yml is specifically about disambiguating when
reusable workflows run in nested contexts, and ci-global-upgrade
doesn't need its own concurrency block at all — the caller
(ci.yml) already governs concurrency for nested invocations.
2. `npm install` in gitnexus/ runs `prepare: node scripts/build.js`,
which depends on gitnexus-shared/dist being built first. Other CI
jobs handle this via the setup-gitnexus composite action. Use it
here too (with build: 'false' — we only need the dep graph, then
npm pack runs prepack which builds gitnexus itself).
3. Removed `pull_request` and `workflow_dispatch` triggers. The
workflow is now pure `workflow_call` — invoked once from ci.yml
via `uses:`. This avoids the duplicate-run problem where both the
top-level pull_request trigger AND the nested workflow_call would
fire on every PR.
* fix(ci): relax vendor build/ guard and use bash shell on Windows
Two fixes for ci-global-upgrade failures on PR #846:
1. The guard after the upgrade step was rejecting vendor/tree-sitter-proto/build/
in the global install. That was too strict. The original #836 bug was
about vendor/tree-sitter-proto/node_modules/ specifically, not build/.
The build/ directory appears because node-gyp-build compiles through the
symlink npm creates at node_modules/gitnexus/node_modules/tree-sitter-proto,
and its contents are plain .node, .obj, .lib files that rmdir handles
without trouble. We know this empirically because the test got past the
upgrade step in the run where the old vendor/node_modules was present.
The guard now only flags nested node_modules, which is what the fix
actually removes.
2. The Windows --ignore-scripts lane failed with ENOENT when npm tried to
open the tarball. The path was computed in a bash step using $(pwd),
which on Windows returns /d/a/... form, but npm install ran in the
default cmd shell and received a mangled Windows path. Adding
shell: bash to the install steps keeps path handling consistent.
When @huggingface/transformers is installed globally (e.g. via npm install -g),
it defaults its cache directory to ./node_modules/.cache inside its own install
dir, which is unwritable by non-root users.
This causes EACCES errors on first use when the model is downloaded:
EACCES: permission denied, mkdir '/usr/lib/node_modules/gitnexus/node_modules/@huggingface/transformers/.cache'
Set env.cacheDir before pipeline() is called in both embedders (CLI and MCP).
Respects HF_HOME env var if set, falls back to ~/.cache/huggingface.
Co-authored-by: Sisyphus <sisyphus@gitnexus.dev>
* Initial plan
* fix: add preinstall cleanup for vendor/tree-sitter-proto to prevent ENOTEMPTY on upgrade
When upgrading gitnexus globally, npm may fail with ENOTEMPTY because it
cannot cleanly remove node_modules/ and build/ directories that a previous
installation's file: dependency resolution created inside
vendor/tree-sitter-proto/.
Add a preinstall script that removes those leftover directories before npm
resolves dependencies. Also add .npmignore entries for vendor build artifacts
as a belt-and-suspenders measure.
Fixes#836
Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/8b7c1fdd-0c20-4cf4-a64a-9e9d1c0b20ed
Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com>
* fix: log warnings in preinstall cleanup catch block
Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/8b7c1fdd-0c20-4cf4-a64a-9e9d1c0b20ed
Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com>
* ci: standardize workflow concurrency and automate release-note labeling
Concurrency — prevent racing CI jobs
- Every top-level workflow now declares an explicit concurrency block.
- PR runs cancel-in-progress on supersede; main/push/workflow_call/publish
runs queue instead of cancelling so every commit and every release is
validated end-to-end.
- ci.yml uses a literal `CI-` prefix (not `${{ github.workflow }}`) and a
per-run nested group for workflow_call invocations, avoiding a potential
deadlock with publish.yml and release-candidate.yml callers whose own
concurrency groups could otherwise collide with the called workflow.
- ci-report.yml falls back to `<head-repo>/<head-branch>` for fork PRs
(stable across reruns) instead of the per-run-unique workflow_run.id
which did not actually serialize anything.
- ci-quality.yml enforces the convention: fails CI if any non-reusable
workflow lacks a concurrency block or a reusable workflow declares one.
Release-note automation
- New pr-labeler.yml: amannn/action-semantic-pull-request enforces
conventional-commit PR titles on pull_request (fork-safe, read-only);
release-drafter/release-drafter with disable-releaser: true applies the
matching label under pull_request_target (write-scoped). sync-labels in
.github/release-drafter.yml removes managed autolabels that no longer
match (e.g. when `!` or `BREAKING CHANGE:` is dropped from a PR).
- .github/release.yml (unchanged) continues to map labels to categorized
release-notes sections.
- dependabot.yml added for the github-actions ecosystem so pinned SHAs
auto-refresh on a weekly cadence.
Docs
- CONTRIBUTING.md documents the concurrency convention, the
conventional-commit PR-title rules, and the reusable-workflow exception.
Follow-up to verify before relying on the labeler in anger
- gh api repos/amannn/action-semantic-pull-request/git/refs/tags/v5.5.3
- gh api repos/release-drafter/release-drafter/git/refs/tags/v6.0.0
- Confirm release-drafter reads its config from the base ref (not fork
head) when invoked via pull_request_target.
* ci: address PR review feedback on concurrency and labeler workflows
Two blocking fixes
- pr-labeler.yml: separate concurrency slots for pull_request and
pull_request_target. Previously both triggers shared a single group
with cancel-in-progress: true, so the privileged autolabel run could
cancel the title-validation check mid-run and leave a required status
in a permanent cancelled state.
- pr-labeler.yml autolabel job: add contents: read. release-drafter's
context.config() reads .github/release-drafter.yml from the default
branch via the repo-contents API and 403s without the scope. Job-level
permissions nullify all unlisted scopes so an explicit grant is needed.
Two non-blocking improvements
- Replace the hardcoded reusable-workflow allowlist in ci-quality.yml
with dynamic on:-block parsing. New workflow_call-only workflows no
longer produce false-positive convention failures.
- Implement actual group-key validation. The check now also asserts that
every concurrency.group expression references either ${{ github.workflow }}
or the literal CI- prefix (the documented ci.yml exception).
- Script extracted to .github/scripts/check-workflow-concurrency.py so
it is runnable locally and independently testable.
* Initial plan
* fix: stale vectors preserved on content edits and vector index missing after zero-node run
Issue 1: Add contentHash to EMBEDDING_SCHEMA and embedding pipeline.
- contentHash column persisted per CodeEmbedding row
- POST /api/embed queries nodeId+contentHash, compares per-node hash
- Stale rows (hash mismatch) are DELETE'd before re-embedding
- Legacy DBs without contentHash treated as stale (full re-embed)
- loadCachedEmbeddings and run-analyze cache restore include contentHash
Issue 2: createVectorIndex called unconditionally before zero-node early return.
Regression tests:
- contentHashForNode determinism and content-change detection
- EMBEDDING_SCHEMA includes contentHash STRING column
- Pipeline exports verified
Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/1581c0c0-f359-4376-b47e-62d24a28fd2d
Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com>
* fix: use parameterized query for stale embedding DELETE, revert package-lock.json
Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/1581c0c0-f359-4376-b47e-62d24a28fd2d
Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com>
* fix: address review feedback — config consistency, narrow catches, extract DB logic
Bug #1: Use finalConfig consistently in contentHashForNode (line 224 was
using raw `config` while line 307 used `finalConfig`). Cache precomputed
hashes in filter phase to avoid double computation (Perf #5).
Bug #2: Narrow catch in loadCachedEmbeddings to only fall back on
column/table-missing errors. Rethrow transient/connection errors.
Bug #3: Log non-trivial DELETE failures instead of silently swallowing.
Arch Violation #3: Extract fetchExistingEmbeddingHashes from api.ts into
lbug-adapter.ts. Server layer now calls a single adapter function instead
of re-implementing the DB query logic with nested try-catch.
Tests: Add config consistency test, note that fetchExistingEmbeddingHashes
tests require native module (run in CI).
Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/b8c4f6b0-4095-4507-a15d-d8469793efac
Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com>
* fix: narrow Column error match to 'contentHash' in lbug-adapter fallback checks
Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/b8c4f6b0-4095-4507-a15d-d8469793efac
Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com>
* fix: address production-readiness review — eliminate competing state, use schema constants, hard-fail on stale DELETE, add incremental filter tests
Gap A / Arch Violation 1: Remove duplicate vectorExtensionLoaded flag from
embedding-pipeline.ts — delegate to lbug-adapter's loadVectorExtension()
which owns the VECTOR extension lifecycle and resets on DB reconnect.
Arch Violation 2: Replace all hardcoded 'CodeEmbedding' and
'code_embedding_idx' strings in embedding-pipeline.ts and run-analyze.ts
with EMBEDDING_TABLE_NAME, EMBEDDING_INDEX_NAME, and CREATE_VECTOR_INDEX_QUERY
imported from schema.ts. Add EMBEDDING_INDEX_NAME export to schema.ts.
Gap B: Make DELETE failure for stale vectors a hard throw (not just a
warning). Continuing after failed DELETE risks Kuzu vector-index corruption
since the constraint requires DELETE-before-INSERT for vector-indexed
properties. "not found" / "does not exist" errors are still safe to ignore.
STALE_HASH_SENTINEL: Define a named constant in embedding types.ts for the
empty-string sentinel convention. Used consistently in lbug-adapter.ts and
run-analyze.ts so the invariant is self-documenting.
Tests: Add comprehensive unit tests for the incremental filter logic with
mocked embedder:
- New node → embedded
- Unchanged node (hash matches) → skipped
- Stale node (hash mismatch) → DELETE + re-embed
- STALE_HASH_SENTINEL → treated as stale
- Zero nodes after filter → createVectorIndex still called
- DELETE failure with non-trivial error → throws
Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/b21edee7-c9c5-4742-947b-d0def4fb26aa
Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com>
* fix: tighten error classification — extract isMissingColumnOrTableError helper, remove broad pattern matching
- Extract isMissingColumnOrTableError() helper in lbug-adapter for
consistent schema-error detection (replaces duplicate inline checks)
- Tighten 'contentHash' match: now requires 'property' AND 'contentHash'
(Kuzu-specific pattern) instead of broad 'contentHash' substring
- Tighten DELETE error check: only ignore 'does not exist' (Kuzu's actual
message), not broad 'not found' which could mask connection errors
- Fix test node ID/name/filePath consistency
Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/b21edee7-c9c5-4742-947b-d0def4fb26aa
Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com>
* fix: CI failures and final review — move STALE_HASH_SENTINEL to schema, tighten error matching, fix test mocking, format
- Move STALE_HASH_SENTINEL from embeddings/types.ts to lbug/schema.ts
(fixes inverted layer dependency: lbug should not import from embeddings)
- Tighten isMissingColumnOrTableError: replace broad msg.includes('not found')
with /(table|column|property).*not found/i regex to avoid matching transient errors
- Add vi.resetModules() in test beforeEach for explicit module isolation
(fixes vi.doMock not intercepting loadVectorExtension in CI)
- Skip precomputedHashes.set() on unchanged (return false) path
- Run prettier on all 5 files flagged by CI format check
Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/e20311fd-4361-47b4-a137-9adc3e533b35
* fix: address remaining review nits — rename precomputedHashes, generalize error matcher, revert package-lock
- Rename precomputedHashes → computedStaleHashes (hashes are computed
on-demand during filter, only cached for stale nodes being re-embedded)
- Remove contentHash-specific clause from isMissingColumnOrTableError —
the regex /(table|column|property).*not found/i already covers it
- Revert package-lock.json ssh→https protocol change
Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/e20311fd-4361-47b4-a137-9adc3e533b35
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com>
* fix(group/sync): wire ManifestExtractor into syncGroup pipeline
ManifestExtractor was fully implemented in extractors/manifest-extractor.ts
but never imported or called in sync.ts. As a result, any links declared in
group.yaml were parsed and validated by config-parser.ts but silently dropped
— config.links was always an empty dead-end as far as syncGroup was concerned.
Changes:
- Import ManifestExtractor in sync.ts
- Call extractFromManifest(config.links, dbExecutors) inside the outer try
block, after all repos are processed but before the finally closes the DB
pools (symbol resolution via resolveSymbol requires open executors)
- Collect the resulting contracts into autoContracts and the cross-links into
a separate manifestCrossLinks array
- Merge manifestCrossLinks into the final crossLinks alongside runExactMatch
results
Without this fix, users who declare explicit service dependencies in
group.yaml links (the documented workaround for HTTP clients that use absolute
URLs and are invisible to the auto-extractors) get 0 cross-links regardless of
what they configure.
* test(group/sync): cover manifest links producing cross-links
Add a unit test that asserts config.links entries produce contract pairs
and a manifest cross-link (matchType: 'manifest') via syncGroup.
Also refactors the manifest extraction call to sit outside the else/try
block so it runs regardless of extractorOverride arity — makes the code
testable without mocked DB pools and ensures links work when callers supply
a zero-arity override (e.g. in tests or programmatic usage).
* style: prettier format sync.ts and sync.test.ts
Also removes the stray empty line in the finally block (noted in review).
* fix(group/sync): dedupe cross-links and warn on dangling manifest repos
Addresses review feedback on PR #827:
1. Dedupe cross-links. Manifest contracts participate in runExactMatch, so a
manifest-declared link also emitted a duplicate matchType:'exact' CrossLink
for the same endpoint pair. Dedupe by (from, to, type, contractId) and
prefer manifest (operator-declared intent).
2. Warn on dangling repos. When a manifest link references a repo not in
config.repos, log a warning. Synthetic UIDs keep the cross-link
deterministic, but the operator probably meant something else.
3. Tests:
- Assert no duplicate 'exact' CrossLink is emitted alongside the manifest one.
- Assert synthetic UID format when no DB executors are available.
- New test: dangling manifest repo still produces a cross-link + logs a warning.
* perf(group/manifest): parallelize and memoize symbol resolution
Previous implementation ran 2N sequential Cypher round-trips per
manifest (one for provider side, one for consumer, awaited in-order
per link). For manifests with tens of links this dominated syncGroup
latency in groups with many declared cross-repo contracts.
Changes:
- Resolve provider + consumer in parallel per link (Promise.all).
- Resolve all links in parallel (outer Promise.all over links.map).
Each repo's executor pool is independent, so cross-repo fan-out
scales with the number of distinct repos in the manifest.
- Memoize by (repo, type, contract). Manifests frequently declare
the same contract from both directions or across sibling groups,
so duplicate triples now hit the DB once instead of 2× per link.
Correctness:
- resolveSymbol is a pure LIMIT 1 read, so caching + concurrent
invocation is safe.
- Iteration order over links is preserved in the final
contracts / crossLinks arrays — result shape is identical.
Test:
- New test asserts that two links sharing (repo, type, contract)
produce exactly one DB call per distinct repo-tuple.
---------
Co-authored-by: jonasvanderhaegen-xve <>
Co-authored-by: Gergo Magyar <gergomagyar@icloud.com>
* fix(lbug): wait for read stream close in splitRelCsvByLabelPair (Windows ENOTEMPTY)
The windows-latest CI job intermittently failed:
FAIL test/unit/rel-csv-split.test.ts > splitRelCsvByLabelPair > handles empty CSV (header only) without errors
Error: ENOTEMPTY: directory not empty, rmdir 'C:\Users\RUNNER~1\AppData\Local\Temp\rel-csv-test-XW5KOu'
Cause: splitRelCsvByLabelPair resolved its Promise on readline's 'close'
event, but the underlying fs.ReadStream's file descriptor is released
asynchronously after that — especially on Windows. For the empty-CSV
test the function returns so quickly that afterEach fires rmSync while
the relations.csv fd is still held, so Windows reports ENOTEMPTY on
the directory.
Fixes:
- Production: after readline 'close', wait for inputStream 'close' (or
resolve immediately if already closed/destroyed). Call inputStream
.destroy() defensively so we never hang if the fd never emits 'close'.
- Test: afterEach now retries rmSync up to 5 times on ENOTEMPTY/EBUSY/
EPERM with a brief back-off — defense-in-depth so the test doesn't
flake on slow CI runners independent of the production change.
The production fix benefits every caller, not just the test: any code
that deletes the CSV's parent directory right after the Promise
resolves previously hit the same race on Windows.
* refactor(lbug): replace custom stream state machines with stdlib primitives
Full audit of splitRelCsvByLabelPair's stream usage after the original
ENOTEMPTY fix. Replaced three hand-rolled mechanisms with their
standard-library equivalents — 147 -> 71 lines in the function, and
the caller's WriteStream closure dropped from 13 lines to 5.
- readline: 'on(line)' + pause/resume/waitingForDrain state machine
-> 'for await (const line of rl)'. Async-iterator delivery naturally
serializes line processing with our awaits, so at most one ws is in
backpressure at a time. We just 'await once(ws, "drain")' when
'write()' returns false — the custom Set, the settled flag and the
'only resume when all streams have drained' logic all go away.
- Multi-stream error coordination: hand-rolled cleanup() that had to
be entered exactly once and had to destroy the inputStream and every
pair ws -> single AbortController shared across every 'once(ws,
'drain', { signal })'. Any stream error aborts every pending wait.
- 'stream/promises.finished(inputStream)' in the 'finally' block
replaces the manual 'rl.on('close', () => inputStream.once('close',
...))' dance, and covers both the success and error paths with the
same primitive. This closes the Windows ENOTEMPTY race root cause —
we never return while the fd might still be in flight.
- Caller closure: 'new Promise((res, rej) => ws.end(cb) + remove
listener on error)' -> 'ws.end(); await finished(ws)'.
- Test 'afterEach': custom retry loop -> 'fs.rmSync(..., { maxRetries:
5, retryDelay: 50 })' (Node added these options specifically for
cross-platform tmpdir cleanup).
- Test 'destroys all streams when one errors': old code leaked
backpressure and created multiple pair streams before the first
blocked; new strict serial backpressure doesn't, so the test now
unblocks the first stream once to advance the loop and create the
second stream before triggering the error.
* fix(csv-generator): deduplicate all node types, not just File nodes
The pipeline can produce duplicate node IDs across all symbol types
(Class, Method, Function, etc.). Only File nodes were guarded by a
seenFileIds Set, leaving every other type unprotected. When the CSV
was COPY'd into LadybugDB, duplicate PKs caused mass "Batch execution
error: Found duplicated primary key value" warnings on gitnexus serve.
Replace the per-type seenFileIds with a single seenNodeIds Set checked
at the top of the iteration loop, before the switch, so every label is
covered by the same O(1) deduplication guard.
Fixes: #822
* fix(embeddings): use MERGE instead of CREATE for CodeEmbedding inserts
CREATE fails with duplicate PK when a CodeEmbedding node already exists,
which happens when:
- A PostToolUse hook triggers a concurrent gitnexus analyze during an
active analyze run (git commits fire the hook)
- A partial prior run left some embeddings in the DB before a crash
Switching to MERGE makes the insert idempotent: existing embeddings are
updated in place, new ones are created, no PK violations.
Fixes: #822
* fix(server): skip already-embedded nodes in POST /api/embed to avoid vector-index SET error
Kuzu/LadybugDB forbids SET on a property that is part of a vector index.
The /api/embed endpoint was calling runEmbeddingPipeline without skipNodeIds,
causing it to attempt MERGE+SET on every node including those already embedded.
Fix: query existing CodeEmbedding nodeIds before running the pipeline and pass
them as skipNodeIds so only new (unembedded) nodes are processed.
* fix(server): narrow catch to table-not-exist errors only in POST /api/embed
Bare catch{} would silently swallow connection errors and proceed to
re-embed all nodes, hiding infrastructure issues. Now only swallows
errors where the CodeEmbedding table does not yet exist.
* style: prettier format gitnexus/src/server/api.ts
* fix(server): log skip-embedding count and table-not-found swallow path
Addresses review feedback on PR #823:
- Log count of already-embedded nodes when skipNodeIds is populated
(aids debugging if Kuzu driver row shape changes).
- Log when the 'table does not exist' swallow path fires so ops can
catch it if Kuzu ever changes error wording.
- Document the {} config positional argument with an inline comment
referencing the runEmbeddingPipeline signature.
---------
Co-authored-by: jonasvanderhaegen-xve <>
Co-authored-by: Gergo Magyar <gergomagyar@icloud.com>
* feat(ci): add release-candidate publish pipeline
Auto-publishes gitnexus@rc on every merge to main. Version scheme is
canonical semver X.Y.Z-rc.N where the base is the current npm 'latest'
bumped by the 'bump' input (default patch) and N auto-increments by
querying existing rc versions on the registry. First rc for a new base
is rc.1; the counter resets naturally when the base advances after a
stable release.
- Reuses ci.yml via workflow_call so tests must pass before publish
- SHA-pinned actions, per-job permission scoping, provenance enabled
- Guard job dedupes duplicate dispatches against HEAD via v*-rc.* tags
- Docs-only pushes skipped via paths-ignore
- workflow_dispatch inputs: bump (patch/minor/major), force (override guard)
- Publishes under the 'rc' dist-tag so 'latest' is never moved
- Tags commits as v<rc-version> and creates GitHub prereleases
* fix(ci): address release-candidate review feedback
- Sort rc tags by creatordate (handles out-of-order pushes correctly)
- Fail fast on npm registry errors; only fall back to package.json on E404
- Drop unused pull-requests: write permission on the reused CI job
- Add secrets: inherit so any future CI secrets are available to sub-jobs
- Remove unused reltag step output
* fix(ci): address Copilot review comments
- Correct concurrency comment (runs serialize on same ref, not overlap)
- Apply E404-only fallback to 'npm view versions' query, matching the
pattern used for the 'npm view version' query
- README: clarify that docs-only merges don't trigger rc publish
- CONTRIBUTING: drop 'from main' claim for publish.yml; the tag-push
trigger does not enforce branch reachability
* fix(ci): address adversarial review — idempotency, cycle continuity, tag integrity
Codex adversarial review flagged three release-safety issues in the rc
pipeline. Fixes:
1. Cycle continuity (H). Non-patch rc trains no longer collapse back to
patch on the next push. 'bump' input accepts a new 'auto' value
(default) that infers the active rc base from the registry: if any
X.Y.Z-rc.* exists with X.Y.Z > latest, continue that base; otherwise
patch-bump. Explicit patch/minor/major still forces a cycle reset and
now also bypasses the dedup guard so an explicit dispatch on a
tagged HEAD is honored.
2. Idempotency across post-publish failures (H). The guard marker
('rc/<HEAD_SHA>' lightweight tag) and the release tag ('v<RC>'
annotated) are now pushed atomically *before* 'npm publish'. A
publish failure leaves the marker in place and the guard refuses to
re-publish. Added a defensive 'npm view <pkg>@<rc> version' check
before publish to catch registry-level races. Recovery path
documented in CONTRIBUTING.md.
3. Tag ↔ package integrity (M). 'v<RC>' now points at a detached
release commit whose tree contains the rewritten package.json, so
the tag's source archive matches the npm tarball exactly. 'main'
stays pristine; the release commit is reachable only via the tag.
* fix(ci): surface registry errors on defensive version check; drop actions: read
- npm view <pkg>@<rc> version now distinguishes E404 (safe) from network
failures (abort) via the same mktemp+grep pattern used for the other
two npm view calls
- Dropped actions: read on the ci workflow_call — no sub-workflow uses
the Actions API
* fix: add setMaxListeners(50) to relationship pair WriteStreams
Dynamically-created per-pair WriteStreams for relationship CSV splitting
default to Node.js's maxListeners limit of 10. On large repositories with
many relationship types, readline backpressure causes repeated
ws.once('drain', ...) calls that exceed this limit, flooding stderr with
MaxListenersExceededWarning messages.
This matches the existing pattern in csv-generator.ts where
BufferedCSVWriter already calls this.ws.setMaxListeners(50).
* fix: address all 3 stream bugs in relationship CSV splitting
Addresses review feedback from @magyargergo and Claude CI analysis:
Bug 1 (High): Add error handlers to per-pair WriteStreams.
Previously, if a WriteStream errored (disk full, EMFILE) while rl was
paused waiting for drain, the drain callback never fired, rl.resume()
was never called, and the outer Promise hung forever — leaking all
open file descriptors until process kill.
Now each WriteStream gets an error handler that destroys all streams,
closes the readline interface + its input ReadStream, and rejects the
Promise.
Bug 2 (Medium): Add waitingForDrain Set to prevent drain listener
accumulation. rl.pause() is not synchronous — buffered line events
continue firing after pause(), and multiple lines targeting the same
pairKey each added another ws.once('drain', ...) listener. This was the
root cause of MaxListenersExceededWarning.
Now a Set<string> tracks which streams are already waiting for drain.
Only the first backpressure event registers the listener; subsequent
lines for the same stream are silently skipped (they're already written
to the stream buffer). This eliminates listener accumulation entirely
and makes setMaxListeners(50) a safety net rather than a band-aid.
Bug 3 (Low): Close readline and destroy input ReadStream in error
handler. Previously only the WriteStreams were destroyed on error,
leaving the ReadStream FD to linger until GC.
* fix: address review feedback — remove setMaxListeners, harden cleanup
- Remove setMaxListeners(50) entirely. The waitingForDrain guard
guarantees at most 1 drain listener per stream at any time. Tested
with 200 pairs x 500 lines (100k total) — max listeners was always 1,
zero warnings. No hard-coded limit needed.
- Wrap destroy() calls in cleanup() with try/catch so already-destroyed
streams don't throw synchronously (addresses @xkonjin review point 1).
- Add ws.once('error', reject) to the ws.end() phase so flush errors
during stream close properly reject instead of hanging Promise.all
(addresses Claude CI Bug 3b finding).
* test: add 8 regression tests for relationship CSV stream fixes
Covers all bugs fixed in this PR:
- Bug 1: WriteStream error rejects Promise and destroys all streams
- Bug 2: waitingForDrain guard keeps drain listeners at max 1 per stream
- Bug 3: cleanup() handles already-destroyed streams safely
Tests use a MockWriteStream with controllable backpressure and error
injection to verify the exact patterns in loadGraphToLbug() without
needing a real LadybugDB instance.
* style: run prettier on changed files
* fix(test): use backpressure to keep promise pending during error tests
The error tests were racing — readline finished reading the tiny CSV
and resolved the Promise before setTimeout fired the error. Now the
mock streams use blocked=true to trigger backpressure, keeping the
Promise pending so the error fires while the split is still in progress.
* fix: use named error handler in ws.end() to prevent listener leak
ws.once() wraps the callback, so removeListener with the original
function reference won't match. Switch to ws.on() with a named
onError function so removeListener correctly detaches it after
successful close.
* refactor: extract splitRelCsvByLabelPair, fix multi-stream drain
1. Extract splitRelCsvByLabelPair as an exported function with optional
wsFactory parameter for dependency injection. loadGraphToLbug now
delegates to it. Tests import and call the real function instead of
a local reimplementation.
2. Fix multi-stream drain coordination: rl.resume() is now guarded by
waitingForDrain.size === 0, so readline only resumes when ALL
backpressured streams have drained. Previously, any single stream
draining would resume readline while other streams were still full,
allowing unbounded buffer growth.
3. Export WriteStreamFactory type and RelCsvSplitResult interface for
test consumption.
