GitNexus/ARCHITECTURE.md
Gergő Magyar 5bf8a17cd5
feat(ingestion): add control-flow-graph layer for TS/JS (#2081) (#2099)
* feat(cfg): language-agnostic CFG construction core (#2081)

U1 of M1 (CFG layer). Plain JSON-serializable CFG data model (BasicBlockData/
CfgEdgeData/FunctionCfg — must survive the worker→main boundary + ParsedFile
store), a CfgBuilder accumulator (leaders→blocks→edges, synthetic ENTRY/EXIT,
idempotent edges), a ControlFlowContext (break/continue/switch + labeled-jump
target stacks), and a TraversalResult ({entry, dangling exits}). AST-agnostic
and unit-tested on the classic control-flow topologies (if/else, while back-edge,
mid-block return, labeled break/continue) the S2 spike validated; reachability
helper backs the R9 property test.

* feat(ingestion): U2 — TS/JS CFG visitor over tree-sitter AST (#2081)

Add the TS/JS CfgVisitor that walks a function's tree-sitter AST and drives
the U1 CfgBuilder to produce a serializable FunctionCfg. One visitor covers
both languages (shared grammar family).

Handles the classic CFG hazards explicitly (R2, R10):
- loops allocate a dedicated loop-exit block so `break` has a concrete target
  before the loop's successor is known; `continue`/back-edge close the loop
  (while, do-while, C-for with init-once + increment-as-continue-target,
  for-in, for-of)
- switch fallthrough falls out naturally: a non-breaking case yields exits we
  wire to the next case as `fallthrough`; a breaking case wires to the switch
  exit via ControlFlowContext
- try/catch/finally: normal completion AND exceptional flow both route through
  finally (post-domination); a conservative exceptional edge models that the
  protected region may raise to its handler (not just explicit `throw`)
- labeled break/continue resolve against the labeled loop's frame
- early return/throw wire to EXIT/handler and terminate their block

19 hazard tests (one per construct) + AC1 10-function fixture; all green.
No change to the committed U1 core or ControlFlowContext.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(ingestion): U3 — worker CFG build + cfgSideChannel + cache coherence (#2081)

Run the CFG visitor in the parse worker (where the AST lives), serialize the
per-function CFG onto a new ParsedFile.cfgSideChannel, and keep it coherent
across the disk-backed store and the warm/durable parse cache (R3, R4).

- gitnexus-shared parsed-file.ts: add `cfgSideChannel?: unknown` as a DISTINCT
  field from captureSideChannel (different producer/consumer/lifecycle; plain
  JSON data — blocks/edges deliberately lack the `nodeId` the store's interning
  reviver keys on, so no mis-interning).
- cfg/types.ts + visitors/typescript.ts: add CfgVisitor.isFunction so the worker
  enumerates functions (and applies the line budget) by a cheap node-type test.
- cfg/collect.ts (new): collectFunctionCfgs walks the tree, builds one CFG per
  function (nested included), applies maxFunctionLines (over-cap = skipped).
- language-provider.ts: add `cfgVisitor?: CfgVisitor<SyntaxNode>` hook;
  typescript.ts attaches it to both the TS and JS providers (shared grammar).
- parse-worker.ts: read pdg + pdgMaxFunctionLines from workerData (read once at
  init — the worker never sees PipelineOptions), gate the build, attach
  cfgSideChannel alongside captureSideChannel.
- parse-cache.ts: bump SCHEMA_BUMP 4→5 (ParsedFile shape changed) and fold the
  pdg flag into computeChunkHash so a pdg-off cached chunk is NOT reused on a
  --pdg run (the #2038-class warm-cache trap). Default path keeps its keys.
- worker-pool.ts + parse-impl.ts + pipeline.ts: thread pdg/pdgMaxFunctionLines
  PipelineOptions → WorkerPoolOptions → workerData, and into the chunk-hash key.

9 boundary tests: collect contract, JSON round-trip identity (no AST leakage),
the pdg cache-key guard, the line-cap skip, and the no-visitor gate. Full CFG
suite (U1+U2+U3) green; build clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(ingestion): U4 — emit BasicBlock + CFG within scope-resolution (#2081)

Emit persisted BasicBlock nodes + CFG edges from each ParsedFile's worker-built
cfgSideChannel, INSIDE scope-resolution's Phase-4 graph emission — the last
point where the worker-built CFGs are loaded (emitParsedFiles carries the
channel; the disk store is cleared right after the orchestrator returns). This
is the architecture the doc-review corrected to: a standalone post-`mro` phase
(the issue's literal subtask) provably reads empty data (KTD1).

- cfg/emit.ts (new): pure emitFileCfgs(graph, cfgs, maxEdgesPerFunction, onWarn).
  BasicBlock id = `BasicBlock:<filePath>:<functionStartLine>:<blockIndex>`
  (KTD3 — funcStart disambiguates blocks across functions in one file; no
  `name` column). CFG edge = CodeRelation type 'CFG' with the edge KIND
  (seq/cond-true/…) in `reason` (kinds can't be their own edge type). Per-
  function edge cap stops at the cap and warns with the dropped count — no
  silent truncation (R6/KTD6).
- run.ts: pdg-gated emit pass over emitParsedFiles after emitPostResolutionEdges
  (store still live); RunScopeResolutionInput gains pdg + pdgMaxEdgesPerFunction.
- phase.ts: thread ctx.options.pdg / pdgMaxEdgesPerFunction into the call.
- pipeline.ts: PipelineOptions.pdgMaxEdgesPerFunction.

6 tests: node/edge shape (KTD3 id, no name, type='CFG', kind in reason),
cross-function id uniqueness, AC2 reachability-from-ENTRY property, the edge
cap's no-silent-truncation contract, and empty-input no-op. Flag-off
byte-identity + full runPipelineFromRepo round-trip land in U7. Build clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(cli): U5 — `--pdg` opt-in plumbing (CLI + .gitnexusrc → both sinks) (#2081)

Expose the CFG/PDG substrate as an opt-in and thread it from CLI/.gitnexusrc to
the single source of truth (PipelineOptions.pdg), which fans out to BOTH sinks
already wired in U3/U4: the worker build gate (workerData.pdg) and the
scope-resolution emit gate. Off by default (R7).

- cli/index.ts: `--pdg` commander flag.
- cli/analyze.ts: AnalyzeOptions.pdg + pass `pdg` into runFullAnalysis options.
- cli/analyze-config.ts: KEY_SPECS `pdg` (boolean) so `.gitnexusrc { "pdg": true }`
  normalizes and a non-boolean value fails closed with GitNexusRcError.
- core/run-analyze.ts: AnalyzeOptions.pdg → runPipelineFromRepo({ pdg }).

(The internal PipelineOptions/WorkerPoolOptions/workerData fields + the
parse-cache key fold landed in U3/U4; this unit adds the user-facing surface.
The budget knobs stay at internal defaults for M1.)

Tests: analyze-config pdg normalization + non-boolean rejection; opt-in.test.ts
covers the CLI/file merge precedence and that pdg perturbs the chunk-dispatch
key. The full worker-build + main-emit round-trip is the U7 integration test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(ingestion): U7 — CFG acceptance fixtures, parity, end-to-end + docs (#2081)

Acceptance criteria for the M1 CFG layer:
- AC1: a 10-function TS fixture's CFG node/edge set matches a committed snapshot
  (cfg-snapshot.test.ts).
- AC2: every BasicBlock is reachable from its function ENTRY (property test over
  the emitted graph; the fixture has no dead code).
- AC3: hazard fixtures lock the classic-bug coverage — try/throw/finally
  post-domination + labeled break/continue resolution.
- AC4: the existing pipeline-graph-golden test stays byte-identical with --pdg
  off (verified; no UPDATE_GOLDEN), proving the opt-in adds zero default-run
  drift.
- End-to-end (pipeline-pdg.test.ts): runPipelineFromRepo({ pdg: true }) on a
  tiny repo emits BasicBlock nodes + CFG edges with both endpoints present —
  the true both-sinks proof (worker builds → store → scope-resolution emits);
  the default run emits zero.

Docs: CHANGELOG M1 entry, ARCHITECTURE "Optional CFG/PDG emission" subsection
(why emit is in-phase, not post-mro), README CFG language-support note.

Full CFG suite (U1–U7): 56 tests green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(ingestion): drop unused helper in cfg-snapshot test (#2081)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(review): apply ce-code-review autofix feedback (#2081)

Review (10 reviewers) confirmed OFF-path byte-identity (adversarial + golden)
and found defects all within the --pdg path. Fixes:

- P1 same-line BasicBlock id collision: add a start-column disambiguator to
  FunctionCfg + the id (`BasicBlock:<file>:<line>:<col>:<idx>`) so two functions
  sharing a start line no longer collide under first-writer-wins addNode.
- P1 worker crash-cascade: per-file try/catch around collectFunctionCfgs so a
  CFG-build throw cannot escape to the language-group catch and silently drop
  every remaining file in the group.
- P2 edge-cap drop now logs unconditionally (input.onWarn is validator-gated/
  silent in prod) — upholds the no-silent-truncation guarantee.
- P2 Array.isArray guard before the cfgSideChannel cast in run.ts.
- P2 maxFunctionLines default: worker applies DEFAULT_PDG_MAX_FUNCTION_LINES=2000
  when unset; caps forwarded through run-analyze AnalyzeOptions (closes the
  server-path drop).
- P3 README duplicate paragraph removed; `0`-vs-default docstrings corrected;
  CLI --pdg flag made language-neutral; reachableBlocks JSDoc corrected.
- Documented the break-through-finally + stacked-label CFG limitations.
- Tests: same-line id-collision regression, standalone throw→EXIT, dead-code-
  after-return, async/generator/method coverage, strengthened labeled-continue.

Refuted: the HTTP-500 getNodeQuery finding — M0 already shipped the BasicBlock
branch + name-floor (R12/web-safety handled).

CFG + analyze-config suites: 95 tests green; golden parity (AC4) byte-identical.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* perf(ingestion): benchmark CFG construction + O(n) block-text accumulation (#2081)

Closes the M1 review's requires_verification perf gap ("no benchmark for
collectFunctionCfgs; a wall-time + cfgSideChannel byte-size regression gate
would catch the extendBlock concatenation before kernel scale").

- bench/cfg/measure.mjs (new): build-free tsx harness timing collectFunctionCfgs
  (parse once, reuse the tree) across three scaling scenarios — straight-line
  (extendBlock path), many-functions (collect walk), branchy (block/edge growth)
  — at 500→2000. Reports a wall-time scaling ratio AND a cfgSideChannel
  byte-size ratio, plus an order-independent sha256 over the emitted blocks/edges
  as the behavior gate. `--check` compares both ratios + the fingerprint against
  bench/cfg/baselines.json; mirrors the scope-capture / python-scope harnesses.
- .github/workflows/ci-tests.yml: run the gate on every test job (build-free,
  alongside the existing scope-capture guards) so an O(n^2) re-regression fails CI.
- cfg-builder.ts: structural fix for the one real hotspot the bench surfaced —
  accumulate basic-block text as fragments joined once in finish(), instead of
  concatenating onto a growing string per coalesced statement (O(n^2) → O(n)).
  Behavior-identical (the CFG fingerprint + the AC1 snapshot are unchanged).

Measured (post-fix): time ratios straight-line ~1.3, many-functions ~1.0,
branchy ~1.1 (all sub-quadratic; a true O(n^2) would be ~4.0). cfgSideChannel
bytes scale linearly (~1.0-1.04). 60 CFG tests green; build clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* perf(ingestion): add memory + disk growth gates to the CFG benchmark (#2081)

Extend bench/cfg/measure.mjs beyond wall-time to the two other scalability
dimensions that matter at kernel scale:

- DISK growth: utf8 byte size of the serialized cfgSideChannel — exactly what a
  --pdg run writes onto every ParsedFile shard (durable store + parse cache).
- MEMORY growth: retained JS heap of the cfgSideChannel payload, measured by the
  release-delta method (heap held minus heap after dropping it) — robust to
  pre-existing garbage and dead-stable run-to-run. Needs `node --expose-gc`;
  without it the heap metric is null and its gate is skipped (local runs still
  work). ci-tests.yml now passes --expose-gc so the heap gate runs in CI.

Both gated on linear scaling in baselines.json (disk_bytes_budget / heap_budget
1.2-1.3). Measured: disk ~1.0-1.04, retained heap ~0.87-1.0 — both linear
(~1KB/function each; ~2MB heap / 1.6MB disk at 2000 functions, --pdg only).
Bumped REPS 7->15 to stabilize the noisier time signal and widened the coarse
time tripwire budgets (the disk/heap gates carry the tight regression detection).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(ingestion): address tri-review + CFG-expert findings (#2081)

Corroborated findings from the tri-review (Codex + CE personas + GitNexus swarm
+ a CFG/program-analysis domain-expert lane). The OFF-path stays byte-identical;
all fixes are within the --pdg path or the benchmark.

- [Codex+CFG-expert] Exceptional `throw` edges now wire EVERY block in a try's
  protected region to the handler, not just the body ENTRY. A branched try body
  (`try { if (x) { use(t); } } catch`) previously left interior blocks with no
  path to `catch` — a taint false-negative into the handler for the M2 PDG pass.
- [Codex+CFG-expert] An unresolved labeled jump (a stacked outer label or a
  labeled non-loop block) now routes to the function EXIT instead of leaving a
  dangling sink — restores the single-exit invariant post-dominator/PDG
  computation needs.
- [Codex] computeChunkHash now folds pdgMaxFunctionLines/pdgMaxEdgesPerFunction
  into the chunk key (not just the pdg boolean), so a warm cache built under one
  cap is never served to a run with a different cap (#2038 class, extended to
  the budgets). Adds PdgCacheKey; boolean form kept for back-compat.
- [perf] visitTry resolves catch/finally in a single namedChild pass (the double
  `namedChildren.find` allocated two throwaway arrays).
- [adversarial] The bench `straight-line` scenario now runs at 2000->8000:
  output is a constant 4 blocks so disk/heap can't see the concat path, and at
  the old N a genuine O(n²) was masked by V8 cons-strings. Verified at the new N:
  the array-join impl ~1.0, a rope-optimized `+=` ~1.0 (correctly not flagged),
  a real O(n²) (re-join-every-append) ~3.8 — budget tightened 2.0->1.5.
- [adversarial+Codex] The bench `--check` now FAILS LOUDLY when run without
  `--expose-gc` instead of silently skipping the retained-heap gate.
- Doc: re-labeled the finally-bypass as a SOUNDNESS (false-negative) limitation
  tracked for M2, not mere "precision."

3 new regression tests (branched-try interior→handler, stacked-label→EXIT,
cap-fold key). 99 CFG tests pass; build clean; bench gate green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(parse-cache): clarify that SCHEMA_BUMP still invalidates caches once (#2099 F6)

The computeChunkHash comment claimed pdg-off warm caches "survive this
change untouched" — true for the key FORMAT, but misleading as an
upgrade-behavior promise: SCHEMA_BUMP 4→5 changes PARSE_CACHE_VERSION
and both stores hard-invalidate on it. Separate the two facts so the
next cache change isn't reasoned about from a false premise.

Review finding F6 (P3) of PR #2099 tri-review.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(cfg): correct for-loop back-edge kinds when no increment clause (#2099 F5)

A for with a body but no increment emitted an unconditional
header→header 'loop-back' self-edge (a path that never executes the
body) while the real back-edge body→header was labeled 'seq'. Any
consumer identifying loops via reason='loop-back' picked the phantom
edge and excluded the body from the natural loop.

Gate the self-edge on the body being absent (the one case where the
header genuinely re-tests itself) and carry 'loop-back' on the body's
exits when they ARE the back-edge, matching visitWhile/visitForIn.

Review finding F5 (P3) of PR #2099 tri-review.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(cfg): treat an empty catch clause as a real handler (#2099 F2)

visitTry keyed handler semantics off the traversal result — null for an
empty body, since visitSeq([]) returns null — instead of the syntactic
clause. An empty `catch {}` was therefore treated as NO catch: the
swallowed exception escaped to the outer handler/EXIT, the no-catch
re-propagation misfired past finally, and code after a try whose body
always throws became unreachable from ENTRY — a hard false-negative
source for the M2 taint pass, on an extremely common pattern.

Synthesize one empty block spanning the clause (entry == sole exit)
when the catch body traverses to null, before the protected region is
walked. Exception flow lands in it and rejoins the normal continuation;
all downstream wiring (handler selection, finally routing, the !catchRes
re-propagation gate) operates on the syntactically-correct shape.

Review finding F2 (P2, reproduced) of PR #2099 tri-review.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(cfg): guard CFG emission per element, not just per outer array (#2099 F4)

The cfgSideChannel guard checked only Array.isArray before casting to
FunctionCfg[] — its own comment promised a wrong-shape value would
'skip emission, not throw a TypeError mid-graph-build', but a malformed
ELEMENT sailed through. Worse, the obvious-looking failure shape never
throws at all: emitFileCfgs string-templates any edge endpoint into the
BasicBlock id and graph inserts are no-throw, so a non-integer endpoint
silently became a dangling 'BasicBlock:…:undefined' edge that degrades
the DB rel-pair COPY to row-by-row fallback inserts much later.

Layered fix matching house precedents (parsedfile-store reviver,
worker-side per-file catch): a per-element shape+content predicate
(arrays + integer edge endpoints) that warns and skips malformed
elements while valid siblings still emit, plus a per-file try/catch
backstop for shapes that genuinely throw (e.g. a null inside blocks).

Review finding F4 (P3) of PR #2099 tri-review.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(parse-cache): drop emit-time edge cap from the pdg chunk key (#2099 F3)

pdgMaxEdgesPerFunction is applied exclusively in emitFileCfgs during
scope-resolution on the main thread — the worker never receives it
(workerData carries only pdg + pdgMaxFunctionLines), so the cached
worker output is byte-identical across cap values. Folding it into the
chunk key (added by a prior review round) only converted a free knob
into a repo-sized cost: every cap change forced a full re-parse and a
durable-store rewrite of unchanged data.

Keep pdg + maxFunctionLines (genuinely worker-visible, shape the cached
cfgSideChannel) and document the classification test in the PdgCacheKey
doc comment so the next option gets sorted deliberately: worker-shard
inputs go in this key; persisted-graph-only inputs belong in the
RepoMeta pdg stamp (F1). Chunks written under the old ns string miss
once and prune — no migration needed.

Review finding F3 (P2) of PR #2099 tri-review.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(analyze): record pdg config in RepoMeta; force full writeback on mode flip (#2099 F1)

Running --pdg against an already-indexed repo silently persisted ~zero
CFG: incremental eligibility had no pdg term, RepoMeta recorded no
mode, and extractChangedSubgraph keeps only changed-file nodes — on a
no-change --pdg re-run every freshly built BasicBlock was dropped from
the written subgraph ('Incremental: changed=0', run succeeds, zero
rows). The converse flip left zombie mixed-coverage blocks only --force
could clean. Worse, a clean-tree flip hit the alreadyUpToDate fast path
and never ran the pipeline at all.

- RepoMeta gains an additive-optional pdg stamp ({maxFunctionLines,
  maxEdgesPerFunction}, resolved values; absent ≡ pdg-off, which covers
  every legacy meta). No INCREMENTAL_SCHEMA_VERSION bump — that would
  force a one-time full rebuild for everyone. The end-of-run meta is a
  fresh literal, so omitting the field on a pdg-off run is what clears
  the stamp after an on→off flip.
- pdgModeMismatch (pure, exported) compares the resolved triple; the
  flip check sits before the fast path and always logs its notice (not
  gated on options.force — --skills implies force with no message of
  its own), naming the .gitnexusrc pdg key that pins the mode.
- The full-rebuild branch now writes the incrementalInProgress dirty
  flag (toWriteCount: 0 sentinel) before the wipe whenever a prior meta
  exists, mirroring the incremental branch. This closes the crash
  window where a rebuild dying between the bulk load and saveMeta left
  meta/DB inconsistent and the fast path certified zombie (or missing)
  CFG rows indefinitely — and incidentally closes the same pre-existing
  hole for user --force runs. Recovery log reworded accordingly.

Tests: pdg-mode-flip.test.ts (real git + LadybugDB; primary assertion
is a direct BasicBlock table count — meta.stats aggregates
nondeterministic Community/Process rows) covering off→on, steady-state
fast path, on→off zombie cleanup, cap-change rebuild, and dirty-flag +
flip composition; pure-helper tests for default resolution and the
0=unlimited carve-out.

Review finding F1 (P1) of PR #2099 tri-review.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-10 19:26:45 +01:00

28 KiB

Architecture — GitNexus

Monorepo: CLI/MCP (gitnexus/) + browser UI (gitnexus-web/).

Repository layout

Path Role
gitnexus/ npm package gitnexus: CLI, MCP server (stdio), HTTP API, ingestion pipeline, LadybugDB graph, embeddings.
gitnexus-web/ Vite + React thin client: graph explorer + AI chat. All queries via gitnexus serve HTTP API.
gitnexus-shared/ Shared TypeScript types and constants (consumed by CLI and Web).
.claude/, gitnexus-claude-plugin/, gitnexus-cursor-integration/ Agent skills and plugin metadata.
eval/ Evaluation harnesses for benchmarking tool usage.
.github/ CI workflows + composite actions (setup-gitnexus/, setup-gitnexus-web/).

End-to-end flow: index → graph → tools

  1. Ingestionanalyze.tsrunFullAnalysis (run-analyze.ts) → runPipelineFromRepo (pipeline.ts). DAG of 14 phases builds a KnowledgeGraph in memory, then loads into LadybugDB under .gitnexus/. Repo registered in ~/.gitnexus/registry.json for MCP discovery.

  2. Persistencerepo-manager.ts (paths, registry, KuzuDB cleanup). lbug-adapter.ts (graph load, queries, embedding batches).

  3. Query layer — three interfaces to the same backend:

    • MCP (stdio): mcp.tsLocalBackend → tools (tools.ts) + resources (resources.ts)
    • HTTP bridge: serve.ts → Express (api.ts, mcp-http.ts) for web UI
    • CLI direct: gitnexus query|context|impact|cypher in tool.ts
  4. Stalenessstaleness.ts compares indexed lastCommit to HEAD, surfaces hints.

MCP tools

Tool Purpose
list_repos Discover indexed repos
query Hybrid BM25 + vector search over the graph
cypher Ad hoc Cypher against the schema
context Callers, callees, processes for one symbol
impact Blast radius (upstream/downstream) with risk summary
detect_changes Map git diffs to affected symbols and processes
rename Graph-assisted multi-file rename with dry_run preview
api_impact Pre-change impact report for an API route handler
route_map API route → handler → consumer mappings
tool_map MCP/RPC tool definitions and handlers
shape_check Response shape vs consumer property access mismatches
group_list List repo groups or details for one group
group_sync Rebuild group Contract Registry (contracts.json) and bridge graph

query, context, and impact are group-aware: pass repo: "@<groupName>" (or "@<groupName>/<memberPath>" to scope to one member) plus optional service: "<monorepo/path>". Group-mode query merges per-repo results via Reciprocal Rank Fusion; group-mode impact runs the local walk in the chosen member and fans out across boundaries via the Contract Bridge (gitnexus/src/core/group/cross-impact.ts). The previously-planned group_query, group_context, group_impact, group_contracts, group_status MCP tools are intentionally not introduced — group-level state is exposed via resources instead:

Resource URI Purpose
gitnexus://group/{name}/contracts Contract Registry (provider/consumer rows + cross-links)
gitnexus://group/{name}/status Per-member index + Contract Registry staleness

Where to change what

Concern Start in
CLI commands/flags src/cli/ (index.ts, per-command modules)
Parsing/graph construction src/core/ingestion/pipeline-phases/ + pipeline.ts
Graph schema/DB src/core/lbug/ (schema.ts, lbug-adapter.ts)
MCP tools/resources src/mcp/server.ts, tools.ts, resources.ts
Cross-repo groups (sync, contracts, @<group> routing) src/core/group/ (service.ts, cross-impact.ts, sync.ts, bridge-db.ts)
Search ranking src/core/search/ (BM25, hybrid fusion)
Embeddings src/core/embeddings/ + src/core/run-analyze.ts
Wiki generation src/core/wiki/
Language support src/core/ingestion/languages/ + tree-sitter-queries.ts + gitnexus-shared/src/languages.ts
Import resolution src/core/ingestion/import-processor.ts + import-resolvers/configs/ + model/resolution-context.ts
Call resolution/inheritance/MRO src/core/ingestion/scope-resolution/ (pipeline, passes, graph-bridge)
Type extraction src/core/ingestion/type-extractors/
Worker pool src/core/ingestion/workers/
Web UI gitnexus-web/src/
CI .github/workflows/*.yml, .github/actions/

Paths above are relative to gitnexus/ unless they start with gitnexus-web/ or .github/.


Pipeline Phase DAG

14 phases defined in gitnexus/src/core/ingestion/pipeline-phases/, each with explicit deps and typed output.

scan → structure → [markdown, cobol] → parse → [routes, tools, orm]
  → crossFile → scopeResolution → pruneLocalSymbols → mro → communities → processes
Phase File Deps Output
scan scan.ts (root) File paths + sizes
structure structure.ts scan File/Folder nodes, CONTAINS edges, allPathSet
markdown markdown.ts structure Section nodes, cross-link edges from .md/.mdx
cobol cobol.ts structure COBOL program/paragraph/section nodes (regex, no tree-sitter)
parse parse.ts + parse-impl.ts structure, markdown, cobol Symbol nodes, IMPORTS/CALLS/EXTENDS edges, extracted routes/tools/ORM queries
routes routes.ts parse Route nodes + HANDLES_ROUTE edges (Next.js, Expo, PHP, decorators)
tools tools.ts parse Tool nodes + HANDLES_TOOL edges
orm orm.ts parse QUERIES edges (Prisma, Supabase)
crossFile cross-file.ts + cross-file-impl.ts parse, routes, tools, orm Cross-file type propagation in topological import order
scopeResolution scope-resolution/pipeline/phase.ts parse, crossFile, structure Binding/reference + inheritance edges; disposes BindingAccumulator
pruneLocalSymbols prune-local-symbols.ts scopeResolution Drops inert block-local Const/Variable/Static nodes (only a File→DEFINES edge) post-resolution
mro mro.ts crossFile, scopeResolution, pruneLocalSymbols, structure METHOD_OVERRIDES + METHOD_IMPLEMENTS edges
communities communities.ts mro, pruneLocalSymbols, structure Community nodes + MEMBER_OF edges (Leiden algorithm)
processes processes.ts communities, routes, tools, pruneLocalSymbols, structure Process nodes + STEP_IN_PROCESS edges

Non-phase files in the same directory: parse-impl.ts, cross-file-impl.ts (implementation), wildcard-synthesis.ts (whole-module import expansion), types.ts, runner.ts, index.ts.

DAG runner

runner.ts — static phase graph, no plugins, compile-time type safety.

  1. Validation — Kahn's topological sort. Rejects on: duplicate names, missing deps, cycles (DFS traces the concrete cycle path, e.g., A -> B -> C -> A, plus count of transitively blocked dependents).

  2. Execution — sequential in topological order. Each phase receives:

    • ctx: PipelineContext — shared mutable KnowledgeGraph, repoPath, progress callback, options
    • deps: ReadonlyMap<string, PhaseResult>declared deps only (runner filters the results map to prevent hidden coupling)
  3. Error handling — wraps phase errors with the phase name, emits terminal error progress event, swallows progress handler errors to preserve the original cause.

  4. Timing — per-phase durationMs in PhaseResult, dev-mode console logging.

Design patterns:

  • Single graph accumulator — all phases mutate the same KnowledgeGraph in ctx; the graph is the primary output.
  • Typed phase accessgetPhaseOutput<T>(deps, 'name') for type-safe upstream results.
  • Binding accumulator lifecycle — created in parse, disposed by crossFile (in finally). No other phase should take ownership.
  • Skippable phasesskipGraphPhases omits MRO/communities/processes (faster tests); pruneLocalSymbols still runs (it is graph cleanup, not analysis). skipWorkers is no longer a sequential escape hatch — it (like --workers 0 / GITNEXUS_WORKER_POOL_SIZE=0) is rejected with an actionable error, since the worker pool is the sole parse path (§ Chunked parse-and-resolve).
  • Local-symbol pruningpruneLocalSymbols removes inert block-local value symbols after scope resolution has consumed them. Opt out per-call with PipelineOptions.keepLocalValueSymbols or globally with the GITNEXUS_KEEP_LOCAL_VALUE_SYMBOLS env var.

How to add a new phase

  1. Create pipeline-phases/my-phase.ts with a PipelinePhase<MyOutput> (name, deps, execute)
  2. Export from pipeline-phases/index.ts
  3. Add to buildPhaseList() in pipeline.ts
import type { PipelinePhase, PhaseResult } from './types.js';
import { getPhaseOutput } from './types.js';
import type { ParseOutput } from './parse.js';

export interface MyPhaseOutput { /* ... */ }

export const myPhase: PipelinePhase<MyPhaseOutput> = {
  name: 'myPhase',
  deps: ['parse'],
  async execute(ctx, deps) {
    const { allPaths } = getPhaseOutput<ParseOutput>(deps, 'parse');
    // ... write to ctx.graph ...
    return { /* typed output */ };
  },
};

Semantic model

SemanticModel (gitnexus/src/core/ingestion/model/semantic-model.ts) is the authoritative store for every symbol-indexed lookup (by nodeId, simpleName, qualifiedName, or filePath). The scope-resolution pipeline reads from here: findOwnedMember, pickOverload, and findExportedDefByName all consult model.methods / model.fields / model.symbols.

ParsedFile (gitnexus-shared/src/scope-resolution/parsed-file.ts) is the single per-file artifact the scope-resolution pipeline consumes. Scope-resolution passes MUST NOT build a parallel parse representation. If a per-language hook needs AST-level facts that ParsedFile doesn't expose, it should reuse the orchestrator's treeCache (RunScopeResolutionInput.treeCache) rather than re-invoking parser.parse(...) on its own — the C# populateNamespaceSiblings hook is the reference implementation of this pattern.

The scope-resolution pipeline additionally carries WorkspaceResolutionIndex for Scope-valued lookups (classScopeByDefId, moduleScopeByFile) that SemanticModel structurally cannot hold. No symbol-indexed duplicates exist outside SemanticModel.

Write / read phase contract. The model is mutable during three ordered phases and read-only afterward:

 Phase 1: parse            ──► symbolTable.add fans into types/methods/fields
 Phase 2: scope-resolution ──► reconcileOwnership() registers corrected ownerIds
 Phase 3: finalize         ──► model.attachScopeIndexes(bundle) — one-shot freeze
 ─────────────────────────── phase boundary ───────────────────────────
 Read phase: all resolution passes + MCP + HTTP + embeddings see
             SemanticModel (read-only handle); writes are type-errors.

runScopeResolution narrows MutableSemanticModelSemanticModel at the phase boundary so downstream passes physically cannot mutate the model even accidentally.

Reconciliation pass. reconcileOwnership (scope-resolution/pipeline/reconcile-ownership.ts) is a shim for languages whose parse-time extractor doesn't resolve enclosingClassId at parse time (Python class-body methods are the canonical case). It walks parsed.localDefs[i].ownerId after populateOwners and registers any missed methods/fields into the model. Idempotent — safe to re-run, safe alongside languages whose extractor already carries ownerId (C#).

The architectural end state is for every language's parse-time extractor to emit the correct ownerId directly, making reconciliation a no-op (tracked as a follow-up refactor). The dev-mode validator validateOwnershipParity surfaces any drift via onWarn under NODE_ENV !== 'production' && VALIDATE_SEMANTIC_MODEL !== '0'.

References: semantic-model.ts file-head (full write/read contract); contract/scope-resolver.ts Contract Invariant I9 (scope-resolution-side rule).


Scope-Resolution Pipeline (RFC #909 Ring 3)

Language-agnostic scope-resolution resolver. This is the resolution path for every language — it owns CALLS/ACCESSES/USES emission and inheritance edges. Adding a language is one interface implementation (ScopeResolver) plus one registration in the SCOPE_RESOLVERS map — no changes to shared code, no new pipeline phase. (RING4-1 #942 removed the legacy call-resolution DAG and the per-language MIGRATED_LANGUAGES flag, so SCOPE_RESOLVERS registration is all that's needed.)

Pipeline stages

 ParsedFile[]  (extractParsedFile per file)
    │  finalizeScopeModel (+ provider hooks)
    ▼
 ScopeResolutionIndexes
    │  resolveReferenceSites  (via MethodRegistry.lookup)
    ▼
 ReferenceIndex
    │  emitReceiverBoundCalls  ── FIRST
    │  emitFreeCallFallback    ── THEN
    │  emitReferencesViaLookup ── LAST (uses handledSites)
    │  emitImportEdges
    ▼
 KnowledgeGraph  (IMPORTS / CALLS / ACCESSES / INHERITS / USES)

Orchestrator: runScopeResolution(input, provider) in scope-resolution/pipeline/run.ts. Pipeline phase: scopeResolutionPhase in scope-resolution/pipeline/phase.ts — iterates the registered SCOPE_RESOLVERS over the worker-serialized ParsedFiles. (Per-language emitScopeCaptures hooks may reuse a cached Tree via the orchestrator's treeCache, but in worker-pool runs that cache is empty — Trees can't cross MessageChannels — so they consume the pre-extracted ParsedFile instead; § Performance notes.)

Optional CFG/PDG emission (--pdg, #2081 M1)

On a --pdg run, the parse worker builds a per-function control-flow graph from the tree-sitter AST (LanguageProvider.cfgVisitor; TypeScript/JavaScript in M1) and serializes it onto ParsedFile.cfgSideChannel as plain data. Scope-resolution then emits BasicBlock nodes + CFG edges from that side-channel inside Phase 4 of runScopeResolution, while the disk-backed ParsedFile store is still live — the only window where the worker-built CFGs are loaded (the store is cleared right after the phase returns). A standalone post-mro phase would read an empty store, so the CFG emit deliberately lives in-phase, mirroring the applyCaptureSideChannel pattern. The opt-in is off by default (graph byte-identical), folded into the parse-cache key (a pdg-off warm cache is never reused on a --pdg run), and bounded by a per-function edge cap that logs any dropped edges. Edge kind (seq/cond-true/loop-back/…) rides in the CFG relationship's reason (CFG is a single CodeRelation type, not one type per kind). See core/ingestion/cfg/.

ScopeResolver contract

Single interface a language implements to plug into the pipeline. Contract fully documented in scope-resolution/contract/scope-resolver.ts.

Hook Purpose
languageProvider Base LanguageProvider (tree-sitter query, emitScopeCaptures, import/binding interpreters, hooks)
populateOwners(parsed) Fill deferred ownerId fields on method defs (captures can't always know the owning class at parse time)
buildMro(graph, parsed, nodeLookup) Produce mroByClassDefId: Map<DefId, DefId[]> — C3, Ruby-mixin, or first-wins per language
resolveImportTarget(target, fromFile, allFiles) (rawImportPath, sourceFile) → targetFilePath (PEP-328 for Python, etc.)
mergeBindings(existing, incoming, scopeId) Shadowing / LEGB precedence
arityCompatibility Provider consumed by registry during MethodRegistry.lookup Step 2
importEdgeReason Confidence-tier string for IMPORTS edge reason field
propagatesReturnTypesAcrossImports? Opt out of cross-file return-type propagation (default on)
fieldFallbackOnMethodLookup? Statically-typed languages turn this OFF — the heuristic over-connects (default on)
unwrapCollectionAccessor? Property-style collection views (data.Values on Dictionary-like receivers) — default off
collapseMemberCallsByCallerTarget? One CALLS edge per (caller, target) instead of per-site — default off
populateNamespaceSiblings? Cross-file implicit visibility (compiler-implicit namespace sharing) — default off; ctx carries treeCache
hoistTypeBindingsToModule? Walk up to Module scope when looking up a method's return-type typeBinding — default off; enable only when bindings are stored at module level

Per-language registration

  1. Implement ScopeResolver in languages/<lang>/scope-resolver.ts.
  2. Add entry to SCOPE_RESOLVERS in scope-resolution/pipeline/registry.ts.

CI auto-discovers the set via tsx. No workflow edit required.

Code references

Module Purpose
scope-resolution/contract/scope-resolver.ts ScopeResolver interface + shared types
scope-resolution/pipeline/run.ts Generic orchestrator
scope-resolution/pipeline/phase.ts Pipeline-phase wrapper (deps: parse, structure)
scope-resolution/pipeline/registry.ts SCOPE_RESOLVERS map
scope-resolution/passes/*.ts Reference-resolution passes (receiver-bound, free-call fallback, compound-receiver, MRO, cross-file return-type propagation)
scope-resolution/graph-bridge/*.ts CLI-local translation from resolved references → KnowledgeGraph edges
scope-resolution/scope/*.ts Generic scope-chain walkers + namespace targets
scope-resolution/workspace-index.ts Build-once O(1) lookup index
languages/python/index.ts Python ScopeResolver hooks + known-limitation docs
languages/python/captures.ts emitPythonScopeCaptures (honors cross-phase Tree cache)
languages/csharp/index.ts C# ScopeResolver hooks + known-limitation docs
languages/csharp/captures.ts emitCsharpScopeCaptures (honors cross-phase Tree cache)
languages/csharp/namespace-siblings.ts Cross-file implicit-namespace visibility hook (reads treeCache)

Performance notes

  • Cross-phase Tree cache: the orchestrator's treeCache (RunScopeResolutionInput.treeCache) lets a scope-resolution per-language hook (emitScopeCaptures) reuse a tree instead of re-parsing. Workers leave it empty — Trees can't cross MessageChannels — so in normal (worker-pool) runs scope-resolution does NOT rely on it: workers serialize each file's ParsedFile (+ capture side-channel) and stream them in, so scope-resolution consumes the pre-extracted artifact rather than re-parsing on the main thread (§ Chunked parse-and-resolve). PROF_SCOPE_RESOLUTION=1 emits hit/miss counters and a worker-engaged warning.
  • Typed relationship iteration: heritage + MRO walk only the EXTENDS / IMPLEMENTS / HAS_METHOD edges via iterRelationshipsByType, not the full relationship map.
  • Workspace-resolution-index: O(1) findOwnedMember / findExportedDef / classScopeByDefId built once per run.
  • SCC-ordered cross-file return-type propagation (PR #1050): propagateImportedReturnTypes walks indexes.sccs in reverse-topological order (leaves first), so multi-hop alias chains like models.User → service.user → app.user collapse to the terminal class in a single linear pass. Within each importer, the source module's typeBindings is chain-followed BEFORE mirroring (so we mirror terminal types, not intermediate refs), and the importer's own typeBindings is chain-followed AFTER mirroring (so local const x = importedFn() resolves before downstream importers run). Cyclic SCCs reach a partial fixpoint within a single pass without iterating to convergence — see the ts-circular cross-file-binding fixture which only asserts pipeline-no-throw. PROF output (PROF_SCOPE_RESOLUTION=1) splits finalize from propagate so quadratic regressions in the chain-follow surface independently.

Language-agnostic graph feeding

16 languages → single unified graph. Four abstraction layers:

 Unified Graph Schema (44 node types, 21 relationship types)
           ↑
 Scope-Resolution Pipeline (registry lookup + 3-tier import resolution + MRO)
           ↑
 Language Providers (import semantics, type config, export checker, MRO strategy)
           ↑
 Tree-Sitter Queries (per-language S-expressions, unified capture tags)

Language providers

Each language implements LanguageProvider (language-provider.ts). Key fields:

Field Purpose
id, extensions Language identity and file matching
treeSitterQueries S-expression queries for AST extraction
importSemantics named / wildcard-leaf / wildcard-transitive / namespace
importResolver Language-specific path → file resolution
exportChecker Public/exported symbol detection
typeConfig Type annotation extraction rules
mroStrategy first-wins / c3 / none

16 providers in languages/index.ts via satisfies Record<SupportedLanguages, LanguageProvider> — missing a language is a compile error.

Unified capture tags

Per-language tree-sitter queries use different AST node names but produce the same semantic capture tags: @definition.class, @definition.function, @call.name, @import.source, @reference.inherits. Downstream extraction needs no language branching. Defined in tree-sitter-queries.ts.

Import resolution

Per-language import resolution uses the configs + factory pattern (like call/method/class extractors). Each language declares an ImportResolutionConfig in import-resolvers/configs/, listing an ordered chain of ImportResolverStrategy functions. createImportResolver() (in resolver-factory.ts) composes them: first non-null result wins. Low-level helpers shared across strategies live alongside the configs in import-resolvers/ (e.g. go.ts, rust.ts, python.ts).

Unified 3-tier algorithm (model/resolution-context.ts), per-language importSemantics controls which tier activates:

Tier Confidence Mechanism
1 — same-file 0.95 Symbol table for caller's file
2 — import-scoped 0.9 NamedImportMap chains (named) or all files in importMap (wildcard)
3 — global 0.5 O(1) index lookups: class, impl, callable. Fallback only
Import strategy Languages Behavior
named TS, JS, Java, C#, Rust, PHP, Kotlin Only explicitly imported names visible
wildcard-leaf Go, Ruby, Swift, Dart Whole-package import, no transitive re-exports
wildcard-transitive C, C++ #include closure chains through re-exports
namespace Python Module aliases resolved at call site

Chunked parse-and-resolve

parse processes files in ~20 MB byte-budget chunks to bound memory. Per chunk:

  1. Worker pool dispatches files (the sole parse path — there is no sequential fallback; skipWorkers, --workers 0, and GITNEXUS_WORKER_POOL_SIZE=0 are rejected with an actionable error)
  2. Each worker: detect language → load grammar → run queries → return unified ParseWorkerResult
  3. Synthesize wildcard bindings (wildcard-synthesis.ts)
  4. Resolve imports
  5. Collect BindingAccumulator entries for cross-file propagation

Inheritance edges are emitted later, by the scope-resolution phase (preEmitInheritanceEdges + emitHeritageEdges), not during parse.

Workers: workers/worker-pool.ts, workers/parse-worker.ts.

Worker-serialized ParsedFiles (#2038). To index very large repos (e.g. the Linux kernel) without OOM, the worker pool is the sole parse path and workers serialize each file's ParsedFile (plus its capture side-channel) in parallel, streaming them to scope-resolution through a disk-backed store. Scope-resolution consumes the pre-extracted artifact instead of re-parsing every file on the main thread — tree-sitter's native input buffers are not GC-reclaimable, so the former main-thread re-parse leaked native memory until the process died. Pool creation is lazy / cache-miss-gated, so a warm all-cache-hit run replays cached worker output without spawning a worker (hence usedWorkerPool can be false even when the repo has parseable files).

Inheritance and MRO

Inheritance is captured by the @reference.inherits tag and emitted by the scope-resolution phase: preEmitInheritanceEdges resolves each base in scope, then emitHeritageEdges writes the EXTENDS/IMPLEMENTS edges. The phase then computes method resolution order via each ScopeResolver's buildMro hook, feeding a MethodDispatchIndex used for owner-scoped lookups. Per-language strategy:

  • first-wins — Java, C#, C++, TS, Ruby, Go
  • c3 — Python (C3 linearization)
  • ruby-mixin — Ruby (mixin-aware linearization)
  • none — single-inheritance languages

Full analysis flow

runFullAnalysis in run-analyze.ts orchestrates everything around the pipeline:

CLI (analyze.ts) → runFullAnalysis(repoPath, options, callbacks)
  1. Early exit if lastCommit == HEAD (unless --force)     [0%]
  2. Cache existing embeddings from prior index             [0%]
  3. runPipelineFromRepo() → KnowledgeGraph                [0-60%]
  4. Clean up legacy KuzuDB files                          [60%]
  5. initLbug() → loadGraphToLbug() via CSV streaming      [60-85%]
  6. Create FTS indexes (File, Function, Class, Method...) [85-90%]
  7. Restore cached embeddings (batch insert)              [88%]
  8. Generate new embeddings if --embeddings               [90-98%]
  9. Save metadata + register repo + update .gitignore     [98-100%]
 10. Generate AI context files (AGENTS.md, CLAUDE.md)      [100%]

Options: --force (rebuild regardless), --embeddings (opt-in, skipped if >50k nodes), --skipGit, --noStats.

Storage

<repo>/.gitnexus/
  ├── lbug           # LadybugDB database
  ├── lbug.wal       # Write-ahead log
  ├── lbug.lock      # Single-writer lock
  └── meta.json      # lastCommit, indexedAt, stats

~/.gitnexus/
  └── registry.json  # Global repo registry (MCP discovery)

Managed by repo-manager.ts.

LadybugDB schema

Defined in lbug/schema.ts. Separate node tables per type, single CodeRelation table.

Node tables: File, Folder, Function, Class, Interface, Method, Constructor, CodeElement, Struct, Enum, Macro, Typedef, Union, Namespace, Trait, Impl, TypeAlias, Const, Static, Property, Record, Delegate, Annotation, Template, Module, Community, Process, Route, Tool, Section, Embedding.

Relation types (CodeRelation.type): CONTAINS, DEFINES, CALLS, IMPORTS, EXTENDS, IMPLEMENTS, HAS_METHOD, HAS_PROPERTY, ACCESSES, METHOD_OVERRIDES, METHOD_IMPLEMENTS, MEMBER_OF, STEP_IN_PROCESS, HANDLES_ROUTE, FETCHES, HANDLES_TOOL, ENTRY_POINT_OF.

Embeddings (src/core/embeddings/): Snowflake arctic-embed-xs (384D). Embeddable: File, Function, Class, Method, Interface. Incremental via SHA1 content hash. Separate Embedding table.

Search (src/core/search/): Hybrid BM25 + semantic vector, merged via Reciprocal Rank Fusion (K=60).

Known limitations

Overloaded method resolution

Node IDs use arity suffix (#<paramCount>): Method:file:Class.method#1 vs #2.

Same-arity disambiguation: type-hash suffix ~type1,type2 when collision detected and type annotations present. Languages without types (Python, Ruby, JS) use arity-only. TS/JS overload signatures excluded (collapse to implementation body). See #651.

C++ const-qualified: $const suffix after type-hash when non-const collision exists: Method:file:Container.begin#0$const.

Generic/template types: type-hash uses rawType (full AST text including generics): ~vector<int> vs ~vector<std::string>.

ID stability: collision-only tags mean IDs change when overloads are added. save#1 becomes save#1~int when save(String) is added.

Variadic matching: confidence 0.7 when one side is variadic and the other has fixed count.

METHOD_IMPLEMENTS confidence tiering:

Match quality Confidence
Exact parameter types match 1.0
Arity match, types unavailable 1.0
Variadic vs fixed 0.7
Insufficient info 0.7
  • MIGRATION.md — breaking changes and migration guidance
  • RUNBOOK.md — operational commands and recovery
  • GUARDRAILS.md — safety boundaries for humans and agents
  • TESTING.md — how to run tests
  • AGENTS.md / CLAUDE.md — agent workflows and tool usage