mirror of
https://github.com/abhigyanpatwari/GitNexus.git
synced 2026-10-07 02:58:02 +00:00
Some checks are pending
CodeQL / Analyze (python) (push) Waiting to run
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Scorecard / Scorecard analysis (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
* docs(plans): add provider-hook value-refs plan (#2437) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(plans): deepen #2437 plan to USES + property-dispatch design Design revised after prior-art research (Kythe ref vs ref/call, Joern METHOD_REF, Feldthaus field-based call graphs, CodeQL impliedReceiverStep): registration sites emit reference-class USES, invocation is recovered by a field-based property-dispatch pass synthesizing CALLS at member-call sites. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(scope-resolution): model provider-hook value references (#2437) Functions referenced as object-literal property values (provider hooks like emitScopeCaptures: emitCppScopeCaptures) previously produced no edge at all, so impact/context reported a false-safe 0 upstream dependents. Two coordinated halves, per prior art (Kythe ref vs ref/call, Joern METHOD_REF, Feldthaus ICSE'13 field-based call graphs, CodeQL impliedReceiverStep): - Registration -> USES: new ReferenceKind 'value-ref'; TS/JS queries capture pair values and shorthand properties (with @reference.property-key); emitted as a reference-class USES edge, reason 'scope-resolution: value-ref'. Resolution is callable-gated so plain values emit nothing. - Dispatch -> CALLS: new shared pass emitPropertyDispatchCalls synthesizes CALLS (reason 'property-dispatch', confidence 0.7, per-key fan-out cap 32 calibrated on this repo's 16-provider hook tables) from member-call sites to every function registered under the same property key. Deviation from plan: the pass owns value-ref resolution entirely via the post-finalize findCallableBindingInScope walker — the shared registries only see pre-finalize local bindings, so imported hooks (the c-cpp.ts case) were unresolvable through lookupForSite; Reference.propertyKey passthrough dropped as unnecessary. SCHEMA_BUMP 13 -> 14: ParsedFile gains value-ref sites + propertyKey. Verified end-to-end: impact(emitCppScopeCaptures, upstream) now reports 8 impacted / HIGH with extractParsedFile (true dispatch caller) at d=1 via property-dispatch and the c-cpp.ts registration via USES. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(scope-resolution): cover value-ref registration and property dispatch (#2437) Integration: same-file/cross-file/aliased/shorthand registrations emit USES; non-callable and destructuring values emit nothing; dispatch sites gain property-dispatch CALLS (incl. JS twins and per-language partitioning); fan-out-capped keys are dropped entirely; factory-call values unchanged. Unit: capture-shape pins for @reference.value-ref + @reference.property-key. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(scope-resolution): surface dropped property-dispatch keys in stats (#2437) Review finding: skippedKeys was returned but discarded — a hook table larger than the fan-out cap silently reopened the #2437 gap for those keys. Log dropped keys and fold value-ref USES + dispatch CALLS into referenceEdgesEmitted. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(plans): add callable reference-flow implementation plan * fix(scope-resolution): close property-dispatch review gaps * feat(scope-resolution): add callable flow facts * feat(scope-resolution): resolve callable value flow * feat(scope-resolution): resolve callable references across providers * fix: harden callable reference flow resolution * fix(scope-resolution): preserve callable binding semantics * docs(plans): add pr-2522-review-fixes plan Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(storage): bump INCREMENTAL_SCHEMA_VERSION for callable-value-flow edges Callable-value-flow CALLS/USES edges (#2437) can connect two files whose content did not change, but the incremental write set only covers changed files — a top-up against a pre-v7 index would silently omit the new edges for every unchanged file pair, indefinitely. Force the one-time full re-analyze (review finding 1, #2522). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(storage): sanitize callable-flow sites per-site at load, log drops The load-time validator rejected the WHOLE ParsedFile when one site was malformed or over-bound, with no logging — and C++ legitimately emits empty-string parameterTypes entries ('' = unknown, the ReferenceSite.argumentTypes convention) for cv-only/ERROR-recovered types, so real repos fell into a permanent, silent warm-cache-miss reparse loop through the #1983-sensitive main-thread path (review finding 7, #2522). Now: '' entries are valid in type arrays; a malformed/over-bound site drops only itself (counted, warned once per load); only non-array garbage — evidence the serialization itself is untrustworthy — rejects the file. Deviation from plan §6 wording: validator-side tolerance replaces emit-side clamps — smaller diff, same asymmetry closed at the single chokepoint. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(scope-resolution): keep declarations in the union for reassigned callable cells The binding-lookup suppression for fact-constrained cells was wholesale: reassigning a declared function through its own name (greet = other; greet()) deferred the call to the solver, which then refused the lexical lookup that resolves the declaration — an unresolvable RHS yielded zero CALLS for a call that resolved pre-flow (review finding 8, #2522). Suppression now applies only to cells bound by FORMAL facts — its actual purpose (a parameter whose grammar emits no declaration binding must not adopt a same-named outer function). Copy/alias/store/load destinations keep their declaration as an inclusion seed (Andersen-style union). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(scope-resolution): count forfeited deferred sites in the budget-bailout warning On work-budget exhaustion the deferred invoke sites end the run with zero CALLS — free-call fallback and reference emission already skipped them — but the warning said 'ordinary graph emission remains untouched', which is false for exactly those sites. The warning context now carries the unresolved deferred-site count and the comment states the real cost (review finding: budget-bailout honesty, #2522). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(scope-resolution): surface dropped property-dispatch keys in stats and warn payload The over-cap warning carried only a count; the dropped key NAMES were discarded and RunScopeResolutionStats had no field, so the PR-body claim 'includes them in resolver statistics' was unimplemented (review finding, #2522; reviewer ask on the fan-out cap). The warn payload now names up to 20 dropped keys and the stats carry propertyDispatchSkippedKeys. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(scope-resolution): drop producer-less ownerQualifiedName from formal sites No capture emitter anywhere produces @callable-flow.owner-qualified-name — the solver branch consuming it was unreachable in production, yet the field was typed, parsed, validated, and unit-tested with hand-built input (review finding 16, #2522; YAGNI). Re-add with a real producer if C++ qualified member declarators ever need it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(scope-resolution): drop dead callable-flow knobs CallableFlowPassingMode 'callable-object' had no producer and no consumer distinguishing it, and CallableFlowCaptureOptions.extractCallArguments had no language providing it (unlike its live sibling extractCallCallee) — review finding 17, #2522 (YAGNI). The invocation-kind 'callable-object' is a different, live concept and stays. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ingestion): bind subscripted callable cells to the container, not the index terminalIdentifier iterates children in reverse, so tbl[i] = handler seeded the INDEX variable's cell (polluting a same-named formal) and tbl[i](7) looked up the callee under i in a different scope — no join, no CALLS edge for the classic function-pointer-array dispatch (review finding 12, #2522). Subscript nodes now recurse into their container field only, in both bindingIdentifier and terminalIdentifier, across the fielded grammars (C/C++/JS/TS/Python/Go/Java). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ingestion): make cross-function file-scope callable bindings resolvable Two stacked gaps killed the canonical C callback-registration pattern (fp assigned in init(), called in run()) — the exact #2437 false-safe this PR exists to fix (review finding H1, #2522): 1. isVisibleValueBinding only consulted assignment regions and formals, so a call in a function OTHER than the assigning one emitted no invoke fact. A declared callable-typed binding is now a value binding wherever its declaration is visible (visibleCallableSignature). 2. The C scope query had no @declaration.variable pattern for function- pointer declarators — void (*fp)(int); created no scope-tree binding, so the seed (init) and invoke (run) cells canonicalized to different keys and never joined. Both bare and initialized forms now bind. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(c): detect variadic parameters via the named variadic_parameter node tree-sitter-c materializes '...' as a named variadic_parameter node; the anonymous-token checks never matched, so variadic function-pointer signatures were emitted with a wrong fixed arity and no '...' sentinel (review finding, #2522). C++ is unaffected ('...' stays an anonymous token there); the token checks remain for such grammars. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ingestion): emit invoke facts for field-stored callable member calls The C ops-vtable pattern (o->run = handler; o->run(1)) captured the store but never the call — the member path in emitCallFacts bailed for languages without protocol methods, and the value-binding index recorded the member store under the OBJECT's name ('o'), not the member's ('run') (review finding 11/M3, #2522). Member destinations now also record their terminal member name, and a member call whose name-cell has a visible store emits an indirect invoke — gated on the store so plain accessor calls (map.get) stay inert. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(cpp): disambiguate (obj->*ptr)() ERROR recovery by token order tree-sitter-cpp groups the recovered '->*' two ways depending on error-recovery cost (identifier lengths): [identifier, ERROR '->*m'] or [ERROR 'obj->*', identifier]. The recovery assumed the first shape, so the second silently swapped receiver/member and dropped the call site — the committed test passed only by name luck (review finding H2, #2522). The identifier's position relative to '->*' inside the ERROR now decides roles. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(cpp): class members are never file-local in hasFileLocalCallableLinkage The name-keyed file-local set is populated from every static declaration, so an in-class 'static void make();' (external linkage — in-class static means no-instance) and any member sharing a name with a static free function were over-marked, refusing legitimate cross-file declaration/definition joins (review finding 13/M2, #2522). Method and Constructor defs now bypass the name-set, per the hook's own linkage-only contract. Deviation from plan step 13: the regression is a unit-level contract pin rather than an end-to-end join test — C++ merges out-of-line member definitions onto the member node by qualified identity, so the graph shape cannot discriminate the join refusal for members. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(cpp): classify parameter passing mode from the declarator chain only A whole-subtree scan for reference_declarator inverted copy vs alias: void reg(void (*cb)(int& out)) marked the by-value pointer cb as 'reference' because of the NESTED parameter's int&, making the solver back-propagate formal targets into every caller's argument cell — alias semantics for a copy (review finding 14/M5, #2522). The chain walk never descends into nested parameter lists; a reference anywhere ON the chain (int& x, void (*&cb)(int)) still aliases. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ruby): bare identifiers are calls, not callable references Ruby parses a receiver-less zero-arg method call identically to a variable read, so 'action = process' — which CALLS process and stores its return — seeded action with the callable and minted a wrong CALLS edge from any dispatch through it, confirmed end-to-end (review finding 15/HIGH, #2522). New provider knob bareNamesAreCalls: a bare name that is not a provably local value binding and not an explicit reference form (method(:x), lambda/proc) emits no flow fact, on both the assignment and argument paths. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(go): pair multi-value := positionally instead of cross-wiring The shared field fallback took the FIRST LHS identifier and the LAST RHS identifier of Go's expression_list pair, cross-wiring 'a, b := f, g' and synthesizing a garbage comma-joined qualified name — the real relationships were silently dropped (review finding 16, #2522). extractAssignment may now return multiple pairs; Go pairs list entries positionally and emits nothing for a length mismatch (multi-return call RHS). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(java): drop get/test from callableProtocolMethods 'get' and 'test' collide with ubiquitous non-functional-interface APIs (Map/List/Optional/Future.get), so every ordinary container access emitted a spurious callable-object invoke fact — high-volume misleading graph facts with a cross-wiring risk on receiver-name reuse (review finding 17, #2522). Supplier.get/Predicate.test dispatch is deliberately traded away until the check can gate on the receiver's declared type. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(rust): pin the qualified-name no-degrade guard as a hard invariant Rust's scoped_identifier callable-reference capture over-includes unit enum variants and associated constants (Shape::Square seeds as if callable); they stay edge-free only because resolveSeedCandidates refuses to degrade an unresolved qualified name to a simple-name lookup (review finding 18, #2522). Capture-side type filtering would false-negative on tuple-variant constructors, so the guard IS the contract: documented as a hard invariant (Go's mis-shaped multi-value forms also rely on it) and pinned end-to-end. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(php): remove nonexistent optional_parameter node type tree-sitter-php has no 'optional_parameter' — defaults ride on simple_parameter — so the entry was dead weight the #1920 literal gate does not cover for capture-option Sets (review finding 19, #2522). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(cobol): detect procedure pointers on fixed-format sources Two stacked defects made the feature a no-op on classic sequence-numbered fixed format (review finding 20/H3, #2522): 1. parseDataItemClauses' USAGE alternation knew POINTER but not PROCEDURE-POINTER/FUNCTION-POINTER, so the dataItems filter was dead. 2. The raw-line fallback scanned UNCLEANED text, where the sequence number satisfied the leading digits and the LEVEL NUMBER got captured as the pointer name. It now scans preprocessed lines and requires a letter- initial name (COBOL data names must contain a letter). 161 COBOL preprocessor/copy-expander tests stay green; free-format matrix case unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(cobol): skip comment lines in SET seed/copy scans A commented-out SET (indicator-column '*'/'/' or free-format '*>') produced a live seed and a false CALLS edge from dead code (review finding 21/M1, #2522). The scan now skips indicator-column comment lines and strips inline '*>' tails before matching. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(architecture): document callable-flow-only mode and skipped-key reporting The Callable-value flow section omitted scopeResolutionEdgeMode: 'callable-flow-only' — a real emit-pipeline branch that suppresses all ordinary emission for standalone providers (review finding 22, #2522) — and predated the skipped-key names/stats surfacing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(scope-resolution): correct value-ref resolution attribution and stale pdg-gating comments The value-ref contract comment claimed MethodRegistry resolution — the mechanism is the post-finalize findCallableBindingInScope walker owned by emitPropertyDispatchCalls (resolveReferenceSites skips these sites). Three 'only under --pdg' calleeIdSink comments were falsified by the #2437 gating change (callee-id-sink.ts's header was updated; these copies were missed). Review finding 23, #2522. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(ingestion): direct unit coverage for synthesizeCallableFlowCaptures The 1,100-line shared synthesizer had no test naming it — only downstream consumers were covered (review finding 24, #2522). Pins seed/invoke/ formal/argument emission, subscript container binding, store-gated member invokes, produced-value guards, and the bareNamesAreCalls knob over a minimal options object so assertions target the synthesizer's own semantics. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(resolvers): deepen shallow-language coverage; fix Kotlin/Swift reassignment gaps it exposed Adds the COBOL SET x TO y copy-branch scenario and conditional-assignment scenarios for Kotlin, C#, Swift, and Dart (10 languages previously had one generic case each — review finding 25, #2522). The new scenarios exposed two real capture gaps, fixed here: - tree-sitter-kotlin's 'assignment' node is fieldless, so nested reassignments (chosen = ::target inside a block) produced no flow facts; Kotlin's extractAssignment now decomposes it positionally. - tree-sitter-swift fields its assignment as target:/result:, neither in the shared fallback's field lists; both added. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(infra): literal-validation gate for callable-capture option Sets The #1920 gate validates query literals and exported configs but not the module-private *_CALLABLE_CAPTURE_OPTIONS Sets consumed by the shared synthesizer — a typo'd node type silently captures nothing (PHP shipped a dead 'optional_parameter'; review finding 26, #2522). Every <key>NodeTypes Set literal is now validated against its language's grammar; name-carrying sets (callableProtocolMethods, memberPointerOperators) are deliberately outside the contract. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(storage): centralize corrupt-fixture casts into makeStoreEntry The callable-flow store tests scattered 'as unknown as' double-casts per fixture (review finding 27, #2522; standing no-as-any rule). One typed helper now owns the single controlled escape hatch for building malformed serialization-boundary payloads. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(bench): refresh capture fingerprints after review fixes python-scope: the committed baseline (8d5c3699) never matched this branch's code — CI's benchmarks arm was red on the PR head (review finding 2/HIGH, #2522); regenerated (a99e69ab), scaling 1.04 in budget. scope-capture: ruby/cpp/swift/java/kotlin drifted from the review-fix commits (bare-name suppression, passing modes + ->* recovery, assignment fields, protocol narrowing, positional assignment); all 14 languages re-verified PASS with ratios <= 1.18 against the 1.5 budget. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(docs): untrack docs/plans working documents docs/ is gitignored (local working docs); the plan files were force-added past the ignore. Untracked from the index only — they stay on disk. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(golden): regenerate captures goldens after callable-flow review fixes The per-language digest guards (csharp/go/php/python/ruby/rust/swift) locked the pre-fix capture output; the review-fix series intentionally changed it — store-gated member invokes, subscript container binding, Ruby bare-name suppression, Swift assignment fields, positional pairing. Regenerated with UPDATE_GOLDEN=1; clean verification run 59/59; all other parity/golden guards (pipeline-graph, spring-route, python parity) pass untouched at 33/33. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ingestion): prototypes are callees, not callable value cells The cross-function visibility fix indexed EVERY signature-bearing declaration as a value binding — including plain function/method prototypes (void f(int);). Every call to a declared function then became an indirect invoke, and with emitCanonicalInvokeReference (C/C++) minted a free-call reference that resolved through the registry, bypassing the precise passes' two-phase/ambiguity/subobject suppression — eight phantom CALLS edges in the cpp resolver suite on CI. Only declarations whose binding identifier sits under a pointer/ parenthesized declarator (callable-typed variables like void (*fp)(int);) create value cells now. cpp resolver suite 331/331; callable-value-flow + C/C++ suites 181/181 (the cross-function fp regression still passes); cpp fingerprint rebaselined, both bench gates PASS. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
460 lines
37 KiB
Markdown
460 lines
37 KiB
Markdown
# Architecture — GitNexus
|
||
|
||
Monorepo: **CLI/MCP** (`gitnexus/`) + **browser UI** (`gitnexus-web/`).
|
||
|
||
## Repository layout
|
||
|
||
| Path | Role |
|
||
|------|------|
|
||
| `gitnexus/` | npm package `gitnexus`: CLI, MCP server (stdio), HTTP API, ingestion pipeline, LadybugDB graph, embeddings. |
|
||
| `gitnexus-web/` | Vite + React thin client: graph explorer + AI chat. All queries via `gitnexus serve` HTTP API. |
|
||
| `gitnexus-shared/` | Shared TypeScript types and constants (consumed by CLI and Web). |
|
||
| `.claude/`, `gitnexus-claude-plugin/`, `gitnexus-cursor-integration/` | Agent skills and plugin metadata. |
|
||
| `eval/` | Evaluation harnesses for benchmarking tool usage. |
|
||
| `.github/` | CI workflows + composite actions (`setup-gitnexus/`, `setup-gitnexus-web/`). |
|
||
|
||
## End-to-end flow: index → graph → tools
|
||
|
||
1. **Ingestion** — `analyze.ts` → `runFullAnalysis` (`run-analyze.ts`) → `runPipelineFromRepo` (`pipeline.ts`). DAG of 15 phases builds a `KnowledgeGraph` in memory, then loads into LadybugDB under `.gitnexus/`. Repo registered in `~/.gitnexus/registry.json` for MCP discovery.
|
||
|
||
2. **Persistence** — `repo-manager.ts` (paths, registry, LadybugDB cleanup). `lbug-adapter.ts` (graph load, queries, embedding batches).
|
||
|
||
3. **Query layer** — three interfaces to the same backend:
|
||
- **MCP (stdio):** `mcp.ts` → `LocalBackend` → tools (`tools.ts`) + resources (`resources.ts`)
|
||
- **HTTP bridge:** `serve.ts` → Express (`api.ts`, `mcp-http.ts`) for web UI
|
||
- **CLI direct:** `gitnexus query|context|impact|cypher` in `tool.ts`
|
||
|
||
4. **Staleness** — `staleness.ts` compares indexed `lastCommit` to `HEAD`, surfaces hints.
|
||
|
||
## MCP tools
|
||
|
||
| Tool | Purpose |
|
||
|------|---------|
|
||
| `list_repos` | Discover indexed repos |
|
||
| `query` | Hybrid BM25 + vector search over the graph |
|
||
| `cypher` | Ad hoc Cypher against the schema |
|
||
| `context` | Callers, callees, processes for one symbol |
|
||
| `impact` | Blast radius (upstream/downstream) with risk summary |
|
||
| `detect_changes` | Map git diffs to affected symbols and processes |
|
||
| `rename` | Graph-assisted multi-file rename with `dry_run` preview |
|
||
| `api_impact` | Pre-change impact report for an API route handler |
|
||
| `trace` | Shortest directed path between two symbols (call + class-member edges); group-aware (`repo: "@<group>"`) for cross-repo traces |
|
||
| `route_map` | API route → handler → consumer mappings |
|
||
| `tool_map` | MCP/RPC tool definitions and handlers |
|
||
| `shape_check` | Response shape vs consumer property access mismatches |
|
||
| `explain` | Persisted taint findings (source→sink data flows) — needs `analyze --pdg` |
|
||
| `pdg_query` | Control/data dependence — CDG (`mode: controls`) / REACHING_DEF (`mode: flows`) — needs `analyze --pdg` |
|
||
| `group_list` | List repo groups or details for one group |
|
||
| `group_sync` | Rebuild group Contract Registry (`contracts.json`) and bridge graph |
|
||
|
||
`query`, `context`, and `impact` are group-aware: pass `repo: "@<groupName>"` (or `"@<groupName>/<memberPath>"` to scope to one member) plus optional `service: "<monorepo/path>"`. Group-mode `query` merges per-repo results via Reciprocal Rank Fusion; group-mode `impact` runs the local walk in the chosen member and fans out across boundaries via the Contract Bridge (`gitnexus/src/core/group/cross-impact.ts`). `trace` is also group-aware via `repo: "@<groupName>"` — but, unlike the others, it resolves `from`/`to` across **all** members (a `@<groupName>/<memberPath>` suffix is advisory for trace, not a scope); pass `from_uid`/`to_uid` to disambiguate a symbol name that occurs in more than one member.
|
||
|
||
Group-mode `trace` (`gitnexus/src/core/group/cross-trace.ts`) stitches a path that crosses repositories: it resolves `from`/`to` across all members, and when they live in different repos it joins the home-repo segment to the target-repo segment over a single `ContractLink` boundary (an HTTP consumer→provider link, joined on `Contract.symbolUid`), reported as a `CONTRACT_LINK` hop in `crossings[]`. The crossing is clamped to one boundary (`MAX_SUPPORTED_CROSS_DEPTH`, shared with cross-impact); deeper `crossDepth` is reported via `notes[]`. With `pdg: true` (experimental, opt-in), each boundary-adjacent segment is enriched with its intra-procedural REACHING_DEF data-flow when that repo was indexed with `--pdg` (reusing the same anchored `flows` query as `pdg_query`); data flow never crosses the repo boundary, and a missing PDG layer degrades to call-level hops with a note. Two stores meet only at the `symbolUid` grain — the per-repo PDG/call graph and the group bridge — so this is the documented join; full cross-program (SDG-like) data flow across the boundary remains deferred (see `docs/plans/2026-06-18-002-feat-unified-pdg-impact-evaluation-plan.md`). The previously-planned `group_query`, `group_context`, `group_impact`, `group_contracts`, `group_status` MCP tools are intentionally not introduced — group-level state is exposed via resources instead:
|
||
|
||
| Resource URI | Purpose |
|
||
|--------------|---------|
|
||
| `gitnexus://group/{name}/contracts` | Contract Registry (provider/consumer rows + cross-links) |
|
||
| `gitnexus://group/{name}/status` | Per-member index + Contract Registry staleness |
|
||
|
||
## Where to change what
|
||
|
||
| Concern | Start in |
|
||
|---------|----------|
|
||
| CLI commands/flags | `src/cli/` (`index.ts`, per-command modules) |
|
||
| Parsing/graph construction | `src/core/ingestion/pipeline-phases/` + `pipeline.ts` |
|
||
| Graph schema/DB | `src/core/lbug/` (`schema.ts`, `lbug-adapter.ts`) |
|
||
| MCP tools/resources | `src/mcp/server.ts`, `tools.ts`, `resources.ts` |
|
||
| Cross-repo groups (sync, contracts, `@<group>` routing) | `src/core/group/` (`service.ts`, `cross-impact.ts`, `sync.ts`, `bridge-db.ts`) |
|
||
| Search ranking | `src/core/search/` (BM25, hybrid fusion) |
|
||
| Embeddings | `src/core/embeddings/` + `src/core/run-analyze.ts` |
|
||
| Wiki generation | `src/core/wiki/` |
|
||
| Language support | `src/core/ingestion/languages/` + `tree-sitter-queries.ts` + `gitnexus-shared/src/languages.ts` |
|
||
| Import resolution | `src/core/ingestion/import-processor.ts` + `import-resolvers/configs/` + `model/resolution-context.ts` |
|
||
| Call resolution/inheritance/MRO | `src/core/ingestion/scope-resolution/` (pipeline, passes, graph-bridge) |
|
||
| Type extraction | `src/core/ingestion/type-extractors/` |
|
||
| Worker pool | `src/core/ingestion/workers/` |
|
||
| Web UI | `gitnexus-web/src/` |
|
||
| CI | `.github/workflows/*.yml`, `.github/actions/` |
|
||
|
||
> Paths above are relative to `gitnexus/` unless they start with `gitnexus-web/` or `.github/`.
|
||
|
||
---
|
||
|
||
## Pipeline Phase DAG
|
||
|
||
15 phases defined in `gitnexus/src/core/ingestion/pipeline-phases/`, each with explicit `deps` and typed output.
|
||
|
||
```
|
||
scan → structure → [markdown, cobol] → parse → [routes, tools, orm]
|
||
→ crossFile → scopeResolution → pruneLocalSymbols → mro → di → communities → processes
|
||
```
|
||
|
||
| Phase | File | Deps | Output |
|
||
|-------|------|------|--------|
|
||
| `scan` | `scan.ts` | (root) | File paths + sizes |
|
||
| `structure` | `structure.ts` | `scan` | File/Folder nodes, CONTAINS edges, `allPathSet` |
|
||
| `markdown` | `markdown.ts` | `structure` | Section nodes, cross-link edges from .md/.mdx |
|
||
| `cobol` | `cobol.ts` | `structure` | COBOL program/paragraph/section nodes (regex, no tree-sitter) |
|
||
| `parse` | `parse.ts` + `parse-impl.ts` | `structure`, `markdown`, `cobol` | Symbol nodes, IMPORTS/CALLS/EXTENDS edges, extracted routes/tools/ORM queries |
|
||
| `routes` | `routes.ts` | `parse` | Route nodes + HANDLES_ROUTE edges (Next.js, Expo, PHP, decorators) |
|
||
| `tools` | `tools.ts` | `parse` | Tool nodes + HANDLES_TOOL edges |
|
||
| `orm` | `orm.ts` | `parse` | QUERIES edges (Prisma, Supabase) |
|
||
| `crossFile` | `cross-file.ts` + `cross-file-impl.ts` | `parse`, `routes`, `tools`, `orm` | Cross-file type propagation in topological import order |
|
||
| `scopeResolution` | `scope-resolution/pipeline/phase.ts` | `parse`, `crossFile`, `structure` | Binding/reference + inheritance edges; disposes BindingAccumulator |
|
||
| `pruneLocalSymbols` | `prune-local-symbols.ts` | `scopeResolution` | Drops inert block-local `Const`/`Variable`/`Static` nodes (only a `File→DEFINES` edge) post-resolution |
|
||
| `mro` | `mro.ts` | `crossFile`, `scopeResolution`, `pruneLocalSymbols`, `structure` | METHOD_OVERRIDES + METHOD_IMPLEMENTS edges |
|
||
| `di` | `di.ts` | `mro` | INJECTS edges (framework-neutral DI resolution; per-language matchers registered in `di-extractors/`) |
|
||
| `communities` | `communities.ts` | `mro`, `pruneLocalSymbols`, `structure` | Community nodes + MEMBER_OF edges (Leiden algorithm) |
|
||
| `processes` | `processes.ts` | `communities`, `routes`, `tools`, `pruneLocalSymbols`, `structure` | Process nodes + STEP_IN_PROCESS edges |
|
||
|
||
**Non-phase files in the same directory:** `parse-impl.ts`, `cross-file-impl.ts` (implementation), `wildcard-synthesis.ts` (whole-module import expansion), `types.ts`, `runner.ts`, `index.ts`.
|
||
|
||
### DAG runner
|
||
|
||
`runner.ts` — static phase graph, no plugins, compile-time type safety.
|
||
|
||
1. **Validation** — Kahn's topological sort. Rejects on: duplicate names, missing deps, cycles (DFS traces the concrete cycle path, e.g., `A -> B -> C -> A`, plus count of transitively blocked dependents).
|
||
|
||
2. **Execution** — sequential in topological order. Each phase receives:
|
||
- `ctx: PipelineContext` — shared mutable `KnowledgeGraph`, `repoPath`, progress callback, options
|
||
- `deps: ReadonlyMap<string, PhaseResult>` — **declared deps only** (runner filters the results map to prevent hidden coupling)
|
||
|
||
3. **Error handling** — wraps phase errors with the phase name, emits terminal `error` progress event, swallows progress handler errors to preserve the original cause.
|
||
|
||
4. **Timing** — per-phase `durationMs` in `PhaseResult`, dev-mode console logging.
|
||
|
||
**Design patterns:**
|
||
- **Single graph accumulator** — all phases mutate the same `KnowledgeGraph` in `ctx`; the graph is the primary output.
|
||
- **Typed phase access** — `getPhaseOutput<T>(deps, 'name')` for type-safe upstream results.
|
||
- **Binding accumulator lifecycle** — created in `parse`, disposed by `crossFile` (in `finally`). No other phase should take ownership.
|
||
- **Skippable phases** — `skipGraphPhases` omits MRO/di/communities/processes (faster tests); `pruneLocalSymbols` still runs (it is graph cleanup, not analysis). `skipWorkers` is no longer a sequential escape hatch — it (like `--workers 0` / `GITNEXUS_WORKER_POOL_SIZE=0`) is rejected with an actionable error, since the worker pool is the sole parse path (§ Chunked parse-and-resolve).
|
||
- **Local-symbol pruning** — `pruneLocalSymbols` removes inert block-local value symbols after scope resolution has consumed them. Opt out per-call with `PipelineOptions.keepLocalValueSymbols` or globally with the `GITNEXUS_KEEP_LOCAL_VALUE_SYMBOLS` env var.
|
||
|
||
### How to add a new phase
|
||
|
||
1. Create `pipeline-phases/my-phase.ts` with a `PipelinePhase<MyOutput>` (name, deps, execute)
|
||
2. Export from `pipeline-phases/index.ts`
|
||
3. Add to `buildPhaseList()` in `pipeline.ts`
|
||
|
||
```typescript
|
||
import type { PipelinePhase, PhaseResult } from './types.js';
|
||
import { getPhaseOutput } from './types.js';
|
||
import type { ParseOutput } from './parse.js';
|
||
|
||
export interface MyPhaseOutput { /* ... */ }
|
||
|
||
export const myPhase: PipelinePhase<MyPhaseOutput> = {
|
||
name: 'myPhase',
|
||
deps: ['parse'],
|
||
async execute(ctx, deps) {
|
||
const { allPaths } = getPhaseOutput<ParseOutput>(deps, 'parse');
|
||
// ... write to ctx.graph ...
|
||
return { /* typed output */ };
|
||
},
|
||
};
|
||
```
|
||
|
||
---
|
||
|
||
## Semantic model
|
||
|
||
`SemanticModel` (`gitnexus/src/core/ingestion/model/semantic-model.ts`) is the authoritative store for every symbol-indexed lookup (by `nodeId`, `simpleName`, `qualifiedName`, or `filePath`). The scope-resolution pipeline reads from here: `findOwnedMember`, `pickOverload`, and `findExportedDefByName` all consult `model.methods` / `model.fields` / `model.symbols`.
|
||
|
||
`ParsedFile` (`gitnexus-shared/src/scope-resolution/parsed-file.ts`) is the single per-file artifact the scope-resolution pipeline consumes. Scope-resolution passes MUST NOT build a parallel parse representation. If a per-language hook needs AST-level facts that `ParsedFile` doesn't expose, it should reuse the orchestrator's `treeCache` (`RunScopeResolutionInput.treeCache`) rather than re-invoking `parser.parse(...)` on its own — the C# `populateNamespaceSiblings` hook is the reference implementation of this pattern.
|
||
|
||
The scope-resolution pipeline additionally carries `WorkspaceResolutionIndex` for `Scope`-valued lookups (`classScopeByDefId`, `moduleScopeByFile`) that `SemanticModel` structurally cannot hold. No symbol-indexed duplicates exist outside `SemanticModel`.
|
||
|
||
**Write / read phase contract.** The model is mutable during three ordered phases and read-only afterward:
|
||
|
||
```
|
||
Phase 1: parse ──► symbolTable.add fans into types/methods/fields
|
||
Phase 2: scope-resolution ──► reconcileOwnership() registers corrected ownerIds
|
||
Phase 3: finalize ──► model.attachScopeIndexes(bundle) — one-shot freeze
|
||
─────────────────────────── phase boundary ───────────────────────────
|
||
Read phase: all resolution passes + MCP + HTTP + embeddings see
|
||
SemanticModel (read-only handle); writes are type-errors.
|
||
```
|
||
|
||
`runScopeResolution` narrows `MutableSemanticModel` → `SemanticModel` at the phase boundary so downstream passes physically cannot mutate the model even accidentally.
|
||
|
||
**Reconciliation pass.** `reconcileOwnership` (`scope-resolution/pipeline/reconcile-ownership.ts`) is a shim for languages whose parse-time extractor doesn't resolve `enclosingClassId` at parse time (Python class-body methods are the canonical case). It walks `parsed.localDefs[i].ownerId` after `populateOwners` and registers any missed methods/fields into the model. Idempotent — safe to re-run, safe alongside languages whose extractor already carries `ownerId` (C#).
|
||
|
||
The architectural end state is for every language's parse-time extractor to emit the correct `ownerId` directly, making reconciliation a no-op (tracked as a follow-up refactor). The dev-mode validator `validateOwnershipParity` surfaces any drift via `onWarn` under `NODE_ENV !== 'production' && VALIDATE_SEMANTIC_MODEL !== '0'`.
|
||
|
||
References: `semantic-model.ts` file-head (full write/read contract); `contract/scope-resolver.ts` Contract Invariant I9 (scope-resolution-side rule).
|
||
|
||
---
|
||
|
||
## Scope-Resolution Pipeline (RFC #909 Ring 3)
|
||
|
||
Language-agnostic scope-resolution resolver. This is the resolution path for every language — it owns CALLS/ACCESSES/USES emission and inheritance edges. Adding a language is one interface implementation (`ScopeResolver`) plus one registration in the `SCOPE_RESOLVERS` map — no changes to shared code, no new pipeline phase. (RING4-1 #942 removed the legacy call-resolution DAG and the per-language `MIGRATED_LANGUAGES` flag, so `SCOPE_RESOLVERS` registration is all that's needed.)
|
||
|
||
### Pipeline stages
|
||
|
||
```
|
||
ParsedFile[] (extractParsedFile per file)
|
||
│ finalizeScopeModel (+ provider hooks)
|
||
▼
|
||
ScopeResolutionIndexes
|
||
│ resolveReferenceSites (via MethodRegistry.lookup)
|
||
▼
|
||
ReferenceIndex
|
||
│ emitReceiverBoundCalls ── FIRST
|
||
│ emitFreeCallFallback ── THEN
|
||
│ emitReferencesViaLookup ── uses handledSites + deferred-site skip set
|
||
│ emitPropertyDispatchCalls ── registration USES + conservative CALLS
|
||
│ emitCallableValueFlow ── assigned/passed callable invocation CALLS
|
||
│ emitImportEdges
|
||
▼
|
||
KnowledgeGraph (IMPORTS / CALLS / ACCESSES / INHERITS / USES)
|
||
```
|
||
|
||
Orchestrator: `runScopeResolution(input, provider)` in `scope-resolution/pipeline/run.ts`.
|
||
Pipeline phase: `scopeResolutionPhase` in `scope-resolution/pipeline/phase.ts` — iterates the registered `SCOPE_RESOLVERS` over the worker-serialized `ParsedFile`s. (Per-language `emitScopeCaptures` hooks may reuse a cached Tree via the orchestrator's `treeCache`, but in worker-pool runs that cache is empty — Trees can't cross MessageChannels — so they consume the pre-extracted `ParsedFile` instead; § Performance notes.)
|
||
|
||
### Callable-value flow
|
||
|
||
First-class callable values use a language-neutral inclusion analysis in `passes/callable-value-flow.ts`. Providers recognize their own syntax and emit JSON-safe `CallableFlowSite` facts (`seed`, `copy`, `alias`, `address`, `load`, `store`, `formal`, `argument`, and `invoke`) into `ParsedFile`; shared ingestion never branches on a language name. These always-on facts cross workers and the durable parse store, whose schema is bumped whenever their semantic shape changes.
|
||
|
||
The emit stage defers only invocation sites proven to reference a flow cell. Ordinary receiver/free/reference passes still resolve direct callees first and record exact callee IDs by file/line/column. Property dispatch then runs before callable flow because a property-dispatched wrapper call can seed actual-to-formal propagation. The callable solver consumes those direct targets, propagates callable sets through lexical cells and formals, and emits `CALLS` at the real indirect invocation site with reason `callable-value-flow` (confidence 0.8 for a singleton, 0.7 for a bounded multi-target set).
|
||
|
||
The solver is flow-insensitive but bounded: dependency-indexed work items rerun only when a cell they read changes; target/address sets cap at 32; a hostile fact graph has a finite work budget; overflow or budget exhaustion emits no partial `CALLS` and produces a structured warning. Lexical shadowing is function/block aware, invocation/constructor results are not reinterpreted as callable designators, and overload selection uses provider-supplied signature metadata. C/C++ additionally associate visible prototypes with unique definitions so actual-to-formal flow crosses translation units; the provider-owned `hasFileLocalCallableLinkage` hook prevents `static` declarations or definitions from leaking across files. C++ member-function pointers preserve parameter/cv shape, keep non-virtual targets exact, and expand virtual targets through `MethodDispatchIndex`/MRO.
|
||
|
||
Property-key dispatch remains a separate conservative fallback. Its per-key fan-out cap is 32; capped keys synthesize no partial calls and are reported at warning level with language, skipped-key count, dropped key names (bounded), and cap; the count also travels in `RunScopeResolutionStats.propertyDispatchSkippedKeys`.
|
||
|
||
Standalone (regex-based) providers such as COBOL participate via `ScopeResolver.scopeResolutionEdgeMode: 'callable-flow-only'`: `runScopeResolution` runs for them, but every ordinary emission path — heritage, interface implementations, receiver-bound, free-call fallback, reference/import edges, post-resolution hooks — is gated off, so their legacy phase (e.g. `cobolPhase`) remains the sole owner of structural edges and the callable solver's `CALLS` are purely additive. A callable-flow-only provider whose files emitted no callable facts exits early, before finalize, keeping the opt-in proportional to source scanning.
|
||
|
||
### Optional CFG/PDG emission (`--pdg`, #2081–#2086)
|
||
|
||
On a `--pdg` run the parse worker builds a per-function control-flow graph from the tree-sitter AST (`LanguageProvider.cfgVisitor`; TypeScript/JavaScript today) and serializes it onto `ParsedFile.cfgSideChannel` as plain data. Scope-resolution then emits the program-dependence layers from that side-channel **inside Phase 4 of `runScopeResolution`, while the disk-backed ParsedFile store is still live** — the only window where the worker-built CFGs are loaded (the store is cleared right after the phase returns). A standalone post-`mro` phase would read an empty store, so the emit deliberately lives in-phase, mirroring the `applyCaptureSideChannel` pattern. The opt-in is off by default (graph byte-identical), folded into the parse-cache key (a pdg-off warm cache is never reused on a `--pdg` run), and each layer is bounded by a per-function edge cap that logs any dropped edges. All layers are `BasicBlock → BasicBlock` edges in the single `CodeRelation` table, keyed by `type`; there is **no** `Function → BasicBlock` edge — the symbol↔block join is reconstructed from the BasicBlock id prefix + line span. The layers build on each other:
|
||
|
||
- **M1 — CFG** (#2081): `BasicBlock` nodes + `CFG` edges. Edge *kind* (`seq`/`cond-true`/`loop-back`/…) rides the `reason` column (CFG is one `CodeRelation` type, not one per kind).
|
||
- **M2 — REACHING_DEF** (#2082): GEN/KILL def→use data dependence from a pure fixpoint solver; the variable name rides `reason`.
|
||
- **M3/M4 — TAINTED / SANITIZES / TAINT_PATH** (#2083–#2084): intra- and inter-procedural taint (source→sink) — the `explain` tool's data.
|
||
- **M5 — CDG** (#2085): Ferrante control dependence over a Cooper–Harvey–Kennedy post-dominator tree (the EXIT-rooted reverse CFG); branch sense (`'T'`/`'F'`) rides `reason`. A CFG whose EXIT is unreachable from some block is skipped for CDG (post-dominance would be unsound) while its CFG/REACHING_DEF layers are kept.
|
||
- **M6 — read surface** (#2086): the `pdg_query` MCP tool answers "what gates X?" (CDG, `mode: controls`) and "where does Y flow?" (REACHING_DEF, `mode: flows`); `explain` is the taint consumer. Both are always anchored + `LIMIT`-bounded (LadybugDB has no rel-property index) and share one `resolveBlockAnchor` helper. These PDG edge types are deliberately kept out of the default `VALID_RELATION_TYPES` / web schema.
|
||
- **Cross-repo trace enrichment**: group-mode `trace` (`pdg: true`) reuses the same anchored REACHING_DEF `flows` query to annotate a boundary-adjacent segment with how a value reaches the cross-repo call — strictly intra-procedural (data flow never crosses the repo boundary). See the group-aware tools note above.
|
||
|
||
See `core/ingestion/cfg/` (emit + the pure CFG / post-dominator / control-dependence / reaching-defs / taint passes) and `mcp/local/local-backend.ts` (`_pdgQueryImpl`, `_explainImpl`, the shared `resolveBlockAnchor`).
|
||
|
||
### `ScopeResolver` contract
|
||
|
||
Single interface a language implements to plug into the pipeline. Contract fully documented in `scope-resolution/contract/scope-resolver.ts`.
|
||
|
||
| Hook | Purpose |
|
||
|------|---------|
|
||
| `languageProvider` | Base `LanguageProvider` (tree-sitter query, `emitScopeCaptures`, import/binding interpreters, hooks) |
|
||
| `populateOwners(parsed)` | Fill deferred `ownerId` fields on method defs (captures can't always know the owning class at parse time) |
|
||
| `buildMro(graph, parsed, nodeLookup)` | Produce `mroByClassDefId: Map<DefId, DefId[]>` — C3, Ruby-mixin, or first-wins per language |
|
||
| `resolveImportTarget(target, fromFile, allFiles)` | `(rawImportPath, sourceFile) → targetFilePath` (PEP-328 for Python, etc.) |
|
||
| `mergeBindings(existing, incoming, scopeId)` | Shadowing / LEGB precedence |
|
||
| `arityCompatibility` | Provider consumed by registry during `MethodRegistry.lookup` Step 2 |
|
||
| `importEdgeReason` | Confidence-tier string for IMPORTS edge reason field |
|
||
| `propagatesReturnTypesAcrossImports?` | Opt out of cross-file return-type propagation (default on) |
|
||
| `fieldFallbackOnMethodLookup?` | Statically-typed languages turn this OFF — the heuristic over-connects (default on) |
|
||
| `unwrapCollectionAccessor?` | Property-style collection views (`data.Values` on Dictionary-like receivers) — default off |
|
||
| `collapseMemberCallsByCallerTarget?` | One CALLS edge per (caller, target) instead of per-site — default off |
|
||
| `populateNamespaceSiblings?` | Cross-file implicit visibility (compiler-implicit namespace sharing) — default off; ctx carries `treeCache` |
|
||
| `hoistTypeBindingsToModule?` | Walk up to Module scope when looking up a method's return-type typeBinding — default off; enable only when bindings are stored at module level |
|
||
| `hasFileLocalCallableLinkage?` | Precise internal-linkage predicate used only when joining callable declarations/prototypes to cross-file definitions; C/C++ use it for `static` free functions |
|
||
|
||
### Per-language registration
|
||
|
||
1. Implement `ScopeResolver` in `languages/<lang>/scope-resolver.ts`.
|
||
2. Add entry to `SCOPE_RESOLVERS` in `scope-resolution/pipeline/registry.ts`.
|
||
|
||
CI auto-discovers the set via `tsx`. No workflow edit required.
|
||
|
||
### Code references
|
||
|
||
| Module | Purpose |
|
||
|--------|---------|
|
||
| `scope-resolution/contract/scope-resolver.ts` | `ScopeResolver` interface + shared types |
|
||
| `scope-resolution/pipeline/run.ts` | Generic orchestrator |
|
||
| `scope-resolution/pipeline/phase.ts` | Pipeline-phase wrapper (deps: `parse`, `structure`) |
|
||
| `scope-resolution/pipeline/registry.ts` | `SCOPE_RESOLVERS` map |
|
||
| `scope-resolution/passes/*.ts` | Reference-resolution passes (receiver-bound, free-call fallback, compound-receiver, MRO, cross-file return-type propagation) |
|
||
| `scope-resolution/graph-bridge/*.ts` | CLI-local translation from resolved references → `KnowledgeGraph` edges |
|
||
| `scope-resolution/scope/*.ts` | Generic scope-chain walkers + namespace targets |
|
||
| `scope-resolution/workspace-index.ts` | Build-once O(1) lookup index |
|
||
| `languages/python/index.ts` | Python `ScopeResolver` hooks + known-limitation docs |
|
||
| `languages/python/captures.ts` | `emitPythonScopeCaptures` (honors cross-phase Tree cache) |
|
||
| `languages/csharp/index.ts` | C# `ScopeResolver` hooks + known-limitation docs |
|
||
| `languages/csharp/captures.ts` | `emitCsharpScopeCaptures` (honors cross-phase Tree cache) |
|
||
| `languages/csharp/namespace-siblings.ts` | Cross-file implicit-namespace visibility hook (reads `treeCache`) |
|
||
|
||
### Performance notes
|
||
|
||
- **Cross-phase Tree cache**: the orchestrator's `treeCache` (`RunScopeResolutionInput.treeCache`) lets a scope-resolution per-language hook (`emitScopeCaptures`) reuse a tree instead of re-parsing. Workers leave it empty — Trees can't cross MessageChannels — so in normal (worker-pool) runs scope-resolution does NOT rely on it: workers serialize each file's `ParsedFile` (+ capture side-channel) and stream them in, so scope-resolution consumes the pre-extracted artifact rather than re-parsing on the main thread (§ Chunked parse-and-resolve). `PROF_SCOPE_RESOLUTION=1` emits hit/miss counters and a worker-engaged warning.
|
||
- **Typed relationship iteration**: heritage + MRO walk only the EXTENDS / IMPLEMENTS / HAS_METHOD edges via `iterRelationshipsByType`, not the full relationship map.
|
||
- **Workspace-resolution-index**: O(1) `findOwnedMember` / `findExportedDef` / `classScopeByDefId` built once per run.
|
||
- **Callable-value worklist**: dependency-indexed inclusion propagation is linear in a reverse-ordered copy-chain fixture; target/address sets cap at 32 and the whole worklist has a finite budget with no partial edge emission on exhaustion.
|
||
- **SCC-ordered cross-file return-type propagation** (PR #1050): `propagateImportedReturnTypes` walks `indexes.sccs` in reverse-topological order (leaves first), so multi-hop alias chains like `models.User → service.user → app.user` collapse to the terminal class in a single linear pass. Within each importer, the source module's `typeBindings` is chain-followed BEFORE mirroring (so we mirror terminal types, not intermediate refs), and the importer's own `typeBindings` is chain-followed AFTER mirroring (so local `const x = importedFn()` resolves before downstream importers run). Cyclic SCCs reach a partial fixpoint within a single pass without iterating to convergence — see the `ts-circular` cross-file-binding fixture which only asserts pipeline-no-throw. PROF output (`PROF_SCOPE_RESOLUTION=1`) splits `finalize` from `propagate` so quadratic regressions in the chain-follow surface independently.
|
||
|
||
---
|
||
|
||
## Language-agnostic graph feeding
|
||
|
||
16 languages → single unified graph. Four abstraction layers:
|
||
|
||
```
|
||
Unified Graph Schema (44 node types, 21 relationship types)
|
||
↑
|
||
Scope-Resolution Pipeline (registry lookup + 3-tier import resolution + MRO)
|
||
↑
|
||
Language Providers (import semantics, type config, export checker, MRO strategy)
|
||
↑
|
||
Tree-Sitter Queries (per-language S-expressions, unified capture tags)
|
||
```
|
||
|
||
### Language providers
|
||
|
||
Each language implements `LanguageProvider` (`language-provider.ts`). Key fields:
|
||
|
||
| Field | Purpose |
|
||
|-------|---------|
|
||
| `id`, `extensions` | Language identity and file matching |
|
||
| `treeSitterQueries` | S-expression queries for AST extraction |
|
||
| `importSemantics` | `named` / `wildcard-leaf` / `wildcard-transitive` / `namespace` |
|
||
| `importResolver` | Language-specific path → file resolution |
|
||
| `exportChecker` | Public/exported symbol detection |
|
||
| `typeConfig` | Type annotation extraction rules |
|
||
| `mroStrategy` | `first-wins` / `c3` / `none` |
|
||
| `descriptionExtractor` | Optional hook returning a symbol's doc-comment text as its `description`; feeds the embedding metadata header so doc-only terms are semantically searchable (issue #2270). Most languages register `createLeadingDocDescriptionExtractor` (shared, language-neutral; per-language comment/wrapper config passed at the call site) |
|
||
|
||
16 providers in `languages/index.ts` via `satisfies Record<SupportedLanguages, LanguageProvider>` — missing a language is a compile error.
|
||
|
||
### Unified capture tags
|
||
|
||
Per-language tree-sitter queries use different AST node names but produce the **same semantic capture tags**: `@definition.class`, `@definition.function`, `@call.name`, `@import.source`, `@reference.inherits`. Downstream extraction needs no language branching. Defined in `tree-sitter-queries.ts`.
|
||
|
||
### Import resolution
|
||
|
||
Per-language import resolution uses the **configs + factory** pattern (like call/method/class extractors). Each language declares an `ImportResolutionConfig` in `import-resolvers/configs/`, listing an ordered chain of `ImportResolverStrategy` functions. `createImportResolver()` (in `resolver-factory.ts`) composes them: first non-null result wins. Low-level helpers shared across strategies live alongside the configs in `import-resolvers/` (e.g. `go.ts`, `rust.ts`, `python.ts`).
|
||
|
||
Unified 3-tier algorithm (`model/resolution-context.ts`), per-language `importSemantics` controls which tier activates:
|
||
|
||
| Tier | Confidence | Mechanism |
|
||
|------|-----------|-----------|
|
||
| 1 — same-file | 0.95 | Symbol table for caller's file |
|
||
| 2 — import-scoped | 0.9 | `NamedImportMap` chains (named) or all files in `importMap` (wildcard) |
|
||
| 3 — global | 0.5 | O(1) index lookups: class, impl, callable. Fallback only |
|
||
|
||
| Import strategy | Languages | Behavior |
|
||
|----------------|-----------|----------|
|
||
| `named` | TS, JS, Java, C#, Rust, PHP, Kotlin | Only explicitly imported names visible |
|
||
| `wildcard-leaf` | Go, Ruby, Swift, Dart | Whole-package import, no transitive re-exports |
|
||
| `wildcard-transitive` | C, C++ | `#include` closure chains through re-exports |
|
||
| `namespace` | Python | Module aliases resolved at call site |
|
||
|
||
### Chunked parse-and-resolve
|
||
|
||
`parse` processes files in ~20 MB byte-budget chunks to bound memory. Per chunk:
|
||
1. Worker pool dispatches files (the sole parse path — there is no sequential fallback; `skipWorkers`, `--workers 0`, and `GITNEXUS_WORKER_POOL_SIZE=0` are rejected with an actionable error)
|
||
2. Each worker: detect language → load grammar → run queries → return unified `ParseWorkerResult`
|
||
3. Synthesize wildcard bindings (`wildcard-synthesis.ts`)
|
||
4. Resolve imports
|
||
5. Collect `BindingAccumulator` entries for cross-file propagation
|
||
|
||
Inheritance edges are emitted later, by the scope-resolution phase (`preEmitInheritanceEdges` + `emitHeritageEdges`), not during `parse`.
|
||
|
||
Workers: `workers/worker-pool.ts`, `workers/parse-worker.ts`.
|
||
|
||
**Worker-serialized ParsedFiles (#2038).** To index very large repos (e.g. the Linux kernel) without OOM, the worker pool is the *sole* parse path and workers serialize each file's `ParsedFile` (plus its capture side-channel) in parallel, streaming them to scope-resolution through a disk-backed store. Scope-resolution consumes the pre-extracted artifact instead of re-parsing every file on the main thread — tree-sitter's native input buffers are not GC-reclaimable, so the former main-thread re-parse leaked native memory until the process died. Pool creation is lazy / cache-miss-gated, so a warm all-cache-hit run replays cached worker output without spawning a worker (hence `usedWorkerPool` can be false even when the repo has parseable files).
|
||
|
||
### Inheritance and MRO
|
||
|
||
Inheritance is captured by the `@reference.inherits` tag and emitted by the scope-resolution phase: `preEmitInheritanceEdges` resolves each base in scope, then `emitHeritageEdges` writes the `EXTENDS`/`IMPLEMENTS` edges. The phase then computes method resolution order via each `ScopeResolver`'s `buildMro` hook, feeding a `MethodDispatchIndex` used for owner-scoped lookups. Per-language strategy:
|
||
- **`first-wins`** — Java, C#, C++, TS, Ruby, Go
|
||
- **`c3`** — Python (C3 linearization)
|
||
- **`ruby-mixin`** — Ruby (mixin-aware linearization)
|
||
- **`none`** — single-inheritance languages
|
||
|
||
---
|
||
|
||
## Full analysis flow
|
||
|
||
`runFullAnalysis` in `run-analyze.ts` orchestrates everything around the pipeline:
|
||
|
||
```
|
||
CLI (analyze.ts) → runFullAnalysis(repoPath, options, callbacks)
|
||
1. Early exit if lastCommit == HEAD (unless --force) [0%]
|
||
2. Cache existing embeddings from prior index [0%]
|
||
3. runPipelineFromRepo() → KnowledgeGraph [0-60%]
|
||
4. Clean up legacy KuzuDB files [60%]
|
||
5. initLbug() → loadGraphToLbug() via CSV streaming [60-85%]
|
||
6. Create FTS indexes (File, Function, Class, Method...) [85-90%]
|
||
7. Restore cached embeddings (batch insert) [88%]
|
||
8. Generate new embeddings if --embeddings [90-98%]
|
||
9. Save metadata + register repo + update .gitignore [98-100%]
|
||
10. Generate AI context files (AGENTS.md, CLAUDE.md) [100%]
|
||
```
|
||
|
||
**Options:** `--force` (rebuild regardless), `--embeddings` (opt-in, skipped if >50k nodes), `--skipGit`, `--noStats`.
|
||
|
||
## Storage
|
||
|
||
```
|
||
<repo>/.gitnexus/
|
||
├── lbug # LadybugDB database
|
||
├── lbug.wal # Write-ahead log
|
||
├── lbug.shadow # Shadow sidecar (checkpoint staging)
|
||
├── lbug.lock # Single-writer lock
|
||
├── lbug.{wal,shadow}.dirty-recovery # parked sidecars from a crashed run; safe to delete
|
||
├── gitnexus.json # lastCommit, indexedAt, stats (primary metadata file)
|
||
└── meta.json # legacy mirror of gitnexus.json, kept in sync (see MIGRATION.md)
|
||
|
||
~/.gitnexus/
|
||
└── registry.json # Global repo registry (MCP discovery)
|
||
```
|
||
|
||
Managed by `repo-manager.ts`.
|
||
|
||
## LadybugDB schema
|
||
|
||
Defined in `lbug/schema.ts`. Separate node tables per type, single `CodeRelation` table.
|
||
|
||
**Node tables:** File, Folder, Function, Class, Interface, Method, Constructor, CodeElement, Struct, Enum, Macro, Typedef, Union, Namespace, Trait, Impl, TypeAlias, Const, Static, Property, Record, Delegate, Annotation, Template, Module, Community, Process, Route, Tool, Section, Embedding.
|
||
|
||
**Relation types** (`CodeRelation.type`): CONTAINS, DEFINES, CALLS, IMPORTS, EXTENDS, IMPLEMENTS, HAS_METHOD, HAS_PROPERTY, ACCESSES, METHOD_OVERRIDES, METHOD_IMPLEMENTS, MEMBER_OF, STEP_IN_PROCESS, HANDLES_ROUTE, FETCHES, HANDLES_TOOL, ENTRY_POINT_OF.
|
||
|
||
**Optional `--pdg` additions** (off by default, opt-in via `gitnexus analyze --pdg`; see _Optional CFG/PDG emission_ above): a `BasicBlock` node table, plus the PDG relation types `CFG`, `REACHING_DEF`, `CDG`, `TAINTED`, `SANITIZES`, and `TAINT_PATH` on the same `CodeRelation` table. These are deliberately kept out of the default `VALID_RELATION_TYPES` / web graph schema — query them via `cypher`, `explain`, or `pdg_query`.
|
||
|
||
## Embeddings and search
|
||
|
||
**Embeddings** (`src/core/embeddings/`): Snowflake arctic-embed-xs (384D). Embeddable: File, Function, Class, Method, Interface. Incremental via SHA1 content hash. Separate `Embedding` table.
|
||
|
||
**Search** (`src/core/search/`): Hybrid BM25 + semantic vector, merged via Reciprocal Rank Fusion (K=60).
|
||
|
||
## Known limitations
|
||
|
||
### Overloaded method resolution
|
||
|
||
Node IDs use arity suffix (`#<paramCount>`): `Method:file:Class.method#1` vs `#2`.
|
||
|
||
**Same-arity disambiguation:** type-hash suffix `~type1,type2` when collision detected and type annotations present. Languages without types (Python, Ruby, JS) use arity-only. TS/JS overload signatures excluded (collapse to implementation body). See #651.
|
||
|
||
**C++ const-qualified:** `$const` suffix after type-hash when non-const collision exists: `Method:file:Container.begin#0$const`.
|
||
|
||
**Generic/template types:** type-hash uses `rawType` (full AST text including generics): `~vector<int>` vs `~vector<std::string>`.
|
||
|
||
**ID stability:** collision-only tags mean IDs change when overloads are added. `save#1` becomes `save#1~int` when `save(String)` is added.
|
||
|
||
**Variadic matching:** confidence 0.7 when one side is variadic and the other has fixed count.
|
||
|
||
**METHOD_IMPLEMENTS confidence tiering:**
|
||
|
||
| Match quality | Confidence |
|
||
|---|---|
|
||
| Exact parameter types match | 1.0 |
|
||
| Arity match, types unavailable | 1.0 |
|
||
| Variadic vs fixed | 0.7 |
|
||
| Insufficient info | 0.7 |
|
||
|
||
## Related docs
|
||
|
||
- [MIGRATION.md](MIGRATION.md) — breaking changes and migration guidance
|
||
- [RUNBOOK.md](RUNBOOK.md) — operational commands and recovery
|
||
- [GUARDRAILS.md](GUARDRAILS.md) — safety boundaries for humans and agents
|
||
- [TESTING.md](TESTING.md) — how to run tests
|
||
- `AGENTS.md` / `CLAUDE.md` — agent workflows and tool usage
|