From 7016b32af2dd70337baab7414297ae8b2d7e0fe0 Mon Sep 17 00:00:00 2001 From: ivkond Date: Mon, 13 Apr 2026 07:36:15 +0000 Subject: [PATCH] docs+fix(group): address remaining Claude review items + add pipeline flow chart MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Fixes **Remaining 🔴 — HTTP contract id wildcard format.** Documented the `http::*::` format as an intentional wildcard for manifest links that omit the HTTP method, alongside the explicit-method form (`GET::/path` → `http::GET::/path`). The docblock on `buildContractId` now states both forms, notes that wildcard-aware matching is the responsibility of the sync / cross-impact layer (#793), and recommends the explicit-method form whenever the author knows the method (it round-trips through exact equality without needing wildcard logic downstream). Tests unchanged — the wildcard format is what they've always asserted. **Minor 1 — stale comment at `manifest-extractor.ts:124-126`.** The comment claimed "creates a contract with an empty symbolUid/ref" but the code switched to `manifestSymbolUid(repo, contractId)` a few commits back. Updated to describe the actual synthetic-uid fallback semantics and the cross-impact path that relies on both sides of the join deriving the same uid. **Minor 2 — exhaustiveness guard on `buildContractId`.** The `switch(type)` covered all five current `ContractType` variants but silently returned `undefined` if a new variant was added. Added a `default: const _exhaustive: never = type; throw new Error(...)` clause so the build fails loudly on an unhandled variant. **Minor 3 — `tree.rootNode.text` in `grpc-patterns/node.ts`.** Already fixed in `2f28bfc` via a dedicated structural query (`LOAD_PACKAGE_DEFINITION_SPEC`). No action needed. ## New: pipeline flow chart (per @magyargergo's request) Added `src/core/group/PIPELINE.md` with four Mermaid diagrams: 1. **High-level overview** — `group.yaml` → extractors + manifest → contract matching → `bridge.lbug` → `runGroupImpact`. 2. **Per-repo extractor two-strategy shape** — graph-assisted Strategy A vs. source-scan Strategy B. 3. **Plugin architecture** — orchestrator → registry → per-language `*-patterns/.ts` → `tree-sitter-scanner.ts` → `ExtractedContract`. 4. **Manifest extraction** — label-scoped `resolveSymbol` with the synthetic-uid fallback. 5. **Cross-impact query (#606)** — local impact → bridge join → cross-repo fan-out. Each diagram is annotated with which PRs own which stage (this PR: extractors + manifest; #795: bridge storage; #606: cross-impact runtime) and points at the concrete files/functions involved. ## Tests - 99/99 extractor tests pass - `npx tsc -p tsconfig.json --noEmit` clean Co-authored-by: Claude --- gitnexus/src/core/group/PIPELINE.md | 139 ++++++++++++++++++ .../group/extractors/manifest-extractor.ts | 36 ++++- 2 files changed, 172 insertions(+), 3 deletions(-) create mode 100644 gitnexus/src/core/group/PIPELINE.md diff --git a/gitnexus/src/core/group/PIPELINE.md b/gitnexus/src/core/group/PIPELINE.md new file mode 100644 index 000000000..7730b48e7 --- /dev/null +++ b/gitnexus/src/core/group/PIPELINE.md @@ -0,0 +1,139 @@ +# Group Analysis Pipeline + +Flow chart of the cross-repo contract extraction + matching pipeline. +This covers what runs **inside this PR** (extractors + manifest) and +the downstream handoff to the bridge storage (PR #795) and +cross-impact query (PR #606). + +## High-level overview + +```mermaid +flowchart TD + A[group.yaml] --> B[GroupConfig parser] + B --> C{For each repo
in group} + C --> D[Per-repo LadybugDB
indexed by main pipeline] + + D --> E1[TopicExtractor] + D --> E2[HttpRouteExtractor] + D --> E3[GrpcExtractor] + + E1 --> F[ExtractedContract array
per repo] + E2 --> F + E3 --> F + + B --> M[ManifestExtractor] + M --> G[Manifest contracts
+ cross-links] + + F --> H[Contract matching
exact + wildcard] + G --> H + + H --> I[(bridge.lbug
#795)] + + I --> J[runGroupImpact
#606] + J --> K[CrossRepoImpact] +``` + +## Per-repo extractor pipeline + +Each extractor under `src/core/group/extractors/` follows the same +two-strategy shape: + +```mermaid +flowchart TD + R[RepoHandle + CypherExecutor
for this repo] --> S{Graph-assisted
Strategy A
available?} + + S -->|yes| A1[Cypher query against
per-repo LadybugDB] + A1 --> A2{non-empty
result?} + A2 -->|yes| OUT[ExtractedContract array] + A2 -->|no| B1 + + S -->|no| B1[Source-scan Strategy B] + B1 --> B2[glob repo source files] + B2 --> B3{ext in registry?} + B3 -->|yes| B4[Per-language plugin
scan parsed tree] + B3 -->|no| SKIP[skip file] + B4 --> OUT + + SKIP --> B2 +``` + +**Strategy A** (graph-assisted) uses Cypher over edges already produced +by the main ingestion pipeline: +- HTTP: `HANDLES_ROUTE` / `FETCHES` edges from `(File)-[]->(Route)` +- topic: none (pipeline doesn't yet produce topic nodes — Strategy B only) +- gRPC: none (Strategy B + proto map only) + +**Strategy B** (source-scan) is 100% tree-sitter based after this PR. +Each `*-patterns/.ts` plugin owns its grammar + S-expression +queries; the top-level orchestrator imports neither. + +## Plugin architecture + +```mermaid +flowchart LR + O[Orchestrator
topic|http|grpc-extractor.ts] --> REG[REGISTRY
*-patterns/index.ts] + REG --> P1[java.ts
tree-sitter-java] + REG --> P2[go.ts
tree-sitter-go] + REG --> P3[python.ts
tree-sitter-python] + REG --> P4[node.ts
JS + TS + TSX] + REG --> P5[php.ts
tree-sitter-php
HTTP only] + REG --> P6[proto.ts
tree-sitter-proto
gRPC only, optional] + + P1 --> SCAN[tree-sitter-scanner.ts
compilePatterns + runCompiledPatterns] + P2 --> SCAN + P3 --> SCAN + P4 --> SCAN + P5 --> SCAN + P6 --> SCAN + + SCAN --> DET[Detection objects
TopicMeta / HttpDetection / GrpcDetection] + DET --> O + O --> CT[ExtractedContract array] +``` + +The orchestrator never imports a grammar. Adding a new language / +framework = drop one file in `*-patterns/`, register it in +`index.ts`. No orchestrator edits required. + +## Manifest extraction + +```mermaid +flowchart TD + Y[group.yaml links] --> ME[ManifestExtractor] + ME --> LOOP{for each link} + LOOP --> RES[resolveSymbol
label-scoped Cypher] + RES --> OK{found?} + OK -->|yes| REF[real symbol uid + ref] + OK -->|no| SYN[synthetic uid
manifest::repo::cid] + + REF --> EMIT[emit provider + consumer
Contract objects
+ CrossLink] + SYN --> EMIT + + EMIT --> BRIDGE[(bridge.lbug
#795)] +``` + +Label-scoped queries in `resolveSymbol` keep accidental cross-matches +out: +- `topic` → `(n:Function|Method|Class|Interface)` +- `grpc` method → `(n:Function|Method)`, service → `(n:Class|Interface)` +- `lib` → `(n:Package|Module)` + +## Cross-impact query (PR #606) + +```mermaid +flowchart TD + U[User changes symbol S
in repo R] --> LI[Local impact engine
per-repo uid expansion] + LI --> IDS[Affected uid set] + + IDS --> BR[Bridge query
MATCH Contract WHERE uid IN ids] + BR --> CL[CrossLink traversal] + CL --> OTHER[Matching contract in
other repo] + + OTHER --> FE[Fan-out impact
to consuming repo] + FE --> OUT[CrossRepoImpact
per affected repo] +``` + +The bridge stores every extracted contract keyed by `symbolUid`. +Manifest-sourced contracts use the synthetic uid form so both sides +of the `(local impact) ↔ (bridge query)` join derive the same uid +without coordinating through any shared state. diff --git a/gitnexus/src/core/group/extractors/manifest-extractor.ts b/gitnexus/src/core/group/extractors/manifest-extractor.ts index 4cfcd5478..29c8f9b21 100644 --- a/gitnexus/src/core/group/extractors/manifest-extractor.ts +++ b/gitnexus/src/core/group/extractors/manifest-extractor.ts @@ -121,9 +121,12 @@ export class ManifestExtractor { // match "/suborders", and a gRPC manifest entry in a repo with any // .proto file would attach to a random proto symbol. // - // If resolveSymbol returns null, the extractor creates a contract with - // an empty symbolUid/ref — cross-impact still works via name-based - // matching through the `hint` path in runGroupImpact. + // If resolveSymbol returns null, the extractor falls back to a + // deterministic synthetic uid via `manifestSymbolUid(repo, contractId)` + // (see the function's docstring for why synthetic rather than empty). + // Cross-impact still works: the bridge query joins on the synthetic + // uid, and the local impact engine derives the same uid for the + // unresolved symbol — name-based hints are the additional safety net. try { let rows: Record[]; if (link.type === 'http') { @@ -219,6 +222,29 @@ export class ManifestExtractor { return null; } + /** + * Build a canonical contract id for a manifest link. + * + * HTTP is the only type with two valid forms: + * - Explicit method: `"GET::/api/orders"` → `"http::GET::/api/orders"` + * (matches exactly against `HttpRouteExtractor` provider/consumer + * contracts, which are also keyed by `http::::`). + * - Method-agnostic: `"/api/orders"` → `"http::*::/api/orders"` + * — the `*` is a wildcard and is intended to match any concrete + * HTTP method on that path. Wildcard-aware matching is the + * responsibility of the sync / cross-impact layer (see #793); + * downstream code should treat `http::*::` as matching + * every `http::::` for the same path. + * + * Recommend the explicit-method form in group.yaml whenever the + * manifest author knows the method — it round-trips through exact + * equality matching without requiring wildcard logic downstream. + * + * NOTE on exhaustiveness: the switch covers every current + * `ContractType` variant and falls through to a `never` assertion so + * TypeScript fails the build if a new variant is added without a + * corresponding case. + */ private buildContractId(type: ContractType, contract: string): string { switch (type) { case 'http': { @@ -233,6 +259,10 @@ export class ManifestExtractor { return `lib::${contract}`; case 'custom': return `custom::${contract}`; + default: { + const _exhaustive: never = type; + throw new Error(`Unhandled ContractType: ${String(_exhaustive)}`); + } } } }