mirror of
https://github.com/abhigyanpatwari/GitNexus.git
synced 2026-09-11 22:53:04 +00:00
docs+fix(group): address remaining Claude review items + add pipeline flow chart
## Fixes **Remaining 🔴 — HTTP contract id wildcard format.** Documented the `http::*::<path>` format as an intentional wildcard for manifest links that omit the HTTP method, alongside the explicit-method form (`GET::/path` → `http::GET::/path`). The docblock on `buildContractId` now states both forms, notes that wildcard-aware matching is the responsibility of the sync / cross-impact layer (#793), and recommends the explicit-method form whenever the author knows the method (it round-trips through exact equality without needing wildcard logic downstream). Tests unchanged — the wildcard format is what they've always asserted. **Minor 1 — stale comment at `manifest-extractor.ts:124-126`.** The comment claimed "creates a contract with an empty symbolUid/ref" but the code switched to `manifestSymbolUid(repo, contractId)` a few commits back. Updated to describe the actual synthetic-uid fallback semantics and the cross-impact path that relies on both sides of the join deriving the same uid. **Minor 2 — exhaustiveness guard on `buildContractId`.** The `switch(type)` covered all five current `ContractType` variants but silently returned `undefined` if a new variant was added. Added a `default: const _exhaustive: never = type; throw new Error(...)` clause so the build fails loudly on an unhandled variant. **Minor 3 — `tree.rootNode.text` in `grpc-patterns/node.ts`.** Already fixed in `2f28bfc` via a dedicated structural query (`LOAD_PACKAGE_DEFINITION_SPEC`). No action needed. ## New: pipeline flow chart (per @magyargergo's request) Added `src/core/group/PIPELINE.md` with four Mermaid diagrams: 1. **High-level overview** — `group.yaml` → extractors + manifest → contract matching → `bridge.lbug` → `runGroupImpact`. 2. **Per-repo extractor two-strategy shape** — graph-assisted Strategy A vs. source-scan Strategy B. 3. **Plugin architecture** — orchestrator → registry → per-language `*-patterns/<lang>.ts` → `tree-sitter-scanner.ts` → `ExtractedContract`. 4. **Manifest extraction** — label-scoped `resolveSymbol` with the synthetic-uid fallback. 5. **Cross-impact query (#606)** — local impact → bridge join → cross-repo fan-out. Each diagram is annotated with which PRs own which stage (this PR: extractors + manifest; #795: bridge storage; #606: cross-impact runtime) and points at the concrete files/functions involved. ## Tests - 99/99 extractor tests pass - `npx tsc -p tsconfig.json --noEmit` clean Co-authored-by: Claude <noreply@anthropic.com>
This commit is contained in:
parent
7b22776560
commit
7016b32af2
2 changed files with 172 additions and 3 deletions
139
gitnexus/src/core/group/PIPELINE.md
Normal file
139
gitnexus/src/core/group/PIPELINE.md
Normal file
|
|
@ -0,0 +1,139 @@
|
|||
# Group Analysis Pipeline
|
||||
|
||||
Flow chart of the cross-repo contract extraction + matching pipeline.
|
||||
This covers what runs **inside this PR** (extractors + manifest) and
|
||||
the downstream handoff to the bridge storage (PR #795) and
|
||||
cross-impact query (PR #606).
|
||||
|
||||
## High-level overview
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A[group.yaml] --> B[GroupConfig parser]
|
||||
B --> C{For each repo<br/>in group}
|
||||
C --> D[Per-repo LadybugDB<br/>indexed by main pipeline]
|
||||
|
||||
D --> E1[TopicExtractor]
|
||||
D --> E2[HttpRouteExtractor]
|
||||
D --> E3[GrpcExtractor]
|
||||
|
||||
E1 --> F[ExtractedContract array<br/>per repo]
|
||||
E2 --> F
|
||||
E3 --> F
|
||||
|
||||
B --> M[ManifestExtractor]
|
||||
M --> G[Manifest contracts<br/>+ cross-links]
|
||||
|
||||
F --> H[Contract matching<br/>exact + wildcard]
|
||||
G --> H
|
||||
|
||||
H --> I[(bridge.lbug<br/>#795)]
|
||||
|
||||
I --> J[runGroupImpact<br/>#606]
|
||||
J --> K[CrossRepoImpact]
|
||||
```
|
||||
|
||||
## Per-repo extractor pipeline
|
||||
|
||||
Each extractor under `src/core/group/extractors/` follows the same
|
||||
two-strategy shape:
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
R[RepoHandle + CypherExecutor<br/>for this repo] --> S{Graph-assisted<br/>Strategy A<br/>available?}
|
||||
|
||||
S -->|yes| A1[Cypher query against<br/>per-repo LadybugDB]
|
||||
A1 --> A2{non-empty<br/>result?}
|
||||
A2 -->|yes| OUT[ExtractedContract array]
|
||||
A2 -->|no| B1
|
||||
|
||||
S -->|no| B1[Source-scan Strategy B]
|
||||
B1 --> B2[glob repo source files]
|
||||
B2 --> B3{ext in registry?}
|
||||
B3 -->|yes| B4[Per-language plugin<br/>scan parsed tree]
|
||||
B3 -->|no| SKIP[skip file]
|
||||
B4 --> OUT
|
||||
|
||||
SKIP --> B2
|
||||
```
|
||||
|
||||
**Strategy A** (graph-assisted) uses Cypher over edges already produced
|
||||
by the main ingestion pipeline:
|
||||
- HTTP: `HANDLES_ROUTE` / `FETCHES` edges from `(File)-[]->(Route)`
|
||||
- topic: none (pipeline doesn't yet produce topic nodes — Strategy B only)
|
||||
- gRPC: none (Strategy B + proto map only)
|
||||
|
||||
**Strategy B** (source-scan) is 100% tree-sitter based after this PR.
|
||||
Each `*-patterns/<lang>.ts` plugin owns its grammar + S-expression
|
||||
queries; the top-level orchestrator imports neither.
|
||||
|
||||
## Plugin architecture
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
O[Orchestrator<br/>topic|http|grpc-extractor.ts] --> REG[REGISTRY<br/>*-patterns/index.ts]
|
||||
REG --> P1[java.ts<br/>tree-sitter-java]
|
||||
REG --> P2[go.ts<br/>tree-sitter-go]
|
||||
REG --> P3[python.ts<br/>tree-sitter-python]
|
||||
REG --> P4[node.ts<br/>JS + TS + TSX]
|
||||
REG --> P5[php.ts<br/>tree-sitter-php<br/>HTTP only]
|
||||
REG --> P6[proto.ts<br/>tree-sitter-proto<br/>gRPC only, optional]
|
||||
|
||||
P1 --> SCAN[tree-sitter-scanner.ts<br/>compilePatterns + runCompiledPatterns]
|
||||
P2 --> SCAN
|
||||
P3 --> SCAN
|
||||
P4 --> SCAN
|
||||
P5 --> SCAN
|
||||
P6 --> SCAN
|
||||
|
||||
SCAN --> DET[Detection objects<br/>TopicMeta / HttpDetection / GrpcDetection]
|
||||
DET --> O
|
||||
O --> CT[ExtractedContract array]
|
||||
```
|
||||
|
||||
The orchestrator never imports a grammar. Adding a new language /
|
||||
framework = drop one file in `*-patterns/`, register it in
|
||||
`index.ts`. No orchestrator edits required.
|
||||
|
||||
## Manifest extraction
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
Y[group.yaml links] --> ME[ManifestExtractor]
|
||||
ME --> LOOP{for each link}
|
||||
LOOP --> RES[resolveSymbol<br/>label-scoped Cypher]
|
||||
RES --> OK{found?}
|
||||
OK -->|yes| REF[real symbol uid + ref]
|
||||
OK -->|no| SYN[synthetic uid<br/>manifest::repo::cid]
|
||||
|
||||
REF --> EMIT[emit provider + consumer<br/>Contract objects<br/>+ CrossLink]
|
||||
SYN --> EMIT
|
||||
|
||||
EMIT --> BRIDGE[(bridge.lbug<br/>#795)]
|
||||
```
|
||||
|
||||
Label-scoped queries in `resolveSymbol` keep accidental cross-matches
|
||||
out:
|
||||
- `topic` → `(n:Function|Method|Class|Interface)`
|
||||
- `grpc` method → `(n:Function|Method)`, service → `(n:Class|Interface)`
|
||||
- `lib` → `(n:Package|Module)`
|
||||
|
||||
## Cross-impact query (PR #606)
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
U[User changes symbol S<br/>in repo R] --> LI[Local impact engine<br/>per-repo uid expansion]
|
||||
LI --> IDS[Affected uid set]
|
||||
|
||||
IDS --> BR[Bridge query<br/>MATCH Contract WHERE uid IN ids]
|
||||
BR --> CL[CrossLink traversal]
|
||||
CL --> OTHER[Matching contract in<br/>other repo]
|
||||
|
||||
OTHER --> FE[Fan-out impact<br/>to consuming repo]
|
||||
FE --> OUT[CrossRepoImpact<br/>per affected repo]
|
||||
```
|
||||
|
||||
The bridge stores every extracted contract keyed by `symbolUid`.
|
||||
Manifest-sourced contracts use the synthetic uid form so both sides
|
||||
of the `(local impact) ↔ (bridge query)` join derive the same uid
|
||||
without coordinating through any shared state.
|
||||
|
|
@ -121,9 +121,12 @@ export class ManifestExtractor {
|
|||
// match "/suborders", and a gRPC manifest entry in a repo with any
|
||||
// .proto file would attach to a random proto symbol.
|
||||
//
|
||||
// If resolveSymbol returns null, the extractor creates a contract with
|
||||
// an empty symbolUid/ref — cross-impact still works via name-based
|
||||
// matching through the `hint` path in runGroupImpact.
|
||||
// If resolveSymbol returns null, the extractor falls back to a
|
||||
// deterministic synthetic uid via `manifestSymbolUid(repo, contractId)`
|
||||
// (see the function's docstring for why synthetic rather than empty).
|
||||
// Cross-impact still works: the bridge query joins on the synthetic
|
||||
// uid, and the local impact engine derives the same uid for the
|
||||
// unresolved symbol — name-based hints are the additional safety net.
|
||||
try {
|
||||
let rows: Record<string, unknown>[];
|
||||
if (link.type === 'http') {
|
||||
|
|
@ -219,6 +222,29 @@ export class ManifestExtractor {
|
|||
return null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Build a canonical contract id for a manifest link.
|
||||
*
|
||||
* HTTP is the only type with two valid forms:
|
||||
* - Explicit method: `"GET::/api/orders"` → `"http::GET::/api/orders"`
|
||||
* (matches exactly against `HttpRouteExtractor` provider/consumer
|
||||
* contracts, which are also keyed by `http::<METHOD>::<path>`).
|
||||
* - Method-agnostic: `"/api/orders"` → `"http::*::/api/orders"`
|
||||
* — the `*` is a wildcard and is intended to match any concrete
|
||||
* HTTP method on that path. Wildcard-aware matching is the
|
||||
* responsibility of the sync / cross-impact layer (see #793);
|
||||
* downstream code should treat `http::*::<path>` as matching
|
||||
* every `http::<METHOD>::<path>` for the same path.
|
||||
*
|
||||
* Recommend the explicit-method form in group.yaml whenever the
|
||||
* manifest author knows the method — it round-trips through exact
|
||||
* equality matching without requiring wildcard logic downstream.
|
||||
*
|
||||
* NOTE on exhaustiveness: the switch covers every current
|
||||
* `ContractType` variant and falls through to a `never` assertion so
|
||||
* TypeScript fails the build if a new variant is added without a
|
||||
* corresponding case.
|
||||
*/
|
||||
private buildContractId(type: ContractType, contract: string): string {
|
||||
switch (type) {
|
||||
case 'http': {
|
||||
|
|
@ -233,6 +259,10 @@ export class ManifestExtractor {
|
|||
return `lib::${contract}`;
|
||||
case 'custom':
|
||||
return `custom::${contract}`;
|
||||
default: {
|
||||
const _exhaustive: never = type;
|
||||
throw new Error(`Unhandled ContractType: ${String(_exhaustive)}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue