mirror of
https://github.com/abhigyanpatwari/GitNexus.git
synced 2026-09-11 22:53:04 +00:00
refactor(group): migrate grpc-extractor source scans to tree-sitter plugins
Phase 3 (final) of the extractor refactor requested by @magyargergo on #796. Same architecture as phase 1 (topic) and phase 2 (http): thin language-agnostic orchestrator + per-language plugins that own tree-sitter grammars and query sources. With this commit the top-level extractors under `src/core/group/extractors/` import ZERO tree-sitter grammars and ZERO query strings — every grammar import lives in a `*-patterns/<lang>.ts` plugin file, and the orchestrators go through the registry indirection. ## Architecture ``` src/core/group/extractors/ ├── tree-sitter-scanner.ts # shared primitives (unchanged) ├── grpc-extractor.ts # orchestrator (only `.proto` parser left) └── grpc-patterns/ ├── types.ts # GrpcDetection, GrpcLanguagePlugin, GrpcRole ├── index.ts # registry: ext → plugin + GRPC_SCAN_GLOB ├── go.ts # tree-sitter-go: RegisterXxxServer, Unimplemented, NewXxxClient ├── java.ts # tree-sitter-java: @GrpcService + XxxImplBase + newBlockingStub ├── python.ts # tree-sitter-python: add_XxxServicer_to_server + XxxStub └── node.ts # tree-sitter-javascript + tree-sitter-typescript: # @GrpcMethod, @GrpcClient field type, # .getService<X>('Svc'), new XxxServiceClient, # loadPackageDefinition dynamic constructors ``` ## Per-language coverage **Go (`go.ts`)** - Provider: `\w+.RegisterXxxServer(...)` via `call_expression → selector_expression → field_identifier` + JS regex filter `^Register(\w+)Server$`. - Provider: `pb.UnimplementedXxxServer` embedded in a struct via `struct_type → field_declaration_list → field_declaration → qualified_type → type_identifier` + JS filter. - Consumer: `\w+.NewXxxClient(...)` via the same call_expression query + JS filter `^New(\w+)Client$`. **Java (`java.ts`)** - Provider: `class X extends YyyGrpc.YyyImplBase` — two queries handle the scoped and plain forms. `scoped_type_identifier`'s children are positional (no `scope:`/`name:` fields), so the query matches the two `type_identifier` children by position. - `#match? @inner "ImplBase$"` restricts matches at query time. - Whether the class has `@GrpcService` or not controls only the `source` metadata label — the plugin walks the class_declaration's `modifiers` child in JS to detect the marker_annotation. - Consumer: `YyyGrpc.newStub(ch)` / `newBlockingStub(ch)` via a `method_invocation` query with `#match? @method "^new(Blocking)?Stub$"`, service name extracted via `^(\w+)Grpc$` on the object identifier. **Python (`python.ts`)** - Single call-expression query covers both bare identifier and `obj.method` attribute forms: `(call function: [(identifier) @fn (attribute attribute: (identifier) @fn)])`. - Plugin filters `@fn.text` against two JS regexes: `^add_(\w+)Servicer_to_server$` (provider) and `^(\w+)Stub$` (consumer), with a reserved-names ignore list for the Stub case (Mock / Test / Fake / Stub). **Node — JavaScript + TypeScript + TSX (`node.ts`)** - Pattern sources defined once, compiled three times (one per grammar) because `Parser.Query` objects are not portable across grammars. Exports three `GrpcLanguagePlugin`s sharing the same `scan`. - `@GrpcMethod('Service', 'Method')`: decorator query captures the two string literals. Confidence is hard-coded 0.8 regardless of proto map resolution (matches the original regex version's behaviour). - `@GrpcClient(...) field: XxxServiceClient`: decorator query captures the decorator node, plugin walks up to find the enclosing `public_field_definition` (decorators on fields are CHILDREN of the field definition in tree-sitter-typescript, not siblings) and reads its first `type_annotation → type_identifier`, then runs the `^(\w+Service)Client$` JS filter. - `client.getService<X>('AuthService')`: call-expression query on `member_expression.property = "getService"` + string literal arg. - `new XxxServiceClient(...)`: `new_expression` with a bare identifier constructor, filtered by `^(\w+Service)Client$` so generic `new AuthClient(...)` (missing the `Service` infix) does NOT falsely register as a consumer. Preserves the regression test `test_extract_ts_non_service_client_constructor_is_ignored`. - `loadPackageDefinition` dynamic loader: gated on `tree.rootNode.text.includes('loadPackageDefinition')`. When set, `new foo.bar.Xxx(...)` qualified constructors with a capitalised property name register as consumers. ## Orchestrator changes `grpc-extractor.ts` loses every `scanGoProviders` / `scanJavaProviders` / ... helper and replaces them with a single source-scan loop that: 1. Parses each file with the plugin's grammar (one shared `Parser` instance across all files, `setLanguage` called per plugin). 2. Calls `plugin.scan(tree)` to get `GrpcDetection[]`. 3. Converts each detection to an `ExtractedContract` via the private `detectionToContract` helper, which: - Looks the short service name up in the proto map (filled by the `.proto` parser). - Picks confidence = `confidenceWithProto` if resolved, else `confidenceWithoutProto`. - Builds a method-level contract id (`grpc::pkg.Svc/Method`) when the detection carries a `methodName` (TS `@GrpcMethod` only), otherwise a service-level id (`grpc::pkg.Svc/*`). Everything else — the `.proto` parser, `buildProtoContext`, `buildProtoMap`, `resolveProtoConflict`, `serviceContractId`, `stripProtoCommentsAndStrings`, `extractServiceBlocks`, the dedupe function — stays exactly as before. The `.proto` parser is kept as a pragmatic exception to the "no regex in extractors" rule because no `tree-sitter-proto` grammar is installed in the repo; a comment at the top of the file explains this and flags the maintainer option of adding `tree-sitter-proto` as a dependency. ## Why this is better than the regex version 1. **Comments and strings are respected for free.** Matched node types are only code constructs, never text inside comments or string literals. 2. **No false positives on partial names.** The old `(\w+?)Grpc`-style regexes would cross-match unrelated identifiers; structural queries restrict matches to the exact AST shape (`scoped_type_identifier → type_identifier` pairs, `method_invocation → identifier` etc.). 3. **NestJS `@GrpcClient` is structural, not regex-based.** The old regex required a specific textual layout (`@GrpcClient(...) private readonly foo!: XxxServiceClient`); the plugin now walks the AST, so modifier order / optional modifiers / multi-line formatting don't break it. 4. **Language-agnostic extension.** Adding Kotlin / Rust / C# gRPC detection later is a one-file edit in `grpc-patterns/index.ts` — no touches to the shared scanner, the orchestrator, or the proto parser. ## Tests - `grpc-extractor.test.ts` — **43/43 pass** (tests unchanged; the contract shape is identical). Covers .proto parsing (including the brace-inside-string regression), Go provider/consumer, Java @GrpcService / plain ImplBase provider + newBlockingStub consumer, Python servicer + stub, TS @GrpcMethod + @GrpcClient + .getService + new XxxServiceClient + loadPackageDefinition + the `AuthClient` vs `AuthServiceClient` discrimination, dedupe across multiple patterns in one file, proto-aware confidence, and the inherited-package resolution for split proto definitions. - `topic-extractor.test.ts` — 30/30 pass. - `http-route-extractor.test.ts` — 18/18 pass. - `manifest-extractor.test.ts` — 8/8 pass. - `service.test.ts`, `sync.test.ts`, `storage.test.ts` — 41/41 pass. - `npx tsc -p tsconfig.json --noEmit` clean. ## Scope discipline (per GUARDRAILS.md) - Only files under `src/core/group/extractors/` are touched. - No pipeline.ts, MCP surface, ingestion, CI / release / security, or test changes. - New tree-sitter grammar imports (`tree-sitter-go`, `tree-sitter-java`, `tree-sitter-python`, `tree-sitter-javascript`, `tree-sitter-typescript`) are all already installed for the ingestion pipeline. ## End of phase series This commit completes the three-phase extractor refactor: - **Phase 1** (`ea06d11`): topic-extractor → `topic-patterns/` - **Phase 2** (`b6015f6`): http-route-extractor → `http-patterns/` - **Phase 3** (this commit): grpc-extractor → `grpc-patterns/` Every remaining regex-based extractor helper under the `src/core/group/ extractors/` directory is either (a) language-agnostic string processing (path normalization, dedupe keys) or (b) the `.proto` parser, which is documented as an explicit exception. Co-authored-by: Claude <noreply@anthropic.com>
This commit is contained in:
parent
6a76ac0753
commit
2d9a830aa0
7 changed files with 824 additions and 315 deletions
|
|
@ -1,8 +1,32 @@
|
|||
import * as fs from 'node:fs';
|
||||
import * as path from 'node:path';
|
||||
import { glob } from 'glob';
|
||||
import Parser from 'tree-sitter';
|
||||
import type { ContractExtractor, CypherExecutor } from '../contract-extractor.js';
|
||||
import type { ExtractedContract, RepoHandle } from '../types.js';
|
||||
import { GRPC_SCAN_GLOB, getPluginForFile, type GrpcDetection } from './grpc-patterns/index.js';
|
||||
|
||||
/**
|
||||
* Language-agnostic orchestrator for gRPC (provider + consumer) contract
|
||||
* extraction.
|
||||
*
|
||||
* Two parts:
|
||||
*
|
||||
* 1. **`.proto` parsing** — done in-process by a small string-sanitizing
|
||||
* parser (see `stripProtoCommentsAndStrings` + `extractServiceBlocks`
|
||||
* below). NOT tree-sitter-based because no `tree-sitter-proto`
|
||||
* grammar is installed in the repo. The parser preserves offsets so
|
||||
* downstream regex scans run against a sanitized copy without
|
||||
* affecting the line numbers of the original. Kept as a pragmatic
|
||||
* exception to the "no regex in extractors" rule until / unless
|
||||
* maintainers want to add a proto grammar.
|
||||
*
|
||||
* 2. **Source-scan providers / consumers** — delegated to per-language
|
||||
* plugins in `./grpc-patterns/`. The orchestrator imports NO
|
||||
* tree-sitter grammars or query strings — each plugin owns its own.
|
||||
*/
|
||||
|
||||
// ─── .proto parsing (not tree-sitter) ────────────────────────────────
|
||||
|
||||
function readSafe(repoPath: string, rel: string): string | null {
|
||||
const abs = path.resolve(repoPath, rel);
|
||||
|
|
@ -322,6 +346,8 @@ export function serviceContractId(pkg: string, serviceName: string): string {
|
|||
return `grpc::${prefix}/*`;
|
||||
}
|
||||
|
||||
// ─── Orchestrator ────────────────────────────────────────────────────
|
||||
|
||||
export class GrpcExtractor implements ContractExtractor {
|
||||
type = 'grpc' as const;
|
||||
|
||||
|
|
@ -337,7 +363,7 @@ export class GrpcExtractor implements ContractExtractor {
|
|||
const out: ExtractedContract[] = [];
|
||||
const protoContext = await buildProtoContext(repoPath);
|
||||
|
||||
// Proto files — definitive provider source
|
||||
// ─── Proto files — definitive provider source ─────────────────
|
||||
const protoFiles = await glob('**/*.proto', {
|
||||
cwd: repoPath,
|
||||
ignore: ['**/node_modules/**', '**/.git/**', '**/vendor/**'],
|
||||
|
|
@ -357,29 +383,29 @@ export class GrpcExtractor implements ContractExtractor {
|
|||
}
|
||||
const protoMap = protoContext.servicesByName;
|
||||
|
||||
// Source files — server/client detection
|
||||
const sourceFiles = await glob('**/*.{go,java,py,ts,tsx,js,jsx}', {
|
||||
// ─── Source files — delegate to per-language plugins ──────────
|
||||
const sourceFiles = await glob(GRPC_SCAN_GLOB, {
|
||||
cwd: repoPath,
|
||||
ignore: ['**/node_modules/**', '**/.git/**', '**/vendor/**', '**/dist/**', '**/build/**'],
|
||||
nodir: true,
|
||||
});
|
||||
|
||||
const parser = new Parser();
|
||||
for (const rel of sourceFiles) {
|
||||
const plugin = getPluginForFile(rel);
|
||||
if (!plugin) continue;
|
||||
const content = readSafe(repoPath, rel);
|
||||
if (!content) continue;
|
||||
const ext = path.extname(rel).toLowerCase();
|
||||
|
||||
if (ext === '.go') {
|
||||
out.push(...this.scanGoProviders(content, rel, protoMap));
|
||||
out.push(...this.scanGoConsumers(content, rel, protoMap));
|
||||
} else if (ext === '.java') {
|
||||
out.push(...this.scanJavaProviders(content, rel, protoMap));
|
||||
out.push(...this.scanJavaConsumers(content, rel, protoMap));
|
||||
} else if (ext === '.py') {
|
||||
out.push(...this.scanPythonProviders(content, rel, protoMap));
|
||||
out.push(...this.scanPythonConsumers(content, rel, protoMap));
|
||||
} else if (['.ts', '.tsx', '.js', '.jsx'].includes(ext)) {
|
||||
out.push(...this.scanTsProviders(content, rel, protoMap));
|
||||
out.push(...this.scanTsConsumers(content, rel, protoMap));
|
||||
let detections: GrpcDetection[] = [];
|
||||
try {
|
||||
parser.setLanguage(plugin.language);
|
||||
const tree = parser.parse(content);
|
||||
detections = plugin.scan(tree);
|
||||
} catch {
|
||||
continue;
|
||||
}
|
||||
for (const d of detections) {
|
||||
out.push(this.detectionToContract(d, rel, protoMap));
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -409,307 +435,33 @@ export class GrpcExtractor implements ContractExtractor {
|
|||
return out;
|
||||
}
|
||||
|
||||
private scanGoProviders(
|
||||
content: string,
|
||||
/**
|
||||
* Convert a plugin `GrpcDetection` into a concrete `ExtractedContract`
|
||||
* by resolving the short service name against the proto map, building
|
||||
* either a service-level (`grpc::pkg.Svc/*`) or method-level
|
||||
* (`grpc::pkg.Svc/Method`) contract id, and selecting confidence
|
||||
* based on whether the proto map had an entry.
|
||||
*/
|
||||
private detectionToContract(
|
||||
d: GrpcDetection,
|
||||
filePath: string,
|
||||
protoMap: Map<string, ProtoServiceInfo[]>,
|
||||
): ExtractedContract[] {
|
||||
const out: ExtractedContract[] = [];
|
||||
|
||||
// pb.RegisterXxxServer(
|
||||
const registerRe = /\w+\.Register(\w+)Server\s*\(/g;
|
||||
let m: RegExpExecArray | null;
|
||||
while ((m = registerRe.exec(content)) !== null) {
|
||||
const serviceName = m[1];
|
||||
const candidates = protoMap.get(serviceName);
|
||||
const proto = resolveProtoConflict(serviceName, filePath, candidates ?? []);
|
||||
const cid = proto
|
||||
? serviceContractId(proto.package, proto.serviceName)
|
||||
: serviceOnlyContractId(serviceName);
|
||||
const conf = proto ? 0.8 : 0.65;
|
||||
out.push(
|
||||
makeContract(cid, 'provider', filePath, `Register${serviceName}Server`, conf, {
|
||||
service: serviceName,
|
||||
source: 'go_register',
|
||||
}),
|
||||
);
|
||||
}
|
||||
|
||||
// pb.UnimplementedXxxServer
|
||||
const unimplRe = /\w+\.Unimplemented(\w+)Server\b/g;
|
||||
while ((m = unimplRe.exec(content)) !== null) {
|
||||
const serviceName = m[1];
|
||||
const candidates = protoMap.get(serviceName);
|
||||
const proto = resolveProtoConflict(serviceName, filePath, candidates ?? []);
|
||||
const cid = proto
|
||||
? serviceContractId(proto.package, proto.serviceName)
|
||||
: serviceOnlyContractId(serviceName);
|
||||
const conf = proto ? 0.8 : 0.65;
|
||||
out.push(
|
||||
makeContract(cid, 'provider', filePath, `Unimplemented${serviceName}Server`, conf, {
|
||||
service: serviceName,
|
||||
source: 'go_unimplemented',
|
||||
}),
|
||||
);
|
||||
}
|
||||
|
||||
return out;
|
||||
}
|
||||
|
||||
private scanGoConsumers(
|
||||
content: string,
|
||||
filePath: string,
|
||||
protoMap: Map<string, ProtoServiceInfo[]>,
|
||||
): ExtractedContract[] {
|
||||
const out: ExtractedContract[] = [];
|
||||
const re = /\w+\.New(\w+)Client\s*\(/g;
|
||||
let m: RegExpExecArray | null;
|
||||
while ((m = re.exec(content)) !== null) {
|
||||
const serviceName = m[1];
|
||||
const candidates = protoMap.get(serviceName);
|
||||
const proto = resolveProtoConflict(serviceName, filePath, candidates ?? []);
|
||||
const cid = proto
|
||||
? serviceContractId(proto.package, proto.serviceName)
|
||||
: serviceOnlyContractId(serviceName);
|
||||
const conf = proto ? 0.75 : 0.55;
|
||||
out.push(
|
||||
makeContract(cid, 'consumer', filePath, `New${serviceName}Client`, conf, {
|
||||
service: serviceName,
|
||||
source: 'go_client',
|
||||
}),
|
||||
);
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
private scanJavaProviders(
|
||||
content: string,
|
||||
filePath: string,
|
||||
protoMap: Map<string, ProtoServiceInfo[]>,
|
||||
): ExtractedContract[] {
|
||||
const out: ExtractedContract[] = [];
|
||||
|
||||
const resolveJava = (svcName: string): { cid: string; conf: number } => {
|
||||
const candidates = protoMap.get(svcName);
|
||||
const proto = resolveProtoConflict(svcName, filePath, candidates ?? []);
|
||||
const cid = proto
|
||||
? serviceContractId(proto.package, proto.serviceName)
|
||||
: serviceOnlyContractId(svcName);
|
||||
const conf = proto ? 0.8 : 0.65;
|
||||
return { cid, conf };
|
||||
): ExtractedContract {
|
||||
const candidates = protoMap.get(d.serviceName);
|
||||
const proto = resolveProtoConflict(d.serviceName, filePath, candidates ?? []);
|
||||
const pkg = proto?.package ?? '';
|
||||
const cid = d.methodName
|
||||
? contractId(pkg, d.serviceName, d.methodName)
|
||||
: proto
|
||||
? serviceContractId(pkg, d.serviceName)
|
||||
: serviceOnlyContractId(d.serviceName);
|
||||
const confidence = proto ? d.confidenceWithProto : d.confidenceWithoutProto;
|
||||
const meta: Record<string, unknown> = {
|
||||
service: d.serviceName,
|
||||
source: d.source,
|
||||
};
|
||||
|
||||
// @GrpcService
|
||||
if (content.includes('@GrpcService')) {
|
||||
const implBaseRe = /extends\s+(\w+)Grpc\.(\w+)ImplBase/;
|
||||
const m = content.match(implBaseRe);
|
||||
if (m) {
|
||||
const { cid, conf } = resolveJava(m[1]);
|
||||
out.push(
|
||||
makeContract(cid, 'provider', filePath, m[2], conf, {
|
||||
service: m[1],
|
||||
source: 'java_grpc_service',
|
||||
}),
|
||||
);
|
||||
} else {
|
||||
// Try extracting service name from class name
|
||||
const classRe =
|
||||
/class\s+(\w*?)(?:Grpc)?(?:Service)?\s+extends\s+(\w+)(?:Grpc\.(\w+))?ImplBase/;
|
||||
const cm = content.match(classRe);
|
||||
if (cm) {
|
||||
const svcName = cm[2].replace(/Grpc$/, '');
|
||||
const { cid, conf } = resolveJava(svcName);
|
||||
out.push(
|
||||
makeContract(cid, 'provider', filePath, cm[1], conf, {
|
||||
service: svcName,
|
||||
source: 'java_grpc_service',
|
||||
}),
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// extends XxxImplBase (without @GrpcService)
|
||||
if (!content.includes('@GrpcService')) {
|
||||
const implRe = /extends\s+(\w+?)(?:Grpc\.(\w+))?ImplBase/;
|
||||
const m = content.match(implRe);
|
||||
if (m) {
|
||||
const svcName = m[2] || m[1].replace(/Grpc$/, '');
|
||||
const { cid, conf } = resolveJava(svcName);
|
||||
out.push(
|
||||
makeContract(cid, 'provider', filePath, svcName, conf, {
|
||||
service: svcName,
|
||||
source: 'java_impl_base',
|
||||
}),
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
return out;
|
||||
}
|
||||
|
||||
private scanJavaConsumers(
|
||||
content: string,
|
||||
filePath: string,
|
||||
protoMap: Map<string, ProtoServiceInfo[]>,
|
||||
): ExtractedContract[] {
|
||||
const out: ExtractedContract[] = [];
|
||||
// XxxGrpc.newBlockingStub( or XxxGrpc.newStub(
|
||||
const re = /(\w+)Grpc\.new(?:Blocking)?Stub\s*\(/g;
|
||||
let m: RegExpExecArray | null;
|
||||
while ((m = re.exec(content)) !== null) {
|
||||
const serviceName = m[1];
|
||||
const candidates = protoMap.get(serviceName);
|
||||
const proto = resolveProtoConflict(serviceName, filePath, candidates ?? []);
|
||||
const cid = proto
|
||||
? serviceContractId(proto.package, proto.serviceName)
|
||||
: serviceOnlyContractId(serviceName);
|
||||
const conf = proto ? 0.75 : 0.55;
|
||||
out.push(
|
||||
makeContract(cid, 'consumer', filePath, `${serviceName}Stub`, conf, {
|
||||
service: serviceName,
|
||||
source: 'java_stub',
|
||||
}),
|
||||
);
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
private scanPythonProviders(
|
||||
content: string,
|
||||
filePath: string,
|
||||
protoMap: Map<string, ProtoServiceInfo[]>,
|
||||
): ExtractedContract[] {
|
||||
const out: ExtractedContract[] = [];
|
||||
// add_XxxServicer_to_server(
|
||||
const re = /add_(\w+?)Servicer_to_server\s*\(/g;
|
||||
let m: RegExpExecArray | null;
|
||||
while ((m = re.exec(content)) !== null) {
|
||||
const serviceName = m[1];
|
||||
const candidates = protoMap.get(serviceName);
|
||||
const proto = resolveProtoConflict(serviceName, filePath, candidates ?? []);
|
||||
const cid = proto
|
||||
? serviceContractId(proto.package, proto.serviceName)
|
||||
: serviceOnlyContractId(serviceName);
|
||||
const conf = proto ? 0.8 : 0.65;
|
||||
out.push(
|
||||
makeContract(cid, 'provider', filePath, `add_${serviceName}Servicer_to_server`, conf, {
|
||||
service: serviceName,
|
||||
source: 'python_servicer',
|
||||
}),
|
||||
);
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
private scanPythonConsumers(
|
||||
content: string,
|
||||
filePath: string,
|
||||
protoMap: Map<string, ProtoServiceInfo[]>,
|
||||
): ExtractedContract[] {
|
||||
const out: ExtractedContract[] = [];
|
||||
// XxxStub(
|
||||
const re = /(\w+)Stub\s*\(/g;
|
||||
let m: RegExpExecArray | null;
|
||||
while ((m = re.exec(content)) !== null) {
|
||||
const name = m[1];
|
||||
// Filter out common false positives
|
||||
if (['Mock', 'Test', 'Fake', 'Stub'].includes(name)) continue;
|
||||
const candidates = protoMap.get(name);
|
||||
const proto = resolveProtoConflict(name, filePath, candidates ?? []);
|
||||
const cid = proto
|
||||
? serviceContractId(proto.package, proto.serviceName)
|
||||
: serviceOnlyContractId(name);
|
||||
const conf = proto ? 0.75 : 0.55;
|
||||
out.push(
|
||||
makeContract(cid, 'consumer', filePath, `${name}Stub`, conf, {
|
||||
service: name,
|
||||
source: 'python_stub',
|
||||
}),
|
||||
);
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
private scanTsProviders(
|
||||
content: string,
|
||||
filePath: string,
|
||||
protoMap: Map<string, ProtoServiceInfo[]>,
|
||||
): ExtractedContract[] {
|
||||
const out: ExtractedContract[] = [];
|
||||
// @GrpcMethod('ServiceName', 'MethodName')
|
||||
const re = /@GrpcMethod\s*\(\s*['"](\w+)['"]\s*,\s*['"](\w+)['"]\s*\)/g;
|
||||
let m: RegExpExecArray | null;
|
||||
while ((m = re.exec(content)) !== null) {
|
||||
const serviceName = m[1];
|
||||
const methodName = m[2];
|
||||
const candidates = protoMap.get(serviceName);
|
||||
const proto = resolveProtoConflict(serviceName, filePath, candidates ?? []);
|
||||
const pkg = proto?.package ?? '';
|
||||
const cid = contractId(pkg, serviceName, methodName);
|
||||
out.push(
|
||||
makeContract(cid, 'provider', filePath, `${serviceName}.${methodName}`, 0.8, {
|
||||
service: serviceName,
|
||||
method: methodName,
|
||||
source: 'ts_grpc_method',
|
||||
}),
|
||||
);
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
private scanTsConsumers(
|
||||
content: string,
|
||||
filePath: string,
|
||||
protoMap: Map<string, ProtoServiceInfo[]>,
|
||||
): ExtractedContract[] {
|
||||
const out: ExtractedContract[] = [];
|
||||
const pushConsumer = (
|
||||
serviceName: string,
|
||||
symbolName: string,
|
||||
source: string,
|
||||
confidenceWithProto = 0.75,
|
||||
confidenceWithoutProto = 0.55,
|
||||
): void => {
|
||||
const candidates = protoMap.get(serviceName);
|
||||
const proto = resolveProtoConflict(serviceName, filePath, candidates ?? []);
|
||||
const cid = proto
|
||||
? serviceContractId(proto.package, proto.serviceName)
|
||||
: serviceOnlyContractId(serviceName);
|
||||
const conf = proto ? confidenceWithProto : confidenceWithoutProto;
|
||||
out.push(
|
||||
makeContract(cid, 'consumer', filePath, symbolName, conf, {
|
||||
service: serviceName,
|
||||
source,
|
||||
}),
|
||||
);
|
||||
};
|
||||
|
||||
const grpcClientDecoratorRe =
|
||||
/@GrpcClient\s*\([^)]*\)\s*(?:private|protected|public)?\s*(?:readonly\s+)?\w+[!?]?\s*:\s*(\w+Service)Client\b/g;
|
||||
let match: RegExpExecArray | null;
|
||||
while ((match = grpcClientDecoratorRe.exec(content)) !== null) {
|
||||
pushConsumer(match[1], `${match[1]}Client`, 'ts_grpc_client_decorator');
|
||||
}
|
||||
|
||||
const getServiceRe = /\.getService(?:<[^>]+>)?\s*\(\s*['"](\w+)['"]\s*\)/g;
|
||||
while ((match = getServiceRe.exec(content)) !== null) {
|
||||
pushConsumer(match[1], `${match[1]}Client`, 'ts_client_grpc_get_service');
|
||||
}
|
||||
|
||||
const clientCtorRe = /new\s+(\w+Service)Client\s*\(/g;
|
||||
while ((match = clientCtorRe.exec(content)) !== null) {
|
||||
pushConsumer(match[1], `${match[1]}Client`, 'ts_generated_client');
|
||||
}
|
||||
|
||||
if (content.includes('loadPackageDefinition')) {
|
||||
const packageCtorRe = /new\s+[\w$.]*\.([A-Z]\w+)\s*\(/g;
|
||||
while ((match = packageCtorRe.exec(content)) !== null) {
|
||||
pushConsumer(match[1], `${match[1]}Client`, 'ts_load_package_definition');
|
||||
}
|
||||
}
|
||||
|
||||
return out;
|
||||
if (d.methodName) meta.method = d.methodName;
|
||||
return makeContract(cid, d.role, filePath, d.symbolName, confidence, meta);
|
||||
}
|
||||
|
||||
private dedupe(items: ExtractedContract[]): ExtractedContract[] {
|
||||
|
|
|
|||
109
gitnexus/src/core/group/extractors/grpc-patterns/go.ts
Normal file
109
gitnexus/src/core/group/extractors/grpc-patterns/go.ts
Normal file
|
|
@ -0,0 +1,109 @@
|
|||
import Go from 'tree-sitter-go';
|
||||
import {
|
||||
compilePatterns,
|
||||
runCompiledPatterns,
|
||||
type LanguagePatterns,
|
||||
} from '../tree-sitter-scanner.js';
|
||||
import type { GrpcDetection, GrpcLanguagePlugin } from './types.js';
|
||||
|
||||
/**
|
||||
* Go gRPC plugin. Detects:
|
||||
* - Provider: `pb.RegisterXxxServer(...)` calls
|
||||
* - Provider: `pb.UnimplementedXxxServer` embedded in a struct
|
||||
* - Consumer: `pb.NewXxxClient(conn)` calls
|
||||
*/
|
||||
|
||||
const REGISTER_RE = /^Register(\w+)Server$/;
|
||||
const UNIMPLEMENTED_RE = /^Unimplemented(\w+)Server$/;
|
||||
const NEW_CLIENT_RE = /^New(\w+)Client$/;
|
||||
|
||||
// Any `xxx.<fn>(...)` call — plugin filters the field identifier text.
|
||||
const SELECTOR_CALL_PATTERNS = compilePatterns({
|
||||
name: 'go-grpc-selector-call',
|
||||
language: Go,
|
||||
patterns: [
|
||||
{
|
||||
meta: {},
|
||||
query: `
|
||||
(call_expression
|
||||
function: (selector_expression
|
||||
field: (field_identifier) @fn))
|
||||
`,
|
||||
},
|
||||
],
|
||||
} satisfies LanguagePatterns<Record<string, never>>);
|
||||
|
||||
// Any `qualified_type` used as a struct field — for `pb.UnimplementedXxxServer`.
|
||||
const STRUCT_EMBEDDING_PATTERNS = compilePatterns({
|
||||
name: 'go-grpc-struct-embedding',
|
||||
language: Go,
|
||||
patterns: [
|
||||
{
|
||||
meta: {},
|
||||
query: `
|
||||
(struct_type
|
||||
(field_declaration_list
|
||||
(field_declaration
|
||||
type: (qualified_type
|
||||
name: (type_identifier) @field_type))))
|
||||
`,
|
||||
},
|
||||
],
|
||||
} satisfies LanguagePatterns<Record<string, never>>);
|
||||
|
||||
export const GO_GRPC_PLUGIN: GrpcLanguagePlugin = {
|
||||
name: 'go-grpc',
|
||||
language: Go,
|
||||
scan(tree) {
|
||||
const out: GrpcDetection[] = [];
|
||||
|
||||
for (const match of runCompiledPatterns(SELECTOR_CALL_PATTERNS, tree)) {
|
||||
const fnNode = match.captures.fn;
|
||||
if (!fnNode) continue;
|
||||
const fnText = fnNode.text;
|
||||
|
||||
const registerMatch = REGISTER_RE.exec(fnText);
|
||||
if (registerMatch) {
|
||||
out.push({
|
||||
role: 'provider',
|
||||
serviceName: registerMatch[1],
|
||||
symbolName: fnText,
|
||||
source: 'go_register',
|
||||
confidenceWithProto: 0.8,
|
||||
confidenceWithoutProto: 0.65,
|
||||
});
|
||||
continue;
|
||||
}
|
||||
|
||||
const newClientMatch = NEW_CLIENT_RE.exec(fnText);
|
||||
if (newClientMatch) {
|
||||
out.push({
|
||||
role: 'consumer',
|
||||
serviceName: newClientMatch[1],
|
||||
symbolName: fnText,
|
||||
source: 'go_client',
|
||||
confidenceWithProto: 0.75,
|
||||
confidenceWithoutProto: 0.55,
|
||||
});
|
||||
continue;
|
||||
}
|
||||
}
|
||||
|
||||
for (const match of runCompiledPatterns(STRUCT_EMBEDDING_PATTERNS, tree)) {
|
||||
const fieldNode = match.captures.field_type;
|
||||
if (!fieldNode) continue;
|
||||
const unimpl = UNIMPLEMENTED_RE.exec(fieldNode.text);
|
||||
if (!unimpl) continue;
|
||||
out.push({
|
||||
role: 'provider',
|
||||
serviceName: unimpl[1],
|
||||
symbolName: fieldNode.text,
|
||||
source: 'go_unimplemented',
|
||||
confidenceWithProto: 0.8,
|
||||
confidenceWithoutProto: 0.65,
|
||||
});
|
||||
}
|
||||
|
||||
return out;
|
||||
},
|
||||
};
|
||||
43
gitnexus/src/core/group/extractors/grpc-patterns/index.ts
Normal file
43
gitnexus/src/core/group/extractors/grpc-patterns/index.ts
Normal file
|
|
@ -0,0 +1,43 @@
|
|||
import * as path from 'node:path';
|
||||
import type { GrpcLanguagePlugin } from './types.js';
|
||||
import { GO_GRPC_PLUGIN } from './go.js';
|
||||
import { JAVA_GRPC_PLUGIN } from './java.js';
|
||||
import { PYTHON_GRPC_PLUGIN } from './python.js';
|
||||
import { JAVASCRIPT_GRPC_PLUGIN, TYPESCRIPT_GRPC_PLUGIN, TSX_GRPC_PLUGIN } from './node.js';
|
||||
|
||||
export type { GrpcDetection, GrpcLanguagePlugin, GrpcRole } from './types.js';
|
||||
|
||||
/**
|
||||
* File-extension → gRPC language plugin registry. Mirrors the shape
|
||||
* of `http-patterns/index.ts` and `topic-patterns/index.ts`.
|
||||
*
|
||||
* To add a new language drop a `grpc-patterns/<lang>.ts` exporting a
|
||||
* `GrpcLanguagePlugin`, import + register it here, and widen
|
||||
* `GRPC_SCAN_GLOB` if needed. No edits to `grpc-extractor.ts` required.
|
||||
*/
|
||||
const REGISTRY: Record<string, GrpcLanguagePlugin> = {
|
||||
'.go': GO_GRPC_PLUGIN,
|
||||
'.java': JAVA_GRPC_PLUGIN,
|
||||
'.py': PYTHON_GRPC_PLUGIN,
|
||||
'.js': JAVASCRIPT_GRPC_PLUGIN,
|
||||
'.jsx': JAVASCRIPT_GRPC_PLUGIN,
|
||||
'.ts': TYPESCRIPT_GRPC_PLUGIN,
|
||||
'.tsx': TSX_GRPC_PLUGIN,
|
||||
};
|
||||
|
||||
/**
|
||||
* Glob for source files worth scanning for gRPC server/client patterns.
|
||||
* `.proto` files are handled directly by the orchestrator's in-tree
|
||||
* string-sanitizing parser (no `tree-sitter-proto` grammar is
|
||||
* installed).
|
||||
*/
|
||||
export const GRPC_SCAN_GLOB = '**/*.{go,java,py,ts,tsx,js,jsx}';
|
||||
|
||||
/**
|
||||
* Return the gRPC plugin registered for the given file's extension,
|
||||
* or `undefined` if the extension is not registered.
|
||||
*/
|
||||
export function getPluginForFile(rel: string): GrpcLanguagePlugin | undefined {
|
||||
const ext = path.extname(rel).toLowerCase();
|
||||
return REGISTRY[ext];
|
||||
}
|
||||
179
gitnexus/src/core/group/extractors/grpc-patterns/java.ts
Normal file
179
gitnexus/src/core/group/extractors/grpc-patterns/java.ts
Normal file
|
|
@ -0,0 +1,179 @@
|
|||
import Parser from 'tree-sitter';
|
||||
import Java from 'tree-sitter-java';
|
||||
import {
|
||||
compilePatterns,
|
||||
runCompiledPatterns,
|
||||
type LanguagePatterns,
|
||||
} from '../tree-sitter-scanner.js';
|
||||
import type { GrpcDetection, GrpcLanguagePlugin } from './types.js';
|
||||
|
||||
/**
|
||||
* Java gRPC plugin. Detects:
|
||||
* - Provider: classes extending `XxxServiceGrpc.XxxServiceImplBase`
|
||||
* (with or without a `@GrpcService` annotation; the annotation
|
||||
* only affects confidence labelling in the original regex version
|
||||
* — here we emit a single detection per class and pick the source
|
||||
* label based on whether the annotation is present).
|
||||
* - Consumer: `XxxServiceGrpc.newBlockingStub(ch)` /
|
||||
* `XxxServiceGrpc.newStub(ch)` calls.
|
||||
*/
|
||||
|
||||
const IMPL_BASE_RE = /^(\w+)ImplBase$/;
|
||||
const GRPC_SUFFIX_RE = /^(\w+)Grpc$/;
|
||||
|
||||
// Classes extending `ScopedType.ScopedType` where the inner name ends
|
||||
// in ImplBase. Covers `XxxServiceGrpc.XxxServiceImplBase`.
|
||||
// Note: tree-sitter-java's `scoped_type_identifier` exposes its two
|
||||
// segments as positional `type_identifier` children, NOT as named
|
||||
// `scope:`/`name:` fields. We match positionally here and rely on the
|
||||
// grammar's left-to-right ordering: first child = outer, second = inner.
|
||||
const SCOPED_IMPL_BASE_PATTERNS = compilePatterns({
|
||||
name: 'java-grpc-scoped-impl-base',
|
||||
language: Java,
|
||||
patterns: [
|
||||
{
|
||||
meta: {},
|
||||
query: `
|
||||
(class_declaration
|
||||
name: (identifier) @class_name
|
||||
superclass: (superclass
|
||||
(scoped_type_identifier
|
||||
(type_identifier) @outer
|
||||
(type_identifier) @inner (#match? @inner "ImplBase$")))) @class
|
||||
`,
|
||||
},
|
||||
],
|
||||
} satisfies LanguagePatterns<Record<string, never>>);
|
||||
|
||||
// Classes extending a simple `XxxImplBase` identifier (no scope).
|
||||
const PLAIN_IMPL_BASE_PATTERNS = compilePatterns({
|
||||
name: 'java-grpc-plain-impl-base',
|
||||
language: Java,
|
||||
patterns: [
|
||||
{
|
||||
meta: {},
|
||||
query: `
|
||||
(class_declaration
|
||||
name: (identifier) @class_name
|
||||
superclass: (superclass
|
||||
(type_identifier) @plain_type (#match? @plain_type "ImplBase$"))) @class
|
||||
`,
|
||||
},
|
||||
],
|
||||
} satisfies LanguagePatterns<Record<string, never>>);
|
||||
|
||||
// gRPC stub factories: `XxxGrpc.newStub(ch)` / `XxxGrpc.newBlockingStub(ch)`.
|
||||
const STUB_PATTERNS = compilePatterns({
|
||||
name: 'java-grpc-stub',
|
||||
language: Java,
|
||||
patterns: [
|
||||
{
|
||||
meta: {},
|
||||
query: `
|
||||
(method_invocation
|
||||
object: (identifier) @grpc_cls
|
||||
name: (identifier) @method (#match? @method "^new(Blocking)?Stub$"))
|
||||
`,
|
||||
},
|
||||
],
|
||||
} satisfies LanguagePatterns<Record<string, never>>);
|
||||
|
||||
/**
|
||||
* Check whether a `class_declaration` node has a `@GrpcService`
|
||||
* annotation in its modifiers list. In tree-sitter-java, class-level
|
||||
* annotations live under `(class_declaration (modifiers (marker_annotation|annotation)))`.
|
||||
*/
|
||||
function hasGrpcServiceAnnotation(classNode: Parser.SyntaxNode): boolean {
|
||||
for (let i = 0; i < classNode.namedChildCount; i++) {
|
||||
const child = classNode.namedChild(i);
|
||||
if (!child || child.type !== 'modifiers') continue;
|
||||
for (let j = 0; j < child.namedChildCount; j++) {
|
||||
const mod = child.namedChild(j);
|
||||
if (!mod) continue;
|
||||
if (mod.type !== 'marker_annotation' && mod.type !== 'annotation') continue;
|
||||
const nameNode = mod.childForFieldName('name');
|
||||
if (nameNode?.text === 'GrpcService') return true;
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
/**
|
||||
* Given the inner type_identifier text like `AuthServiceImplBase`,
|
||||
* return the service name (`AuthService`), or null if the text
|
||||
* doesn't end in `ImplBase`.
|
||||
*/
|
||||
function extractServiceFromImplBase(text: string): string | null {
|
||||
const m = IMPL_BASE_RE.exec(text);
|
||||
if (!m) return null;
|
||||
// Strip a trailing `Grpc` on the service name too — the original
|
||||
// regex replaces `Grpc$` on the extracted prefix.
|
||||
return m[1].replace(/Grpc$/, '');
|
||||
}
|
||||
|
||||
export const JAVA_GRPC_PLUGIN: GrpcLanguagePlugin = {
|
||||
name: 'java-grpc',
|
||||
language: Java,
|
||||
scan(tree) {
|
||||
const out: GrpcDetection[] = [];
|
||||
const emittedClassIds = new Set<number>();
|
||||
|
||||
// ─── Providers: scoped form (`...Grpc.XxxImplBase`) ─────────────
|
||||
for (const match of runCompiledPatterns(SCOPED_IMPL_BASE_PATTERNS, tree)) {
|
||||
const classNode = match.captures.class;
|
||||
const innerNode = match.captures.inner;
|
||||
if (!classNode || !innerNode) continue;
|
||||
const serviceName = extractServiceFromImplBase(innerNode.text);
|
||||
if (!serviceName) continue;
|
||||
emittedClassIds.add(classNode.id);
|
||||
const annotated = hasGrpcServiceAnnotation(classNode);
|
||||
out.push({
|
||||
role: 'provider',
|
||||
serviceName,
|
||||
symbolName: serviceName,
|
||||
source: annotated ? 'java_grpc_service' : 'java_impl_base',
|
||||
confidenceWithProto: 0.8,
|
||||
confidenceWithoutProto: 0.65,
|
||||
});
|
||||
}
|
||||
|
||||
// ─── Providers: plain form (`XxxImplBase`) ──────────────────────
|
||||
for (const match of runCompiledPatterns(PLAIN_IMPL_BASE_PATTERNS, tree)) {
|
||||
const classNode = match.captures.class;
|
||||
const plainNode = match.captures.plain_type;
|
||||
if (!classNode || !plainNode) continue;
|
||||
if (emittedClassIds.has(classNode.id)) continue;
|
||||
const serviceName = extractServiceFromImplBase(plainNode.text);
|
||||
if (!serviceName) continue;
|
||||
emittedClassIds.add(classNode.id);
|
||||
const annotated = hasGrpcServiceAnnotation(classNode);
|
||||
out.push({
|
||||
role: 'provider',
|
||||
serviceName,
|
||||
symbolName: serviceName,
|
||||
source: annotated ? 'java_grpc_service' : 'java_impl_base',
|
||||
confidenceWithProto: 0.8,
|
||||
confidenceWithoutProto: 0.65,
|
||||
});
|
||||
}
|
||||
|
||||
// ─── Consumers: `XxxGrpc.newBlockingStub(...)` / `newStub(...)` ─
|
||||
for (const match of runCompiledPatterns(STUB_PATTERNS, tree)) {
|
||||
const grpcClsNode = match.captures.grpc_cls;
|
||||
if (!grpcClsNode) continue;
|
||||
const grpcMatch = GRPC_SUFFIX_RE.exec(grpcClsNode.text);
|
||||
if (!grpcMatch) continue;
|
||||
const serviceName = grpcMatch[1];
|
||||
out.push({
|
||||
role: 'consumer',
|
||||
serviceName,
|
||||
symbolName: `${serviceName}Stub`,
|
||||
source: 'java_stub',
|
||||
confidenceWithProto: 0.75,
|
||||
confidenceWithoutProto: 0.55,
|
||||
});
|
||||
}
|
||||
|
||||
return out;
|
||||
},
|
||||
};
|
||||
295
gitnexus/src/core/group/extractors/grpc-patterns/node.ts
Normal file
295
gitnexus/src/core/group/extractors/grpc-patterns/node.ts
Normal file
|
|
@ -0,0 +1,295 @@
|
|||
import Parser from 'tree-sitter';
|
||||
import JavaScript from 'tree-sitter-javascript';
|
||||
import TypeScript from 'tree-sitter-typescript';
|
||||
import {
|
||||
compilePatterns,
|
||||
runCompiledPatterns,
|
||||
unquoteLiteral,
|
||||
type CompiledPatterns,
|
||||
type LanguagePatterns,
|
||||
type PatternSpec,
|
||||
} from '../tree-sitter-scanner.js';
|
||||
import type { GrpcDetection, GrpcLanguagePlugin } from './types.js';
|
||||
|
||||
/**
|
||||
* Node.js / TypeScript gRPC plugin family. Detects:
|
||||
* - Provider: NestJS `@GrpcMethod('Service', 'Method')` decorators
|
||||
* - Consumer: NestJS `@GrpcClient(...) readonly x!: XxxServiceClient`
|
||||
* - Consumer: `client.getService<X>('AuthService')`
|
||||
* - Consumer: `new XxxServiceClient(...)` (generated client constructor)
|
||||
* - Consumer: `new foo.bar.Xxx(...)` when the file uses
|
||||
* `loadPackageDefinition` (gRPC dynamic proto loader)
|
||||
*
|
||||
* As with the HTTP `node.ts`, pattern sources are defined once and
|
||||
* compiled against three grammar variants (JS / TS / TSX) because
|
||||
* `Parser.Query` is not portable across grammar objects.
|
||||
*/
|
||||
|
||||
const SERVICE_CLIENT_RE = /^(\w+Service)Client$/;
|
||||
const CAPITALIZED_SERVICE_RE = /^[A-Z]\w+$/;
|
||||
|
||||
// @GrpcMethod('Service', 'Method')
|
||||
const GRPC_METHOD_SPEC: PatternSpec<Record<string, never>> = {
|
||||
meta: {},
|
||||
query: `
|
||||
(decorator
|
||||
(call_expression
|
||||
function: (identifier) @dec (#eq? @dec "GrpcMethod")
|
||||
arguments: (arguments
|
||||
. [(string) (template_string)] @service
|
||||
. [(string) (template_string)] @method)))
|
||||
`,
|
||||
};
|
||||
|
||||
// @GrpcClient(...) standalone decorator — the plugin walks to the next
|
||||
// sibling (a field definition) to read its type annotation.
|
||||
const GRPC_CLIENT_SPEC: PatternSpec<Record<string, never>> = {
|
||||
meta: {},
|
||||
query: `
|
||||
(decorator
|
||||
(call_expression
|
||||
function: (identifier) @dec (#eq? @dec "GrpcClient"))) @grpc_client_decorator
|
||||
`,
|
||||
};
|
||||
|
||||
// `.getService<X>('AuthService')` / `.getService('AuthService')`
|
||||
const GET_SERVICE_SPEC: PatternSpec<Record<string, never>> = {
|
||||
meta: {},
|
||||
query: `
|
||||
(call_expression
|
||||
function: (member_expression
|
||||
property: (property_identifier) @method (#eq? @method "getService"))
|
||||
arguments: (arguments . [(string) (template_string)] @service))
|
||||
`,
|
||||
};
|
||||
|
||||
// `new XxxServiceClient(...)` — bare identifier constructor.
|
||||
const NEW_SIMPLE_CTOR_SPEC: PatternSpec<Record<string, never>> = {
|
||||
meta: {},
|
||||
query: `
|
||||
(new_expression
|
||||
constructor: (identifier) @ctor)
|
||||
`,
|
||||
};
|
||||
|
||||
// `new foo.bar.XxxService(...)` — qualified constructor.
|
||||
const NEW_QUALIFIED_CTOR_SPEC: PatternSpec<Record<string, never>> = {
|
||||
meta: {},
|
||||
query: `
|
||||
(new_expression
|
||||
constructor: (member_expression
|
||||
property: (property_identifier) @ctor))
|
||||
`,
|
||||
};
|
||||
|
||||
interface NodeGrpcPatternBundle {
|
||||
grpcMethod: CompiledPatterns<Record<string, never>>;
|
||||
grpcClient: CompiledPatterns<Record<string, never>>;
|
||||
getService: CompiledPatterns<Record<string, never>>;
|
||||
newSimpleCtor: CompiledPatterns<Record<string, never>>;
|
||||
newQualifiedCtor: CompiledPatterns<Record<string, never>>;
|
||||
}
|
||||
|
||||
function compileBundle(language: unknown, name: string): NodeGrpcPatternBundle {
|
||||
const mk = (spec: PatternSpec<Record<string, never>>, suffix: string) =>
|
||||
compilePatterns({
|
||||
name: `${name}-${suffix}`,
|
||||
language,
|
||||
patterns: [spec],
|
||||
} satisfies LanguagePatterns<Record<string, never>>);
|
||||
return {
|
||||
grpcMethod: mk(GRPC_METHOD_SPEC, 'grpc-method'),
|
||||
grpcClient: mk(GRPC_CLIENT_SPEC, 'grpc-client'),
|
||||
getService: mk(GET_SERVICE_SPEC, 'get-service'),
|
||||
newSimpleCtor: mk(NEW_SIMPLE_CTOR_SPEC, 'new-simple-ctor'),
|
||||
newQualifiedCtor: mk(NEW_QUALIFIED_CTOR_SPEC, 'new-qualified-ctor'),
|
||||
};
|
||||
}
|
||||
|
||||
const JAVASCRIPT_BUNDLE = compileBundle(JavaScript, 'javascript-grpc');
|
||||
const TYPESCRIPT_BUNDLE = compileBundle(TypeScript.typescript, 'typescript-grpc');
|
||||
const TSX_BUNDLE = compileBundle(TypeScript.tsx, 'tsx-grpc');
|
||||
|
||||
/**
|
||||
* Given a `@GrpcClient(...)` decorator node, find the type annotation
|
||||
* text of the field it decorates (e.g. `AuthServiceClient`).
|
||||
*
|
||||
* In tree-sitter-typescript, decorators on class fields can appear in
|
||||
* two configurations:
|
||||
* - As a CHILD of `public_field_definition` alongside the field's
|
||||
* type annotation (the common case for NestJS `@GrpcClient`).
|
||||
* - As a SIBLING of the field in `class_body` (for method
|
||||
* decorators, but kept for resilience against grammar variants).
|
||||
* We walk the parent container and search for a type annotation.
|
||||
*/
|
||||
function resolveGrpcClientFieldType(decoratorNode: Parser.SyntaxNode): string | null {
|
||||
const parent = decoratorNode.parent;
|
||||
if (!parent) return null;
|
||||
|
||||
// Case 1: decorator is a child of the field definition — search
|
||||
// the parent itself (which is the field definition) for a
|
||||
// type_annotation child.
|
||||
if (parent.type === 'public_field_definition' || parent.type.endsWith('field_definition')) {
|
||||
return findFirstTypeAnnotationText(parent);
|
||||
}
|
||||
|
||||
// Case 2: decorator is a sibling of the field in a class_body — walk
|
||||
// forward through subsequent siblings until we find a node containing
|
||||
// a type annotation.
|
||||
for (let i = 0; i < parent.namedChildCount; i++) {
|
||||
const child = parent.namedChild(i);
|
||||
if (child && child.id === decoratorNode.id) {
|
||||
for (let j = i + 1; j < parent.namedChildCount; j++) {
|
||||
const next = parent.namedChild(j);
|
||||
if (!next) continue;
|
||||
if (next.type === 'decorator') continue;
|
||||
const typeText = findFirstTypeAnnotationText(next);
|
||||
if (typeText) return typeText;
|
||||
return null;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Recursively search `node` for the first `type_annotation` child and
|
||||
* return the text of its inner `type_identifier`, or null. Handles
|
||||
* both `public_field_definition` and its variants.
|
||||
*/
|
||||
function findFirstTypeAnnotationText(node: Parser.SyntaxNode): string | null {
|
||||
if (node.type === 'type_annotation') {
|
||||
for (let i = 0; i < node.namedChildCount; i++) {
|
||||
const child = node.namedChild(i);
|
||||
if (!child) continue;
|
||||
if (child.type === 'type_identifier') return child.text;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
for (let i = 0; i < node.namedChildCount; i++) {
|
||||
const child = node.namedChild(i);
|
||||
if (!child) continue;
|
||||
const found = findFirstTypeAnnotationText(child);
|
||||
if (found) return found;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
function scanBundle(bundle: NodeGrpcPatternBundle, tree: Parser.Tree): GrpcDetection[] {
|
||||
const out: GrpcDetection[] = [];
|
||||
|
||||
// ─── Provider: @GrpcMethod('Service', 'Method') ──────────────────
|
||||
for (const match of runCompiledPatterns(bundle.grpcMethod, tree)) {
|
||||
const svcNode = match.captures.service;
|
||||
const methodNode = match.captures.method;
|
||||
if (!svcNode || !methodNode) continue;
|
||||
const svc = unquoteLiteral(svcNode.text);
|
||||
const mth = unquoteLiteral(methodNode.text);
|
||||
if (!svc || !mth) continue;
|
||||
out.push({
|
||||
role: 'provider',
|
||||
serviceName: svc,
|
||||
symbolName: `${svc}.${mth}`,
|
||||
source: 'ts_grpc_method',
|
||||
methodName: mth,
|
||||
// @GrpcMethod hard-coded confidence 0.8 in the original code
|
||||
// regardless of whether the proto map has a match.
|
||||
confidenceWithProto: 0.8,
|
||||
confidenceWithoutProto: 0.8,
|
||||
});
|
||||
}
|
||||
|
||||
// ─── Consumer: @GrpcClient() field with XxxServiceClient type ────
|
||||
for (const match of runCompiledPatterns(bundle.grpcClient, tree)) {
|
||||
const decoratorNode = match.captures.grpc_client_decorator;
|
||||
if (!decoratorNode) continue;
|
||||
const typeText = resolveGrpcClientFieldType(decoratorNode);
|
||||
if (!typeText) continue;
|
||||
const svcMatch = SERVICE_CLIENT_RE.exec(typeText);
|
||||
if (!svcMatch) continue;
|
||||
const serviceName = svcMatch[1];
|
||||
out.push({
|
||||
role: 'consumer',
|
||||
serviceName,
|
||||
symbolName: `${serviceName}Client`,
|
||||
source: 'ts_grpc_client_decorator',
|
||||
confidenceWithProto: 0.75,
|
||||
confidenceWithoutProto: 0.55,
|
||||
});
|
||||
}
|
||||
|
||||
// ─── Consumer: client.getService<X>('Service') ───────────────────
|
||||
for (const match of runCompiledPatterns(bundle.getService, tree)) {
|
||||
const svcNode = match.captures.service;
|
||||
if (!svcNode) continue;
|
||||
const svc = unquoteLiteral(svcNode.text);
|
||||
if (!svc) continue;
|
||||
out.push({
|
||||
role: 'consumer',
|
||||
serviceName: svc,
|
||||
symbolName: `${svc}Client`,
|
||||
source: 'ts_client_grpc_get_service',
|
||||
confidenceWithProto: 0.75,
|
||||
confidenceWithoutProto: 0.55,
|
||||
});
|
||||
}
|
||||
|
||||
// ─── Consumer: new XxxServiceClient(...) ─────────────────────────
|
||||
for (const match of runCompiledPatterns(bundle.newSimpleCtor, tree)) {
|
||||
const ctorNode = match.captures.ctor;
|
||||
if (!ctorNode) continue;
|
||||
const svcMatch = SERVICE_CLIENT_RE.exec(ctorNode.text);
|
||||
if (!svcMatch) continue;
|
||||
const serviceName = svcMatch[1];
|
||||
out.push({
|
||||
role: 'consumer',
|
||||
serviceName,
|
||||
symbolName: `${serviceName}Client`,
|
||||
source: 'ts_generated_client',
|
||||
confidenceWithProto: 0.75,
|
||||
confidenceWithoutProto: 0.55,
|
||||
});
|
||||
}
|
||||
|
||||
// ─── Consumer: loadPackageDefinition dynamic proto loader ────────
|
||||
// Only emit when the file uses loadPackageDefinition, otherwise a
|
||||
// generic `new foo.bar.Something()` in unrelated code would falsely
|
||||
// register as a gRPC consumer.
|
||||
const usesLoadPackage = tree.rootNode.text.includes('loadPackageDefinition');
|
||||
if (usesLoadPackage) {
|
||||
for (const match of runCompiledPatterns(bundle.newQualifiedCtor, tree)) {
|
||||
const ctorNode = match.captures.ctor;
|
||||
if (!ctorNode) continue;
|
||||
if (!CAPITALIZED_SERVICE_RE.test(ctorNode.text)) continue;
|
||||
out.push({
|
||||
role: 'consumer',
|
||||
serviceName: ctorNode.text,
|
||||
symbolName: `${ctorNode.text}Client`,
|
||||
source: 'ts_load_package_definition',
|
||||
confidenceWithProto: 0.75,
|
||||
confidenceWithoutProto: 0.55,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
return out;
|
||||
}
|
||||
|
||||
export const JAVASCRIPT_GRPC_PLUGIN: GrpcLanguagePlugin = {
|
||||
name: 'javascript-grpc',
|
||||
language: JavaScript,
|
||||
scan: (tree) => scanBundle(JAVASCRIPT_BUNDLE, tree),
|
||||
};
|
||||
|
||||
export const TYPESCRIPT_GRPC_PLUGIN: GrpcLanguagePlugin = {
|
||||
name: 'typescript-grpc',
|
||||
language: TypeScript.typescript,
|
||||
scan: (tree) => scanBundle(TYPESCRIPT_BUNDLE, tree),
|
||||
};
|
||||
|
||||
export const TSX_GRPC_PLUGIN: GrpcLanguagePlugin = {
|
||||
name: 'tsx-grpc',
|
||||
language: TypeScript.tsx,
|
||||
scan: (tree) => scanBundle(TSX_BUNDLE, tree),
|
||||
};
|
||||
77
gitnexus/src/core/group/extractors/grpc-patterns/python.ts
Normal file
77
gitnexus/src/core/group/extractors/grpc-patterns/python.ts
Normal file
|
|
@ -0,0 +1,77 @@
|
|||
import Python from 'tree-sitter-python';
|
||||
import {
|
||||
compilePatterns,
|
||||
runCompiledPatterns,
|
||||
type LanguagePatterns,
|
||||
} from '../tree-sitter-scanner.js';
|
||||
import type { GrpcDetection, GrpcLanguagePlugin } from './types.js';
|
||||
|
||||
/**
|
||||
* Python gRPC plugin. Detects:
|
||||
* - Provider: `add_XxxServicer_to_server(...)` calls (bare identifier
|
||||
* or qualified attribute form `auth_pb2_grpc.add_XxxServicer_to_server`)
|
||||
* - Consumer: `XxxStub(channel)` calls (bare or `auth_pb2_grpc.XxxStub`)
|
||||
*/
|
||||
|
||||
const ADD_SERVICER_RE = /^add_(\w+)Servicer_to_server$/;
|
||||
const STUB_RE = /^(\w+)Stub$/;
|
||||
/** Reserved names that would produce garbage service names. */
|
||||
const STUB_IGNORE = new Set(['Mock', 'Test', 'Fake', 'Stub']);
|
||||
|
||||
// Any call whose target is either a bare identifier or an attribute
|
||||
// access (`obj.method`). The plugin filters the function name in JS.
|
||||
const CALL_PATTERNS = compilePatterns({
|
||||
name: 'python-grpc-call',
|
||||
language: Python,
|
||||
patterns: [
|
||||
{
|
||||
meta: {},
|
||||
query: `
|
||||
(call
|
||||
function: [
|
||||
(identifier) @fn
|
||||
(attribute attribute: (identifier) @fn)
|
||||
])
|
||||
`,
|
||||
},
|
||||
],
|
||||
} satisfies LanguagePatterns<Record<string, never>>);
|
||||
|
||||
export const PYTHON_GRPC_PLUGIN: GrpcLanguagePlugin = {
|
||||
name: 'python-grpc',
|
||||
language: Python,
|
||||
scan(tree) {
|
||||
const out: GrpcDetection[] = [];
|
||||
for (const match of runCompiledPatterns(CALL_PATTERNS, tree)) {
|
||||
const fnNode = match.captures.fn;
|
||||
if (!fnNode) continue;
|
||||
const fnText = fnNode.text;
|
||||
|
||||
const addServicer = ADD_SERVICER_RE.exec(fnText);
|
||||
if (addServicer) {
|
||||
out.push({
|
||||
role: 'provider',
|
||||
serviceName: addServicer[1],
|
||||
symbolName: fnText,
|
||||
source: 'python_servicer',
|
||||
confidenceWithProto: 0.8,
|
||||
confidenceWithoutProto: 0.65,
|
||||
});
|
||||
continue;
|
||||
}
|
||||
|
||||
const stubMatch = STUB_RE.exec(fnText);
|
||||
if (stubMatch && !STUB_IGNORE.has(stubMatch[1])) {
|
||||
out.push({
|
||||
role: 'consumer',
|
||||
serviceName: stubMatch[1],
|
||||
symbolName: fnText,
|
||||
source: 'python_stub',
|
||||
confidenceWithProto: 0.75,
|
||||
confidenceWithoutProto: 0.55,
|
||||
});
|
||||
}
|
||||
}
|
||||
return out;
|
||||
},
|
||||
};
|
||||
54
gitnexus/src/core/group/extractors/grpc-patterns/types.ts
Normal file
54
gitnexus/src/core/group/extractors/grpc-patterns/types.ts
Normal file
|
|
@ -0,0 +1,54 @@
|
|||
import type Parser from 'tree-sitter';
|
||||
|
||||
/**
|
||||
* Shared types for the grpc-extractor language plugins.
|
||||
*
|
||||
* Each plugin lives in its own file (java.ts, go.ts, ...) and owns the
|
||||
* tree-sitter grammar import + query sources. The top-level
|
||||
* `grpc-extractor.ts` orchestrator only knows about this type module
|
||||
* and the plugin registry (`./index.ts`). It MUST NOT import any
|
||||
* grammar or query text directly.
|
||||
*/
|
||||
|
||||
export type GrpcRole = 'provider' | 'consumer';
|
||||
|
||||
/**
|
||||
* One raw gRPC detection produced by a plugin's `scan()` function. The
|
||||
* orchestrator uses the proto map to resolve the full package-qualified
|
||||
* contract id and choose a confidence based on whether the proto was
|
||||
* found.
|
||||
*
|
||||
* Most patterns produce service-level detections; `TS @GrpcMethod` is
|
||||
* the only pattern that captures an explicit `methodName`, producing
|
||||
* a method-level contract (`grpc::pkg.Service/Method`).
|
||||
*/
|
||||
export interface GrpcDetection {
|
||||
role: GrpcRole;
|
||||
/** Short service name, e.g. `"AuthService"`. */
|
||||
serviceName: string;
|
||||
/** Symbol name emitted into the contract's symbolRef. */
|
||||
symbolName: string;
|
||||
/** Metadata source label (goes into `meta.source`). */
|
||||
source: string;
|
||||
/** Explicit method name; set only by TS `@GrpcMethod`. */
|
||||
methodName?: string;
|
||||
/** Confidence when the proto map resolves the service. */
|
||||
confidenceWithProto: number;
|
||||
/** Confidence when the proto map has no entry. */
|
||||
confidenceWithoutProto: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* One language-scoped gRPC plugin. Plugins own the tree-sitter grammar
|
||||
* and a `scan(tree)` function that returns zero or more
|
||||
* `GrpcDetection`s. The plugin is free to run multiple compiled query
|
||||
* bundles and walk the AST to cross-reference captures.
|
||||
*
|
||||
* `language` is typed `unknown` for the same reason as in
|
||||
* `tree-sitter-scanner.ts`.
|
||||
*/
|
||||
export interface GrpcLanguagePlugin {
|
||||
name: string;
|
||||
language: unknown;
|
||||
scan(tree: Parser.Tree): GrpcDetection[];
|
||||
}
|
||||
Loading…
Add table
Reference in a new issue