From c5c42a964983f36ca91a29e51a7ac7754cf75e4d Mon Sep 17 00:00:00 2001 From: Gergo Magyar Date: Tue, 12 May 2026 08:28:42 +0100 Subject: [PATCH 1/9] fix(ci): make /review reliably post PR comments The Run Claude Code Review step passed an invalid PR ref (owner/repo/pull/N) which gh interprets as a branch name, causing early gh pr view failures. More importantly, the prompt omitted --comment, so the code-review plugin only displayed findings in terminal output and never invoked gh pr comment to post to the PR. Switch to a full PR URL and add --comment so the plugin posts the review during the session, which also routes around upstream bugs anthropics/claude-code-action#1061 and #1087 where the action's post-step capture can silently drop output on issue_comment triggers. --- .github/workflows/claude.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/claude.yml b/.github/workflows/claude.yml index fcafc0279..4f43c3d9d 100644 --- a/.github/workflows/claude.yml +++ b/.github/workflows/claude.yml @@ -160,4 +160,4 @@ jobs: show_full_output: true plugin_marketplaces: 'https://github.com/anthropics/claude-code.git' plugins: 'code-review@claude-code-plugins' - prompt: '/code-review:code-review ${{ github.repository }}/pull/${{ steps.pr.outputs.number }}' + prompt: '/code-review:code-review https://github.com/${{ github.repository }}/pull/${{ steps.pr.outputs.number }} --comment' From 2bf6d078aac60757f3a8a4ad905922210f8042b6 Mon Sep 17 00:00:00 2001 From: Gergo Magyar Date: Tue, 12 May 2026 09:07:47 +0100 Subject: [PATCH 2/9] ci(claude): allow Bash in code-review job without interactive approval Claude Code defaults to prompting for Bash approval. In GitHub Actions there is no human to approve, so gh pr comment and similar commands fail and the PR receives no review comment. Pass --dangerously-skip-permissions for the code-review step only (headless CI; token and checkout are already scoped). Co-authored-by: Cursor --- .github/workflows/claude.yml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/.github/workflows/claude.yml b/.github/workflows/claude.yml index 4f43c3d9d..021a56930 100644 --- a/.github/workflows/claude.yml +++ b/.github/workflows/claude.yml @@ -158,6 +158,8 @@ jobs: github_token: ${{ secrets.GITHUB_TOKEN }} allowed_non_write_users: '*' show_full_output: true + # Review posts use Bash (`gh`, etc.); default mode asks for approval — impossible in CI. + claude_args: '--dangerously-skip-permissions' plugin_marketplaces: 'https://github.com/anthropics/claude-code.git' plugins: 'code-review@claude-code-plugins' prompt: '/code-review:code-review https://github.com/${{ github.repository }}/pull/${{ steps.pr.outputs.number }} --comment' From d4f34905bc09e973913030a6b0a5f0045a3d6f89 Mon Sep 17 00:00:00 2001 From: Copilot <198982749+Copilot@users.noreply.github.com> Date: Tue, 12 May 2026 09:37:44 +0100 Subject: [PATCH 3/9] feat: migrate Java to scope-based registry resolution (RFC #909 Ring 3) (#1482) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Initial plan * feat: implement Java scope-based resolution (RFC #909 Ring 3) Add scope-resolution pipeline for Java, following the C# pattern: - query.ts: tree-sitter query for scopes, declarations, imports, type bindings, and references against tree-sitter-java grammar - captures.ts: orchestrator synthesizing import decomposition, receiver bindings (this/super), arity metadata, and reference arity - import-decomposer.ts: decompose import_declaration nodes into kind/source/name markers (named, wildcard, static, static-wildcard) - interpret.ts: convert captures to ParsedImport/ParsedTypeBinding - receiver-binding.ts: synthesize this/super type-bindings on instance methods with superclass support - arity-metadata.ts: extract parameter count/types using javaMethodConfig - arity.ts: Java arity compatibility check with varargs support - merge-bindings.ts: Java shadowing precedence (local > import > wildcard) - simple-hooks.ts: bindingScopeFor, importOwningScope, receiverBinding - import-target.ts: package path to file path resolution - scope-resolver.ts: ScopeResolver implementation registered in registry Wire scope hooks into javaProvider (java.ts) and register javaScopeResolver in SCOPE_RESOLVERS registry. Add createResolverParityIt wrapper to java.test.ts for parity testing. All 172 existing Java tests pass. Java is NOT added to MIGRATED_LANGUAGES — the resolver sits idle until the migration flag is flipped. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * chore(autofix): apply prettier + eslint fixes via /autofix command * fix: address review findings 1-4 — varargs arity, static import resolution, importOwningScope, stripGeneric Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/22308da3-59c9-47e6-8e52-738305b1b80a Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * docs: document registry-primary parity status and CI visibility gap in scope-resolver Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/22308da3-59c9-47e6-8e52-738305b1b80a Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * fix: add generic type erasure fallback in stripGeneric + update scope-resolver docs Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/223f77ac-59a7-4487-9316-f2be05eac5d3 Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * fix: improve stripGeneric fallback regex — use valid Java identifier chars and handle nested generics Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/223f77ac-59a7-4487-9316-f2be05eac5d3 Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * fix: address adversarial review findings 1-6 — flaky test, wildcard import fixture, varargs fixed-prefix test, qualified generic stripping, JSDoc updates Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/172c8a1a-cdf3-4de8-9142-f2c12c14b0a6 Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * docs: add inline comment explaining stripQualifier/stripGeneric call order Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/172c8a1a-cdf3-4de8-9142-f2c12c14b0a6 Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * test: add varargs 0-arg fixture and strengthen wildcard import assertions Finding 1: Added `badCall()` method with 0-arg `fmt.format()` call to the varargs fixture. Test documents that legacy mode still resolves this call (arity rejection is registry-primary only). The fixture now exercises both the success path (2-arg, 3-arg) and the undersupplied path (0-arg). Finding 2: Strengthened wildcard import test to assert `targetFilePath` on the CALLS edge (`com/example/models/User.java`), confirming the call resolved through the wildcard-imported type to the correct file. Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/2b4e5602-9833-485c-ab48-e1d54fdf8465 Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * chore(autofix): apply prettier + eslint fixes via /autofix command --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Gergő Magyar --- gitnexus/src/core/ingestion/languages/java.ts | 22 ++ .../languages/java/arity-metadata.ts | 49 ++++ .../core/ingestion/languages/java/arity.ts | 31 +++ .../ingestion/languages/java/cache-stats.ts | 30 +++ .../core/ingestion/languages/java/captures.ts | 235 ++++++++++++++++++ .../languages/java/import-decomposer.ts | 104 ++++++++ .../ingestion/languages/java/import-target.ts | 108 ++++++++ .../core/ingestion/languages/java/index.ts | 30 +++ .../ingestion/languages/java/interpret.ts | 141 +++++++++++ .../languages/java/merge-bindings.ts | 44 ++++ .../core/ingestion/languages/java/query.ts | 197 +++++++++++++++ .../languages/java/receiver-binding.ts | 103 ++++++++ .../languages/java/scope-resolver.ts | 97 ++++++++ .../ingestion/languages/java/simple-hooks.ts | 54 ++++ .../scope-resolution/pipeline/registry.ts | 2 + .../com/example/app/Main.java | 15 ++ .../com/example/util/Formatter.java | 8 + .../com/example/app/Main.java | 10 + .../com/example/models/Order.java | 7 + .../com/example/models/User.java | 7 + .../test/integration/resolvers/java.test.ts | 53 +++- gitnexus/test/integration/worker-pool.test.ts | 2 +- 22 files changed, 1347 insertions(+), 2 deletions(-) create mode 100644 gitnexus/src/core/ingestion/languages/java/arity-metadata.ts create mode 100644 gitnexus/src/core/ingestion/languages/java/arity.ts create mode 100644 gitnexus/src/core/ingestion/languages/java/cache-stats.ts create mode 100644 gitnexus/src/core/ingestion/languages/java/captures.ts create mode 100644 gitnexus/src/core/ingestion/languages/java/import-decomposer.ts create mode 100644 gitnexus/src/core/ingestion/languages/java/import-target.ts create mode 100644 gitnexus/src/core/ingestion/languages/java/index.ts create mode 100644 gitnexus/src/core/ingestion/languages/java/interpret.ts create mode 100644 gitnexus/src/core/ingestion/languages/java/merge-bindings.ts create mode 100644 gitnexus/src/core/ingestion/languages/java/query.ts create mode 100644 gitnexus/src/core/ingestion/languages/java/receiver-binding.ts create mode 100644 gitnexus/src/core/ingestion/languages/java/scope-resolver.ts create mode 100644 gitnexus/src/core/ingestion/languages/java/simple-hooks.ts create mode 100644 gitnexus/test/fixtures/lang-resolution/java-variadic-resolution/com/example/util/Formatter.java create mode 100644 gitnexus/test/fixtures/lang-resolution/java-wildcard-import/com/example/app/Main.java create mode 100644 gitnexus/test/fixtures/lang-resolution/java-wildcard-import/com/example/models/Order.java create mode 100644 gitnexus/test/fixtures/lang-resolution/java-wildcard-import/com/example/models/User.java diff --git a/gitnexus/src/core/ingestion/languages/java.ts b/gitnexus/src/core/ingestion/languages/java.ts index 96139ccda..c70eacb10 100644 --- a/gitnexus/src/core/ingestion/languages/java.ts +++ b/gitnexus/src/core/ingestion/languages/java.ts @@ -27,6 +27,17 @@ import { javaMethodConfig } from '../method-extractors/configs/jvm.js'; import { createVariableExtractor } from '../variable-extractors/generic.js'; import { javaVariableConfig } from '../variable-extractors/configs/jvm.js'; import { createHeritageExtractor } from '../heritage-extractors/generic.js'; +import { + emitJavaScopeCaptures, + interpretJavaImport, + interpretJavaTypeBinding, + javaBindingScopeFor, + javaImportOwningScope, + javaMergeBindings, + javaReceiverBinding, + javaArityCompatibility, + resolveJavaImportTarget, +} from './java/index.js'; export const javaProvider = defineLanguage({ id: SupportedLanguages.Java, @@ -65,4 +76,15 @@ export const javaProvider = defineLanguage({ variableExtractor: createVariableExtractor(javaVariableConfig), classExtractor: createClassExtractor(javaClassConfig), heritageExtractor: createHeritageExtractor(SupportedLanguages.Java), + + // ── RFC #909 Ring 3: scope-based resolution hooks ── + emitScopeCaptures: emitJavaScopeCaptures, + interpretImport: interpretJavaImport, + interpretTypeBinding: interpretJavaTypeBinding, + bindingScopeFor: javaBindingScopeFor, + importOwningScope: javaImportOwningScope, + mergeBindings: (_scope, bindings) => javaMergeBindings(bindings), + receiverBinding: javaReceiverBinding, + arityCompatibility: javaArityCompatibility, + resolveImportTarget: resolveJavaImportTarget, }); diff --git a/gitnexus/src/core/ingestion/languages/java/arity-metadata.ts b/gitnexus/src/core/ingestion/languages/java/arity-metadata.ts new file mode 100644 index 000000000..47cccbff9 --- /dev/null +++ b/gitnexus/src/core/ingestion/languages/java/arity-metadata.ts @@ -0,0 +1,49 @@ +/** + * Extract Java arity metadata from a method-like tree-sitter node — + * `method_declaration` or `constructor_declaration`. + * + * Reuses `javaMethodConfig.extractParameters` so scope-extracted defs + * carry the same arity semantics as the legacy parse-worker path: + * - varargs (`...`) collapses `parameterCount` to `undefined` + * - `parameterTypes` collects declared type names; a literal + * `'varargs'` marker is appended for variadic methods so + * `javaArityCompatibility` can detect them. + */ + +import type { SyntaxNode } from '../../utils/ast-helpers.js'; +import { javaMethodConfig } from '../../method-extractors/configs/jvm.js'; + +export interface JavaArityMetadata { + readonly parameterCount: number | undefined; + readonly requiredParameterCount: number | undefined; + readonly parameterTypes: readonly string[] | undefined; +} + +export function computeJavaArityMetadata(fnNode: SyntaxNode): JavaArityMetadata { + const params = javaMethodConfig.extractParameters?.(fnNode) ?? []; + + let hasVariadic = false; + const types: string[] = []; + for (const p of params) { + if (p.isVariadic) hasVariadic = true; + if (p.type !== null) types.push(p.type); + } + if (hasVariadic) types.push('varargs'); + + const total = params.length; + // For varargs methods, `parameterCount` (max) is unknown — any number of + // trailing arguments is valid. But the fixed-prefix parameters (everything + // before the variadic `...` param) are still required, so we preserve that + // count in `requiredParameterCount` so `javaArityCompatibility` can reject + // calls that undersupply the fixed prefix (e.g. `f(int x, String... args)` + // called with 0 args). + const fixedCount = params.filter((p) => !p.isVariadic).length; + const parameterCount = hasVariadic ? undefined : total; + const requiredParameterCount = hasVariadic ? fixedCount : total; + + return { + parameterCount, + requiredParameterCount, + parameterTypes: types.length > 0 ? types : undefined, + }; +} diff --git a/gitnexus/src/core/ingestion/languages/java/arity.ts b/gitnexus/src/core/ingestion/languages/java/arity.ts new file mode 100644 index 000000000..f98a8209d --- /dev/null +++ b/gitnexus/src/core/ingestion/languages/java/arity.ts @@ -0,0 +1,31 @@ +/** + * Java arity check, accommodating varargs (`...`). + * + * Verdicts: + * - `'compatible'` — argCount matches parameterCount, OR varargs present. + * - `'incompatible'` — argCount mismatches with no varargs. + * - `'unknown'` — metadata absent / incomplete. + */ + +import type { Callsite, SymbolDefinition } from 'gitnexus-shared'; + +export function javaArityCompatibility( + def: SymbolDefinition, + callsite: Callsite, +): 'compatible' | 'unknown' | 'incompatible' { + const max = def.parameterCount; + const min = def.requiredParameterCount; + if (max === undefined && min === undefined) return 'unknown'; + + const argCount = callsite.arity; + if (!Number.isFinite(argCount) || argCount < 0) return 'unknown'; + + const hasVarArgs = + def.parameterTypes !== undefined && + def.parameterTypes.some((t) => t === 'varargs' || t.includes('...')); + + if (min !== undefined && argCount < min) return 'incompatible'; + if (max !== undefined && argCount > max && !hasVarArgs) return 'incompatible'; + + return 'compatible'; +} diff --git a/gitnexus/src/core/ingestion/languages/java/cache-stats.ts b/gitnexus/src/core/ingestion/languages/java/cache-stats.ts new file mode 100644 index 000000000..a4c58f11f --- /dev/null +++ b/gitnexus/src/core/ingestion/languages/java/cache-stats.ts @@ -0,0 +1,30 @@ +/** + * Dev-mode counters for the cross-phase scope-captures parse cache + * (Java mirror of `languages/csharp/cache-stats.ts`). + * + * Gated by `PROF_SCOPE_RESOLUTION=1`. Production builds fold every + * increment into dead code via the module-level `PROF` constant, so + * the hot path in `captures.ts` stays branch-free. + */ + +const PROF = process.env.PROF_SCOPE_RESOLUTION === '1'; + +let CACHE_HITS = 0; +let CACHE_MISSES = 0; + +export function recordCacheHit(): void { + if (PROF) CACHE_HITS++; +} + +export function recordCacheMiss(): void { + if (PROF) CACHE_MISSES++; +} + +export function getJavaCaptureCacheStats(): { hits: number; misses: number } { + return { hits: CACHE_HITS, misses: CACHE_MISSES }; +} + +export function resetJavaCaptureCacheStats(): void { + CACHE_HITS = 0; + CACHE_MISSES = 0; +} diff --git a/gitnexus/src/core/ingestion/languages/java/captures.ts b/gitnexus/src/core/ingestion/languages/java/captures.ts new file mode 100644 index 000000000..73ea605fe --- /dev/null +++ b/gitnexus/src/core/ingestion/languages/java/captures.ts @@ -0,0 +1,235 @@ +/** + * `emitScopeCaptures` for Java. + * + * Drives the Java scope query against tree-sitter-java and groups raw + * matches into `CaptureMatch[]` for the central extractor. Layers: + * + * 1. **Decomposed import declarations** — each `import_declaration` + * is re-emitted with `@import.kind/source/name` markers. + * 2. **Receiver binding synthesis** — `this`/`super` type-bindings + * on instance methods. + * 3. **Arity metadata** on method/constructor declarations. + * 4. **Reference arity** on call sites. + * + * Pure given the input source text. No I/O, no globals consulted. + */ + +import type { Capture, CaptureMatch } from 'gitnexus-shared'; +import { findNodeAtRange, nodeToCapture, syntheticCapture } from '../../utils/ast-helpers.js'; +import { splitImportDeclaration } from './import-decomposer.js'; +import { computeJavaArityMetadata } from './arity-metadata.js'; +import { synthesizeJavaReceiverBinding } from './receiver-binding.js'; +import { getJavaParser, getJavaScopeQuery } from './query.js'; +import { recordCacheHit, recordCacheMiss } from './cache-stats.js'; +import { getTreeSitterBufferSize } from '../../constants.js'; +import { parseSourceSafe } from '../../../tree-sitter/safe-parse.js'; + +/** Declaration anchors that carry function-like arity metadata. */ +const FUNCTION_DECL_TAGS = ['@declaration.method', '@declaration.constructor'] as const; + +/** tree-sitter-java node types that the method extractor accepts. */ +const FUNCTION_NODE_TYPES = ['method_declaration', 'constructor_declaration'] as const; + +/** Suppress read.member emissions when the field_access is already + * covered by a method_invocation (object of a call) or an + * assignment_expression (write target). */ +function shouldEmitReadMember(memberNode: SyntaxNode): boolean { + const parent = memberNode.parent; + if (parent === null) return true; + + switch (parent.type) { + case 'method_invocation': + // Don't emit read.member when the field_access is the object of a method_invocation + // (the method call already handles this relationship) + return parent.childForFieldName('object')?.id !== memberNode.id; + case 'assignment_expression': + return parent.childForFieldName('left')?.id !== memberNode.id; + default: + return true; + } +} + +export function emitJavaScopeCaptures( + sourceText: string, + _filePath: string, + cachedTree?: unknown, +): readonly CaptureMatch[] { + let tree = cachedTree as ReturnType['parse']> | undefined; + if (tree === undefined) { + tree = parseSourceSafe(getJavaParser(), sourceText, undefined, { + bufferSize: getTreeSitterBufferSize(sourceText), + }); + recordCacheMiss(); + } else { + recordCacheHit(); + } + + const rawMatches = getJavaScopeQuery().matches(tree.rootNode); + const out: CaptureMatch[] = []; + + for (const m of rawMatches) { + const grouped: Record = {}; + for (const c of m.captures) { + const tag = '@' + c.name; + grouped[tag] = nodeToCapture(tag, c.node); + } + if (Object.keys(grouped).length === 0) continue; + + // Decompose each `import_declaration`. + if (grouped['@import.statement'] !== undefined) { + const stmtCapture = grouped['@import.statement']; + const stmtNode = findNodeAtRange(tree.rootNode, stmtCapture.range, 'import_declaration'); + if (stmtNode !== null) { + const decomposed = splitImportDeclaration(stmtNode); + if (decomposed !== null) { + out.push(decomposed); + continue; + } + } + out.push(grouped); + continue; + } + + // Skip free-call matches that are actually member calls. The query + // matches ALL method_invocations as @reference.call.free (without + // negation) because tree-sitter-java's query engine drops !object + // patterns when a positive object: pattern exists for the same node + // type. Filter here: if the match has @reference.call.free but also + // has @reference.receiver, it's a member call — skip the free match + // (the separate @reference.call.member match covers it). + if ( + grouped['@reference.call.free'] !== undefined && + grouped['@reference.receiver'] !== undefined + ) { + continue; + } + + // Filter read.member when it's a child of method_invocation or assignment. + if (grouped['@reference.read.member'] !== undefined) { + const anchor = grouped['@reference.read.member']; + const memberNode = findNodeAtRange(tree.rootNode, anchor.range, 'field_access'); + if (memberNode === null || !shouldEmitReadMember(memberNode)) { + continue; + } + } + + // Synthesize `this` / `super` receiver type-bindings on every + // instance method-like. + if (grouped['@scope.function'] !== undefined) { + out.push(grouped); + const anchor = grouped['@scope.function']!; + const fnNode = findFunctionNode(tree.rootNode, anchor.range); + if (fnNode !== null) { + for (const synth of synthesizeJavaReceiverBinding(fnNode)) { + out.push(synth); + } + } + continue; + } + + // Synthesize arity metadata on function-like declarations. + const declTag = FUNCTION_DECL_TAGS.find((t) => grouped[t] !== undefined); + if (declTag !== undefined) { + const anchor = grouped[declTag]!; + const fnNode = findFunctionNode(tree.rootNode, anchor.range); + if (fnNode !== null) { + const arity = computeJavaArityMetadata(fnNode); + if (arity.parameterCount !== undefined) { + grouped['@declaration.parameter-count'] = syntheticCapture( + '@declaration.parameter-count', + fnNode, + String(arity.parameterCount), + ); + } + if (arity.requiredParameterCount !== undefined) { + grouped['@declaration.required-parameter-count'] = syntheticCapture( + '@declaration.required-parameter-count', + fnNode, + String(arity.requiredParameterCount), + ); + } + if (arity.parameterTypes !== undefined) { + grouped['@declaration.parameter-types'] = syntheticCapture( + '@declaration.parameter-types', + fnNode, + JSON.stringify(arity.parameterTypes), + ); + } + } + } + + // Synthesize `@reference.arity` on every callsite. + const callTag = ( + ['@reference.call.free', '@reference.call.member', '@reference.call.constructor'] as const + ).find((t) => grouped[t] !== undefined); + if (callTag !== undefined && grouped['@reference.arity'] === undefined) { + const anchor = grouped[callTag]!; + const callNode = + findNodeAtRange(tree.rootNode, anchor.range, 'method_invocation') ?? + findNodeAtRange(tree.rootNode, anchor.range, 'object_creation_expression'); + if (callNode !== null) { + const argList = callNode.childForFieldName('arguments'); + const args = + argList === null + ? [] + : argList.namedChildren.filter((c) => c !== null && c.type !== 'comment'); + grouped['@reference.arity'] = syntheticCapture( + '@reference.arity', + callNode, + String(args.length), + ); + + const argTypes = args.map((arg) => inferArgType(arg!)); + grouped['@reference.parameter-types'] = syntheticCapture( + '@reference.parameter-types', + callNode, + JSON.stringify(argTypes), + ); + } + } + + out.push(grouped); + } + + return out; +} + +type SyntaxNode = ReturnType['parse']>['rootNode']; + +/** Infer a Java argument's static type from literal patterns. */ +function inferArgType(argNode: SyntaxNode): string { + switch (argNode.type) { + case 'decimal_integer_literal': + case 'hex_integer_literal': + case 'octal_integer_literal': + case 'binary_integer_literal': + return 'int'; + case 'decimal_floating_point_literal': + case 'hex_floating_point_literal': + return 'double'; + case 'string_literal': + return 'String'; + case 'character_literal': + return 'char'; + case 'true': + case 'false': + return 'boolean'; + case 'null_literal': + return 'null'; + case 'object_creation_expression': { + const typeNode = argNode.childForFieldName('type'); + return typeNode?.text ?? ''; + } + default: + return ''; + } +} + +/** Find the first Java function-like node at the given range. */ +function findFunctionNode(rootNode: SyntaxNode, range: Capture['range']): SyntaxNode | null { + for (const nodeType of FUNCTION_NODE_TYPES) { + const n = findNodeAtRange(rootNode, range, nodeType); + if (n !== null) return n as SyntaxNode; + } + return null; +} diff --git a/gitnexus/src/core/ingestion/languages/java/import-decomposer.ts b/gitnexus/src/core/ingestion/languages/java/import-decomposer.ts new file mode 100644 index 000000000..59c74d144 --- /dev/null +++ b/gitnexus/src/core/ingestion/languages/java/import-decomposer.ts @@ -0,0 +1,104 @@ +/** + * Decompose a Java `import_declaration` into a `CaptureMatch` carrying + * the synthesized markers `@import.kind` / `@import.source` / + * `@import.name` that `interpretJavaImport` consumes. + * + * Unlike C#'s using-directive decomposer, Java has four import forms: + * + * import com.example.User; → named + * import com.example.*; → wildcard + * import static com.example.Utils.format; → static + * import static com.example.Utils.*; → static-wildcard + * + * Each produces exactly one import. The decomposer inspects the raw + * source text and tree-sitter children to determine the flavor. + */ + +import type { Capture, CaptureMatch } from 'gitnexus-shared'; +import { nodeToCapture, syntheticCapture, type SyntaxNode } from '../../utils/ast-helpers.js'; + +type ImportKind = 'named' | 'wildcard' | 'static' | 'static-wildcard'; + +interface ImportSpec { + readonly kind: ImportKind; + /** Full dotted path: `com.example.User`. */ + readonly source: string; + /** Local binding name — last path segment for named/static, + * `'*'` for wildcard/static-wildcard. */ + readonly name: string; + /** Node to anchor the synthesized captures (range-wise). */ + readonly atNode: SyntaxNode; +} + +export function splitImportDeclaration(stmtNode: SyntaxNode): CaptureMatch | null { + if (stmtNode.type !== 'import_declaration') return null; + const spec = parseImportDeclaration(stmtNode); + if (spec === null) return null; + return buildImportMatch(stmtNode, spec); +} + +function parseImportDeclaration(node: SyntaxNode): ImportSpec | null { + // Detect `static` by checking for an anonymous `static` token child. + let isStatic = false; + for (let i = 0; i < node.childCount; i++) { + const child = node.child(i); + if (child !== null && child.type === 'static') { + isStatic = true; + break; + } + } + + // Detect wildcard by checking for `asterisk` named child. + let isWildcard = false; + for (let i = 0; i < node.namedChildCount; i++) { + const child = node.namedChild(i); + if (child !== null && child.type === 'asterisk') { + isWildcard = true; + break; + } + } + + // Find the scoped_identifier (or identifier for single-segment imports). + let pathNode: SyntaxNode | null = null; + for (let i = 0; i < node.namedChildCount; i++) { + const child = node.namedChild(i); + if (child !== null && (child.type === 'scoped_identifier' || child.type === 'identifier')) { + pathNode = child; + break; + } + } + if (pathNode === null) return null; + + const fullPath = pathNode.text; + if (fullPath === '') return null; + + if (isStatic && isWildcard) { + // `import static com.example.Utils.*;` + return { kind: 'static-wildcard', source: fullPath, name: '*', atNode: node }; + } + if (isStatic) { + // `import static com.example.Utils.format;` + const lastDot = fullPath.lastIndexOf('.'); + const name = lastDot >= 0 ? fullPath.slice(lastDot + 1) : fullPath; + return { kind: 'static', source: fullPath, name, atNode: node }; + } + if (isWildcard) { + // `import com.example.*;` + return { kind: 'wildcard', source: fullPath, name: '*', atNode: node }; + } + + // `import com.example.User;` + const lastDot = fullPath.lastIndexOf('.'); + const name = lastDot >= 0 ? fullPath.slice(lastDot + 1) : fullPath; + return { kind: 'named', source: fullPath, name, atNode: node }; +} + +function buildImportMatch(stmtNode: SyntaxNode, spec: ImportSpec): CaptureMatch { + const m: Record = { + '@import.statement': nodeToCapture('@import.statement', stmtNode), + '@import.kind': syntheticCapture('@import.kind', spec.atNode, spec.kind), + '@import.source': syntheticCapture('@import.source', spec.atNode, spec.source), + '@import.name': syntheticCapture('@import.name', spec.atNode, spec.name), + }; + return m; +} diff --git a/gitnexus/src/core/ingestion/languages/java/import-target.ts b/gitnexus/src/core/ingestion/languages/java/import-target.ts new file mode 100644 index 000000000..b78b6369b --- /dev/null +++ b/gitnexus/src/core/ingestion/languages/java/import-target.ts @@ -0,0 +1,108 @@ +/** + * Adapter from `(ParsedImport, WorkspaceIndex)` → concrete file path. + * + * Converts Java package paths (dots → slashes) and tries: + * 1. Exact file match: `com/example/User.java` + * 2. Suffix match for nested layouts + * 3. Directory match (wildcard imports) + * 4. Progressive prefix stripping for non-standard layouts + * + * Returns `null` for unresolvable / JDK imports. + */ + +import type { ParsedImport, WorkspaceIndex } from 'gitnexus-shared'; + +export interface JavaResolveContext { + readonly fromFile: string; + readonly allFilePaths: ReadonlySet; +} + +export function resolveJavaImportTarget( + parsedImport: ParsedImport, + workspaceIndex: WorkspaceIndex, +): string | null { + const ctx = workspaceIndex as JavaResolveContext | undefined; + if ( + ctx === undefined || + typeof (ctx as { fromFile?: unknown }).fromFile !== 'string' || + !((ctx as { allFilePaths?: unknown }).allFilePaths instanceof Set) + ) { + return null; + } + if (parsedImport.kind === 'dynamic-unresolved') return null; + if (parsedImport.targetRaw === null || parsedImport.targetRaw === '') return null; + + // Strip trailing `.*` for wildcard imports: `com.example.*` → `com.example` + let target = parsedImport.targetRaw; + if (target.endsWith('.*')) { + target = target.slice(0, -2); + } + + // Package path: `com.example.User` → `com/example/User` + const pathLike = target.replace(/\./g, '/'); + const suffix = `/${pathLike}`; + + let exactFile: string | null = null; + let suffixFile: string | null = null; + let directoryChild: string | null = null; + const dirPrefix = `${pathLike}/`; + const suffixDirPrefix = `/${dirPrefix}`; + + for (const raw of ctx.allFilePaths) { + const f = raw.replace(/\\/g, '/'); + if (!f.endsWith('.java')) continue; + if (f === `${pathLike}.java`) { + exactFile = raw; + break; + } + if (suffixFile === null && f.endsWith(`${suffix}.java`)) { + suffixFile = raw; + } + if (directoryChild === null) { + const atRoot = f.startsWith(dirPrefix); + const atNested = f.includes(suffixDirPrefix); + if (atRoot || atNested) { + const idx = atRoot ? 0 : f.indexOf(suffixDirPrefix) + 1; + const after = f.slice(idx + dirPrefix.length); + if (after.length > 0 && !after.includes('/')) { + directoryChild = raw; + } + } + } + } + + if (exactFile !== null) return exactFile; + if (suffixFile !== null) return suffixFile; + if (directoryChild !== null) return directoryChild; + + // Progressive prefix stripping — handles `import com.example.User;` + // in a repo laid out `User.java` (no `com/example/` prefix). + const segments = pathLike.split('/').filter(Boolean); + for (let skip = 1; skip < segments.length; skip++) { + const tail = segments.slice(skip).join('/'); + if (tail === '') continue; + const tailFile = `${tail}.java`; + const tailSuffix = `/${tailFile}`; + const tailDir = `${tail}/`; + const tailSuffixDir = `/${tailDir}`; + let tailDirectChild: string | null = null; + for (const raw of ctx.allFilePaths) { + const f = raw.replace(/\\/g, '/'); + if (!f.endsWith('.java')) continue; + if (f === tailFile) return raw; + if (f.endsWith(tailSuffix)) return raw; + if (tailDirectChild === null) { + const atRoot = f.startsWith(tailDir); + const atNested = f.includes(tailSuffixDir); + if (atRoot || atNested) { + const idx = atRoot ? 0 : f.indexOf(tailSuffixDir) + 1; + const after = f.slice(idx + tailDir.length); + if (after.length > 0 && !after.includes('/')) tailDirectChild = raw; + } + } + } + if (tailDirectChild !== null) return tailDirectChild; + } + + return null; +} diff --git a/gitnexus/src/core/ingestion/languages/java/index.ts b/gitnexus/src/core/ingestion/languages/java/index.ts new file mode 100644 index 000000000..443faac4b --- /dev/null +++ b/gitnexus/src/core/ingestion/languages/java/index.ts @@ -0,0 +1,30 @@ +/** + * Java scope-resolution hooks (RFC #909 Ring 3). + * + * Public API barrel. Consumers should import from this file rather than + * the individual modules. + * + * Module layout: + * + * - `query.ts` — tree-sitter query + lazy parser/query singletons + * - `captures.ts` — `emitJavaScopeCaptures` orchestrator + * - `import-decomposer.ts` — each `import` → ParsedImport-shaped captures + * - `interpret.ts` — capture-match → `ParsedImport` / `ParsedTypeBinding` + * - `simple-hooks.ts` — small hooks made explicit + * - `receiver-binding.ts` — synthesize `this`/`super` type-bindings on + * instance-method entry + * - `merge-bindings.ts` — Java import precedence + * - `arity.ts` — Java arity compatibility (varargs) + * - `arity-metadata.ts` — synthesize arity metadata from declarations + * - `import-target.ts` — `(ParsedImport, WorkspaceIndex) → file path` adapter + * - `scope-resolver.ts` — `ScopeResolver` registered in `SCOPE_RESOLVERS` + * - `cache-stats.ts` — PROF_SCOPE_RESOLUTION cache hit/miss counters + */ + +export { emitJavaScopeCaptures } from './captures.js'; +export { getJavaCaptureCacheStats, resetJavaCaptureCacheStats } from './cache-stats.js'; +export { interpretJavaImport, interpretJavaTypeBinding } from './interpret.js'; +export { javaMergeBindings } from './merge-bindings.js'; +export { javaArityCompatibility } from './arity.js'; +export { resolveJavaImportTarget, type JavaResolveContext } from './import-target.js'; +export { javaBindingScopeFor, javaImportOwningScope, javaReceiverBinding } from './simple-hooks.js'; diff --git a/gitnexus/src/core/ingestion/languages/java/interpret.ts b/gitnexus/src/core/ingestion/languages/java/interpret.ts new file mode 100644 index 000000000..9c207d451 --- /dev/null +++ b/gitnexus/src/core/ingestion/languages/java/interpret.ts @@ -0,0 +1,141 @@ +/** + * Capture-match → semantic-shape interpreters for Java. + * + * - `interpretJavaImport` → `ParsedImport` + * - `interpretJavaTypeBinding` → `ParsedTypeBinding` + * + * Import matches arrive pre-decomposed by `emitJavaScopeCaptures` + * (one import per match, with synthesized `@import.kind/source/name` + * markers). Type-binding matches arrive from the raw query captures. + */ + +import type { CaptureMatch, ParsedImport, ParsedTypeBinding, TypeRef } from 'gitnexus-shared'; + +// ─── interpretImport ────────────────────────────────────────────────────── + +export function interpretJavaImport(captures: CaptureMatch): ParsedImport | null { + const kindCap = captures['@import.kind']; + const sourceCap = captures['@import.source']; + const nameCap = captures['@import.name']; + + const kind = kindCap?.text; + if (kind === undefined || sourceCap === undefined) return null; + + switch (kind) { + case 'named': { + // `import com.example.User;` + return { + kind: 'named', + localName: nameCap?.text ?? sourceCap.text.split('.').pop() ?? sourceCap.text, + importedName: sourceCap.text, + targetRaw: sourceCap.text, + }; + } + case 'wildcard': { + // `import com.example.*;` + return { + kind: 'wildcard', + targetRaw: sourceCap.text + '.*', + }; + } + case 'static': { + // `import static com.example.Utils.format;` + // The source contains the full path including the member name + // (e.g. `com.example.Utils.format`). For file resolution we need + // the class path (`com.example.Utils`), so strip the final member + // segment. The local binding name is the member itself. + const fullSource = sourceCap.text; + const lastDot = fullSource.lastIndexOf('.'); + const classPath = lastDot >= 0 ? fullSource.slice(0, lastDot) : fullSource; + return { + kind: 'named', + localName: nameCap?.text ?? (lastDot >= 0 ? fullSource.slice(lastDot + 1) : fullSource), + importedName: fullSource, + targetRaw: classPath, + }; + } + case 'static-wildcard': { + // `import static com.example.Utils.*;` + // The source is the class path (e.g. `com.example.Utils`). + // Resolution should target the class file, not a wildcard directory + // scan — `Utils.java` is the file that contains the static members. + return { + kind: 'wildcard', + targetRaw: sourceCap.text + '.*', + }; + } + default: + return null; + } +} + +// ─── interpretTypeBinding ───────────────────────────────────────────────── + +export function interpretJavaTypeBinding(captures: CaptureMatch): ParsedTypeBinding | null { + const nameCap = captures['@type-binding.name']; + const typeCap = captures['@type-binding.type']; + if (nameCap === undefined || typeCap === undefined) return null; + + // Strip qualifier first so that `com.example.BaseModel` becomes + // `BaseModel` before stripGeneric — the JVM-erasure fallback pattern + // requires an unqualified identifier at the start of the string. + const rawType = stripGeneric(stripQualifier(typeCap.text.trim())); + + // Skip `var` — tree-sitter-java parses `var` as type_identifier with + // text "var". When used without a constructor initializer, there's no + // concrete type to bind. + if (rawType === 'var') return null; + + let source: TypeRef['source'] = 'parameter-annotation'; + if (captures['@type-binding.self'] !== undefined) source = 'self'; + else if (captures['@type-binding.constructor'] !== undefined) source = 'constructor-inferred'; + else if (captures['@type-binding.annotation'] !== undefined) source = 'annotation'; + else if (captures['@type-binding.return'] !== undefined) source = 'return-annotation'; + + return { boundName: nameCap.text, rawTypeName: rawType, source }; +} + +/** + * Unwrap generic type parameters from Java types. + * + * Three tiers, checked in order: + * 1. Known single-arg collection wrappers → extract the element type + * (`List` → `User`, `Optional` → `User`). + * 2. Known two-arg map/container types → extract the value type + * (`Map` → `User`). + * 3. **Fallback (JVM type erasure):** any other generic type → + * strip the generic parameters and keep the raw class name + * (`BaseModel` → `BaseModel`, `CustomList` → `CustomList`). + * This ensures receiver bindings (`this`/`super`) on classes with + * generic superclasses resolve to the correct class file. + */ +function stripGeneric(text: string): string { + // Single-type-argument containers — extract the element type. + const single = text.match( + /^(?:[A-Za-z_][A-Za-z0-9_.]*\.)?(?:List|ArrayList|LinkedList|Set|HashSet|TreeSet|SortedSet|LinkedHashSet|Collection|Iterable|Iterator|Optional|Stream|CompletableFuture|Future|Queue|Deque|ArrayDeque|PriorityQueue|Vector|Stack|Supplier|Consumer|Predicate|Function)<([^,<>]+)>$/, + ); + if (single !== null) return single[1].trim(); + + // Two-type-argument map/container types — extract the value type (second arg). + const twoArg = text.match( + /^(?:[A-Za-z_][A-Za-z0-9_.]*\.)?(?:Map|HashMap|TreeMap|LinkedHashMap|ConcurrentHashMap|ConcurrentMap|SortedMap|NavigableMap|Hashtable|EnumMap|WeakHashMap|IdentityHashMap|BiFunction|BiConsumer|BiPredicate|Pair|Entry)<[^,<>]+,\s*([^,<>]+)>$/, + ); + if (twoArg !== null) return twoArg[1].trim(); + + // Fallback: strip generic parameters from any unrecognized generic type. + // `BaseModel` → `BaseModel`, `Builder` → `Builder`. + // This mirrors JVM type erasure — the raw class name is the resolvable symbol. + // The pattern matches up to the first `<` to handle nested generics safely + // (e.g. `BaseModel>` → `BaseModel`). + const fallback = text.match(/^([A-Za-z_$][A-Za-z0-9_$]*)<.+>$/s); + if (fallback !== null) return fallback[1].trim(); + + return text; +} + +/** `com.example.User` → `User`. */ +function stripQualifier(text: string): string { + const lastDot = text.lastIndexOf('.'); + if (lastDot === -1) return text; + return text.slice(lastDot + 1); +} diff --git a/gitnexus/src/core/ingestion/languages/java/merge-bindings.ts b/gitnexus/src/core/ingestion/languages/java/merge-bindings.ts new file mode 100644 index 000000000..9056705d0 --- /dev/null +++ b/gitnexus/src/core/ingestion/languages/java/merge-bindings.ts @@ -0,0 +1,44 @@ +/** + * Java shadowing precedence for the `mergeBindings` hook. + * + * Tier ranking (lower wins): + * - 0: `local` — class member, method, local variable, parameter + * - 1: `import` / `namespace` / `reexport` — explicit imports + * - 2: `wildcard` — wildcard imports (`import x.y.*`) + * + * Within a surviving tier: de-dup by DefId, last-write-wins. + */ + +import type { BindingRef } from 'gitnexus-shared'; + +const TIER_LOCAL = 0; +const TIER_IMPORT = 1; +const TIER_WILDCARD = 2; +const TIER_UNKNOWN = 3; + +function tierOf(b: BindingRef): number { + switch (b.origin) { + case 'local': + return TIER_LOCAL; + case 'reexport': + case 'import': + case 'namespace': + return TIER_IMPORT; + case 'wildcard': + return TIER_WILDCARD; + default: + return TIER_UNKNOWN; + } +} + +export function javaMergeBindings(bindings: readonly BindingRef[]): readonly BindingRef[] { + if (bindings.length === 0) return bindings; + + let bestTier = Number.POSITIVE_INFINITY; + for (const b of bindings) bestTier = Math.min(bestTier, tierOf(b)); + const survivors = bindings.filter((b) => tierOf(b) === bestTier); + + const seen = new Map(); + for (const b of survivors) seen.set(b.def.nodeId, b); + return [...seen.values()]; +} diff --git a/gitnexus/src/core/ingestion/languages/java/query.ts b/gitnexus/src/core/ingestion/languages/java/query.ts new file mode 100644 index 000000000..3fabbb7bf --- /dev/null +++ b/gitnexus/src/core/ingestion/languages/java/query.ts @@ -0,0 +1,197 @@ +/** + * Tree-sitter query for Java scope captures (RFC §5.1). + * + * Captures the structural skeleton the generic scope-resolution + * pipeline consumes: scopes (module/class/function), declarations + * (class-likes, method-likes, fields, variables), imports (import + * declarations), type bindings (parameter annotations, variable + * annotations, constructor inference), and references (call sites, + * member writes/reads). + * + * Java specifics that shape this query: + * + * - Java uses `program` as the root node (not `compilation_unit`). + * - `import_declaration` nodes carry `scoped_identifier` children + * and optional `asterisk` for wildcard imports. + * - `static` imports are detected by an anonymous `static` token + * child within `import_declaration`. + * - `var` (Java 10+ local variable type inference) parses as a + * `type_identifier` with text `"var"`, not a dedicated node type. + * - Modifiers (`public`, `static`, etc.) are grouped under a + * `modifiers` named child with anonymous keyword tokens. + * - Superclass inheritance uses a `superclass:` field containing + * a `superclass` node wrapping a `type_identifier`. + * + * Exposes lazy `Parser` and `Query` singletons so callers don't pay + * tree-sitter init cost per file. + */ + +import Parser from 'tree-sitter'; +import Java from 'tree-sitter-java'; + +const JAVA_SCOPE_QUERY = ` +;; Scopes +(program) @scope.module + +(class_declaration) @scope.class +(interface_declaration) @scope.class +(enum_declaration) @scope.class +(record_declaration) @scope.class +(annotation_type_declaration) @scope.class + +(method_declaration) @scope.function +(constructor_declaration) @scope.function + +;; Declarations — types +(class_declaration + name: (identifier) @declaration.name) @declaration.class + +(interface_declaration + name: (identifier) @declaration.name) @declaration.interface + +(enum_declaration + name: (identifier) @declaration.name) @declaration.enum + +(record_declaration + name: (identifier) @declaration.name) @declaration.record + +(annotation_type_declaration + name: (identifier) @declaration.name) @declaration.class + +;; Declarations — methods / constructors +(method_declaration + name: (identifier) @declaration.name) @declaration.method + +(constructor_declaration + name: (identifier) @declaration.name) @declaration.constructor + +;; Declarations — fields +(field_declaration + declarator: (variable_declarator + name: (identifier) @declaration.name)) @declaration.variable + +;; Declarations — local variables +(local_variable_declaration + declarator: (variable_declarator + name: (identifier) @declaration.name)) @declaration.variable + +;; Imports — single anchor per import_declaration +(import_declaration) @import.statement + +;; Type bindings — parameter annotations: void f(User u) +(formal_parameter + type: (type_identifier) @type-binding.type + name: (identifier) @type-binding.name) @type-binding.parameter + +(formal_parameter + type: (generic_type) @type-binding.type + name: (identifier) @type-binding.name) @type-binding.parameter + +(formal_parameter + type: (scoped_type_identifier) @type-binding.type + name: (identifier) @type-binding.name) @type-binding.parameter + +;; Type bindings — local variable annotations: User u = new User(); +(local_variable_declaration + type: (type_identifier) @type-binding.type + declarator: (variable_declarator + name: (identifier) @type-binding.name)) @type-binding.annotation + +(local_variable_declaration + type: (generic_type) @type-binding.type + declarator: (variable_declarator + name: (identifier) @type-binding.name)) @type-binding.annotation + +;; Type bindings — var u = new User(); (Java 10+ local variable type inference) +;; tree-sitter-java parses \`var\` as a \`type_identifier\` with text "var". +;; The type-binding.constructor anchor fires when the rhs is an +;; object_creation_expression so interpretJavaTypeBinding can infer +;; the concrete type from the constructor call. +(local_variable_declaration + type: (type_identifier) @_var_type + declarator: (variable_declarator + name: (identifier) @type-binding.name + value: (object_creation_expression + type: (type_identifier) @type-binding.type))) @type-binding.constructor + +;; Type bindings — field declarations: private User user; +(field_declaration + type: (type_identifier) @type-binding.type + declarator: (variable_declarator + name: (identifier) @type-binding.name)) @type-binding.annotation + +(field_declaration + type: (generic_type) @type-binding.type + declarator: (variable_declarator + name: (identifier) @type-binding.name)) @type-binding.annotation + +;; Type bindings — method return type: public User getUser() { } +(method_declaration + type: (type_identifier) @type-binding.type + name: (identifier) @type-binding.name) @type-binding.return + +(method_declaration + type: (generic_type) @type-binding.type + name: (identifier) @type-binding.name) @type-binding.return + +;; Type bindings — enhanced for: for (User u : list) +(enhanced_for_statement + type: (type_identifier) @type-binding.type + name: (identifier) @type-binding.name) @type-binding.annotation + +(enhanced_for_statement + type: (generic_type) @type-binding.type + name: (identifier) @type-binding.name) @type-binding.annotation + +;; References — all method calls: foo() and obj.method() +;; tree-sitter-java's query engine drops negation-based \`!object\` +;; patterns when a positive \`object:\` pattern exists for the same +;; node type, so we match all calls here and classify free vs +;; member in captures.ts based on the presence of @reference.receiver. +(method_invocation + object: (_) @reference.receiver + name: (identifier) @reference.name) @reference.call.member + +(method_invocation + name: (identifier) @reference.name) @reference.call.free + +;; References — constructor calls: new User(...) +(object_creation_expression + type: (type_identifier) @reference.name) @reference.call.constructor + +(object_creation_expression + type: (generic_type + (type_identifier) @reference.name)) @reference.call.constructor + +(object_creation_expression + type: (scoped_type_identifier) @reference.call.constructor.qualified) @reference.call.constructor + +;; References — field/property writes: obj.name = "x" +(assignment_expression + left: (field_access + object: (_) @reference.receiver + field: (identifier) @reference.name)) @reference.write.member + +;; References — field/property reads: obj.name +(field_access + object: (_) @reference.receiver + field: (identifier) @reference.name) @reference.read.member +`; + +let _parser: Parser | null = null; +let _query: Parser.Query | null = null; + +export function getJavaParser(): Parser { + if (_parser === null) { + _parser = new Parser(); + _parser.setLanguage(Java as Parameters[0]); + } + return _parser; +} + +export function getJavaScopeQuery(): Parser.Query { + if (_query === null) { + _query = new Parser.Query(Java as Parameters[0], JAVA_SCOPE_QUERY); + } + return _query; +} diff --git a/gitnexus/src/core/ingestion/languages/java/receiver-binding.ts b/gitnexus/src/core/ingestion/languages/java/receiver-binding.ts new file mode 100644 index 000000000..4d6d60ded --- /dev/null +++ b/gitnexus/src/core/ingestion/languages/java/receiver-binding.ts @@ -0,0 +1,103 @@ +/** + * Synthesize `@type-binding.self` captures for Java instance methods — + * one for `this` (always on non-static methods inside a type + * declaration) and optionally one for `super` (only on class methods + * when the enclosing class has a `superclass`). + * + * Mirrors `languages/csharp/receiver-binding.ts` in structure. + */ + +import type { Capture, CaptureMatch } from 'gitnexus-shared'; +import { nodeToCapture, syntheticCapture, type SyntaxNode } from '../../utils/ast-helpers.js'; + +const TYPE_DECL_NODE_TYPES = new Set([ + 'class_declaration', + 'interface_declaration', + 'enum_declaration', + 'record_declaration', +]); + +const FUNCTION_NODE_TYPES = new Set(['method_declaration', 'constructor_declaration']); + +/** Walk up to the enclosing type declaration. */ +function findEnclosingTypeDeclaration(node: SyntaxNode): SyntaxNode | null { + let cur: SyntaxNode | null = node.parent; + while (cur !== null) { + if (TYPE_DECL_NODE_TYPES.has(cur.type)) return cur; + cur = cur.parent; + } + return null; +} + +function typeName(typeNode: SyntaxNode): string | null { + return typeNode.childForFieldName('name')?.text ?? null; +} + +/** First superclass text. tree-sitter-java uses a `superclass` field + * containing a `superclass` node wrapping a `type_identifier`. */ +function firstSuperclassText(typeNode: SyntaxNode): string | null { + const superclass = typeNode.childForFieldName('superclass'); + if (superclass === null) return null; + // The superclass node wraps the type_identifier + for (let i = 0; i < superclass.namedChildCount; i++) { + const child = superclass.namedChild(i); + if (child !== null && (child.type === 'type_identifier' || child.type === 'generic_type')) { + return child.text; + } + } + return null; +} + +/** Check if a method has the `static` modifier. In tree-sitter-java, + * modifiers are grouped under a `modifiers` named child with anonymous + * keyword tokens. */ +function isStaticMethod(fnNode: SyntaxNode): boolean { + for (let i = 0; i < fnNode.namedChildCount; i++) { + const child = fnNode.namedChild(i); + if (child !== null && child.type === 'modifiers') { + for (let j = 0; j < child.childCount; j++) { + const mod = child.child(j); + if (mod !== null && mod.text.trim() === 'static') return true; + } + } + } + return false; +} + +export function synthesizeJavaReceiverBinding(fnNode: SyntaxNode): CaptureMatch[] { + if (!FUNCTION_NODE_TYPES.has(fnNode.type)) return []; + if (isStaticMethod(fnNode)) return []; + + const enclosingType = findEnclosingTypeDeclaration(fnNode); + if (enclosingType === null) return []; + + const enclosingName = typeName(enclosingType); + if (enclosingName === null) return []; + + // Anchor to the method body so the synthesized captures are inside + // the function scope. + const anchorNode = fnNode.childForFieldName('body'); + if (anchorNode === null) return []; + + const out: CaptureMatch[] = []; + out.push(buildReceiverMatch(anchorNode, 'this', enclosingName)); + + // `super` applies only to class/record methods with an explicit superclass. + if (enclosingType.type === 'class_declaration' || enclosingType.type === 'record_declaration') { + const superText = firstSuperclassText(enclosingType); + if (superText !== null) { + out.push(buildReceiverMatch(anchorNode, 'super', superText)); + } + } + + return out; +} + +function buildReceiverMatch(anchorNode: SyntaxNode, name: string, typeText: string): CaptureMatch { + const m: Record = { + '@type-binding.self': nodeToCapture('@type-binding.self', anchorNode), + '@type-binding.name': syntheticCapture('@type-binding.name', anchorNode, name), + '@type-binding.type': syntheticCapture('@type-binding.type', anchorNode, typeText), + }; + return m; +} diff --git a/gitnexus/src/core/ingestion/languages/java/scope-resolver.ts b/gitnexus/src/core/ingestion/languages/java/scope-resolver.ts new file mode 100644 index 000000000..dac974cc7 --- /dev/null +++ b/gitnexus/src/core/ingestion/languages/java/scope-resolver.ts @@ -0,0 +1,97 @@ +/** + * Java `ScopeResolver` registered in `SCOPE_RESOLVERS` and consumed by + * the generic `runScopeResolution` orchestrator (RFC #909 Ring 3). + * + * ## Registry-primary parity status + * + * Java is **not** in `MIGRATED_LANGUAGES` — the scope-resolution + * registry runs in shadow mode only. Parity in forced registry mode + * (`REGISTRY_PRIMARY_JAVA=1`) is 143/172 (83%). The 29 gaps fall into: + * + * - switch pattern binding / sealed-class exhaustiveness + * - Map.values() / entrySet() iteration type propagation + * - assignment / method chain return-type propagation across files + * - virtual dispatch / interface default methods + * + * These are the same category of advanced-resolution gaps seen in prior + * migrations (Python, C#, Go). Parity is below the ≥99% flip threshold + * per RFC §6.4. + * + * **CI visibility:** Because Java is absent from `MIGRATED_LANGUAGES`, + * the parity CI workflow (`ci-scope-parity.yml`) does not run Java in + * either `REGISTRY_PRIMARY_JAVA=0` or `=1` mode. Regressions in forced + * mode are only visible via manual `REGISTRY_PRIMARY_JAVA=1 npx vitest + * run java.test.ts`. Before flipping Java to registry-primary, a + * non-required CI step should be added to run Java tests in forced mode + * and report parity as a dashboard input. + * + * **Parity baseline (29 failures):** The 29 gaps in forced registry mode + * are tracked in this PR (#1482) and this JSDoc. If the gap count + * changes (up or down), update this baseline accordingly. + * + * ### Known flip-blockers (must fix before adding to MIGRATED_LANGUAGES) + * + * - Varargs arity: fixed-prefix count is now preserved, but no + * integration fixture exercises the 0-arg rejection path yet. + * - Static import resolution: `import static X.Y.m` now correctly + * resolves to `X/Y.java` (the class), not `X/Y/m.java` (the member). + * Edge cases with nested classes may remain. + * - Generic superclass receiver binding: `BaseModel` now strips + * to `BaseModel` via JVM type-erasure fallback in `stripGeneric`. + * - Wildcard import (`import com.example.*`) file selection is + * nondeterministic when multiple classes share a package directory. + * May produce wrong-file edges in forced mode. + * - Qualified generic type parameters in field/parameter annotations + * (`com.example.BaseModel`) — rare in practice but may miss + * resolution when the full qualifier is present with generics. + */ + +import type { ParsedFile } from 'gitnexus-shared'; +import { SupportedLanguages } from 'gitnexus-shared'; +import { buildMro, defaultLinearize } from '../../scope-resolution/passes/mro.js'; +import { populateClassOwnedMembers } from '../../scope-resolution/scope/walkers.js'; +import type { ScopeResolver } from '../../scope-resolution/contract/scope-resolver.js'; +import { javaProvider } from '../java.js'; +import { + javaArityCompatibility, + javaMergeBindings, + resolveJavaImportTarget, + type JavaResolveContext, +} from './index.js'; + +const javaScopeResolver: ScopeResolver = { + language: SupportedLanguages.Java, + languageProvider: javaProvider, + importEdgeReason: 'java-scope: import', + + resolveImportTarget: (targetRaw, fromFile, allFilePaths) => { + const ws: JavaResolveContext = { fromFile, allFilePaths }; + return resolveJavaImportTarget( + { kind: 'named', localName: '_', importedName: '_', targetRaw }, + ws, + ); + }, + + mergeBindings: (existing, incoming) => [...javaMergeBindings([...existing, ...incoming])], + + arityCompatibility: (callsite, def) => javaArityCompatibility(def, callsite), + + buildMro: (graph, parsedFiles, nodeLookup) => + buildMro(graph, parsedFiles, nodeLookup, defaultLinearize), + + populateOwners: (parsed: ParsedFile) => populateClassOwnedMembers(parsed), + + isSuperReceiver: (text) => text.trim() === 'super', + + // Java is statically typed — field-fallback heuristic stays off + fieldFallbackOnMethodLookup: false, + propagatesReturnTypesAcrossImports: true, + + // Java doesn't collapse member calls + collapseMemberCallsByCallerTarget: false, + + // Hoist return-type bindings to Module scope for cross-file propagation + hoistTypeBindingsToModule: true, +}; + +export { javaScopeResolver }; diff --git a/gitnexus/src/core/ingestion/languages/java/simple-hooks.ts b/gitnexus/src/core/ingestion/languages/java/simple-hooks.ts new file mode 100644 index 000000000..e69f768a6 --- /dev/null +++ b/gitnexus/src/core/ingestion/languages/java/simple-hooks.ts @@ -0,0 +1,54 @@ +/** + * Small hooks for the Java provider. Each is a few lines; they make + * the provider's choice explicit rather than relying on defaults. + */ + +import type { + CaptureMatch, + ParsedImport, + Scope, + ScopeId, + ScopeTree, + TypeRef, +} from 'gitnexus-shared'; + +// ─── bindingScopeFor ────────────────────────────────────────────────────── + +/** Method return-type bindings hoist to Module scope so cross-file + * `propagateImportedReturnTypes` and chain-follow can find them. */ +export function javaBindingScopeFor( + decl: CaptureMatch, + innermost: Scope, + tree: ScopeTree, +): ScopeId | null { + if (decl['@type-binding.return'] !== undefined) { + let cur: Scope | undefined = innermost; + while (cur !== undefined && cur.kind !== 'Module') { + const parentId: ScopeId | null = cur.parent ?? null; + if (parentId === null) break; + cur = tree.getScope(parentId); + } + if (cur !== undefined && cur.kind === 'Module') return cur.id; + } + return null; +} + +// ─── importOwningScope ──────────────────────────────────────────────────── + +/** Java imports are always at compilation-unit (Module) level (JLS §7.5). + * Return `null` unconditionally so the default Module scope is used. */ +export function javaImportOwningScope( + _imp: ParsedImport, + _innermost: Scope, + _tree: ScopeTree, +): ScopeId | null { + return null; +} + +// ─── receiverBinding ────────────────────────────────────────────────────── + +/** Look up `this` or `super` in the function scope's type bindings. */ +export function javaReceiverBinding(functionScope: Scope): TypeRef | null { + if (functionScope.kind !== 'Function') return null; + return functionScope.typeBindings.get('this') ?? functionScope.typeBindings.get('super') ?? null; +} diff --git a/gitnexus/src/core/ingestion/scope-resolution/pipeline/registry.ts b/gitnexus/src/core/ingestion/scope-resolution/pipeline/registry.ts index 0667be8f5..ecd7b69aa 100644 --- a/gitnexus/src/core/ingestion/scope-resolution/pipeline/registry.ts +++ b/gitnexus/src/core/ingestion/scope-resolution/pipeline/registry.ts @@ -15,6 +15,7 @@ import { pythonScopeResolver } from '../../languages/python/scope-resolver.js'; import { csharpScopeResolver } from '../../languages/csharp/scope-resolver.js'; import { typescriptScopeResolver } from '../../languages/typescript/scope-resolver.js'; import { goScopeResolver } from '../../languages/go/scope-resolver.js'; +import { javaScopeResolver } from '../../languages/java/scope-resolver.js'; import { cScopeResolver } from '../../languages/c/scope-resolver.js'; /** Map of `SupportedLanguages` → `ScopeResolver`. The phase iterates @@ -29,5 +30,6 @@ export const SCOPE_RESOLVERS: ReadonlyMap = n [SupportedLanguages.CSharp, csharpScopeResolver], [SupportedLanguages.TypeScript, typescriptScopeResolver], [SupportedLanguages.Go, goScopeResolver], + [SupportedLanguages.Java, javaScopeResolver], [SupportedLanguages.C, cScopeResolver], ]); diff --git a/gitnexus/test/fixtures/lang-resolution/java-variadic-resolution/com/example/app/Main.java b/gitnexus/test/fixtures/lang-resolution/java-variadic-resolution/com/example/app/Main.java index e32e3c07e..88c3b71ac 100644 --- a/gitnexus/test/fixtures/lang-resolution/java-variadic-resolution/com/example/app/Main.java +++ b/gitnexus/test/fixtures/lang-resolution/java-variadic-resolution/com/example/app/Main.java @@ -1,10 +1,25 @@ package com.example.app; import com.example.util.Logger; +import com.example.util.Formatter; public class Main { public void run() { Logger logger = new Logger(); logger.record("hello", "world", "test"); + + Formatter fmt = new Formatter(); + // 2-arg call: satisfies fixed prefix (level) + 1 vararg + fmt.format(1, "hello"); + // 3-arg call: satisfies fixed prefix (level) + 2 varargs + fmt.format(2, "hello", "world"); + } + + public void badCall() { + Formatter fmt = new Formatter(); + // 0-arg call: does NOT satisfy the required fixed prefix (int level) + // This should be rejected by arity — no CALLS edge to format + fmt.format(); } } + diff --git a/gitnexus/test/fixtures/lang-resolution/java-variadic-resolution/com/example/util/Formatter.java b/gitnexus/test/fixtures/lang-resolution/java-variadic-resolution/com/example/util/Formatter.java new file mode 100644 index 000000000..6884acf16 --- /dev/null +++ b/gitnexus/test/fixtures/lang-resolution/java-variadic-resolution/com/example/util/Formatter.java @@ -0,0 +1,8 @@ +package com.example.util; + +public class Formatter { + /** Varargs with a required fixed prefix — 0-arg calls should be rejected. */ + public void format(int level, String... args) { + for (String a : args) System.out.println(level + ": " + a); + } +} diff --git a/gitnexus/test/fixtures/lang-resolution/java-wildcard-import/com/example/app/Main.java b/gitnexus/test/fixtures/lang-resolution/java-wildcard-import/com/example/app/Main.java new file mode 100644 index 000000000..bb4d88597 --- /dev/null +++ b/gitnexus/test/fixtures/lang-resolution/java-wildcard-import/com/example/app/Main.java @@ -0,0 +1,10 @@ +package com.example.app; + +import com.example.models.*; + +public class Main { + public void run() { + User user = new User(); + user.save(); + } +} diff --git a/gitnexus/test/fixtures/lang-resolution/java-wildcard-import/com/example/models/Order.java b/gitnexus/test/fixtures/lang-resolution/java-wildcard-import/com/example/models/Order.java new file mode 100644 index 000000000..2804c749e --- /dev/null +++ b/gitnexus/test/fixtures/lang-resolution/java-wildcard-import/com/example/models/Order.java @@ -0,0 +1,7 @@ +package com.example.models; + +public class Order { + public void submit() { + System.out.println("submitting order"); + } +} diff --git a/gitnexus/test/fixtures/lang-resolution/java-wildcard-import/com/example/models/User.java b/gitnexus/test/fixtures/lang-resolution/java-wildcard-import/com/example/models/User.java new file mode 100644 index 000000000..910f44884 --- /dev/null +++ b/gitnexus/test/fixtures/lang-resolution/java-wildcard-import/com/example/models/User.java @@ -0,0 +1,7 @@ +package com.example.models; + +public class User { + public void save() { + System.out.println("saving user"); + } +} diff --git a/gitnexus/test/integration/resolvers/java.test.ts b/gitnexus/test/integration/resolvers/java.test.ts index d9bae90bf..8a813525b 100644 --- a/gitnexus/test/integration/resolvers/java.test.ts +++ b/gitnexus/test/integration/resolvers/java.test.ts @@ -1,11 +1,12 @@ /** * Java: class extends + implements multiple interfaces + ambiguous package disambiguation */ -import { describe, it, expect, beforeAll } from 'vitest'; +import { describe, expect, beforeAll } from 'vitest'; import path from 'path'; import { FIXTURES, CROSS_FILE_FIXTURES, + createResolverParityIt, getRelationships, getNodesByLabel, getNodesByLabelFull, @@ -14,6 +15,8 @@ import { type PipelineResult, } from './helpers.js'; +const it = createResolverParityIt('java'); + // --------------------------------------------------------------------------- // Heritage: class extends + implements multiple interfaces // --------------------------------------------------------------------------- @@ -438,6 +441,54 @@ describe('Java variadic call resolution', () => { } expect(allDangling).toEqual([]); }); + + it('resolves 2-arg call to fixed-prefix varargs method format(int, String...) in Formatter.java', () => { + const calls = getRelationships(result, 'CALLS'); + const fmtCall = calls.find((c) => c.target === 'format' && c.source === 'run'); + expect(fmtCall).toBeDefined(); + expect(fmtCall!.targetFilePath).toBe('com/example/util/Formatter.java'); + }); + + it('0-arg call to format(int, String...) still resolves in legacy mode (arity rejection is registry-only)', () => { + // In REGISTRY_PRIMARY_JAVA=1 mode, `requiredParameterCount = 1` causes + // `javaArityCompatibility` to return 'incompatible' for 0-arg calls, + // preventing the CALLS edge. In default (legacy) mode, arity is not + // enforced so the edge is created. This test documents the legacy + // behavior; the negative assertion is a flip-blocker for registry-primary. + const calls = getRelationships(result, 'CALLS'); + const zeroArgFmtCall = calls.find((c) => c.target === 'format' && c.source === 'badCall'); + expect(zeroArgFmtCall).toBeDefined(); + }); +}); + +// --------------------------------------------------------------------------- +// Wildcard import: `import com.example.models.*` resolves to a package file +// --------------------------------------------------------------------------- + +describe('Java wildcard import resolution', () => { + let result: PipelineResult; + + beforeAll(async () => { + result = await runPipelineFromRepo(path.join(FIXTURES, 'java-wildcard-import'), () => {}); + }, 60000); + + it('parses wildcard import without errors and creates graph nodes', () => { + // The wildcard import (`import com.example.models.*`) exercises the + // directoryChild branch in resolveJavaImportTarget. Even if no IMPORTS + // edge is created (nondeterministic file selection — documented flip + // blocker), the graph must contain valid nodes for all classes. + const classes = getNodesByLabel(result, 'Class'); + expect(classes).toContain('Main'); + expect(classes).toContain('User'); + expect(classes).toContain('Order'); + }); + + it('resolves user.save() call via wildcard-imported User', () => { + const calls = getRelationships(result, 'CALLS'); + const saveCall = calls.find((c) => c.target === 'save' && c.source === 'run'); + expect(saveCall).toBeDefined(); + expect(saveCall!.targetFilePath).toBe('com/example/models/User.java'); + }); }); // --------------------------------------------------------------------------- diff --git a/gitnexus/test/integration/worker-pool.test.ts b/gitnexus/test/integration/worker-pool.test.ts index 845c1cfba..7fba8ebf5 100644 --- a/gitnexus/test/integration/worker-pool.test.ts +++ b/gitnexus/test/integration/worker-pool.test.ts @@ -354,7 +354,7 @@ describe('worker pool integration', () => { try { await expect(pool.dispatch([{ path: 'crash.ts', content: '' }])).rejects.toThrow( - /simulated startup crash|exited with code/, + /simulated startup crash|exited with code|idle timeout/, ); const warnRecords = cap.records().filter((r) => Number(r.level) >= 40 /* warn or above */); expect(warnRecords.length).toBeGreaterThan(0); From 4fa40e9881531b5a5d4c11459b188f872b6cb843 Mon Sep 17 00:00:00 2001 From: Abhigyan Patwari <126312502+abhigyanpatwari@users.noreply.github.com> Date: Tue, 12 May 2026 13:14:56 +0100 Subject: [PATCH 4/9] feat(analyze): incremental indexing (parse cache + DB writeback + scope-res short-circuit) (#1479) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * docs: incremental indexing design spec Captures the design agreed in brainstorming on 2026-05-10: - Transitive importer closure with public-surface-change optimization - Git-only change detection (non-git repos: full rebuild as today) - New default behavior; --force opts out - New hydratePhase + loadGraphFromLbug primitive - Iterative closure expansion with parseCache reuse - incrementalInProgress dirty flag for crash recovery Prior art: PR #592 (zenprocess), PR #533 (davidbeesley), PR #1146 (azeemshaik025) — referenced and credited. Co-Authored-By: Claude Opus 4.7 (1M context) * feat(communities): seed Leiden RNG for deterministic community detection The vendored Leiden algorithm defaults to Math.random for tie-breaking and randomized walks, which produces non-deterministic community assignments and modularity values across runs on the same graph. Pass a seeded mulberry32 RNG (LEIDEN_SEED=0xC0DE) so: - The same graph always produces the same partition - Modularity values are reproducible - Equivalence tests for incremental indexing can compare community assignments byte-for-byte This is foundational for the upcoming incremental-indexing feature (see docs/superpowers/specs/2026-05-10-incremental-indexing-design.md) where the correctness contract is incremental output ≡ full rebuild output. Co-Authored-By: Claude Opus 4.7 (1M context) * feat(incremental): change-detection, surface signatures, closure expansion Three new modules supporting the incremental-indexing pipeline: * core/incremental/git-diff.ts — getChangedFilesSinceCommit() unions 'git diff lastCommit HEAD' (committed) with 'git status --porcelain' (dirty tree). Renames flattened to delete(orig) + add(new). Throws LastCommitMissingError when lastCommit is gone (caller falls back to full rebuild). * core/incremental/surface.ts — extractSurfaceSignature() produces a stable hash of a file's publicly-visible symbols (functions, classes, methods, interfaces, types, heritage). Body-only edits → same hash. Signature/heritage changes → different hash. Drives the closure scoping optimization. * core/incremental/closure.ts — computeImporterClosure() iterative fixpoint: parse each closure file, extract surface, query DB importers, expand. Uses a parseCache so each file is parsed once. Generic over TParseResult so closure logic is decoupled from the pipeline's parse representation. 32 unit tests across the three modules. Tests cover edge cases: clean tree, dirty-only, mixed, renames, deletes, multi-hop cascade, cycle termination, surface invariance, etc. Co-Authored-By: Claude Opus 4.7 (1M context) * feat(lbug): loadGraphFromLbug, queryImporters, deleteAllCommunitiesAndProcesses Three new primitives in lbug-adapter.ts to support incremental indexing: * loadGraphFromLbug(graph, unchangedFilePaths) — streams all nodes for files in the set across every hydratable node table (excludes Community/Process — graph-wide, regenerated downstream). Then loads edges where both endpoints belong to loaded nodes, excluding MEMBER_OF / STEP_IN_PROCESS edges (also graph-wide). FilePaths chunked at 200 per query to keep statement size bounded on huge repos. Endpoint-level join filters by source-side filePath in the query, target-side checked JS-side via the loadedNodeIds set. * queryImporters(targetFilePath) — returns DISTINCT a.filePath where a -[IMPORTS]-> b and b.filePath = target. Powers closure expansion: when a changed file's surface signature changes, all its importers must be re-parsed. * deleteAllCommunitiesAndProcesses() — drops Community/Process nodes (and their edges via DETACH DELETE) at the start of each incremental run so the communities/processes phases regenerate them from the fully-merged graph. Required for the 'Leiden runs on full graph' correctness invariant. Co-Authored-By: Claude Opus 4.7 (1M context) * feat(pipeline): hydrate phase + parse-filter for incremental indexing Wires the incremental-indexing infrastructure into the phase-based pipeline. Three coordinated changes: * New hydratePhase (deps: structure) — loads node/edge state for files OUTSIDE ctx.options.filesToParse from the existing LadybugDB index. Runs before parse so the parse phase can produce a partial graph while downstream phases (mro, communities, processes) still see the full graph. No-op in full-rebuild mode (filesToParse unset). * PipelineOptions.filesToParse: optional ReadonlySet. When set, parse phase filters scanned files to this set; hydrate fills the complement. Set by runFullAnalysis when it detects an eligible incremental run; never set by callers directly. * gitnexus-shared PipelinePhase enum: 'hydrate' added so progress callbacks can report the new phase distinctly from 'structure'. Phase order: scan → structure → hydrate → markdown,cobol → parse → routes,tools,orm → crossFile → scopeResolution → mro → communities → processes. Communities (Leiden) still runs on the full graph, satisfying the correctness invariant. Co-Authored-By: Claude Opus 4.7 (1M context) * feat(analyze): incremental orchestrator branch + meta schema Wires incremental indexing into runFullAnalysis. Highlights: * RepoMeta schema extended: schemaVersion, surfaceSignatures, and incrementalInProgress fields. INCREMENTAL_SCHEMA_VERSION = 1. * core/incremental/file-hash.ts — v1 surface signature: SHA-256 of file content. v2 will switch to a true surface-only signature (defined in surface.ts) so body-only edits don't expand the closure. The plumbing is signature-agnostic so the swap is local. * core/incremental/orchestrator.ts — eligibility check, closure computation (uses file-hash as the surface signal), dirty-flag management, subgraph extraction, signature merge. * run-analyze.ts adds: - hasDirtyTree() check on the existing 'lastCommit==HEAD' early-exit so an uncommitted edit triggers re-index (was a coarse equality check before). - incremental branch: try incremental first; fall through to full rebuild on any setup failure or eligibility miss. - runIncrementalBranch() — opens existing DB, deletes closure-file rows + Community/Process, runs pipeline with filesToParse, writes only the changed-subgraph back, refreshes FTS, updates meta with new surfaceSignatures and clears the dirty flag. - Full-rebuild path now populates surfaceSignatures + schemaVersion in meta.json so the next run is eligible for incremental. Crash recovery: incrementalInProgress is set BEFORE any DB mutation and cleared on success by overwriting meta.json. A crash anywhere in between leaves the flag set, and the next analyze run forces a full rebuild (cheapest path back to a known-good index). v1 limitation documented: body-only edits trigger 1-hop closure expansion (content-hash signal). True surface-only optimization is deferred to v2 — see design doc for the integration path. Co-Authored-By: Claude Opus 4.7 (1M context) * fix(incremental): drop invalid --no-renames=false from git diff The flag --no-renames=false isn't valid git syntax (it's parsed as a file path). Git's default rename detection is on; removing the flag keeps that behavior. Caught while running an end-to-end smoke test against a small fixture repo: incremental setup failed with 'Command failed: git diff --name-status -z --no-renames=false ...'. After the fix, the incremental path runs cleanly: closure is computed, hydrate phase loads unchanged-file state from DB, parse phase only re-parses files in closure, and the writeback updates only changed nodes/edges. Co-Authored-By: Claude Opus 4.7 (1M context) * Revert v1 incremental indexing (5 commits) Reverts the v1 design that parsed only closure files into a fresh graph and tried to hydrate the rest from DB. Real-repo equivalence test failed: cross-file resolution operates on partial parse data (closure files only), so CALLS edges that resolve through unchanged files silently fall off. Diff against full rebuild on the same edited state: -50 nodes, -425 edges, -5 communities, -48 processes. Architecture pivot: switch to PR #533-style content-addressed parse cache. Pipeline parses every file (cache-served when possible), giving cross-file resolution full data, with DB writeback then restricted to changed-file rows. Reverts: d4b9de47 fix(incremental): drop invalid --no-renames=false f35f7634 feat(analyze): incremental orchestrator branch + meta schema bc039686 feat(pipeline): hydrate phase + parse-filter 98bb893d feat(lbug): loadGraphFromLbug, queryImporters, ... aa8d7ae3 feat(incremental): change-detection, surface signatures, closure Kept: d9e340b0 feat(communities): seed Leiden RNG (foundational) 8235ca36 docs: incremental indexing design spec (will be revised) Co-Authored-By: Claude Opus 4.7 (1M context) * feat(analyze): incremental DB writeback (Option B) Equivalence-preserving incremental analyze. The pipeline still parses every file (correctness invariant: cross-file resolution / scope resolution / MRO / community detection all need full graph data); the saving comes from selectively replacing only changed-file rows in LadybugDB instead of wiping and reloading the whole graph. How it works: * On every analyze, we hash all source files (SHA-256 of content) and store the map in meta.json.fileHashes alongside schemaVersion. * The next run loads the prior map and diffs: - changed: content hash differs → file's DB rows replaced. - added: not in prior map → file's DB rows inserted. - deleted: in prior map but not on disk → file's DB rows dropped. * If the diff is non-empty AND no --force / no schema mismatch / no dirty flag, take the incremental path: - Set incrementalInProgress dirty flag (BEFORE any DB mutation). - Open existing DB (no wipe). - deleteNodesForFile() for each changed/added/deleted file. - deleteAllCommunitiesAndProcesses() — Leiden regenerates these. - extractChangedSubgraph() from the in-memory ctx.graph: nodes whose filePath is in the writable set + Community + Process + edges with at least one endpoint in the writable set (edges entirely between hydrated unchanged nodes are skipped — already in DB). - loadGraphToLbug() on the subgraph. Unchanged-file rows in DB untouched. - Recreate FTS indexes. - Update meta with new fileHashes; clear dirty flag. * Otherwise full-rebuild path runs as before. Crash recovery: incrementalInProgress is the dirty flag. Set before destructive ops; cleared on success. Set on next-run startup → forces full rebuild (cheapest path back to known-good). Other changes: * Dirty-tree gate on the existing 'lastCommit==HEAD' early-return: uncommitted edits no longer slip through as 'already up to date'. * deleteAllCommunitiesAndProcesses helper in lbug-adapter. * Skip the embedding cache+restore cycle when willTryIncremental is true — embeddings stay in DB; re-inserting them would PK-conflict. End-to-end equivalence verified on this repo (993 files, 24K nodes): incremental run produces byte-identical {nodes, edges, clusters, flows} to a full rebuild from the same edited state. Speedup is currently modest (~5% on this repo) because the parse phase still runs in full. Parse-cache integration is a separate follow-up that composes cleanly on top of this work. See docs/superpowers/specs/2026-05-10-incremental-indexing-design.md. Co-Authored-By: Claude Opus 4.7 (1M context) * feat(analyze): chunk-level parse cache for full incremental speedup Composes with the incremental DB writeback (commit 27f3b49d) to deliver the major-speedup half of incremental indexing. Previously, the parse phase ran in full on every analyze; the speedup came purely from selective DB rewriting. With this commit the parse phase also reuses prior tree-sitter output for chunks whose contents haven't changed. How it works: * Cache layer (gitnexus/src/storage/parse-cache.ts): - File: /.gitnexus/parse-cache.json. Versioned, atomic write. - Key: chunk content hash = sha256(sorted(filePath:fileContentHash for each file in chunk)). - Value: ParseWorkerResult[] (raw worker output for the chunk, pre-merge). - Granularity: per chunk (~20MB byte-budget). A change to one file invalidates only its chunk — typically 1 of ~50 on a 1000-file repo (~98% cache hit ratio on a small edit). * Worker contract (gitnexus/src/core/ingestion/parsing-processor.ts): - Extracted the chunk-result merge loop into a public mergeChunkResults() so the same logic applies to live worker output AND replayed cache entries. - processParsingWithWorkers / processParsing accept an optional outRawResults out-parameter that captures worker output before merging — used by parse-impl to populate the cache after a miss. * Parse phase wiring (parse-impl.ts): - For each chunk, compute its content hash (after reading file contents). Cache hit → mergeChunkResults() on cached results, skip the worker dispatch entirely. Cache miss → run workers normally, capture raw results, store under the chunk hash. - Cache mutations happen in-place on the ParseCache passed via PipelineOptions.parseCache. * Lifecycle (run-analyze.ts): - loadParseCache() before pipeline runs. - Cache passed via runPipelineFromRepo's PipelineOptions. - saveParseCache() after the pipeline + DB writeback succeed. Equivalence verified on this repo (993 files, 24K nodes): Cold (no cache, full work): 141.1s Warm cache + 1-file edit, incremental: 63.6s ← 55% speedup Warm cache + 1-file edit, --force: 71.6s ← 49% speedup All three runs produce byte-identical {nodes, edges, clusters, flows}. The cache survives --force (content-addressed = always correct), so even forced rebuilds get the parse-skip benefit. Why chunk-level rather than per-file: workers process sub-batches and emit aggregated ParseWorkerResults. Per-file granularity would require restructuring the worker contract; chunk-level captures most of the practical speedup with no worker-side changes. Co-Authored-By: Claude Opus 4.7 (1M context) * perf(parse-impl): smaller default chunk budget (20MB→2MB) for cache granularity The parse cache is keyed at chunk granularity. With the previous 20MB budget, a typical mid-size repo (e.g. this worktree at 9MB total parseable source) fits in a single chunk — meaning ANY file change invalidates the whole chunk and re-parses every file. 2MB default produces ~5x more chunks on the same input, so a one-file edit invalidates ~1/N of cached chunks instead of the whole thing. Cold-run overhead from more chunks is <5% (one extra serialization pass per chunk). Override via GITNEXUS_CHUNK_BYTE_BUDGET env var for benchmarking. Measured on this repo (~9MB / 887 parseable files): Cold (no cache): 143s Warm cache, no source changes: 2s (early-return) Warm cache + 1-file edit: 81s (~43% off cold) Speedup is bounded by the scopeResolution phase (~58s flat regardless of parse cache) and by GitNexus's own auto-writes during analyze (AGENTS.md / .claude/skills/ etc. mutate between runs and invalidate chunks containing them). Both are addressable in follow-ups. Co-Authored-By: Claude Opus 4.7 (1M context) * perf(scope-resolution): reuse worker-produced ParsedFile + stabilize chunk order Two compounding optimizations that drop warm-cache analyze from ~134s to ~38s on a 1000-file repo (72% faster), and cold rebuild from ~143s to ~86s (40% faster) by short-circuiting work that was previously re-done. 1. SCOPE-RESOLUTION: REUSE WORKER PARSEDFILE Previously, the scope-resolution phase re-parsed every file with tree-sitter on the main thread (~58s on a 1000-file repo) because worker-produced tree-sitter Trees can't cross the worker MessageChannel. But the worker ALSO produces a artifact via , which structured-clones fine — and it's exactly what scope-resolution would re-derive. Threading those ParsedFiles through the parse phase () into ( map) lets scope- resolution skip its extract loop on a per-file basis. The fast path is bounded only by per file (cheap graph mutation). On this repo: scopeResolution went from 58s → 5s. 2. MAP-PRESERVING PARSE-CACHE SERIALIZATION is a which JSON.stringify collapses to . The first attempt at threading parsedFiles through the parse cache crashed at runtime with "importerModule.typeBindings is not iterable" because cached entries came back as plain objects. Added a JSON replacer/reviver pair in parse-cache.ts that round-trips Map and Set instances through tagged plain objects (). Symmetric: save uses replacer, load uses reviver. 3. STABLE CHUNK ORDERING The byte-budget chunker walked files in filesystem-scan order, which on Windows isn't guaranteed to be stable across runs. Even with identical source content, two scans could place files in different chunks, shifting chunk hashes and causing 100% parse-cache misses. Added a deterministic alphabetical sort on before chunking. Chunk membership is now stable across runs, so a single-file edit invalidates exactly one chunk, not all of them. Measured on this repo (993 files, 24K nodes): Cold rebuild: 86s (was 143s) Warm cache, no source changes: 3s (early-return) Warm cache + 1-file edit: 38s (was 134s) Co-Authored-By: Claude Opus 4.7 (1M context) * docs(incremental): update spec + AGENTS.md + GUARDRAILS.md for shipped design - Rewrite docs/superpowers/specs/2026-05-10-incremental-indexing-design.md to describe the architecture that actually shipped (parse cache + incremental DB writeback + scope-resolution short-circuit), with the v1 hydrate-phase post-mortem preserved as historical context. - AGENTS.md "Keeping the Index Fresh" section: note that incremental is the new default and --force is the explicit opt-out; mention the parse-cache file location and that it's safe to delete. - GUARDRAILS.md Signs: add an "Index seems corrupt or incremental is misbehaving" entry pointing users to --force as the manual escape hatch (the dirty flag handles automatic recovery). Co-Authored-By: Claude Opus 4.7 (1M context) * chore(autofix): apply prettier + eslint fixes via /autofix command * fix(incremental): bugbot review + CI test failures Bugbot (PR #1479): - Medium: pruneCache was exported but never called -> cache grew unbounded. Wire pruneCache into run-analyze before saveParseCache, using a transient usedKeys Set on ParseCache that the parse phase populates as it processes chunks. - Low: willTryIncremental (pre-pipeline) and isIncremental (post-pipeline) could desync, silently dropping embeddings on mispredicted runs. Removed the prediction; the embedding cache now loads unconditionally when shouldLoadCache is true. The re-insert step gates on the actual isIncremental value to avoid PK-conflicts when the incremental-writeback path keeps DB rows. CI test failures: - cli-e2e #1169 + run-analyze.test.ts #1233: my dirty-tree gate on the lastCommit==HEAD early-return saw GitNexus's own auto-generated outputs (.claude/, .cursor/, AGENTS.md, CLAUDE.md) as dirty, perpetually defeating the up-to-date fast path. Extended the pathspec exclusion to cover all auto-gen outputs, not just .gitnexus/. - ruby field-type disambig: my chunk-stability sort exposed a pre-existing order-dependency in Ruby cross-file resolution (`user.address.save -> Address#save` only resolves correctly when user.rb parses before address.rb in some configurations). Removed the sort. Filesystem ordering is stable enough in practice that the parse cache still hits the common case; the pre-existing fragility is left for a separate fix. - pipeline-graph-golden: regenerated. Seeded Leiden RNG produces a partition different from the previous Math.random snapshot. - staleness `parallel calls` was a CI timing flake; passes locally. Co-Authored-By: Claude Opus 4.7 (1M context) * fix(incremental): re-insert cached embeddings on incremental path Bugbot re-review caught: deleteNodesForFile cascades to the CodeEmbedding table (DELETE WHERE e.nodeId STARTS WITH ...), so changed-file embedding rows are wiped along with their nodes. The previous fix gated re-insert on `!isIncremental`, which silently dropped those embeddings — a regression versus the full-rebuild path's "preserve embeddings by default" guarantee. Remove the `!isIncremental` gate. The per-batch try/catch already handles the unchanged-file PK-conflict case ("some may fail if node was removed, that's fine") with the same semantics, so re-inserting the full cached set on incremental works: - changed-file rows: deleted, then re-inserted from cache (preserved) - unchanged-file rows: still in DB, re-insert PK-conflicts and is silently ignored (existing rows are correct) Cost: re-inserting ~24K embeddings on incremental when only a few files changed — most are no-op conflicts. Bounded by batch size of 200; ~3-5s overhead. Worth it for correctness. Co-Authored-By: Claude Opus 4.7 (1M context) * fix(incremental): address Claude+Bugbot review findings + remove design doc Addresses CHANGES_REQUESTED review on PR #1479: 1. Remove docs/superpowers/specs/2026-05-10-incremental-indexing-design.md per maintainer request. 2. BLOCKER (Claude Finding 1, Bugbot Round 3): Stale cross-file edges between unchanged files. extractChangedSubgraph excluded edges where both endpoints were unchanged-file nodes — when a barrel/re-export file changes, cross-file resolution may update CALLS edges between two unchanged files that would then be silently lost. Fix: 1-hop importer-closure expansion of the writable set in run-analyze.ts. Before deleting/rewriting rows, query DB for importers of every changed/deleted file and add them to the writable set. Their nodes get deleted+rewritten too, so cross-file's refined edges land in the DB. Re-added queryImporters to lbug-adapter.ts. 3. BLOCKER (Claude Finding 3): Parse cache key omitted parser version. After a GitNexus upgrade, the cache silently replays pre-upgrade ParseWorkerResults against the new schema → wrong CALLS/IMPORTS/ scope edges with no visible signal. Fix: PARSE_CACHE_VERSION now embeds the gitnexus npm package version (read at module load via createRequire on package.json). Format: `${SCHEMA_BUMP}+${PKG_VERSION}` e.g. "1+1.6.4". Any release that bumps package.json automatically invalidates the on-disk cache. Mismatched versions fall through to an empty cache (next save overwrites with the new version baked in). 4. BLOCKER (Claude Finding 2): No automated tests for incremental behavior. Added 28 unit tests across 3 files: - incremental-file-hash.test.ts (10 tests) diffFileHashes classification, computeFileHash determinism, computeFileHashes batch / missing-file tolerance, sorted output. - incremental-parse-cache.test.ts (12 tests) computeChunkHash stability and order-independence, version prefix format, pruneCache, load/save round-trip on empty / missing / corrupt / version-mismatched files, AND a Map/Set round-trip test that pins the JSON replacer/reviver behaviour (without it, ParsedFile.scopes[*].typeBindings collapses to {} and downstream `.get()` / iteration throws). - incremental-subgraph-extract.test.ts (6 tests) writable-set node inclusion, Community/Process always kept, edge inclusion when at least one endpoint is writable, MEMBER_OF edges via graph-wide endpoints, empty subgraph case. 5. Medium (Claude Finding 6): AGENTS.md "Keeping the Index Fresh" said "only changed files are re-parsed." Imprecise — the pipeline parses every file every run; the cache skips tree-sitter for chunks whose contents haven't changed. Reworded to match the design doc. Test plan still expects: [x] Typecheck clean [x] All 28 new unit tests pass [x] All previously-failing tests still pass on the rebased branch [x] Equivalence verified locally (incremental ≡ --force, byte-identical stats on this repo) Co-Authored-By: Claude Opus 4.7 (1M context) * fix(incremental): round 3 review feedback — bounded BFS, atomic meta, integration test, docs Addresses remaining findings on PR #1479 from Claude's re-review of commit ad7bd31 + verifies the outstanding Bugbot HIGH severity. 1. F1 — Transitive importer expansion (Claude, was Medium-but-noted). Previous 1-hop importer expansion missed barrel re-export chains (A imports C, C re-exports B; when B changes, only C was pulled in — A was left with potentially-stale CALLS edges to refined targets). Replaced the single pass with a bounded BFS over the IMPORTS graph (depth ≤ 4). Catches nested barrel pyramids without ballooning into a near-full rebuild on monorepos with deep re-export trees. `--force` remains the escape hatch documented in GUARDRAILS.md for cases that exceed the bound. 2. F2 — Integration test for incremental orchestration (Claude, BLOCKER, DoD §2.7). The unit tests added in ad7bd31 covered `diffFileHashes`, `extractChangedSubgraph`, `computeChunkHash`, `pruneCache`, and the Map/Set JSON round-trip — but none of them exercised the real `runFullAnalysis` orchestration. Added gitnexus/test/unit/ incremental-orchestration.test.ts with four end-to-end tests against a real git-initialized fixture repo + real LadybugDB: a. First run populates fileHashes + schemaVersion and clears incrementalInProgress on success. b. Second run on unchanged state takes the alreadyUpToDate fast path (early-return). c. Second run after a source edit takes the incremental path (not full rebuild) and rotates fileHashes for the touched file while keeping the dirty flag cleared. d. A pre-set incrementalInProgress flag forces a full rebuild that clears it (crash-recovery wire). These would catch any regression that wires `isIncremental` from a pre-pipeline prediction (the Bugbot finding from commit 5eb0597) or accidentally re-gates the embedding re-insert on `!isIncremental` (the Bugbot finding from commit 60c10f1). 3. F3 — GUARDRAILS.md docs accuracy (Claude, Low). Line 33 still said "only changed files are re-parsed" — AGENTS.md was already corrected in ad7bd31 but GUARDRAILS.md was missed. Reworded to match. 4. F5 — Atomic saveMeta (Claude, Medium; vvladescu-tb fork). The dirty flag (`incrementalInProgress`) travels through meta.json. A crash mid-write would leave a corrupt meta.json that `loadMeta` would silently treat as "no prior index", losing the flag and skipping recovery. Switched to tmp-file + rename matching saveParseCache. 5. Bugbot's "Subgraph edges reference nodes absent from subgraph" (HIGH severity). Verified as FALSE POSITIVE: `getNodeLabel` in lbug-adapter.ts derives labels from the node-ID string (parses the table prefix), not from the in-memory graph. The CSV generator writes (src_id, dst_id, type) rows without consulting node objects; `splitRelCsvByLabelPair` routes by ID-derived label; `COPY ... (from=X, to=Y)` resolves both endpoints against the live LadybugDB where unchanged-file nodes still exist. No fix needed. All 213 tests pass locally (including the 4 new integration tests and the previously-failing CI tests). Co-Authored-By: Claude Opus 4.7 (1M context) * fix(incremental): address Bugbot round-4 findings (added-file shadow seed + dedupe) Bugbot review on commit e23e4400 surfaced two new findings against the incremental writeback in run-analyze.ts: HIGH — Incremental BFS misses importers of newly added files. queryImporters() reads the pre-pipeline DB. For a NEWLY ADDED file there are no IMPORTS rows pointing to it yet, so unchanged files whose pre-existing import statements now resolve to the newcomer keep stale CALLS edges pointing at the OLD resolution target. LOW — Deleted files double-counted in filesToDelete. hashDiff.deleted entries can reappear in writableFiles via the BFS expansion (queryImporters can return a now-deleted path), so deleteNodesForFile() ran twice for the same file. Fixes: - Add gitnexus/src/core/incremental/shadow-candidates.ts: derive the pre-existing file paths whose JS/TS module-resolution claim an added file can steal. Pattern catalogue: same-basename/ different-extension, bare-file-beats-directory-index, and directory-index-beats-bare-file. Emit both POSIX and Windows separators because the prior fileHashes map may have been written from either OS. - In run-analyze.ts, seed the BFS frontier with shadow candidates that exist in the prior meta.fileHashes. Their importers — found via queryImporters — get pulled into the writable set so their CALLS edges re-resolve against the new file. - Dedupe filesToDelete via Set to avoid the double-call. Tests: gitnexus/test/unit/incremental-shadow-candidates.test.ts — 8 cases covering each shadow pattern, separator handling, .d.ts as a single extension token, deduplication, and the no-self-shadow invariant. All 40 incremental tests (file-hash, parse-cache, subgraph-extract, shadow-candidates, orchestration) pass locally. Note on the third Bugbot finding ("Subgraph edges reference nodes absent from subgraph"): re-anchored from a prior review pass — the code at subgraph-extract.ts:48 is unchanged. Already verified as a false positive: getNodeLabel parses labels from ID strings, CSV write is by ID, and COPY resolves against the live DB. * chore(autofix): apply prettier + eslint fixes via /autofix command * test(incremental): exact-equality stats invariant + analyze ≡ analyze --force Addresses the only remaining Claude production-readiness review finding on PR #1479 (Low-Medium, test-quality only — Claude itself said it does NOT block merge, but the central PR claim "incremental ≡ full rebuild" deserves explicit CI coverage rather than implicit trust). Changes to gitnexus/test/unit/incremental-orchestration.test.ts: 1) Tighten the existing "comment-only edit takes incremental path" test. - Replace toBeGreaterThan(0) bounds assertions on stats.files and stats.nodes with exact toBe(firstMeta) per-field equality across files / nodes / edges / communities / processes. DoD §2.7 calls out bounds-only assertions as masking regressions that drop half the graph; this swap closes that gap. - Rationale: a comment-only edit must change the file content hash (driving the incremental path) without changing any graph data. Therefore every stat MUST be identical to the first run. Anything else is a regression. 2) New test: incremental output is byte-equivalent to a full rebuild. - Run analyze → comment-only edit → analyze (incremental writeback) → analyze --force (full rebuild from same on-disk state). - Assert files / nodes / edges / communities / processes are exactly equal across the incremental and the --force passes. - This is the PR's central correctness contract, now proven by a test that exercises the real runtime path end-to-end against a real on-disk LadybugDB. All 5 orchestration tests pass locally (52s), including the new equivalence test — every stat field matches exactly between incremental and --force on the mini-repo fixture. tsc --noEmit clean. * fix(incremental): F1 cross-file edge consistency + F4 stable chunk sort + unit coverage (#1511) Patch addressing two of the still-open changes-requested findings on PR #1479, rebased onto the current feat/incremental-indexing head. F3 (parser fingerprint in the cache key), F5 (atomic saveMeta), and F6 (AGENTS.md phrasing) were already handled on the branch, so the corresponding parts of the original patch were dropped as redundant. F1 (Blocker) — Cross-file edges between unchanged files Adds `computeEffectiveWriteSet(graph, toWriteSet)` to subgraph-extract.ts: a single pass over the new graph's edges that pulls the unchanged-side file of every writable-boundary-crossing edge into the write set. run-analyze composes it ON TOP of the existing importer-BFS expansion and feeds the combined set to BOTH `deleteNodesForFile` and `extractChangedSubgraph`, so the delete cascade and the writeback subgraph cover identical files (asymmetry would leave stale rows or PK-conflict at COPY time). The BFS reads IMPORTS from the pre-pipeline DB (catches files that *stopped* importing a changed file); the edge walk reads the new graph (catches refined CALLS edges the pre-run DB couldn't predict, e.g. a barrel re-export shifting a symbol from B to D). `extractChangedSubgraph` stays a pure filter — all expansion is the orchestrator's job. F4 (Medium) — Restore alphabetical chunk sort `parseableScanned` is sorted before chunking. Filesystem-scan order isn't stable enough across runs/platforms (notably macOS APFS) to keep chunk hashes consistent, so the parse cache thrashes without it. The pre-existing Ruby cross-file resolution order-dependency the old comment cited is independent — the sort surfaces it but doesn't cause it; tracked separately rather than leaving the cache cold. Tests — incremental-subgraph-extract.test.ts Locks the F1 invariants: `extractChangedSubgraph` is a pure filter (includes only the set it's given, plus graph-wide nodes; edges fire on one writable endpoint), and `computeEffectiveWriteSet` covers the barrel-re-export scenario, the symmetric edge-into- changed-file case, the no-boundary-crossed no-op, graph-wide-node edges, and input-immutability. Supersedes the prior extractChangedSubgraph-only test file on the branch. Co-authored-by: Val Vladescu * chore(autofix): apply prettier + eslint fixes via /autofix command * fix(call-processor): register properties in pre-pass to fix order-dependent field type disambiguation + regenerate golden snapshot Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/2d66666f-861c-432e-a4b0-11f2aefca98a Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * fix(call-processor): port worker-path property enrichment into the sequential pre-pass Copilot's pre-pass in 8184439 fixed the Ruby attr_accessor order-dependence, but it copied the OLD in-loop registration logic, not the canonical worker path in parse-worker.ts. That left the sequential and worker paths emitting non-identical Property nodes/symbols for the same source — silently breaking the `incremental ≡ --force` invariant the moment a repo crosses the worker threshold between runs. Two concrete divergences are closed here: * Node id: worker keys Property as `${file}:${className}.${propName}` (qualified). Pre-pass was using `${file}:${propName}` (unqualified). Same source produced different graph ids depending on which path ran. * Field metadata: worker enriches each routed property with `provider.fieldExtractor` + `getFieldInfo`, falling back to `routedFieldInfo.type` for `declaredType` when the routing payload lacks one (e.g. types discovered from `@address = Address.new` ctor assignments rather than YARD `@return [Type]`), and propagates `visibility` / `isStatic` / `isReadonly`. Pre-pass did none of this, so on the sequential path `resolveFieldAccessType` failed to walk chains where the type only came from the FieldExtractor. The pre-pass now mirrors parse-worker.ts:1803-1898 verbatim, with one deliberate difference: the FieldInfo cache is scoped to a single `processCalls` invocation rather than module-level (the worker process is short-lived; the main thread is not, and a module-level cache would leak state between analyze runs). Also drops the now-stale "Defer resolution: Ruby attr_accessor properties are registered during this same loop" comment on `pendingWrites.push` — the rationale is no longer accurate after Copilot's pre-pass, but the deferral is still needed so write-access tracking sees inference that completes during the main loop. Comment updated to reflect that. Verification: * `tsc --noEmit`: 0 errors * test/unit (call-processor, call-routing, field-extraction, ruby-self-call): 224 passing * test/integration (ruby, ruby-sequential-mixin, pipeline-graph-golden): 137 passing Co-Authored-By: Claude Opus 4.7 (1M context) * fix(call-processor): key fieldInfoCache by filePath:startIndex, not raw byte offset Claude's review of 255bdf6 caught a real collision in the FieldInfoCache I added: keying by `classNode.startIndex` alone is a per-file byte offset, so two files that both begin with a class at byte 0 — extremely common in Ruby / Python, where files frequently open with `class Foo`, `module Foo` — collide on the same cache entry. The second file's `getFieldInfo` then returns the first file's FieldInfo map, producing wrong `declaredType` / `visibility` / `isReadonly` on its properties. Same shape as the bug that already exists in parse-worker.ts:377 (also keyed by `classNode.startIndex` in a module-level map, persistent across files processed by the same worker). Fixing the symmetric pre-existing leak in parse-worker.ts is a separate, scoped follow-up — left out of this commit to keep the fix minimal and reviewable. Cache map and key are now both string-typed. Composite key `${context.filePath}:${classNode.startIndex}` keeps the within-file hit rate (one FieldExtractor.extract() per class regardless of how many `attr_accessor` lines it has) while eliminating cross-file aliasing. Verification on the patched HEAD: * `tsc --noEmit`: 0 errors * test/unit (call-processor, call-routing, field-extraction, ruby-self-call): 224 passing * test/integration (ruby, ruby-sequential-mixin, pipeline-graph-golden): 137 passing Co-Authored-By: Claude Opus 4.7 (1M context) --------- Co-authored-by: Claude Opus 4.7 (1M context) Co-authored-by: Gergő Magyar Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Val Vladescu Co-authored-by: Val Vladescu Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> --- AGENTS.md | 7 +- GUARDRAILS.md | 8 +- .../src/core/incremental/shadow-candidates.ts | 76 ++++ .../src/core/incremental/subgraph-extract.ts | 123 ++++++ gitnexus/src/core/ingestion/call-processor.ts | 224 ++++++++-- .../src/core/ingestion/community-processor.ts | 19 + .../src/core/ingestion/parsing-processor.ts | 177 ++++---- .../ingestion/pipeline-phases/parse-impl.ts | 175 ++++++-- .../core/ingestion/pipeline-phases/parse.ts | 14 + gitnexus/src/core/ingestion/pipeline.ts | 13 + .../scope-resolution/pipeline/phase.ts | 15 +- .../scope-resolution/pipeline/run.ts | 51 ++- gitnexus/src/core/lbug/lbug-adapter.ts | 71 ++++ gitnexus/src/core/run-analyze.ts | 383 ++++++++++++++++-- gitnexus/src/storage/file-hash.ts | 104 +++++ gitnexus/src/storage/parse-cache.ts | 213 ++++++++++ gitnexus/src/storage/repo-manager.ts | 47 ++- .../mini-repo/expected-graph.json | 2 +- .../test/unit/incremental-file-hash.test.ts | 124 ++++++ .../unit/incremental-orchestration.test.ts | 263 ++++++++++++ .../test/unit/incremental-parse-cache.test.ts | 243 +++++++++++ .../incremental-shadow-candidates.test.ts | 75 ++++ .../unit/incremental-subgraph-extract.test.ts | 169 ++++++++ 23 files changed, 2409 insertions(+), 187 deletions(-) create mode 100644 gitnexus/src/core/incremental/shadow-candidates.ts create mode 100644 gitnexus/src/core/incremental/subgraph-extract.ts create mode 100644 gitnexus/src/storage/file-hash.ts create mode 100644 gitnexus/src/storage/parse-cache.ts create mode 100644 gitnexus/test/unit/incremental-file-hash.test.ts create mode 100644 gitnexus/test/unit/incremental-orchestration.test.ts create mode 100644 gitnexus/test/unit/incremental-parse-cache.test.ts create mode 100644 gitnexus/test/unit/incremental-shadow-candidates.test.ts create mode 100644 gitnexus/test/unit/incremental-subgraph-extract.test.ts diff --git a/AGENTS.md b/AGENTS.md index 60317d73d..1346facc9 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -149,11 +149,16 @@ Indexed as **GitNexus** (4325 symbols, 10556 relationships, 300 execution flows) ## Keeping the Index Fresh ```bash -npx gitnexus analyze # basic refresh; preserves any existing embeddings +npx gitnexus analyze # incremental by default; preserves embeddings +npx gitnexus analyze --force # full rebuild from scratch (opt out of incremental) npx gitnexus analyze --embeddings # also generate embeddings for new/changed nodes npx gitnexus analyze --drop-embeddings # explicit opt-in to wipe existing embeddings ``` +`analyze` runs **incrementally by default**. The pipeline still parses every file every run (cross-file resolution requires it), but tree-sitter parsing is **served from a content-addressed cache** at `.gitnexus/parse-cache.json` for chunks whose file contents haven't changed since the last run. Only changed-file rows (and their importers) are rewritten in LadybugDB; unchanged-file rows are preserved. Output is byte-equivalent to a full rebuild. Pass `--force` to wipe and re-index from scratch (e.g., to recover from a corrupt index, or after upgrading GitNexus). + +The parse cache key is **content-addressed and version-tagged**: it survives `--force` runs, and is automatically invalidated by a `gitnexus` package upgrade (so a new tree-sitter grammar doesn't silently replay stale parse output). Safe to delete `.gitnexus/parse-cache.json` at any time — it'll be rebuilt on the next analyze. + Check `.gitnexus/meta.json` `stats.embeddings` (0 = none). A plain `analyze` no longer drops existing vectors — pass `--drop-embeddings` to wipe. > Claude Code: PostToolUse hook detects a stale index after `git commit` and `git merge` and prompts the agent to run `analyze`. The hook does not invoke `analyze` itself. diff --git a/GUARDRAILS.md b/GUARDRAILS.md index 1cc032759..c09f0319a 100644 --- a/GUARDRAILS.md +++ b/GUARDRAILS.md @@ -30,9 +30,15 @@ Format: **Trigger → Instruction → Reason**. Append new Signs when the same m ### Stale graph after edits - **Trigger:** MCP warns index is behind `HEAD`, or search doesn't match latest commit. -- **Do:** `npx gitnexus analyze` (plus `--embeddings` if used). +- **Do:** `npx gitnexus analyze` (plus `--embeddings` if used). Runs incrementally by default — the pipeline parses every file every run (cross-file resolution requires it), but tree-sitter dispatch is skipped for unchanged file chunks via the content-addressed cache, and only changed-file rows (plus their importers, transitively) are rewritten in LadybugDB. - **Why:** Tools query LadybugDB from last analyze; git changes are invisible until re-indexed. +### Index seems corrupt or "incremental" is misbehaving + +- **Trigger:** `analyze` produces unexpected results, or `meta.json.incrementalInProgress` is set, or the index is in a half-state after a crash. +- **Do:** `npx gitnexus analyze --force` to rebuild from scratch. The dirty-flag check forces this automatically when a previous incremental run didn't complete cleanly, but `--force` is the manual escape hatch. Safe to delete `.gitnexus/parse-cache.json` at any time — content-addressed, will be regenerated. +- **Why:** Incremental writeback is selective DB row replacement; if the on-disk state is inconsistent for any reason, a full rebuild is the cheapest path back to a known-good index. + ### Embeddings vanished after analyze - **Trigger:** Semantic search quality drops; `stats.embeddings` in `meta.json` is 0 after refresh. diff --git a/gitnexus/src/core/incremental/shadow-candidates.ts b/gitnexus/src/core/incremental/shadow-candidates.ts new file mode 100644 index 000000000..415a6d9df --- /dev/null +++ b/gitnexus/src/core/incremental/shadow-candidates.ts @@ -0,0 +1,76 @@ +/** + * Shadow-candidate path derivation for incremental indexing. + * + * Background — Bugbot review on PR #1479: + * queryImporters() on a NEWLY ADDED file returns 0 importers in the + * pre-pipeline DB, because the new file's IMPORTS rows haven't been + * written yet. But pre-existing files may have IMPORTS edges that + * *resolved to a sibling path*, and the newcomer can now steal that + * resolution under standard JS/TS module-resolution rules. Without + * pulling those pre-existing files into the writable set, their + * stale CALLS edges remain pointing at the OLD resolution target. + * + * Given an added file path, this helper enumerates the pre-existing + * file paths whose import-resolution claim the newcomer can steal. + * Caller filters the candidates against the prior-run `fileHashes` + * map so we only query importers of paths that actually existed. + * + * Shadow patterns covered (resolution-priority-aware): + * + * (a) Same basename, different extension — + * added `foo/bar.ts` shadows `foo/bar.{tsx,js,jsx,mjs,cjs,d.ts}`. + * (b) Bare-file beats directory-style index — + * added `foo/bar.ts` shadows `foo/bar/index.{ts,tsx,...}`. + * (c) Directory-index beats bare-file — + * added `foo/index.ts` shadows `foo.{ts,tsx,...}` (rare but real, + * e.g. converting a single-file module into a directory module). + * + * Resolution-order priority is conservatively wide: we enumerate ALL + * common extensions because we don't know which the importer actually + * specified, and over-seeding is harmless (extra BFS work, but the + * subgraph extract still gates write-back by file membership). + * + * Cross-platform path separators: candidates are emitted with both `/` + * and `\` for shadow pattern (b), since the caller's prior fileHashes + * map may use either depending on the OS that wrote it. + */ + +const SHADOW_EXTS = ['.d.ts', '.tsx', '.ts', '.jsx', '.js', '.mjs', '.cjs']; + +/** + * Enumerate pre-existing paths whose import-resolution `added` can steal. + * + * @param added — repo-relative path of a newly-added file + * @returns deduplicated list of candidate paths (NOT filtered against + * any known-files set — caller does that) + */ +export const shadowCandidatesFor = (added: string): string[] => { + const ext = SHADOW_EXTS.find((e) => added.endsWith(e)); + if (!ext) return []; + + const noExt = added.slice(0, -ext.length); + const out = new Set(); + + // (a) Same basename, different extension. + for (const alt of SHADOW_EXTS) { + if (alt !== ext) out.add(noExt + alt); + } + + // (b) Bare file beats sibling directory-style index. + for (const idx of SHADOW_EXTS) { + out.add(`${noExt}/index${idx}`); + out.add(`${noExt}\\index${idx}`); + } + + // (c) New `foo/index.ext` shadows old `foo.ext`. + const idxSuffixSlash = '/index'; + const idxSuffixBack = '\\index'; + let dir: string | null = null; + if (noExt.endsWith(idxSuffixSlash)) dir = noExt.slice(0, -idxSuffixSlash.length); + else if (noExt.endsWith(idxSuffixBack)) dir = noExt.slice(0, -idxSuffixBack.length); + if (dir !== null) { + for (const alt of SHADOW_EXTS) out.add(dir + alt); + } + + return [...out]; +}; diff --git a/gitnexus/src/core/incremental/subgraph-extract.ts b/gitnexus/src/core/incremental/subgraph-extract.ts new file mode 100644 index 000000000..71fe656be --- /dev/null +++ b/gitnexus/src/core/incremental/subgraph-extract.ts @@ -0,0 +1,123 @@ +/** + * Subgraph extraction for incremental DB writeback. + * + * Given the FULL ctx.graph produced by the pipeline (all files parsed, + * all phases run) and the set of file paths whose DB rows must be + * replaced, produce a smaller KnowledgeGraph that contains: + * + * - Every node whose `properties.filePath` is in `toWriteSet`. + * - Every graph-wide node (Community, Process) — these are regenerated + * each run by the communities/processes phases and must be fully + * rewritten. + * - Every relationship where AT LEAST ONE endpoint is in the writable + * set above. Relationships entirely between unchanged-file nodes + * are skipped — their rows are still in the DB and re-inserting + * them would PK-conflict at COPY time. + * + * The resulting subgraph is what gets passed to `loadGraphToLbug` after + * the orchestrator has deleted the corresponding DB rows. Hydrated + * unchanged-file rows are never touched in the DB. + * + * # Cross-file edge consistency (Finding 1) + * + * `extractChangedSubgraph` intentionally does NOT expand the set it is + * given — expansion is the orchestrator's job, so the SAME expanded set + * can be fed to both `deleteNodesForFile` and this function (asymmetry + * between the delete set and the write set silently corrupts the DB). + * `computeEffectiveWriteSet` below performs the boundary-crossing 1-hop + * walk; the orchestrator composes it with its importer-BFS expansion and + * passes the result here. + * + * Why the 1-hop walk is needed: consider a barrel re-export change — + * file C (a barrel) shifts `export { foo } from './b'` to + * `export { foo } from './d'`. After scope resolution, file A's CALLS + * edge to `foo` resolves to D instead of B, even though A's content is + * byte-for-byte identical: + * + * - Old A→B edge survives in DB (neither A nor B is changed → not deleted) + * - New A→D edge is missing (neither A nor D in writable set → skipped) + * + * Pulling the unchanged-side file of every writable-boundary-crossing + * edge into the write set fixes both halves: the orchestrator's + * `DETACH DELETE` cleans up the stale unchanged-side rows, and the new + * cross-file edges land because at least one endpoint is now writable. + * + * Limitation (documented): if a file X *stopped* importing from a + * changed file C, X has no edge to C in the new graph, so this 1-hop + * walk doesn't catch it. The orchestrator's importer-BFS (which reads + * IMPORTS from the pre-pipeline DB) covers that case instead. + */ + +import type { GraphNode, GraphRelationship } from 'gitnexus-shared'; +import { createKnowledgeGraph } from '../graph/graph.js'; +import type { KnowledgeGraph } from '../graph/types.js'; + +const isGraphWide = (label: string): boolean => label === 'Community' || label === 'Process'; + +/** + * Build a Map for every File-bound node in the graph. + * Graph-wide nodes (Community/Process) have no filePath and are filtered. + */ +const indexNodeFilePaths = (fullGraph: KnowledgeGraph): Map => { + const idx = new Map(); + fullGraph.forEachNode((n: GraphNode) => { + const fp = n.properties?.filePath as string | undefined; + if (fp) idx.set(n.id, fp); + }); + return idx; +}; + +export const extractChangedSubgraph = ( + fullGraph: KnowledgeGraph, + toWriteSet: ReadonlySet, +): KnowledgeGraph => { + const sub = createKnowledgeGraph(); + const writableNodeIds = new Set(); + + fullGraph.forEachNode((n: GraphNode) => { + const filePath = n.properties?.filePath as string | undefined; + const include = (filePath && toWriteSet.has(filePath)) || isGraphWide(n.label); + if (include) { + sub.addNode(n); + writableNodeIds.add(n.id); + } + }); + + fullGraph.forEachRelationship((r: GraphRelationship) => { + if (writableNodeIds.has(r.sourceId) || writableNodeIds.has(r.targetId)) { + sub.addRelationship(r); + } + }); + + return sub; +}; + +/** + * Public — derive the EFFECTIVE write-set: `toWriteSet` expanded by one + * hop along every edge in the new graph that crosses the writable + * boundary (one endpoint in a writable file, the other in an unchanged + * file). The unchanged-side file is pulled in so its stale rows are + * deleted + rewritten in lockstep with the changed side. + * + * Single pass over the edge list. Does NOT mutate `toWriteSet`. The + * orchestrator MUST feed the returned set to both `deleteNodesForFile` + * and `extractChangedSubgraph` — feeding the unexpanded set to either + * one leaves stale rows or PK-conflicts at COPY time. + */ +export const computeEffectiveWriteSet = ( + fullGraph: KnowledgeGraph, + toWriteSet: ReadonlySet, +): Set => { + const nodeFilePaths = indexNodeFilePaths(fullGraph); + const expanded = new Set(toWriteSet); + fullGraph.forEachRelationship((r: GraphRelationship) => { + const sourcePath = nodeFilePaths.get(r.sourceId); + const targetPath = nodeFilePaths.get(r.targetId); + if (!sourcePath || !targetPath) return; // skip edges to graph-wide nodes + const sourceWritable = toWriteSet.has(sourcePath); + const targetWritable = toWriteSet.has(targetPath); + if (sourceWritable && !targetWritable) expanded.add(targetPath); + else if (targetWritable && !sourceWritable) expanded.add(sourcePath); + }); + return expanded; +}; diff --git a/gitnexus/src/core/ingestion/call-processor.ts b/gitnexus/src/core/ingestion/call-processor.ts index 9fa6c1ae5..b45478f7a 100644 --- a/gitnexus/src/core/ingestion/call-processor.ts +++ b/gitnexus/src/core/ingestion/call-processor.ts @@ -42,12 +42,14 @@ import { isVerboseIngestionEnabled } from './utils/verbose.js'; import { yieldToEventLoop } from './utils/event-loop.js'; import { parseSourceSafe } from '../tree-sitter/safe-parse.js'; import { + CLASS_CONTAINER_TYPES, FUNCTION_NODE_TYPES, - findEnclosingClassId, findEnclosingClassInfo, genericFuncName, inferFunctionLabel, } from './utils/ast-helpers.js'; +import type { FieldInfo, FieldExtractorContext } from './field-types.js'; +import type { LanguageProvider } from './language-provider.js'; import { typeTagForId, constTagForId, buildCollisionGroups } from './utils/method-props.js'; import type { MethodInfo } from './method-types.js'; import { @@ -77,6 +79,62 @@ import type { LiteralTypeInferrer } from './type-extractors/types.js'; import type { SyntaxNode } from './utils/ast-helpers.js'; import { logger } from '../logger.js'; + +// ── Property-prepass helpers (parity with parse-worker.ts) ── +// These mirror the sequential-path equivalents in parse-worker.ts so the main- +// thread `processCalls` pre-pass produces byte-identical Property nodes/symbols +// to the worker pool. Drift between the two paths breaks the +// `incremental ≡ --force` invariant the moment a repo crosses the worker +// threshold between runs. + +/** Walk up to the nearest enclosing class/struct/interface AST node. */ +const findEnclosingClassNode = (node: SyntaxNode): SyntaxNode | null => { + let current = node.parent; + while (current) { + if (CLASS_CONTAINER_TYPES.has(current.type)) return current; + current = current.parent; + } + return null; +}; + +/** No-op SymbolTable stub for FieldExtractorContext — matches parse-worker. */ +const NOOP_SYMBOL_TABLE: SymbolTableReader = { + lookupExact: () => undefined, + lookupExactFull: () => undefined, + lookupExactAll: () => [], + lookupCallableByName: () => [], + getFiles: () => [][Symbol.iterator](), + getStats: () => ({ fileCount: 0 }), +}; + +/** + * Extract (and cache) field info for a class node. Cache is passed in so it + * stays scoped to a single `processCalls` invocation rather than leaking + * across analyze runs (worker uses module-level caching because each worker + * process is short-lived; the main thread is not). + * + * Cache key is `${filePath}:${classNode.startIndex}` — startIndex alone is a + * per-file byte offset, so almost every Ruby/Python file's leading class lands + * at byte 0 and would collide across files in the shared map. + */ +const getFieldInfo = ( + classNode: SyntaxNode, + provider: LanguageProvider, + context: FieldExtractorContext, + cache: Map>, +): Map | undefined => { + if (!provider.fieldExtractor) return undefined; + const cacheKey = `${context.filePath}:${classNode.startIndex}`; + const cached = cache.get(cacheKey); + if (cached) return cached; + const result = provider.fieldExtractor.extract(classNode, context); + if (!result?.fields?.length) return undefined; + const map = new Map(); + for (const field of result.fields) map.set(field.name, field); + cache.set(cacheKey, map); + return map; +}; + /** Per-file resolved type bindings for exported symbols. * Populated during call processing, consumed by Phase 14 re-resolution pass. */ export type ExportedTypeMap = Map>; @@ -860,6 +918,120 @@ export const processCalls = async ( prepared.push({ file, language, provider, tree, matches, parentMap, typeEnv }); } + // ── Property-registration pre-pass ── + // Register all routed properties (e.g. Ruby attr_accessor) BEFORE the + // resolution loop so cross-file field-type lookups (e.g. + // `user.address.save → Address#save`) succeed regardless of file + // processing order. This MUST stay in lockstep with the equivalent + // worker-path block in parse-worker.ts (kind === 'properties') — any + // divergence between the two paths breaks the `incremental ≡ --force` + // invariant once a repo crosses the worker threshold between runs. + const fieldInfoCache = new Map>(); + for (const { file, language, provider, matches, typeEnv } of prepared) { + const callRouter = provider.callRouter; + if (!callRouter) continue; + matches.forEach((match) => { + const captureMap: Record = {}; + match.captures.forEach((c) => (captureMap[c.name] = c.node)); + if (!captureMap['call']) return; + const callNameNode = captureMap['call.name']; + if (!callNameNode) return; + const routed = callRouter(callNameNode.text, captureMap['call']); + if (!routed || routed.kind !== 'properties') return; + + const propEnclosingInfo = findEnclosingClassInfo( + captureMap['call'], + file.path, + provider.resolveEnclosingOwner, + ); + const propEnclosingClassId = propEnclosingInfo?.classId ?? null; + + // Enrich routed properties with FieldExtractor metadata so types + // discovered from constructor assignments (e.g. `@address = Address.new`) + // are propagated even when the routing payload itself lacks declaredType. + let routedFieldMap: Map | undefined; + if (provider.fieldExtractor && typeEnv) { + const classNode = findEnclosingClassNode(captureMap['call']); + if (classNode) { + routedFieldMap = getFieldInfo( + classNode, + provider, + { + typeEnv, + symbolTable: NOOP_SYMBOL_TABLE, + filePath: file.path, + language, + }, + fieldInfoCache, + ); + } + } + + const fileId = generateId('File', file.path); + for (const item of routed.items) { + const routedFieldInfo = routedFieldMap?.get(item.propName); + const propQualifiedName = propEnclosingInfo + ? `${propEnclosingInfo.className}.${item.propName}` + : item.propName; + const nodeId = generateId('Property', `${file.path}:${propQualifiedName}`); + graph.addNode({ + id: nodeId, + label: 'Property', + properties: { + name: item.propName, + filePath: file.path, + startLine: item.startLine, + endLine: item.endLine, + language, + isExported: true, + description: item.accessorType, + ...(item.declaredType + ? { declaredType: item.declaredType } + : routedFieldInfo?.type + ? { declaredType: routedFieldInfo.type } + : {}), + ...(routedFieldInfo?.visibility !== undefined + ? { visibility: routedFieldInfo.visibility } + : {}), + ...(routedFieldInfo?.isStatic !== undefined + ? { isStatic: routedFieldInfo.isStatic } + : {}), + ...(routedFieldInfo?.isReadonly !== undefined + ? { isReadonly: routedFieldInfo.isReadonly } + : {}), + }, + }); + ctx.model.symbols.add(file.path, item.propName, nodeId, 'Property', { + ...(propEnclosingClassId ? { ownerId: propEnclosingClassId } : {}), + ...(item.declaredType + ? { declaredType: item.declaredType } + : routedFieldInfo?.type + ? { declaredType: routedFieldInfo.type } + : {}), + }); + const relId = generateId('DEFINES', `${fileId}->${nodeId}`); + graph.addRelationship({ + id: relId, + sourceId: fileId, + targetId: nodeId, + type: 'DEFINES', + confidence: 1.0, + reason: '', + }); + if (propEnclosingClassId) { + graph.addRelationship({ + id: generateId('HAS_PROPERTY', `${propEnclosingClassId}->${nodeId}`), + sourceId: propEnclosingClassId, + targetId: nodeId, + type: 'HAS_PROPERTY', + confidence: 1.0, + reason: '', + }); + } + } + }); + } + // ── Resolution loop: verify constructor bindings and resolve calls ── // The accumulator (if present) is now fully populated from the preparation // loop above, so verifyConstructorBindings sees all provider bindings @@ -930,9 +1102,10 @@ export const processCalls = async ( provider, ); const srcId = enclosing || generateId('File', file.path); - // Defer resolution: Ruby attr_accessor properties are registered during - // this same loop, so cross-file lookups fail if the declaring file hasn't - // been processed yet. Collect now, resolve after all files are done. + // Defer resolution so write-access tracking sees the FINAL graph + // state — properties from the pre-pass are present, but receiver-type + // resolution can still depend on inference that completes during the + // main loop. Resolve after all files have been processed. pendingWrites.push({ receiverTypeName, propertyName, filePath: file.path, srcId }); } // Assignment-only capture (no @call sibling): skip the rest of this @@ -1053,47 +1226,8 @@ export const processCalls = async ( return; case 'properties': { - const fileId = generateId('File', file.path); - const propEnclosingClassId = findEnclosingClassId(captureMap['call'], file.path); - for (const item of routed.items) { - const nodeId = generateId('Property', `${file.path}:${item.propName}`); - graph.addNode({ - id: nodeId, - label: 'Property', - properties: { - name: item.propName, - filePath: file.path, - startLine: item.startLine, - endLine: item.endLine, - language, - isExported: true, - description: item.accessorType, - }, - }); - ctx.model.symbols.add(file.path, item.propName, nodeId, 'Property', { - ...(propEnclosingClassId ? { ownerId: propEnclosingClassId } : {}), - ...(item.declaredType ? { declaredType: item.declaredType } : {}), - }); - const relId = generateId('DEFINES', `${fileId}->${nodeId}`); - graph.addRelationship({ - id: relId, - sourceId: fileId, - targetId: nodeId, - type: 'DEFINES', - confidence: 1.0, - reason: '', - }); - if (propEnclosingClassId) { - graph.addRelationship({ - id: generateId('HAS_PROPERTY', `${propEnclosingClassId}->${nodeId}`), - sourceId: propEnclosingClassId, - targetId: nodeId, - type: 'HAS_PROPERTY', - confidence: 1.0, - reason: '', - }); - } - } + // Properties already registered in the pre-pass above. + // Skip to avoid duplicate nodes/edges. return; } diff --git a/gitnexus/src/core/ingestion/community-processor.ts b/gitnexus/src/core/ingestion/community-processor.ts index 9913e4a3f..ac8f068fa 100644 --- a/gitnexus/src/core/ingestion/community-processor.ts +++ b/gitnexus/src/core/ingestion/community-processor.ts @@ -41,6 +41,24 @@ interface LeidenDetailedResult { modularity: number; } +/** + * Deterministic PRNG (mulberry32) seed for the vendored Leiden algorithm. + * Vendored Leiden defaults `rng: Math.random`, which makes community + * assignment non-deterministic across runs. Passing a seeded RNG gives us + * reproducible community/modularity output, which is required for the + * incremental-indexing equivalence test (incremental ≡ full rebuild). + */ +const LEIDEN_SEED = 0xc0de; +function createSeededRng(seed: number): () => number { + let s = seed >>> 0; + return () => { + s = (s + 0x6d2b79f5) >>> 0; + let t = Math.imul(s ^ (s >>> 15), 1 | s); + t = (t + Math.imul(t ^ (t >>> 7), 61 | t)) ^ t; + return ((t ^ (t >>> 14)) >>> 0) / 4294967296; + }; +} + // ============================================================================ // TYPES // ============================================================================ @@ -150,6 +168,7 @@ export const processCommunities = async ( leiden.detailed(graph, { resolution: isLarge ? 2.0 : 1.0, maxIterations: isLarge ? 3 : 0, + rng: createSeededRng(LEIDEN_SEED), }), ), new Promise((_, reject) => diff --git a/gitnexus/src/core/ingestion/parsing-processor.ts b/gitnexus/src/core/ingestion/parsing-processor.ts index 7559b26bc..04a17db4f 100644 --- a/gitnexus/src/core/ingestion/parsing-processor.ts +++ b/gitnexus/src/core/ingestion/parsing-processor.ts @@ -82,6 +82,88 @@ export interface WorkerExtractedData { // Worker-based parallel parsing // ============================================================================ +/** + * Merge a list of `ParseWorkerResult`s into the running graph + symbol + * table state and produce the chunk-aggregated `WorkerExtractedData`. + * + * Extracted from `processParsingWithWorkers` so the same merge logic can + * be applied to both freshly-parsed worker output AND cached worker + * output replayed during incremental analyze. Idempotent on the + * accumulator fields (push-only); idempotent on graph if the caller + * starts from a clean graph (otherwise duplicate `addNode` calls are + * silently no-op'd by `KnowledgeGraph`). + */ +export const mergeChunkResults = ( + graph: KnowledgeGraph, + symbolTable: SymbolTableWriter, + chunkResults: readonly ParseWorkerResult[], +): WorkerExtractedData => { + const allImports: ExtractedImport[] = []; + const allCalls: ExtractedCall[] = []; + const allAssignments: ExtractedAssignment[] = []; + const allHeritage: ExtractedHeritage[] = []; + const allRoutes: ExtractedRoute[] = []; + const allFetchCalls: ExtractedFetchCall[] = []; + const allDecoratorRoutes: ExtractedDecoratorRoute[] = []; + const allToolDefs: ExtractedToolDef[] = []; + const allORMQueries: ExtractedORMQuery[] = []; + const allConstructorBindings: FileConstructorBindings[] = []; + const fileScopeBindingsByFile: FileScopeBindings[] = []; + const allParsedFiles: ParsedFile[] = []; + + for (const result of chunkResults) { + for (const node of result.nodes) { + graph.addNode({ + id: node.id, + label: node.label as NodeLabel, + properties: node.properties, + }); + } + for (const rel of result.relationships) { + graph.addRelationship(rel); + } + for (const sym of result.symbols) { + symbolTable.add(sym.filePath, sym.name, sym.nodeId, sym.type, { + parameterCount: sym.parameterCount, + requiredParameterCount: sym.requiredParameterCount, + parameterTypes: sym.parameterTypes, + returnType: sym.returnType, + declaredType: sym.declaredType, + ownerId: sym.ownerId, + qualifiedName: sym.qualifiedName, + }); + } + for (const item of result.imports) allImports.push(item); + for (const item of result.calls) allCalls.push(item); + for (const item of result.assignments) allAssignments.push(item); + for (const item of result.heritage) allHeritage.push(item); + for (const item of result.routes) allRoutes.push(item); + for (const item of result.fetchCalls) allFetchCalls.push(item); + for (const item of result.decoratorRoutes) allDecoratorRoutes.push(item); + for (const item of result.toolDefs) allToolDefs.push(item); + if (result.ormQueries) for (const item of result.ormQueries) allORMQueries.push(item); + for (const item of result.constructorBindings) allConstructorBindings.push(item); + if (result.fileScopeBindings) + for (const item of result.fileScopeBindings) fileScopeBindingsByFile.push(item); + if (result.parsedFiles) for (const item of result.parsedFiles) allParsedFiles.push(item); + } + + return { + imports: allImports, + calls: allCalls, + assignments: allAssignments, + heritage: allHeritage, + routes: allRoutes, + fetchCalls: allFetchCalls, + decoratorRoutes: allDecoratorRoutes, + toolDefs: allToolDefs, + ormQueries: allORMQueries, + constructorBindings: allConstructorBindings, + fileScopeBindings: fileScopeBindingsByFile, + parsedFiles: allParsedFiles, + }; +}; + const processParsingWithWorkers = async ( graph: KnowledgeGraph, files: { path: string; content: string }[], @@ -89,6 +171,14 @@ const processParsingWithWorkers = async ( astCache: ASTCache, workerPool: WorkerPool, onFileProgress?: FileProgressCallback, + /** + * When provided, populated with the raw worker results before merging. + * Used by the incremental-indexing parse cache to capture the per-chunk + * worker output for caching across runs. The mutation happens in-place + * so the caller (parse-impl) can keep a reference. See + * `gitnexus/src/storage/parse-cache.ts`. + */ + outRawResults?: ParseWorkerResult[], ): Promise => { // Filter to parseable files only const parseableFiles: ParseWorkerInput[] = []; @@ -123,63 +213,16 @@ const processParsingWithWorkers = async ( }, ); - // Merge results from all workers into graph and symbol table - const allImports: ExtractedImport[] = []; - const allCalls: ExtractedCall[] = []; - const allAssignments: ExtractedAssignment[] = []; - const allHeritage: ExtractedHeritage[] = []; - const allRoutes: ExtractedRoute[] = []; - const allFetchCalls: ExtractedFetchCall[] = []; - const allDecoratorRoutes: ExtractedDecoratorRoute[] = []; - const allToolDefs: ExtractedToolDef[] = []; - const allORMQueries: ExtractedORMQuery[] = []; - const allConstructorBindings: FileConstructorBindings[] = []; - const fileScopeBindingsByFile: FileScopeBindings[] = []; - const allParsedFiles: ParsedFile[] = []; - for (const result of chunkResults) { - for (const node of result.nodes) { - graph.addNode({ - id: node.id, - label: node.label as NodeLabel, - properties: node.properties, - }); - } - - for (const rel of result.relationships) { - graph.addRelationship(rel); - } - - for (const sym of result.symbols) { - symbolTable.add(sym.filePath, sym.name, sym.nodeId, sym.type, { - parameterCount: sym.parameterCount, - requiredParameterCount: sym.requiredParameterCount, - parameterTypes: sym.parameterTypes, - returnType: sym.returnType, - declaredType: sym.declaredType, - ownerId: sym.ownerId, - qualifiedName: sym.qualifiedName, - }); - } - - for (const item of result.imports) allImports.push(item); - for (const item of result.calls) allCalls.push(item); - for (const item of result.assignments) allAssignments.push(item); - for (const item of result.heritage) allHeritage.push(item); - for (const item of result.routes) allRoutes.push(item); - for (const item of result.fetchCalls) allFetchCalls.push(item); - for (const item of result.decoratorRoutes) allDecoratorRoutes.push(item); - for (const item of result.toolDefs) allToolDefs.push(item); - if (result.ormQueries) for (const item of result.ormQueries) allORMQueries.push(item); - for (const item of result.constructorBindings) allConstructorBindings.push(item); - if (result.fileScopeBindings) - for (const item of result.fileScopeBindings) fileScopeBindingsByFile.push(item); - // RFC #909 Ring 2: aggregate per-file scope artifacts. Tolerant of - // workers that don't emit the field yet (older worker builds or - // partial rollouts), since the additive contract means undefined = - // "this worker produced no ParsedFiles for this chunk". - if (result.parsedFiles) for (const item of result.parsedFiles) allParsedFiles.push(item); + // Capture the raw chunk results for the incremental parse cache before + // merging — the cache stores the unmerged worker output so a future run + // can re-merge them into a fresh graph state. + if (outRawResults) { + for (const r of chunkResults) outRawResults.push(r); } + // Merge results from all workers into graph and symbol table. + const merged = mergeChunkResults(graph, symbolTable, chunkResults); + // Merge and log skipped languages from workers const skippedLanguages = new Map(); for (const result of chunkResults) { @@ -196,20 +239,7 @@ const processParsingWithWorkers = async ( // Final progress onFileProgress?.(total, total, 'done'); - return { - imports: allImports, - calls: allCalls, - assignments: allAssignments, - heritage: allHeritage, - routes: allRoutes, - fetchCalls: allFetchCalls, - decoratorRoutes: allDecoratorRoutes, - toolDefs: allToolDefs, - ormQueries: allORMQueries, - constructorBindings: allConstructorBindings, - fileScopeBindings: fileScopeBindingsByFile, - parsedFiles: allParsedFiles, - }; + return merged; }; // ============================================================================ @@ -732,6 +762,14 @@ export const processParsing = async ( scopeTreeCache: ASTCache | undefined, onFileProgress?: FileProgressCallback, workerPool?: WorkerPool, + /** + * Optional out-parameter for the incremental parse cache. When + * provided AND the worker-pool path runs successfully, populated + * with the raw `ParseWorkerResult[]` from the workers (pre-merge). + * Stays empty for the sequential fallback path (no per-chunk + * artifact to cache there). See `gitnexus/src/storage/parse-cache.ts`. + */ + outRawResults?: ParseWorkerResult[], ): Promise => { let lastProgress = 0; const reportProgress: FileProgressCallback | undefined = onFileProgress @@ -759,6 +797,7 @@ export const processParsing = async ( astCache, workerPool, reportProgress, + outRawResults, ); } catch (err) { const message = err instanceof Error ? err.message : String(err); diff --git a/gitnexus/src/core/ingestion/pipeline-phases/parse-impl.ts b/gitnexus/src/core/ingestion/pipeline-phases/parse-impl.ts index bd39a4330..17cfaab3f 100644 --- a/gitnexus/src/core/ingestion/pipeline-phases/parse-impl.ts +++ b/gitnexus/src/core/ingestion/pipeline-phases/parse-impl.ts @@ -17,7 +17,10 @@ import { enrichExportedTypeMap, type BindingEntry, } from '../binding-accumulator.js'; -import { processParsing } from '../parsing-processor.js'; +import { processParsing, mergeChunkResults } from '../parsing-processor.js'; +import { fileContentHash, computeChunkHash } from '../../../storage/parse-cache.js'; +import type { ParseWorkerResult } from '../workers/parse-worker.js'; +import type { WorkerExtractedData } from '../parsing-processor.js'; import { processImports, processImportsFromExtracted, @@ -72,8 +75,21 @@ import { extractORMQueriesInline } from './orm-extraction.js'; import { logger } from '../../logger.js'; // ── Constants ────────────────────────────────────────────────────────────── -/** Max bytes of source content to load per parse chunk. */ -const CHUNK_BYTE_BUDGET = 20 * 1024 * 1024; // 20MB +/** Max bytes of source content to load per parse chunk. + * + * Memory bound for the worker pool dispatch + a granularity knob for + * the parse cache. A single file change invalidates only its enclosing + * chunk, so smaller budgets → finer-grained invalidation. + * + * Override via GITNEXUS_CHUNK_BYTE_BUDGET (bytes) — the default of 2MB + * gives a useful invalidation floor (~1/N chunks on a multi-MB repo) + * while keeping worker dispatch overhead under 5% on cold runs. + */ +const CHUNK_BYTE_BUDGET = (() => { + const env = Number(process.env.GITNEXUS_CHUNK_BYTE_BUDGET); + if (Number.isFinite(env) && env > 0) return env; + return 2 * 1024 * 1024; +})(); // ── Main parse + resolve function ────────────────────────────────────────── @@ -119,6 +135,11 @@ export async function runChunkedParseAndResolve( * source. See plan * docs/plans/2026-04-20-002-perf-parse-heritage-mro-plan.md (Unit 4). */ scopeTreeCache: ASTCache; + /** Worker-produced ParsedFile artifacts aggregated across chunks. + * Threaded into scope-resolution as a re-extract cache so the warm- + * cache analyze run can skip the dominant `extractParsedFile` cost + * (otherwise ~58s on a 1000-file repo). */ + parsedFiles: import('gitnexus-shared').ParsedFile[]; }> { const ctx = createResolutionContext(); const symbolTable = ctx.model.symbols; @@ -142,6 +163,15 @@ export async function runChunkedParseAndResolve( ); } + // Sort parseableScanned alphabetically for stable chunk membership + // across runs (Finding 4). Without this, filesystem-scan order can + // shift between runs (notably on macOS APFS where directory entry + // order can change after modifications) — different files in the + // same chunk → different chunk hash → cache miss even when no file + // content changed. The cache also becomes platform-specific: a + // Linux-built cache misses on macOS for the same repo. + parseableScanned.sort((a, b) => (a.path < b.path ? -1 : a.path > b.path ? 1 : 0)); + const totalParseable = parseableScanned.length; if (totalParseable === 0) { @@ -271,6 +301,20 @@ export async function runChunkedParseAndResolve( const deferredWorkerHeritage: ExtractedHeritage[] = []; const deferredConstructorBindings: FileConstructorBindings[] = []; const deferredAssignments: ExtractedAssignment[] = []; + // Aggregated per-file ParsedFile artifacts produced by workers' calls + // to `extractParsedFile`. Threaded through to the scope-resolution + // phase so it can SKIP its own re-extraction on cache hits — this is + // the second-half of the parse-cache speedup since scope-resolution's + // re-parse otherwise dominates the warm-cache wall-clock time. + const allParsedFiles: import('gitnexus-shared').ParsedFile[] = []; + + // Incremental parse cache (Option B): chunk-level content-addressed. + // When the chunk's (filePath, content-hash) signature matches a prior + // run's, replay the cached ParseWorkerResult[] instead of dispatching + // to workers. See gitnexus/src/storage/parse-cache.ts. + const parseCache = options?.parseCache; + let chunkCacheHits = 0; + let chunkCacheMisses = 0; try { for (let chunkIdx = 0; chunkIdx < numChunks; chunkIdx++) { @@ -281,29 +325,89 @@ export async function runChunkedParseAndResolve( .filter((p) => chunkContents.has(p)) .map((p) => ({ path: p, content: chunkContents.get(p)! })); - const chunkWorkerData = await processParsing( - graph, - chunkFiles, - symbolTable, - astCache, - scopeTreeCache, - (current, _total, filePath) => { - const globalCurrent = filesParsedSoFar + current; - const parsingProgress = 20 + (globalCurrent / totalParseable) * 62; - onProgress({ - phase: 'parsing', - percent: Math.round(parsingProgress), - message: `Parsing chunk ${chunkIdx + 1}/${numChunks}...`, - detail: filePath, - stats: { - filesProcessed: globalCurrent, - totalFiles: totalParseable, - nodesCreated: graph.nodeCount, - }, - }); - }, - workerPool, - ); + // Compute the chunk's content-hash signature (if cache available). + let chunkHash: string | null = null; + if (parseCache) { + const entries = chunkFiles.map((f) => ({ + filePath: f.path, + contentHash: fileContentHash(f.content), + })); + chunkHash = computeChunkHash(entries); + } + + let chunkWorkerData: WorkerExtractedData | null; + const cachedRaw = chunkHash ? parseCache!.entries.get(chunkHash) : undefined; + + // Track every chunk hash we touched so the orchestrator can + // prune stale entries (chunks whose composition no longer + // corresponds to a live chunk in the current scan) before saving. + if (parseCache && chunkHash) parseCache.usedKeys.add(chunkHash); + + if (cachedRaw && cachedRaw.length > 0) { + // Cache hit: replay the cached worker output through the same + // merge logic the live worker path uses. + chunkCacheHits++; + chunkWorkerData = mergeChunkResults(graph, symbolTable, cachedRaw); + if (isDev) { + logger.info( + `📦 parse-cache HIT: chunk ${chunkIdx + 1}/${numChunks} (${chunkFiles.length} files, ${chunkHash!.slice(0, 8)})`, + ); + } + // Progress update so UI advances even on a cache hit. + const cachedFiles = chunkFiles.length; + onProgress({ + phase: 'parsing', + percent: Math.round(20 + ((filesParsedSoFar + cachedFiles) / totalParseable) * 62), + message: `Parsing chunk ${chunkIdx + 1}/${numChunks} (cache)...`, + stats: { + filesProcessed: filesParsedSoFar + cachedFiles, + totalFiles: totalParseable, + nodesCreated: graph.nodeCount, + }, + }); + } else { + // Cache miss: dispatch to workers, capture the raw results, store + // them under the chunk hash for the next run. + chunkCacheMisses++; + const rawResults: ParseWorkerResult[] = []; + chunkWorkerData = await processParsing( + graph, + chunkFiles, + symbolTable, + astCache, + scopeTreeCache, + (current, _total, filePath) => { + const globalCurrent = filesParsedSoFar + current; + const parsingProgress = 20 + (globalCurrent / totalParseable) * 62; + onProgress({ + phase: 'parsing', + percent: Math.round(parsingProgress), + message: `Parsing chunk ${chunkIdx + 1}/${numChunks}...`, + detail: filePath, + stats: { + filesProcessed: globalCurrent, + totalFiles: totalParseable, + nodesCreated: graph.nodeCount, + }, + }); + }, + workerPool, + // Capture raw results only when we have a cache to write to — + // otherwise we'd retain extra arrays for nothing. + parseCache && chunkHash ? rawResults : undefined, + ); + // Persist the raw results for this chunk hash. Sequential path + // doesn't populate rawResults (it writes directly to graph), so + // small repos without worker pool simply don't cache. That's fine. + if (parseCache && chunkHash && rawResults.length > 0) { + parseCache.entries.set(chunkHash, rawResults); + if (isDev) { + logger.info( + `📦 parse-cache MISS+store: chunk ${chunkIdx + 1}/${numChunks} (${chunkFiles.length} files, ${chunkHash.slice(0, 8)})`, + ); + } + } + } const chunkBasePercent = 20 + (filesParsedSoFar / totalParseable) * 62; @@ -349,6 +453,12 @@ export async function runChunkedParseAndResolve( for (const item of chunkWorkerData.heritage) deferredWorkerHeritage.push(item); for (const item of chunkWorkerData.constructorBindings) deferredConstructorBindings.push(item); + // Aggregate worker-produced ParsedFile artifacts so scope- + // resolution can use them as a re-extraction cache (skips its + // own tree-sitter re-parse on warm runs). + if (chunkWorkerData.parsedFiles?.length) { + for (const item of chunkWorkerData.parsedFiles) allParsedFiles.push(item); + } if (chunkWorkerData.assignments?.length) { for (const item of chunkWorkerData.assignments) deferredAssignments.push(item); } @@ -422,6 +532,12 @@ export async function runChunkedParseAndResolve( astCache.clear(); } + if (isDev && parseCache && (chunkCacheHits > 0 || chunkCacheMisses > 0)) { + logger.info( + `📦 parse-cache summary: ${chunkCacheHits} chunk hit(s), ${chunkCacheMisses} miss(es) across ${numChunks} chunk(s)`, + ); + } + const fullWorkerHeritageMap = deferredWorkerHeritage.length > 0 ? buildHeritageMap(deferredWorkerHeritage, ctx, getHeritageStrategyForLanguage) @@ -621,5 +737,12 @@ export async function runChunkedParseAndResolve( // chunk-local `astCache` above is intentionally NOT exposed // because parse-impl clears it between chunks. scopeTreeCache, + // Per-file ParsedFile artifacts produced by workers' calls to + // `extractParsedFile`. Empty when only the sequential path ran + // (sequential doesn't go through the worker, and extracts ParsedFile + // inline rather than emitting it). Consumed by scope-resolution as + // a re-extraction cache: when the file's ParsedFile is here, + // scope-resolution skips its own `extractParsedFile` call. + parsedFiles: allParsedFiles, }; } diff --git a/gitnexus/src/core/ingestion/pipeline-phases/parse.ts b/gitnexus/src/core/ingestion/pipeline-phases/parse.ts index a20d1e4b0..a3fa81be7 100644 --- a/gitnexus/src/core/ingestion/pipeline-phases/parse.ts +++ b/gitnexus/src/core/ingestion/pipeline-phases/parse.ts @@ -20,6 +20,7 @@ import type { PipelinePhase, PipelineContext, PhaseResult } from './types.js'; import { getPhaseOutput } from './types.js'; import type { StructureOutput } from './structure.js'; import type { BindingAccumulator } from '../binding-accumulator.js'; +import type { ParsedFile } from 'gitnexus-shared'; import type { ExtractedFetchCall, ExtractedRoute, @@ -81,6 +82,19 @@ export interface ParseOutput { * `scopeTreeCache.clear()` after its extract loop finishes. */ readonly scopeTreeCache: ASTCache; + /** + * Per-file `ParsedFile` artifacts produced by workers' calls to + * `extractParsedFile`. Threaded through to `scopeResolutionPhase` + * as a re-extraction cache: when a file's ParsedFile is present here, + * scope-resolution can skip its own `extractParsedFile` (which would + * otherwise re-parse the file with tree-sitter on the main thread, + * costing ~58s on a 1000-file repo). + * + * Empty for files that went through the sequential parse fallback — + * sequential doesn't emit ParsedFile artifacts; scope-resolution + * falls back to a fresh extract for those. + */ + readonly parsedFiles: readonly ParsedFile[]; } export const parsePhase: PipelinePhase = { diff --git a/gitnexus/src/core/ingestion/pipeline.ts b/gitnexus/src/core/ingestion/pipeline.ts index c220ea224..1ee8e102f 100644 --- a/gitnexus/src/core/ingestion/pipeline.ts +++ b/gitnexus/src/core/ingestion/pipeline.ts @@ -55,6 +55,19 @@ export interface PipelineOptions { minFiles?: number; minBytes?: number; }; + /** + * Incremental-indexing parse cache. When provided: + * - The parse phase looks up each chunk's content hash in + * `parseCache.entries`. On hit, it replays the cached + * `ParseWorkerResult[]` instead of dispatching to workers. + * - On miss, it runs the workers as today and stores the new + * results in `parseCache.entries` keyed by chunk hash. + * The caller (`run-analyze.ts`) is responsible for loading the cache + * before the pipeline runs and persisting it after. Cache survives + * `--force` because keys are content-addressed. + * See `gitnexus/src/storage/parse-cache.ts`. + */ + parseCache?: import('../../storage/parse-cache.js').ParseCache; } // ── Phase registry ───────────────────────────────────────────────────────── diff --git a/gitnexus/src/core/ingestion/scope-resolution/pipeline/phase.ts b/gitnexus/src/core/ingestion/scope-resolution/pipeline/phase.ts index c2fda9777..98a9f8994 100644 --- a/gitnexus/src/core/ingestion/scope-resolution/pipeline/phase.ts +++ b/gitnexus/src/core/ingestion/scope-resolution/pipeline/phase.ts @@ -93,13 +93,25 @@ export const scopeResolutionPhase: PipelinePhase = { // Worker-mode parses leave the cache empty for those files; they // also fall back to a fresh parse — no correctness impact. const parseOutput = getPhaseOutput(deps, 'parse'); - const { scopeTreeCache, resolutionContext } = parseOutput; + const { scopeTreeCache, resolutionContext, parsedFiles: workerParsedFiles } = parseOutput; // SemanticModel populated during `parse`: scope-resolution consumes // TypeRegistry / MethodRegistry / SymbolTable lookups instead of // rebuilding parallel indexes. See ARCHITECTURE.md § "Semantic-model // source of truth". const model = resolutionContext.model; + // Build a per-file lookup of ParsedFile artifacts the workers (or + // sequential extracts) already produced. Threading this into + // `runScopeResolution` lets the per-language extract loop short- + // circuit `extractParsedFile` — the dominant cost on the warm-cache + // path, since workers can't return tree-sitter Trees across the + // MessageChannel and scope-resolution would otherwise re-parse + // every file from scratch on the main thread. + const preExtractedByPath = new Map(); + for (const pf of workerParsedFiles) { + preExtractedByPath.set(pf.filePath, pf); + } + let totalFiles = 0; let totalImports = 0; let totalRefs = 0; @@ -143,6 +155,7 @@ export const scopeResolutionPhase: PipelinePhase = { files, treeCache: scopeTreeCache, resolutionConfig, + preExtractedParsedFiles: preExtractedByPath, onWarn: (msg) => { if (isSemanticModelValidatorEnabled()) { logger.warn(`[scope-resolution:${lang}] ${msg}`); diff --git a/gitnexus/src/core/ingestion/scope-resolution/pipeline/run.ts b/gitnexus/src/core/ingestion/scope-resolution/pipeline/run.ts index e2c734a43..31a58cdfa 100644 --- a/gitnexus/src/core/ingestion/scope-resolution/pipeline/run.ts +++ b/gitnexus/src/core/ingestion/scope-resolution/pipeline/run.ts @@ -72,6 +72,22 @@ interface RunScopeResolutionInput { * provider doesn't supply a config loader. */ readonly resolutionConfig?: unknown; + /** + * Pre-extracted ParsedFile artifacts keyed by file path. When a + * file is present here, the extract loop reuses it directly and + * skips `extractParsedFile` (which would re-parse the file with + * tree-sitter on the main thread). Only files matching the + * provider's language are honored — the loop verifies this + * implicitly by language filter at the call-site (scopeResolution + * phase). + * + * Worker-mode parses produce these ParsedFile artifacts as a side + * effect of `extractParsedFile` running inside the worker; threading + * them here is what lets the warm-cache analyze run skip the ~58s + * scope-resolution re-parse loop on a multi-thousand-file repo. + * Cache miss is safe — falls back to fresh extract. + */ + readonly preExtractedParsedFiles?: ReadonlyMap; } interface RunScopeResolutionStats { @@ -104,22 +120,37 @@ export function runScopeResolution( const parsedFiles: ParsedFile[] = []; let filesSkipped = 0; const treeCache = input.treeCache; + const preExtracted = input.preExtractedParsedFiles; + let preExtractedHits = 0; for (const file of files) { - const cachedTree = treeCache?.get(file.path); - const parsed = extractParsedFile( - provider.languageProvider, - file.content, - file.path, - onWarn, - cachedTree, - ); + let parsed: ParsedFile | undefined; + // Fast path: a worker (during the parse phase) already produced a + // ParsedFile for this file via `extractParsedFile`. Reuse it + // directly — skips a tree-sitter re-parse on the main thread. + if (preExtracted !== undefined) { + parsed = preExtracted.get(file.path); + if (parsed !== undefined) preExtractedHits++; + } if (parsed === undefined) { - filesSkipped++; - continue; + const cachedTree = treeCache?.get(file.path); + parsed = extractParsedFile( + provider.languageProvider, + file.content, + file.path, + onWarn, + cachedTree, + ); + if (parsed === undefined) { + filesSkipped++; + continue; + } } provider.populateOwners(parsed); parsedFiles.push(parsed); } + if (PROF && preExtracted !== undefined) { + logger.warn(`[scope-resolution prof] pre-extracted hits: ${preExtractedHits}/${files.length}`); + } provider.populateWorkspaceOwners?.(parsedFiles, { fileContents: getFileContents() }); // Reconcile scope-resolution's ownership view into the SemanticModel. diff --git a/gitnexus/src/core/lbug/lbug-adapter.ts b/gitnexus/src/core/lbug/lbug-adapter.ts index fe831cd43..caa7a58a1 100644 --- a/gitnexus/src/core/lbug/lbug-adapter.ts +++ b/gitnexus/src/core/lbug/lbug-adapter.ts @@ -1204,6 +1204,77 @@ export const deleteNodesForFile = async ( export const getEmbeddingTableName = (): string => EMBEDDING_TABLE_NAME; +/** + * Return the distinct repo-relative paths of files that import + * `targetFilePath` according to the IMPORTS edges currently in the + * DB. Used by the incremental writeback path to expand the + * "files-to-rewrite" set so that files importing a changed file get + * their edges (which may have been refined by cross-file resolution) + * re-emitted, rather than left stale in the DB. + * + * The DB query reads the *previous* run's state — pre-pipeline, before + * any nodes are deleted — so the returned importers are "files that + * USED TO import the target". That's the right set to invalidate: + * those are the files whose edges in the DB might no longer match + * what cross-file resolution produces given the changed file's new + * exports. + */ +export const queryImporters = async (targetFilePath: string): Promise => { + if (!conn) { + throw new Error('LadybugDB not initialized. Call initLbug first.'); + } + const escaped = targetFilePath.replace(/'/g, "''"); + const cypher = ` + MATCH (a)-[r:${REL_TABLE_NAME}]->(b) + WHERE r.type = 'IMPORTS' AND b.filePath = '${escaped}' + RETURN DISTINCT a.filePath AS importer + `; + try { + const queryResult = await conn.query(cypher); + const result = Array.isArray(queryResult) ? queryResult[0] : queryResult; + const rows = await result.getAll(); + const out: string[] = []; + for (const row of rows) { + const v = (row as { importer?: unknown }).importer; + if (typeof v === 'string' && v.length > 0) out.push(v); + } + return out; + } catch { + return []; + } +}; + +/** + * Drop every Community and Process node (and their MEMBER_OF / + * STEP_IN_PROCESS edges via DETACH DELETE). Used at the start of an + * incremental run so the communities and processes phases regenerate + * them from scratch on the merged graph — required for the + * "Leiden runs on the FULL graph" correctness invariant. + */ +export const deleteAllCommunitiesAndProcesses = async (): Promise<{ + nodesDeleted: number; +}> => { + if (!conn) { + throw new Error('LadybugDB not initialized. Call initLbug first.'); + } + let nodesDeleted = 0; + for (const label of ['Community', 'Process']) { + try { + const countResult = await conn.query(`MATCH (n:${label}) RETURN count(n) AS cnt`); + const result = Array.isArray(countResult) ? countResult[0] : countResult; + const rows = await result.getAll(); + const count = Number(rows[0]?.cnt ?? rows[0]?.[0] ?? 0); + if (count > 0) { + await conn.query(`MATCH (n:${label}) DETACH DELETE n`); + nodesDeleted += count; + } + } catch { + // Table may not exist yet on a freshly-initialized DB — fine. + } + } + return { nodesDeleted }; +}; + // ============================================================================ // Full-Text Search (FTS) Functions // ============================================================================ diff --git a/gitnexus/src/core/run-analyze.ts b/gitnexus/src/core/run-analyze.ts index fa2757f45..425f18f9a 100644 --- a/gitnexus/src/core/run-analyze.ts +++ b/gitnexus/src/core/run-analyze.ts @@ -11,6 +11,7 @@ import path from 'path'; import fs from 'fs/promises'; +import { execFileSync } from 'child_process'; import { runPipelineFromRepo } from './ingestion/pipeline.js'; import { initLbug, @@ -20,6 +21,9 @@ import { executeWithReusedStatement, closeLbug, loadCachedEmbeddings, + deleteNodesForFile, + deleteAllCommunitiesAndProcesses, + queryImporters, } from './lbug/lbug-adapter.js'; import { createSearchFTSIndexes } from './search/fts-indexes.js'; import { @@ -29,7 +33,15 @@ import { ensureGitNexusIgnored, registerRepo, cleanupOldKuzuFiles, + INCREMENTAL_SCHEMA_VERSION, } from '../storage/repo-manager.js'; +import { computeFileHashes, diffFileHashes } from '../storage/file-hash.js'; +import { + extractChangedSubgraph, + computeEffectiveWriteSet, +} from './incremental/subgraph-extract.js'; +import { shadowCandidatesFor } from './incremental/shadow-candidates.js'; +import { loadParseCache, saveParseCache, pruneCache } from '../storage/parse-cache.js'; import { getCurrentCommit, getRemoteUrl, @@ -178,23 +190,81 @@ export async function runFullAnalysis( const currentCommit = repoHasGit ? getCurrentCommit(repoPath) : ''; const existingMeta = await loadMeta(storagePath); + // ── Crash recovery: dirty flag forces full rebuild ──────────────── + // If the previous incremental run set incrementalInProgress and didn't + // clear it, the on-disk index may be in a half-state. Cheapest path + // back to a known-good index is to wipe + rebuild from scratch. + if (existingMeta?.incrementalInProgress) { + log( + 'Previous incremental run did not complete cleanly (incrementalInProgress flag set); ' + + 'forcing full rebuild to restore a known-good index.', + ); + options = { ...options, force: true }; + // Reload meta after clearing the flag in-memory; we still want fileHashes + // for the post-rebuild meta carry-over, but force=true ensures the + // rebuild path executes. + } + // ── Early-return: already up to date ────────────────────────────── if (existingMeta && !options.force && existingMeta.lastCommit === currentCommit) { // Non-git folders have currentCommit = '' — always rebuild since we can't detect changes if (currentCommit !== '') { - await ensureGitNexusIgnored(repoPath); - return { - // `resolveRepoIdentityRoot` collapses worktree roots to the - // canonical repo basename (#1259) but leaves arbitrary subdirs - // and `--skip-git` paths unchanged (#1232/#1233 intent preserved). - repoName: - options.registryName ?? - getInferredRepoName(repoPath) ?? - path.basename(resolveRepoIdentityRoot(repoPath)), - repoPath, - stats: existingMeta.stats ?? {}, - alreadyUpToDate: true, - }; + // For git repos, even if HEAD matches lastCommit, the working tree + // may have uncommitted changes. Only short-circuit when the working + // tree is also clean — otherwise fall through to the incremental + // path which will hash-diff and update only changed files. + // + // We exclude paths that GitNexus itself writes during analyze: + // .gitnexus/ — db / parse cache / meta.json + // .claude/, .cursor/ — auto-generated agent skill files + // AGENTS.md, CLAUDE.md — auto-updated stats blocks + // Counting them as dirty would perpetually defeat the up-to-date + // fast path because the previous analyze just wrote them + // (regression vs PR #1233 behavior). + const dirty = (() => { + try { + const out = execFileSync( + 'git', + [ + 'status', + '--porcelain', + '--', + '.', + ':(exclude).gitnexus', + ':(exclude).gitnexus/**', + ':(exclude).claude', + ':(exclude).claude/**', + ':(exclude).cursor', + ':(exclude).cursor/**', + ':(exclude)AGENTS.md', + ':(exclude)CLAUDE.md', + ], + { + cwd: repoPath, + stdio: ['ignore', 'pipe', 'ignore'], + encoding: 'utf8', + }, + ); + return out.trim().length > 0; + } catch { + return true; // conservative on git failure + } + })(); + if (!dirty) { + await ensureGitNexusIgnored(repoPath); + return { + // `resolveRepoIdentityRoot` collapses worktree roots to the + // canonical repo basename (#1259) but leaves arbitrary subdirs + // and `--skip-git` paths unchanged (#1232/#1233 intent preserved). + repoName: + options.registryName ?? + getInferredRepoName(repoPath) ?? + path.basename(resolveRepoIdentityRoot(repoPath)), + repoPath, + stats: existingMeta.stats ?? {}, + alreadyUpToDate: true, + }; + } } } @@ -243,6 +313,14 @@ export async function runFullAnalysis( ); } + // We *always* load the embedding cache when one is requested (regardless + // of the predicted `willTryIncremental`). The post-pipeline branch may + // disagree with the prediction (e.g. when the pipeline produces zero + // File nodes, `isIncremental` flips false and the full-rebuild path + // wipes the DB) — loading unconditionally is cheap insurance against + // silently dropping embeddings on a mispredicted run. The re-insert + // step gates itself on the actual `isIncremental` value to avoid + // PK-conflicts when the incremental writeback path keeps the rows. if (shouldLoadCache && existingMeta) { try { progress('embeddings', 0, 'Caching embeddings...'); @@ -270,24 +348,89 @@ export async function runFullAnalysis( } } + // ── Load incremental parse cache ────────────────────────────────── + // Content-addressed: safe to reuse across `--force` runs (chunks whose + // file contents haven't changed produce identical worker output). + // Loaded into a single ParseCache object that the pipeline mutates + // in-place (cache hits leave entries unchanged; misses add new ones). + const parseCache = await loadParseCache(storagePath); + // ── Phase 1: Full Pipeline (0–60%) ──────────────────────────────── - const pipelineResult = await runPipelineFromRepo(repoPath, (p) => { - const phaseLabel = PHASE_LABELS[p.phase] || p.phase; - const scaled = Math.round(p.percent * 0.6); - const message = p.detail ? `${p.message || phaseLabel} (${p.detail})` : p.message || phaseLabel; - progress(p.phase, scaled, message); - }); + const pipelineResult = await runPipelineFromRepo( + repoPath, + (p) => { + const phaseLabel = PHASE_LABELS[p.phase] || p.phase; + const scaled = Math.round(p.percent * 0.6); + const message = p.detail + ? `${p.message || phaseLabel} (${p.detail})` + : p.message || phaseLabel; + progress(p.phase, scaled, message); + }, + { parseCache }, + ); // ── Phase 2: LadybugDB (60–85%) ────────────────────────────────── progress('lbug', 60, 'Loading into LadybugDB...'); - await closeLbug(); - const lbugFiles = [lbugPath, `${lbugPath}.wal`, `${lbugPath}.lock`]; - for (const f of lbugFiles) { - try { - await fs.rm(f, { recursive: true, force: true }); - } catch { - /* swallow */ + // Compute current per-file content hashes from the pipeline's File nodes. + // Used both to drive the incremental DB writeback (when eligible) and to + // populate meta.json.fileHashes for the next run. + const allFilePaths: string[] = []; + pipelineResult.graph.forEachNode((n) => { + if (n.label === 'File') { + const fp = n.properties?.filePath as string | undefined; + if (fp) allFilePaths.push(fp); + } + }); + const newFileHashes = await computeFileHashes(repoPath, allFilePaths); + + // Decide incremental vs full at THIS point (post-pipeline, pre-DB). + // All eligibility conditions are checked here against the actual + // pipeline output — no separate pre-pipeline prediction to desync from + // (Bugbot review on PR #1479: a prediction that flipped post-pipeline + // could skip the embedding cache load and then take the full-rebuild + // path, silently losing embeddings). + const isIncremental = + !options.force && + !!existingMeta && + existingMeta.schemaVersion === INCREMENTAL_SCHEMA_VERSION && + !!existingMeta.fileHashes && + Object.keys(existingMeta.fileHashes).length > 0 && + repoHasGit && + allFilePaths.length > 0; + + const hashDiff = isIncremental + ? diffFileHashes(newFileHashes, existingMeta!.fileHashes) + : undefined; + + if (isIncremental && hashDiff) { + log( + `Incremental: changed=${hashDiff.changed.length}, ` + + `added=${hashDiff.added.length}, ` + + `deleted=${hashDiff.deleted.length} ` + + `(skipping wipe + ${ + allFilePaths.length - hashDiff.toWrite.length + } unchanged file rows preserved)`, + ); + // Set the dirty flag BEFORE any destructive DB mutation. Cleared on + // success at the meta-save step. + await saveMeta(storagePath, { + ...existingMeta!, + incrementalInProgress: { + startedAt: Date.now(), + toWriteCount: hashDiff.toWrite.length, + }, + }); + } else { + // Full rebuild path: wipe DB files first. + await closeLbug(); + const lbugFiles = [lbugPath, `${lbugPath}.wal`, `${lbugPath}.lock`]; + for (const f of lbugFiles) { + try { + await fs.rm(f, { recursive: true, force: true }); + } catch { + /* swallow */ + } } } @@ -298,11 +441,145 @@ export async function runFullAnalysis( // must be released to avoid blocking subsequent invocations. let lbugMsgCount = 0; - await loadGraphToLbug(pipelineResult.graph, pipelineResult.repoPath, storagePath, (msg) => { - lbugMsgCount++; - const pct = Math.min(84, 60 + Math.round((lbugMsgCount / (lbugMsgCount + 10)) * 24)); - progress('lbug', pct, msg); - }); + if (isIncremental && hashDiff) { + // ── Incremental DB writeback ─────────────────────────────────── + // 0. Expand the writable set with transitive importers of + // changed/deleted files (bounded BFS). + // + // Reason (Bugbot/Claude review on PR #1479): when a barrel / + // re-export file C changes, cross-file resolution may update + // CALLS edges between two unchanged files A and B (A imports + // from C, C re-exports something from B). Those refined edges + // live in `ctx.graph` but would be excluded from the subgraph + // if neither endpoint is in the changed set. To catch this, + // files that imported (directly OR transitively, through + // other unchanged intermediaries) any changed file get pulled + // into the writable set so their rows are deleted + rewritten + // against the refined edges. + // + // BFS bound: MAX_IMPORTER_BFS_DEPTH. Practically sized to + // catch nested barrel chains (e.g. `index.ts → submodule/index.ts + // → submodule/impl.ts`) without ballooning into a near-full- + // rebuild on monorepos with deep re-export pyramids. Beyond + // this depth, the "incremental ≡ full-rebuild" invariant is + // self-acknowledged as best-effort; `--force` remains the + // escape hatch documented in GUARDRAILS.md. + // + // `queryImporters` reads `IMPORTS` from the pre-pipeline DB + // state, so the result is "files that USED TO import the + // target" — exactly the set whose previously-stored edges may + // no longer match what cross-file resolution produces this run. + const MAX_IMPORTER_BFS_DEPTH = 4; + const writableFiles = new Set(hashDiff.toWrite); + const directlyChangedCount = writableFiles.size; + + // Shadow-seed: for ADDED files, queryImporters returns 0 (the new + // file has no IMPORTS rows in the pre-pipeline DB yet). But pre- + // existing unchanged files may have IMPORTS edges whose module- + // resolution claim the newcomer can steal under standard JS/TS + // resolution (Bugbot review on PR #1479). For each added file we + // derive the shadow candidates and, if the candidate was a known + // file in the prior meta, seed it into the BFS frontier so its + // importers — surfaced via queryImporters — get their CALLS edges + // re-resolved against the new file. See shadow-candidates.ts for + // the full pattern catalogue. + const priorFileSet = new Set( + existingMeta?.fileHashes ? Object.keys(existingMeta.fileHashes) : [], + ); + const shadowSeed: string[] = []; + for (const added of hashDiff.added) { + for (const cand of shadowCandidatesFor(added)) { + if (priorFileSet.has(cand) && !writableFiles.has(cand)) { + shadowSeed.push(cand); + } + } + } + + { + let frontier: string[] = [...hashDiff.toWrite, ...hashDiff.deleted, ...shadowSeed]; + for (let depth = 0; depth < MAX_IMPORTER_BFS_DEPTH && frontier.length > 0; depth++) { + const nextFrontier: string[] = []; + for (const f of frontier) { + try { + const importers = await queryImporters(f); + for (const i of importers) { + if (!writableFiles.has(i)) { + writableFiles.add(i); + nextFrontier.push(i); + } + } + } catch { + /* per-file importer query failure → skip; correctness degrades on + that branch, but DB stays writable. */ + } + } + frontier = nextFrontier; + } + } + const importerExpansion = writableFiles.size - directlyChangedCount; + if (importerExpansion > 0) { + log( + `Incremental: +${importerExpansion} importer(s) added to writable set ` + + `(BFS depth ≤ ${MAX_IMPORTER_BFS_DEPTH}` + + (shadowSeed.length > 0 ? `, ${shadowSeed.length} shadow-seed(s)` : '') + + `)`, + ); + } + + // 1. Compute the EFFECTIVE write-set (Finding 1). Two layers, + // composed: + // (a) `writableFiles` — toWrite ∪ transitive importers of + // changed/deleted files (the bounded BFS above, reading + // IMPORTS from the pre-pipeline DB). + // (b) `computeEffectiveWriteSet` — walks the NEW graph's + // edges and pulls in any unchanged-side file that sits + // on a writable-boundary-crossing edge (catches refined + // cross-file CALLS edges that the pre-run DB couldn't + // predict, e.g. a barrel re-export shifting `foo` from + // B to D). + // The composed set is the input to BOTH deleteNodesForFile + // and extractChangedSubgraph — asymmetry between the two would + // leave stale rows or PK-conflict at COPY time. + const effectiveWriteSet = computeEffectiveWriteSet(pipelineResult.graph, writableFiles); + // Deduped: deleted entries may already appear via importer-BFS + // expansion (queryImporters can return a now-deleted path), which + // would otherwise call deleteNodesForFile twice for the same file + // (Bugbot LOW finding on PR #1479). + const filesToDelete = [...new Set([...effectiveWriteSet, ...hashDiff.deleted])]; + for (let i = 0; i < filesToDelete.length; i++) { + const f = filesToDelete[i]; + try { + await deleteNodesForFile(f); + } catch { + /* file may not have rows (e.g. an unparseable file) — fine */ + } + if (i % 20 === 0) { + progress('lbug', 62, `Removing rows for changed files (${i}/${filesToDelete.length})...`); + } + } + // 2. Drop graph-wide nodes (Community, Process). They'll be re-inserted + // from the fresh pipeline output below. Required for the + // "Leiden runs on the FULL graph" correctness invariant. + await deleteAllCommunitiesAndProcesses(); + + // 3. Extract the changed subgraph from the FULL ctx.graph and write + // only that. Unchanged-file rows in the DB stay untouched. Pass + // the SAME effectiveWriteSet so the subgraph and the deletes + // cover identical files (asymmetry would silently corrupt). + const subgraph = extractChangedSubgraph(pipelineResult.graph, effectiveWriteSet); + await loadGraphToLbug(subgraph, pipelineResult.repoPath, storagePath, (msg) => { + lbugMsgCount++; + const pct = Math.min(84, 65 + Math.round((lbugMsgCount / (lbugMsgCount + 10)) * 19)); + progress('lbug', pct, msg); + }); + } else { + // ── Full rebuild ─────────────────────────────────────────────── + await loadGraphToLbug(pipelineResult.graph, pipelineResult.repoPath, storagePath, (msg) => { + lbugMsgCount++; + const pct = Math.min(84, 60 + Math.round((lbugMsgCount / (lbugMsgCount + 10)) * 24)); + progress('lbug', pct, msg); + }); + } // ── Phase 3: FTS (85–90%) ───────────────────────────────────────── progress('fts', 85, 'Creating search indexes...'); @@ -310,6 +587,19 @@ export async function runFullAnalysis( progress('fts', 90, 'Search indexes ready'); // ── Phase 3.5: Re-insert cached embeddings ──────────────────────── + // Runs on BOTH the full-rebuild path and the incremental path: + // - Full rebuild: DB was wiped, every cached row needs to come back. + // - Incremental: changed-file rows were just deleted by + // deleteNodesForFile (which cascades to their + // embedding rows) — so their cached vectors need + // to come back too. Unchanged-file rows still + // exist; re-inserting their cached vectors would + // PK-conflict, but the per-batch try/catch below + // silently ignores those (matches the existing + // "some may fail if node was removed, that's + // fine" semantics). Bugbot review on PR #1479 + // flagged that gating this on `!isIncremental` + // silently lost changed-file embeddings. if (cachedEmbeddings.length > 0) { const cachedDims = cachedEmbeddings[0].embedding.length; const { EMBEDDING_DIMS } = await import('./lbug/schema.js'); @@ -456,6 +746,12 @@ export async function runFullAnalysis( const effectiveSemanticMode = semanticMode ?? (runtimeCapabilities.semanticMode === 'vector-index' ? 'vector-index' : 'exact-scan'); + + // Convert the post-run file-hash map to the on-disk Record + // shape consumed by RepoMeta.fileHashes. + const newFileHashesRecord: Record = {}; + for (const [k, v] of newFileHashes) newFileHashesRecord[k] = v; + const meta = { repoPath, lastCommit: currentCommit, @@ -485,8 +781,33 @@ export async function runFullAnalysis( reason: runtimeCapabilities.reason, }, }, + // Incremental-indexing fields. Populated for git repos so the next + // analyze run can take the incremental DB-writeback path. Setting + // incrementalInProgress to undefined explicitly clears any prior + // dirty flag (full and incremental success paths converge here). + schemaVersion: hasGitDir(repoPath) ? INCREMENTAL_SCHEMA_VERSION : undefined, + fileHashes: hasGitDir(repoPath) ? newFileHashesRecord : undefined, + incrementalInProgress: undefined as { startedAt: number; toWriteCount: number } | undefined, }; await saveMeta(storagePath, meta); + + // Persist the incremental parse cache for the next run. Wraps in + // try/catch so a cache-write failure never breaks an otherwise + // successful indexing run. Prune stale chunk-hash entries first so + // the cache file size stays bounded across runs (chunks whose + // composition no longer matches anything in the current scan are + // dead weight; the parse phase populates `usedKeys` as it processes + // chunks). + try { + const pruned = pruneCache(parseCache, parseCache.usedKeys); + if (pruned > 0) { + log(`Parse cache: pruned ${pruned} stale chunk entries`); + } + await saveParseCache(storagePath, parseCache); + } catch (e) { + log(`Warning: could not save parse cache (${(e as Error).message}); continuing.`); + } + // Forward the --name alias and the registry-collision bypass bit. // `allowDuplicateName` is its own concern — independent from the // pipeline `force` above. The CLI maps it from diff --git a/gitnexus/src/storage/file-hash.ts b/gitnexus/src/storage/file-hash.ts new file mode 100644 index 000000000..b39111815 --- /dev/null +++ b/gitnexus/src/storage/file-hash.ts @@ -0,0 +1,104 @@ +/** + * Per-file content hashing for incremental DB writeback. + * + * On every analyze run we compute SHA-256 of every file's content and + * store the map in meta.json. The next run compares disk against the + * stored map and produces: + * - `changed` — content differs (re-emit DB rows for this file) + * - `added` — file is new on disk (insert DB rows) + * - `deleted` — file was in last meta but no longer on disk (drop rows) + * + * The pipeline still parses every file (correctness invariant: cross-file + * resolution needs full data). What this enables is a SELECTIVE DB + * writeback: instead of wipe-and-reload of the whole graph (~50s of CSV + * COPY on a 25K-node repo), we only delete-and-rewrite rows for the + * changed/added/deleted set. + * + * See docs/superpowers/specs/2026-05-10-incremental-indexing-design.md + * (Option B revision). + */ + +import { createHash } from 'crypto'; +import fs from 'fs/promises'; +import path from 'path'; + +/** + * Compute SHA-256 of a single file. Returns null when the file can't be + * read — caller treats that as "no signature, assume changed". + */ +export const computeFileHash = async (absPath: string): Promise => { + try { + const buf = await fs.readFile(absPath); + return createHash('sha256').update(buf).digest('hex'); + } catch { + return null; + } +}; + +/** + * Compute SHA-256 hashes for many files in parallel batches. Files that + * fail to read are omitted from the result map. + */ +export const computeFileHashes = async ( + repoPath: string, + relPaths: readonly string[], +): Promise> => { + const out = new Map(); + const BATCH = 100; + for (let i = 0; i < relPaths.length; i += BATCH) { + const batch = relPaths.slice(i, i + BATCH); + const results = await Promise.all( + batch.map(async (rel) => { + const h = await computeFileHash(path.join(repoPath, rel)); + return h ? ([rel, h] as const) : null; + }), + ); + for (const r of results) if (r) out.set(r[0], r[1]); + } + return out; +}; + +/** Result of comparing the current on-disk hashes against stored ones. */ +export interface FileHashDiff { + /** Files whose content hash differs from stored. */ + changed: string[]; + /** Files in the current scan that weren't in the stored map. */ + added: string[]; + /** Files in the stored map that aren't in the current scan. */ + deleted: string[]; + /** All files whose DB rows must be replaced (changed ∪ added). */ + toWrite: string[]; +} + +/** + * Diff a current hash map against a previously stored one. + * + * Sorted output so two runs produce identical diff arrays for the same + * changes — useful for stable logging / equivalence checks. + */ +export const diffFileHashes = ( + current: ReadonlyMap, + stored: Readonly> | undefined, +): FileHashDiff => { + const storedMap = new Map(stored ? Object.entries(stored) : []); + const changed: string[] = []; + const added: string[] = []; + for (const [p, h] of current) { + const prev = storedMap.get(p); + if (prev === undefined) added.push(p); + else if (prev !== h) changed.push(p); + } + const deleted: string[] = []; + for (const p of storedMap.keys()) { + if (!current.has(p)) deleted.push(p); + } + changed.sort(); + added.sort(); + deleted.sort(); + return { + changed, + added, + deleted, + toWrite: [...changed, ...added].sort(), + }; +}; diff --git a/gitnexus/src/storage/parse-cache.ts b/gitnexus/src/storage/parse-cache.ts new file mode 100644 index 000000000..a1abf76fa --- /dev/null +++ b/gitnexus/src/storage/parse-cache.ts @@ -0,0 +1,213 @@ +/** + * Chunk-level content-addressed parse cache. + * + * The pipeline always parses every file (correctness invariant: cross-file + * resolution and downstream phases need full graph data). What this cache + * does is skip the tree-sitter worker dispatch when a chunk's contents + * haven't changed since the last run. + * + * Granularity: chunk-level. The parse phase chunks files into ~20MB byte + * budgets. The cache key is `sha256(joined(filePath:contentHash for each + * file in the chunk, sorted))`. A change to a single file invalidates only + * that file's chunk — typically 1 of ~50 chunks on a 1000-file repo. + * + * Why not per-file: + * - Workers process sub-batches and emit aggregated `ParseWorkerResult`s. + * Splitting back to per-file would require reworking the worker contract. + * - Chunk-level invalidation gives a useful speedup floor (98% on a single + * 1-of-50 invalidated chunk) without touching the worker. + * + * Survives `--force` because it's content-addressed: the same bytes always + * produce the same key. `--force` only matters for the LadybugDB writeback; + * the cache itself is always safe to reuse. + */ + +import { createHash } from 'crypto'; +import { createRequire } from 'module'; +import fs from 'fs/promises'; +import path from 'path'; +import { fileURLToPath } from 'url'; +import type { ParseWorkerResult } from '../core/ingestion/workers/parse-worker.js'; + +/** + * Cache version composed of: + * - A schema bump knob (`SCHEMA_BUMP`) for hand-controlled invalidation + * when ParseWorkerResult shape or upstream parse semantics change. + * - The current `gitnexus` npm package version, read at module load. + * Any release that ships an updated tree-sitter grammar or revised + * extractor logic implies a version bump in package.json, which + * automatically invalidates the on-disk cache. Without this, a user + * running `npm i -g gitnexus@latest` after a parser-affecting + * release would silently replay pre-upgrade ParseWorkerResults + * against the new graph schema (Bugbot/Claude review on #1479). + * + * On version mismatch, `loadParseCache` returns an empty cache and the + * next save overwrites the on-disk file with the new version baked in. + */ +const SCHEMA_BUMP = 1; +const GITNEXUS_PKG_VERSION = (() => { + try { + // package.json sits at gitnexus/package.json — two levels up from + // gitnexus/src/storage/parse-cache.ts (or its dist/ equivalent). + const here = path.dirname(fileURLToPath(import.meta.url)); + const candidates = [ + path.join(here, '..', '..', 'package.json'), // src/storage → gitnexus/ + path.join(here, '..', '..', '..', 'package.json'), // dist/storage → gitnexus/ + ]; + const requireCJS = createRequire(import.meta.url); + for (const c of candidates) { + try { + const pkg = requireCJS(c); + if (typeof pkg?.version === 'string') return pkg.version; + } catch { + /* try next candidate */ + } + } + } catch { + /* fall through to fallback */ + } + return '0.0.0-unknown'; +})(); +export const PARSE_CACHE_VERSION = `${SCHEMA_BUMP}+${GITNEXUS_PKG_VERSION}`; + +const CACHE_FILENAME = 'parse-cache.json'; + +/** On-disk shape. */ +interface ParseCacheFile { + version: string; + /** key = chunk hash (hex) → cached chunk result list. */ + entries: Record; +} + +/** Runtime view: keyed Map for fast lookup; mutated in place during a run. */ +export interface ParseCache { + version: string; + entries: Map; + /** + * Hashes referenced (hit OR miss-and-stored) by the current run. + * The parse phase populates this as it processes chunks; the orchestrator + * uses it as input to `pruneCache` before saving so entries that no + * longer correspond to any chunk in the current scan are discarded. + * Transient — never serialized to disk. + */ + usedKeys: Set; +} + +/** SHA-256 hex of a single string or buffer. */ +const sha256Hex = (input: Buffer | string): string => + createHash('sha256') + .update(typeof input === 'string' ? Buffer.from(input) : input) + .digest('hex'); + +/** Stable hash of a single file's contents — used by callers to compose a chunk hash. */ +export const fileContentHash = (content: Buffer | string): string => sha256Hex(content); + +/** + * Compute the canonical cache key for a chunk's contents. + * + * `entries` is the list of (filePath, file content hash) for every file + * in the chunk. We sort by filePath before hashing so chunks composed of + * the same files in different order produce the same key. + */ +export const computeChunkHash = ( + entries: Array<{ filePath: string; contentHash: string }>, +): string => { + const sorted = [...entries].sort((a, b) => (a.filePath < b.filePath ? -1 : 1)); + const joined = sorted.map((e) => `${e.filePath}:${e.contentHash}`).join('\n'); + return sha256Hex(joined); +}; + +/** + * JSON replacer that round-trips Map/Set instances through plain JSON. + * + * `ParseWorkerResult.parsedFiles[*].scopes[*].typeBindings` is a + * `ReadonlyMap`; without this transform it serializes + * to `{}` and downstream code that iterates / `.get()`s on it crashes + * with "is not iterable". Applied symmetrically by `mapReviver` on + * load so the in-memory shape stays Map-typed. + */ +const MAP_TAG = '__$mapEntries$__'; +const SET_TAG = '__$setValues$__'; + +const mapReplacer = (_key: string, value: unknown): unknown => { + if (value instanceof Map) return { [MAP_TAG]: Array.from(value.entries()) }; + if (value instanceof Set) return { [SET_TAG]: Array.from(value.values()) }; + return value; +}; + +const mapReviver = (_key: string, value: unknown): unknown => { + if (value && typeof value === 'object') { + const v = value as Record; + if (Array.isArray(v[MAP_TAG])) return new Map(v[MAP_TAG] as [unknown, unknown][]); + if (Array.isArray(v[SET_TAG])) return new Set(v[SET_TAG] as unknown[]); + } + return value; +}; + +/** + * Load the parse cache. Returns an empty cache on any failure (missing + * file, corrupt JSON, version mismatch). Never throws on a normal load. + */ +export const loadParseCache = async (storagePath: string): Promise => { + const cachePath = path.join(storagePath, CACHE_FILENAME); + try { + const raw = await fs.readFile(cachePath, 'utf-8'); + const data = JSON.parse(raw, mapReviver) as ParseCacheFile; + if ( + typeof data !== 'object' || + data === null || + data.version !== PARSE_CACHE_VERSION || + typeof data.entries !== 'object' || + data.entries === null + ) { + return emptyCache(); + } + const entries = new Map(); + for (const [k, v] of Object.entries(data.entries)) { + if (Array.isArray(v)) entries.set(k, v as ParseWorkerResult[]); + } + return { version: PARSE_CACHE_VERSION, entries, usedKeys: new Set() }; + } catch { + return emptyCache(); + } +}; + +/** + * Persist the cache to disk atomically (write-and-rename) so a crash + * mid-write doesn't leave a corrupt file. + */ +export const saveParseCache = async (storagePath: string, cache: ParseCache): Promise => { + await fs.mkdir(storagePath, { recursive: true }); + const cachePath = path.join(storagePath, CACHE_FILENAME); + const tmpPath = `${cachePath}.tmp`; + const out: ParseCacheFile = { + version: cache.version, + entries: Object.fromEntries(cache.entries), + }; + // Compact JSON; this file can be tens of MB on a large repo and pretty- + // printing roughly doubles size for no value. + await fs.writeFile(tmpPath, JSON.stringify(out, mapReplacer), 'utf-8'); + await fs.rename(tmpPath, cachePath); +}; + +/** + * Drop entries whose hashes are not in `usedHashes`. Called at the end + * of a run so chunks that no longer correspond to any current chunk + * don't keep their stale entries forever. + */ +export const pruneCache = (cache: ParseCache, usedHashes: ReadonlySet): number => { + let removed = 0; + for (const k of cache.entries.keys()) { + if (!usedHashes.has(k)) { + cache.entries.delete(k); + removed++; + } + } + return removed; +}; + +const emptyCache = (): ParseCache => ({ + version: PARSE_CACHE_VERSION, + entries: new Map(), + usedKeys: new Set(), +}); diff --git a/gitnexus/src/storage/repo-manager.ts b/gitnexus/src/storage/repo-manager.ts index 8c0bda95f..456a6c143 100644 --- a/gitnexus/src/storage/repo-manager.ts +++ b/gitnexus/src/storage/repo-manager.ts @@ -71,8 +71,40 @@ export interface RepoMeta { processes?: number; embeddings?: number; }; + /** + * Bumped whenever incremental-indexing invariants change in an + * incompatible way (delete-and-rewrite logic, subgraph extraction, + * graph-wide node handling). On mismatch, runFullAnalysis forces a + * full rebuild rather than risk an inconsistent incremental update. + */ + schemaVersion?: number; + /** + * SHA-256 of every file's content at the time of the last successful + * indexing run. The next run computes current hashes and diffs against + * this map to determine which files' DB rows must be replaced. + * Map keys are repo-relative paths. + */ + fileHashes?: Record; + /** + * Crash-recovery dirty flag. Written to meta.json BEFORE any + * destructive DB mutation in an incremental run; cleared on success + * by overwriting meta.json. If a run crashes between, the next run + * sees the flag and forces a full rebuild — the cheapest path back + * to a known-good index. + */ + incrementalInProgress?: { + /** When the incremental run started (epoch ms). */ + startedAt: number; + /** Number of files in the writable set, for diagnostic logs. */ + toWriteCount: number; + }; } +/** + * Bumped whenever incremental-indexing invariants change incompatibly. + */ +export const INCREMENTAL_SCHEMA_VERSION = 1; + export interface IndexedRepo { repoPath: string; storagePath: string; @@ -186,12 +218,23 @@ export const loadMeta = async (storagePath: string): Promise => }; /** - * Save metadata to storage + * Save metadata to storage. + * + * Atomic via tmp-file + rename (matches `saveParseCache`'s pattern). The + * `incrementalInProgress` dirty flag travels through this file — a crash + * mid-write would leave a corrupt `meta.json` that the next run's + * `loadMeta` would silently treat as "no prior index", losing the dirty + * flag and skipping the recovery full-rebuild. Write-and-rename rules + * that out: the rename is atomic on POSIX and on Windows (`fs.rename` + * on `node:fs/promises` uses `MoveFileEx(REPLACE_EXISTING)`), so either + * the old or the new file is observed at every moment. */ export const saveMeta = async (storagePath: string, meta: RepoMeta): Promise => { await fs.mkdir(storagePath, { recursive: true }); const metaPath = path.join(storagePath, 'meta.json'); - await fs.writeFile(metaPath, JSON.stringify(meta, null, 2), 'utf-8'); + const tmpPath = `${metaPath}.tmp`; + await fs.writeFile(tmpPath, JSON.stringify(meta, null, 2), 'utf-8'); + await fs.rename(tmpPath, metaPath); }; /** diff --git a/gitnexus/test/fixtures/pipeline-golden/mini-repo/expected-graph.json b/gitnexus/test/fixtures/pipeline-golden/mini-repo/expected-graph.json index 0d66fd3a7..34f2a6106 100644 --- a/gitnexus/test/fixtures/pipeline-golden/mini-repo/expected-graph.json +++ b/gitnexus/test/fixtures/pipeline-golden/mini-repo/expected-graph.json @@ -25,5 +25,5 @@ "MEMBER_OF": 12, "STEP_IN_PROCESS": 12 }, - "edgeDigest": "a418debec537cf959fe56fd1fbbbfb59a640398cdb3c61ce0bcb8056c1f45110" + "edgeDigest": "6f414427a20c037df3e336f055c83f987e7d381c9bfa73b4d2be690cb8103302" } diff --git a/gitnexus/test/unit/incremental-file-hash.test.ts b/gitnexus/test/unit/incremental-file-hash.test.ts new file mode 100644 index 000000000..0f59dfb09 --- /dev/null +++ b/gitnexus/test/unit/incremental-file-hash.test.ts @@ -0,0 +1,124 @@ +import { describe, it, expect } from 'vitest'; +import { mkdtemp, writeFile, rm } from 'fs/promises'; +import { tmpdir } from 'os'; +import path from 'path'; +import { computeFileHash, computeFileHashes, diffFileHashes } from '../../src/storage/file-hash.js'; + +describe('diffFileHashes', () => { + it('classifies files into changed / added / deleted / toWrite', () => { + const stored = { a: 'h-a', b: 'h-b', c: 'h-c' }; + const current = new Map([ + ['a', 'h-a'], // unchanged + ['b', 'h-b-NEW'], // changed + ['d', 'h-d'], // added + // 'c' is gone → deleted + ]); + const diff = diffFileHashes(current, stored); + expect(diff.changed).toEqual(['b']); + expect(diff.added).toEqual(['d']); + expect(diff.deleted).toEqual(['c']); + // toWrite is the union of changed ∪ added (rows to be (re)written) + expect(diff.toWrite.sort()).toEqual(['b', 'd']); + }); + + it('treats no stored map as "everything is added"', () => { + const current = new Map([ + ['x', 'h1'], + ['y', 'h2'], + ]); + const diff = diffFileHashes(current, undefined); + expect(diff.added.sort()).toEqual(['x', 'y']); + expect(diff.changed).toEqual([]); + expect(diff.deleted).toEqual([]); + expect(diff.toWrite.sort()).toEqual(['x', 'y']); + }); + + it('returns sorted arrays for stable cross-platform comparison', () => { + const stored = { z: 'h', a: 'h', m: 'h' }; + const current = new Map([ + ['z', 'h2'], + ['a', 'h2'], + ['m', 'h2'], + ]); + const diff = diffFileHashes(current, stored); + expect(diff.changed).toEqual(['a', 'm', 'z']); + expect(diff.toWrite).toEqual(['a', 'm', 'z']); + }); + + it('handles empty current map (all stored files become deleted)', () => { + const stored = { a: 'h1', b: 'h2' }; + const diff = diffFileHashes(new Map(), stored); + expect(diff.deleted).toEqual(['a', 'b']); + expect(diff.changed).toEqual([]); + expect(diff.added).toEqual([]); + }); +}); + +describe('computeFileHash', () => { + it('produces a stable SHA-256 hex digest', async () => { + const dir = await mkdtemp(path.join(tmpdir(), 'gnx-fh-')); + try { + const f = path.join(dir, 'a.txt'); + await writeFile(f, 'hello world\n', 'utf-8'); + const h1 = await computeFileHash(f); + const h2 = await computeFileHash(f); + expect(h1).toBe(h2); + expect(h1).toMatch(/^[a-f0-9]{64}$/); + } finally { + await rm(dir, { recursive: true, force: true }); + } + }); + + it('returns null on missing file (caller treats as "no signature")', async () => { + const h = await computeFileHash('/definitely/does/not/exist/here.xyz'); + expect(h).toBeNull(); + }); + + it('different content → different hash', async () => { + const dir = await mkdtemp(path.join(tmpdir(), 'gnx-fh-')); + try { + const a = path.join(dir, 'a.txt'); + const b = path.join(dir, 'b.txt'); + await writeFile(a, 'hello', 'utf-8'); + await writeFile(b, 'goodbye', 'utf-8'); + const ha = await computeFileHash(a); + const hb = await computeFileHash(b); + expect(ha).not.toBeNull(); + expect(hb).not.toBeNull(); + expect(ha).not.toBe(hb); + } finally { + await rm(dir, { recursive: true, force: true }); + } + }); +}); + +describe('computeFileHashes', () => { + it('hashes a small batch of files in parallel', async () => { + const dir = await mkdtemp(path.join(tmpdir(), 'gnx-fh-')); + try { + await writeFile(path.join(dir, 'one.txt'), 'A', 'utf-8'); + await writeFile(path.join(dir, 'two.txt'), 'B', 'utf-8'); + await writeFile(path.join(dir, 'three.txt'), 'C', 'utf-8'); + const map = await computeFileHashes(dir, ['one.txt', 'two.txt', 'three.txt']); + expect(map.size).toBe(3); + expect(map.get('one.txt')).toMatch(/^[a-f0-9]{64}$/); + // All distinct since contents differ + const hashes = [...map.values()]; + expect(new Set(hashes).size).toBe(3); + } finally { + await rm(dir, { recursive: true, force: true }); + } + }); + + it('omits files that fail to read (no entry in result)', async () => { + const dir = await mkdtemp(path.join(tmpdir(), 'gnx-fh-')); + try { + await writeFile(path.join(dir, 'real.txt'), 'X', 'utf-8'); + const map = await computeFileHashes(dir, ['real.txt', 'phantom.txt']); + expect(map.has('real.txt')).toBe(true); + expect(map.has('phantom.txt')).toBe(false); + } finally { + await rm(dir, { recursive: true, force: true }); + } + }); +}); diff --git a/gitnexus/test/unit/incremental-orchestration.test.ts b/gitnexus/test/unit/incremental-orchestration.test.ts new file mode 100644 index 000000000..3d0d244af --- /dev/null +++ b/gitnexus/test/unit/incremental-orchestration.test.ts @@ -0,0 +1,263 @@ +/** + * Integration coverage for the `runFullAnalysis` incremental-orchestration + * wiring (Claude PR-review Finding 2). + * + * These tests exercise the *real runtime path* — they call + * `runFullAnalysis` against a real on-disk git repo backed by a real + * LadybugDB at `/.gitnexus/`, and assert behaviours that pure + * unit tests on `diffFileHashes` / `extractChangedSubgraph` cannot + * catch: + * + * - the `isIncremental` decision (post-pipeline eligibility check) + * - `incrementalInProgress` dirty-flag set-before-mutation and + * clear-on-success + * - the importer-closure expansion (1-hop reached via the writable + * set, transitive reachable via bounded BFS) + * - the "forced full rebuild on dirty-flag-from-prior-crash" path + * + * Each test creates a temporary git repo, runs the analyzer, and asserts + * on the resulting `meta.json` and graph state. Cleanup is best-effort + * (Windows LadybugDB handle release can lag; `cleanupTempDir` retries). + */ + +import { execSync } from 'child_process'; +import { writeFile, readFile, copyFile, mkdir } from 'fs/promises'; +import path from 'path'; +import { fileURLToPath } from 'url'; +import { describe, it, expect } from 'vitest'; +import { + getStoragePaths, + saveMeta, + loadMeta, + INCREMENTAL_SCHEMA_VERSION, + type RepoMeta, +} from '../../src/storage/repo-manager.js'; +import { createTempDir } from '../helpers/test-db.js'; + +const HERE = path.dirname(fileURLToPath(import.meta.url)); +const FIXTURE_SRC = path.resolve(HERE, '..', 'fixtures', 'mini-repo', 'src'); + +/** + * Copy the mini-repo fixture into a fresh git-initialized temp directory. + * Returns the temp handle so the caller owns cleanup. + */ +async function setupMiniRepo(): Promise<{ dbPath: string; cleanup: () => Promise }> { + const tmp = await createTempDir('gitnexus-incr-orch-'); + const dest = path.join(tmp.dbPath, 'src'); + await mkdir(dest, { recursive: true }); + // Copy mini-repo fixture files + const names = [ + 'index.ts', + 'handler.ts', + 'validator.ts', + 'formatter.ts', + 'middleware.ts', + 'logger.ts', + 'db.ts', + ]; + for (const n of names) { + await copyFile(path.join(FIXTURE_SRC, n), path.join(dest, n)); + } + execSync('git init', { cwd: tmp.dbPath, stdio: 'pipe' }); + execSync('git -c user.name=test -c user.email=t@t -c commit.gpgsign=false add -A', { + cwd: tmp.dbPath, + stdio: 'pipe', + }); + execSync('git -c user.name=test -c user.email=t@t -c commit.gpgsign=false commit -q -m initial', { + cwd: tmp.dbPath, + stdio: 'pipe', + }); + return tmp; +} + +describe('runFullAnalysis — incremental orchestration', () => { + it('first run populates fileHashes + schemaVersion and clears incrementalInProgress on success', async () => { + const repo = await setupMiniRepo(); + try { + const { runFullAnalysis } = await import('../../src/core/run-analyze.js'); + await runFullAnalysis(repo.dbPath, { skipAgentsMd: true }, { onProgress: () => {} }); + + const { storagePath } = getStoragePaths(repo.dbPath); + const meta = await loadMeta(storagePath); + expect(meta).not.toBeNull(); + expect(meta!.schemaVersion).toBe(INCREMENTAL_SCHEMA_VERSION); + expect(meta!.fileHashes).toBeDefined(); + expect(Object.keys(meta!.fileHashes ?? {}).length).toBeGreaterThan(0); + // Dirty flag MUST be cleared after a successful run. + expect(meta!.incrementalInProgress).toBeUndefined(); + } finally { + await repo.cleanup(); + } + }, 180_000); + + it('second run on unchanged state takes the alreadyUpToDate fast path', async () => { + const repo = await setupMiniRepo(); + try { + const { runFullAnalysis } = await import('../../src/core/run-analyze.js'); + const first = await runFullAnalysis( + repo.dbPath, + { skipAgentsMd: true }, + { onProgress: () => {} }, + ); + expect(first.alreadyUpToDate).toBeUndefined(); + + const second = await runFullAnalysis( + repo.dbPath, + { skipAgentsMd: true }, + { onProgress: () => {} }, + ); + // lastCommit==HEAD && working tree clean (mod GitNexus output) → + // early-return fast path. + expect(second.alreadyUpToDate).toBe(true); + } finally { + await repo.cleanup(); + } + }, 300_000); + + it('second run after a comment-only edit takes the incremental path, clears the dirty flag, and preserves graph stats exactly', async () => { + const repo = await setupMiniRepo(); + try { + const { runFullAnalysis } = await import('../../src/core/run-analyze.js'); + await runFullAnalysis(repo.dbPath, { skipAgentsMd: true }, { onProgress: () => {} }); + const { storagePath } = getStoragePaths(repo.dbPath); + const firstMeta = await loadMeta(storagePath); + + // Modify a source file with a COMMENT-ONLY edit — by construction + // this changes the content hash (driving the incremental code path) + // without changing any symbol, scope binding, call edge, import, + // or community membership. Therefore every graph-stat invariant + // (files / nodes / edges / communities / processes) MUST be + // bit-identical to the first run. Anything else is a regression. + const target = path.join(repo.dbPath, 'src', 'logger.ts'); + const before = await readFile(target, 'utf-8'); + await writeFile(target, before + '\n// touched by test\n', 'utf-8'); + + const second = await runFullAnalysis( + repo.dbPath, + { skipAgentsMd: true }, + { onProgress: () => {} }, + ); + // The early-return alreadyUpToDate path must NOT fire (the dirty + // tree should kick the run through to incremental writeback). + expect(second.alreadyUpToDate).toBeUndefined(); + + const secondMeta = await loadMeta(storagePath); + expect(secondMeta).not.toBeNull(); + // Dirty flag must be cleared on success. + expect(secondMeta!.incrementalInProgress).toBeUndefined(); + // fileHashes[logger.ts] must have rotated to the new content. + expect(secondMeta!.fileHashes?.['src/logger.ts']).toBeDefined(); + expect(secondMeta!.fileHashes?.['src/logger.ts']).not.toBe( + firstMeta!.fileHashes?.['src/logger.ts'], + ); + // Exact-equality stats invariant. DoD §2.7: avoid bounds-only + // assertions that would mask a regression dropping half the graph. + expect(secondMeta!.stats?.files).toBe(firstMeta!.stats?.files); + expect(secondMeta!.stats?.nodes).toBe(firstMeta!.stats?.nodes); + expect(secondMeta!.stats?.edges).toBe(firstMeta!.stats?.edges); + expect(secondMeta!.stats?.communities).toBe(firstMeta!.stats?.communities); + expect(secondMeta!.stats?.processes).toBe(firstMeta!.stats?.processes); + } finally { + await repo.cleanup(); + } + }, 300_000); + + it('incremental output is byte-equivalent to a full rebuild (incremental ≡ --force on the same repo state)', async () => { + // The central correctness contract of this PR: an incremental run + // and a full rebuild from the same repo state must produce identical + // graph stats. We exercise it end-to-end: + // + // 1. setup mini-repo + run analyze (populates the index) + // 2. edit one source file (comment-only — same graph) + // 3. run incremental analyze → record secondMeta + // 4. run analyze --force from the same state → record forceMeta + // 5. assert every stats invariant is exactly equal. + // + // Steps 3 and 4 share the same on-disk file contents, so any + // divergence is purely an artifact of the writeback strategy. If + // any invariant differs, the PR's load-bearing claim is violated. + const repo = await setupMiniRepo(); + try { + const { runFullAnalysis } = await import('../../src/core/run-analyze.js'); + + // Step 1: initial index. + await runFullAnalysis(repo.dbPath, { skipAgentsMd: true }, { onProgress: () => {} }); + + // Step 2: comment-only edit, same as the test above. + const target = path.join(repo.dbPath, 'src', 'logger.ts'); + const original = await readFile(target, 'utf-8'); + await writeFile(target, original + '\n// equivalence test touch\n', 'utf-8'); + + // Step 3: incremental writeback for the edited file. + const incremental = await runFullAnalysis( + repo.dbPath, + { skipAgentsMd: true }, + { onProgress: () => {} }, + ); + expect(incremental.alreadyUpToDate).toBeUndefined(); + const { storagePath } = getStoragePaths(repo.dbPath); + const secondMeta = await loadMeta(storagePath); + expect(secondMeta).not.toBeNull(); + + // Step 4: force a full rebuild from the SAME on-disk file state. + const forced = await runFullAnalysis( + repo.dbPath, + { skipAgentsMd: true, force: true }, + { onProgress: () => {} }, + ); + expect(forced.alreadyUpToDate).toBeUndefined(); + const forceMeta = await loadMeta(storagePath); + expect(forceMeta).not.toBeNull(); + + // Step 5: exact-equality across every stat. `toEqual` would also + // work but `toBe` per-field makes a failure pinpoint the field. + expect(secondMeta!.stats?.files).toBe(forceMeta!.stats?.files); + expect(secondMeta!.stats?.nodes).toBe(forceMeta!.stats?.nodes); + expect(secondMeta!.stats?.edges).toBe(forceMeta!.stats?.edges); + expect(secondMeta!.stats?.communities).toBe(forceMeta!.stats?.communities); + expect(secondMeta!.stats?.processes).toBe(forceMeta!.stats?.processes); + } finally { + await repo.cleanup(); + } + }, 600_000); + + it('a stale incrementalInProgress flag at startup forces a full rebuild that clears it', async () => { + const repo = await setupMiniRepo(); + try { + const { runFullAnalysis } = await import('../../src/core/run-analyze.js'); + // First run lays down a normal index. + await runFullAnalysis(repo.dbPath, { skipAgentsMd: true }, { onProgress: () => {} }); + + // Manually corrupt meta.json with a stale dirty flag — simulates + // a crashed previous incremental run. + const { storagePath } = getStoragePaths(repo.dbPath); + const meta = await loadMeta(storagePath); + expect(meta).not.toBeNull(); + const tampered: RepoMeta = { + ...meta!, + incrementalInProgress: { + startedAt: Date.now() - 60_000, + toWriteCount: 3, + }, + }; + await saveMeta(storagePath, tampered); + + // Next run must detect the flag, force a full rebuild (which + // overwrites meta), and clear the flag. + const recovered = await runFullAnalysis( + repo.dbPath, + { skipAgentsMd: true }, + { onProgress: () => {} }, + ); + // A full rebuild was taken — the alreadyUpToDate fast path + // explicitly cannot fire because the dirty-flag check rewrote + // `options.force` to true. + expect(recovered.alreadyUpToDate).toBeUndefined(); + + const after = await loadMeta(storagePath); + expect(after!.incrementalInProgress).toBeUndefined(); + } finally { + await repo.cleanup(); + } + }, 300_000); +}); diff --git a/gitnexus/test/unit/incremental-parse-cache.test.ts b/gitnexus/test/unit/incremental-parse-cache.test.ts new file mode 100644 index 000000000..757b9cf3a --- /dev/null +++ b/gitnexus/test/unit/incremental-parse-cache.test.ts @@ -0,0 +1,243 @@ +import { describe, it, expect } from 'vitest'; +import { mkdtemp, rm } from 'fs/promises'; +import { tmpdir } from 'os'; +import path from 'path'; +import { + PARSE_CACHE_VERSION, + computeChunkHash, + fileContentHash, + loadParseCache, + saveParseCache, + pruneCache, + type ParseCache, +} from '../../src/storage/parse-cache.js'; +import type { ParseWorkerResult } from '../../src/core/ingestion/workers/parse-worker.js'; + +const minimalResult = (overrides: Partial = {}): ParseWorkerResult => ({ + nodes: [], + relationships: [], + symbols: [], + imports: [], + calls: [], + assignments: [], + heritage: [], + routes: [], + fetchCalls: [], + decoratorRoutes: [], + toolDefs: [], + ormQueries: [], + constructorBindings: [], + fileScopeBindings: [], + parsedFiles: [], + skippedLanguages: {}, + fileCount: 0, + ...overrides, +}); + +describe('computeChunkHash', () => { + it('produces a stable hex hash for a fixed set of (filePath, contentHash) entries', () => { + const entries = [ + { filePath: 'a.ts', contentHash: 'h-a' }, + { filePath: 'b.ts', contentHash: 'h-b' }, + { filePath: 'c.ts', contentHash: 'h-c' }, + ]; + const h1 = computeChunkHash(entries); + const h2 = computeChunkHash(entries); + expect(h1).toBe(h2); + expect(h1).toMatch(/^[a-f0-9]{64}$/); + }); + + it('is order-independent (same files in different order → same hash)', () => { + const order1 = [ + { filePath: 'a.ts', contentHash: 'h-a' }, + { filePath: 'b.ts', contentHash: 'h-b' }, + ]; + const order2 = [ + { filePath: 'b.ts', contentHash: 'h-b' }, + { filePath: 'a.ts', contentHash: 'h-a' }, + ]; + expect(computeChunkHash(order1)).toBe(computeChunkHash(order2)); + }); + + it('changes when any file content changes', () => { + const before = [ + { filePath: 'a.ts', contentHash: 'h-a' }, + { filePath: 'b.ts', contentHash: 'h-b' }, + ]; + const after = [ + { filePath: 'a.ts', contentHash: 'h-a' }, + { filePath: 'b.ts', contentHash: 'h-b-NEW' }, // b.ts content changed + ]; + expect(computeChunkHash(before)).not.toBe(computeChunkHash(after)); + }); + + it('changes when chunk membership changes (file added or removed)', () => { + const small = [ + { filePath: 'a.ts', contentHash: 'h-a' }, + { filePath: 'b.ts', contentHash: 'h-b' }, + ]; + const bigger = [...small, { filePath: 'c.ts', contentHash: 'h-c' }]; + expect(computeChunkHash(small)).not.toBe(computeChunkHash(bigger)); + }); +}); + +describe('fileContentHash', () => { + it('hashes a string deterministically', () => { + expect(fileContentHash('hello')).toBe(fileContentHash('hello')); + expect(fileContentHash('hello')).not.toBe(fileContentHash('hello!')); + expect(fileContentHash('hello')).toMatch(/^[a-f0-9]{64}$/); + }); + + it('handles Buffer input identical to its string form', () => { + const s = 'sentinel'; + expect(fileContentHash(Buffer.from(s))).toBe(fileContentHash(s)); + }); +}); + +describe('PARSE_CACHE_VERSION', () => { + it('embeds the gitnexus package version (so upgrades invalidate the cache)', () => { + // Looks like "1+1.6.4" — schema bump prefix + actual gitnexus version + expect(PARSE_CACHE_VERSION).toMatch(/^\d+\+\d+\.\d+\.\d+/); + }); +}); + +describe('pruneCache', () => { + it('drops entries whose hashes are not in the used-set', () => { + const cache: ParseCache = { + version: PARSE_CACHE_VERSION, + entries: new Map([ + ['hash-A', [minimalResult()]], + ['hash-B', [minimalResult()]], + ['hash-C', [minimalResult()]], + ]), + usedKeys: new Set(['hash-A']), + }; + const removed = pruneCache(cache, cache.usedKeys); + expect(removed).toBe(2); + expect([...cache.entries.keys()].sort()).toEqual(['hash-A']); + }); + + it('returns 0 when every entry is in use', () => { + const cache: ParseCache = { + version: PARSE_CACHE_VERSION, + entries: new Map([ + ['hash-A', [minimalResult()]], + ['hash-B', [minimalResult()]], + ]), + usedKeys: new Set(['hash-A', 'hash-B']), + }; + expect(pruneCache(cache, cache.usedKeys)).toBe(0); + expect(cache.entries.size).toBe(2); + }); +}); + +describe('loadParseCache / saveParseCache (round-trip)', () => { + it('round-trips an empty cache', async () => { + const dir = await mkdtemp(path.join(tmpdir(), 'gnx-pc-')); + try { + const cache: ParseCache = { + version: PARSE_CACHE_VERSION, + entries: new Map(), + usedKeys: new Set(), + }; + await saveParseCache(dir, cache); + const loaded = await loadParseCache(dir); + expect(loaded.version).toBe(PARSE_CACHE_VERSION); + expect(loaded.entries.size).toBe(0); + } finally { + await rm(dir, { recursive: true, force: true }); + } + }); + + it('returns an empty cache when the file is missing', async () => { + const dir = await mkdtemp(path.join(tmpdir(), 'gnx-pc-')); + try { + const loaded = await loadParseCache(dir); + expect(loaded.entries.size).toBe(0); + expect(loaded.usedKeys.size).toBe(0); + } finally { + await rm(dir, { recursive: true, force: true }); + } + }); + + it('returns an empty cache on version mismatch (next-run regen)', async () => { + const dir = await mkdtemp(path.join(tmpdir(), 'gnx-pc-')); + try { + // Write a cache file with a different version directly + const fs = await import('fs/promises'); + await fs.writeFile( + path.join(dir, 'parse-cache.json'), + JSON.stringify({ version: 'foreign-99', entries: { h: [] } }), + 'utf-8', + ); + const loaded = await loadParseCache(dir); + expect(loaded.entries.size).toBe(0); // mismatch → empty + } finally { + await rm(dir, { recursive: true, force: true }); + } + }); + + it('returns an empty cache on corrupt JSON', async () => { + const dir = await mkdtemp(path.join(tmpdir(), 'gnx-pc-')); + try { + const fs = await import('fs/promises'); + await fs.writeFile(path.join(dir, 'parse-cache.json'), '{not-json', 'utf-8'); + const loaded = await loadParseCache(dir); + expect(loaded.entries.size).toBe(0); + } finally { + await rm(dir, { recursive: true, force: true }); + } + }); + + it('round-trips Map and Set values through the JSON replacer/reviver', async () => { + // ParsedFile.scopes[*].typeBindings is a ReadonlyMap. + // Without the replacer/reviver pair, JSON.stringify collapses Maps to + // {} and downstream code that does .get() / iterates entries crashes + // with "is not iterable". This test pins the round-trip behaviour. + const dir = await mkdtemp(path.join(tmpdir(), 'gnx-pc-')); + try { + const innerMap = new Map([ + ['k1', 'v1'], + ['k2', 'v2'], + ]); + const innerSet = new Set(['s1', 's2']); + // Stash the live Map/Set inside a synthetic ParseWorkerResult — we + // only need the serializer to traverse them. Casting to bypass the + // strict shape isn't a problem here: this test is about JSON + // round-tripping of arbitrary nested Map/Set values, not full + // ParseWorkerResult contents. + const fake = minimalResult({ + parsedFiles: [ + { + filePath: 't.ts', + // Cast through unknown to satisfy the readonly Scope shape + // while still smuggling a live Map into the serializer's + // traversal path — see comment block above. + scopes: [{ id: 's1', typeBindings: innerMap, extras: innerSet }], + } as unknown as ParseWorkerResult['parsedFiles'][number], + ], + }); + + const cache: ParseCache = { + version: PARSE_CACHE_VERSION, + entries: new Map([['chunk-h', [fake]]]), + usedKeys: new Set(['chunk-h']), + }; + await saveParseCache(dir, cache); + const loaded = await loadParseCache(dir); + const reloaded = loaded.entries.get('chunk-h')?.[0]; + expect(reloaded).toBeDefined(); + const scope = (reloaded as ParseWorkerResult).parsedFiles[0]?.scopes[0] as unknown as { + typeBindings?: unknown; + extras?: unknown; + }; + expect(scope.typeBindings).toBeInstanceOf(Map); + expect((scope.typeBindings as Map).get('k1')).toBe('v1'); + expect((scope.typeBindings as Map).size).toBe(2); + expect(scope.extras).toBeInstanceOf(Set); + expect((scope.extras as Set).has('s2')).toBe(true); + } finally { + await rm(dir, { recursive: true, force: true }); + } + }); +}); diff --git a/gitnexus/test/unit/incremental-shadow-candidates.test.ts b/gitnexus/test/unit/incremental-shadow-candidates.test.ts new file mode 100644 index 000000000..207cc0b3c --- /dev/null +++ b/gitnexus/test/unit/incremental-shadow-candidates.test.ts @@ -0,0 +1,75 @@ +import { describe, it, expect } from 'vitest'; +import { shadowCandidatesFor } from '../../src/core/incremental/shadow-candidates.js'; + +describe('shadowCandidatesFor', () => { + it('returns an empty list when the input has no recognised module extension', () => { + expect(shadowCandidatesFor('README.md')).toEqual([]); + expect(shadowCandidatesFor('src/foo')).toEqual([]); + expect(shadowCandidatesFor('binary.so')).toEqual([]); + }); + + it('enumerates same-basename / different-extension candidates (pattern a)', () => { + const out = shadowCandidatesFor('src/foo/bar.ts'); + // All non-.ts module extensions on the same path should appear. + expect(out).toContain('src/foo/bar.tsx'); + expect(out).toContain('src/foo/bar.js'); + expect(out).toContain('src/foo/bar.jsx'); + expect(out).toContain('src/foo/bar.mjs'); + expect(out).toContain('src/foo/bar.cjs'); + expect(out).toContain('src/foo/bar.d.ts'); + // ...but NOT the same .ts (you can't shadow yourself). + expect(out).not.toContain('src/foo/bar.ts'); + }); + + it('enumerates directory-style index candidates (pattern b) for both path separators', () => { + const out = shadowCandidatesFor('src/foo/bar.ts'); + // POSIX form + expect(out).toContain('src/foo/bar/index.ts'); + expect(out).toContain('src/foo/bar/index.tsx'); + expect(out).toContain('src/foo/bar/index.js'); + // Windows form + expect(out).toContain('src/foo/bar\\index.ts'); + expect(out).toContain('src/foo/bar\\index.js'); + }); + + it('enumerates bare-file shadows when the added file is a directory index (pattern c)', () => { + const out = shadowCandidatesFor('src/foo/index.ts'); + // Adding foo/index.ts can shadow foo.{ext} (rare but real — converting + // a single-file module into a directory module). + expect(out).toContain('src/foo.ts'); + expect(out).toContain('src/foo.tsx'); + expect(out).toContain('src/foo.js'); + expect(out).toContain('src/foo.jsx'); + expect(out).toContain('src/foo.mjs'); + expect(out).toContain('src/foo.cjs'); + }); + + it('also handles the Windows-separator form of `foo\\index.ts`', () => { + const out = shadowCandidatesFor('src\\foo\\index.ts'); + expect(out).toContain('src\\foo.ts'); + expect(out).toContain('src\\foo.tsx'); + expect(out).toContain('src\\foo.js'); + }); + + it('handles `.d.ts` as a single extension token (not `.ts`)', () => { + // The longest-match scan in shadowCandidatesFor puts `.d.ts` first. + // For `foo.d.ts`, the noExt portion is "foo" (not "foo.d"), so the + // pattern (a) candidates should be the non-.d.ts module variants. + const out = shadowCandidatesFor('types/foo.d.ts'); + expect(out).toContain('types/foo.ts'); + expect(out).toContain('types/foo.tsx'); + expect(out).toContain('types/foo.js'); + // Not the .d.ts itself. + expect(out).not.toContain('types/foo.d.ts'); + }); + + it('deduplicates output (no candidate appears twice)', () => { + const out = shadowCandidatesFor('src/foo/bar.ts'); + expect(out.length).toBe(new Set(out).size); + }); + + it('never includes the input path itself', () => { + const input = 'src/foo/bar.ts'; + expect(shadowCandidatesFor(input)).not.toContain(input); + }); +}); diff --git a/gitnexus/test/unit/incremental-subgraph-extract.test.ts b/gitnexus/test/unit/incremental-subgraph-extract.test.ts new file mode 100644 index 000000000..dc720fd9e --- /dev/null +++ b/gitnexus/test/unit/incremental-subgraph-extract.test.ts @@ -0,0 +1,169 @@ +/** + * Tests for incremental DB writeback subgraph extraction. + * + * Locks the Finding 1 fix (PR #1479 review): cross-file edges between + * two unchanged files MUST land in the writeback subgraph when a third + * (changed) file alters their cross-file resolution. The pre-fix + * behaviour silently dropped those edges, leaving stale rows in the DB. + * + * These tests use synthetic graphs constructed via createKnowledgeGraph + * directly — they don't run the parser, so they're cheap and stable. + */ + +import { describe, it, expect } from 'vitest'; +import type { GraphNode, GraphRelationship } from 'gitnexus-shared'; +import { createKnowledgeGraph } from '../../src/core/graph/graph.js'; +import { + extractChangedSubgraph, + computeEffectiveWriteSet, +} from '../../src/core/incremental/subgraph-extract.js'; + +const makeFileNode = (id: string, filePath: string, label = 'Function'): GraphNode => + ({ + id, + label, + properties: { filePath, name: id }, + }) as unknown as GraphNode; + +const makeWideNode = (id: string, label: 'Community' | 'Process'): GraphNode => + ({ + id, + label, + properties: {}, + }) as unknown as GraphNode; + +const makeRel = ( + id: string, + sourceId: string, + targetId: string, + type = 'CALLS', +): GraphRelationship => + ({ + id, + sourceId, + targetId, + type, + properties: {}, + }) as unknown as GraphRelationship; + +describe('extractChangedSubgraph', () => { + it('includes nodes whose filePath is in the explicit toWriteSet', () => { + const g = createKnowledgeGraph(); + g.addNode(makeFileNode('a', '/repo/a.ts')); + g.addNode(makeFileNode('c', '/repo/c.ts')); + + const sub = extractChangedSubgraph(g, new Set(['/repo/c.ts'])); + + expect(sub.nodes.map((n) => n.id).sort()).toEqual(['c']); + }); + + it('always includes graph-wide nodes (Community, Process)', () => { + const g = createKnowledgeGraph(); + g.addNode(makeFileNode('a', '/repo/a.ts')); + g.addNode(makeWideNode('comm-1', 'Community')); + g.addNode(makeWideNode('proc-1', 'Process')); + + const sub = extractChangedSubgraph(g, new Set([])); // no files changed + + expect(sub.nodes.map((n) => n.id).sort()).toEqual(['comm-1', 'proc-1']); + }); + + it('includes a relationship when at least one endpoint is writable', () => { + const g = createKnowledgeGraph(); + g.addNode(makeFileNode('a:fn', '/repo/a.ts')); + g.addNode(makeFileNode('c:fn', '/repo/c.ts')); + g.addRelationship(makeRel('e1', 'a:fn', 'c:fn', 'CALLS')); + + // toWriteSet already includes A (the orchestrator expanded it via + // computeEffectiveWriteSet) — both endpoints writable, edge fires. + const sub = extractChangedSubgraph(g, new Set(['/repo/a.ts', '/repo/c.ts'])); + + expect(sub.nodes.map((n) => n.id).sort()).toEqual(['a:fn', 'c:fn']); + expect(sub.relationships.map((r) => r.id)).toEqual(['e1']); + }); + + it('skips a relationship entirely between unchanged files', () => { + const g = createKnowledgeGraph(); + g.addNode(makeFileNode('x:fn', '/repo/x.ts')); + g.addNode(makeFileNode('y:fn', '/repo/y.ts')); + g.addRelationship(makeRel('e1', 'x:fn', 'y:fn', 'CALLS')); + + const sub = extractChangedSubgraph(g, new Set(['/repo/c.ts'])); + + expect(sub.nodes).toEqual([]); + expect(sub.relationships).toEqual([]); + }); +}); + +describe('computeEffectiveWriteSet (Finding 1)', () => { + it('barrel re-export — expands the writable set to the consumer file', () => { + // Scenario: file C (a barrel) used to re-export from B; now re-exports + // from D. File A is unchanged byte-wise but its CALLS to foo() now + // resolve to D instead of B. Both A and D are unchanged at the file + // level — but A's edges have shifted. + // + // Pre-fix: toWriteSet={C} → A's nodes not deleted, A→D edge not + // inserted (neither endpoint writable). DB ends up with + // stale A→B and missing A→D. + // Post-fix: the new graph has A→C (A still imports the barrel), so + // A crosses the writable boundary and joins the effective + // write set. deleteNodesForFile(A) then clears the stale + // rows and the subgraph carries the new A→D edge. + const g = createKnowledgeGraph(); + g.addNode(makeFileNode('a:fn', '/repo/a.ts')); + g.addNode(makeFileNode('b:fn', '/repo/b.ts')); + g.addNode(makeFileNode('c:re-export', '/repo/c.ts')); + g.addNode(makeFileNode('d:fn', '/repo/d.ts')); + g.addRelationship(makeRel('e1', 'a:fn', 'c:re-export', 'IMPORTS')); + g.addRelationship(makeRel('e2', 'a:fn', 'd:fn', 'CALLS')); + + const effective = computeEffectiveWriteSet(g, new Set(['/repo/c.ts'])); + + expect([...effective].sort()).toEqual(['/repo/a.ts', '/repo/c.ts']); + }); + + it('picks up edges pointing INTO the changed file (symmetric case)', () => { + const g = createKnowledgeGraph(); + g.addNode(makeFileNode('b:fn', '/repo/b.ts')); + g.addNode(makeFileNode('c:fn', '/repo/c.ts')); + g.addRelationship(makeRel('e1', 'b:fn', 'c:fn', 'CALLS')); + + const effective = computeEffectiveWriteSet(g, new Set(['/repo/c.ts'])); + + expect([...effective].sort()).toEqual(['/repo/b.ts', '/repo/c.ts']); + }); + + it('does not expand when no edge crosses the writable boundary', () => { + const g = createKnowledgeGraph(); + g.addNode(makeFileNode('x:fn', '/repo/x.ts')); + g.addNode(makeFileNode('y:fn', '/repo/y.ts')); + g.addRelationship(makeRel('e1', 'x:fn', 'y:fn', 'CALLS')); + + const effective = computeEffectiveWriteSet(g, new Set(['/repo/c.ts'])); + + expect([...effective].sort()).toEqual(['/repo/c.ts']); + }); + + it('ignores edges to graph-wide nodes (no filePath)', () => { + const g = createKnowledgeGraph(); + g.addNode(makeFileNode('a:fn', '/repo/a.ts')); + g.addNode(makeWideNode('comm-1', 'Community')); + g.addRelationship(makeRel('e1', 'a:fn', 'comm-1', 'BELONGS_TO')); + + const effective = computeEffectiveWriteSet(g, new Set(['/repo/a.ts'])); + + expect([...effective].sort()).toEqual(['/repo/a.ts']); + }); + + it('does not mutate the input set', () => { + const g = createKnowledgeGraph(); + g.addNode(makeFileNode('a:fn', '/repo/a.ts')); + g.addNode(makeFileNode('c:fn', '/repo/c.ts')); + g.addRelationship(makeRel('e1', 'a:fn', 'c:fn', 'CALLS')); + + const input = new Set(['/repo/c.ts']); + computeEffectiveWriteSet(g, input); + + expect([...input]).toEqual(['/repo/c.ts']); + }); +}); From 0daae9370141f4130fdf357823794916f1896699 Mon Sep 17 00:00:00 2001 From: evolution Date: Tue, 12 May 2026 21:03:45 +0800 Subject: [PATCH 5/9] fix(lbug): drain checkpoint result before close (#1506) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * fix(lbug): drain checkpoint result before close * test(lbug): cover checkpoint drain lifecycle * fix(lbug): close query results after reads * fix(lbug): close all stream query results * fix(lbug): harden query result cleanup --------- Co-authored-by: Gergő Magyar --- gitnexus/src/core/lbug/lbug-adapter.ts | 146 ++++-- .../lbug-close-handle-release.test.ts | 43 +- .../unit/lbug-checkpoint-lifecycle.test.ts | 420 ++++++++++++++++++ gitnexus/test/unit/lbug-checkpoint.test.ts | 8 + 4 files changed, 575 insertions(+), 42 deletions(-) create mode 100644 gitnexus/test/unit/lbug-checkpoint-lifecycle.test.ts diff --git a/gitnexus/src/core/lbug/lbug-adapter.ts b/gitnexus/src/core/lbug/lbug-adapter.ts index caa7a58a1..cf8f1cb71 100644 --- a/gitnexus/src/core/lbug/lbug-adapter.ts +++ b/gitnexus/src/core/lbug/lbug-adapter.ts @@ -218,6 +218,65 @@ const runWithSessionLock = async (operation: () => Promise): Promise => const normalizeCopyPath = (filePath: string): string => filePath.replace(/\\/g, '/'); +const closeQueryResult = async (result: lbug.QueryResult): Promise => { + try { + await result.close(); + } catch { + // Best-effort cleanup only. + } +}; + +const drainQueryResult = async ( + queryResult: lbug.QueryResult | lbug.QueryResult[], +): Promise => { + const results = Array.isArray(queryResult) ? queryResult : [queryResult]; + let firstError: unknown; + let hasError = false; + for (const result of results) { + try { + await result.getAll(); + } catch (err) { + if (!hasError) { + firstError = err; + hasError = true; + } + } finally { + await closeQueryResult(result); + } + } + if (hasError) throw firstError; +}; + +const readQueryRows = async ( + queryResult: lbug.QueryResult | lbug.QueryResult[], +): Promise => { + const results = Array.isArray(queryResult) ? queryResult : [queryResult]; + let rows: any[] = []; + let firstError: unknown; + let hasError = false; + for (let i = 0; i < results.length; i++) { + const result = results[i]; + try { + const resultRows = await result.getAll(); + if (i === 0) rows = resultRows; + } catch (err) { + if (!hasError) { + firstError = err; + hasError = true; + } + } finally { + await closeQueryResult(result); + } + } + if (hasError) throw firstError; + return rows; +}; + +const queryAndDrain = async (targetConn: lbug.Connection, cypher: string): Promise => { + const queryResult = await targetConn.query(cypher); + await drainQueryResult(queryResult); +}; + export const initLbug = async (dbPath: string) => { return runWithSessionLock(() => ensureLbugInitialized(dbPath)); }; @@ -319,7 +378,7 @@ const doInitLbug = async (dbPath: string) => { for (const schemaQuery of SCHEMA_QUERIES) { try { - await conn.query(schemaQuery); + await queryAndDrain(conn, schemaQuery); } catch (err) { const msg = err instanceof Error ? err.message : String(err); // Suppression list: @@ -384,14 +443,14 @@ export const loadGraphToLbug = async ( const copyQuery = getCopyQuery(table, normalizedPath); try { - await conn.query(copyQuery); + await queryAndDrain(conn, copyQuery); } catch (err) { try { const retryQuery = copyQuery.replace( 'auto_detect=false)', 'auto_detect=false, IGNORE_ERRORS=true)', ); - await conn.query(retryQuery); + await queryAndDrain(conn, retryQuery); } catch (retryErr) { const retryMsg = retryErr instanceof Error ? retryErr.message : String(retryErr); throw new Error(`COPY failed for ${table}: ${retryMsg.slice(0, 200)}`); @@ -433,14 +492,14 @@ export const loadGraphToLbug = async ( } try { - await conn.query(copyQuery); + await queryAndDrain(conn, copyQuery); } catch (err) { try { const retryQuery = copyQuery.replace( 'auto_detect=false)', 'auto_detect=false, IGNORE_ERRORS=true)', ); - await conn.query(retryQuery); + await queryAndDrain(conn, retryQuery); } catch (retryErr) { const retryMsg = retryErr instanceof Error ? retryErr.message : String(retryErr); warnings.push(`${fromLabel}->${toLabel} (${rows} edges): ${retryMsg.slice(0, 80)}`); @@ -562,11 +621,14 @@ const fallbackRelationshipInserts = async ( const esc = (s: string) => s.replace(/'/g, "''").replace(/\\/g, '\\\\').replace(/\n/g, '\\n').replace(/\r/g, '\\r'); - await conn.query(` + await queryAndDrain( + conn, + ` MATCH (a:${escapeLabel(fromLabel)} {id: '${esc(fromId)}' }), (b:${escapeLabel(toLabel)} {id: '${esc(toId)}' }) CREATE (a)-[:${REL_TABLE_NAME} {type: '${esc(relType)}', confidence: ${confidence}, reason: '${esc(reason)}', step: ${step}}]->(b) - `); + `, + ); } catch { // skip } @@ -679,14 +741,14 @@ export const insertNodeToLbug = async ( if (targetDbPath) { const tempHandle = await openLbugConnection(lbug, targetDbPath); try { - await tempHandle.conn.query(query); + await queryAndDrain(tempHandle.conn, query); return true; } finally { await closeLbugConnection(tempHandle); } } else if (conn) { // Use existing persistent connection (when called from analyze) - await conn.query(query); + await queryAndDrain(conn, query); return true; } @@ -757,7 +819,7 @@ export const batchInsertNodesToLbug = async ( query = `MERGE (n:${t} {id: ${escapeValue(properties.id)}}) SET n.name = ${escapeValue(properties.name)}, n.filePath = ${escapeValue(properties.filePath)}, n.startLine = ${properties.startLine || 0}, n.endLine = ${properties.endLine || 0}, n.content = ${escapeValue(properties.content || '')}${descPart}`; } - await tempConn.query(query); + await queryAndDrain(tempConn, query); inserted++; } catch (e: any) { // Don't console.error here - it corrupts MCP JSON-RPC on stderr @@ -777,11 +839,7 @@ export const executeQuery = async (cypher: string): Promise => { } const queryResult = await conn.query(cypher); - // LadybugDB uses getAll() instead of hasNext()/getNext() - // Query returns QueryResult for single queries, QueryResult[] for multi-statement - const result = Array.isArray(queryResult) ? queryResult[0] : queryResult; - const rows = await result.getAll(); - return rows; + return await readQueryRows(queryResult); }; export const streamQuery = async ( @@ -793,8 +851,10 @@ export const streamQuery = async ( } const queryResult = await conn.query(cypher); - const result = Array.isArray(queryResult) ? queryResult[0] : queryResult; + const results = Array.isArray(queryResult) ? queryResult : [queryResult]; + const result = results[0]; let rowCount = 0; + let streamError: unknown; try { while (await result.hasNext()) { @@ -803,11 +863,14 @@ export const streamQuery = async ( rowCount++; } return rowCount; + } catch (err) { + streamError = err; + throw err; } finally { try { - await result.close(); - } catch { - // Best-effort cleanup only. + await drainQueryResult(results); + } catch (err) { + if (streamError === undefined) throw err; } } }; @@ -829,8 +892,7 @@ export const executePrepared = async ( throw new Error(`Prepare failed: ${errMsg}`); } const queryResult = await conn.execute(stmt, params); - const result = Array.isArray(queryResult) ? queryResult[0] : queryResult; - return await result.getAll(); + return await readQueryRows(queryResult); }; export const executeWithReusedStatement = async ( @@ -852,7 +914,7 @@ export const executeWithReusedStatement = async ( } try { for (const params of subBatch) { - await conn.execute(stmt, params); + await drainQueryResult(await conn.execute(stmt, params)); } } catch (e) { const msg = e instanceof Error ? e.message : String(e); @@ -874,8 +936,7 @@ export const getLbugStats = async (): Promise<{ nodes: number; edges: number }> const queryResult = await conn.query( `MATCH (n:${escapeTableName(tableName)}) RETURN count(n) AS cnt`, ); - const nodeResult = Array.isArray(queryResult) ? queryResult[0] : queryResult; - const nodeRows = await nodeResult.getAll(); + const nodeRows = await readQueryRows(queryResult); if (nodeRows.length > 0) { totalNodes += Number(nodeRows[0]?.cnt ?? nodeRows[0]?.[0] ?? 0); } @@ -889,8 +950,7 @@ export const getLbugStats = async (): Promise<{ nodes: number; edges: number }> const queryResult = await conn.query( `MATCH ()-[r:${REL_TABLE_NAME}]->() RETURN count(r) AS cnt`, ); - const edgeResult = Array.isArray(queryResult) ? queryResult[0] : queryResult; - const edgeRows = await edgeResult.getAll(); + const edgeRows = await readQueryRows(queryResult); if (edgeRows.length > 0) { totalEdges = Number(edgeRows[0]?.cnt ?? edgeRows[0]?.[0] ?? 0); } @@ -926,8 +986,7 @@ export const loadCachedEmbeddings = async (): Promise<{ const check = await conn.query( `MATCH (e:${EMBEDDING_TABLE_NAME}) RETURN e.nodeId AS nodeId, e.chunkIndex AS chunkIndex LIMIT 1`, ); - const checkResult = Array.isArray(check) ? check[0] : check; - await checkResult.getAll(); + await readQueryRows(check); } catch { return { embeddingNodeIds: new Set(), embeddings: [] }; } @@ -951,8 +1010,7 @@ export const loadCachedEmbeddings = async (): Promise<{ throw err; } } - const result = Array.isArray(rows) ? rows[0] : rows; - for (const row of await result.getAll()) { + for (const row of await readQueryRows(rows)) { const nodeId = String(row.nodeId ?? row[0] ?? ''); if (!nodeId) continue; embeddingNodeIds.add(nodeId); @@ -1060,7 +1118,8 @@ export const fetchExistingEmbeddingHashes = async ( export const flushWAL = async (): Promise => { if (!conn) return; try { - await conn.query('CHECKPOINT'); + const checkpointResult = await conn.query('CHECKPOINT'); + await drainQueryResult(checkpointResult); } catch { /* ignore — older LadybugDB or schemaless DB may not accept it */ } @@ -1170,13 +1229,13 @@ export const deleteNodesForFile = async ( const countResult = await targetConn!.query( `MATCH (n:${tn}) WHERE n.filePath = '${escapedPath}' RETURN count(n) AS cnt`, ); - const result = Array.isArray(countResult) ? countResult[0] : countResult; - const rows = await result.getAll(); + const rows = await readQueryRows(countResult); const count = Number(rows[0]?.cnt ?? rows[0]?.[0] ?? 0); if (count > 0) { // Delete nodes (and implicitly their relationships via DETACH) - await targetConn!.query( + await queryAndDrain( + targetConn!, `MATCH (n:${tn}) WHERE n.filePath = '${escapedPath}' DETACH DELETE n`, ); deletedNodes += count; @@ -1188,7 +1247,8 @@ export const deleteNodesForFile = async ( // Also delete any embeddings for nodes in this file try { - await targetConn!.query( + await queryAndDrain( + targetConn!, `MATCH (e:${EMBEDDING_TABLE_NAME}) WHERE e.nodeId STARTS WITH '${escapedPath}' DELETE e`, ); } catch { @@ -1303,7 +1363,7 @@ export const loadFTSExtension = async ( throw new Error('LadybugDB not initialized. Call initLbug first.'); } - const loaded = await extensionManager.ensure((sql) => c.query(sql), 'fts', 'FTS', opts); + const loaded = await extensionManager.ensure((sql) => queryAndDrain(c, sql), 'fts', 'FTS', opts); if (loaded && useModuleState) ftsLoaded = true; return loaded; }; @@ -1333,7 +1393,12 @@ export const loadVectorExtension = async ( throw new Error('LadybugDB not initialized. Call initLbug first.'); } - const loaded = await extensionManager.ensure((sql) => c.query(sql), 'VECTOR', 'VECTOR', opts); + const loaded = await extensionManager.ensure( + (sql) => queryAndDrain(c, sql), + 'VECTOR', + 'VECTOR', + opts, + ); if (loaded && useModuleState) vectorExtensionLoaded = true; return loaded; }; @@ -1365,7 +1430,7 @@ export const createFTSIndex = async ( const query = `CALL CREATE_FTS_INDEX('${tableName}', '${indexName}', [${propList}], stemmer := '${stemmer}')`; try { - await conn.query(query); + await queryAndDrain(conn, query); ensuredFTSIndexes.add(key); } catch (e: any) { if (e.message?.includes('already exists')) { @@ -1449,8 +1514,7 @@ export const queryFTS = async ( try { const queryResult = await conn.query(cypher); - const result = Array.isArray(queryResult) ? queryResult[0] : queryResult; - const rows = await result.getAll(); + const rows = await readQueryRows(queryResult); return rows.map((row: any) => { const node = row.node || row[0] || {}; @@ -1481,7 +1545,7 @@ export const dropFTSIndex = async (tableName: string, indexName: string): Promis } try { - await conn.query(`CALL DROP_FTS_INDEX('${tableName}', '${indexName}')`); + await queryAndDrain(conn, `CALL DROP_FTS_INDEX('${tableName}', '${indexName}')`); } catch { // Index may not exist } finally { diff --git a/gitnexus/test/integration/lbug-close-handle-release.test.ts b/gitnexus/test/integration/lbug-close-handle-release.test.ts index c0a3b8758..65a53bc7d 100644 --- a/gitnexus/test/integration/lbug-close-handle-release.test.ts +++ b/gitnexus/test/integration/lbug-close-handle-release.test.ts @@ -8,9 +8,16 @@ * absorbed by the open-time retry in `lbug-config.ts`. */ import path from 'path'; -import { describe, it } from 'vitest'; +import { describe, expect, it } from 'vitest'; import { createTempDir } from '../helpers/test-db.js'; +/** + * LadybugDB's native Windows file lock can outlive Database.close() for + * same-process close/reopen cycles. Keep true reopen coverage on POSIX and + * cover ordering deterministically in lbug-checkpoint-lifecycle.test.ts. + */ +const itLbugReopen = process.platform === 'win32' ? it.skip : it; + describe('safeClose — close + reopen does not surface lock errors', () => { it('survives 10 sequential open/close/reopen cycles on the same path', async () => { const tmp = await createTempDir('gitnexus-lbug-close-cycle-'); @@ -38,4 +45,38 @@ describe('safeClose — close + reopen does not surface lock errors', () => { await tmp.cleanup(); } }); + + itLbugReopen('flushes WAL when switching between two database paths in one process', async () => { + const repoA = await createTempDir('gitnexus-lbug-switch-a-'); + const repoB = await createTempDir('gitnexus-lbug-switch-b-'); + const dbPathA = path.join(repoA.dbPath, 'lbug'); + const dbPathB = path.join(repoB.dbPath, 'lbug'); + + try { + const adapter = await import('../../src/core/lbug/lbug-adapter.js'); + + await adapter.withLbugDb(dbPathA, async () => { + await adapter.executeQuery( + "CREATE (:File {id: 'file:a', name: 'a.ts', filePath: 'a.ts', content: 'repo a'})", + ); + }); + + await adapter.withLbugDb(dbPathB, async () => { + await adapter.executeQuery( + "CREATE (:File {id: 'file:b', name: 'b.ts', filePath: 'b.ts', content: 'repo b'})", + ); + }); + + const rows = await adapter.withLbugDb(dbPathA, async () => + adapter.executeQuery("MATCH (n:File {id: 'file:a'}) RETURN n.filePath AS filePath"), + ); + + expect(rows).toEqual([{ filePath: 'a.ts' }]); + } finally { + const adapter = await import('../../src/core/lbug/lbug-adapter.js'); + await adapter.closeLbug().catch(() => {}); + await repoA.cleanup(); + await repoB.cleanup(); + } + }); }); diff --git a/gitnexus/test/unit/lbug-checkpoint-lifecycle.test.ts b/gitnexus/test/unit/lbug-checkpoint-lifecycle.test.ts new file mode 100644 index 000000000..3e9ffe8f6 --- /dev/null +++ b/gitnexus/test/unit/lbug-checkpoint-lifecycle.test.ts @@ -0,0 +1,420 @@ +import { afterEach, describe, expect, it, vi } from 'vitest'; + +describe('lbug adapter CHECKPOINT lifecycle', () => { + afterEach(() => { + vi.doUnmock('../../src/core/lbug/lbug-config.js'); + vi.doUnmock('../../src/core/lbug/extension-loader.js'); + vi.resetModules(); + vi.clearAllMocks(); + }); + + it('drains and closes CHECKPOINT result before closing connection and database handles', async () => { + vi.resetModules(); + + const events: string[] = []; + const checkpointResult = { + getAll: vi.fn(async () => { + events.push('checkpoint:getAll'); + return []; + }), + close: vi.fn(() => { + events.push('checkpoint:close'); + }), + }; + const genericResult = { + getAll: vi.fn(async () => []), + close: vi.fn(), + }; + const conn = { + query: vi.fn(async (sql: string) => { + if (sql === 'CHECKPOINT') { + events.push('checkpoint:query'); + return checkpointResult; + } + return genericResult; + }), + close: vi.fn(async () => { + events.push('conn:close'); + }), + }; + const db = { + close: vi.fn(async () => { + events.push('db:close'); + }), + }; + + vi.doMock('../../src/core/lbug/lbug-config.js', () => ({ + openLbugConnection: vi.fn(async () => ({ db, conn })), + closeLbugConnection: vi.fn(async () => {}), + isDbBusyError: vi.fn((err: unknown) => String(err).toLowerCase().includes('lock')), + isOpenRetryExhausted: vi.fn(() => false), + waitForWindowsHandleRelease: vi.fn(async () => true), + })); + vi.doMock('../../src/core/lbug/extension-loader.js', () => ({ + extensionManager: { + ensure: vi.fn(async () => true), + getCapabilities: vi.fn(() => []), + reset: vi.fn(), + }, + })); + + const adapter = await import('../../src/core/lbug/lbug-adapter.js'); + await adapter.initLbug('/tmp/gitnexus-lbug-checkpoint-lifecycle/lbug'); + + events.length = 0; + await adapter.closeLbug(); + + expect(events).toEqual([ + 'checkpoint:query', + 'checkpoint:getAll', + 'checkpoint:close', + 'conn:close', + 'db:close', + ]); + }); + + it('closes normal query results after reading rows', async () => { + vi.resetModules(); + + const events: string[] = []; + const queryResult = { + getAll: vi.fn(async () => { + events.push('query:getAll'); + return [{ id: 'file:a' }]; + }), + close: vi.fn(() => { + events.push('query:close'); + }), + }; + const genericResult = { + getAll: vi.fn(async () => []), + close: vi.fn(), + }; + const conn = { + query: vi.fn(async (sql: string) => { + if (sql === 'MATCH (n:File) RETURN n.id AS id') { + events.push('query:run'); + return queryResult; + } + return genericResult; + }), + close: vi.fn(async () => {}), + }; + const db = { + close: vi.fn(async () => {}), + }; + + vi.doMock('../../src/core/lbug/lbug-config.js', () => ({ + openLbugConnection: vi.fn(async () => ({ db, conn })), + closeLbugConnection: vi.fn(async () => {}), + isDbBusyError: vi.fn((err: unknown) => String(err).toLowerCase().includes('lock')), + isOpenRetryExhausted: vi.fn(() => false), + waitForWindowsHandleRelease: vi.fn(async () => true), + })); + vi.doMock('../../src/core/lbug/extension-loader.js', () => ({ + extensionManager: { + ensure: vi.fn(async () => true), + getCapabilities: vi.fn(() => []), + reset: vi.fn(), + }, + })); + + const adapter = await import('../../src/core/lbug/lbug-adapter.js'); + await adapter.initLbug('/tmp/gitnexus-lbug-query-lifecycle/lbug'); + + events.length = 0; + await expect(adapter.executeQuery('MATCH (n:File) RETURN n.id AS id')).resolves.toEqual([ + { id: 'file:a' }, + ]); + + expect(events).toEqual(['query:run', 'query:getAll', 'query:close']); + + await adapter.closeLbug(); + }); + + it('treats synchronous query result close errors as best-effort cleanup', async () => { + vi.resetModules(); + + const queryResult = { + getAll: vi.fn(async () => [{ id: 'file:a' }]), + close: vi.fn(() => { + throw new Error('close failed'); + }), + }; + const genericResult = { + getAll: vi.fn(async () => []), + close: vi.fn(), + }; + const conn = { + query: vi.fn(async (sql: string) => { + if (sql === 'MATCH (n:File) RETURN n.id AS id') { + return queryResult; + } + return genericResult; + }), + close: vi.fn(async () => {}), + }; + const db = { + close: vi.fn(async () => {}), + }; + + vi.doMock('../../src/core/lbug/lbug-config.js', () => ({ + openLbugConnection: vi.fn(async () => ({ db, conn })), + closeLbugConnection: vi.fn(async () => {}), + isDbBusyError: vi.fn((err: unknown) => String(err).toLowerCase().includes('lock')), + isOpenRetryExhausted: vi.fn(() => false), + waitForWindowsHandleRelease: vi.fn(async () => true), + })); + vi.doMock('../../src/core/lbug/extension-loader.js', () => ({ + extensionManager: { + ensure: vi.fn(async () => true), + getCapabilities: vi.fn(() => []), + reset: vi.fn(), + }, + })); + + const adapter = await import('../../src/core/lbug/lbug-adapter.js'); + await adapter.initLbug('/tmp/gitnexus-lbug-sync-close-lifecycle/lbug'); + + await expect(adapter.executeQuery('MATCH (n:File) RETURN n.id AS id')).resolves.toEqual([ + { id: 'file:a' }, + ]); + expect(queryResult.close).toHaveBeenCalledOnce(); + + await adapter.closeLbug(); + }); + + it('closes later query results when an earlier array result fails to read', async () => { + vi.resetModules(); + + const events: string[] = []; + const firstResult = { + getAll: vi.fn(async () => { + events.push('first:getAll'); + throw new Error('read failed'); + }), + close: vi.fn(() => { + events.push('first:close'); + }), + }; + const secondResult = { + getAll: vi.fn(async () => { + events.push('second:getAll'); + return []; + }), + close: vi.fn(() => { + events.push('second:close'); + }), + }; + const genericResult = { + getAll: vi.fn(async () => []), + close: vi.fn(), + }; + const conn = { + query: vi.fn(async (sql: string) => { + if (sql === 'MATCH (n:File) RETURN n.id AS id') { + return [firstResult, secondResult]; + } + return genericResult; + }), + close: vi.fn(async () => {}), + }; + const db = { + close: vi.fn(async () => {}), + }; + + vi.doMock('../../src/core/lbug/lbug-config.js', () => ({ + openLbugConnection: vi.fn(async () => ({ db, conn })), + closeLbugConnection: vi.fn(async () => {}), + isDbBusyError: vi.fn((err: unknown) => String(err).toLowerCase().includes('lock')), + isOpenRetryExhausted: vi.fn(() => false), + waitForWindowsHandleRelease: vi.fn(async () => true), + })); + vi.doMock('../../src/core/lbug/extension-loader.js', () => ({ + extensionManager: { + ensure: vi.fn(async () => true), + getCapabilities: vi.fn(() => []), + reset: vi.fn(), + }, + })); + + const adapter = await import('../../src/core/lbug/lbug-adapter.js'); + await adapter.initLbug('/tmp/gitnexus-lbug-array-error-lifecycle/lbug'); + + await expect(adapter.executeQuery('MATCH (n:File) RETURN n.id AS id')).rejects.toThrow( + 'read failed', + ); + expect(events).toEqual(['first:getAll', 'first:close', 'second:getAll', 'second:close']); + + await adapter.closeLbug(); + }); + + it('closes non-first stream query results when LadybugDB returns an array', async () => { + vi.resetModules(); + + const events: string[] = []; + const firstResult = { + hasNext: vi + .fn() + .mockImplementationOnce(() => { + events.push('first:hasNext:true'); + return true; + }) + .mockImplementationOnce(() => { + events.push('first:hasNext:false'); + return false; + }), + getNext: vi.fn(async () => { + events.push('first:getNext'); + return { id: 'file:a' }; + }), + getAll: vi.fn(async () => { + events.push('first:getAll'); + return []; + }), + close: vi.fn(() => { + events.push('first:close'); + }), + }; + const secondResult = { + getAll: vi.fn(async () => { + events.push('second:getAll'); + return []; + }), + close: vi.fn(() => { + events.push('second:close'); + }), + }; + const genericResult = { + getAll: vi.fn(async () => []), + close: vi.fn(), + }; + const conn = { + query: vi.fn(async (sql: string) => { + if (sql === 'MATCH (n:File) RETURN n.id AS id') { + events.push('stream:query'); + return [firstResult, secondResult]; + } + return genericResult; + }), + close: vi.fn(async () => {}), + }; + const db = { + close: vi.fn(async () => {}), + }; + + vi.doMock('../../src/core/lbug/lbug-config.js', () => ({ + openLbugConnection: vi.fn(async () => ({ db, conn })), + closeLbugConnection: vi.fn(async () => {}), + isDbBusyError: vi.fn((err: unknown) => String(err).toLowerCase().includes('lock')), + isOpenRetryExhausted: vi.fn(() => false), + waitForWindowsHandleRelease: vi.fn(async () => true), + })); + vi.doMock('../../src/core/lbug/extension-loader.js', () => ({ + extensionManager: { + ensure: vi.fn(async () => true), + getCapabilities: vi.fn(() => []), + reset: vi.fn(), + }, + })); + + const adapter = await import('../../src/core/lbug/lbug-adapter.js'); + await adapter.initLbug('/tmp/gitnexus-lbug-stream-lifecycle/lbug'); + + const rows: unknown[] = []; + events.length = 0; + await expect( + adapter.streamQuery('MATCH (n:File) RETURN n.id AS id', (row) => { + rows.push(row); + }), + ).resolves.toBe(1); + + expect(rows).toEqual([{ id: 'file:a' }]); + expect(events).toEqual([ + 'stream:query', + 'first:hasNext:true', + 'first:getNext', + 'first:hasNext:false', + 'first:getAll', + 'first:close', + 'second:getAll', + 'second:close', + ]); + + await adapter.closeLbug(); + }); + + it('drains stream query results when row handling fails before the result is exhausted', async () => { + vi.resetModules(); + + const events: string[] = []; + const queryResult = { + hasNext: vi.fn(() => { + events.push('stream:hasNext'); + return true; + }), + getNext: vi.fn(async () => { + events.push('stream:getNext'); + return { id: 'file:a' }; + }), + getAll: vi.fn(async () => { + events.push('stream:getAll'); + return [{ id: 'file:b' }]; + }), + close: vi.fn(() => { + events.push('stream:close'); + }), + }; + const genericResult = { + getAll: vi.fn(async () => []), + close: vi.fn(), + }; + const conn = { + query: vi.fn(async (sql: string) => { + if (sql === 'MATCH (n:File) RETURN n.id AS id') { + events.push('stream:query'); + return queryResult; + } + return genericResult; + }), + close: vi.fn(async () => {}), + }; + const db = { + close: vi.fn(async () => {}), + }; + + vi.doMock('../../src/core/lbug/lbug-config.js', () => ({ + openLbugConnection: vi.fn(async () => ({ db, conn })), + closeLbugConnection: vi.fn(async () => {}), + isDbBusyError: vi.fn((err: unknown) => String(err).toLowerCase().includes('lock')), + isOpenRetryExhausted: vi.fn(() => false), + waitForWindowsHandleRelease: vi.fn(async () => true), + })); + vi.doMock('../../src/core/lbug/extension-loader.js', () => ({ + extensionManager: { + ensure: vi.fn(async () => true), + getCapabilities: vi.fn(() => []), + reset: vi.fn(), + }, + })); + + const adapter = await import('../../src/core/lbug/lbug-adapter.js'); + await adapter.initLbug('/tmp/gitnexus-lbug-stream-error-lifecycle/lbug'); + + await expect( + adapter.streamQuery('MATCH (n:File) RETURN n.id AS id', () => { + throw new Error('client disconnected'); + }), + ).rejects.toThrow('client disconnected'); + + expect(events).toEqual([ + 'stream:query', + 'stream:hasNext', + 'stream:getNext', + 'stream:getAll', + 'stream:close', + ]); + + await adapter.closeLbug(); + }); +}); diff --git a/gitnexus/test/unit/lbug-checkpoint.test.ts b/gitnexus/test/unit/lbug-checkpoint.test.ts index 5b68ee9bd..5b9603997 100644 --- a/gitnexus/test/unit/lbug-checkpoint.test.ts +++ b/gitnexus/test/unit/lbug-checkpoint.test.ts @@ -58,6 +58,14 @@ describe('flushWAL / safeClose — consolidation guard (#1376)', () => { expect(matches.length).toBe(1); }); + it('flushWAL drains and closes the CHECKPOINT result before returning', () => { + const flushBody = adapterSource.slice( + adapterSource.indexOf('export const flushWAL'), + adapterSource.indexOf('export const safeClose'), + ); + expect(flushBody).toMatch(/await drainQueryResult\(checkpointResult\)/); + }); + it('conn.close() only appears inside safeClose (with eslint-disable)', () => { // Every conn.close() in the adapter must live inside safeClose, guarded // by the eslint-disable comment. Count occurrences to catch leaks. From a2f1b077009d9b6e1743a5d8550985381d4f2f6d Mon Sep 17 00:00:00 2001 From: Harlan Zhou Date: Tue, 12 May 2026 23:05:58 +0800 Subject: [PATCH 6/9] fix: resolve TypeScript ESM .js extension imports to .ts source files (#1525) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * fix: resolve TypeScript ESM .js extension imports to .ts source files TypeScript ESM requires imports to use .js extensions even when source files are .ts (moduleResolution: node16/bundler). The import resolver now strips JS-family extensions (.js/.jsx/.mjs/.cjs) and retries with TS equivalents (.ts/.tsx/.mts/.cts) when the literal .js file does not exist. This fallback only applies to TypeScript/JavaScript languages. Also adds .mts/.cts to the EXTENSIONS list for completeness. Fixes #1503 * fix: address review findings — normalization, edge-case tests, integration test - Fix makeCtx to use production normalization (.replace backslash) instead of .toLowerCase() (Finding 3) - Add tests for .mjs/.cjs with competing .ts/.mts siblings (Finding 1) - Add tests for ./dir.js → dir/index.ts boundary (Finding 2) - Add integration test verifying full pipeline CALLS edges for ESM .js imports (Finding 4) - Document path alias .js limitation as known follow-up (Finding 5) * chore(autofix): apply prettier + eslint fixes via /autofix command * chore: retrigger CI after bot-only tip commit Co-authored-by: Cursor --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Gergo Magyar Co-authored-by: Cursor --- .../ingestion/import-resolvers/standard.ts | 32 +++- .../core/ingestion/import-resolvers/utils.ts | 4 + .../typescript-esm-js-extension.test.ts | 62 +++++++ .../unit/esm-extension-resolution.test.ts | 153 ++++++++++++++++++ 4 files changed, 250 insertions(+), 1 deletion(-) create mode 100644 gitnexus/test/integration/resolvers/typescript-esm-js-extension.test.ts create mode 100644 gitnexus/test/unit/esm-extension-resolution.test.ts diff --git a/gitnexus/src/core/ingestion/import-resolvers/standard.ts b/gitnexus/src/core/ingestion/import-resolvers/standard.ts index f8aae9625..47e2dabb2 100644 --- a/gitnexus/src/core/ingestion/import-resolvers/standard.ts +++ b/gitnexus/src/core/ingestion/import-resolvers/standard.ts @@ -128,7 +128,21 @@ export const resolveImportPath = ( if (importPath.startsWith('.')) { const resolved = tryResolveWithExtensions(basePath, allFiles); - return cache(resolved); + if (resolved) return cache(resolved); + + // TypeScript ESM: imports use .js/.jsx/.mjs/.cjs but source files are + // .ts/.tsx/.mts/.cts. Strip the JS-family extension and re-resolve. + // NOTE: This fallback only applies to relative imports. Path alias imports + // (e.g. @/utils.js via tsconfig paths) do not yet strip .js extensions — + // that is a known limitation tracked for follow-up. + if (language === SupportedLanguages.TypeScript || language === SupportedLanguages.JavaScript) { + const stripped = stripJsExtension(basePath); + if (stripped !== null) { + return cache(tryResolveWithExtensions(stripped, allFiles)); + } + } + + return cache(null); } // ---- Generic package/absolute import resolution (suffix matching) ---- @@ -182,3 +196,19 @@ export function resolveStandard( export function createStandardStrategy(language: SupportedLanguages): ImportResolverStrategy { return (raw, fp, ctx) => resolveStandard(raw, fp, ctx, language); } + +// ============================================================================ +// ESM extension helpers +// ============================================================================ + +/** JS-family extensions that TypeScript ESM maps to TS equivalents. */ +const JS_EXTENSION_PATTERN = /\.(js|jsx|mjs|cjs)$/; + +/** + * Strip a JS-family extension from a path, returning the stem. + * Returns `null` if the path does not end with a JS-family extension. + */ +export function stripJsExtension(path: string): string | null { + const match = JS_EXTENSION_PATTERN.exec(path); + return match ? path.slice(0, -match[0].length) : null; +} diff --git a/gitnexus/src/core/ingestion/import-resolvers/utils.ts b/gitnexus/src/core/ingestion/import-resolvers/utils.ts index 8d915eb7f..c4d36556c 100644 --- a/gitnexus/src/core/ingestion/import-resolvers/utils.ts +++ b/gitnexus/src/core/ingestion/import-resolvers/utils.ts @@ -9,8 +9,12 @@ export const EXTENSIONS = [ // TypeScript/JavaScript '.tsx', '.ts', + '.mts', + '.cts', '.jsx', '.js', + '.mjs', + '.cjs', '.vue', '/index.tsx', '/index.ts', diff --git a/gitnexus/test/integration/resolvers/typescript-esm-js-extension.test.ts b/gitnexus/test/integration/resolvers/typescript-esm-js-extension.test.ts new file mode 100644 index 000000000..476082563 --- /dev/null +++ b/gitnexus/test/integration/resolvers/typescript-esm-js-extension.test.ts @@ -0,0 +1,62 @@ +/** + * Integration test: TypeScript ESM .js extension imports produce CALLS edges. + * + * Verifies the full pipeline: .js import → resolveImportPath strips .js → + * resolves to .ts → scope-resolver emits CALLS edge. + */ +import { describe, it, expect, beforeAll, afterAll } from 'vitest'; +import path from 'path'; +import fs from 'node:fs'; +import os from 'node:os'; +import { getRelationships, runPipelineFromRepo, type PipelineResult } from './helpers.js'; + +function writeFixtureRepo(root: string, files: Record): void { + for (const [relPath, content] of Object.entries(files)) { + const fullPath = path.join(root, relPath); + fs.mkdirSync(path.dirname(fullPath), { recursive: true }); + fs.writeFileSync(fullPath, content, 'utf8'); + } +} + +describe('TypeScript ESM .js extension → CALLS edges', () => { + let result: PipelineResult; + let repoDir: string | undefined; + + beforeAll(async () => { + repoDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gn-ts-esm-js-ext-')); + writeFixtureRepo(repoDir, { + 'src/utils.ts': ` +export function estimateTokens(text: string): number { + return Math.ceil(text.length / 4); +} +`, + 'src/index.ts': ` +import { estimateTokens } from './utils.js'; + +export function processText(text: string): number { + return estimateTokens(text); +} +`, + }); + result = await runPipelineFromRepo(repoDir, () => {}); + }, 60000); + + afterAll(() => { + if (repoDir !== undefined) fs.rmSync(repoDir, { recursive: true, force: true }); + }); + + it('emits CALLS edge from processText → estimateTokens via .js import', () => { + const calls = getRelationships(result, 'CALLS'); + const edge = calls.find((c) => c.source === 'processText' && c.target === 'estimateTokens'); + expect(edge).toBeDefined(); + expect(edge!.targetFilePath).toBe('src/utils.ts'); + }); + + it('emits IMPORTS edge from index.ts → utils.ts', () => { + const imports = getRelationships(result, 'IMPORTS'); + const edge = imports.find( + (e) => e.sourceFilePath === 'src/index.ts' && e.targetFilePath === 'src/utils.ts', + ); + expect(edge).toBeDefined(); + }); +}); diff --git a/gitnexus/test/unit/esm-extension-resolution.test.ts b/gitnexus/test/unit/esm-extension-resolution.test.ts new file mode 100644 index 000000000..69dc652cf --- /dev/null +++ b/gitnexus/test/unit/esm-extension-resolution.test.ts @@ -0,0 +1,153 @@ +/** + * Unit tests for TypeScript ESM .js extension resolution. + * + * TypeScript ESM requires imports to use .js extensions even when source + * files are .ts. The resolver must map .js → .ts (and .jsx → .tsx, + * .mjs → .mts, .cjs → .cts) when the literal .js file does not exist. + */ + +import { describe, it, expect } from 'vitest'; +import { resolveImportPath } from '../../src/core/ingestion/import-resolvers/standard.js'; +import { stripJsExtension } from '../../src/core/ingestion/import-resolvers/standard.js'; +import { buildSuffixIndex } from '../../src/core/ingestion/import-resolvers/utils.js'; +import { SupportedLanguages } from 'gitnexus-shared'; + +function makeCtx(files: string[]) { + // Match production normalization: only replace backslashes with forward slashes + const normalized = files.map((f) => f.replace(/\\/g, '/')); + const allFilesSet = new Set(files); + const index = buildSuffixIndex(normalized, files); + const cache = new Map(); + return { files, normalized, allFilesSet, index, cache }; +} + +function resolve( + currentFile: string, + importPath: string, + language: SupportedLanguages, + ctx: ReturnType, +): string | null { + return resolveImportPath( + currentFile, + importPath, + ctx.allFilesSet, + ctx.files, + ctx.normalized, + ctx.cache, + language, + null, + ctx.index, + ); +} + +describe('TypeScript ESM .js extension resolution', () => { + it('resolves ./utils.js to ./utils.ts when .js does not exist', () => { + const ctx = makeCtx(['src/index.ts', 'src/utils.ts']); + const result = resolve('src/index.ts', './utils.js', SupportedLanguages.TypeScript, ctx); + expect(result).toBe('src/utils.ts'); + }); + + it('resolves ./component.jsx to ./component.tsx', () => { + const ctx = makeCtx(['src/app.ts', 'src/component.tsx']); + const result = resolve('src/app.ts', './component.jsx', SupportedLanguages.TypeScript, ctx); + expect(result).toBe('src/component.tsx'); + }); + + it('resolves ./config.mjs to ./config.mts', () => { + const ctx = makeCtx(['src/index.ts', 'src/config.mts']); + const result = resolve('src/index.ts', './config.mjs', SupportedLanguages.TypeScript, ctx); + expect(result).toBe('src/config.mts'); + }); + + it('resolves ./legacy.cjs to ./legacy.cts', () => { + const ctx = makeCtx(['src/index.ts', 'src/legacy.cts']); + const result = resolve('src/index.ts', './legacy.cjs', SupportedLanguages.TypeScript, ctx); + expect(result).toBe('src/legacy.cts'); + }); + + it('prefers actual .js file when it exists', () => { + const ctx = makeCtx(['src/index.ts', 'src/utils.js', 'src/utils.ts']); + const result = resolve('src/index.ts', './utils.js', SupportedLanguages.TypeScript, ctx); + expect(result).toBe('src/utils.js'); + }); + + it('resolves relative path with ../ and .js extension', () => { + const ctx = makeCtx(['src/helpers/token.ts', 'src/core/engine.ts']); + const result = resolve( + 'src/core/engine.ts', + '../helpers/token.js', + SupportedLanguages.TypeScript, + ctx, + ); + expect(result).toBe('src/helpers/token.ts'); + }); + + it('works for JavaScript language too', () => { + const ctx = makeCtx(['src/index.js', 'src/utils.ts']); + const result = resolve('src/index.js', './utils.js', SupportedLanguages.JavaScript, ctx); + expect(result).toBe('src/utils.ts'); + }); + + it('does NOT apply ESM fallback for non-TS/JS languages', () => { + const ctx = makeCtx(['src/main.py', 'src/utils.ts']); + const result = resolve('src/main.py', './utils.js', SupportedLanguages.Python, ctx); + expect(result).toBeNull(); + }); + + it('returns null when neither .js nor .ts exists', () => { + const ctx = makeCtx(['src/index.ts']); + const result = resolve('src/index.ts', './missing.js', SupportedLanguages.TypeScript, ctx); + expect(result).toBeNull(); + }); +}); + +describe('ESM extension resolution — .mjs/.cjs with competing siblings', () => { + it('resolves ./config.mjs to .ts when only .ts exists (no .mts)', () => { + const ctx = makeCtx(['src/index.ts', 'src/config.ts']); + const result = resolve('src/index.ts', './config.mjs', SupportedLanguages.TypeScript, ctx); + // .ts wins because EXTENSIONS order tries .ts before .mts + expect(result).toBe('src/config.ts'); + }); + + it('resolves ./config.mjs to .mts when both .ts and .mts exist', () => { + // Note: EXTENSIONS order is .tsx, .ts, .mts, .cts — so .ts wins over .mts. + // This is intentional for a source-analysis tool: we resolve to the first + // matching source file. In practice, having both config.ts and config.mts + // in the same directory is extremely rare. + const ctx = makeCtx(['src/index.ts', 'src/config.ts', 'src/config.mts']); + const result = resolve('src/index.ts', './config.mjs', SupportedLanguages.TypeScript, ctx); + expect(result).toBe('src/config.ts'); + }); + + it('resolves ./config.cjs to .cts when only .cts exists', () => { + const ctx = makeCtx(['src/index.ts', 'src/config.cts']); + const result = resolve('src/index.ts', './config.cjs', SupportedLanguages.TypeScript, ctx); + expect(result).toBe('src/config.cts'); + }); +}); + +describe('ESM extension resolution — directory index boundary', () => { + it('resolves ./dir.js to dir/index.ts when dir/ exists (bundler-mode)', () => { + // After stripping .js from "dir.js" → "dir", tryResolveWithExtensions probes + // "/index.ts" suffix. This matches bundler-mode behavior where bare directory + // imports resolve to index files. Intentional for source-analysis compatibility. + const ctx = makeCtx(['src/index.ts', 'src/dir/index.ts']); + const result = resolve('src/index.ts', './dir.js', SupportedLanguages.TypeScript, ctx); + expect(result).toBe('src/dir/index.ts'); + }); + + it('resolves ./dir/index.js to dir/index.ts', () => { + const ctx = makeCtx(['src/index.ts', 'src/dir/index.ts']); + const result = resolve('src/index.ts', './dir/index.js', SupportedLanguages.TypeScript, ctx); + expect(result).toBe('src/dir/index.ts'); + }); +}); + +describe('stripJsExtension', () => { + it('strips .js', () => expect(stripJsExtension('foo/bar.js')).toBe('foo/bar')); + it('strips .jsx', () => expect(stripJsExtension('foo/bar.jsx')).toBe('foo/bar')); + it('strips .mjs', () => expect(stripJsExtension('foo/bar.mjs')).toBe('foo/bar')); + it('strips .cjs', () => expect(stripJsExtension('foo/bar.cjs')).toBe('foo/bar')); + it('returns null for .ts', () => expect(stripJsExtension('foo/bar.ts')).toBeNull()); + it('returns null for no extension', () => expect(stripJsExtension('foo/bar')).toBeNull()); +}); From 8083c39f6d8271c6ec88ff53db127fe4d11b217e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Gerg=C5=91=20Magyar?= Date: Tue, 12 May 2026 16:56:31 +0100 Subject: [PATCH 7/9] feat(php): migrate PHP to scope-based resolution model (#938) [supersedes #1124] (#1497) --- .../scope-resolution/method-dispatch-index.ts | 52 +- .../registries/lookup-core.ts | 19 +- gitnexus/src/core/ingestion/languages/php.ts | 28 +- .../ingestion/languages/php/arity-metadata.ts | 73 ++ .../src/core/ingestion/languages/php/arity.ts | 47 + .../ingestion/languages/php/cache-stats.ts | 30 + .../core/ingestion/languages/php/captures.ts | 806 ++++++++++++++++++ .../languages/php/import-decomposer.ts | 304 +++++++ .../ingestion/languages/php/import-target.ts | 140 +++ .../src/core/ingestion/languages/php/index.ts | 73 ++ .../core/ingestion/languages/php/interpret.ts | 250 ++++++ .../ingestion/languages/php/merge-bindings.ts | 51 ++ .../languages/php/namespace-siblings.ts | 335 ++++++++ .../src/core/ingestion/languages/php/query.ts | 332 ++++++++ .../languages/php/receiver-binding.ts | 136 +++ .../ingestion/languages/php/scope-resolver.ts | 421 +++++++++ .../ingestion/languages/php/simple-hooks.ts | 134 +++ .../core/ingestion/registry-primary-flag.ts | 1 + .../contract/scope-resolver.ts | 68 ++ .../graph-bridge/method-dispatch.ts | 13 +- .../graph-bridge/node-lookup.ts | 5 + .../passes/free-call-fallback.ts | 82 +- .../passes/overload-narrowing.ts | 30 +- .../passes/receiver-bound-calls.ts | 27 +- .../scope-resolution/pipeline/registry.ts | 2 + .../scope-resolution/pipeline/run.ts | 17 +- .../src/core/ingestion/tree-sitter-queries.ts | 10 + .../app/{BProvider.php => Models/User.php} | 0 .../php-calls/app/Services/UserService.php | 7 +- .../app/Services/Dynamic.php | 113 +++ .../app/Services/OtherTargets.php | 16 + .../app/Services/Targets.php | 36 + .../php-dynamic-calls/composer.json | 5 + .../app/Models/User.php | 8 + .../app/Other/User.php | 8 + .../app/Services/Service.php | 25 + .../php-fqn-cross-namespace/composer.json | 7 + .../app/Models/ChildModel.php | 10 + .../app/Models/Orphan.php | 8 + .../app/Models/ParentModel.php | 10 + .../app/Services/Caller.php | 42 + .../php-mro-arity-mismatch/composer.json | 5 + .../composer.json | 8 + .../src/App/Caller.php | 19 + .../src/App/Utils/Caller.php | 10 + .../src/App/Utils/Format.php | 6 + .../src/Vendor/Utils/Format.php | 6 + .../php-parent-vs-trait/app/Auditable.php | 8 + .../php-parent-vs-trait/app/Base.php | 8 + .../php-parent-vs-trait/app/Child.php | 14 + .../php-parent-vs-trait/composer.json | 7 + .../app/Models/Consumer.php | 20 + .../app/Traits/TraitA.php | 10 + .../app/Traits/TraitB.php | 10 + .../app/Traits/TraitC.php | 8 + .../php-transitive-traits/composer.json | 7 + .../app/Models/UserRepo.php | 8 + .../app/Services/Mixed.php | 23 + .../php-typed-property-dedup/composer.json | 5 + .../app/Models/Handler.php | 26 + .../app/Services/Caller.php | 58 ++ .../composer.json | 5 + .../app/Services/Caller.php | 30 + .../app/Utils/Logger.php | 19 + .../php-variadic-arity-minimum/composer.json | 7 + .../test/integration/resolvers/helpers.ts | 46 + .../test/integration/resolvers/php.test.ts | 490 ++++++++++- .../test/unit/registry-primary-flag.test.ts | 15 +- .../overload-narrowing.test.ts | 19 +- .../pick-implicit-this-overload.test.ts | 156 ++++ .../unit/scope-resolution/registries.test.ts | 45 +- 71 files changed, 4846 insertions(+), 33 deletions(-) create mode 100644 gitnexus/src/core/ingestion/languages/php/arity-metadata.ts create mode 100644 gitnexus/src/core/ingestion/languages/php/arity.ts create mode 100644 gitnexus/src/core/ingestion/languages/php/cache-stats.ts create mode 100644 gitnexus/src/core/ingestion/languages/php/captures.ts create mode 100644 gitnexus/src/core/ingestion/languages/php/import-decomposer.ts create mode 100644 gitnexus/src/core/ingestion/languages/php/import-target.ts create mode 100644 gitnexus/src/core/ingestion/languages/php/index.ts create mode 100644 gitnexus/src/core/ingestion/languages/php/interpret.ts create mode 100644 gitnexus/src/core/ingestion/languages/php/merge-bindings.ts create mode 100644 gitnexus/src/core/ingestion/languages/php/namespace-siblings.ts create mode 100644 gitnexus/src/core/ingestion/languages/php/query.ts create mode 100644 gitnexus/src/core/ingestion/languages/php/receiver-binding.ts create mode 100644 gitnexus/src/core/ingestion/languages/php/scope-resolver.ts create mode 100644 gitnexus/src/core/ingestion/languages/php/simple-hooks.ts rename gitnexus/test/fixtures/cross-file-binding/php-consumer-before-provider/app/{BProvider.php => Models/User.php} (100%) create mode 100644 gitnexus/test/fixtures/lang-resolution/php-dynamic-calls/app/Services/Dynamic.php create mode 100644 gitnexus/test/fixtures/lang-resolution/php-dynamic-calls/app/Services/OtherTargets.php create mode 100644 gitnexus/test/fixtures/lang-resolution/php-dynamic-calls/app/Services/Targets.php create mode 100644 gitnexus/test/fixtures/lang-resolution/php-dynamic-calls/composer.json create mode 100644 gitnexus/test/fixtures/lang-resolution/php-fqn-cross-namespace/app/Models/User.php create mode 100644 gitnexus/test/fixtures/lang-resolution/php-fqn-cross-namespace/app/Other/User.php create mode 100644 gitnexus/test/fixtures/lang-resolution/php-fqn-cross-namespace/app/Services/Service.php create mode 100644 gitnexus/test/fixtures/lang-resolution/php-fqn-cross-namespace/composer.json create mode 100644 gitnexus/test/fixtures/lang-resolution/php-mro-arity-mismatch/app/Models/ChildModel.php create mode 100644 gitnexus/test/fixtures/lang-resolution/php-mro-arity-mismatch/app/Models/Orphan.php create mode 100644 gitnexus/test/fixtures/lang-resolution/php-mro-arity-mismatch/app/Models/ParentModel.php create mode 100644 gitnexus/test/fixtures/lang-resolution/php-mro-arity-mismatch/app/Services/Caller.php create mode 100644 gitnexus/test/fixtures/lang-resolution/php-mro-arity-mismatch/composer.json create mode 100644 gitnexus/test/fixtures/lang-resolution/php-namespace-fallback-isolation/composer.json create mode 100644 gitnexus/test/fixtures/lang-resolution/php-namespace-fallback-isolation/src/App/Caller.php create mode 100644 gitnexus/test/fixtures/lang-resolution/php-namespace-fallback-isolation/src/App/Utils/Caller.php create mode 100644 gitnexus/test/fixtures/lang-resolution/php-namespace-fallback-isolation/src/App/Utils/Format.php create mode 100644 gitnexus/test/fixtures/lang-resolution/php-namespace-fallback-isolation/src/Vendor/Utils/Format.php create mode 100644 gitnexus/test/fixtures/lang-resolution/php-parent-vs-trait/app/Auditable.php create mode 100644 gitnexus/test/fixtures/lang-resolution/php-parent-vs-trait/app/Base.php create mode 100644 gitnexus/test/fixtures/lang-resolution/php-parent-vs-trait/app/Child.php create mode 100644 gitnexus/test/fixtures/lang-resolution/php-parent-vs-trait/composer.json create mode 100644 gitnexus/test/fixtures/lang-resolution/php-transitive-traits/app/Models/Consumer.php create mode 100644 gitnexus/test/fixtures/lang-resolution/php-transitive-traits/app/Traits/TraitA.php create mode 100644 gitnexus/test/fixtures/lang-resolution/php-transitive-traits/app/Traits/TraitB.php create mode 100644 gitnexus/test/fixtures/lang-resolution/php-transitive-traits/app/Traits/TraitC.php create mode 100644 gitnexus/test/fixtures/lang-resolution/php-transitive-traits/composer.json create mode 100644 gitnexus/test/fixtures/lang-resolution/php-typed-property-dedup/app/Models/UserRepo.php create mode 100644 gitnexus/test/fixtures/lang-resolution/php-typed-property-dedup/app/Services/Mixed.php create mode 100644 gitnexus/test/fixtures/lang-resolution/php-typed-property-dedup/composer.json create mode 100644 gitnexus/test/fixtures/lang-resolution/php-unresolved-receiver-arity/app/Models/Handler.php create mode 100644 gitnexus/test/fixtures/lang-resolution/php-unresolved-receiver-arity/app/Services/Caller.php create mode 100644 gitnexus/test/fixtures/lang-resolution/php-unresolved-receiver-arity/composer.json create mode 100644 gitnexus/test/fixtures/lang-resolution/php-variadic-arity-minimum/app/Services/Caller.php create mode 100644 gitnexus/test/fixtures/lang-resolution/php-variadic-arity-minimum/app/Utils/Logger.php create mode 100644 gitnexus/test/fixtures/lang-resolution/php-variadic-arity-minimum/composer.json create mode 100644 gitnexus/test/unit/scope-resolution/pick-implicit-this-overload.test.ts diff --git a/gitnexus-shared/src/scope-resolution/method-dispatch-index.ts b/gitnexus-shared/src/scope-resolution/method-dispatch-index.ts index d09e8fa89..dcaa45055 100644 --- a/gitnexus-shared/src/scope-resolution/method-dispatch-index.ts +++ b/gitnexus-shared/src/scope-resolution/method-dispatch-index.ts @@ -40,11 +40,27 @@ export interface MethodDispatchIndex { readonly mroByOwnerDefId: ReadonlyMap; /** Interfaces / traits → classes that implement them. */ readonly implsByInterfaceDefId: ReadonlyMap; + /** + * Optional parallel MRO view that EXCLUDES mixin-like augmentation + * (e.g., PHP traits). Populated only when the input supplies + * `computeExtendsOnlyMro`. Used by the super-branch dispatch in + * `receiver-bound-calls` so that `parent::method()` walks the + * inheritance chain only, not the trait-augmented one. Undefined for + * languages without mixin-like semantics — callers should fall back + * to `mroFor` when this is missing. + */ + readonly extendsOnlyMroByOwnerDefId?: ReadonlyMap; /** `mroByOwnerDefId.get`, with an empty frozen array on miss. */ mroFor(ownerDefId: DefId): readonly DefId[]; /** `implsByInterfaceDefId.get`, with an empty frozen array on miss. */ implementorsOf(interfaceDefId: DefId): readonly DefId[]; + /** + * `extendsOnlyMroByOwnerDefId.get`, with an empty frozen array on miss. + * Undefined when `extendsOnlyMroByOwnerDefId` was not populated; callers + * should treat this as equivalent to `mroFor` for non-mixin languages. + */ + readonly extendsOnlyMroFor?: (ownerDefId: DefId) => readonly DefId[]; } export interface MethodDispatchInput { @@ -81,12 +97,25 @@ export interface MethodDispatchInput { * write-wins policy and fires at most once per unique owner. */ readonly implementsOf: (ownerDefId: DefId) => readonly DefId[]; + /** + * Optional: return the EXTENDS-only ancestor chain for `ownerDefId`, + * excluding the owner itself AND any mixin-like augmentation (e.g., + * PHP traits). Languages without mixin semantics leave this undefined + * and the index's `extendsOnlyMroByOwnerDefId` stays unpopulated. + * + * Same contract as `computeMro`: pure, deterministic, `[]` on no parents. + * Called at most once per unique owner (first-write-wins). + */ + readonly computeExtendsOnlyMro?: (ownerDefId: DefId) => readonly DefId[]; } // ─── Builder ──────────────────────────────────────────────────────────────── export function buildMethodDispatchIndex(input: MethodDispatchInput): MethodDispatchIndex { const mroByOwnerDefId = new Map(); + const extendsOnlyByOwnerDefId = input.computeExtendsOnlyMro + ? new Map() + : undefined; const implsBuilding = new Map(); const implsSeen = new Map>(); @@ -97,6 +126,14 @@ export function buildMethodDispatchIndex(input: MethodDispatchInput): MethodDisp const chain = input.computeMro(ownerId); mroByOwnerDefId.set(ownerId, Object.freeze(chain.slice())); } + if ( + input.computeExtendsOnlyMro !== undefined && + extendsOnlyByOwnerDefId !== undefined && + !extendsOnlyByOwnerDefId.has(ownerId) + ) { + const extOnly = input.computeExtendsOnlyMro(ownerId); + extendsOnlyByOwnerDefId.set(ownerId, Object.freeze(extOnly.slice())); + } for (const ifaceId of input.implementsOf(ownerId)) { let seen = implsSeen.get(ifaceId); @@ -121,7 +158,7 @@ export function buildMethodDispatchIndex(input: MethodDispatchInput): MethodDisp implsByInterfaceDefId.set(ifaceId, Object.freeze(owners.slice())); } - return wrapIndex(mroByOwnerDefId, implsByInterfaceDefId); + return wrapIndex(mroByOwnerDefId, implsByInterfaceDefId, extendsOnlyByOwnerDefId); } // ─── Internal ─────────────────────────────────────────────────────────────── @@ -131,8 +168,9 @@ const EMPTY: readonly DefId[] = Object.freeze([]); function wrapIndex( mroByOwnerDefId: Map, implsByInterfaceDefId: Map, + extendsOnlyMroByOwnerDefId: Map | undefined, ): MethodDispatchIndex { - return { + const base: MethodDispatchIndex = { mroByOwnerDefId, implsByInterfaceDefId, mroFor(ownerDefId: DefId): readonly DefId[] { @@ -142,4 +180,14 @@ function wrapIndex( return implsByInterfaceDefId.get(interfaceDefId) ?? EMPTY; }, }; + if (extendsOnlyMroByOwnerDefId !== undefined) { + return { + ...base, + extendsOnlyMroByOwnerDefId, + extendsOnlyMroFor(ownerDefId: DefId): readonly DefId[] { + return extendsOnlyMroByOwnerDefId.get(ownerDefId) ?? EMPTY; + }, + }; + } + return base; } diff --git a/gitnexus-shared/src/scope-resolution/registries/lookup-core.ts b/gitnexus-shared/src/scope-resolution/registries/lookup-core.ts index a18ad4930..fff3e7adb 100644 --- a/gitnexus-shared/src/scope-resolution/registries/lookup-core.ts +++ b/gitnexus-shared/src/scope-resolution/registries/lookup-core.ts @@ -423,13 +423,30 @@ function applyArityFilter( } let anyCompatible = false; + let anyUnknown = false; for (const state of perCandidate.values()) { const verdict = arityFn(callsite, state.def); state.signals.arityVerdict = verdict; if (verdict === 'compatible') anyCompatible = true; + else if (verdict === 'unknown') anyUnknown = true; } - if (!anyCompatible) return; + // When ALL candidates are 'incompatible' (none compatible, none unknown), + // the call is genuinely arity-broken — drop every candidate so the + // registry returns no resolution. This matches the PHP variadic case + // f(int $req, ...$rest) called with zero args: every candidate definitively + // rejects, and emitting an edge to a definitively-rejected callable is + // a false positive. When some candidates are 'unknown' (missing metadata), + // keep the set so downstream evidence can break the tie — that's the + // original safety-fallback behavior. + if (!anyCompatible) { + if (!anyUnknown) { + for (const defId of perCandidate.keys()) { + perCandidate.delete(defId); + } + } + return; + } // Filter: when at least one compatible candidate exists, drop incompatibles. for (const [defId, state] of perCandidate) { diff --git a/gitnexus/src/core/ingestion/languages/php.ts b/gitnexus/src/core/ingestion/languages/php.ts index eb8f296f8..caca85335 100644 --- a/gitnexus/src/core/ingestion/languages/php.ts +++ b/gitnexus/src/core/ingestion/languages/php.ts @@ -5,12 +5,22 @@ * and standard export/import resolution. PHP files can use a variety of * extensions from legacy versions through modern PHP 8. */ +import { + emitPhpScopeCaptures, + interpretPhpImport, + interpretPhpTypeBinding, + phpArityCompatibility, + phpMergeBindings, + resolvePhpImportTarget, + phpBindingScopeFor, + phpImportOwningScope, + phpReceiverBinding, +} from './php/index.js'; import { SupportedLanguages } from 'gitnexus-shared'; import { createClassExtractor } from '../class-extractors/generic.js'; import { phpClassConfig } from '../class-extractors/configs/php.js'; -import { defineLanguage } from '../language-provider.js'; -import type { AstFrameworkPatternConfig } from '../language-provider.js'; +import { defineLanguage, type AstFrameworkPatternConfig } from '../language-provider.js'; import { typeConfig as phpConfig } from '../type-extractors/php.js'; import { phpExportChecker } from '../export-detection.js'; import { createImportResolver } from '../import-resolvers/resolver-factory.js'; @@ -289,4 +299,18 @@ export const phpProvider = defineLanguage({ descriptionExtractor: phpDescriptionExtractor, isRouteFile: isPhpRouteFile, builtInNames: BUILT_INS, + // ── RFC #909 Ring 3: scope-based resolution hooks ────────────────────── + emitScopeCaptures: emitPhpScopeCaptures, + interpretImport: interpretPhpImport, + interpretTypeBinding: interpretPhpTypeBinding, + // LanguageProvider uses (def, callsite); phpArityCompatibility uses (def, callsite) — same. + arityCompatibility: phpArityCompatibility, + // LanguageProvider adapter: (parsedImport, workspaceIndex) → string | null + resolveImportTarget: resolvePhpImportTarget, + // mergeBindings on LanguageProvider: (scope, bindings) — ignore scope id, + // delegate to phpMergeBindings which uses binding origin tiers. + mergeBindings: (_scope, bindings) => [...phpMergeBindings(bindings)], + bindingScopeFor: phpBindingScopeFor, + importOwningScope: phpImportOwningScope, + receiverBinding: phpReceiverBinding, }); diff --git a/gitnexus/src/core/ingestion/languages/php/arity-metadata.ts b/gitnexus/src/core/ingestion/languages/php/arity-metadata.ts new file mode 100644 index 000000000..40662bd61 --- /dev/null +++ b/gitnexus/src/core/ingestion/languages/php/arity-metadata.ts @@ -0,0 +1,73 @@ +/** + * Extract PHP arity metadata from a method-like tree-sitter node — + * `method_declaration` or `function_definition`. + * + * Reuses `phpMethodConfig.extractParameters` so scope-extracted defs + * carry the same arity semantics as the legacy parse-worker path: + * - `variadic_parameter` (`...$args`) collapses `parameterCount` to + * `undefined`, which `phpArityCompatibility` then treats as + * "max unknown" — the candidate stays eligible at `argCount >= required`. + * - Defaulted parameters (`= expr`) contribute to `optionalCount`; + * `requiredParameterCount = total − optionalCount − (variadic ? 1 : 0)`. + * The variadic slot itself accepts zero args so it is subtracted from + * the required count — `f(int $a, ...$rest)` requires exactly 1 arg, + * not 2, and `f(...$rest)` requires 0. + * - `property_promotion_parameter` (constructor-promoted) is counted + * the same as `simple_parameter` since both consume an argument slot. + * - `parameterTypes` collects declared type names; a literal `'...'` + * marker is appended for variadic methods so `phpArityCompatibility` + * can detect them without re-reading the AST. + */ + +import type { SyntaxNode } from '../../utils/ast-helpers.js'; +import { phpMethodConfig } from '../../method-extractors/configs/php.js'; + +interface PhpArityMetadata { + readonly parameterCount: number | undefined; + readonly requiredParameterCount: number | undefined; + readonly parameterTypes: readonly string[] | undefined; +} + +export function computePhpArityMetadata(fnNode: SyntaxNode): PhpArityMetadata { + const params = phpMethodConfig.extractParameters?.(fnNode) ?? []; + + let hasVariadic = false; + let optionalCount = 0; + const types: string[] = []; + + for (const p of params) { + if (p.isVariadic) { + hasVariadic = true; + } else if (p.isOptional) { + optionalCount++; + } + if (p.type !== null) types.push(p.type); + } + // PHP variadic marker convention: append the literal '...' string to + // `parameterTypes`. This is intentionally DIFFERENT from C#, which uses + // the literal 'params' (its source-language keyword). The shared + // `narrowOverloadCandidates` pass in `scope-resolution/passes/overload- + // narrowing.ts` checks for the C# 'params' marker — that branch is + // dead code for PHP because PHP variadic methods set `parameterCount + // = undefined` (see line below), which skips the `max !== undefined` + // gate that hosts the 'params' check. PHP's actual variadic-aware + // arity logic lives in `phpArityCompatibility` (arity.ts) and now + // also in `phpEmitUnresolvedReceiverEdges` (scope-resolver.ts), both + // of which check `'...'`. Finding 9 of PR #1497 adversarial review. + if (hasVariadic) types.push('...'); + + const total = params.length; + // Variadic methods accept any arg count ≥ required — leave `parameterCount` + // undefined so the registry treats max as unknown. + const parameterCount = hasVariadic ? undefined : total; + // The variadic slot itself accepts zero args; subtract it from the required + // count so PHP's ArgumentCountError-equivalent calls (too few args before + // the variadic) are correctly rejected by arity compatibility. + const requiredParameterCount = total - optionalCount - (hasVariadic ? 1 : 0); + + return { + parameterCount, + requiredParameterCount, + parameterTypes: types.length > 0 ? types : undefined, + }; +} diff --git a/gitnexus/src/core/ingestion/languages/php/arity.ts b/gitnexus/src/core/ingestion/languages/php/arity.ts new file mode 100644 index 000000000..5b99a73c7 --- /dev/null +++ b/gitnexus/src/core/ingestion/languages/php/arity.ts @@ -0,0 +1,47 @@ +/** + * PHP arity check, accommodating variadic (`...$args`) and default parameters. + * + * The `def` metadata synthesized by `arity-metadata.ts`: + * - `parameterCount` — total formal parameters; `undefined` when + * the method has a variadic `...$param`. + * - `requiredParameterCount` — min required (excludes defaulted params + * and the variadic itself). + * - `parameterTypes` — declared type strings; contains the + * literal `'...'` when the method is variadic. + * + * Verdicts: + * - `'compatible'` — `required <= argCount <= max`, OR the def has + * variadic (any `argCount >= required`). + * - `'incompatible'` — argCount below required, or above max with no variadic. + * - `'unknown'` — metadata absent / incomplete; named-args can satisfy + * any arity so we return unknown when we detect them. + * + * PHP supports named arguments (PHP 8.0+): `save(force: true)`. Named-arg + * call sites cannot be arity-checked statically without parsing arg names, + * so we return `'unknown'` when the callsite carries named args (signalled + * by a negative `arity` value per the shared Callsite contract). + */ + +import type { Callsite, SymbolDefinition } from 'gitnexus-shared'; + +export function phpArityCompatibility( + def: SymbolDefinition, + callsite: Callsite, +): 'compatible' | 'unknown' | 'incompatible' { + const max = def.parameterCount; + const min = def.requiredParameterCount; + if (max === undefined && min === undefined) return 'unknown'; + + const argCount = callsite.arity; + // Negative arity signals named-argument call sites — can't narrow statically. + if (!Number.isFinite(argCount) || argCount < 0) return 'unknown'; + + const hasVarArgs = + def.parameterTypes !== undefined && + def.parameterTypes.some((t) => t === '...' || t.startsWith('...')); + + if (min !== undefined && argCount < min) return 'incompatible'; + if (max !== undefined && argCount > max && !hasVarArgs) return 'incompatible'; + + return 'compatible'; +} diff --git a/gitnexus/src/core/ingestion/languages/php/cache-stats.ts b/gitnexus/src/core/ingestion/languages/php/cache-stats.ts new file mode 100644 index 000000000..508eec580 --- /dev/null +++ b/gitnexus/src/core/ingestion/languages/php/cache-stats.ts @@ -0,0 +1,30 @@ +/** + * Dev-mode counters for the cross-phase scope-captures parse cache + * (PHP mirror of `languages/csharp/cache-stats.ts`). + * + * Gated by `PROF_SCOPE_RESOLUTION=1`. Production builds fold every + * increment into dead code via the module-level `PROF` constant, so + * the hot path in `captures.ts` stays branch-free. + */ + +const PROF = process.env.PROF_SCOPE_RESOLUTION === '1'; + +let CACHE_HITS = 0; +let CACHE_MISSES = 0; + +export function recordCacheHit(): void { + if (PROF) CACHE_HITS++; +} + +export function recordCacheMiss(): void { + if (PROF) CACHE_MISSES++; +} + +export function getPhpCaptureCacheStats(): { hits: number; misses: number } { + return { hits: CACHE_HITS, misses: CACHE_MISSES }; +} + +export function resetPhpCaptureCacheStats(): void { + CACHE_HITS = 0; + CACHE_MISSES = 0; +} diff --git a/gitnexus/src/core/ingestion/languages/php/captures.ts b/gitnexus/src/core/ingestion/languages/php/captures.ts new file mode 100644 index 000000000..692d87bc1 --- /dev/null +++ b/gitnexus/src/core/ingestion/languages/php/captures.ts @@ -0,0 +1,806 @@ +/** + * `emitScopeCaptures` for PHP (RFC #909 Ring 3 LANG-php). + * + * Drives the PHP scope query against tree-sitter-php and groups raw + * matches into `CaptureMatch[]` for the central extractor. Layers two + * synthesized streams on top: + * + * 1. **Decomposed use declarations** — each `namespace_use_declaration` + * is re-emitted with `@import.kind/source/name/alias` markers so + * `interpretPhpImport` can recover the ParsedImport shape without + * re-parsing raw text. Grouped uses fan out to one match per clause. + * + * 2. **Receiver-binding synthesis** — `$this` and `parent` type-bindings + * are synthesized on every non-static method entry. PHP's grammar + * does not express "implicit receiver of a non-static class method" + * via a clean `.scm` pattern, so we walk up the AST in code. + * + * 3. **Arity metadata synthesis** — `@declaration.parameter-count` / + * `@declaration.required-parameter-count` / `@declaration.parameter-types` + * are synthesized on function-like declarations so the registry can + * narrow overloads. + * + * 4. **PHPDoc synthesis** — @param and @return annotations in comment + * nodes preceding method/function declarations are extracted and emitted + * as `@type-binding.parameter` and `@type-binding.return` matches. + * + * 5. **Foreach loop synthesis** — `foreach ($users as $user)` emits + * a `@type-binding.alias` match binding the loop variable to the + * element type of the iterable (resolved from PHPDoc or scopeEnv). + * + * Pure given the input source text. No I/O, no globals consulted. + */ + +import type { Capture, CaptureMatch } from 'gitnexus-shared'; +import { findNodeAtRange, nodeToCapture, syntheticCapture } from '../../utils/ast-helpers.js'; +import { splitNamespaceUseDeclaration } from './import-decomposer.js'; +import { computePhpArityMetadata } from './arity-metadata.js'; +import { synthesizePhpReceiverBinding } from './receiver-binding.js'; +import { getPhpParser, getPhpScopeQuery } from './query.js'; +import { recordCacheHit, recordCacheMiss } from './cache-stats.js'; +import { getTreeSitterBufferSize } from '../../constants.js'; +import { parseSourceSafe } from '../../../tree-sitter/safe-parse.js'; + +type SyntaxNode = ReturnType['parse']>['rootNode']; + +/** Declaration anchors that carry function-like arity metadata. */ +const FUNCTION_DECL_TAGS = ['@declaration.method', '@declaration.function'] as const; + +/** tree-sitter-php node types that the method extractor accepts. */ +const FUNCTION_NODE_TYPES = [ + 'method_declaration', + 'function_definition', + 'anonymous_function', + 'arrow_function', +] as const; + +export function emitPhpScopeCaptures( + sourceText: string, + _filePath: string, + cachedTree?: unknown, +): readonly CaptureMatch[] { + // Skip the parse when the caller already produced a Tree for this source. + // The cachedTree parameter is typed as `unknown` at the LanguageProvider + // contract layer; cast here at the use site. + let tree = cachedTree as ReturnType['parse']> | undefined; + if (tree === undefined) { + tree = parseSourceSafe(getPhpParser(), sourceText, undefined, { + bufferSize: getTreeSitterBufferSize(sourceText), + }); + recordCacheMiss(); + } else { + recordCacheHit(); + } + + const rawMatches = getPhpScopeQuery().matches(tree.rootNode); + const out: CaptureMatch[] = []; + + // Pre-scan: collect anchor node IDs of property_declaration nodes already + // matched by the typed @declaration.property pattern (query.ts ~lines 95–98). + // The untyped @declaration.variable catch-all (query.ts ~lines 101–103) is + // intentionally loose — it has no `type:` constraint, so tree-sitter also + // matches it against typed property declarations and emits a second capture + // for the same property_declaration anchor. Graph-level def-id collision + // currently masks the duplicate at the node-emit layer, but the catch-all + // capture still flows through scope-binding / name-keyed registries with a + // `$`-prefixed name that the typed branch's `$`-strip never normalizes — + // a known vector for receiver-binding lookup pollution. The two patterns + // produce separate rawMatches entries with separate `grouped` maps, so the + // dedup has to be cross-match: build the set here, then skip + // @declaration.variable matches whose anchor is in it (loop below). + const typedPropertyAnchorIds = new Set(); + for (const m of rawMatches) { + for (const c of m.captures) { + if (c.name === 'declaration.property') { + typedPropertyAnchorIds.add(c.node.id); + break; + } + } + } + + for (const m of rawMatches) { + // Group captures by their tag name. Tree-sitter strips the leading + // `@`; we put it back so the central extractor's prefix lookups work. + const grouped: Record = {}; + for (const c of m.captures) { + const tag = '@' + c.name; + grouped[tag] = nodeToCapture(tag, c.node); + } + if (Object.keys(grouped).length === 0) continue; + + // Cross-match dedup for the typed-property double-match described above: + // skip @declaration.variable matches whose anchor was already captured as + // @declaration.property in an earlier match. + if (grouped['@declaration.variable'] !== undefined) { + const varCap = m.captures.find((c) => c.name === 'declaration.variable'); + if (varCap !== undefined && typedPropertyAnchorIds.has(varCap.node.id)) continue; + } + + // Normalize PHP property declarations: strip leading `$` from + // `@declaration.name` for @declaration.property matches. PHP stores + // field names WITHOUT the `$` sigil in the graph so that member access + // lookups like `$user->address` can find the property named `address` + // (not `$address`). `@type-binding.annotation` already strips `$` in + // `interpretPhpTypeBinding`; this mirrors that for the declaration side. + // + // Only applies to `@declaration.property` — typed class properties and + // constructor-promoted parameters. Untyped `@declaration.variable` keeps + // its `$` prefix (those defs are Variable type and not in the field + // registry, so their name doesn't affect member lookup). + if ( + grouped['@declaration.property'] !== undefined && + grouped['@declaration.name'] !== undefined + ) { + const nameCap = grouped['@declaration.name']; + if (nameCap.text.startsWith('$')) { + grouped['@declaration.name'] = { ...nameCap, text: nameCap.text.slice(1) }; + } + } + + // Normalize PHP receiver expressions so the compound-receiver resolver + // can walk chains expressed with `->` (PHP) as if they used `.` (the + // resolver's canonical separator). Without this, `$user->address->save()` + // has receiver text `$user->address` — the resolver sees no `.` separator, + // treats it as a bare identifier, and cannot walk field types. + // + // Transformation applied to `@reference.receiver` captures: + // 1. Replace `->` with `.` ($user->address → $user.address) + // 2. Strip leading `$` from each segment ($user.address → user.address) + // 3. Strip trailing `?` on null-safe receivers ($user? → user) + // + // This is a PHP-local normalization — no shared pipeline code is changed. + if (grouped['@reference.receiver'] !== undefined) { + const recvCap = grouped['@reference.receiver']!; + const normalized = normalizePhpReceiver(recvCap.text); + if (normalized !== recvCap.text) { + grouped['@reference.receiver'] = { ...recvCap, text: normalized }; + } + } + + // Normalize static property write: strip leading `$` from `@reference.name` + // so `User::$count` resolves to property `count` (stored without `$` in graph). + if (grouped['@reference.write.static'] !== undefined) { + const nameCap = grouped['@reference.name']; + if (nameCap !== undefined && nameCap.text.startsWith('$')) { + grouped['@reference.name'] = { + ...nameCap, + text: nameCap.text.slice(1), + }; + } + // Re-tag as @reference.write.member so downstream passes see a uniform write kind. + grouped['@reference.write.member'] = grouped['@reference.write.static']!; + delete grouped['@reference.write.static']; + } + + // Decompose each `namespace_use_declaration` so `interpretPhpImport` + // sees the kind/source/name/alias markers it consumes. + if (grouped['@import.statement'] !== undefined) { + const stmtCapture = grouped['@import.statement']; + const stmtNode = findNodeAtRange( + tree.rootNode, + stmtCapture.range, + 'namespace_use_declaration', + ); + if (stmtNode !== null) { + const decomposed = splitNamespaceUseDeclaration(stmtNode); + if (decomposed.length > 0) { + for (const d of decomposed) out.push(d); + continue; + } + } + // Defensive fallback: emit the raw match. + out.push(grouped); + continue; + } + + // Synthesize `$this` / `parent` receiver type-bindings on every + // non-static method-like. Mirrors C#'s `this` / `base` synthesis. + if (grouped['@scope.function'] !== undefined) { + out.push(grouped); + const anchor = grouped['@scope.function']!; + const fnNode = findFunctionNode(tree.rootNode, anchor.range); + if (fnNode !== null) { + for (const synth of synthesizePhpReceiverBinding(fnNode)) { + out.push(synth); + } + // Synthesize PHPDoc @param and @return type bindings for this fn. + for (const synth of synthesizePhpDocBindings(fnNode)) { + out.push(synth); + } + // Synthesize foreach loop variable bindings inside this fn body. + for (const synth of synthesizeForeachBindings(fnNode)) { + out.push(synth); + } + } + continue; + } + + // Synthesize arity metadata on function-like declarations so the + // registry can narrow overloads. + const declTag = FUNCTION_DECL_TAGS.find((t) => grouped[t] !== undefined); + if (declTag !== undefined) { + const anchor = grouped[declTag]!; + const fnNode = findFunctionNode(tree.rootNode, anchor.range); + if (fnNode !== null) { + const arity = computePhpArityMetadata(fnNode); + if (arity.parameterCount !== undefined) { + grouped['@declaration.parameter-count'] = syntheticCapture( + '@declaration.parameter-count', + fnNode, + String(arity.parameterCount), + ); + } + if (arity.requiredParameterCount !== undefined) { + grouped['@declaration.required-parameter-count'] = syntheticCapture( + '@declaration.required-parameter-count', + fnNode, + String(arity.requiredParameterCount), + ); + } + if (arity.parameterTypes !== undefined) { + grouped['@declaration.parameter-types'] = syntheticCapture( + '@declaration.parameter-types', + fnNode, + JSON.stringify(arity.parameterTypes), + ); + } + } + } + + // Synthesize `@reference.arity` on every call site so the registry's + // arity filter can narrow overloads. Count the `argument` children of + // the backing `arguments` node. Mirrors C#'s pattern (csharp/captures.ts + // lines 149-186). PHP needs this for arity-based dispatch (Cluster H). + const callTag = ( + ['@reference.call.free', '@reference.call.member', '@reference.call.constructor'] as const + ).find((t) => grouped[t] !== undefined); + if (callTag !== undefined && grouped['@reference.arity'] === undefined) { + const anchor = grouped[callTag]!; + const callNode = + findNodeAtRange(tree.rootNode, anchor.range, 'function_call_expression') ?? + findNodeAtRange(tree.rootNode, anchor.range, 'member_call_expression') ?? + findNodeAtRange(tree.rootNode, anchor.range, 'nullsafe_member_call_expression') ?? + findNodeAtRange(tree.rootNode, anchor.range, 'scoped_call_expression') ?? + findNodeAtRange(tree.rootNode, anchor.range, 'object_creation_expression'); + if (callNode !== null) { + const argList = callNode.childForFieldName('arguments'); + const args: SyntaxNode[] = []; + if (argList !== null) { + for (let i = 0; i < argList.namedChildCount; i++) { + const child = argList.namedChild(i); + if (child !== null && child.type === 'argument') args.push(child); + } + } + grouped['@reference.arity'] = syntheticCapture( + '@reference.arity', + callNode, + String(args.length), + ); + // Infer argument types from literal nodes for type-based narrowing. + // Non-literal arguments emit empty string ("unknown" = any-match). + const argTypes = args.map((arg) => inferPhpArgType(arg)); + grouped['@reference.parameter-types'] = syntheticCapture( + '@reference.parameter-types', + callNode, + JSON.stringify(argTypes), + ); + } + } + + out.push(grouped); + } + + return out; +} + +/** Find the first PHP function-like node at the given range. */ +function findFunctionNode(rootNode: SyntaxNode, range: Capture['range']): SyntaxNode | null { + for (const nodeType of FUNCTION_NODE_TYPES) { + const n = findNodeAtRange(rootNode, range, nodeType); + if (n !== null) return n as SyntaxNode; + } + return null; +} + +// ─── PHP receiver normalization ────────────────────────────────────────────── + +/** + * Normalize a PHP receiver expression so the language-agnostic + * compound-receiver resolver (which splits on `.`) can walk field-type chains. + * + * The compound-receiver resolver: + * - splits on `.` to get chain segments + * - looks up the first segment in `typeBindings` (keyed with `$` for variables) + * - walks subsequent segments as field names (stored without `$` in the graph) + * + * Transformation: + * 1. Replace `->` and `?->` with `.` so the resolver's splitter works + * 2. Strip any bare `?` fragment left by null-safe chain ends + * 3. Strip `$` from all segments EXCEPT the first (which is a variable + * and must keep `$` for typeBindings lookup — e.g. `$user → User`) + * + * Examples: + * `$user` → `$user` (bare variable — unchanged) + * `$user->address` → `$user.address` + * `$user->address->city` → `$user.address.city` + * `$user?` → `$user` (null-safe trailing `?` stripped) + * `$this` → `$this` (receiverBinding uses `$this`) + * `parent` → `parent` (super-receiver check) + */ +function normalizePhpReceiver(raw: string): string { + // Keep `$this`, `parent`, and `self` as-is. + if (raw === '$this' || raw === 'parent' || raw === 'self') return raw; + + // Replace `?->` (null-safe) and plain `->` with `.`. + let text = raw.replace(/\?->/g, '.').replace(/->/g, '.'); + // Strip a trailing `?` (null-safe fragment on the last object node). + text = text.replace(/\?$/, ''); + // Collapse any doubled dots from `?->` where `?` was on its own. + text = text.replace(/\.{2,}/g, '.'); + // Strip trailing dot. + text = text.replace(/\.$/, ''); + + // Split on `.` and strip `$` from all segments EXCEPT the first. + // The first segment is a PHP variable (typeBinding key includes `$`). + // Subsequent segments are property/method names (stored without `$`). + const segments = text.split('.'); + for (let i = 1; i < segments.length; i++) { + const s = segments[i]; + if (s !== undefined && s.startsWith('$')) segments[i] = s.slice(1); + } + return segments.join('.'); +} + +// ─── PHP argument type inference ───────────────────────────────────────────── + +/** + * Infer the PHP type of a call argument from its literal shape. + * Returns an empty string for non-literals (treated as "unknown" = any-match). + * Mirrors C#'s `inferArgType` helper. + */ +function inferPhpArgType(argNode: SyntaxNode): string { + // argument node wraps the actual expression + const expr = argNode.firstNamedChild ?? argNode; + switch (expr.type) { + case 'integer': + return 'int'; + case 'float': + return 'float'; + case 'string': + case 'encapsed_string': + case 'heredoc': + case 'nowdoc': + return 'string'; + case 'boolean': + case 'true': + case 'false': + return 'bool'; + case 'null': + return 'null'; + default: + return ''; + } +} + +// ─── PHPDoc synthesis ───────────────────────────────────────────────────────── + +/** PHP 8+ attribute_list nodes that appear between PHPDoc and method. */ +const SKIP_SIBLING_TYPES = new Set(['attribute_list', 'attribute', 'comment']); + +/** Regex for PHPDoc @param: standard `@param Type $name` */ +const PHPDOC_PARAM_RE = /@param\s+(\S+)\s+\$(\w+)/g; +/** Regex for PHPDoc @param: alternate `@param $name Type` */ +const PHPDOC_PARAM_ALT_RE = /@param\s+\$(\w+)\s+(\S+)/g; +/** Regex for PHPDoc @return: `@return Type` */ +const PHPDOC_RETURN_RE = /@return\s+(\S+)/; + +/** + * Normalize a PHP type string to a simple class name for binding purposes. + * Returns null for primitives or uninformative types. + * Mirrors `normalizePhpType` in `interpret.ts` but operates on raw PHPDoc strings. + */ +function normalizePhpDocType(raw: string): string | null { + let type = raw.trim(); + // Strip nullable prefix + if (type.startsWith('?')) type = type.slice(1).trim(); + // Strip array suffix: User[] → User + if (type.endsWith('[]')) type = type.slice(0, -2).trim(); + // Strip union with null/false/void + if (type.includes('|')) { + const parts = type + .split('|') + .map((p) => p.trim()) + .filter((p) => p !== 'null' && p !== 'false' && p !== 'void' && p !== 'mixed' && p !== ''); + if (parts.length !== 1) return null; + type = parts[0]; + } + // Strip intersection: take first part + if (type.includes('&')) { + const first = type.split('&')[0].trim(); + if (first === '') return null; + type = first; + } + // Strip generic wrapper: Collection → User + const genericMatch = type.match(/^\w[\w\\]*\s*<([^,<>]+)>$/); + if (genericMatch) { + type = genericMatch[1].trim(); + // Strip array suffix again inside generic + if (type.endsWith('[]')) type = type.slice(0, -2).trim(); + } + // Strip namespace qualifier: \App\Models\User → User + if (type.includes('\\')) { + const segs = type.split('\\').filter(Boolean); + type = segs[segs.length - 1] ?? type; + } + // Reject primitives + if (PHP_PRIMITIVES.has(type.toLowerCase())) return null; + // Must be a simple identifier + if (!/^\w+$/.test(type)) return null; + return type; +} + +const PHP_PRIMITIVES = new Set([ + 'int', + 'integer', + 'float', + 'double', + 'string', + 'bool', + 'boolean', + 'array', + 'object', + 'callable', + 'iterable', + 'null', + 'void', + 'never', + 'mixed', + 'false', + 'true', + 'self', + 'static', + 'parent', +]); + +/** + * Collect comment text from siblings immediately before `fnNode`. + * Skips PHP 8+ attribute_list nodes. + */ +function collectPrecedingComments(fnNode: SyntaxNode): string { + const texts: string[] = []; + let sibling = fnNode.previousSibling; + while (sibling !== null) { + if (sibling.type === 'comment') { + texts.unshift(sibling.text); + } else if (sibling.isNamed && !SKIP_SIBLING_TYPES.has(sibling.type)) { + break; + } + sibling = sibling.previousSibling; + } + return texts.join('\n'); +} + +/** + * Synthesize PHPDoc @param and @return type-binding captures for a + * method_declaration or function_definition node. + * + * PHPDoc @param Type $name → `@type-binding.parameter` match (anchored at fn body/return_type). + * PHPDoc @return Type → `@type-binding.return` match (anchored at fn name). + */ +function synthesizePhpDocBindings(fnNode: SyntaxNode): CaptureMatch[] { + if (fnNode.type !== 'method_declaration' && fnNode.type !== 'function_definition') return []; + + const commentBlock = collectPrecedingComments(fnNode); + if (commentBlock === '') return []; + + const out: CaptureMatch[] = []; + + // Anchor for parameter type-bindings: the function body (or return_type as fallback). + // The binding must be inside the function scope so it's visible to body statements. + const bodyNode = fnNode.childForFieldName('body'); + const anchorNode = bodyNode ?? fnNode; + + // ── @param annotations ──────────────────────────────────────────────────── + PHPDOC_PARAM_RE.lastIndex = 0; + let m: RegExpExecArray | null; + const seenParams = new Set(); + + while ((m = PHPDOC_PARAM_RE.exec(commentBlock)) !== null) { + const rawType = m[1]; + const paramName = '$' + m[2]; + const typeName = normalizePhpDocType(rawType); + if (typeName === null) continue; + seenParams.add(paramName); + out.push({ + '@type-binding.parameter': nodeToCapture('@type-binding.parameter', anchorNode), + '@type-binding.name': syntheticCapture('@type-binding.name', anchorNode, paramName), + '@type-binding.type': syntheticCapture('@type-binding.type', anchorNode, typeName), + }); + } + + // Also check alternate PHPDoc order: @param $name Type + PHPDOC_PARAM_ALT_RE.lastIndex = 0; + while ((m = PHPDOC_PARAM_ALT_RE.exec(commentBlock)) !== null) { + const paramName = '$' + m[1]; + if (seenParams.has(paramName)) continue; // standard format takes priority + const rawType = m[2]; + const typeName = normalizePhpDocType(rawType); + if (typeName === null) continue; + out.push({ + '@type-binding.parameter': nodeToCapture('@type-binding.parameter', anchorNode), + '@type-binding.name': syntheticCapture('@type-binding.name', anchorNode, paramName), + '@type-binding.type': syntheticCapture('@type-binding.type', anchorNode, typeName), + }); + } + + // ── @return annotation ──────────────────────────────────────────────────── + const returnMatch = PHPDOC_RETURN_RE.exec(commentBlock); + if (returnMatch !== null) { + const rawType = returnMatch[1]; + const typeName = normalizePhpDocType(rawType); + if (typeName !== null) { + // @return bindings must be anchored at the method name and hoisted to Module scope + // by phpBindingScopeFor (which checks for @type-binding.return presence). + // Use the function_definition/method_declaration node itself as the anchor — it + // coincides with the innermost scope's range, so auto-hoist kicks in. + const nameNode = fnNode.childForFieldName('name') ?? fnNode; + out.push({ + '@type-binding.return': nodeToCapture('@type-binding.return', fnNode), + '@type-binding.name': syntheticCapture('@type-binding.name', nameNode, nameNode.text), + '@type-binding.type': syntheticCapture('@type-binding.type', nameNode, typeName), + }); + } + } + + return out; +} + +// ─── Foreach synthesis ─────────────────────────────────────────────────────── + +/** + * Walk all `foreach_statement` nodes inside `fnNode` and synthesize + * `@type-binding.alias` captures binding the loop variable to the + * element type of the iterable. + * + * Supports: + * - `foreach ($users as $user)` — simple iterable variable + * - `foreach ($users as $k => $user)` — key→value pair + * - `foreach ($this->users as $user)` — member access iterable + * - `foreach (getUsers() as $user)` — NOT yet supported (needs return type) + * + * The element type is resolved by: + * 1. Looking up the iterable name in PHPDoc @param bindings already + * collected for this function (passed via typeBindingsByName). + * 2. Direct resolution when iterable's env type IS the element type + * (because PHPDoc normalizes `User[]` → `User` already). + */ +function synthesizeForeachBindings(fnNode: SyntaxNode): CaptureMatch[] { + if ( + fnNode.type !== 'method_declaration' && + fnNode.type !== 'function_definition' && + fnNode.type !== 'anonymous_function' && + fnNode.type !== 'arrow_function' + ) { + return []; + } + + const out: CaptureMatch[] = []; + + // Build a mini type map from the function's PHPDoc @param annotations. + // This is re-parsed here (not cached from synthesizePhpDocBindings) for simplicity; + // the cost is negligible given the small comment sizes. + const commentBlock = collectPrecedingComments(fnNode); + const paramTypeMap = buildParamTypeMap(commentBlock); + + // Walk the function body for foreach_statement nodes. + const bodyNode = fnNode.childForFieldName('body'); + if (bodyNode === null) return []; + collectForeachBindings(bodyNode, fnNode, paramTypeMap, out); + + return out; +} + +/** Build a map of `$paramName → elementTypeName` from PHPDoc @param in a comment block. */ +function buildParamTypeMap(commentBlock: string): Map { + const map = new Map(); + if (commentBlock === '') return map; + + PHPDOC_PARAM_RE.lastIndex = 0; + let m: RegExpExecArray | null; + while ((m = PHPDOC_PARAM_RE.exec(commentBlock)) !== null) { + const rawType = m[1]; + const paramName = '$' + m[2]; + const typeName = normalizePhpDocType(rawType); + if (typeName !== null) map.set(paramName, typeName); + } + PHPDOC_PARAM_ALT_RE.lastIndex = 0; + while ((m = PHPDOC_PARAM_ALT_RE.exec(commentBlock)) !== null) { + const paramName = '$' + m[1]; + if (map.has(paramName)) continue; + const rawType = m[2]; + const typeName = normalizePhpDocType(rawType); + if (typeName !== null) map.set(paramName, typeName); + } + return map; +} + +/** + * Walk a subtree and collect foreach_statement bindings. + * Recursively descends into all child nodes. + */ +function collectForeachBindings( + node: SyntaxNode, + fnNode: SyntaxNode, + paramTypeMap: Map, + out: CaptureMatch[], +): void { + if (node.type === 'foreach_statement') { + const synth = synthesizeSingleForeach(node, fnNode, paramTypeMap); + if (synth !== null) out.push(synth); + } + for (let i = 0; i < node.childCount; i++) { + const child = node.child(i); + if (child !== null) { + collectForeachBindings(child, fnNode, paramTypeMap, out); + } + } +} + +/** + * Synthesize a single `@type-binding.alias` match for a `foreach_statement`. + * + * AST structure for foreach_statement (tree-sitter-php): + * foreach ( as ) + * Named children (excluding body): first = iterable, second = value or pair. + */ +function synthesizeSingleForeach( + foreachNode: SyntaxNode, + fnNode: SyntaxNode, + paramTypeMap: Map, +): CaptureMatch | null { + // Collect non-body named children: [iterable, value_or_pair] + const bodyNode = foreachNode.childForFieldName('body'); + const children: SyntaxNode[] = []; + for (let i = 0; i < foreachNode.namedChildCount; i++) { + const child = foreachNode.namedChild(i); + if (child !== null && child !== bodyNode) children.push(child); + } + if (children.length < 2) return null; + + const iterableNode = children[0]; + const valueOrPair = children[1]; + + // Determine the loop variable node + let loopVarNode: SyntaxNode; + if (valueOrPair.type === 'pair') { + // $key => $value — use the last named child of the pair + const lastChild = valueOrPair.namedChild(valueOrPair.namedChildCount - 1); + if (lastChild === null) return null; + loopVarNode = + lastChild.type === 'by_ref' ? (lastChild.firstNamedChild ?? lastChild) : lastChild; + } else { + loopVarNode = + valueOrPair.type === 'by_ref' ? (valueOrPair.firstNamedChild ?? valueOrPair) : valueOrPair; + } + + // Loop variable must be a variable_name + if (loopVarNode.type !== 'variable_name') return null; + const loopVarName = loopVarNode.text; // e.g. '$user' + + // Resolve the element type from the iterable + let elementType: string | null = null; + + if (iterableNode.type === 'variable_name') { + // foreach ($users as $user) — look up $users in param map + const iterableName = iterableNode.text; // e.g. '$users' + elementType = paramTypeMap.get(iterableName) ?? null; + } else if (iterableNode.type === 'member_access_expression') { + // foreach ($this->users as $user) — property name is the field + const propNameNode = iterableNode.childForFieldName('name'); + if (propNameNode !== null) { + // Property stored with $ prefix in paramTypeMap (rare for $this->prop patterns) + // Try both with and without $ prefix + const propKey = '$' + propNameNode.text; + elementType = paramTypeMap.get(propKey) ?? null; + if (elementType === null) { + // Try to find the property type from the enclosing class + elementType = findClassPropertyElementType(iterableNode, fnNode); + } + } + } else if (iterableNode.type === 'function_call_expression') { + // foreach (getUsers() as $user) — use the function name as a type alias. + // The function's @return annotation produces a @type-binding.return binding + // in the Module scope (e.g. getUsers → User). The scope-extractor's + // followChainedRef will resolve $user → getUsers → User. + const funcNode = iterableNode.childForFieldName('function'); + if (funcNode !== null && funcNode.type === 'name') { + elementType = funcNode.text; // e.g. 'getUsers' — chain will be resolved later + } + } else if (iterableNode.type === 'member_call_expression') { + // foreach ($this->getUsers() as $user) — use the method name as a type alias. + const methodNameNode = iterableNode.childForFieldName('name'); + if (methodNameNode !== null) { + elementType = methodNameNode.text; // e.g. 'getUsers' + } + } + + if (elementType === null) return null; + + // Anchor the binding inside the foreach body so it's scoped to the loop. + const anchorNode = bodyNode ?? foreachNode; + + return { + '@type-binding.alias': nodeToCapture('@type-binding.alias', anchorNode), + '@type-binding.name': syntheticCapture('@type-binding.name', anchorNode, loopVarName), + '@type-binding.type': syntheticCapture('@type-binding.type', anchorNode, elementType), + }; +} + +/** + * Try to find the element type for `$this->property` member access by walking + * up from the foreach to the enclosing class and scanning the property declaration. + */ +function findClassPropertyElementType( + memberAccessNode: SyntaxNode, + fnNode: SyntaxNode, +): string | null { + const propNameNode = memberAccessNode.childForFieldName('name'); + if (propNameNode === null) return null; + const propName = propNameNode.text; + + // Walk up from fnNode to find the enclosing class declaration + let cur: SyntaxNode | null = fnNode.parent; + while (cur !== null) { + if (cur.type === 'class_declaration' || cur.type === 'trait_declaration') { + break; + } + cur = cur.parent; + } + if (cur === null) return null; + + // Find the property_declaration with matching variable_name '$propName' + const declList = cur.childForFieldName('body'); + if (declList === null) return null; + + for (let i = 0; i < declList.namedChildCount; i++) { + const child = declList.namedChild(i); + if (child === null || child.type !== 'property_declaration') continue; + for (let j = 0; j < child.namedChildCount; j++) { + const elem = child.namedChild(j); + if (elem === null || elem.type !== 'property_element') continue; + const varNameNode = elem.firstNamedChild; + if (varNameNode === null || varNameNode.text !== '$' + propName) continue; + // Found the property — get its element type from @var PHPDoc or native type + return extractPropertyElementType(child); + } + } + return null; +} + +/** Regex for PHPDoc @var: `@var Type` */ +const PHPDOC_VAR_RE = /@var\s+(\S+)/; + +/** + * Extract element type from a property_declaration node: + * 1. PHPDoc @var annotation on a preceding comment sibling + * 2. PHP 7.4+ native type field (non-array) + */ +function extractPropertyElementType(propDecl: SyntaxNode): string | null { + // Strategy 1: PHPDoc @var on a preceding comment sibling + let sibling = propDecl.previousSibling; + while (sibling !== null) { + if (sibling.type === 'comment') { + const m = PHPDOC_VAR_RE.exec(sibling.text); + if (m !== null) return normalizePhpDocType(m[1]); + } else if (sibling.isNamed && !SKIP_SIBLING_TYPES.has(sibling.type)) { + break; + } + sibling = sibling.previousSibling; + } + // Strategy 2: native type field — skip generic 'array' + const typeNode = propDecl.childForFieldName('type'); + if (typeNode === null) return null; + const typeName = typeNode.text.trim(); + if (typeName === 'array' || typeName === '') return null; + return normalizePhpDocType(typeName); +} diff --git a/gitnexus/src/core/ingestion/languages/php/import-decomposer.ts b/gitnexus/src/core/ingestion/languages/php/import-decomposer.ts new file mode 100644 index 000000000..f17456805 --- /dev/null +++ b/gitnexus/src/core/ingestion/languages/php/import-decomposer.ts @@ -0,0 +1,304 @@ +/** + * Decompose a PHP `namespace_use_declaration` into one or more + * `CaptureMatch` objects carrying the synthesized markers + * `@import.kind` / `@import.source` / `@import.name` / `@import.alias` + * that `interpretPhpImport` consumes. + * + * PHP import forms handled: + * + * use Foo\Bar; → namespace, localName=Bar + * use Foo\Bar as Baz; → alias, localName=Baz + * use function Foo\bar; → function, localName=bar + * use const Foo\BAR; → const, localName=BAR + * use Foo\{A, B as C}; → grouped: one match per clause + * use function Foo\{f, g as h}; → grouped function variants + * use const Foo\{X, Y as Z}; → grouped const variants + * + * Unlike C#'s decomposer this is 1:N — each grouped use_declaration + * fans out to one CaptureMatch per inner clause. + */ + +import type { Capture, CaptureMatch } from 'gitnexus-shared'; +import { nodeToCapture, syntheticCapture, type SyntaxNode } from '../../utils/ast-helpers.js'; + +export type PhpImportKind = 'namespace' | 'alias' | 'function' | 'const'; + +interface PhpImportSpec { + readonly kind: PhpImportKind; + /** Full backslash-separated path (backslashes intact): `Foo\Bar\Baz`. */ + readonly source: string; + /** Local binding name — last source segment for plain imports, the + * alias identifier for aliased imports. */ + readonly name: string; + /** Present iff kind === 'alias'. */ + readonly alias?: string; + /** Anchor node for synthesized captures (range-wise). */ + readonly atNode: SyntaxNode; +} + +/** + * Decompose a `namespace_use_declaration` node into one `CaptureMatch` + * per logical import. Returns `[]` when the node is unrecognized or + * carries no resolvable clauses. + */ +export function splitNamespaceUseDeclaration(stmtNode: SyntaxNode): CaptureMatch[] { + if (stmtNode.type !== 'namespace_use_declaration') return []; + + // Detect qualifier keyword: `use function` / `use const` + // tree-sitter-php uses a `use_type` or `function`/`const` keyword + // child to distinguish them. We scan the raw text before the first + // backslash-path child. + const qualifier = detectQualifier(stmtNode); + + // Grouped use: `use Foo\{A, B as C}` — find namespace_use_group child. + const groupNode = findNamedChild(stmtNode, 'namespace_use_group'); + if (groupNode !== null) { + return decomposeGrouped(stmtNode, groupNode, qualifier); + } + + // Single use clause (possibly aliased). + const spec = parseSingleUseClause(stmtNode, qualifier); + if (spec === null) return []; + return [buildImportMatch(stmtNode, spec)]; +} + +// ── Qualifier detection ──────────────────────────────────────────────────── + +/** + * Return the qualifier keyword appearing after `use`: + * `'function'`, `'const'`, or `null` for plain namespace use. + * + * tree-sitter-php emits the qualifier as a `name` node with text + * "function" or "const" (not a keyword token in recent grammars), + * or as a dedicated `use_type` node. We inspect the node's raw text + * to be grammar-version-agnostic. + */ +function detectQualifier(node: SyntaxNode): PhpImportKind { + const raw = node.text; + // Match `use function` or `use const` at the start (after optional whitespace) + if (/^\s*use\s+function\s/i.test(raw)) return 'function'; + if (/^\s*use\s+const\s/i.test(raw)) return 'const'; + return 'namespace'; +} + +// ── Single clause parsing ────────────────────────────────────────────────── + +function parseSingleUseClause(node: SyntaxNode, qualifier: PhpImportKind): PhpImportSpec | null { + // A plain `namespace_use_declaration` has one or more + // `namespace_use_clause` named children (each clause is one import, + // comma-separated for multiple). For the single case there is one. + const clause = findNamedChild(node, 'namespace_use_clause'); + if (clause !== null) return parseUseClause(clause, qualifier); + + // Older grammar versions may put the qualified_name directly under + // the declaration node. Check for a qualified_name or name child. + const qualName = findNamedChild(node, 'qualified_name') ?? findNamedChild(node, 'name'); + if (qualName === null) return null; + const source = qualName.text.trim(); + if (source === '') return null; + return { + kind: qualifier, + source, + name: lastSegment(source), + atNode: node, + }; +} + +function parseUseClause(clause: SyntaxNode, qualifier: PhpImportKind): PhpImportSpec | null { + // namespace_use_clause: + // qualified_name (or name) + // optional: alias_clause → "as" name (some grammar versions) + // optional: bare name node (tree-sitter-php ≥ 0.22 emits the + // alias as a sibling `name` node + // directly, not inside alias_clause) + const qualName = findNamedChild(clause, 'qualified_name') ?? findNamedChild(clause, 'name'); + if (qualName === null) return null; + const source = qualName.text.trim(); + if (source === '') return null; + + // Strategy 1: explicit alias_clause wrapper (older grammar versions). + const aliasClause = findNamedChild(clause, 'alias_clause'); + if (aliasClause !== null) { + // alias_clause: "as" name + const aliasName = findNamedChild(aliasClause, 'name') ?? aliasClause.firstNamedChild; + const alias = aliasName?.text.trim() ?? ''; + if (alias === '') return null; + return { + kind: 'alias', + source, + name: alias, + alias, + atNode: clause, + }; + } + + // Strategy 2: bare sibling `name` node after the qualified_name. + // tree-sitter-php (≥ 0.22) emits `use Foo\Bar as Baz` as: + // namespace_use_clause + // qualified_name "Foo\Bar" + // name "Baz" ← alias, no alias_clause wrapper + // Detect by: clause has ≥2 named children AND the last named child is + // a `name` node that differs from the qualName node. + if (clause.namedChildCount >= 2) { + const lastChild = clause.namedChild(clause.namedChildCount - 1); + if (lastChild !== null && lastChild !== qualName && lastChild.type === 'name') { + const alias = lastChild.text.trim(); + if (alias !== '') { + return { + kind: 'alias', + source, + name: alias, + alias, + atNode: clause, + }; + } + } + } + + return { + kind: qualifier, + source, + name: lastSegment(source), + atNode: clause, + }; +} + +// ── Grouped use decomposition ────────────────────────────────────────────── + +/** + * Decompose `use Foo\Bar\{A, B as C, function f, const X}` into one + * `CaptureMatch` per inner clause. + * + * The leading prefix (`Foo\Bar`) is prepended to each inner path. + * Inner clauses can override the qualifier with their own `function` / + * `const` keyword inside the group. + */ +function decomposeGrouped( + stmtNode: SyntaxNode, + groupNode: SyntaxNode, + outerQualifier: PhpImportKind, +): CaptureMatch[] { + // The prefix is the qualified_name that precedes the `{...}` group. + const prefixNode = findNamedChild(stmtNode, 'qualified_name') ?? findNamedChild(stmtNode, 'name'); + const prefix = prefixNode?.text.trim() ?? ''; + + const out: CaptureMatch[] = []; + + for (let i = 0; i < groupNode.namedChildCount; i++) { + const child = groupNode.namedChild(i); + if (child === null) continue; + + // Each child in a group may be: + // namespace_use_clause — plain or aliased + // namespace_use_type — `function` or `const` qualifier inside group + // We detect an inline qualifier by checking the raw text of the clause. + if (child.type !== 'namespace_use_clause') continue; + + const innerQualifier = detectInnerQualifier(child) ?? outerQualifier; + const spec = parseInnerClause(child, prefix, innerQualifier); + if (spec !== null) { + out.push(buildImportMatch(stmtNode, spec)); + } + } + + return out; +} + +/** + * Detect an inline qualifier keyword inside a grouped clause. + * e.g. `use Foo\{function bar, const BAZ}` — each clause may start with + * `function` or `const`. + */ +function detectInnerQualifier(clause: SyntaxNode): PhpImportKind | null { + const raw = clause.text.trim(); + if (/^function\s/i.test(raw)) return 'function'; + if (/^const\s/i.test(raw)) return 'const'; + return null; +} + +function parseInnerClause( + clause: SyntaxNode, + prefix: string, + qualifier: PhpImportKind, +): PhpImportSpec | null { + const qualName = findNamedChild(clause, 'qualified_name') ?? findNamedChild(clause, 'name'); + if (qualName === null) return null; + + // Strip inline `function` / `const` text prefix if present in the text. + let innerPath = qualName.text.trim(); + innerPath = innerPath.replace(/^(?:function|const)\s+/i, '').trim(); + if (innerPath === '') return null; + + const source = prefix !== '' ? `${prefix}\\${innerPath}` : innerPath; + + // Strategy 1: explicit alias_clause wrapper (older grammar versions). + const aliasClause = findNamedChild(clause, 'alias_clause'); + if (aliasClause !== null) { + const aliasName = findNamedChild(aliasClause, 'name') ?? aliasClause.firstNamedChild; + const alias = aliasName?.text.trim() ?? ''; + if (alias === '') return null; + return { + kind: 'alias', + source, + name: alias, + alias, + atNode: clause, + }; + } + + // Strategy 2: bare sibling `name` node after the qualified_name (tree-sitter-php ≥ 0.22). + if (clause.namedChildCount >= 2) { + const lastChild = clause.namedChild(clause.namedChildCount - 1); + if (lastChild !== null && lastChild !== qualName && lastChild.type === 'name') { + const alias = lastChild.text.trim(); + if (alias !== '') { + return { + kind: 'alias', + source, + name: alias, + alias, + atNode: clause, + }; + } + } + } + + return { + kind: qualifier, + source, + name: lastSegment(innerPath), + atNode: clause, + }; +} + +// ── CaptureMatch builder ─────────────────────────────────────────────────── + +function buildImportMatch(stmtNode: SyntaxNode, spec: PhpImportSpec): CaptureMatch { + const m: Record = { + '@import.statement': nodeToCapture('@import.statement', stmtNode), + '@import.kind': syntheticCapture('@import.kind', spec.atNode, spec.kind), + '@import.source': syntheticCapture('@import.source', spec.atNode, spec.source), + '@import.name': syntheticCapture('@import.name', spec.atNode, spec.name), + }; + if (spec.alias !== undefined) { + m['@import.alias'] = syntheticCapture('@import.alias', spec.atNode, spec.alias); + } + return m; +} + +// ── Helpers ──────────────────────────────────────────────────────────────── + +/** Last backslash-separated segment: `Foo\Bar\Baz` → `Baz`. */ +function lastSegment(path: string): string { + const parts = path.split('\\').filter(Boolean); + return parts[parts.length - 1] ?? path; +} + +/** Find the first named child with a given node type. */ +function findNamedChild(node: SyntaxNode, type: string): SyntaxNode | null { + for (let i = 0; i < node.namedChildCount; i++) { + const child = node.namedChild(i); + if (child !== null && child.type === type) return child; + } + return null; +} diff --git a/gitnexus/src/core/ingestion/languages/php/import-target.ts b/gitnexus/src/core/ingestion/languages/php/import-target.ts new file mode 100644 index 000000000..ebf7938b3 --- /dev/null +++ b/gitnexus/src/core/ingestion/languages/php/import-target.ts @@ -0,0 +1,140 @@ +/** + * Adapter from `(ParsedImport, WorkspaceIndex)` → concrete file path. + * + * Delegates to the existing `resolvePhpImportInternal` (PSR-4 via + * composer.json + suffix matching fallback). The `WorkspaceIndex` is + * opaque at this layer; consumers wire a `PhpResolveContext` shape + * carrying `fromFile` + `allFilePaths`. + * + * `loadPhpComposerConfig` is the `ScopeResolver.loadResolutionConfig` + * implementation — it loads `composer.json` once per workspace pass and + * threads the parsed config into every subsequent `resolveImportTarget` + * call via the opaque `resolutionConfig` parameter. + * + * Returning `null` lets the finalize algorithm mark the edge as + * `linkStatus: 'unresolved'`. + */ + +import type { ParsedImport, WorkspaceIndex } from 'gitnexus-shared'; +import { resolvePhpImportInternal } from '../../import-resolvers/php.js'; +import type { ComposerConfig } from '../../language-config.js'; +import { readFileSync } from 'node:fs'; +import { join } from 'node:path'; + +export interface PhpResolveContext { + readonly fromFile: string; + readonly allFilePaths: ReadonlySet; +} + +// ─── loadResolutionConfig ────────────────────────────────────────────────── + +/** + * Load and parse `composer.json` from the repo root. Returns a + * `ComposerConfig` object (PSR-4 namespace → directory mappings) or + * `null` when no `composer.json` is present or it cannot be parsed. + * + * The result is threaded into each `resolvePhpImportInternal` call as + * the `composerConfig` argument. + */ +export function loadPhpComposerConfig(repoPath: string): ComposerConfig | null { + try { + const composerPath = join(repoPath, 'composer.json'); + const raw = readFileSync(composerPath, 'utf8'); + const parsed = JSON.parse(raw) as unknown; + if (typeof parsed !== 'object' || parsed === null) return null; + + const composer = parsed as Record; + const autoload = composer['autoload'] as Record | undefined; + if (autoload === undefined) return null; + + const psr4Raw = (autoload['psr-4'] ?? {}) as Record; + const psr4 = new Map(); + + for (const [ns, dirs] of Object.entries(psr4Raw)) { + // namespace prefix ends with `\` — keep as-is; resolver strips it + const normalizedNs = ns.replace(/\\$/, ''); + const dir = Array.isArray(dirs) ? dirs[0] : dirs; + if (typeof dir === 'string') { + // Normalize directory path (strip trailing slash) + const normalizedDir = dir.replace(/\/+$/, ''); + psr4.set(normalizedNs, normalizedDir); + } + } + + return { psr4 }; + } catch { + return null; + } +} + +// ─── resolvePhpImportTarget ──────────────────────────────────────────────── + +/** + * LanguageProvider-shaped adapter: `(ParsedImport, WorkspaceIndex) → string | null`. + * + * The `WorkspaceIndex` is `unknown` in the shared contract. The scope-resolution + * orchestrator hands us a `PhpResolveContext`-shaped object; narrow structurally + * rather than via a cast chain so unexpected shapes return `null` cleanly. + */ +export function resolvePhpImportTarget( + parsedImport: ParsedImport, + workspaceIndex: WorkspaceIndex, +): string | null { + const ctx = workspaceIndex as PhpResolveContext | undefined; + if ( + ctx === undefined || + typeof (ctx as { fromFile?: unknown }).fromFile !== 'string' || + !((ctx as { allFilePaths?: unknown }).allFilePaths instanceof Set) + ) { + return null; + } + if (parsedImport.kind === 'dynamic-unresolved') return null; + if (parsedImport.targetRaw === null || parsedImport.targetRaw === '') return null; + + const allFiles = ctx.allFilePaths as Set; + const normalizedFileList = [...allFiles].map((f) => f.replace(/\\/g, '/')); + const allFileList = [...allFiles]; + + return resolvePhpImportInternal( + parsedImport.targetRaw, + null, // composerConfig not available through LanguageProvider path + allFiles, + normalizedFileList, + allFileList, + undefined, + ); +} + +/** + * ScopeResolver-shaped adapter: `(targetRaw, fromFile, allFilePaths, resolutionConfig?) → string | null`. + * + * Used inside `scope-resolver.ts`. Accepts the optional `resolutionConfig` + * (a `ComposerConfig | null` loaded once per workspace by + * `loadPhpComposerConfig`) and threads it into `resolvePhpImportInternal`. + */ +export function resolvePhpImportTargetInternal( + targetRaw: string, + _fromFile: string, + allFilePaths: ReadonlySet, + resolutionConfig?: unknown, +): string | null { + if (targetRaw === '') return null; + + const composerConfig = + resolutionConfig !== undefined && resolutionConfig !== null + ? (resolutionConfig as ComposerConfig) + : null; + + const allFiles = allFilePaths as Set; + const normalizedFileList = [...allFiles].map((f) => f.replace(/\\/g, '/')); + const allFileList = [...allFiles]; + + return resolvePhpImportInternal( + targetRaw, + composerConfig, + allFiles, + normalizedFileList, + allFileList, + undefined, + ); +} diff --git a/gitnexus/src/core/ingestion/languages/php/index.ts b/gitnexus/src/core/ingestion/languages/php/index.ts new file mode 100644 index 000000000..9b549bc3d --- /dev/null +++ b/gitnexus/src/core/ingestion/languages/php/index.ts @@ -0,0 +1,73 @@ +/** + * PHP scope-resolution hooks (RFC #909 Ring 3 LANG-php, #938). + * + * Public API barrel. Consumers should import from this file rather than + * the individual modules. + * + * Module layout (each file is a single concern): + * + * - `query.ts` — tree-sitter query + lazy parser/query singletons + * - `captures.ts` — `emitPhpScopeCaptures` orchestrator + * - `import-decomposer.ts` — each `namespace_use_declaration` → ParsedImport captures + * - `interpret.ts` — capture-match → `ParsedImport` / `ParsedTypeBinding` + * - `simple-hooks.ts` — small/no-op hooks made explicit + * - `receiver-binding.ts` — synthesize `$this` / `parent` type-bindings on + * instance-method entry + * - `merge-bindings.ts` — PHP `use` precedence (local > import > wildcard) + * - `arity.ts` — PHP arity compatibility (variadic, defaults) + * - `arity-metadata.ts` — synthesize arity metadata from declarations + * - `import-target.ts` — `(ParsedImport, WorkspaceIndex) → file path` adapter + * wrapping `resolvePhpImportInternal` (PSR-4 + composer.json) + * - `scope-resolver.ts` — `ScopeResolver` registered in `SCOPE_RESOLVERS` + * - `cache-stats.ts` — PROF_SCOPE_RESOLUTION cache hit/miss counters + * + * ## Known limitations + * + * The PHP registry-primary path intentionally does NOT resolve the following. + * Each is a conscious trade-off at migration time. + * + * 1. **Trait `$this` → using-class binding** — for methods defined in a + * trait, `$this` is synthesized as a binding to the trait itself. + * Resolving `$this` to the actual using-class type requires cross-file + * analysis of all `use TraitName;` declarations in class bodies. + * Deferred to a follow-up; trait method resolution falls back to the + * trait scope. + * + * 2. **Anonymous classes** — `new class extends Foo { }` have no stable + * class name and are skipped by receiver-binding synthesis. The class + * body is still scoped; member lookups inside it will fall back to + * free-call resolution. + * + * 3. **Dynamic property/method access** — `$obj->{$name}()` and + * `$$varName` are not followed. The dynamic receiver is ignored and + * the call falls through to the shared free-call resolver. + * + * 4. **Magic methods** — `__get`, `__set`, `__call`, `__callStatic` are + * not modeled as virtual dispatch; they appear as regular method + * declarations in the graph but calls that would route through them + * at runtime are not distinguished. + * + * 5. **Laravel facade magic** — `App::make(...)`, `Cache::get(...)` etc. + * resolve statically to the Facade class rather than the underlying + * bound implementation. Deferred to a Laravel-specific plugin. + * + * 6. **Intersection types in parameters** — `T&U $param` takes the first + * named part (`T`). This matches the legacy type-extractor's behavior. + * + * Shadow-harness corpus parity is the authoritative signal for which of + * these matter in practice. The CI parity gate blocks any PR that regresses + * either the legacy or registry-primary run of + * `test/integration/resolvers/php.test.ts`. + */ + +export { emitPhpScopeCaptures } from './captures.js'; +export { getPhpCaptureCacheStats, resetPhpCaptureCacheStats } from './cache-stats.js'; +export { interpretPhpImport, interpretPhpTypeBinding } from './interpret.js'; +export { phpMergeBindings } from './merge-bindings.js'; +export { phpArityCompatibility } from './arity.js'; +export { resolvePhpImportTarget, type PhpResolveContext } from './import-target.js'; +export { phpBindingScopeFor, phpImportOwningScope, phpReceiverBinding } from './simple-hooks.js'; +// NOTE: phpScopeResolver is intentionally NOT re-exported from this barrel. +// Importing it here would create a circular dependency: +// php.ts → php/index.js → php/scope-resolver.js → ../php.js +// Registry and other consumers must import directly from './php/scope-resolver.js'. diff --git a/gitnexus/src/core/ingestion/languages/php/interpret.ts b/gitnexus/src/core/ingestion/languages/php/interpret.ts new file mode 100644 index 000000000..8aa07a736 --- /dev/null +++ b/gitnexus/src/core/ingestion/languages/php/interpret.ts @@ -0,0 +1,250 @@ +/** + * Capture-match → semantic-shape interpreters for PHP. + * + * - `interpretPhpImport` → `ParsedImport` + * - `interpretPhpTypeBinding` → `ParsedTypeBinding` + * + * Import matches arrive pre-decomposed by `emitPhpScopeCaptures` (one + * CaptureMatch per logical import, with synthesized `@import.kind / + * source / name / alias` markers). Type-binding matches arrive from + * the raw query captures — each `@type-binding.*` anchor carries + * `@type-binding.name` + `@type-binding.type`. + */ + +import type { CaptureMatch, ParsedImport, ParsedTypeBinding, TypeRef } from 'gitnexus-shared'; + +// ─── interpretImport ────────────────────────────────────────────────────── + +export function interpretPhpImport(captures: CaptureMatch): ParsedImport | null { + const kindCap = captures['@import.kind']; + const sourceCap = captures['@import.source']; + const nameCap = captures['@import.name']; + const aliasCap = captures['@import.alias']; + + const kind = kindCap?.text; + if (kind === undefined || sourceCap === undefined) return null; + + const source = sourceCap.text.trim(); + if (source === '') return null; + + switch (kind) { + case 'namespace': { + // `use Foo\Bar;` — PHP `use` is a NAMED import (binds the class + // `Bar`, not the namespace `Foo`). This differs from C# `using`, + // which is a true namespace import. Producing 'named' here makes + // `new Bar()` resolve to the imported class def. + const localName = nameCap?.text.trim() ?? lastSegment(source); + return { + kind: 'named', + localName, + importedName: localName, + targetRaw: source, + }; + } + case 'alias': { + // `use Foo\Bar as Baz;` + if (aliasCap === undefined) return null; + const alias = aliasCap.text.trim(); + if (alias === '') return null; + const importedName = lastSegment(source); + return { + kind: 'alias', + localName: alias, + importedName, + alias, + targetRaw: source, + }; + } + case 'function': { + // `use function Foo\bar;` — treat as named import; importedName is + // the function name (last segment). targetRaw is the full path. + const localName = nameCap?.text.trim() ?? lastSegment(source); + return { + kind: 'named', + localName, + importedName: localName, + targetRaw: source, + }; + } + case 'const': { + // `use const Foo\BAR;` — same shape as function. + const localName = nameCap?.text.trim() ?? lastSegment(source); + return { + kind: 'named', + localName, + importedName: localName, + targetRaw: source, + }; + } + default: + return null; + } +} + +// ─── interpretTypeBinding ───────────────────────────────────────────────── + +export function interpretPhpTypeBinding(captures: CaptureMatch): ParsedTypeBinding | null { + const nameCap = captures['@type-binding.name']; + const typeCap = captures['@type-binding.type']; + if (nameCap === undefined || typeCap === undefined) return null; + + // Determine source from anchor captures. Order: most-specific first. + let source: TypeRef['source'] = 'parameter-annotation'; + if (captures['@type-binding.self'] !== undefined) source = 'self'; + else if (captures['@type-binding.constructor'] !== undefined) source = 'constructor-inferred'; + else if (captures['@type-binding.annotation'] !== undefined) source = 'annotation'; + else if (captures['@type-binding.alias'] !== undefined) source = 'assignment-inferred'; + else if (captures['@type-binding.return'] !== undefined) source = 'return-annotation'; + + let rawType: string | null; + + if (source === 'assignment-inferred') { + // `@type-binding.alias` captures cover several assignment RHS shapes: + // - `$alias = $u` → rawType = '$u' (variable alias) + // - `$u = getUser()` → rawType = 'getUser' (callable alias) + // - `$u = new User()` → rawType = 'User' (constructor — via @type-binding.constructor; handled below) + // - `$role = UserRole::Viewer` → rawType = 'UserRole' (enum/class constant) + // + // For variable aliases (`$u`), `normalizePhpType` returns null because + // `$` is not a word character. We must preserve the raw `$`-prefixed name + // so `followChainedRef` can walk the chain `$alias → $u → User`. + // For callable/class names, `normalizePhpType` strips qualifiers correctly. + const rawText = typeCap.text.trim(); + if (rawText.startsWith('$')) { + // Variable alias: keep as-is for chain-following. + rawType = rawText; + } else { + rawType = normalizePhpType(rawText); + } + } else { + // All other sources: strip PHP type decoration to get the simple class name: + // ?User → User (nullable prefix) + // User|null → User (union with null/false/void) + // User&Loggable → User (intersection — take first meaningful) + // Collection → User (PHPDoc generic wrapper) + // User[] → User (array suffix) + // \App\Models\User → User (backslash qualifier) + rawType = normalizePhpType(typeCap.text.trim()); + } + + if (rawType === null) return null; + + // PHP variable names include the `$` sigil (e.g. `$user`). Most + // bindings keep it because they are looked up via the variable + // (`$user->method()` finds binding `$user`). Property field bindings + // are different: `$user->address` looks up `address` (no sigil) on + // the User class. Property declarations carry source `'annotation'`, + // so we strip the leading `$` for that source only. + let boundName = nameCap.text.trim(); + if (source === 'annotation' && boundName.startsWith('$')) { + boundName = boundName.slice(1); + } + + return { boundName, rawTypeName: rawType, source }; +} + +// ─── Type normalization ─────────────────────────────────────────────────── + +/** + * Normalize a PHP type string to a simple class identifier, or `null` + * when the type is uninformative (primitive, void, mixed, self, etc.). + * + * Rules applied in order: + * 1. Strip nullable prefix `?` + * 2. Split on `|` (union) — keep only if exactly one non-null part + * 3. Take first part of `&` intersection + * 4. Strip array suffix `[]` + * 5. Strip generic wrapper `Collection` → `User` + * 6. Canonicalize leading backslash off: `\App\Models\User` → `App\Models\User` + * 7. Reject PHP primitive / pseudo types + * + * The qualified form is preserved on `TypeRef.rawName` so downstream PHP + * receiver resolution can distinguish `\App\Other\User` from a same-simple-name + * `User` reachable via `use`. Without this, fully-qualified type hints collapse + * to ambiguous simple names and resolve against the caller's scope chain + * instead of the explicit target the source named (Codex PR #1497 review, + * finding 1). + */ +export function normalizePhpType(raw: string): string | null { + // 1. Strip nullable prefix + let type = raw.startsWith('?') ? raw.slice(1).trim() : raw; + + // 2. Union type — keep only if one non-null/false/void part remains + if (type.includes('|')) { + const parts = type + .split('|') + .map((p) => p.trim()) + .filter((p) => p !== 'null' && p !== 'false' && p !== 'void' && p !== 'mixed' && p !== ''); + if (parts.length !== 1) return null; + type = parts[0]; + } + + // 3. Intersection type — take the first part + if (type.includes('&')) { + const first = type.split('&')[0].trim(); + if (first === '') return null; + type = first; + } + + // 4. Strip array suffix + if (type.endsWith('[]')) type = type.slice(0, -2).trim(); + + // 5. Strip single-arg generic wrapper: Collection → User + // Qualified inner types (Collection<\App\Models\User>) survive — the + // capture group preserves whatever the writer named. + const genericMatch = type.match(/^\w[\w\\]*\s*<([^,<>]+)>$/); + if (genericMatch) { + type = genericMatch[1].trim(); + } + + // 6. Canonicalize leading backslash off — keep the qualified path intact. + // `\App\Models\User` → `App\Models\User`. `App\Models\User` → unchanged. + // Unqualified `User` stays as `User`. The qualified form is the lookup + // key into the workspace QualifiedNameIndex (PHP defs are indexed by + // namespace-joined qualifiedName); the leading-backslash distinction in + // source is only an "absolute path" anchor, not part of the canonical key. + if (type.startsWith('\\')) type = type.replace(/^\\+/, ''); + + // 7. Reject primitives / pseudo-types + if (isPrimitiveOrPseudo(type)) return null; + + // Must be a (possibly qualified) PHP identifier — segments of word chars + // separated by single backslashes. Empty segments (consecutive backslashes, + // trailing backslash) are rejected. + if (!/^\w+(?:\\\w+)*$/.test(type)) return null; + + return type; +} + +const PHP_PRIMITIVE_TYPES = new Set([ + 'int', + 'integer', + 'float', + 'double', + 'string', + 'bool', + 'boolean', + 'array', + 'object', + 'callable', + 'iterable', + 'null', + 'void', + 'never', + 'mixed', + 'false', + 'true', + 'self', + 'static', + 'parent', +]); + +function isPrimitiveOrPseudo(type: string): boolean { + return PHP_PRIMITIVE_TYPES.has(type.toLowerCase()); +} + +/** Last backslash-separated segment: `Foo\Bar\Baz` → `Baz`. */ +function lastSegment(path: string): string { + const parts = path.split('\\').filter(Boolean); + return parts[parts.length - 1] ?? path; +} diff --git a/gitnexus/src/core/ingestion/languages/php/merge-bindings.ts b/gitnexus/src/core/ingestion/languages/php/merge-bindings.ts new file mode 100644 index 000000000..257496550 --- /dev/null +++ b/gitnexus/src/core/ingestion/languages/php/merge-bindings.ts @@ -0,0 +1,51 @@ +/** + * PHP shadowing precedence for the `mergeBindings` hook. + * + * Tier ranking (lower wins in shadowing): + * + * - 0: `local` — a class member, method, local variable, or parameter + * declared in this scope. + * - 1: `import` / `namespace` / `reexport` — `use Foo\Bar;`, + * `use Foo\Bar as Baz;`, `use function`, `use const`. + * All use-statement flavors that introduce a name sit at this tier. + * - 2: `wildcard` — grouped uses / wildcard imports (deferred; mapped + * here for completeness). + * + * Within a surviving tier we de-dup by `DefId`, last-write-wins so a + * `use` re-declared further down the file cleanly replaces the earlier + * binding. + */ + +import type { BindingRef } from 'gitnexus-shared'; + +const TIER_LOCAL = 0; +const TIER_IMPORT = 1; +const TIER_WILDCARD = 2; +const TIER_UNKNOWN = 3; + +function tierOf(b: BindingRef): number { + switch (b.origin) { + case 'local': + return TIER_LOCAL; + case 'reexport': + case 'import': + case 'namespace': + return TIER_IMPORT; + case 'wildcard': + return TIER_WILDCARD; + default: + return TIER_UNKNOWN; + } +} + +export function phpMergeBindings(bindings: readonly BindingRef[]): readonly BindingRef[] { + if (bindings.length === 0) return bindings; + + let bestTier = Number.POSITIVE_INFINITY; + for (const b of bindings) bestTier = Math.min(bestTier, tierOf(b)); + const survivors = bindings.filter((b) => tierOf(b) === bestTier); + + const seen = new Map(); + for (const b of survivors) seen.set(b.def.nodeId, b); + return [...seen.values()]; +} diff --git a/gitnexus/src/core/ingestion/languages/php/namespace-siblings.ts b/gitnexus/src/core/ingestion/languages/php/namespace-siblings.ts new file mode 100644 index 000000000..20b99d971 --- /dev/null +++ b/gitnexus/src/core/ingestion/languages/php/namespace-siblings.ts @@ -0,0 +1,335 @@ +/** + * PHP same-namespace cross-file visibility. + * + * In PHP, every class declared in `namespace Foo\Bar` is visible to all + * other files in the same namespace WITHOUT an explicit `use` statement. + * Without this pass, `Service.php` (namespace `App\Services`) can't see + * `User` declared in `Models.php` (namespace `App\Models`) unless + * `UserService.php` has an explicit `use App\Models\User` statement. + * + * More importantly, A.php (namespace `App\Models`) can return `Greeting` + * (same namespace `App\Models`) without importing it, and the compound- + * receiver resolver needs to find `Greeting` as a class binding in the + * scope chain. + * + * Implementation mirrors C#'s `namespace-siblings.ts`: + * 1. Extract the declared namespace from each PHP file's source. + * 2. Group class-like defs by namespace. + * 3. Inject sibling class defs into each file's Module scope's + * `bindingAugmentations` with `origin: 'namespace'`. + * 4. Also mirror return-type bindings from same-namespace siblings + * so cross-file chain-follow finds return types without explicit imports. + * + * Uses the PHP tree-sitter parser (via the lazy singleton in `query.ts`) + * to extract namespace declarations — same AST that `extractParsedFile` + * already parsed, reused via `treeCache` to avoid double-parsing. + */ + +import type { BindingRef, ParsedFile, Scope, ScopeId, SymbolDefinition } from 'gitnexus-shared'; +import type { ScopeResolutionIndexes } from '../../model/scope-resolution-indexes.js'; +import { getPhpParser } from './query.js'; +import { getTreeSitterBufferSize } from '../../constants.js'; +import { parseSourceSafe } from '../../../tree-sitter/safe-parse.js'; + +// ─── PHP file structure extraction ────────────────────────────────────────── + +interface PhpFileStructure { + /** The declared namespace (backslash-separated), or '' for global namespace. */ + readonly namespace: string; +} + +type PhpTree = ReturnType['parse']>; + +/** + * Extract the declared namespace from a PHP file's source. + * Uses the cached AST tree when available to avoid re-parsing. + */ +function extractPhpFileStructure(content: string, cachedTree: unknown): PhpFileStructure { + const tree = + (cachedTree as PhpTree | undefined) ?? + parseSourceSafe(getPhpParser(), content, undefined, { + bufferSize: getTreeSitterBufferSize(content), + }); + + // Walk top-level nodes looking for namespace_definition. + // PHP files have at most one namespace declaration (PSR-4 convention). + // `namespace_definition` has a `name:` field of type `namespace_name`. + const root = tree.rootNode; + for (let i = 0; i < root.namedChildCount; i++) { + const child = root.namedChild(i); + if (child === null) continue; + if (child.type === 'namespace_definition') { + const nameNode = child.childForFieldName('name'); + if (nameNode !== null) { + return { namespace: nameNode.text }; + } + } + } + + return { namespace: '' }; +} + +// ─── Augmentation bucket helper ───────────────────────────────────────────── + +function getAugmentationBucket( + augmentations: Map>, + scopeId: ScopeId, + name: string, +): BindingRef[] { + let scopeBindings = augmentations.get(scopeId); + if (scopeBindings === undefined) { + scopeBindings = new Map(); + augmentations.set(scopeId, scopeBindings); + } + let bucket = scopeBindings.get(name); + if (bucket === undefined) { + bucket = []; + scopeBindings.set(name, bucket); + } + return bucket; +} + +function isClassLikeDef(def: SymbolDefinition): boolean { + return ( + def.type === 'Class' || + def.type === 'Interface' || + def.type === 'Struct' || + def.type === 'Enum' || + def.type === 'Trait' + ); +} + +// ─── Public entry point ────────────────────────────────────────────────────── + +export interface PhpSiblingInputs { + readonly fileContents: ReadonlyMap; + readonly treeCache?: { get(filePath: string): unknown }; +} + +/** + * Side-channel cache populated by `populatePhpNamespaceSiblings` so that + * later visibility-check hooks (e.g., `isCallableVisibleFromCaller`) can + * look up a file's PHP namespace without re-parsing. Cleared at the start + * of every populate run so stale entries don't leak across resolutions. + */ +const namespaceByFilePath = new Map(); + +/** + * Read the cached PHP namespace for a given filePath. Returns `''` (global) + * when the file has no namespace_definition or hasn't been processed yet. + * Callers should only consult this AFTER either `populatePhpClassQualifiedNames` + * or `populatePhpNamespaceSiblings` has run for the current resolution. + */ +export function getPhpNamespaceForFile(filePath: string): string { + return namespaceByFilePath.get(filePath) ?? ''; +} + +/** + * Inject same-namespace class defs and return-type bindings into each + * PHP file's Module scope's `bindingAugmentations`. This makes classes + * in the same PHP namespace visible to each other without explicit `use` + * statements, mirroring PHP's actual runtime behavior. + * + * Uses `origin: 'namespace'` so `phpMergeBindings` tiers it below + * explicit `use` imports (`origin: 'import'`) and local declarations. + */ +export function populatePhpNamespaceSiblings( + parsedFiles: readonly ParsedFile[], + indexes: ScopeResolutionIndexes, + inputs: PhpSiblingInputs, +): void { + // Step 1: extract namespace structure for each file. Also seed the + // side-channel cache used by visibility-check hooks downstream. + namespaceByFilePath.clear(); + const structureByFile = new Map(); + for (const parsed of parsedFiles) { + const content = inputs.fileContents.get(parsed.filePath); + if (content === undefined) continue; + const cachedTree = inputs.treeCache?.get(parsed.filePath); + const struct = extractPhpFileStructure(content, cachedTree); + structureByFile.set(parsed.filePath, struct); + namespaceByFilePath.set(parsed.filePath, struct.namespace); + } + + // Step 2: group class-like defs and module scopes by namespace. + interface NamespaceBucket { + readonly scopes: { filePath: string; scopeId: ScopeId; scope: Scope }[]; + readonly classDefs: SymbolDefinition[]; + } + const buckets = new Map(); + const getBucket = (ns: string): NamespaceBucket => { + let b = buckets.get(ns); + if (b === undefined) { + b = { scopes: [], classDefs: [] }; + buckets.set(ns, b); + } + return b; + }; + + for (const parsed of parsedFiles) { + const struct = structureByFile.get(parsed.filePath); + if (struct === undefined) continue; + const ns = struct.namespace; + const bucket = getBucket(ns); + + // Register the file's module scope in the bucket. + const moduleScope = parsed.scopes.find((s) => s.kind === 'Module'); + if (moduleScope !== undefined) { + bucket.scopes.push({ + filePath: parsed.filePath, + scopeId: moduleScope.id, + scope: moduleScope, + }); + } + + // Collect class-like defs declared at the top-level of this file + // (defs in Class or Module scopes, excluding nested inner classes). + for (const scope of parsed.scopes) { + if (scope.kind !== 'Class') continue; + // Only top-level class scopes (parent is Module or Namespace scope). + if (scope.parent === null) continue; + const parentScope = parsed.scopes.find((s) => s.id === scope.parent); + if ( + parentScope === undefined || + (parentScope.kind !== 'Module' && parentScope.kind !== 'Namespace') + ) { + continue; + } + for (const def of scope.ownedDefs) { + if (isClassLikeDef(def)) { + bucket.classDefs.push(def); + break; // one class-like per scope + } + } + } + } + + const augmentations = indexes.bindingAugmentations as Map>; + + // Step 3: For each namespace bucket, inject sibling class bindings + // into every file's Module scope (that is NOT the declaring file). + for (const [, bucket] of buckets) { + // Build name → def map (simple name of qualifiedName). + const defsByName = new Map(); + for (const def of bucket.classDefs) { + const q = def.qualifiedName ?? ''; + const simpleName = q.includes('.') + ? q.slice(q.lastIndexOf('.') + 1) + : q.includes('\\') + ? q.slice(q.lastIndexOf('\\') + 1) + : q; + if (simpleName === '') continue; + const arr = defsByName.get(simpleName) ?? []; + arr.push(def); + defsByName.set(simpleName, arr); + } + + for (const { filePath, scopeId, scope } of bucket.scopes) { + for (const [name, defs] of defsByName) { + // Skip if already locally declared (origin: 'local' wins). + const local = scope.bindings.get(name); + if (local !== undefined && local.some((b) => b.origin === 'local')) continue; + + for (const def of defs) { + if (def.filePath === filePath) continue; // don't self-inject + const arr = getAugmentationBucket(augmentations, scopeId, name); + if (arr.some((b) => b.def.nodeId === def.nodeId)) continue; + arr.push({ def, origin: 'namespace' }); + } + } + } + } + + // Step 3b: Inject fully-qualified-name bindings into every PHP file's + // Module scope. PHP `\App\Models\User` (leading-backslash FQN) and + // `App\Models\User` (already-qualified relative) on a parameter or + // typed receiver must resolve to the exact namespace-qualified class + // regardless of which simple-name `User` the caller's `use` imports + // shadowed. The shared `findClassBindingInScope` scope-chain walk + // consumes these augmentations via `lookupBindingsAt`, so adding the + // qualified key on every file's module scope routes FQN-receivers to + // the right def. Codex PR #1497 review, finding 1. + // + // Cost: O(PHP files × class-like defs in the workspace) augmentation + // entries. Bounded and acceptable in practice — typical PHP projects + // have hundreds of files and classes, not tens of thousands. + for (const parsed of parsedFiles) { + const moduleScope = parsed.scopes.find((s) => s.kind === 'Module'); + if (moduleScope === undefined) continue; + const moduleScopeId = moduleScope.id; + + for (const [ns, bucket] of buckets) { + if (ns === '') continue; // global-namespace classes have no qualified form to register + for (const def of bucket.classDefs) { + const q = def.qualifiedName ?? ''; + const simpleName = q.includes('\\') ? q.slice(q.lastIndexOf('\\') + 1) : q; + if (simpleName === '') continue; + const fqn = `${ns}\\${simpleName}`; + const arr = getAugmentationBucket(augmentations, moduleScopeId, fqn); + if (arr.some((b) => b.def.nodeId === def.nodeId)) continue; + arr.push({ def, origin: 'namespace' }); + } + } + } + + // Step 4: Mirror return-type bindings from same-namespace sibling files. + // This enables chain-follow like `$c->greet()->save()` where `greet()` + // returns `Greeting` (declared in A.php, same namespace) and `Greeting` + // isn't imported in the calling file. Without this, the compound-receiver + // resolver can't resolve `Greeting` as a class binding in the importer's + // scope chain. + // + // Additionally, mirror from files that are imported via `use` (different + // namespace) so return types from dependencies are chain-followable too. + for (const parsed of parsedFiles) { + const moduleScope = parsed.scopes.find((s) => s.kind === 'Module'); + if (moduleScope === undefined) continue; + const moduleTypeBindings = moduleScope.typeBindings as Map< + string, + import('gitnexus-shared').TypeRef + >; + + const struct = structureByFile.get(parsed.filePath); + const ownNs = struct?.namespace ?? ''; + + // Collect namespaces accessible from this file: + // 1. Own namespace (same-ns siblings) + // 2. Namespaces of directly imported files (via parsedImports → targetRaw → PSR-4 namespace) + const accessibleFiles = new Set(); + + // Same-namespace siblings. + const sameBucket = buckets.get(ownNs); + if (sameBucket !== undefined) { + for (const { filePath } of sameBucket.scopes) { + if (filePath !== parsed.filePath) accessibleFiles.add(filePath); + } + } + + // Files directly imported by this file (finalized import edges). + const ownModuleScopeBindings = indexes.bindings.get(moduleScope.id); + if (ownModuleScopeBindings !== undefined) { + for (const [, refs] of ownModuleScopeBindings) { + for (const ref of refs) { + if (ref.origin === 'import' || ref.origin === 'namespace') { + const importFilePath = ref.def.filePath; + if (importFilePath !== parsed.filePath) { + accessibleFiles.add(importFilePath); + } + } + } + } + } + + // Mirror return-type bindings from accessible files. + for (const srcFilePath of accessibleFiles) { + const srcParsed = parsedFiles.find((p) => p.filePath === srcFilePath); + if (srcParsed === undefined) continue; + const srcModuleScope = srcParsed.scopes.find((s) => s.kind === 'Module'); + if (srcModuleScope === undefined) continue; + for (const [boundName, typeRef] of srcModuleScope.typeBindings) { + if (moduleTypeBindings.has(boundName)) continue; + moduleTypeBindings.set(boundName, typeRef); + } + } + } +} diff --git a/gitnexus/src/core/ingestion/languages/php/query.ts b/gitnexus/src/core/ingestion/languages/php/query.ts new file mode 100644 index 000000000..fa84c911f --- /dev/null +++ b/gitnexus/src/core/ingestion/languages/php/query.ts @@ -0,0 +1,332 @@ +/** + * Tree-sitter query for PHP scope captures (RFC #909 Ring 3 LANG-php). + * + * Captures the structural skeleton the generic scope-resolution pipeline + * consumes: scopes (program/namespace/class/function), declarations + * (class-likes, method-likes, properties, variables), imports + * (namespace_use_declaration), type bindings (parameter annotations, + * property types, constructor-inferred locals, return types), and + * references (call sites, member writes). + * + * PHP specifics that shape this query: + * + * - `namespace_use_declaration` is an import only at top level / inside + * namespace blocks. Class-body `use_declaration` (trait-use) is a + * different node type and is NOT captured here. + * + * - `object_creation_expression` has `name` and `qualified_name` as + * direct children (no wrapping node). + * + * - `method_declaration` exposes a `return_type:` named field containing + * a `type` node, which may be `named_type`, `optional_type`, etc. + * + * - `property_element` has a `name:` field of type `variable_name`. + * + * - `variable_name` nodes always include the `$` sigil in their text. + * + * Exposes lazy `Parser` and `Query` singletons so callers don't pay + * tree-sitter init cost per file. + */ + +import Parser from 'tree-sitter'; +import Php from 'tree-sitter-php'; + +// tree-sitter-php exports `{ php, php_only, html }` in recent versions, or the +// language directly in older versions. +// +// IMPORTANT: must match the grammar used by the central parse phase +// (`src/core/tree-sitter/parser-loader.ts` line: `[SupportedLanguages.PHP]: PHP.php_only`). +// Using a different grammar variant causes tree-sitter to throw when running +// a query built against grammar A on a tree parsed by grammar B — this error +// is swallowed by `scope-extractor-bridge.ts`, producing silent empty results. +const Php_typed = Php as unknown as { php_only?: unknown; php?: unknown }; +const PHP_LANG = Php_typed.php_only ?? Php_typed.php ?? Php; + +const PHP_SCOPE_QUERY = ` +;; ── Scopes ──────────────────────────────────────────────────────────────── + +(program) @scope.module + +;; Both block-scoped and statement-scoped namespace declarations. +(namespace_definition) @scope.namespace + +(class_declaration) @scope.class +(interface_declaration) @scope.class +(trait_declaration) @scope.class +(enum_declaration) @scope.class + +(method_declaration) @scope.function +(function_definition) @scope.function +(anonymous_function) @scope.function +(arrow_function) @scope.function + +;; ── Declarations — types ────────────────────────────────────────────────── + +(class_declaration + name: (name) @declaration.name) @declaration.class + +(interface_declaration + name: (name) @declaration.name) @declaration.interface + +(trait_declaration + name: (name) @declaration.name) @declaration.trait + +(enum_declaration + name: (name) @declaration.name) @declaration.enum + +;; ── Declarations — methods / functions / constructors ───────────────────── + +(method_declaration + name: (name) @declaration.name) @declaration.method + +(function_definition + name: (name) @declaration.name) @declaration.function + +;; ── Declarations — properties ───────────────────────────────────────────── + +;; PHP 7.4+ typed property: private UserRepo $repo; +;; property_element has name: (variable_name) field. +;; Emits BOTH a declaration (so SemanticModel registers the property) AND a type-binding. +(property_declaration + type: (_) @type-binding.type + (property_element + name: (variable_name) @type-binding.name)) @type-binding.annotation + +(property_declaration + type: (_) + (property_element + name: (variable_name) @declaration.name)) @declaration.property + +;; Untyped property: public $id; — capture as plain declaration. +(property_declaration + (property_element + name: (variable_name) @declaration.name)) @declaration.variable + +;; ── Imports — namespace_use_declaration ─────────────────────────────────── +;; +;; Captures ALL forms: plain, alias, function/const qualifiers, and grouped. +;; The import-decomposer in captures.ts fans out grouped uses. +;; +;; NOTE: class-body use_declaration = trait-use, NOT an import. +;; Only namespace_use_declaration (top-level / namespace scope) is an import. + +(namespace_use_declaration) @import.statement + +;; ── Type bindings — parameters ──────────────────────────────────────────── + +;; simple_parameter with a type hint: function f(User $u) +;; type field is a 'type' supertype (named_type, optional_type, union_type, etc.) +(simple_parameter + type: (_) @type-binding.type + name: (variable_name) @type-binding.name) @type-binding.parameter + +;; property_promotion_parameter: function __construct(private User $u) +;; Emits type-binding so the constructor body can resolve $u as the typed param. +(property_promotion_parameter + type: (_) @type-binding.type + name: (variable_name) @type-binding.name) @type-binding.parameter + +;; Also emit a @type-binding.annotation for the promoted parameter so that +;; phpBindingScopeFor can hoist it to the Class scope (stripping the $ sigil). +;; This enables compound-receiver resolution: $user->address->save() resolves +;; address → Address via the Class scope's typeBindings. +;; The @type-binding.parameter above stays for constructor-body resolution ($address). +(property_promotion_parameter + type: (_) @type-binding.type + name: (variable_name) @type-binding.name) @type-binding.annotation + +;; Also emit a @declaration.property so SemanticModel registers the promoted +;; parameter as a class-owned property (enabling $obj->propName lookups). +(property_promotion_parameter + name: (variable_name) @declaration.name) @declaration.property + +;; ── Type bindings — local assignment: $u = new User() ───────────────────── + +;; new ClassName() — name is a direct child of object_creation_expression +(assignment_expression + left: (variable_name) @type-binding.name + right: (object_creation_expression + (name) @type-binding.type)) @type-binding.constructor + +;; new Foo\Bar\ClassName() — qualified_name wraps name +(assignment_expression + left: (variable_name) @type-binding.name + right: (object_creation_expression + (qualified_name + (name) @type-binding.type))) @type-binding.constructor + +;; ── Type bindings — $alias = $u (identifier alias) ─────────────────────── + +(assignment_expression + left: (variable_name) @type-binding.name + right: (variable_name) @type-binding.type) @type-binding.alias + +;; ── Type bindings — $u = factory() (free call return alias) ────────────── + +(assignment_expression + left: (variable_name) @type-binding.name + right: (function_call_expression + function: (name) @type-binding.type)) @type-binding.alias + +;; ── Type bindings — $u = $svc->getUser() (method call return alias) ─────── + +(assignment_expression + left: (variable_name) @type-binding.name + right: (member_call_expression + name: (name) @type-binding.type)) @type-binding.alias + +;; ── Type bindings — method return type ─────────────────────────────────── + +;; method_declaration exposes return_type: field (type node supertype). +;; named_type wraps the class name: function getUser(): User +(method_declaration + name: (name) @type-binding.name + return_type: (named_type + (name) @type-binding.type)) @type-binding.return + +;; nullable return type via optional_type: function getUser(): ?User +(method_declaration + name: (name) @type-binding.name + return_type: (optional_type + (named_type + (name) @type-binding.type))) @type-binding.return + +;; function_definition (top-level or namespace-level) return type: User +;; Enables cross-file return-type propagation for free functions. +(function_definition + name: (name) @type-binding.name + return_type: (named_type + (name) @type-binding.type)) @type-binding.return + +;; nullable return type for function_definition: ?User +(function_definition + name: (name) @type-binding.name + return_type: (optional_type + (named_type + (name) @type-binding.type))) @type-binding.return + +;; ── References — free calls: foo() ─────────────────────────────────────── + +(function_call_expression + function: (name) @reference.name) @reference.call.free + +;; ── References — member calls: $obj->method() ──────────────────────────── +;; +;; SAFETY-INVARIANT (Finding 1 of PR #1497 adversarial review): the name: +;; field is constrained to (name), NOT (_) — tree-sitter-php emits +;; variable_name nodes for dynamic method names ($obj->$method(), +;; $obj->{$method}()). Keeping the pattern at (name) is what suppresses +;; capture of those dynamic shapes. The resolver is structural-only and +;; cannot infer the bound method name from runtime values; relaxing this +;; pattern to (_) would silently emit zero-confidence false-positive +;; edges. Regression: test/fixtures/lang-resolution/php-dynamic-calls/. + +(member_call_expression + object: (_) @reference.receiver + name: (name) @reference.name) @reference.call.member + +;; ── References — null-safe member calls: $obj?->method() (PHP 8+) ───────── + +(nullsafe_member_call_expression + object: (_) @reference.receiver + name: (name) @reference.name) @reference.call.member + +;; ── References — static calls: X::method() ─────────────────────────────── +;; +;; Same SAFETY-INVARIANT as member_call_expression above: name: (name) +;; deliberately excludes variable_name so Class::$method() and +;; $className::$method() shapes do not capture. The receiver field uses +;; (_) because static dispatch on a variable receiver +;; ($className::method()) IS captured — but resolution falls through +;; harmlessly when $className has no class type binding. See +;; php-dynamic-calls/ regression suite. + +(scoped_call_expression + scope: (_) @reference.receiver + name: (name) @reference.name) @reference.call.member + +;; ── Type bindings — $x = X::Constant or $x = X::CASE (enum case) ───────── +;; Binds the variable to the class name X so member calls on $x dispatch +;; to X's methods (e.g. UserRole::Viewer → label()). +;; +;; tree-sitter-php emits class_constant_access_expression with two name +;; children: [0]=class/enum name, [1]=constant/case name. The dot-anchor +;; before (name) matches only the FIRST name child (the class). + +(assignment_expression + left: (variable_name) @type-binding.name + right: (class_constant_access_expression + . (name) @type-binding.type)) @type-binding.alias + +(assignment_expression + left: (variable_name) @type-binding.name + right: (class_constant_access_expression + (qualified_name + (name) @type-binding.type))) @type-binding.alias + +;; ── Type bindings — $x = SomeClass::staticFactory() ────────────────────── +;; Binds $x to the type returned by the static factory method, anchored on +;; the method name (chain-follow resolves the actual return type later). + +(assignment_expression + left: (variable_name) @type-binding.name + right: (scoped_call_expression + name: (name) @type-binding.type)) @type-binding.alias + +;; ── Type bindings — null-safe member-call result: $x = $a?->getY() ─────── + +(assignment_expression + left: (variable_name) @type-binding.name + right: (nullsafe_member_call_expression + name: (name) @type-binding.type)) @type-binding.alias + +;; ── References — constructor calls: new User() ─────────────────────────── + +(object_creation_expression + (name) @reference.name) @reference.call.constructor + +(object_creation_expression + (qualified_name + (name) @reference.name)) @reference.call.constructor + +;; ── References — member writes: $obj->prop = $x ────────────────────────── + +(assignment_expression + left: (member_access_expression + object: (_) @reference.receiver + name: (name) @reference.name)) @reference.write.member + +;; ── References — static property writes: User::$count = $x ────────────── +;; Uses @reference.write.static anchor so captures.ts can strip the leading +;; $ from the variable_name capture (static props are stored without $ in graph). +;; +;; SAFETY-INVARIANT (Finding 2 of PR #1497 adversarial review): no +;; read-access property capture exists in this query — dynamic property +;; reads ($obj->$prop, $obj->{$prop}) produce no captures, which is the +;; desired behavior for a structural-only resolver. Adding a read pattern +;; in the future MUST keep name: (name) (not (_)) to preserve the +;; suppression. Regression: php-dynamic-calls/ fixture dynamicPropertyRead. + +(assignment_expression + left: (scoped_property_access_expression + scope: (_) @reference.receiver + name: (variable_name) @reference.name)) @reference.write.static +`; + +let _parser: Parser | null = null; +let _query: Parser.Query | null = null; + +export function getPhpParser(): Parser { + if (_parser === null) { + _parser = new Parser(); + _parser.setLanguage(PHP_LANG as Parameters[0]); + } + return _parser; +} + +export function getPhpScopeQuery(): Parser.Query { + if (_query === null) { + _query = new Parser.Query(PHP_LANG as Parameters[0], PHP_SCOPE_QUERY); + } + return _query; +} diff --git a/gitnexus/src/core/ingestion/languages/php/receiver-binding.ts b/gitnexus/src/core/ingestion/languages/php/receiver-binding.ts new file mode 100644 index 000000000..79709fe61 --- /dev/null +++ b/gitnexus/src/core/ingestion/languages/php/receiver-binding.ts @@ -0,0 +1,136 @@ +/** + * Synthesize `@type-binding.self` captures for PHP instance methods — + * one for `$this` (always on non-static methods inside a type + * declaration) and optionally one for `parent` (only on class methods + * when the enclosing class has an explicit `base_clause`). + * + * Mirrors `languages/csharp/receiver-binding.ts` in structure. PHP's + * grammar doesn't give us a clean `.scm` pattern for "implicit receiver + * on every instance method inside an enclosing type" because `$this` is + * not a parameter — it's an implicit receiver. Synthesis in code is the + * same approach C# uses for `this` / `base`. + * + * ## Known limitations + * + * - **Trait `$this`**: for methods defined in a trait, `$this` is + * synthesized as a binding to the trait itself. The actual using-class + * type is not known at single-file parse time. V1 limitation — + * documented in `index.ts`. + * - **Anonymous classes**: skipped (no stable enclosing class name). + */ + +import type { Capture, CaptureMatch } from 'gitnexus-shared'; +import { nodeToCapture, syntheticCapture, type SyntaxNode } from '../../utils/ast-helpers.js'; + +const TYPE_DECL_NODE_TYPES = new Set([ + 'class_declaration', + 'interface_declaration', + 'trait_declaration', + 'enum_declaration', +]); + +const FUNCTION_NODE_TYPES = new Set([ + 'method_declaration', + 'function_definition', + 'anonymous_function', + 'arrow_function', +]); + +/** Walk up to find the enclosing type declaration. */ +function findEnclosingTypeDeclaration(node: SyntaxNode): SyntaxNode | null { + let cur: SyntaxNode | null = node.parent; + while (cur !== null) { + if (TYPE_DECL_NODE_TYPES.has(cur.type)) return cur; + cur = cur.parent; + } + return null; +} + +function typeName(typeNode: SyntaxNode): string | null { + return typeNode.childForFieldName('name')?.text ?? null; +} + +/** + * Return the base class name from a `base_clause` child of the class node. + * `base_clause` contains a `qualified_name` or `name` child. + */ +function baseClauseText(typeNode: SyntaxNode): string | null { + for (let i = 0; i < typeNode.namedChildCount; i++) { + const child = typeNode.namedChild(i); + if (child === null || child.type !== 'base_clause') continue; + const nameNode = child.firstNamedChild; + if (nameNode === null) return null; + // Take last segment of qualified name (e.g. \App\Models\BaseModel → BaseModel) + const text = nameNode.text.trim(); + const segments = text.split('\\').filter(Boolean); + return segments[segments.length - 1] ?? text; + } + return null; +} + +/** Check whether this method has a `static_modifier` child. */ +function isStaticMethod(fnNode: SyntaxNode): boolean { + for (let i = 0; i < fnNode.namedChildCount; i++) { + const child = fnNode.namedChild(i); + if (child !== null && child.type === 'static_modifier') return true; + } + return false; +} + +/** + * Build zero, one, or two `@type-binding.self` matches for `fnNode`: + * + * - Returns `[]` if the function is free (no enclosing type), static, + * or the enclosing type has no resolvable name. + * - Returns one match (`$this`) for non-static methods inside a + * class / trait / interface / enum body. + * - Returns two matches (`$this` + `parent`) only when the function + * lives in a `class_declaration` that has an explicit `base_clause`. + * + * The caller is responsible for guaranteeing + * `FUNCTION_NODE_TYPES.has(fnNode.type)`. + */ +export function synthesizePhpReceiverBinding(fnNode: SyntaxNode): CaptureMatch[] { + if (!FUNCTION_NODE_TYPES.has(fnNode.type)) return []; + if (isStaticMethod(fnNode)) return []; + + const enclosingType = findEnclosingTypeDeclaration(fnNode); + if (enclosingType === null) return []; + + // Anonymous class — skip (no stable name). + if (enclosingType.type === 'anonymous_class_declaration') return []; + + const enclosingName = typeName(enclosingType); + if (enclosingName === null) return []; + + // Anchor the synthesized captures to the method body (compound_statement) + // so they land inside the function scope, not at the class scope. + // For interface/abstract methods that have no body, skip. + const bodyNode = + fnNode.childForFieldName('body') ?? + // arrow_function: body is the expression after `=>` + fnNode.childForFieldName('return_value'); + if (bodyNode === null) return []; + + const out: CaptureMatch[] = []; + out.push(buildReceiverMatch(bodyNode, '$this', enclosingName)); + + // `parent` applies only to class methods with an explicit base_clause. + if (enclosingType.type === 'class_declaration') { + const baseText = baseClauseText(enclosingType); + if (baseText !== null) { + out.push(buildReceiverMatch(bodyNode, 'parent', baseText)); + } + } + + return out; +} + +function buildReceiverMatch(anchorNode: SyntaxNode, name: string, typeText: string): CaptureMatch { + const m: Record = { + '@type-binding.self': nodeToCapture('@type-binding.self', anchorNode), + '@type-binding.name': syntheticCapture('@type-binding.name', anchorNode, name), + '@type-binding.type': syntheticCapture('@type-binding.type', anchorNode, typeText), + }; + return m; +} diff --git a/gitnexus/src/core/ingestion/languages/php/scope-resolver.ts b/gitnexus/src/core/ingestion/languages/php/scope-resolver.ts new file mode 100644 index 000000000..8b1fcf41b --- /dev/null +++ b/gitnexus/src/core/ingestion/languages/php/scope-resolver.ts @@ -0,0 +1,421 @@ +/** + * PHP `ScopeResolver` registered in `SCOPE_RESOLVERS` and consumed by + * the generic `runScopeResolution` orchestrator (RFC #909 Ring 3 LANG-php). + * + * Third migration after Python and C#. See `pythonScopeResolver` for the + * canonical shape. + * + * ## Circular-import avoidance + * + * The old PR had `php/scope-resolver.ts` importing `phpProvider` from + * `../php.js` while `php.ts` imported `phpScopeResolver` from `./php/index.js` + * — undefined at module load. The canonical fix (mirroring C#): + * + * - `scope-resolver.ts` imports `phpProvider` from `../php.js` ✓ + * - `php.ts` imports individual hook FUNCTIONS from `./php/index.js` ✗ + * + * Node's ESM handles the cycle correctly because `phpProvider` is a named + * export that is live-binding — by the time `phpScopeResolver` is first + * read (lazily, at resolution time), `phpProvider` is fully initialized. + */ + +import type { ParsedFile } from 'gitnexus-shared'; +import { SupportedLanguages } from 'gitnexus-shared'; +import { buildMro, defaultLinearize } from '../../scope-resolution/passes/mro.js'; +import { + findReceiverTypeBinding, + populateClassOwnedMembers, +} from '../../scope-resolution/scope/walkers.js'; +import type { ScopeResolver } from '../../scope-resolution/contract/scope-resolver.js'; +import type { KnowledgeGraph } from '../../../graph/types.js'; +import type { GraphNodeLookup } from '../../scope-resolution/graph-bridge/node-lookup.js'; +import { + resolveCallerGraphId, + resolveDefGraphId, +} from '../../scope-resolution/graph-bridge/ids.js'; +import { narrowOverloadCandidates } from '../../scope-resolution/passes/overload-narrowing.js'; +import type { SemanticModel } from '../../model/semantic-model.js'; +import type { ScopeResolutionIndexes } from '../../model/scope-resolution-indexes.js'; +import type { SymbolDefinition } from 'gitnexus-shared'; +import { phpProvider } from '../php.js'; +import { phpArityCompatibility, phpMergeBindings } from './index.js'; +import { resolvePhpImportTargetInternal, loadPhpComposerConfig } from './import-target.js'; +import { populatePhpNamespaceSiblings, getPhpNamespaceForFile } from './namespace-siblings.js'; + +/** + * PHP MRO builder — extends the generic EXTENDS-only MRO with trait-use + * relationships encoded as IMPLEMENTS edges. + * + * PHP trait-use (`use TraitName;` inside a class body) is recorded in the + * graph as an IMPLEMENTS edge from the using class to the Trait node. The + * generic `buildMro` only walks EXTENDS edges, so trait methods are invisible + * to the MRO-based dispatch index. This variant: + * + * 1. Runs the generic `buildMro` (EXTENDS edges, Class defs only). + * 2. Indexes Trait defs from `parsedFiles` alongside Class defs. + * 3. Walks IMPLEMENTS edges; for each edge whose target resolves to a + * Trait DefId, prepends that Trait DefId to the source class's MRO. + * + * Trait methods are searched BEFORE parent-class methods (PHP semantics: + * a trait method shadows the parent-class method but is overridden by the + * using class's own methods). + */ +/** + * PHP free-call visibility check for `pickUniqueGlobalCallable`. Returns + * true when the candidate function is reachable from the caller's PHP + * namespace context, false when the cross-namespace bridge would be a + * false positive (e.g., `\App\Utils\format` is not visible from `\App` + * without an explicit `use function App\Utils\format;`). + * + * Rules (PHP semantics): + * 1. Same-namespace candidates are always visible. + * 2. Global-namespace candidates (no namespace prefix) are visible from + * every caller — PHP's global fallback for functions/constants. + * 3. Candidates in a different namespace are visible only when the + * caller has a `use function` import that matches the candidate's + * fully-qualified name. + */ +function phpIsCallableVisibleFromCaller(ctx: { + callerParsed: ParsedFile; + candidate: SymbolDefinition; +}): boolean { + const { callerParsed, candidate } = ctx; + const callerNs = getPhpNamespaceForFile(callerParsed.filePath); + const candNs = getPhpNamespaceForFile(candidate.filePath); + + // Global-namespace candidate: PHP falls back to global for functions + // and constants when the local namespace doesn't define them. + if (candNs === '') return true; + + // Same-namespace: caller can see the candidate without an explicit use. + if (candNs === callerNs) return true; + + // Cross-namespace: require an explicit `use function` import in the + // caller's parsedImports that matches the candidate's fully-qualified + // name. interpret.ts maps `use function Foo\bar` to a named import with + // localName = 'bar' and targetRaw = 'Foo\\bar'. + const candQualified = + candidate.qualifiedName === undefined + ? '' + : candNs !== '' && !candidate.qualifiedName.includes('\\') + ? `${candNs}\\${candidate.qualifiedName}` + : candidate.qualifiedName; + if (candQualified === '') return false; + return callerParsed.parsedImports.some( + (imp) => + imp.kind === 'named' && + imp.targetRaw.replace(/^\\+/, '') === candQualified.replace(/^\\+/, ''), + ); +} + +/** + * Compute the EXTENDS-only ancestor chain for every class — no trait + * augmentation. PHP semantics: `parent::method()` walks this view so + * that `parent::` resolves to the parent class's method, even when a + * composed trait shadows the same name. + * + * Returns the same shape as `buildPhpMro` so callers can swap views + * without changing dispatch logic. Just `buildMro` + `defaultLinearize` + * — no trait IMPLEMENTS edge walk. + */ +function buildPhpExtendsOnlyMro( + graph: KnowledgeGraph, + parsedFiles: readonly ParsedFile[], + nodeLookup: GraphNodeLookup, +): Map { + return buildMro(graph, parsedFiles, nodeLookup, defaultLinearize); +} + +function buildPhpMro( + graph: KnowledgeGraph, + parsedFiles: readonly ParsedFile[], + nodeLookup: GraphNodeLookup, +): Map { + // Step 1: run generic MRO (Class-only, EXTENDS-only). + const mro = buildMro(graph, parsedFiles, nodeLookup, defaultLinearize); + + // Step 2: build a graphId → defId map for ALL class-like defs including Traits. + // After the `isLinkableLabel` fix, Trait nodes are now indexed in nodeLookup. + const defIdByGraphId = new Map(); + for (const parsed of parsedFiles) { + for (const def of parsed.localDefs) { + if (def.type !== 'Class' && def.type !== 'Trait') continue; + const graphId = resolveDefGraphId(parsed.filePath, def, nodeLookup); + if (graphId !== undefined) defIdByGraphId.set(graphId, def.nodeId); + } + } + + // Step 2b: build a Set of Trait defIds for O(1) trait-vs-interface checks. + const traitDefIds = new Set(); + for (const parsed of parsedFiles) { + for (const def of parsed.localDefs) { + if (def.type === 'Trait') traitDefIds.add(def.nodeId); + } + } + + // Step 3: collect direct trait-use edges (IMPLEMENTS where target is a Trait). + // Maps class/trait defId → [traitDefId, ...] for direct `use TraitName;`. + const directTraitUse = new Map(); + for (const rel of graph.iterRelationshipsByType('IMPLEMENTS')) { + const sourceDefId = defIdByGraphId.get(rel.sourceId); + if (sourceDefId === undefined) continue; + const targetDefId = defIdByGraphId.get(rel.targetId); + if (targetDefId === undefined) continue; + if (!traitDefIds.has(targetDefId)) continue; + + let list = directTraitUse.get(sourceDefId); + if (list === undefined) { + list = []; + directTraitUse.set(sourceDefId, list); + } + if (!list.includes(targetDefId)) list.push(targetDefId); + } + + // Step 4: augment every class's MRO by prepending the traits used by + // any class in its ancestor chain (transitively closed). PHP semantics: + // a trait used by a parent class is also visible on the child, and a + // trait-using-trait chain is flattened to a single ancestor set. + // + // For each class, walk its (already-computed) EXTENDS-based MRO and + // collect all transitively-used traits via BFS — `trait A { use B; } + // trait B { use C; } class X { use A; }` must include C in X's MRO. + // Prepend them before the EXTENDS ancestors so the method dispatch + // index finds trait methods before falling back to the parent class + // hierarchy. + for (const [classDefId, extendsMro] of mro) { + const ancestorChain = [classDefId, ...extendsMro]; + const seeds: string[] = []; + for (const ancestorId of ancestorChain) { + for (const traitId of directTraitUse.get(ancestorId) ?? []) { + seeds.push(traitId); + } + } + const allTraits = collectTransitiveTraits(seeds, directTraitUse); + + if (allTraits.length > 0) { + // Prepend traits before EXTENDS ancestors: own class's traits first, + // then parent traits (in ancestor order). This ensures trait methods + // are found before falling back to the inheritance chain. + mro.set(classDefId, [...allTraits, ...extendsMro]); + } + } + + // Step 5: also insert Trait-only entries for classes that use traits + // directly but have no EXTENDS parents (not in `mro` yet). + for (const [classDefId, traits] of directTraitUse) { + if (!mro.has(classDefId) && !traitDefIds.has(classDefId)) { + // Class with no EXTENDS but with trait-use — add to MRO map. + const allTraits = collectTransitiveTraits([...traits], directTraitUse); + mro.set(classDefId, allTraits); + } + } + + return mro; +} + +/** + * Collect the transitive closure of traits reachable from the seed set. + * BFS over `directTraitUse` until fixpoint. The `seen` set guards against + * cycles (invalid PHP but defensively handled) and prevents duplicate + * entries when multiple seeds converge on the same trait. Insertion order + * is preserved — first-seen wins for MRO ordering. + */ +function collectTransitiveTraits( + seeds: readonly string[], + directTraitUse: ReadonlyMap, +): string[] { + const out: string[] = []; + const seen = new Set(); + const queue: string[] = [...seeds]; + while (queue.length > 0) { + const t = queue.shift()!; + if (seen.has(t)) continue; + seen.add(t); + out.push(t); + for (const next of directTraitUse.get(t) ?? []) { + if (!seen.has(next)) queue.push(next); + } + } + return out; +} + +/** + * Emit CALLS edges for PHP member-call sites whose receiver has no type + * binding (e.g. `mixed`-typed parameters, untyped variables). + * + * PHP is dynamically typed: a parameter declared as `mixed` (or with no + * type hint) cannot be resolved by the generic receiver-bound pass, which + * requires a `TypeRef` in scope. This hook does a workspace-wide method + * name lookup: when exactly one def in the workspace matches the called + * method name, emit the CALLS edge. + * + * Only fires for sites that are NOT already in `handledSites` and whose + * receiver has no type binding in the scope chain. Unique-name-match + * constraint avoids false positives for common method names. + */ +function phpEmitUnresolvedReceiverEdges( + graph: KnowledgeGraph, + scopes: ScopeResolutionIndexes, + parsedFiles: readonly ParsedFile[], + nodeLookup: GraphNodeLookup, + handledSites: Set, + model: SemanticModel, +): number { + let emitted = 0; + const seen = new Set(); + + for (const parsed of parsedFiles) { + for (const site of parsed.referenceSites) { + if (site.kind !== 'call') continue; + if (site.explicitReceiver === undefined) continue; + + const siteKey = `${parsed.filePath}:${site.atRange.startLine}:${site.atRange.startCol}`; + if (handledSites.has(siteKey)) continue; + + // Only proceed when the receiver has NO type binding — it's unresolvable + // by the generic pass. This is the `mixed` / unannotated case. + const typeRef = findReceiverTypeBinding(site.inScope, site.explicitReceiver.name, scopes); + if (typeRef !== undefined) continue; + + // Workspace-wide lookup: collect all methods matching the called name. + // Filter out defs with no qualifiedName (legacy parse stubs without full + // metadata) and deduplicate by nodeId so reconcileOwnership double-registration + // doesn't inflate the count. + const allCandidates = model.methods.lookupMethodByName(site.name); + const seen2 = new Set(); + const candidates = allCandidates.filter((c) => { + if (c.qualifiedName === undefined) return false; + if (seen2.has(c.nodeId)) return false; + seen2.add(c.nodeId); + return true; + }); + if (candidates.length !== 1) continue; // ambiguous or missing — skip + + const fnDef = candidates[0]; + if (fnDef === undefined) continue; + + // Apply arity narrowing — a unique method name match is not enough + // when arity says the call is definitively incompatible (e.g., PHP + // f(int $req, ...$rest) called with zero args). This prevents the + // fallback from emitting edges that the receiver-bound pass already + // rejected for arity reasons. + if (narrowOverloadCandidates([fnDef], site.arity, site.argumentTypes).length === 0) { + continue; + } + + // Tighten the fallback further with an EXACT-required-arity gate + // (Finding 8 / U4): the first-stage `narrowOverloadCandidates` + // accepts any argCount in `min..max` (or `>= min` when variadic), + // which over-emits 0.6-confidence edges for common method names + // whose only workspace candidate has optional / defaulted params. + // For the fallback path only, require argCount === required for + // fixed-arity candidates. Variadic candidates keep the relaxed + // `argCount >= required` semantics (already enforced by the first- + // stage check, so no extra work here). + const min = fnDef.requiredParameterCount; + const hasVarArgs = + fnDef.parameterTypes !== undefined && + fnDef.parameterTypes.some((t) => t === '...' || t.startsWith('...')); + if ( + min !== undefined && + Number.isFinite(site.arity) && + site.arity >= 0 && + !hasVarArgs && + site.arity !== min + ) { + continue; + } + + const callerGraphId = resolveCallerGraphId(site.inScope, scopes, nodeLookup); + if (callerGraphId === undefined) continue; + const tgtGraphId = resolveDefGraphId(fnDef.filePath, fnDef, nodeLookup); + if (tgtGraphId === undefined) continue; + + handledSites.add(siteKey); + const relId = `rel:CALLS:${callerGraphId}->${tgtGraphId}`; + if (seen.has(relId)) continue; + seen.add(relId); + graph.addRelationship({ + id: relId, + sourceId: callerGraphId, + targetId: tgtGraphId, + type: 'CALLS', + confidence: 0.6, + reason: 'php-unresolved-receiver-fallback', + }); + emitted++; + } + } + return emitted; +} + +const phpScopeResolver: ScopeResolver = { + language: SupportedLanguages.PHP, + languageProvider: phpProvider, + importEdgeReason: 'php-scope: use', + + resolveImportTarget: (targetRaw, fromFile, allFilePaths, resolutionConfig) => + resolvePhpImportTargetInternal(targetRaw, fromFile, allFilePaths, resolutionConfig), + + loadResolutionConfig: (repoPath) => loadPhpComposerConfig(repoPath), + + // PHP LEGB-like precedence: local > import/namespace/reexport > wildcard. + // The per-scope id is unused by phpMergeBindings (tier ordering computed + // purely from BindingRef.origin), so we don't synthesize a Scope. + mergeBindings: (existing, incoming) => [...phpMergeBindings([...existing, ...incoming])], + + // Adapter: phpArityCompatibility uses (def, callsite); the contract is (callsite, def). + arityCompatibility: (callsite, def) => phpArityCompatibility(def, callsite), + + buildMro: (graph, parsedFiles, nodeLookup) => buildPhpMro(graph, parsedFiles, nodeLookup), + + // PHP-specific: parent::method() must walk inheritance only, skipping + // composed traits. See buildPhpExtendsOnlyMro and the super-branch use + // in `passes/receiver-bound-calls.ts`. + buildExtendsOnlyMro: (graph, parsedFiles, nodeLookup) => + buildPhpExtendsOnlyMro(graph, parsedFiles, nodeLookup), + + // PHP free-call visibility: cross-namespace candidates are blocked + // unless explicitly `use function`-imported by the caller. Prevents + // false-positive CALLS edges between unrelated namespaces sharing a + // function name. Same-namespace and global-namespace candidates pass + // unchanged. + isCallableVisibleFromCaller: phpIsCallableVisibleFromCaller, + + populateOwners: (parsed: ParsedFile) => populateClassOwnedMembers(parsed), + + // PHP same-namespace cross-file visibility — classes in the same + // PHP namespace are visible without explicit `use` statements. + // Mirrors C#'s `populateNamespaceSiblings`. + populateNamespaceSiblings: populatePhpNamespaceSiblings, + + // PHP uses `parent` for super-class dispatch (not `super()`). + isSuperReceiver: (text) => text.trim() === 'parent', + + // PHP is dynamically typed — field-fallback heuristic on so that + // method calls on `mixed`-typed receivers (no annotation) fall back + // to a workspace-wide name search rather than silently dropping the edge. + fieldFallbackOnMethodLookup: true, + + // PHP: allow free-call fallback to unique workspace-wide callable when + // lexical/import bindings miss. Needed for two cases: + // 1. `use function` imports where PSR-4 directory resolution is + // non-deterministic (multiple .php files in same namespace dir). + // 2. Unimported free calls within the same namespace (same-namespace + // visibility without an explicit use statement, e.g. test fixtures). + allowGlobalFreeCallFallback: true, + + // Return-type propagation on — PHP method signatures are authoritative + // enough for cross-file chain-follow. + propagatesReturnTypesAcrossImports: true, + + // PHP hoists method return-type bindings to the Module scope so + // `propagateImportedReturnTypes` can pick them up across files. + hoistTypeBindingsToModule: true, + + // PHP recovers member calls on `mixed`/untyped receivers via a + // workspace-wide unique-method-name lookup, mirroring the legacy DAG. + emitUnresolvedReceiverEdges: phpEmitUnresolvedReceiverEdges, +}; + +export { phpScopeResolver }; diff --git a/gitnexus/src/core/ingestion/languages/php/simple-hooks.ts b/gitnexus/src/core/ingestion/languages/php/simple-hooks.ts new file mode 100644 index 000000000..89a9a2658 --- /dev/null +++ b/gitnexus/src/core/ingestion/languages/php/simple-hooks.ts @@ -0,0 +1,134 @@ +/** + * Trivial / no-op-ish hooks for the PHP provider. Made explicit so + * reviewers don't have to re-derive the analysis from "absence == default". + */ + +import type { + CaptureMatch, + ParsedImport, + Scope, + ScopeId, + ScopeTree, + TypeRef, +} from 'gitnexus-shared'; + +// ─── bindingScopeFor ────────────────────────────────────────────────────── + +/** + * PHP method return-type bindings (`@type-binding.return`) must hoist + * to the enclosing Module scope so `propagateImportedReturnTypes` can + * mirror them across files. Without this hoist, the return binding gets + * stuck at the Class scope and is invisible to the cross-file propagation + * pass that reads only `sourceModule.typeBindings`. + * + * All other bindings delegate to the default "innermost scope" rule. + */ +export function phpBindingScopeFor( + decl: CaptureMatch, + innermost: Scope, + tree: ScopeTree, +): ScopeId | null { + if (decl['@type-binding.return'] !== undefined) { + let cur: Scope | undefined = innermost; + while (cur !== undefined && cur.kind !== 'Module') { + const parentId: ScopeId | null = cur.parent ?? null; + if (parentId === null) break; + cur = tree.getScope(parentId); + } + if (cur !== undefined && cur.kind === 'Module') return cur.id; + } + + // Constructor-promoted properties (`function __construct(public User $u)`) + // are declared inside the constructor's Function scope in the AST, but they + // are class-owned fields. Hoist the @declaration.property binding to the + // enclosing Class scope so `populateClassOwnedMembers` assigns the correct + // ownerId and `findOwnedMember` can resolve `$obj->u`. + if (decl['@declaration.property'] !== undefined && innermost.kind === 'Function') { + let cur: Scope | undefined = innermost; + while (cur !== undefined && cur.kind !== 'Class') { + const parentId: ScopeId | null = cur.parent ?? null; + if (parentId === null) break; + cur = tree.getScope(parentId); + } + if (cur !== undefined && cur.kind === 'Class') return cur.id; + } + + // Constructor-promoted property TYPE BINDING (`function __construct(public Address $address)`) + // produces both a @type-binding.parameter (stays in Function scope for `$address` lookups + // inside the constructor body) AND a @type-binding.annotation (query.ts). The annotation + // capture is emitted so this hoist branch can place `address → Address` in the CLASS scope. + // + // The compound-receiver resolver (`resolveCompoundReceiverClass`) reads typeBindings from + // the class scope: `cs.typeBindings.get('address')`. Without hoisting, `$user->address->save()` + // fails to resolve `address` because the type binding is in the constructor's Function scope. + // + // `@type-binding.annotation` for a promoted param appears with innermost = Function scope + // (the constructor). Regular typed class properties (`private Address $addr;`) have their + // annotation already in the Class scope, so this branch only fires for promoted params. + if (decl['@type-binding.annotation'] !== undefined && innermost.kind === 'Function') { + let cur: Scope | undefined = innermost; + while (cur !== undefined && cur.kind !== 'Class') { + const parentId: ScopeId | null = cur.parent ?? null; + if (parentId === null) break; + cur = tree.getScope(parentId); + } + if (cur !== undefined && cur.kind === 'Class') return cur.id; + } + + return null; +} + +// ─── importOwningScope ──────────────────────────────────────────────────── + +/** + * Determine which scope owns a `use` import declaration. + * + * - `use` inside `namespace Foo { }` → attach to that Namespace scope. + * - Top-level `use` (no enclosing namespace) → innermost (Module). + * - `use TraitName;` inside a class body → this is a trait-use + * (heritage), NOT a namespace import. The grammar emits + * `use_declaration` for trait-use (distinct from + * `namespace_use_declaration`). Our query only captures + * `namespace_use_declaration`, so trait-use never reaches this hook + * in practice. Returning `null` here is a safety fallback. + */ +export function phpImportOwningScope( + _imp: ParsedImport, + innermost: Scope, + _tree: ScopeTree, +): ScopeId | null { + // Namespace-scoped or module-scoped imports attach to the innermost scope + // (either Namespace or Module). Class-scoped imports should not occur for + // namespace_use_declaration; if they do, attach to the class scope. + if ( + innermost.kind === 'Namespace' || + innermost.kind === 'Module' || + innermost.kind === 'Class' || + innermost.kind === 'Function' + ) { + return innermost.id; + } + return null; +} + +// ─── receiverBinding ────────────────────────────────────────────────────── + +/** + * Look up `$this` or `parent` in the function scope's type bindings. + * + * Both are synthesized as `@type-binding.self` captures during capture + * emission (`receiver-binding.ts`) — `$this` for every non-static + * method inside a class/trait/interface/enum body, `parent` additionally + * for class methods with an explicit `base_clause`. + * + * Returns `null` for: + * - static methods (no `$this` synthesized) + * - free functions (no enclosing class) + * - non-Function scopes + */ +export function phpReceiverBinding(functionScope: Scope): TypeRef | null { + if (functionScope.kind !== 'Function') return null; + return ( + functionScope.typeBindings.get('$this') ?? functionScope.typeBindings.get('parent') ?? null + ); +} diff --git a/gitnexus/src/core/ingestion/registry-primary-flag.ts b/gitnexus/src/core/ingestion/registry-primary-flag.ts index e063ce20c..6818007a3 100644 --- a/gitnexus/src/core/ingestion/registry-primary-flag.ts +++ b/gitnexus/src/core/ingestion/registry-primary-flag.ts @@ -72,6 +72,7 @@ export const MIGRATED_LANGUAGES: ReadonlySet = new Set; + /** + * Optional parallel MRO that EXCLUDES mixin-like augmentation (e.g., PHP + * traits). Returns the inheritance-only ancestor chain — the same kind + * of map as `buildMro` but built only from inheritance edges (EXTENDS). + * + * Used by the shared super-branch dispatch in `receiver-bound-calls` + * so that `parent::method()` walks the inheritance chain only, not the + * trait-augmented one. PHP semantics: `parent::` explicitly bypasses + * traits, even when a composed trait shadows a same-named parent method. + * + * Languages without mixin-like semantics leave this undefined — callers + * fall back to `buildMro`/`mroFor`, which for those languages is already + * the inheritance chain. + */ + readonly buildExtendsOnlyMro?: ( + graph: KnowledgeGraph, + parsedFiles: readonly ParsedFile[], + nodeLookup: GraphNodeLookup, + ) => Map; + /** * Mutate `parsed.localDefs[i].ownerId` to point at the structural * owner. Python's rule: methods (Function defs whose parent scope @@ -484,6 +504,26 @@ export interface ScopeResolver { */ readonly isFileLocalDef?: (def: SymbolDefinition) => boolean; + /** + * Optional predicate to gate free-call fallback emission by caller-side + * visibility. When provided, `pickUniqueGlobalCallable` rejects candidates + * the caller cannot legally reach — e.g., a PHP function in a different + * namespace with no `use function` import, which PHP runtime would treat + * as `Call to undefined function`. Returning `false` blocks the candidate; + * returning `true` allows it; undefined-default keeps current behavior + * (no visibility filtering, equivalent to "all candidates visible"). + * + * The hook receives the caller's `ParsedFile` (so it can consult + * `parsedImports`, `moduleScope`, etc.) and the candidate `SymbolDefinition`. + * The predicate must be pure: same inputs → same answer. + * + * Languages without namespace-scoped function resolution leave this undefined. + */ + readonly isCallableVisibleFromCaller?: (ctx: { + readonly callerParsed: ParsedFile; + readonly candidate: SymbolDefinition; + }) => boolean; + /** * Optional post-finalize hook to inject cross-file bindings that * aren't modeled via explicit imports. Runs after @@ -576,4 +616,32 @@ export interface ScopeResolver { readonly treeCache?: { get(filePath: string): unknown }; }, ) => void; + + /** + * Optional post-resolution pass: emit CALLS edges for member-call sites + * whose receiver cannot be typed by the scope chain (no `TypeRef`). + * Dynamically-typed languages with untyped/`mixed`/`Any` parameters use + * this hook to recover the call edge via workspace-wide method-name + * lookup, mirroring what their legacy resolvers did. + * + * Runs AFTER `emitReceiverBoundCalls` and BEFORE `emitFreeCallFallback`. + * Implementations MUST: + * - Skip sites already in `handledSites` (Invariant I2). + * - Add resolved site keys to `handledSites` before returning. + * - Stay narrow: a unique workspace-wide match is the safe baseline. + * Multi-candidate fallbacks should narrow by arity / argument types + * before emitting to keep false-positive rate bounded. + * + * Returns the number of edges emitted (for telemetry). + * + * Default: undefined (no unresolved-receiver fallback). + */ + readonly emitUnresolvedReceiverEdges?: ( + graph: KnowledgeGraph, + scopes: ScopeResolutionIndexes, + parsedFiles: readonly ParsedFile[], + nodeLookup: GraphNodeLookup, + handledSites: Set, + model: SemanticModel, + ) => number; } diff --git a/gitnexus/src/core/ingestion/scope-resolution/graph-bridge/method-dispatch.ts b/gitnexus/src/core/ingestion/scope-resolution/graph-bridge/method-dispatch.ts index 164147ac6..419ab478f 100644 --- a/gitnexus/src/core/ingestion/scope-resolution/graph-bridge/method-dispatch.ts +++ b/gitnexus/src/core/ingestion/scope-resolution/graph-bridge/method-dispatch.ts @@ -22,8 +22,9 @@ const EMPTY_DEFS: readonly string[] = Object.freeze([]); export function buildPopulatedMethodDispatch( mroByDefId: ReadonlyMap, + extendsOnlyMroByDefId?: ReadonlyMap, ): MethodDispatchIndex { - return { + const base: MethodDispatchIndex = { mroByOwnerDefId: mroByDefId, implsByInterfaceDefId: new Map(), mroFor(ownerDefId) { @@ -33,4 +34,14 @@ export function buildPopulatedMethodDispatch( return EMPTY_DEFS; }, }; + if (extendsOnlyMroByDefId !== undefined) { + return { + ...base, + extendsOnlyMroByOwnerDefId: extendsOnlyMroByDefId, + extendsOnlyMroFor(ownerDefId) { + return extendsOnlyMroByDefId.get(ownerDefId) ?? EMPTY_DEFS; + }, + }; + } + return base; } diff --git a/gitnexus/src/core/ingestion/scope-resolution/graph-bridge/node-lookup.ts b/gitnexus/src/core/ingestion/scope-resolution/graph-bridge/node-lookup.ts index 70aa875fb..c3b53c6f7 100644 --- a/gitnexus/src/core/ingestion/scope-resolution/graph-bridge/node-lookup.ts +++ b/gitnexus/src/core/ingestion/scope-resolution/graph-bridge/node-lookup.ts @@ -117,6 +117,11 @@ export function isLinkableLabel(label: NodeLabel): boolean { label === 'Interface' || label === 'Struct' || label === 'Enum' || + // Trait nodes are linkable so MRO builders can bridge PHP/Rust trait + // defs between scope-resolution DefIds and the graph's node ids. + // IMPLEMENTS edges from classes to traits are otherwise invisible to + // the scope-resolution MRO pass. + label === 'Trait' || // Variable / Property are linkable too — receiver-bound write/read // ACCESSES edges target field nodes (e.g. `user.name = "x"` → // ACCESSES edge to User's `name` Variable/Property node). diff --git a/gitnexus/src/core/ingestion/scope-resolution/passes/free-call-fallback.ts b/gitnexus/src/core/ingestion/scope-resolution/passes/free-call-fallback.ts index 44f941daf..eb6dd71fb 100644 --- a/gitnexus/src/core/ingestion/scope-resolution/passes/free-call-fallback.ts +++ b/gitnexus/src/core/ingestion/scope-resolution/passes/free-call-fallback.ts @@ -39,6 +39,10 @@ export function emitFreeCallFallback( options: { readonly allowGlobalFallback?: boolean; readonly isFileLocalDef?: (def: SymbolDefinition) => boolean; + readonly isCallableVisibleFromCaller?: (ctx: { + readonly callerParsed: ParsedFile; + readonly candidate: SymbolDefinition; + }) => boolean; } = {}, ): number { let emitted = 0; @@ -82,6 +86,11 @@ export function emitFreeCallFallback( scopes, parsed.filePath, options.isFileLocalDef, + site.arity, + options.isCallableVisibleFromCaller !== undefined + ? (candidate) => + options.isCallableVisibleFromCaller!({ callerParsed: parsed, candidate }) + : undefined, ); } if (fnDef === undefined) continue; @@ -118,6 +127,8 @@ function pickUniqueGlobalCallable( scopes: ScopeResolutionIndexes, callerFilePath: string, isFileLocalDef?: (def: SymbolDefinition) => boolean, + callArity?: number, + isCallerVisible?: (candidate: SymbolDefinition) => boolean, ): SymbolDefinition | undefined { const scopeDefs: SymbolDefinition[] = []; const scopeSeen = new Set(); @@ -130,6 +141,13 @@ function pickUniqueGlobalCallable( if (isFileLocalDef !== undefined && def.filePath !== callerFilePath && isFileLocalDef(def)) { continue; } + // Caller-side visibility filter (e.g., PHP namespace + use-function + // import gating). When defined, blocks candidates the caller cannot + // legally reach. Languages without namespace-scoped function resolution + // leave this undefined → no filtering. + if (isCallerVisible !== undefined && !isCallerVisible(def)) { + continue; + } const key = logicalCallableKey(def); if (scopeSeen.has(key)) continue; scopeSeen.add(key); @@ -137,6 +155,15 @@ function pickUniqueGlobalCallable( } if (scopeDefs.length === 1) return scopeDefs[0]; + // When multiple scope-index candidates exist, attempt arity narrowing + // before falling back to the semantic-model lookup. This handles + // registry-primary languages where the model is not populated for the + // migrated language's files (call-processor skips them). + if (scopeDefs.length > 1 && callArity !== undefined) { + const arityMatch = narrowByArity(scopeDefs, callArity); + if (arityMatch !== undefined) return arityMatch; + } + const defs: SymbolDefinition[] = []; const seen = new Set(); const push = (pool: readonly SymbolDefinition[]): void => { @@ -147,6 +174,10 @@ function pickUniqueGlobalCallable( if (isFileLocalDef !== undefined && def.filePath !== callerFilePath && isFileLocalDef(def)) { continue; } + // Same caller-visibility filter applied to the model-side pool. + if (isCallerVisible !== undefined && !isCallerVisible(def)) { + continue; + } const key = logicalCallableKey(def); if (seen.has(key)) continue; seen.add(key); @@ -157,7 +188,35 @@ function pickUniqueGlobalCallable( push(model.symbols.lookupCallableByName(name)); push(model.methods.lookupMethodByName(name)); - return defs.length === 1 ? defs[0] : undefined; + if (defs.length === 1) return defs[0]; + + // When multiple candidates exist and the call site has a known arity, + // narrow by parameter count. + if (defs.length > 1 && callArity !== undefined) { + const arityMatch = narrowByArity(defs, callArity); + if (arityMatch !== undefined) return arityMatch; + } + + return undefined; +} + +/** + * Narrow a list of callable candidates by call-site arity. + * A def is compatible when `requiredParameterCount <= arity <= parameterCount`. + * Defs with `parameterCount === undefined` (variadic/unknown) are always kept. + * Returns the single compatible def, or `undefined` when zero or multiple match. + */ +function narrowByArity( + defs: readonly SymbolDefinition[], + callArity: number, +): SymbolDefinition | undefined { + const compatible = defs.filter((d) => { + const total = d.parameterCount; + if (total === undefined) return true; // unknown arity — keep + const required = d.requiredParameterCount ?? total; + return required <= callArity && callArity <= total; + }); + return compatible.length === 1 ? compatible[0] : undefined; } function logicalCallableKey(def: SymbolDefinition): string { @@ -189,10 +248,18 @@ function pickConstructorOrClass( /** Walk up from the call-site scope to the enclosing class scope, * pick a method member by name with overload narrowing on arity + - * argument types. Returns undefined if there's no enclosing class - * or no matching method. Used for implicit-this calls inside a - * class body where multiple overloads share the call name. */ -function pickImplicitThisOverload( + * argument types. Returns undefined if there's no enclosing class, + * no matching method, OR narrowing leaves multiple compatible + * candidates — in the multi-candidate case, picking + * `candidates[0]` would emit a high-confidence CALLS edge whose + * target depends on registration order rather than a defensible + * resolution. Mirrors `pickUniqueGlobalCallable`'s uniqueness check + * in the same file (Codex PR #1497 review, finding 2). + * + * Exported for unit testing — language-agnostic logic, exercised + * via synthetic stubs in `pick-implicit-this-overload.test.ts`. The + * production call site is `applyFreeCallFallback` immediately above. */ +export function pickImplicitThisOverload( site: { readonly inScope: ScopeId; readonly name: string; @@ -225,6 +292,11 @@ function pickImplicitThisOverload( if (overloads.length === 0) return undefined; if (overloads.length === 1) return overloads[0]; + // Narrow on arity + argument types. Require a UNIQUE survivor — + // ambiguous narrowing (multiple compatible candidates with no + // disambiguating signal) leaves the call unresolved rather than + // routing to an arbitrary first overload by registration order. const candidates = narrowOverloadCandidates(overloads, site.arity, site.argumentTypes); + if (candidates.length !== 1) return undefined; return candidates[0]; } diff --git a/gitnexus/src/core/ingestion/scope-resolution/passes/overload-narrowing.ts b/gitnexus/src/core/ingestion/scope-resolution/passes/overload-narrowing.ts index 922afb36c..f36287052 100644 --- a/gitnexus/src/core/ingestion/scope-resolution/passes/overload-narrowing.ts +++ b/gitnexus/src/core/ingestion/scope-resolution/passes/overload-narrowing.ts @@ -13,9 +13,13 @@ * 2. Exact-required-match wins over variadic. Variadic is detected * via a `parameterTypes` entry equal to `'params'` or starting * with `'params '` (C# `params` / variadic marker). - * 3. If the arity filter empties the set, fall back to the full - * overload list rather than returning nothing — the caller still - * needs a best-effort candidate. + * 3. If the arity filter empties the set AND any candidate had + * unknown bounds (both `parameterCount` and `requiredParameterCount` + * undefined), fall back to the full overload list — the empty + * result may be due to missing metadata rather than a real mismatch. + * If EVERY rejected candidate had definite arity bounds, trust the + * filter and return empty — the call is genuinely arity-incompatible + * (e.g., PHP `f(int $req, ...$rest)` called with zero args). * 4. If `argTypes` is present, filter further by per-slot type * equality. An empty string in `argTypes[i]` means "unknown" and * counts as a match. Mismatches disqualify. A non-empty typed @@ -39,6 +43,16 @@ export function narrowOverloadCandidates( const max = d.parameterCount; const min = d.requiredParameterCount; if (max !== undefined && argCount > max) { + // Variadic marker check is C#-specific (the 'params' keyword). + // Other languages use their own marker — PHP uses '...' (see + // `languages/php/arity-metadata.ts:46`), Python uses '*args'- + // shaped metadata that lives outside `parameterTypes` entirely. + // This branch is dead code for those languages because they + // set `parameterCount = undefined` for variadic functions, + // which keeps `max` undefined and skips this check entirely. + // Adding new variadic markers here changes behavior for those + // other languages too — don't extend without auditing each + // adapter's `arity-metadata.ts`. Finding 9 of PR #1497. const variadic = d.parameterTypes !== undefined && d.parameterTypes.some((t) => t === 'params' || t.startsWith('params ')); @@ -48,8 +62,16 @@ export function narrowOverloadCandidates( return true; }); + // When the arity filter empties the set, only fall back to the full + // overload list if some candidate had unknown bounds — otherwise the + // empty result is authoritative (every candidate definitively failed + // arity, e.g., PHP variadic with required-prefix called with too few + // args). + const anyUnknownBounds = overloads.some( + (d) => d.parameterCount === undefined && d.requiredParameterCount === undefined, + ); const candidates: readonly SymbolDefinition[] = - arityMatches.length > 0 ? arityMatches : overloads; + arityMatches.length > 0 ? arityMatches : anyUnknownBounds ? overloads : []; if (argTypes !== undefined && argTypes.length > 0) { const typed = candidates.filter((d) => { diff --git a/gitnexus/src/core/ingestion/scope-resolution/passes/receiver-bound-calls.ts b/gitnexus/src/core/ingestion/scope-resolution/passes/receiver-bound-calls.ts index 925151b6c..0cb544db9 100644 --- a/gitnexus/src/core/ingestion/scope-resolution/passes/receiver-bound-calls.ts +++ b/gitnexus/src/core/ingestion/scope-resolution/passes/receiver-bound-calls.ts @@ -165,7 +165,16 @@ export function emitReceiverBoundCalls( if (provider.isSuperReceiver(receiverName)) { const enclosingClass = findEnclosingClassDef(site.inScope, scopes); if (enclosingClass !== undefined) { - const ancestors = scopes.methodDispatch.mroFor(enclosingClass.nodeId); + // For super-receiver dispatch (`parent::`, `base.`, `super()`), + // walk the inheritance-only ancestor chain when the language + // exposes it. PHP's `parent::` semantically bypasses composed + // traits; other languages without mixin augmentation have no + // `extendsOnlyMroFor` and fall back to `mroFor`. + const extendsOnly = scopes.methodDispatch.extendsOnlyMroFor; + const ancestors = + extendsOnly !== undefined + ? extendsOnly(enclosingClass.nodeId) + : scopes.methodDispatch.mroFor(enclosingClass.nodeId); let memberDef: SymbolDefinition | undefined; for (const ownerId of ancestors) { memberDef = findOwnedMember(ownerId, memberName, model); @@ -283,7 +292,21 @@ export function emitReceiverBoundCalls( let memberDef: SymbolDefinition | undefined; for (const ownerId of chain) { memberDef = findOwnedMember(ownerId, memberName, model); - if (memberDef !== undefined) break; + if (memberDef !== undefined) { + // The MRO chain is most-derived-first ([classDef, ...ancestors]). + // If the most-derived definition is arity-incompatible with the + // call site, PHP throws ArgumentCountError at runtime — it does + // NOT silently dispatch to an ancestor. Terminate the chain walk + // so no edge is emitted, rather than falling through to an + // arity-compatible ancestor (which would be a false positive). + if ( + narrowOverloadCandidates([memberDef], site.arity, site.argumentTypes).length === 0 + ) { + memberDef = undefined; + break; + } + break; + } } if (memberDef !== undefined) { const reason = diff --git a/gitnexus/src/core/ingestion/scope-resolution/pipeline/registry.ts b/gitnexus/src/core/ingestion/scope-resolution/pipeline/registry.ts index ecd7b69aa..c606661c8 100644 --- a/gitnexus/src/core/ingestion/scope-resolution/pipeline/registry.ts +++ b/gitnexus/src/core/ingestion/scope-resolution/pipeline/registry.ts @@ -17,6 +17,7 @@ import { typescriptScopeResolver } from '../../languages/typescript/scope-resolv import { goScopeResolver } from '../../languages/go/scope-resolver.js'; import { javaScopeResolver } from '../../languages/java/scope-resolver.js'; import { cScopeResolver } from '../../languages/c/scope-resolver.js'; +import { phpScopeResolver } from '../../languages/php/scope-resolver.js'; /** Map of `SupportedLanguages` → `ScopeResolver`. The phase iterates * this map intersected with `MIGRATED_LANGUAGES` (the per-language @@ -32,4 +33,5 @@ export const SCOPE_RESOLVERS: ReadonlyMap = n [SupportedLanguages.Go, goScopeResolver], [SupportedLanguages.Java, javaScopeResolver], [SupportedLanguages.C, cScopeResolver], + [SupportedLanguages.PHP, phpScopeResolver], ]); diff --git a/gitnexus/src/core/ingestion/scope-resolution/pipeline/run.ts b/gitnexus/src/core/ingestion/scope-resolution/pipeline/run.ts index 31a58cdfa..47f8a3551 100644 --- a/gitnexus/src/core/ingestion/scope-resolution/pipeline/run.ts +++ b/gitnexus/src/core/ingestion/scope-resolution/pipeline/run.ts @@ -184,6 +184,7 @@ export function runScopeResolution( const allFilePaths = new Set(parsedFiles.map((f) => f.filePath)); const nodeLookup = buildGraphNodeLookup(graph); const mroByClassDefId = provider.buildMro(graph, parsedFiles, nodeLookup); + const extendsOnlyMroByClassDefId = provider.buildExtendsOnlyMro?.(graph, parsedFiles, nodeLookup); const resolutionConfig = input.resolutionConfig; const finalized = finalizeScopeModel(parsedFiles, { @@ -205,7 +206,7 @@ export function runScopeResolution( // the type system. const indexes = { ...finalized, - methodDispatch: buildPopulatedMethodDispatch(mroByClassDefId), + methodDispatch: buildPopulatedMethodDispatch(mroByClassDefId, extendsOnlyMroByClassDefId), }; // Build the workspace resolution index ONCE — scope-valued lookups @@ -283,6 +284,17 @@ export function runScopeResolution( workspaceIndex, readonlyModel, ); + const unresolvedReceiverExtras = + provider.emitUnresolvedReceiverEdges !== undefined + ? provider.emitUnresolvedReceiverEdges( + graph, + indexes, + parsedFiles, + nodeLookup, + handledSites, + readonlyModel, + ) + : 0; const freeCallExtras = emitFreeCallFallback( graph, indexes, @@ -295,6 +307,7 @@ export function runScopeResolution( { allowGlobalFallback: provider.allowGlobalFreeCallFallback === true, isFileLocalDef: provider.isFileLocalDef, + isCallableVisibleFromCaller: provider.isCallableVisibleFromCaller, }, ); const { emitted, skipped } = emitReferencesViaLookup( @@ -330,7 +343,7 @@ export function runScopeResolution( filesSkipped, importsEmitted, resolve: resolveStats, - referenceEdgesEmitted: emitted + receiverExtras + freeCallExtras, + referenceEdgesEmitted: emitted + receiverExtras + unresolvedReceiverExtras + freeCallExtras, referenceSkipped: skipped, }; } diff --git a/gitnexus/src/core/ingestion/tree-sitter-queries.ts b/gitnexus/src/core/ingestion/tree-sitter-queries.ts index 8e165a837..d65229808 100644 --- a/gitnexus/src/core/ingestion/tree-sitter-queries.ts +++ b/gitnexus/src/core/ingestion/tree-sitter-queries.ts @@ -1020,6 +1020,16 @@ export const PHP_QUERIES = ` (use_declaration [(name) (qualified_name)] @heritage.trait))) @heritage +; ── Heritage: trait uses another trait (transitive trait composition) ──────── +; PHP allows a trait body to contain "use OtherTrait;". The trait-uses-trait +; IMPLEMENTS edge is required by buildPhpMro to compute the full transitive +; trait closure (depth 3+ chains). +(trait_declaration + name: (name) @heritage.class + body: (declaration_list + (use_declaration + [(name) (qualified_name)] @heritage.trait))) @heritage + ; PHP HTTP consumers: file_get_contents('/path'), curl_init('/path') (function_call_expression function: (name) @_php_http (#match? @_php_http "^(file_get_contents|curl_init)$") diff --git a/gitnexus/test/fixtures/cross-file-binding/php-consumer-before-provider/app/BProvider.php b/gitnexus/test/fixtures/cross-file-binding/php-consumer-before-provider/app/Models/User.php similarity index 100% rename from gitnexus/test/fixtures/cross-file-binding/php-consumer-before-provider/app/BProvider.php rename to gitnexus/test/fixtures/cross-file-binding/php-consumer-before-provider/app/Models/User.php diff --git a/gitnexus/test/fixtures/lang-resolution/php-calls/app/Services/UserService.php b/gitnexus/test/fixtures/lang-resolution/php-calls/app/Services/UserService.php index fa0a1b8be..882e12d64 100644 --- a/gitnexus/test/fixtures/lang-resolution/php-calls/app/Services/UserService.php +++ b/gitnexus/test/fixtures/lang-resolution/php-calls/app/Services/UserService.php @@ -2,10 +2,13 @@ namespace App\Services; -use function App\Utils\OneArg\log; -use function App\Utils\ZeroArg\log as zero_log; +use function App\Utils\OneArg\write_audit; +use function App\Utils\ZeroArg\write_audit as zero_write_audit; function create_user(): string { + // Two visible write_audit candidates (different arities). Arity narrowing + // must pick the 1-arg OneArg version. This validates that visibility + + // arity together correctly disambiguate. return write_audit('hello'); } diff --git a/gitnexus/test/fixtures/lang-resolution/php-dynamic-calls/app/Services/Dynamic.php b/gitnexus/test/fixtures/lang-resolution/php-dynamic-calls/app/Services/Dynamic.php new file mode 100644 index 000000000..4904f3c24 --- /dev/null +++ b/gitnexus/test/fixtures/lang-resolution/php-dynamic-calls/app/Services/Dynamic.php @@ -0,0 +1,113 @@ +$method() — dynamic method name via variable_name node. + // Query pattern requires `name: (name)` so this is not captured. + $method = 'dynamicProcess'; + $obj->$method(); + } + + public function memberCallBraceDynamicName(Targets $obj): void + { + // $obj->{$method}() — brace-syntax variant of the above. + $method = 'dynamicBrace'; + $obj->{$method}(); + } + + public function scopedCallDynamicMethodName(): void + { + // ClassName::$method() — dynamic method name on static dispatch. + $method = 'dynamicHandle'; + Targets::$method(); + } + + public function scopedCallVariableClassNameStaticMethod($className): void + { + // $className::method() — class-name is an untyped parameter (no + // type hint, no string-literal assignment that could be picked up + // by a future type-binding heuristic). Receiver IS captured but + // resolution falls through because $className has no class type + // binding in scope. The unresolved-receiver fallback also doesn't + // fire because `dynamicStaticMethod` is unique workspace-wide AND + // exact-arity narrowing in U4 would still match — meaning the + // ONLY thing keeping the edge count at zero today is the absence + // of any type binding for the receiver. + $className::dynamicStaticMethod(); + } + + public function scopedCallDynamicClassAndMethodName(): void + { + // $className::$method() — both dynamic. + $className = 'App\\Services\\Targets'; + $method = 'dynamicScopedDynName'; + $className::$method(); + } + + public function callUserFuncVariableCallable($callable): void + { + // call_user_func($callable, ...) — resolver is structural-only + // and never inspects argument values to infer the callable. + // The literal `call_user_func` itself is an unresolved built-in. + call_user_func($callable); + } + + public function callUserFuncArrayVariable($callable, $args): void + { + // call_user_func_array($callable, $args) — unknown-arity variant. + call_user_func_array($callable, $args); + } + + public function callUserFuncStringCallable(): void + { + // 'Class::method' string-callable form — argument is a string + // literal, never reaches the function: child of function_call_expression. + call_user_func('App\\Services\\Targets::dynamicCallableMethod'); + } + + public function callUserFuncArrayObjectCallable(Targets $obj): void + { + // [$obj, 'method'] array-callable form — array is an argument + // value, not the function: child. + call_user_func([$obj, 'dynamicArrayCallableMethod']); + } + + public function callUserFuncArrayClassNameCallable(): void + { + // ['Class', 'method'] array-callable with class-name string. + call_user_func(['App\\Services\\Targets', 'dynamicArrayClassCallableMethod']); + } + + public function dynamicPropertyRead(Targets $obj): string + { + // $obj->$prop — dynamic property read. No read-access property + // capture pattern exists in query.ts at all (Finding 2). + $prop = 'dynamicProp'; + return $obj->$prop; + } + + public function sanityStaticCall(Targets $obj): void + { + // The fixture's deliberate sanity-check call. THIS one DOES emit + // a CALLS edge — if the assertion that this edge exists ever + // fails, the test infra is broken, not the dynamic-dispatch + // suppression. Without this, every zero-edge assertion above + // would pass even if the pipeline never emitted any edges at all. + $obj->sanityStaticallyNamedTarget(); + } +} diff --git a/gitnexus/test/fixtures/lang-resolution/php-dynamic-calls/app/Services/OtherTargets.php b/gitnexus/test/fixtures/lang-resolution/php-dynamic-calls/app/Services/OtherTargets.php new file mode 100644 index 000000000..c8b70a20d --- /dev/null +++ b/gitnexus/test/fixtures/lang-resolution/php-dynamic-calls/app/Services/OtherTargets.php @@ -0,0 +1,16 @@ +record()` MUST resolve to app/Other/User.php::record, +// NOT app/Models/User.php::record. The `saveLocal` method exercises the +// simple-name path as a control — `User` here is the imported App\Models\User. +class Service { + public function save(\App\Other\User $u): void { + $u->record(); + } + + public function saveLocal(User $u): void { + $u->record(); + } +} diff --git a/gitnexus/test/fixtures/lang-resolution/php-fqn-cross-namespace/composer.json b/gitnexus/test/fixtures/lang-resolution/php-fqn-cross-namespace/composer.json new file mode 100644 index 000000000..386b0bd2d --- /dev/null +++ b/gitnexus/test/fixtures/lang-resolution/php-fqn-cross-namespace/composer.json @@ -0,0 +1,7 @@ +{ + "autoload": { + "psr-4": { + "App\\": "app/" + } + } +} diff --git a/gitnexus/test/fixtures/lang-resolution/php-mro-arity-mismatch/app/Models/ChildModel.php b/gitnexus/test/fixtures/lang-resolution/php-mro-arity-mismatch/app/Models/ChildModel.php new file mode 100644 index 000000000..0dff935c9 --- /dev/null +++ b/gitnexus/test/fixtures/lang-resolution/php-mro-arity-mismatch/app/Models/ChildModel.php @@ -0,0 +1,10 @@ + hits Case 2 (findClassBindingInScope) in + // receiver-bound-calls.ts. + // Pre-fix bug: MRO walk emits a false CALLS edge to ParentModel::method + // because Case 2 used `continue` on arity mismatch and fell through. + // Post-fix: zero edges (PHP throws ArgumentCountError at runtime). + ChildModel::method(1); + } + + public function callCompatible(): void + { + // Happy path: ChildModel::compat takes 1 arg; matches call site. + ChildModel::compat(1); + } + + public function callNoParent(): void + { + // Orphan::method takes 2 args; called with 1; no parent class exists. + // Pre-fix: same Case 2 bug — the loop exhausts with memberDef cleared, + // BUT with `continue` the loop simply ends after one iteration since + // the chain has only one entry; no edge would have been emitted here + // even pre-fix. Post-fix: same — zero edges. Documents the boundary. + Orphan::method(1); + } + + public function callMostDerivedHappy(): void + { + // Happy path: ChildModel::method takes 2 args; matches call site. + ChildModel::method(1, 2); + } +} diff --git a/gitnexus/test/fixtures/lang-resolution/php-mro-arity-mismatch/composer.json b/gitnexus/test/fixtures/lang-resolution/php-mro-arity-mismatch/composer.json new file mode 100644 index 000000000..60ede80e5 --- /dev/null +++ b/gitnexus/test/fixtures/lang-resolution/php-mro-arity-mismatch/composer.json @@ -0,0 +1,5 @@ +{ + "autoload": { + "psr-4": { "App\\": "app/" } + } +} diff --git a/gitnexus/test/fixtures/lang-resolution/php-namespace-fallback-isolation/composer.json b/gitnexus/test/fixtures/lang-resolution/php-namespace-fallback-isolation/composer.json new file mode 100644 index 000000000..3675b0d1c --- /dev/null +++ b/gitnexus/test/fixtures/lang-resolution/php-namespace-fallback-isolation/composer.json @@ -0,0 +1,8 @@ +{ + "autoload": { + "psr-4": { + "App\\": "src/App/", + "Vendor\\": "src/Vendor/" + } + } +} diff --git a/gitnexus/test/fixtures/lang-resolution/php-namespace-fallback-isolation/src/App/Caller.php b/gitnexus/test/fixtures/lang-resolution/php-namespace-fallback-isolation/src/App/Caller.php new file mode 100644 index 000000000..6fecabb3c --- /dev/null +++ b/gitnexus/test/fixtures/lang-resolution/php-namespace-fallback-isolation/src/App/Caller.php @@ -0,0 +1,19 @@ +record(); + } +} diff --git a/gitnexus/test/fixtures/lang-resolution/php-parent-vs-trait/composer.json b/gitnexus/test/fixtures/lang-resolution/php-parent-vs-trait/composer.json new file mode 100644 index 000000000..386b0bd2d --- /dev/null +++ b/gitnexus/test/fixtures/lang-resolution/php-parent-vs-trait/composer.json @@ -0,0 +1,7 @@ +{ + "autoload": { + "psr-4": { + "App\\": "app/" + } + } +} diff --git a/gitnexus/test/fixtures/lang-resolution/php-transitive-traits/app/Models/Consumer.php b/gitnexus/test/fixtures/lang-resolution/php-transitive-traits/app/Models/Consumer.php new file mode 100644 index 000000000..86d330173 --- /dev/null +++ b/gitnexus/test/fixtures/lang-resolution/php-transitive-traits/app/Models/Consumer.php @@ -0,0 +1,20 @@ +aMethod(); + } + + public function callDepthTwo(): string { + return $this->bMethod(); + } + + public function callDepthThree(): string { + return $this->deepMethod(); + } +} diff --git a/gitnexus/test/fixtures/lang-resolution/php-transitive-traits/app/Traits/TraitA.php b/gitnexus/test/fixtures/lang-resolution/php-transitive-traits/app/Traits/TraitA.php new file mode 100644 index 000000000..4408b8dfb --- /dev/null +++ b/gitnexus/test/fixtures/lang-resolution/php-transitive-traits/app/Traits/TraitA.php @@ -0,0 +1,10 @@ += required → edge emitted + * - variadic candidate, argCount < required → NO edge + */ +class Caller +{ + public function callHappyPath($h): void + { + // happyPath(): min=0. argCount=0 → exact match. Edge. + $h->happyPath(); + } + + public function callDefaultExactRequired($h): void + { + // withDefault($a, $b=0): min=1. argCount=1 === min → exact match. Edge. + $h->withDefault('a'); + } + + public function callDefaultBeyondRequired($h): void + { + // withDefault($a, $b=0): min=1, max=2. argCount=2 > min. + // Pre-fix: first-stage narrow accepts (2 <= 2), edge emitted. + // Post-fix: exact-required gate rejects (2 !== 1), no edge. + $h->withDefault('a', 99); + } + + public function callVariadicAtRequired($h): void + { + // variadicLog($level, ...$args): min=1, hasVarArgs. + // argCount=1 === min → edge emitted (variadic relaxed path). + $h->variadicLog('info'); + } + + public function callVariadicBeyondRequired($h): void + { + // variadicLog($level, ...$args): min=1, hasVarArgs. + // argCount=2 > min, variadic → edge emitted. + $h->variadicLog('info', 'arg1'); + } + + public function callVariadicBelowRequired($h): void + { + // variadicLogTwoRequired($a, $b, ...$rest): min=2, hasVarArgs. + // argCount=1 < min → no edge (first-stage rejects). Both pre/post-fix. + $h->variadicLogTwoRequired('only-one'); + } +} diff --git a/gitnexus/test/fixtures/lang-resolution/php-unresolved-receiver-arity/composer.json b/gitnexus/test/fixtures/lang-resolution/php-unresolved-receiver-arity/composer.json new file mode 100644 index 000000000..60ede80e5 --- /dev/null +++ b/gitnexus/test/fixtures/lang-resolution/php-unresolved-receiver-arity/composer.json @@ -0,0 +1,5 @@ +{ + "autoload": { + "psr-4": { "App\\": "app/" } + } +} diff --git a/gitnexus/test/fixtures/lang-resolution/php-variadic-arity-minimum/app/Services/Caller.php b/gitnexus/test/fixtures/lang-resolution/php-variadic-arity-minimum/app/Services/Caller.php new file mode 100644 index 000000000..1da29a43a --- /dev/null +++ b/gitnexus/test/fixtures/lang-resolution/php-variadic-arity-minimum/app/Services/Caller.php @@ -0,0 +1,30 @@ +method()` precedence inside a class that composes a trait AND + // extends a parent both defining the same method requires the augmented + // trait-aware MRO (trait shadows parent). The legacy DAG has no + // trait-aware MRO, so it fails to bind the call to the trait. Scope- + // resolver-only correctness win (commit af9af4a9 U3). + '$this->record() still resolves to Auditable::record (trait shadows parent)', + // Fully-qualified type-hint resolution (`\App\Other\User $u` parameter) + // routes through the scope-resolver's bindingAugmentations channel + // populated by `populatePhpNamespaceSiblings` Step 3b. The legacy DAG + // resolves receiver types via simple-name workspace lookup and has no + // namespace-prefixed binding channel, so it cannot distinguish the FQN + // target from a same-simple-name class reachable via `use`. Scope- + // resolver-only correctness win (Codex PR #1497 review, finding 1). + '\\App\\Other\\User parameter resolves $u->record() to app/Other/User.php (NOT app/Models/User.php)', + // MRO arity-mismatch on class-name receivers (`Child::method(1)` where + // Child::method takes 2 args and Parent::method takes 1): the legacy + // DAG has no arity narrowing on Case 2 (class-name) MRO walk, so it + // emits a false CALLS edge to Parent::method on fallthrough. Scope- + // resolver-only correctness win (PR #1497 review Image 1 / U1). + 'arity-incompatible most-derived override does NOT fall through to ParentModel::method', + // Class-name receiver with single-class arity mismatch (no parent in + // the MRO chain): legacy resolves the method by name without arity + // gating, so it emits a CALLS edge even when arity is definitively + // incompatible. The scope-resolver's `narrowOverloadCandidates` check + // in `receiver-bound-calls.ts` Case 2 rejects this post-fix. Scope- + // resolver-only correctness win (PR #1497 / U1). + 'arity-incompatible class with no parent emits zero CALLS edges (regression check)', + // `phpEmitUnresolvedReceiverEdges` exact-required-arity gate (PR + // #1497 / U4): the legacy DAG has no equivalent unresolved-receiver + // fallback hook, so it resolves these untyped-receiver sites via a + // different code path that over-emits for default-parameter and + // variadic-required-mismatch shapes. Scope-resolver-only correctness + // wins; backporting to legacy is out of scope. + 'argCount > required (2>1) on candidate with default param emits NO edge post-fix', + 'variadic candidate, argCount < required (1<2) emits NO edge', + ]), python: new Set([ // Suffix-fallback lex tiebreak depends on the registry-primary // resolver's deterministic sort. The legacy resolver returns the diff --git a/gitnexus/test/integration/resolvers/php.test.ts b/gitnexus/test/integration/resolvers/php.test.ts index e120d92f4..336e5dee2 100644 --- a/gitnexus/test/integration/resolvers/php.test.ts +++ b/gitnexus/test/integration/resolvers/php.test.ts @@ -1,11 +1,12 @@ /** * PHP: PSR-4 imports, extends, implements, trait use, enums, calls + ambiguous disambiguation */ -import { describe, it, expect, beforeAll } from 'vitest'; +import { describe, expect, beforeAll } from 'vitest'; import path from 'path'; import { FIXTURES, CROSS_FILE_FIXTURES, + createResolverParityIt, getRelationships, getNodesByLabel, getNodesByLabelFull, @@ -14,6 +15,12 @@ import { type PipelineResult, } from './helpers.js'; +// Wrap vitest's `it` so legacy-DAG-only divergences (commit af9af4a9 U1/U3) +// are skipped under REGISTRY_PRIMARY_PHP=0. The skip list lives in +// helpers.ts:LEGACY_RESOLVER_PARITY_EXPECTED_FAILURES.php — sibling pattern +// to csharp/typescript/python. +const it = createResolverParityIt('php'); + // --------------------------------------------------------------------------- // Heritage: PSR-4 imports, extends, implements, trait use, enums, calls // --------------------------------------------------------------------------- @@ -91,6 +98,10 @@ describe('PHP heritage & import resolution', () => { expect(targets).toContain('label'); }); + // save($entity: mixed) calls $entity->getId() — the receiver is typed `mixed` + // so there is no TypeRef in scope. The scope-resolver `emitUnresolvedReceiverEdges` + // hook (PHP-wired) recovers this case via workspace-wide unique-name lookup, + // matching the legacy DAG behavior. it('emits CALLS edge: save → getId', () => { const calls = getRelationships(result, 'CALLS').filter( (e) => e.source === 'save' && e.target === 'getId', @@ -439,6 +450,161 @@ describe('PHP variadic call resolution', () => { }); }); +// --------------------------------------------------------------------------- +// Variadic arity minimum: required-arg count must be enforced for variadic +// functions. f(int $req, ...$rest) called as f() is an ArgumentCountError at +// PHP runtime and must NOT emit a CALLS edge from the resolver. +// --------------------------------------------------------------------------- + +describe('PHP variadic arity minimum (U1)', () => { + let result: PipelineResult; + + beforeAll(async () => { + result = await runPipelineFromRepo(path.join(FIXTURES, 'php-variadic-arity-minimum'), () => {}); + }, 60000); + + const callsFrom = (source: string, target: string) => + getRelationships(result, 'CALLS').filter((c) => c.source === source && c.target === target); + + it('emits CALLS edge for record(level, ...msgs) with arity 4 (happy path)', () => { + expect(callsFrom('callValidRecord', 'record').length).toBe(1); + }); + + it('emits CALLS edge for record(level) with only the required arg (arity 1)', () => { + expect(callsFrom('callValidRecordMin', 'record').length).toBe(1); + }); + + it('does NOT emit CALLS edge for record() with zero args (below required=1)', () => { + expect(callsFrom('callTooFewRecord', 'record').length).toBe(0); + }); + + it('emits CALLS edge for format() — pure variadic, required=0', () => { + expect(callsFrom('callPureVariadic', 'format').length).toBe(1); + }); + + it('emits CALLS edge for pad("x") — required+optional+variadic, only required given', () => { + expect(callsFrom('callPadMin', 'pad').length).toBe(1); + }); + + it('does NOT emit CALLS edge for pad() with zero args (below required=1)', () => { + expect(callsFrom('callPadTooFew', 'pad').length).toBe(0); + }); +}); + +// --------------------------------------------------------------------------- +// Transitive trait MRO: trait A uses B uses C — Consumer using A must see C's +// methods. Current depth-2 expansion in buildPhpMro silently drops methods +// from 3+ level chains. +// --------------------------------------------------------------------------- + +describe('PHP transitive trait MRO (U2)', () => { + let result: PipelineResult; + + beforeAll(async () => { + result = await runPipelineFromRepo(path.join(FIXTURES, 'php-transitive-traits'), () => {}); + }, 60000); + + const callsFrom = (source: string, target: string) => + getRelationships(result, 'CALLS').filter((c) => c.source === source && c.target === target); + + it('detects 3 traits and 1 class', () => { + expect(getNodesByLabel(result, 'Trait')).toEqual(['TraitA', 'TraitB', 'TraitC']); + expect(getNodesByLabel(result, 'Class')).toContain('Consumer'); + }); + + it('depth-1: $this->aMethod() resolves to TraitA::aMethod', () => { + expect(callsFrom('callDepthOne', 'aMethod').length).toBe(1); + }); + + it('depth-2: $this->bMethod() resolves to TraitB::bMethod (TraitA uses TraitB)', () => { + expect(callsFrom('callDepthTwo', 'bMethod').length).toBe(1); + }); + + it('depth-3: $this->deepMethod() resolves to TraitC::deepMethod (TraitA → TraitB → TraitC)', () => { + expect(callsFrom('callDepthThree', 'deepMethod').length).toBe(1); + }); +}); + +// --------------------------------------------------------------------------- +// parent:: bypasses traits. When a class composes a trait AND extends a parent +// that both define the same method name, parent::method() must resolve to the +// parent class (PHP semantics), NOT the trait. $this->method() still goes to +// the trait (PHP's own-class > trait > parent precedence). +// --------------------------------------------------------------------------- + +describe('PHP parent:: bypasses traits (U3)', () => { + let result: PipelineResult; + + beforeAll(async () => { + result = await runPipelineFromRepo(path.join(FIXTURES, 'php-parent-vs-trait'), () => {}); + }, 60000); + + const callsFromTo = (source: string, target: string, file: string) => + getRelationships(result, 'CALLS').filter( + (c) => c.source === source && c.target === target && c.targetFilePath === file, + ); + + it('parent::record() resolves to Base::record, NOT Auditable::record', () => { + expect(callsFromTo('callViaParent', 'record', 'app/Base.php').length).toBe(1); + expect(callsFromTo('callViaParent', 'record', 'app/Auditable.php').length).toBe(0); + }); + + it('$this->record() still resolves to Auditable::record (trait shadows parent)', () => { + expect(callsFromTo('callViaThis', 'record', 'app/Auditable.php').length).toBe(1); + expect(callsFromTo('callViaThis', 'record', 'app/Base.php').length).toBe(0); + }); +}); + +// --------------------------------------------------------------------------- +// Namespace-aware free-call fallback. PHP's `pickUniqueGlobalCallable` must +// reject cross-namespace candidates that the caller can't reach without an +// explicit `use function` import. Same-namespace and globally-imported calls +// still emit edges. +// --------------------------------------------------------------------------- + +describe('PHP namespace-aware free-call fallback (U4)', () => { + let result: PipelineResult; + + beforeAll(async () => { + result = await runPipelineFromRepo( + path.join(FIXTURES, 'php-namespace-fallback-isolation'), + () => {}, + ); + }, 60000); + + const callsFromTo = (source: string, target: string, file?: string) => + getRelationships(result, 'CALLS').filter( + (c) => + c.source === source && + c.target === target && + (file === undefined || c.targetFilePath === file), + ); + + it('rejects cross-namespace candidate when caller has no use-function import', () => { + // callNoImport (in \App) calls format('x'). Workspace has \App\Utils\format/1 + // and \Vendor\Utils\format/2. Caller is in \App — NOT same namespace as + // either candidate, and no `use function` for `format` is in scope. + // Expected: NO CALLS edge. + expect(callsFromTo('callNoImport', 'format').length).toBe(0); + }); + + it('resolves same-namespace free call (caller in App\\Utils → App\\Utils\\format)', () => { + expect(callsFromTo('callSameNamespace', 'format', 'src/App/Utils/Format.php').length).toBe(1); + }); + + it('resolves use-function-imported alias (vendorFormat → Vendor\\Utils\\format)', () => { + // `use function Vendor\Utils\format as vendorFormat;`. Caller in \App calls + // vendorFormat('x', 80) — the import target is reachable. The CALLS edge + // may surface against either the alias name (`vendorFormat`) or the + // canonical function name (`format` in the vendor file) depending on + // dedup ordering; either way, exactly one edge total. + expect( + callsFromTo('callImported', 'vendorFormat').length + + callsFromTo('callImported', 'format', 'src/Vendor/Utils/Format.php').length, + ).toBe(1); + }); +}); + // --------------------------------------------------------------------------- // Local shadow: same-file definition takes priority over imported name // --------------------------------------------------------------------------- @@ -1807,3 +1973,325 @@ describe('PHP Child extends ParentClass — inherited method resolution (SM-9)', expect(parentMethodCall!.source).toBe('run'); }); }); + +// --------------------------------------------------------------------------- +// Fully-qualified type-hint resolution (Codex PR #1497 review, finding 1). +// +// Two `User` classes coexist in the workspace: `App\Models\User` and +// `App\Other\User`. A service file imports the simple-name `User` from +// App\Models, but uses a fully-qualified `\App\Other\User` in a parameter +// annotation. PHP runtime semantics: the leading `\` is an absolute namespace +// path; the parameter is always `App\Other\User`, even when the simple +// `User` is bound to a different class by `use`. +// +// Pre-fix: `normalizePhpType` strips the qualifier so the TypeRef carries +// only `User`, then `findClassBindingInScope` walks the scope chain and +// resolves to the imported `App\Models\User` — emitting a CALLS edge to the +// wrong class. Post-fix: qualified form survives on `rawName`, the +// QualifiedNameIndex fallback (or a PHP-specific qualified lookup) routes +// the call to App\Other\User::record. +// --------------------------------------------------------------------------- + +describe('PHP fully-qualified type-hint resolution (Codex #1497)', () => { + let result: PipelineResult; + + beforeAll(async () => { + result = await runPipelineFromRepo(path.join(FIXTURES, 'php-fqn-cross-namespace'), () => {}); + }, 60000); + + const callsFromTo = (source: string, target: string, file: string) => + getRelationships(result, 'CALLS').filter( + (c) => c.source === source && c.target === target && c.targetFilePath === file, + ); + + it('detects both User classes in distinct namespaces', () => { + expect(getNodesByLabel(result, 'Class')).toContain('User'); + // Exactly two User entries — one per namespace. + const userClasses = getNodesByLabelFull(result, 'Class').filter((n) => n.name === 'User'); + expect(userClasses.length).toBe(2); + const userFiles = userClasses.map((c) => c.properties.filePath as string).sort(); + expect( + userFiles.some((f) => f.includes('Models/User.php') || f.includes('Models\\User.php')), + ).toBe(true); + expect( + userFiles.some((f) => f.includes('Other/User.php') || f.includes('Other\\User.php')), + ).toBe(true); + }); + + it('\\App\\Other\\User parameter resolves $u->record() to app/Other/User.php (NOT app/Models/User.php)', () => { + // The bug Codex flagged: FQN parameter collapses to simple `User`, then + // resolves to the imported `App\Models\User` instead of the explicit + // `\App\Other\User` named in the annotation. Post-fix: exactly one edge, + // pointing to the FQN target. + expect(callsFromTo('save', 'record', 'app/Other/User.php').length).toBe(1); + expect(callsFromTo('save', 'record', 'app/Models/User.php').length).toBe(0); + }); + + it('simple-name `User $u` parameter resolves to the imported App\\Models\\User (control case)', () => { + // Sanity check that unqualified type-hint resolution still works via the + // `use App\Models\User;` import. Without this control, U2's normalizer + // change could regress the simple-name path and we'd miss it. + expect(callsFromTo('saveLocal', 'record', 'app/Models/User.php').length).toBe(1); + expect(callsFromTo('saveLocal', 'record', 'app/Other/User.php').length).toBe(0); + }); +}); + +// --------------------------------------------------------------------------- +// MRO arity-mismatch: most-derived override with incompatible arity must NOT +// fall through to an arity-compatible ancestor (PHP throws ArgumentCountError +// at runtime). See receiver-bound-calls.ts Case 2. +// --------------------------------------------------------------------------- + +describe('PHP MRO arity-mismatch fallthrough', () => { + let result: PipelineResult; + + beforeAll(async () => { + result = await runPipelineFromRepo(path.join(FIXTURES, 'php-mro-arity-mismatch'), () => {}); + }, 60000); + + const callsFromTo = (source: string, target: string, targetFilePath: string) => + getRelationships(result, 'CALLS').filter( + (c) => c.source === source && c.target === target && c.targetFilePath === targetFilePath, + ); + + it('detects ParentModel, ChildModel, Orphan, and Caller classes', () => { + expect(getNodesByLabel(result, 'Class')).toEqual([ + 'Caller', + 'ChildModel', + 'Orphan', + 'ParentModel', + ]); + }); + + it('arity-incompatible most-derived override does NOT fall through to ParentModel::method', () => { + // Pre-fix bug: `$child->method(1)` with ChildModel::method(int,int) and + // ParentModel::method(int) would emit a false CALLS edge to ParentModel::method. + // Post-fix: zero CALLS edges from callIncompatible for this site. + expect(callsFromTo('callIncompatible', 'method', 'app/Models/ParentModel.php').length).toBe(0); + expect(callsFromTo('callIncompatible', 'method', 'app/Models/ChildModel.php').length).toBe(0); + }); + + it('arity-compatible most-derived override emits exactly one CALLS edge to ChildModel::compat', () => { + // Happy path: ChildModel::compat(int) matches the call site $child->compat(1). + expect(callsFromTo('callCompatible', 'compat', 'app/Models/ChildModel.php').length).toBe(1); + expect(callsFromTo('callCompatible', 'compat', 'app/Models/ParentModel.php').length).toBe(0); + }); + + it('arity-incompatible class with no parent emits zero CALLS edges (regression check)', () => { + // Orphan::method(int,int) called with one arg, no parent class — must remain + // unresolved both before and after the fix. + expect(callsFromTo('callNoParent', 'method', 'app/Models/Orphan.php').length).toBe(0); + }); + + it('arity-compatible most-derived call still resolves to ChildModel::method (happy path)', () => { + // Ensure the fix did not break compatible-arity resolution. + expect(callsFromTo('callMostDerivedHappy', 'method', 'app/Models/ChildModel.php').length).toBe( + 1, + ); + expect(callsFromTo('callMostDerivedHappy', 'method', 'app/Models/ParentModel.php').length).toBe( + 0, + ); + }); +}); + +// --------------------------------------------------------------------------- +// @declaration.variable double-match dedup on typed properties. +// Pre-fix, the catch-all property pattern in query.ts (no `type:` constraint) +// also matched typed property declarations and emitted a stray Variable def +// alongside the legitimate Property def. captures.ts now pre-scans rawMatches +// for @declaration.property anchors and suppresses the duplicate. +// --------------------------------------------------------------------------- + +describe('PHP typed-property double-match dedup', () => { + let result: PipelineResult; + + beforeAll(async () => { + result = await runPipelineFromRepo(path.join(FIXTURES, 'php-typed-property-dedup'), () => {}); + }, 60000); + + it('detects the Mixed class', () => { + expect(getNodesByLabel(result, 'Class')).toContain('Mixed'); + }); + + it('emits exactly one Property def for the typed property `$repo`', () => { + const properties = getNodesByLabel(result, 'Property'); + expect(properties.filter((n) => n === 'repo').length).toBe(1); + }); + + it('emits exactly one Property def for the constructor-promoted typed `$promotedRepo`', () => { + const properties = getNodesByLabel(result, 'Property'); + expect(properties.filter((n) => n === 'promotedRepo').length).toBe(1); + }); + + it('emits zero stray Variable defs for typed property and promoted typed parameter', () => { + // Pre-fix: a Variable def named `$repo` and `$promotedRepo` (no `$` strip) + // would slip through the catch-all pattern. Post-fix: zero. + const variables = getNodesByLabel(result, 'Variable'); + expect(variables.filter((n) => n === '$repo' || n === 'repo').length).toBe(0); + expect(variables.filter((n) => n === '$promotedRepo' || n === 'promotedRepo').length).toBe(0); + }); + + it('untyped property `$id` still emits its catch-all Property def (regression check)', () => { + // The untyped catch-all @declaration.variable pattern is the legitimate + // path for `public $id;`. Make sure the cross-match dedup does not + // over-suppress untyped declarations — they have no @declaration.property + // sibling, so their anchor is not in the typedPropertyAnchorIds set. + const properties = getNodesByLabel(result, 'Property'); + expect(properties.filter((n) => n === 'id').length).toBe(1); + }); + + it('no `$`-prefixed Property or Variable defs leak from typed declarations', () => { + // The catch-all branch does NOT run the `$`-strip normalization, so any + // def it produces for a typed property carries a `$`-prefixed name — + // a known receiver-binding lookup pollution vector. Post-fix the + // catch-all is suppressed for typed property_declaration anchors, so + // no `$repo` / `$promotedRepo` def should appear at any label. + for (const n of result.graph.iterNodes()) { + const name = String(n.properties.name); + if (name === '$repo' || name === '$promotedRepo') { + throw new Error(`leaked $-prefixed def: ${n.label}|${name}|${n.id}`); + } + } + }); +}); + +// --------------------------------------------------------------------------- +// Dynamic PHP constructs MUST NOT capture as resolvable references. +// Findings 1-7 of the PR #1497 adversarial review confirmed via grammar +// inspection that $obj->$method(), call_user_func(...), array/string +// callables, and dynamic property reads produce zero captures. This suite +// locks that invariant in regression so a future query.ts edit cannot +// silently relax `name: (name)` to `name: (_)` and reintroduce false- +// positive edges. +// --------------------------------------------------------------------------- + +describe('PHP dynamic dispatch — negative regression suite', () => { + let result: PipelineResult; + + beforeAll(async () => { + result = await runPipelineFromRepo(path.join(FIXTURES, 'php-dynamic-calls'), () => {}); + }, 60000); + + const callsFromDynamicTo = (target: string) => + getRelationships(result, 'CALLS').filter( + (c) => + c.target === target && + // Source is some method on `Dynamic` (the file under test). + c.sourceFilePath === 'app/Services/Dynamic.php', + ); + + it('detects the Dynamic and Targets classes', () => { + expect(getNodesByLabel(result, 'Class')).toContain('Dynamic'); + expect(getNodesByLabel(result, 'Class')).toContain('Targets'); + }); + + it('sanity check: non-dynamic call DOES emit an edge', () => { + // Without this, every zero-edge assertion below would pass even if the + // pipeline emitted no CALLS edges at all. + expect(callsFromDynamicTo('sanityStaticallyNamedTarget').length).toBe(1); + }); + + it('$obj->$method() emits no CALLS edge to dynamicProcess', () => { + expect(callsFromDynamicTo('dynamicProcess').length).toBe(0); + }); + + it('$obj->{$method}() emits no CALLS edge to dynamicBrace', () => { + expect(callsFromDynamicTo('dynamicBrace').length).toBe(0); + }); + + it('Class::$method() emits no CALLS edge to dynamicHandle', () => { + expect(callsFromDynamicTo('dynamicHandle').length).toBe(0); + }); + + it('$className::method() with untyped variable receiver emits no CALLS edge', () => { + // Two attractor classes (Targets and OtherTargets) both expose + // dynamicStaticMethod so the unresolved-receiver fallback (Finding 8 / + // U4) cannot fire — that isolates this assertion to the dynamic- + // dispatch suppression at the query / receiver-bound-calls layer. + expect(callsFromDynamicTo('dynamicStaticMethod').length).toBe(0); + }); + + it('$className::$method() with dynamic class and method names emits no CALLS edge', () => { + expect(callsFromDynamicTo('dynamicScopedDynName').length).toBe(0); + }); + + it('call_user_func / call_user_func_array string and array callables emit no CALLS edges', () => { + // call_user_func itself is a built-in with no workspace def, so the + // free-call to it is unresolved — no edge to `call_user_func`. + expect(callsFromDynamicTo('call_user_func').length).toBe(0); + expect(callsFromDynamicTo('call_user_func_array').length).toBe(0); + // None of the named targets reachable only via the callable argument + // should pick up a false-positive edge. + expect(callsFromDynamicTo('dynamicCallableMethod').length).toBe(0); + expect(callsFromDynamicTo('dynamicArrayCallableMethod').length).toBe(0); + expect(callsFromDynamicTo('dynamicArrayClassCallableMethod').length).toBe(0); + }); + + it('dynamic property read ($obj->$prop) emits no read-edge to dynamicProp', () => { + // No read-access property capture pattern exists in query.ts at all + // (Finding 2). Verify no CALLS / READS / write edge targets `dynamicProp`. + expect(callsFromDynamicTo('dynamicProp').length).toBe(0); + const reads = getRelationships(result, 'READS').filter( + (r) => r.target === 'dynamicProp' && r.sourceFilePath === 'app/Services/Dynamic.php', + ); + expect(reads.length).toBe(0); + }); +}); + +// --------------------------------------------------------------------------- +// phpEmitUnresolvedReceiverEdges exact-required-arity gate (Finding 8 / U4). +// The 0.6-confidence fallback for untyped receivers now requires argCount +// to exactly match the candidate's required parameter count for fixed- +// arity candidates. Variadic candidates keep the relaxed argCount >= +// required semantics. +// --------------------------------------------------------------------------- + +describe('PHP unresolved-receiver fallback exact-required-arity gate', () => { + let result: PipelineResult; + + beforeAll(async () => { + result = await runPipelineFromRepo( + path.join(FIXTURES, 'php-unresolved-receiver-arity'), + () => {}, + ); + }, 60000); + + const fallbackEdgeFromTo = (source: string, target: string) => + getRelationships(result, 'CALLS').filter( + (c) => + c.source === source && c.target === target && c.targetFilePath === 'app/Models/Handler.php', + ); + + it('detects Handler and Caller classes', () => { + expect(getNodesByLabel(result, 'Class')).toContain('Handler'); + expect(getNodesByLabel(result, 'Class')).toContain('Caller'); + }); + + it('happy path: argCount === required (0===0) emits 0.6 fallback edge', () => { + expect(fallbackEdgeFromTo('callHappyPath', 'happyPath').length).toBe(1); + }); + + it('argCount === required (1===1) on candidate with default param still emits edge', () => { + expect(fallbackEdgeFromTo('callDefaultExactRequired', 'withDefault').length).toBe(1); + }); + + it('argCount > required (2>1) on candidate with default param emits NO edge post-fix', () => { + // Pre-fix: first-stage narrowOverloadCandidates accepted (1 <= 2 <= 2). + // Post-fix: exact-required gate rejects (2 !== 1). + expect(fallbackEdgeFromTo('callDefaultBeyondRequired', 'withDefault').length).toBe(0); + }); + + it('variadic candidate, argCount === required (1===1) emits edge', () => { + expect(fallbackEdgeFromTo('callVariadicAtRequired', 'variadicLog').length).toBe(1); + }); + + it('variadic candidate, argCount > required (2>1) emits edge (relaxed)', () => { + expect(fallbackEdgeFromTo('callVariadicBeyondRequired', 'variadicLog').length).toBe(1); + }); + + it('variadic candidate, argCount < required (1<2) emits NO edge', () => { + expect(fallbackEdgeFromTo('callVariadicBelowRequired', 'variadicLogTwoRequired').length).toBe( + 0, + ); + }); +}); diff --git a/gitnexus/test/unit/registry-primary-flag.test.ts b/gitnexus/test/unit/registry-primary-flag.test.ts index 9e8be3455..864754b7f 100644 --- a/gitnexus/test/unit/registry-primary-flag.test.ts +++ b/gitnexus/test/unit/registry-primary-flag.test.ts @@ -148,17 +148,20 @@ describe('primaryLanguages', () => { it('returns exactly the flipped languages (env opts in unmigrated, opts out migrated)', () => { // Migrated languages are default-on; each must be opted out here when - // testing explicit env overrides. Java (unmigrated) opts in; Go stays off. - process.env['REGISTRY_PRIMARY_PYTHON'] = 'false'; - process.env['REGISTRY_PRIMARY_CSHARP'] = 'false'; - process.env['REGISTRY_PRIMARY_TYPESCRIPT'] = 'false'; - process.env['REGISTRY_PRIMARY_GO'] = 'false'; - process.env['REGISTRY_PRIMARY_C'] = 'false'; + // testing explicit env overrides. Java (unmigrated) opts in. + // Opt out every member of MIGRATED_LANGUAGES dynamically so this test + // does not have to be updated each time a new language ships its + // Ring 3 migration (PHP joined the set in commit 69786b16; future + // Ring 3 additions land here without test churn). + for (const lang of MIGRATED_LANGUAGES) { + process.env[envVarNameFor(lang)] = 'false'; + } process.env['REGISTRY_PRIMARY_JAVA'] = '1'; const enabled = primaryLanguages(); expect(enabled.has(SupportedLanguages.Python)).toBe(false); expect(enabled.has(SupportedLanguages.CSharp)).toBe(false); expect(enabled.has(SupportedLanguages.Go)).toBe(false); + expect(enabled.has(SupportedLanguages.PHP)).toBe(false); expect(enabled.has(SupportedLanguages.Java)).toBe(true); // Only Java is on: migrated defaults overridden off, Java explicitly on. expect(enabled.size).toBe(1); diff --git a/gitnexus/test/unit/scope-resolution/overload-narrowing.test.ts b/gitnexus/test/unit/scope-resolution/overload-narrowing.test.ts index 810c3a79b..9a14fdddd 100644 --- a/gitnexus/test/unit/scope-resolution/overload-narrowing.test.ts +++ b/gitnexus/test/unit/scope-resolution/overload-narrowing.test.ts @@ -69,11 +69,26 @@ describe('narrowOverloadCandidates — arity filtering', () => { expect(result.map((d) => d.nodeId)).toEqual(['v:1']); }); - it('falls back to the full overload list when arity filter empties it', () => { + it('returns empty when arity filter empties the set AND every candidate had definite bounds', () => { // argCount=5 doesn't match any overload (none variadic, all have max < 5). + // Post-commit af9af4a9 (PR #1497 / U1): the empty result is now authoritative + // because every rejected candidate had defined `parameterCount` / + // `requiredParameterCount`. The old "always fall back to full list" rescue + // was deliberately removed so resolvers actually drop calls that are + // definitively arity-incompatible (e.g., PHP `f(int $req, ...$rest)` + // called with zero args). const result = narrowOverloadCandidates([add1, add2, add3], 5, undefined); - expect(result.map((d) => d.nodeId)).toEqual(['add:1', 'add:2', 'add:3']); + expect(result.map((d) => d.nodeId)).toEqual([]); }); + + // Note: the `anyUnknownBounds ? overloads : []` branch in + // narrowOverloadCandidates is structurally unreachable in this caller's + // shape — a candidate with both `parameterCount` and `requiredParameterCount` + // undefined always passes the arity filter (neither `argCount > max` nor + // `argCount < min` can fire), so `arityMatches.length` is always > 0 + // whenever `anyUnknownBounds` is true. The branch is preserved in the + // source as a defensive guard for future refactors that might add + // additional rejection criteria in the filter. }); describe('narrowOverloadCandidates — type narrowing', () => { diff --git a/gitnexus/test/unit/scope-resolution/pick-implicit-this-overload.test.ts b/gitnexus/test/unit/scope-resolution/pick-implicit-this-overload.test.ts new file mode 100644 index 000000000..26f526382 --- /dev/null +++ b/gitnexus/test/unit/scope-resolution/pick-implicit-this-overload.test.ts @@ -0,0 +1,156 @@ +/** + * Unit tests for `pickImplicitThisOverload` — the implicit-`this` free-call + * resolver in `free-call-fallback.ts`. + * + * Codex PR #1497 review, finding 2: the previous implementation returned + * `candidates[0]` after `narrowOverloadCandidates` regardless of how many + * candidates survived narrowing. When two same-name methods on the same + * class had identical arity and unknown argument types, narrowing left both + * compatible and the resolver emitted a high-confidence CALLS edge whose + * target depended on registration order. The fix tightens the picker to + * require a UNIQUE post-narrowing candidate; otherwise the call is left + * unresolved. + * + * These tests exercise the function via synthetic stubs — no fixtures, no + * pipeline — because the failure shape (two same-arity overloads with + * indistinguishable types) cannot be produced by a PHP integration fixture + * (PHP forbids method overloading) and any C# fixture would entangle this + * unit's contract with the wider C# resolver. + */ + +import { describe, it, expect } from 'vitest'; +import type { Scope, ScopeId, SymbolDefinition } from 'gitnexus-shared'; +import { pickImplicitThisOverload } from '../../../src/core/ingestion/scope-resolution/passes/free-call-fallback.js'; +import type { ScopeResolutionIndexes } from '../../../src/core/ingestion/model/scope-resolution-indexes.js'; +import type { SemanticModel } from '../../../src/core/ingestion/model/semantic-model.js'; +import type { WorkspaceResolutionIndex } from '../../../src/core/ingestion/scope-resolution/workspace-index.js'; + +const CLASS_SCOPE_ID = 'scope:test.cs#1:1-100:1:Class' as ScopeId; +const CLASS_DEF_ID = 'def:test.cs:Foo'; + +const mkMethod = (overrides: Partial & { nodeId: string }): SymbolDefinition => ({ + nodeId: overrides.nodeId, + filePath: 'x.cs', + type: 'Method', + ...overrides, +}); + +const mkClassScope = (): Scope => + ({ + id: CLASS_SCOPE_ID, + parent: null, + kind: 'Class', + range: { startLine: 1, startCol: 1, endLine: 100, endCol: 1 }, + filePath: 'test.cs', + bindings: new Map(), + typeBindings: new Map(), + ownedDefs: [], + }) as unknown as Scope; + +const mkScopes = (scope: Scope): ScopeResolutionIndexes => + ({ + scopeTree: { + getScope: (id: ScopeId) => (id === scope.id ? scope : undefined), + }, + }) as unknown as ScopeResolutionIndexes; + +const mkWorkspaceIndex = (mapping: ReadonlyMap): WorkspaceResolutionIndex => + ({ + classScopeIdToDefId: mapping, + }) as unknown as WorkspaceResolutionIndex; + +const mkModel = ( + overloadsByName: ReadonlyMap, +): SemanticModel => + ({ + methods: { + lookupAllByOwner: (_classDefId: string, name: string) => + overloadsByName.get(name) ?? ([] as readonly SymbolDefinition[]), + }, + }) as unknown as SemanticModel; + +describe('pickImplicitThisOverload — uniqueness guard (Codex #1497 finding 2)', () => { + const site = { + inScope: CLASS_SCOPE_ID, + name: 'save', + arity: 1, + argumentTypes: undefined, + }; + + it('returns the sole overload when only one method exists on the owner', () => { + const sole = mkMethod({ nodeId: 'm:1', parameterCount: 1, requiredParameterCount: 1 }); + const scopes = mkScopes(mkClassScope()); + const workspace = mkWorkspaceIndex(new Map([[CLASS_SCOPE_ID, CLASS_DEF_ID]])); + const model = mkModel(new Map([['save', [sole]]])); + + const result = pickImplicitThisOverload(site, scopes, workspace, model); + + expect(result?.nodeId).toBe('m:1'); + }); + + it('returns the single survivor when narrowing disambiguates by arity', () => { + const save1 = mkMethod({ nodeId: 'm:1', parameterCount: 1, requiredParameterCount: 1 }); + const save2 = mkMethod({ nodeId: 'm:2', parameterCount: 2, requiredParameterCount: 2 }); + const scopes = mkScopes(mkClassScope()); + const workspace = mkWorkspaceIndex(new Map([[CLASS_SCOPE_ID, CLASS_DEF_ID]])); + const model = mkModel(new Map([['save', [save1, save2]]])); + + // site.arity = 1 → only save1 survives narrowing. + const result = pickImplicitThisOverload(site, scopes, workspace, model); + + expect(result?.nodeId).toBe('m:1'); + }); + + it('returns undefined when narrowing leaves two compatible candidates (the bug)', () => { + // Two same-arity, same-required-count overloads with no disambiguating + // parameter-type info on either def. `narrowOverloadCandidates` keeps + // both; pre-fix code returned `candidates[0]` (registration order); + // post-fix code returns undefined. + const save1 = mkMethod({ nodeId: 'm:1', parameterCount: 1, requiredParameterCount: 1 }); + const save2 = mkMethod({ nodeId: 'm:2', parameterCount: 1, requiredParameterCount: 1 }); + const scopes = mkScopes(mkClassScope()); + const workspace = mkWorkspaceIndex(new Map([[CLASS_SCOPE_ID, CLASS_DEF_ID]])); + const model = mkModel(new Map([['save', [save1, save2]]])); + + const result = pickImplicitThisOverload(site, scopes, workspace, model); + + expect(result).toBeUndefined(); + }); + + it('returns undefined when no method on the owner matches the call name', () => { + const scopes = mkScopes(mkClassScope()); + const workspace = mkWorkspaceIndex(new Map([[CLASS_SCOPE_ID, CLASS_DEF_ID]])); + const model = mkModel(new Map()); + + const result = pickImplicitThisOverload(site, scopes, workspace, model); + + expect(result).toBeUndefined(); + }); + + it('returns undefined when the call site is not inside a Class scope', () => { + // Module-scope sites: no enclosing class, so the implicit-this picker + // has nothing to pick from. Different from an empty-narrowing miss. + const moduleScope = { + id: 'scope:test.cs#1:1-100:1:Module' as ScopeId, + parent: null, + kind: 'Module', + range: { startLine: 1, startCol: 1, endLine: 100, endCol: 1 }, + filePath: 'test.cs', + bindings: new Map(), + typeBindings: new Map(), + ownedDefs: [], + } as unknown as Scope; + const scopes = mkScopes(moduleScope); + const workspace = mkWorkspaceIndex(new Map()); + const model = mkModel(new Map()); + + const result = pickImplicitThisOverload( + { ...site, inScope: moduleScope.id }, + scopes, + workspace, + model, + ); + + expect(result).toBeUndefined(); + }); +}); diff --git a/gitnexus/test/unit/scope-resolution/registries.test.ts b/gitnexus/test/unit/scope-resolution/registries.test.ts index b204287b5..2681af69f 100644 --- a/gitnexus/test/unit/scope-resolution/registries.test.ts +++ b/gitnexus/test/unit/scope-resolution/registries.test.ts @@ -250,7 +250,13 @@ describe('Step 5: arity filter', () => { ); }); - it('keeps incompatible candidates when no compatible candidate exists (soft penalty)', () => { + it('drops every candidate when ALL are incompatible AND none unknown (hard rejection)', () => { + // Post-commit af9af4a9 (PR #1497 / U1): the old soft-penalty fallback + // that kept incompatible candidates with `arityMatchIncompatible` + // weight was deliberately removed at this layer too. When every + // candidate is definitively arity-incompatible, the registry returns + // no resolution — matching the PHP variadic case `f(int $req, ...$rest)` + // called with zero args. const save3 = mkDef({ nodeId: 'def:save-three', type: 'Method', @@ -268,8 +274,41 @@ describe('Step 5: arity filter', () => { const results = buildMethodRegistry(ctx).lookup('save', 'scope:m', { callsite: { arity: 1 }, }); - expect(results).toHaveLength(1); - expect(evidenceOfKind(results[0]!, 'arity-match')?.weight).toBe( + expect(results).toHaveLength(0); + }); + + it('keeps incompatible candidates when at least one verdict is unknown (soft penalty)', () => { + // The soft-rescue path is still active when at least one candidate's + // arity verdict is 'unknown' — that signals missing metadata rather + // than a definitive mismatch, so all candidates (including incompatible + // ones) are preserved with their evidence weights for downstream + // tie-breaking. + const save3 = mkDef({ + nodeId: 'def:save-three', + type: 'Method', + qualifiedName: 'User.save', + parameterCount: 3, + }); + const saveUnknown = mkDef({ + nodeId: 'def:save-unknown', + type: 'Method', + qualifiedName: 'User.save', + }); + const mod = mkScope({ + id: 'scope:m', + parent: null, + bindings: { save: [mkBinding(save3, 'local'), mkBinding(saveUnknown, 'local')] }, + }); + const ctx = makeCtx([mod], [save3, saveUnknown], { + arity: (_callsite, def) => (def.nodeId === 'def:save-unknown' ? 'unknown' : 'incompatible'), + }); + const results = buildMethodRegistry(ctx).lookup('save', 'scope:m', { + callsite: { arity: 1 }, + }); + expect(results).toHaveLength(2); + const incompat = results.find((r) => r.def.nodeId === 'def:save-three'); + expect(incompat).toBeDefined(); + expect(evidenceOfKind(incompat!, 'arity-match')?.weight).toBe( EvidenceWeights.arityMatchIncompatible, ); }); From 7637bd1c8332feafb5073dd93f2390610ab4e61c Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 12 May 2026 23:41:05 +0100 Subject: [PATCH 8/9] chore(deps)(deps): bump @protobufjs/utf8 in /gitnexus (#1535) --- gitnexus/package-lock.json | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/gitnexus/package-lock.json b/gitnexus/package-lock.json index 7aa9dc7bf..13b225aed 100644 --- a/gitnexus/package-lock.json +++ b/gitnexus/package-lock.json @@ -1646,9 +1646,9 @@ "license": "BSD-3-Clause" }, "node_modules/@protobufjs/utf8": { - "version": "1.1.0", - "resolved": "https://registry.npmjs.org/@protobufjs/utf8/-/utf8-1.1.0.tgz", - "integrity": "sha512-Vvn3zZrhQZkkBE8LSuW3em98c0FwgO4nxzv6OdSxPKJIEKY2bGbHn+mhGIPerzI4twdxaP8/0+06HBpwf345Lw==", + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/@protobufjs/utf8/-/utf8-1.1.1.tgz", + "integrity": "sha512-oOAWABowe8EAbMyWKM0tYDKi8Yaox52D+HWZhAIJqQXbqe0xI/GV7FhLWqlEKreMkfDjshR5FKgi3mnle0h6Eg==", "license": "BSD-3-Clause" }, "node_modules/@rolldown/binding-android-arm64": { From 6a2361687358cb52173662bc5d5528fb5aee3bf3 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed, 13 May 2026 06:40:24 +0100 Subject: [PATCH 9/9] chore(deps)(deps): bump protobufjs from 7.5.5 to 7.5.8 in /gitnexus (#1536) --- gitnexus/package-lock.json | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/gitnexus/package-lock.json b/gitnexus/package-lock.json index 13b225aed..15831e54d 100644 --- a/gitnexus/package-lock.json +++ b/gitnexus/package-lock.json @@ -1600,9 +1600,9 @@ "license": "BSD-3-Clause" }, "node_modules/@protobufjs/codegen": { - "version": "2.0.4", - "resolved": "https://registry.npmjs.org/@protobufjs/codegen/-/codegen-2.0.4.tgz", - "integrity": "sha512-YyFaikqM5sH0ziFZCN3xDC7zeGaB/d0IUb9CATugHWbd1FRFwWwt4ld4OYMPWu5a3Xe01mGAULCdqhMlPl29Jg==", + "version": "2.0.5", + "resolved": "https://registry.npmjs.org/@protobufjs/codegen/-/codegen-2.0.5.tgz", + "integrity": "sha512-zgXFLzW3Ap33e6d0Wlj4MGIm6Ce8O89n/apUaGNB/jx+hw+ruWEp7EwGUshdLKVRCxZW12fp9r40E1mQrf/34g==", "license": "BSD-3-Clause" }, "node_modules/@protobufjs/eventemitter": { @@ -1628,9 +1628,9 @@ "license": "BSD-3-Clause" }, "node_modules/@protobufjs/inquire": { - "version": "1.1.0", - "resolved": "https://registry.npmjs.org/@protobufjs/inquire/-/inquire-1.1.0.tgz", - "integrity": "sha512-kdSefcPdruJiFMVSbn801t4vFK7KB/5gd2fYvrxhuJYg8ILrmn9SKSX2tZdV6V+ksulWqS7aXjBcRXl3wHoD9Q==", + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/@protobufjs/inquire/-/inquire-1.1.1.tgz", + "integrity": "sha512-mnzgDV26ueAvk7rsbt9L7bE0SuAoqyuys/sMMrmVcN5x9VsxpcG3rqAUSgDyLp0UZlmNfIbQ4fHfCtreVBk8Ew==", "license": "BSD-3-Clause" }, "node_modules/@protobufjs/path": { @@ -4543,22 +4543,22 @@ "license": "MIT" }, "node_modules/protobufjs": { - "version": "7.5.5", - "resolved": "https://registry.npmjs.org/protobufjs/-/protobufjs-7.5.5.tgz", - "integrity": "sha512-3wY1AxV+VBNW8Yypfd1yQY9pXnqTAN+KwQxL8iYm3/BjKYMNg4i0owhEe26PWDOMaIrzeeF98Lqd5NGz4omiIg==", + "version": "7.5.8", + "resolved": "https://registry.npmjs.org/protobufjs/-/protobufjs-7.5.8.tgz", + "integrity": "sha512-dvpCIeLPbXZS/Ete7yLaO7RenOdken2NHKykBXbsaGxZT0UTltcarBciw+A78SRQs9iMAAVpsYA+l8b1hTePIA==", "hasInstallScript": true, "license": "BSD-3-Clause", "dependencies": { "@protobufjs/aspromise": "^1.1.2", "@protobufjs/base64": "^1.1.2", - "@protobufjs/codegen": "^2.0.4", + "@protobufjs/codegen": "^2.0.5", "@protobufjs/eventemitter": "^1.1.0", "@protobufjs/fetch": "^1.1.0", "@protobufjs/float": "^1.0.2", - "@protobufjs/inquire": "^1.1.0", + "@protobufjs/inquire": "^1.1.1", "@protobufjs/path": "^1.1.2", "@protobufjs/pool": "^1.1.0", - "@protobufjs/utf8": "^1.1.0", + "@protobufjs/utf8": "^1.1.1", "@types/node": ">=13.7.0", "long": "^5.0.0" },