diff --git a/docs/languages/local-import-scoping.md b/docs/languages/local-import-scoping.md new file mode 100644 index 000000000..66ff40116 --- /dev/null +++ b/docs/languages/local-import-scoping.md @@ -0,0 +1,191 @@ +# Local import scoping + +GitNexus resolves statically known local imports through their lexical owner and +actual declaration identity. A nearer name that is unresolved, unavailable, or +noncallable blocks an unrelated outer or workspace fallback. An absent name can +continue through the existing lookup rules. This is static analysis: it does not +execute loaders or infer arbitrary runtime initialization order. + +## Shared contract + +Language rules stay in providers. The shared +[name-claim lookup](../../gitnexus-shared/src/scope-resolution/name-claims.ts) +distinguishes `absent`, `blocked`, and `resolved` before filtering by target kind. +Its [regressions](../../gitnexus/test/unit/scope-resolution/lexical-name-claims.test.ts) +cover noncallable barriers, unresolved imports, type/value lookup, declaration +positions, and inferred aliases that retain their initializer's lookup position. +Calls, constructors, receiver/type inference, inheritance, namespace lookup, and +fallback consumers use these ownership facts rather than independently choosing a +same-named symbol. + +The [shared records](../../gitnexus-shared/src/scope-resolution/types.ts) carry +ownership, activation positions, directive destinations, and lookup purpose as +serializable data. They retain declaration identity and ambiguity; they contain +no AST nodes or callbacks. `declaredAtScope` describes ownership, while +`runsOnlyWhenCalled` describes deferred execution. A function-local Rust `use` or +C `#include` is not deferred merely because of its position; Python and Ruby +loading can be. The +[function-local import chain tests](../../gitnexus/test/unit/scope-resolution/function-local-import-chain.test.ts) +pin that distinction. + +## Providers with expanded lexical support + +These seven entries are part of the +[18-provider registry](../../gitnexus/src/core/ingestion/languages/index.ts). +The evidence links cover exact targets and negative cases, not just edge counts. + +| Provider | Supported behavior | Source and regression evidence | +| --- | --- | --- | +| Python | Function-local named, aliased, and namespace imports; compiler-local ownership; ordered class lookup; `global`/`nonlocal` destinations. | [Lexical facts](../../gitnexus/src/core/ingestion/languages/python/lexical-bindings.ts), [local imports](../../gitnexus/test/unit/scope-resolution/python/python-local-imports.test.ts), [namespace semantics](../../gitnexus/test/integration/resolvers/python-namespace-semantics.test.ts) | +| TypeScript | Literal genuine `require`, immediately awaited `import`, and import-equals; namespace, named, and renamed bindings; type/value separation. | [Loader collector](../../gitnexus/src/core/ingestion/languages/typescript/local-loaders.ts), [loader tests](../../gitnexus/test/unit/scope-resolution/typescript/local-loader-imports.test.ts), [type ownership](../../gitnexus/test/unit/scope-resolution/typescript/lexical-type-ownership.test.ts) | +| JavaScript | The same supported runtime loaders and lexical barriers, using JavaScript captures for JS/JSX. | [Captures](../../gitnexus/src/core/ingestion/languages/javascript/captures.ts), [JS/JSX loader cases](../../gitnexus/test/unit/scope-resolution/typescript/local-loader-imports.test.ts) | +| Vue | Local loader facts in extracted scripts, with original declaration graph positions preserved. Explicit JS/JSX uses its own semantic grammar inside the worker; TS reuses the cached tree. | [Script delegation](../../gitnexus/src/core/ingestion/languages/vue/captures.ts), [direct/pre-extracted parity](../../gitnexus/test/unit/scope-resolution/typescript/local-loader-imports.test.ts), [real workers](../../gitnexus/test/integration/local-import-worker-parity.test.ts) | +| Rust | Lexical `use`, producer identity for imported factories/types, and Cargo-aware crate/module identity. Unresolved explicit imports block unrelated fallback. | [Lexical facts](../../gitnexus/src/core/ingestion/languages/rust/lexical-bindings.ts), [import targets](../../gitnexus/src/core/ingestion/languages/rust/import-target.ts), [local scope](../../gitnexus/test/integration/resolvers/rust-import-scope.test.ts), [Cargo boundaries](../../gitnexus/test/integration/resolvers/rust-cargo-target-fallback.test.ts) | +| C++ | Position-sensitive local `using` declarations/directives, separate from includes; propagated header claims retain unresolved named `using` barriers. | [Captures](../../gitnexus/src/core/ingestion/languages/cpp/captures.ts), [local using](../../gitnexus/test/integration/resolvers/cpp-local-using.test.ts), [header using](../../gitnexus/test/integration/resolvers/cpp-local-using-headers.test.ts) | +| Zig | Literal `@import` namespace/member bindings, block activation, container hoisting, stable deep aliases, and inline `@import` receivers. | [Captures](../../gitnexus/src/core/ingestion/languages/zig/captures.ts), [local imports](../../gitnexus/test/integration/resolvers/zig-local-imports.test.ts), [static gating](../../gitnexus/test/integration/resolvers/zig-static-gating.test.ts) | + +### Python boundaries + +Compiler-local binders include parameters, assignment patterns, loop/with/except +targets, match captures, augmented assignment, deletion, and comprehension/walrus +bindings. Comprehension targets do not leak, and the first iterable is evaluated +outside the comprehension environment. Class bodies use ordered locals; methods +skip class locals for bare-name lookup while explicit instance receivers remain +available. Annotation-only declarations distinguish compiler ownership from a +runtime store. These cases are characterized in the +[Python local-import tests](../../gitnexus/test/unit/scope-resolution/python/python-local-imports.test.ts). + +Mixed import/value rebinding and conflicting import identities conservatively +abstain throughout the environment, including simple sequential rebinding; this +is not a control-flow evaluator. A definite `def`/`class` declaration can still +supply its own target after activation. The +[provider policy](../../gitnexus/src/core/ingestion/languages/python/lexical-bindings.ts) +and [rebinding regressions](../../gitnexus/test/integration/resolvers/python-namespace-semantics.test.ts) +define this boundary. There is no full annotation evaluator, `exec` evaluation, +or general closure scheduling model. A deferred `global`/`nonlocal` installer is +not assumed to have executed for unrelated callers. + +Conditional imports with one compatible identity remain available for structural +reference resolution, including `TYPE_CHECKING` annotations and `try` imports. +Their runtime execution is not guaranteed by a resolved graph edge. + +Default expressions look up names in the enclosing environment. FastAPI's +synthetic dependency graph caller is recorded separately as `callerScope`; it +does not move lexical lookup into the handler body. Both +[direct extraction](../../gitnexus/test/unit/scope-resolution/python/python-local-imports.test.ts) +and [worker transport](../../gitnexus/test/integration/local-import-worker-parity.test.ts) +check this distinction. + +### JavaScript-family boundaries + +Loader recognition checks AST shape and binding identity, including parameters, +catch/destructuring bindings, temporal-dead-zone ownership, and writes to the +loader or imported handle. CommonJS forwarding uses that same identity check. +Reassigning one destructured import leaves unchanged sibling re-exports intact. +Writes to ordinary locals retain their declared type and callable-value facts. +Unawaited dynamic imports remain Promise/file-dependency facts rather than module +namespace aliases. Computed specifiers, indirect loader/Promise chains, and +unsupported destructuring remain unresolved; see the +[loader regressions](../../gitnexus/test/unit/scope-resolution/typescript/local-loader-imports.test.ts). + +Vue's explicit JS/JSX script requires an additional bounded semantic parse because +the worker's structural tree uses TypeScript. That parse runs inside the real +worker. The [parity test](../../gitnexus/test/integration/local-import-worker-parity.test.ts) +witnesses returned worker facts; a main-thread fallback does not satisfy it. + +### Rust, C++, and Zig boundaries + +Rust uses Cargo editions, target roots, and module identities, including files +with multiple valid identities. Reusable per-analysis indexes bound repeated +lookup work; this is not a claim of a global indexing speedup. See +[Cargo metadata](../../gitnexus/src/core/ingestion/languages/rust/cargo-targets.ts) +and its [tests](../../gitnexus/test/unit/scope-resolution/rust-cargo-targets.test.ts). +Exact same-name inline `pub use` forwarding is supported. A forwarding alias that +changes symbol spelling through a file-only target conservatively remains +unresolved; ambiguous same-file homonyms and re-export cycles cannot select an +unrelated declaration. These limits are explicit in the +[Cargo fallback regressions](../../gitnexus/test/integration/resolvers/rust-cargo-target-fallback.test.ts). +Inferred receiver types retain their receiving declaration range. When same-block +shadowing replaces a retained type fact, earlier uses cannot consume the later +declaration's type and may remain unresolved; this is not full SSA inference. See +the [factory ownership regressions](../../gitnexus/test/unit/scope-resolution/rust/rust-range-binding-order.test.ts). + +C++ includes keep their compilation-unit behavior alongside lexical `using`. +The header tests cover literal includes, declaration positions, unresolved named +using claims, and overload identity. They do not establish a full preprocessor, +namespace-alias evaluator, or arbitrary transitive `using namespace` chain support. +Zig nonliteral imports remain unresolved; its block-local names do not escape or +activate before their declarations. + +## Retained provider behavior + +The other eleven providers retain their existing import/loading models. This +work does not reinterpret every dependency construct as a local value import. + +| Provider | Retained model and boundary | Evidence | +| --- | --- | --- | +| Ruby | Wildcard loading; method-local loading is deferred. The extractor recognizes string arguments to `require`, `require_relative`, and `load`. The linked behavioral tests establish `require`/`require_relative` and deferral, not arbitrary `load` execution. | [Extractor](../../gitnexus/src/core/ingestion/languages/ruby/captures.ts), [loading](../../gitnexus/test/integration/resolvers/ruby-scope.test.ts), [deferral](../../gitnexus/test/unit/scope-resolution/function-local-import-chain.test.ts) | +| PHP | Namespace `use` imports are distinct from closure capture and trait `use`. The scope provider does not propagate names through `include`/`require`. | [Query](../../gitnexus/src/core/ingestion/languages/php/query.ts), [ownership](../../gitnexus/src/core/ingestion/languages/php/simple-hooks.ts), [resolver coverage](../../gitnexus/test/integration/resolvers/php.test.ts) | +| C | Header includes remain preprocessor/compilation-unit dependencies, including includes written inside functions; no full preprocessing is added. | [Include interpretation](../../gitnexus/src/core/ingestion/languages/c/interpret.ts), [include tests](../../gitnexus/test/unit/scope-resolution/c/c-imports.test.ts), [deferral boundary](../../gitnexus/test/unit/scope-resolution/function-local-import-chain.test.ts) | +| Objective-C | Headers, modules, and compilation-unit siblings retain the dedicated provider rules. | [Provider guide](objective-c-provider.md), [resolver](../../gitnexus/src/core/ingestion/languages/objective-c/scope-resolver.ts), [workspace tests](../../gitnexus/test/unit/scope-resolution/objective-c/objc-workspace.test.ts) | +| COBOL | `COPY` stays on the dedicated copybook path with module ownership. | [Interpretation](../../gitnexus/src/core/ingestion/languages/cobol/interpret.ts), [COPY target parity](../../gitnexus/test/unit/scope-resolution/cobol-import-target-parity.test.ts) | +| Go | File imports and package sibling visibility retain separate provider rules. | [Import ownership](../../gitnexus/src/core/ingestion/languages/go/simple-hooks.ts), [imports](../../gitnexus/test/unit/scope-resolution/go/go-imports.test.ts), [package siblings](../../gitnexus/test/unit/scope-resolution/go/go-package-siblings.test.ts) | +| Java | Imports belong to the compilation unit. | [Ownership](../../gitnexus/src/core/ingestion/languages/java/simple-hooks.ts), [named/wildcard import resolution](../../gitnexus/test/integration/resolvers/java.test.ts) | +| Kotlin | File import headers retain their existing alias/package resolution. | [Header decomposition](../../gitnexus/src/core/ingestion/languages/kotlin/import-decomposer.ts), [resolver tests](../../gitnexus/test/integration/resolvers/kotlin.test.ts) | +| C# | Namespace/file `using` directives remain imports; resource `using` statements are not import directives. | [Directive query](../../gitnexus/src/core/ingestion/languages/csharp/query.ts), [import tests](../../gitnexus/test/unit/scope-resolution/csharp/csharp-imports.test.ts) | +| Swift | Imports and target/module visibility retain provider ownership. | [Ownership](../../gitnexus/src/core/ingestion/languages/swift/simple-hooks.ts), [import decomposition](../../gitnexus/test/unit/scope-resolution/swift/import-decomposer.test.ts), [target siblings](../../gitnexus/test/unit/scope-resolution/swift/target-siblings.test.ts) | +| Dart | Library imports retain existing package identity and visibility rules. | [Ownership](../../gitnexus/src/core/ingestion/languages/dart/simple-hooks.ts), [library/package regressions](../../gitnexus/test/integration/resolvers/dart.test.ts) | + +## Taint provenance + +Taint source, sink, and sanitizer matching uses the same lexical provenance. +A genuine external import may match its raw module name even when that package +has no indexed source. A sibling import, local shadow, or type-only import cannot +authorize that match. General copied-handle tracking is outside this support. +The [taint regressions](../../gitnexus/test/unit/taint/lexical-imports.test.ts) +cover extraction, emission, and function summaries, including abstention when +exact call-site positions are missing. + +## Cache migration and verification evidence + +[Parse-cache schema 136](../../gitnexus/src/storage/parse-cache.ts) replaces 133, +134, and 135. It invalidates overly broad Python/TS/JS binding barriers, +incorrect parameter-annotation lookup ownership, and embedded Vue grammar and +source positions. The same version gate applies to the +[durable ParsedFile store](../../gitnexus/src/storage/parsedfile-store.ts). +Incompatible records are rejected and rebuilt on the next analysis; unchanged +records can then be reused. The +[schema regression](../../gitnexus/test/unit/incremental-parse-cache.test.ts) +explicitly rejects valid schema-133, schema-134, and schema-135 records in both stores. +PDG-mode chunk keys also advance to `pdg:6`, so embedded-script CFGs with +incorrect call anchors are rebuilt before the resolved-callee join. + +The [real-worker parity suite](../../gitnexus/test/integration/local-import-worker-parity.test.ts) +witnesses worker boot, dispatch, and returned ParsedFiles, then compares complete +provider facts with direct extraction, including `callerScope` and C++ side-channel +data. Optional grammar availability is explicit in the test gate. + +The [persistence suite](../../gitnexus/test/integration/resolvers/local-import-persistence.test.ts) +loads both stores afresh from disk. Its parse-cache chunks contain no embedded +ParsedFiles, so warm equality requires durable-store reuse. It checks identical +`CALLS`/`IMPORTS` identities with zero extraction and zero worker dispatch on an +unchanged warm run. Editing a local import target or shadowing `require` removes +the stale target, and a second warm run preserves the edited graph. + +Both suites are in the [cross-platform manifest](../../gitnexus/scripts/cross-platform-tests.ts); +the [shard regression](../../gitnexus/test/unit/cross-platform-shard.test.ts) +ensures each is selected exactly once across the four Windows/macOS shards. +[CI](../../.github/workflows/ci-tests.yml) owns native execution on those systems; +Ubuntu's full suite discovers the tests normally. These are focused evidence +boundaries, not a statement that the final full suite or every platform has passed. + +## Relationship to main and issue #3499 + +The integrated main baseline `50aa4be3` already includes #3502's nested Python +declaration ownership fix for the original #3499 case and #3504's aliased package +re-export fix. Their [Python integration coverage](../../gitnexus/test/integration/resolvers/python.test.ts) +and [module namespace coverage](../../gitnexus/test/unit/scope-resolution/python/python-module-namespace-construction.test.ts) +remain relevant. This branch extends that base with the lexical claims, provider +semantics, and persistence evidence described above. Issue coverage is **Related +#3499**; this document does not assert that all dynamic Python behavior is solved +or make an issue-status claim. diff --git a/gitnexus-shared/src/index.ts b/gitnexus-shared/src/index.ts index b5fbe37a5..10bf7c0ad 100644 --- a/gitnexus-shared/src/index.ts +++ b/gitnexus-shared/src/index.ts @@ -223,3 +223,22 @@ export { stripGitSuffix, } from './integrations/understand-quickly.js'; export type { UqDispatchPayload } from './integrations/understand-quickly.js'; + +export type { + SourcePosition, + LookupPurpose, + NameClaim, + ScopeLookupPolicy, + NameLookupOptions, +} from './scope-resolution/types.js'; +export { + lookupLexicalName, + nameClaimsFor, + selectClaimsAtScope, + rangesOverlap, +} from './scope-resolution/name-claims.js'; +export type { + NameClaimResult, + NameClaimSources, + ScopeClaimSelection, +} from './scope-resolution/name-claims.js'; diff --git a/gitnexus-shared/src/scope-resolution/finalize-algorithm.ts b/gitnexus-shared/src/scope-resolution/finalize-algorithm.ts index 2254a835a..c31605780 100644 --- a/gitnexus-shared/src/scope-resolution/finalize-algorithm.ts +++ b/gitnexus-shared/src/scope-resolution/finalize-algorithm.ts @@ -488,13 +488,14 @@ function makeEdgeDrafts( workspace: WorkspaceIndex, ): ImportEdgeDraft[] { const fromScope = - hooks.importsBindAtLexicalScope === true + (parsed.bindsAtLexicalScope ?? hooks.importsBindAtLexicalScope) === true ? (parsed.declaredAtScope ?? file.moduleScope) : file.moduleScope; // Dynamic-unresolved passes through — no `BindingRef`, no target file. if (parsed.kind === 'dynamic-unresolved') { const base: ImportEdge = { localName: parsed.localName, + ...(parsed.atRange !== undefined ? { atRange: parsed.atRange } : {}), targetFile: null, targetExportedName: '', kind: 'dynamic-unresolved', @@ -522,6 +523,7 @@ function makeEdgeDrafts( if (targetFile === null) { const base: ImportEdge = { localName: extractLocalName(parsed), + ...(parsed.atRange !== undefined ? { atRange: parsed.atRange } : {}), targetFile: null, targetExportedName: extractExportedName(parsed), kind: edgeKindFor(parsed), @@ -560,6 +562,7 @@ function makeEdgeDrafts( return targetFiles.map((tf) => { const base: ImportEdge = { localName: extractLocalName(parsed), + ...(parsed.atRange !== undefined ? { atRange: parsed.atRange } : {}), targetFile: tf, targetExportedName: extractExportedName(parsed), kind: @@ -611,22 +614,25 @@ function typeOnlyFor(parsed: ParsedImport): { typeOnly?: true } { } /** - * Re-carry both runtime-presence flags from an existing edge onto a derived - * one. + * Carry runtime-presence flags and source provenance onto a derived edge. * * `expandWildcard` builds each `wildcard-expanded` edge from scratch rather * than spreading the source (three fields differ per exported name), so every - * field it does not name is dropped. That is exactly how both flags were lost - * once already. Naming the pair here keeps "these two travel together" in one - * place, so a third presence flag is added in one place too. + * field it does not name is dropped. Keep presence flags and the source range + * together here so wildcard expansion preserves both runtime policy and the + * location used to select the owning import claim. */ -function carriedPresenceFlags(edge: Pick): { +function carriedPresenceFlags( + edge: Pick, +): { typeOnly?: true; runsOnlyWhenCalled?: true; + atRange?: ImportEdge['atRange']; } { return { ...(edge.typeOnly === true ? { typeOnly: true } : {}), ...(edge.runsOnlyWhenCalled === true ? { runsOnlyWhenCalled: true } : {}), + ...(edge.atRange !== undefined ? { atRange: edge.atRange } : {}), }; } diff --git a/gitnexus-shared/src/scope-resolution/name-claims.ts b/gitnexus-shared/src/scope-resolution/name-claims.ts new file mode 100644 index 000000000..7af01c30d --- /dev/null +++ b/gitnexus-shared/src/scope-resolution/name-claims.ts @@ -0,0 +1,337 @@ +/** Pure lexical ownership shared by registries, inference, and import provenance. */ +import type { + BindingRef, + ImportEdge, + NameClaim, + NameLookupOptions, + Range, + Scope, + ScopeId, + ScopeLookup, + SourcePosition, + TypeRef, +} from './types.js'; + +export interface ScopeClaimSelection { + readonly status: 'absent' | 'blocked' | 'selected' | 'module'; + readonly claims: readonly NameClaim[]; +} +export interface NameClaimResult { + readonly status: 'absent' | 'blocked' | 'resolved'; + readonly scope?: Scope; + readonly typeBinding?: TypeRef; + readonly bindings: readonly BindingRef[]; + readonly imports: readonly ImportEdge[]; + readonly claims: readonly NameClaim[]; +} +export interface NameClaimSources { + readonly scopes: ScopeLookup; + readonly bindingsAt?: (scope: Scope, name: string) => readonly BindingRef[]; + readonly importsAt?: (scope: Scope) => readonly ImportEdge[]; +} +const ABSENT: NameClaimResult = Object.freeze({ + status: 'absent', + bindings: [], + imports: [], + claims: [], +}); +// Published scope arrays are readonly. Cache by the array, not the scope ID: +// finalization and cache hydration can replace a scope's import/claim snapshot. +const claimsByName = new WeakMap>(); +const importsByName = new WeakMap< + readonly ImportEdge[], + ReadonlyMap +>(); +const NO_CLAIMS: readonly NameClaim[] = Object.freeze([]); +const NO_IMPORTS: readonly ImportEdge[] = Object.freeze([]); + +export function nameClaimsFor(scope: Scope, name: string): readonly NameClaim[] { + const claims = scope.nameClaims; + if (claims === undefined || claims.length === 0) return NO_CLAIMS; + let byName = claimsByName.get(claims); + if (byName === undefined) { + const indexed = new Map(); + for (const claim of claims) { + const bucket = indexed.get(claim.name); + if (bucket === undefined) indexed.set(claim.name, [claim]); + else bucket.push(claim); + } + byName = indexed; + claimsByName.set(claims, byName); + } + return byName.get(name) ?? NO_CLAIMS; +} + +function importsFor(imports: readonly ImportEdge[], name: string): readonly ImportEdge[] { + if (imports.length === 0) return NO_IMPORTS; + let byName = importsByName.get(imports); + if (byName === undefined) { + const indexed = new Map(); + for (const edge of imports) { + if (edge.kind === 'side-effect' || edge.kind === 'dynamic-resolved') continue; + const bucket = indexed.get(edge.localName); + if (bucket === undefined) indexed.set(edge.localName, [edge]); + else bucket.push(edge); + } + byName = indexed; + importsByName.set(imports, byName); + } + return byName.get(name) ?? NO_IMPORTS; +} +const compare = (a: SourcePosition, b: SourcePosition): number => + a.startLine - b.startLine || a.startCol - b.startCol; +export function rangesOverlap(a: Range, b: Range): boolean { + return ( + compare(a, { startLine: b.endLine, startCol: b.endCol }) < 0 && + compare(b, { startLine: a.endLine, startCol: a.endCol }) < 0 + ); +} + +/** Ownership is selected before target availability or target kind is inspected. */ +export function selectClaimsAtScope( + scope: Scope, + name: string, + options: NameLookupOptions = {}, +): ScopeClaimSelection { + const purpose = options.purpose ?? 'value'; + const claims = nameClaimsFor(scope, name).filter( + (c) => c.purpose === undefined || c.purpose === 'both' || c.purpose === purpose, + ); + if (claims.length === 0) return { status: 'absent', claims }; + const active = claims.filter( + (c) => + c.availableFrom === undefined || + options.position === undefined || + compare(c.availableFrom, options.position) <= 0, + ); + if (active.length === 0) { + const first = claims.reduce((a, b) => (compare(a.range, b.range) <= 0 ? a : b)); + return { + status: + first.inactive === 'outer' ? 'absent' : first.inactive === 'module' ? 'module' : 'blocked', + claims, + }; + } + // Same-position alternatives retain ambiguity. Providers can mark uncertain + // rebinding explicitly blocked rather than claiming a control-flow winner. + const ordered = active.filter((c) => c.hoisted !== true); + const relevant = ordered.length > 0 ? ordered : active; + const latest = relevant.reduce((a, b) => (compare(a.range, b.range) >= 0 ? a : b)).range; + let selected = relevant.filter((c) => compare(c.range, latest) === 0); + if (selected.some((c) => c.merge === true)) + selected = active.filter((c) => c.merge === true || compare(c.range, latest) === 0); + return { + status: selected.some((c) => c.kind === 'blocked' && c.redirect === undefined) + ? 'blocked' + : 'selected', + claims: selected, + }; +} + +function moduleScope(start: Scope, scopes: ScopeLookup): Scope | undefined { + let scope: Scope | undefined = start; + const seen = new Set(); + while (scope !== undefined && !seen.has(scope.id)) { + if (scope.kind === 'Module') return scope; + seen.add(scope.id); + scope = scope.parent === null ? undefined : scopes.getScope(scope.parent); + } + return undefined; +} +function redirectScope( + scope: Scope, + name: string, + claim: NameClaim, + scopes: ScopeLookup, +): Scope | undefined { + if (claim.redirect === 'module') return moduleScope(scope, scopes); + let parent = scope.parent === null ? undefined : scopes.getScope(scope.parent); + const seen = new Set(); + while (parent !== undefined && !seen.has(parent.id)) { + seen.add(parent.id); + if ( + parent.kind === 'Function' && + (nameClaimsFor(parent, name).some((c) => c.redirect === undefined) || + parent.lexicalNames?.has(name) || + parent.bindings.has(name) || + parent.typeBindings.has(name)) + ) + return parent; + parent = parent.parent === null ? undefined : scopes.getScope(parent.parent); + } + return undefined; +} +function matchesClaim(range: Range | undefined, claim: NameClaim): boolean { + return range === undefined || rangesOverlap(range, claim.range); +} + +function matchesBindingClaim(range: Range | undefined, claim: NameClaim): boolean { + return ( + range === undefined || + (compare(claim.range, range) <= 0 && + compare( + { startLine: range.endLine, startCol: range.endCol }, + { startLine: claim.range.endLine, startCol: claim.range.endCol }, + ) <= 0) + ); +} + +export function lookupLexicalName( + startScope: ScopeId, + name: string, + sources: NameClaimSources, + options: NameLookupOptions = {}, +): NameClaimResult { + let id: ScopeId | null = startScope; + let position = options.position; + let forcedClaims: readonly NameClaim[] | undefined; + const visited = new Set(); + while (id !== null && !visited.has(id)) { + visited.add(id); + const scope = sources.scopes.getScope(id); + if (scope === undefined) return { ...ABSENT, status: 'blocked' }; + if ( + scope.kind === 'Object' || + (id !== startScope && + ((scope.lookupPolicy?.skipFromChildren && + !scope.lookupPolicy.visibleNamesFromChildren?.includes(name)) || + (options.skipEnclosingClasses && scope.kind === 'Class'))) + ) { + id = scope.lookupPolicy?.parentScope ?? scope.parent; + continue; + } + const selected: ScopeClaimSelection = + forcedClaims === undefined + ? selectClaimsAtScope(scope, name, { ...options, position }) + : { status: 'selected', claims: forcedClaims }; + forcedClaims = undefined; + const redirected = selected.claims.find((c) => c.redirect !== undefined); + if (redirected !== undefined) { + const target = redirectScope(scope, name, redirected, sources.scopes); + if (target === undefined || target.id === id) + return { ...ABSENT, status: 'blocked', scope, claims: selected.claims }; + if (selected.status === 'selected' && redirected.kind !== 'blocked') { + forcedClaims = selected.claims.map((c) => ({ ...c, redirect: undefined })); + } + id = target.id; + position = undefined; + continue; + } + if (selected.status === 'module') { + const target = moduleScope(scope, sources.scopes); + if (target === undefined || target.id === id) + return { ...ABSENT, status: 'blocked', scope, claims: selected.claims }; + let skipped: Scope | undefined = scope; + const skippedIds = new Set(); + while (skipped !== undefined && skipped.id !== target.id && !skippedIds.has(skipped.id)) { + skippedIds.add(skipped.id); + if (skipped.lookupPolicy?.deferParentActivation) position = undefined; + skipped = skipped.parent === null ? undefined : sources.scopes.getScope(skipped.parent); + } + id = target.id; + continue; + } + if (selected.status === 'blocked') + return { ...ABSENT, status: 'blocked', scope, claims: selected.claims }; + const nameClaims = nameClaimsFor(scope, name); + const hasClaimsForName = nameClaims.length > 0; + const allBindings = sources.bindingsAt?.(scope, name) ?? scope.bindings.get(name) ?? []; + const allImports = importsFor(sources.importsAt?.(scope) ?? scope.imports ?? NO_IMPORTS, name); + let bindings = allBindings.filter( + (b) => + b.availableFrom === undefined || + position === undefined || + compare(b.availableFrom, position) <= 0, + ); + let imports = allImports; + if (selected.status === 'selected') { + bindings = bindings.filter((b) => + selected.claims.some((c) => + c.kind === 'import' + ? b.via !== undefined && matchesClaim(b.via.atRange, c) + : c.kind === 'binding' && + (b.origin === 'local' || b.declarationRange !== undefined) && + matchesBindingClaim(b.declarationRange, c), + ), + ); + imports = allImports.filter((e) => + selected.claims.some((c) => c.kind === 'import' && matchesClaim(e.atRange, c)), + ); + } else if (hasClaimsForName) { + // A declaration in another namespace, or a not-yet-visible declaration, + // must not sneak back through the unfiltered legacy binding channels. + bindings = bindings.filter( + (b) => + b.availableFrom !== undefined && + b.declarationRange !== undefined && + !nameClaims.some((c) => rangesOverlap(c.range, b.declarationRange!)), + ); + imports = []; + } + const ownType = scope.typeBindings.get(name); + const typeFactOwnsName = + ownType !== undefined && + !(scope.lookupPolicy?.nameClaimsComplete && ownType.source === 'assignment-inferred'); + const owns = + selected.status === 'selected' || + bindings.length > 0 || + imports.length > 0 || + scope.ownsReceivers?.has(name) === true || + (!hasClaimsForName && (scope.lexicalNames?.has(name) === true || typeFactOwnsName)); + if (owns) { + let typeBinding = ownType; + if ( + selected.status === 'selected' && + ownType?.bindingRange !== undefined && + !selected.claims.some( + (c) => + c.kind === 'import' || + (c.kind === 'binding' && rangesOverlap(c.range, ownType.bindingRange!)), + ) + ) { + // A later assignment can infer the type of the already-owned binding. + // Check ownership after that fact binds its value, not at the producer's + // lookup position: a later shadowing let still reads the earlier binding + // in its initializer, but must not lend that binding its inferred type. + const factClaims = + ownType.source === 'constructor-inferred' || ownType.source === 'assignment-inferred' + ? selectClaimsAtScope(scope, name, { + ...options, + position: { + startLine: ownType.bindingRange.endLine, + startCol: ownType.bindingRange.endCol, + }, + }) + : undefined; + if ( + factClaims?.status !== 'selected' || + factClaims.claims.length !== selected.claims.length || + !selected.claims.every( + (claim) => claim.kind === 'binding' && factClaims.claims.includes(claim), + ) + ) + typeBinding = undefined; + } + const usable = bindings.filter( + (b) => (options.purpose ?? 'value') !== 'value' || b.via?.typeOnly !== true, + ); + return { + status: + usable.length > 0 || + imports.some( + (e) => e.kind === 'namespace' && e.targetFile !== null && e.linkStatus !== 'unresolved', + ) + ? 'resolved' + : 'blocked', + scope, + typeBinding, + bindings: usable, + imports, + claims: selected.claims, + }; + } + if (scope.lookupPolicy?.deferParentActivation) position = undefined; + id = scope.lookupPolicy?.parentScope ?? scope.parent; + } + return id === null ? ABSENT : { ...ABSENT, status: 'blocked' }; +} diff --git a/gitnexus-shared/src/scope-resolution/reference-site.ts b/gitnexus-shared/src/scope-resolution/reference-site.ts index 933e8d8a3..05191dc84 100644 --- a/gitnexus-shared/src/scope-resolution/reference-site.ts +++ b/gitnexus-shared/src/scope-resolution/reference-site.ts @@ -68,6 +68,12 @@ export type ReferenceKind = export type CallForm = 'free' | 'member' | 'constructor' | 'index'; export interface ReferenceSite { + /** Graph caller for a synthesized invocation whose callable is looked up elsewhere. + * Never changes lexical resolution from `inScope` (for example FastAPI Depends). */ + readonly callerScope?: ScopeId; + /** Lexical lookup anchor when it differs from the reference's structural owner. */ + readonly lookupScope?: ScopeId; + readonly lookupPurpose?: import('./types.js').LookupPurpose; /** The name being referenced (e.g., `'save'`, `'User'`, `'count'`). */ readonly name: string; /** diff --git a/gitnexus-shared/src/scope-resolution/registries/class-registry.ts b/gitnexus-shared/src/scope-resolution/registries/class-registry.ts index 332f5a34d..7e3c12e43 100644 --- a/gitnexus-shared/src/scope-resolution/registries/class-registry.ts +++ b/gitnexus-shared/src/scope-resolution/registries/class-registry.ts @@ -24,7 +24,11 @@ export interface ClassRegistry { * `scope`. Returns a confidence-ranked `Resolution[]`; consume `[0]` * for the best answer. */ - lookup(name: string, scope: ScopeId): readonly Resolution[]; + lookup( + name: string, + scope: ScopeId, + options?: Pick, + ): readonly Resolution[]; } export function buildClassRegistry(ctx: RegistryContext): ClassRegistry { @@ -34,8 +38,12 @@ export function buildClassRegistry(ctx: RegistryContext): ClassRegistry { ownerScopedContributor: null, }; return { - lookup(name: string, scope: ScopeId) { - return lookupCore(name, scope, params, ctx); + lookup( + name: string, + scope: ScopeId, + options: Pick = {}, + ) { + return lookupCore(name, scope, { ...params, lookupPurpose: 'type', ...options }, ctx); }, }; } diff --git a/gitnexus-shared/src/scope-resolution/registries/field-registry.ts b/gitnexus-shared/src/scope-resolution/registries/field-registry.ts index 9e6a7aa0f..141e4597b 100644 --- a/gitnexus-shared/src/scope-resolution/registries/field-registry.ts +++ b/gitnexus-shared/src/scope-resolution/registries/field-registry.ts @@ -17,7 +17,10 @@ import { lookupCore, type CoreLookupParams } from './lookup-core.js'; import type { OwnerScopedContributor, RegistryContext } from './context.js'; import { FIELD_KINDS } from './context.js'; -export interface FieldLookupOptions { +export interface FieldLookupOptions extends Pick< + CoreLookupParams, + 'lookupPosition' | 'lookupPurpose' +> { readonly explicitReceiver?: { readonly name: string }; readonly ownerScopedContributor?: OwnerScopedContributor; } @@ -30,6 +33,8 @@ export function buildFieldRegistry(ctx: RegistryContext): FieldRegistry { return { lookup(name: string, scope: ScopeId, options: FieldLookupOptions = {}) { const params: CoreLookupParams = { + lookupPosition: options.lookupPosition, + lookupPurpose: options.lookupPurpose, acceptedKinds: FIELD_KINDS, useReceiverTypeBinding: true, ownerScopedContributor: options.ownerScopedContributor ?? null, diff --git a/gitnexus-shared/src/scope-resolution/registries/lookup-core.ts b/gitnexus-shared/src/scope-resolution/registries/lookup-core.ts index 4c0e360a8..699b5185f 100644 --- a/gitnexus-shared/src/scope-resolution/registries/lookup-core.ts +++ b/gitnexus-shared/src/scope-resolution/registries/lookup-core.ts @@ -67,27 +67,23 @@ * - No caching. Callers that want memoization can wrap this function. */ +import { lookupLexicalName } from '../name-claims.js'; import type { NodeLabel } from '../../graph/types.js'; import type { SymbolDefinition } from '../symbol-definition.js'; -import type { - BindingRef, - Callsite, - DefId, - LookupParams, - Resolution, - Scope, - ScopeId, -} from '../types.js'; +import type { BindingRef, Callsite, DefId, LookupParams, Resolution, ScopeId } from '../types.js'; import type { OriginForTieBreak } from '../origin-priority.js'; import { composeEvidence, confidenceFromEvidence, type RawSignals } from './evidence.js'; import { compareByConfidenceWithTiebreaks, type TieBreakKey } from './tie-breaks.js'; import { lookupQualified } from './lookup-qualified.js'; import type { ArityVerdict, OwnerScopedContributor, RegistryContext } from './context.js'; +import { CLASS_KINDS } from './context.js'; // ─── Public entry point ───────────────────────────────────────────────────── /** Extended `LookupParams` narrowing `ownerScopedContributor` to the concrete shape. */ export interface CoreLookupParams extends Omit { + /** A separate namespace whose ownership is determined only by accepted kinds. */ + readonly independentKindNamespace?: boolean; readonly ownerScopedContributor: OwnerScopedContributor | null; /** Call-site description forwarded to `arityCompatibility`. Optional — for non-call lookups. */ readonly callsite?: Callsite; @@ -132,15 +128,22 @@ export function lookupCore( !IMPLICIT_RECEIVERS.includes(params.explicitReceiver.name); const lexicalShadowed = skipLexical ? false - : walkLexicalChain(name, startScope, acceptedKinds, ctx, perCandidate); + : walkLexicalChain(name, startScope, acceptedKinds, ctx, perCandidate, params); // ── Step 2: type-binding / MRO walk (methods/fields) ────────────────── - if (params.useReceiverTypeBinding && ctx.methodDispatch !== undefined) { + if ( + (!lexicalShadowed || perCandidate.size > 0 || skipLexical) && + params.useReceiverTypeBinding && + ctx.methodDispatch !== undefined + ) { walkReceiverTypeBinding(name, startScope, acceptedKinds, params, ctx, perCandidate); } // ── Step 3: owner-scoped contributor ────────────────────────────────── - if (params.ownerScopedContributor !== null) { + if ( + (!lexicalShadowed || perCandidate.size > 0 || skipLexical) && + params.ownerScopedContributor !== null + ) { seedFromOwnerScopedContributor( name, params.ownerScopedContributor, @@ -222,32 +225,50 @@ function walkLexicalChain( acceptedKinds: ReadonlySet, ctx: RegistryContext, perCandidate: Map, + params: CoreLookupParams, ): boolean { - let currentId: ScopeId | null = startScope; - let depth = 0; - const visited = new Set(); - - while (currentId !== null) { - if (visited.has(currentId)) return false; - visited.add(currentId); - - const scope: Scope | undefined = ctx.scopes.getScope(currentId); - if (scope === undefined) return false; - - const bindings = scope.bindings.get(name); - if (bindings !== undefined && bindings.length > 0) { - for (const binding of bindings) { - if (!acceptedKinds.has(binding.def.type)) continue; - recordLexicalHit(perCandidate, binding, depth); + if (params.independentKindNamespace === true) { + let id: ScopeId | null = startScope; + let depth = 0; + const visited = new Set(); + while (id !== null && !visited.has(id)) { + visited.add(id); + const scope = ctx.scopes.getScope(id); + if (scope === undefined) return true; + const bindings = (scope.bindings.get(name) ?? []).filter((binding) => + acceptedKinds.has(binding.def.type), + ); + if (bindings.length > 0) { + for (const binding of bindings) recordLexicalHit(perCandidate, binding, depth); + return true; } - return true; // hard shadow regardless of kind-filter survivorship + id = scope.lookupPolicy?.parentScope ?? scope.parent; + depth++; } - - currentId = scope.parent; - depth++; + return id !== null; } - - return false; + const options = { position: params.lookupPosition, purpose: params.lookupPurpose }; + const claim = lookupLexicalName(startScope, name, { scopes: ctx.scopes }, options); + if (claim.status !== 'absent') { + let depth = 0; + let owner = ctx.scopes.getScope(startScope); + const visited = new Set(); + while (owner !== undefined && owner.id !== claim.scope?.id && !visited.has(owner.id)) { + visited.add(owner.id); + const parent = owner.lookupPolicy?.parentScope ?? owner.parent; + owner = parent === null ? undefined : ctx.scopes.getScope(parent); + depth++; + } + for (const binding of claim.bindings) { + if (acceptedKinds.has(binding.def.type)) recordLexicalHit(perCandidate, binding, depth); + } + return true; + } + const rootName = name.split('.', 1)[0]!; + return ( + rootName !== name && + lookupLexicalName(startScope, rootName, { scopes: ctx.scopes }, options).status !== 'absent' + ); } function recordLexicalHit( @@ -310,12 +331,12 @@ function resolveReceiverOwner( // ready resolveTypeRef call (that module is separate), we do a direct // lookup and trust the caller to have populated the binding. if (params.explicitReceiver !== undefined) { - return lookupReceiverType(startScope, params.explicitReceiver.name, ctx); + return lookupReceiverType(startScope, params.explicitReceiver.name, ctx, params); } // Implicit `self` / `this` — the scope's typeBindings should carry it. for (const implicitName of IMPLICIT_RECEIVERS) { - const owner = lookupReceiverType(startScope, implicitName, ctx); + const owner = lookupReceiverType(startScope, implicitName, ctx, params); if (owner !== undefined) return owner; } return undefined; @@ -353,38 +374,38 @@ function lookupReceiverType( startScope: ScopeId, receiverName: string, ctx: RegistryContext, + params: CoreLookupParams, ): DefId | undefined { - let currentId: ScopeId | null = startScope; - const visited = new Set(); - while (currentId !== null) { - if (visited.has(currentId)) return undefined; - visited.add(currentId); - - const scope = ctx.scopes.getScope(currentId); - if (scope === undefined) return undefined; - - const typeRef = scope.typeBindings.get(receiverName); - if (typeRef !== undefined) { - // rawName must resolve to a def via qualifiedNames; if it doesn't, we - // can't claim the receiver type. No fallback — that's what - // `resolveTypeRef` would do, but we keep this path lean and let - // callers pre-resolve if they want the richer semantics. - const candidateIds = ctx.qualifiedNames.get(typeRef.rawName); - if (candidateIds.length === 1) return candidateIds[0]; - // Ambiguous (≥ 2) or missing (0) — caller must pre-resolve via - // `resolveTypeRef` (#916) if they want the richer semantics. We - // intentionally do NOT re-implement a simple-name fallback here. - return undefined; - } - // The scope binds this receiver itself but carries no type for it — a - // JS/TS ordinary `function` whose `this` is bound at call time, not the - // enclosing instance (#2701). Stop rather than borrowing an enclosing - // scope's binding; see `Scope.ownsReceivers`. Mirrors the same gate in - // the ingestion-side twin of this walk, `findReceiverTypeBinding`. - if (scope.ownsReceivers?.has(receiverName) === true) return undefined; - currentId = scope.parent; + const claim = lookupLexicalName( + startScope, + receiverName, + { scopes: ctx.scopes }, + { + position: params.lookupPosition, + purpose: 'value', + }, + ); + const typeRef = claim.typeBinding; + if (typeRef === undefined) return undefined; + const typeClaim = lookupLexicalName( + typeRef.declaredAtScope, + typeRef.rawName, + { scopes: ctx.scopes }, + { + position: typeRef.lookupPosition, + purpose: typeRef.lookupPurpose ?? 'type', + }, + ); + if (typeClaim.status !== 'absent') { + const candidates = new Map( + typeClaim.bindings + .filter((binding) => CLASS_KINDS.includes(binding.def.type)) + .map((binding) => [binding.def.nodeId, binding.def]), + ); + return candidates.size === 1 ? candidates.keys().next().value : undefined; } - return undefined; + const candidateIds = ctx.qualifiedNames.get(typeRef.rawName); + return candidateIds.length === 1 ? candidateIds[0] : undefined; } function collectOwnedMembers( diff --git a/gitnexus-shared/src/scope-resolution/registries/macro-registry.ts b/gitnexus-shared/src/scope-resolution/registries/macro-registry.ts index 0eaf0e96a..f135bd34d 100644 --- a/gitnexus-shared/src/scope-resolution/registries/macro-registry.ts +++ b/gitnexus-shared/src/scope-resolution/registries/macro-registry.ts @@ -32,6 +32,7 @@ export interface MacroRegistry { export function buildMacroRegistry(ctx: RegistryContext): MacroRegistry { const params: CoreLookupParams = { acceptedKinds: MACRO_KINDS, + independentKindNamespace: true, useReceiverTypeBinding: false, ownerScopedContributor: null, }; diff --git a/gitnexus-shared/src/scope-resolution/registries/method-registry.ts b/gitnexus-shared/src/scope-resolution/registries/method-registry.ts index ed206d164..f0b14b5b9 100644 --- a/gitnexus-shared/src/scope-resolution/registries/method-registry.ts +++ b/gitnexus-shared/src/scope-resolution/registries/method-registry.ts @@ -23,7 +23,10 @@ import { METHOD_KINDS } from './context.js'; * concise while still exposing the explicit-receiver + owner-contributor + * arity knobs the RFC algorithm needs. */ -export interface MethodLookupOptions { +export interface MethodLookupOptions extends Pick< + CoreLookupParams, + 'lookupPosition' | 'lookupPurpose' +> { /** Call-site arity for `provider.arityCompatibility`. */ readonly callsite?: Callsite; /** Explicit receiver (e.g., `user` in `user.save()`). See §4.1. */ @@ -40,6 +43,8 @@ export function buildMethodRegistry(ctx: RegistryContext): MethodRegistry { return { lookup(name: string, scope: ScopeId, options: MethodLookupOptions = {}) { const params: CoreLookupParams = { + lookupPosition: options.lookupPosition, + lookupPurpose: options.lookupPurpose, acceptedKinds: METHOD_KINDS, useReceiverTypeBinding: true, ownerScopedContributor: options.ownerScopedContributor ?? null, diff --git a/gitnexus-shared/src/scope-resolution/resolve-type-ref.ts b/gitnexus-shared/src/scope-resolution/resolve-type-ref.ts index 793322688..88d130df4 100644 --- a/gitnexus-shared/src/scope-resolution/resolve-type-ref.ts +++ b/gitnexus-shared/src/scope-resolution/resolve-type-ref.ts @@ -38,7 +38,8 @@ import type { NodeLabel } from '../graph/types.js'; import type { SymbolDefinition } from './symbol-definition.js'; -import type { BindingRef, ScopeId, ScopeLookup, TypeRef } from './types.js'; +import { lookupLexicalName } from './name-claims.js'; +import type { BindingRef, ScopeLookup, TypeRef } from './types.js'; import type { DefIndex } from './def-index.js'; import type { QualifiedNameIndex } from './qualified-name-index.js'; @@ -100,36 +101,27 @@ const TYPE_KINDS: ReadonlySet = new Set([ // ─── Main entry point ────────────────────────────────────────────────────── export function resolveTypeRef(ref: TypeRef, ctx: ResolveTypeRefContext): SymbolDefinition | null { - // Phase 1: scope-chain walk anchored at the declaration site. - let currentId: ScopeId | null = ref.declaredAtScope; - const visited = new Set(); - - while (currentId !== null) { - // Cycle guard — a well-formed scope tree never loops, but a bug in the - // construction path should fail fast here rather than hanging. - if (visited.has(currentId)) return null; - visited.add(currentId); - - const scope = ctx.scopes.getScope(currentId); - if (scope === undefined) return null; // broken chain = unresolvable - - const bindings = scope.bindings.get(ref.rawName); - if (bindings !== undefined && bindings.length > 0) { - // At least one binding exists at this scope → it is the shadowing site. - // Either one of them qualifies, or the name is shadowed by a non-type. - for (const binding of bindings) { - if (!STRICT_ORIGINS.has(binding.origin)) continue; - if (TYPE_KINDS.has(binding.def.type)) { - return binding.def; - } - } - // Shadowed by a non-type / non-strict-origin binding. Fail fast — no - // global fallback, no walk to the parent. - return null; - } - - currentId = scope.parent; + const options = { position: ref.lookupPosition, purpose: ref.lookupPurpose ?? ('type' as const) }; + const claim = lookupLexicalName( + ref.declaredAtScope, + ref.rawName, + { scopes: ctx.scopes }, + options, + ); + if (claim.status !== 'absent') { + const candidates = claim.bindings.filter( + (binding) => STRICT_ORIGINS.has(binding.origin) && TYPE_KINDS.has(binding.def.type), + ); + const distinct = new Map(candidates.map((binding) => [binding.def.nodeId, binding.def])); + return distinct.size === 1 ? distinct.values().next().value! : null; } + const rootName = ref.rawName.split('.', 1)[0]!; + if ( + rootName !== ref.rawName && + lookupLexicalName(ref.declaredAtScope, rootName, { scopes: ctx.scopes }, options).status !== + 'absent' + ) + return null; // Phase 2: dotted fallback via `QualifiedNameIndex`. Only accept a unique // type-kind hit; anything ambiguous returns null (strict: no guesses). diff --git a/gitnexus-shared/src/scope-resolution/symbol-definition.ts b/gitnexus-shared/src/scope-resolution/symbol-definition.ts index 4334d1534..8d59e475d 100644 --- a/gitnexus-shared/src/scope-resolution/symbol-definition.ts +++ b/gitnexus-shared/src/scope-resolution/symbol-definition.ts @@ -10,6 +10,7 @@ */ import type { NodeLabel } from '../graph/types.js'; +import type { SourcePosition } from './types.js'; export interface ParameterTypeClass { /** Normalized base type, matching the coarse `parameterTypes` vocabulary when known. */ @@ -60,6 +61,9 @@ export interface SymbolDefinition { nodeId: string; filePath: string; type: NodeLabel; + /** Optional graph-source position (1-based line, 0-based column) for an + * embedded source buffer. Lexical scope identity remains in `nodeId`. */ + graphPosition?: SourcePosition; /** Canonical dot-separated qualified type name for class-like symbols * (e.g. `App.Models.User`). Falls back to the simple symbol name when no * package/namespace/module scope exists or no explicit qualified metadata is provided. */ diff --git a/gitnexus-shared/src/scope-resolution/types.ts b/gitnexus-shared/src/scope-resolution/types.ts index c62ae2625..d6a511945 100644 --- a/gitnexus-shared/src/scope-resolution/types.ts +++ b/gitnexus-shared/src/scope-resolution/types.ts @@ -112,6 +112,9 @@ export type ParsedImport = ParsedImportSyntax & { * Absent for legacy or synthesized imports. Binding semantics remain opt-in * through FinalizeHooks.importsBindAtLexicalScope. */ readonly declaredAtScope?: ScopeId; + readonly atRange?: Range; + /** Per-import lexical binding override; useful when a provider also has file-wide dependencies. */ + readonly bindsAtLexicalScope?: boolean; }; type ParsedImportSyntax = @@ -526,6 +529,7 @@ export interface Callsite { * bounded-fixpoint linking (RFC §3.2). */ export interface ImportEdge { + readonly atRange?: Range; /** How this scope sees the imported name (after alias). */ readonly localName: string; /** Namespace-import alias syntax, retained for providers whose unaliased imports bind a path prefix. */ @@ -595,6 +599,9 @@ export interface ImportEdge { * stamping first-class instead of reconstructing provenance from a side table. */ export interface BindingRef { + readonly availableFrom?: SourcePosition; + /** Source declaration range, independent of the target definition's identity. */ + readonly declarationRange?: Range; readonly def: SymbolDefinition; readonly origin: 'local' | 'import' | 'namespace' | 'wildcard' | 'reexport'; /** Non-null for non-local origins; carries the `ImportEdge` that brought the name into this scope. */ @@ -618,6 +625,10 @@ export interface BindingRef { * re-exports, and nested modules. Generics deferred to V2 via `typeArgs`. */ export interface TypeRef { + /** Declaration that binds this value, independent of its type's lookup anchor. */ + readonly bindingRange?: Range; + readonly lookupPosition?: SourcePosition; + readonly lookupPurpose?: LookupPurpose; /** * The type name AFTER the language's capture-time normalization — NOT * necessarily what the source says. Every provider's `interpretTypeBinding` @@ -665,6 +676,49 @@ export interface TypeRef { // ─── §2.2 Scope ───────────────────────────────────────────────────────────── +export interface SourcePosition { + readonly startLine: number; + readonly startCol: number; +} + +export type LookupPurpose = 'value' | 'type'; + +/** Provider-owned name ownership; a claim does not require an indexed target. */ +export interface NameClaim { + readonly name: string; + readonly range: Range; + readonly kind: 'binding' | 'import' | 'blocked'; + /** Proven compatible declarations may accumulate their import paths. */ + readonly merge?: boolean; + /** Hoisted declarations rank before active ordered rebinding at this scope. */ + readonly hoisted?: boolean; + readonly purpose?: LookupPurpose | 'both'; + readonly availableFrom?: SourcePosition; + readonly inactive?: 'blocked' | 'outer' | 'module'; + readonly redirect?: 'module' | 'enclosing-function'; +} + +export interface ScopeLookupPolicy { + /** All lexical declarations have claims; inferred assignments alone do not declare names. */ + readonly nameClaimsComplete?: boolean; + /** Reference scopes already identify the evaluating environment; do not infer + * a nested caller from physical source containment. */ + readonly callerScopeIsAuthoritative?: boolean; + /** Exact lexical self names retained when other bindings are skipped by children. */ + readonly visibleNamesFromChildren?: readonly string[]; + readonly parentScope?: ScopeId; + /** The scope's locals are not an enclosing lexical environment. */ + readonly skipFromChildren?: boolean; + /** Child execution is deferred, so source order in parents cannot imply a TDZ. */ + readonly deferParentActivation?: boolean; +} + +export interface NameLookupOptions { + readonly skipEnclosingClasses?: boolean; + readonly position?: SourcePosition; + readonly purpose?: LookupPurpose; +} + /** * The canonical lexical-scope node. Forms the spine of the SemanticModel. * @@ -672,6 +726,8 @@ export interface TypeRef { * — deterministic, stable across reparses of the same source, interned. */ export interface Scope { + readonly nameClaims?: readonly NameClaim[]; + readonly lookupPolicy?: ScopeLookupPolicy; readonly id: ScopeId; readonly parent: ScopeId | null; readonly kind: ScopeKind; @@ -762,7 +818,7 @@ export interface Resolution { * during the emit phase. */ export interface Reference { - /** Innermost lexical scope containing `atRange`. */ + /** Scope owning the graph source; may differ from lexical lookup for a synthesized call. */ readonly fromScope: ScopeId; readonly toDef: DefId; /** Location of the reference in source. */ @@ -818,6 +874,8 @@ export type RegistryContributor = unknown; * RFC §4.4 for per-registry specializations. */ export interface LookupParams { + readonly lookupPosition?: SourcePosition; + readonly lookupPurpose?: LookupPurpose; readonly acceptedKinds: readonly NodeLabel[]; /** Class lookups: false. Method/Field lookups: true. */ readonly useReceiverTypeBinding: boolean; diff --git a/gitnexus/bench/import-target/baselines.json b/gitnexus/bench/import-target/baselines.json index 006014d18..85f84fa21 100644 --- a/gitnexus/bench/import-target/baselines.json +++ b/gitnexus/bench/import-target/baselines.json @@ -97,6 +97,7 @@ "zig": 7 }, "heap_ceiling_bytes": { + "rust": 31300000, "kotlin": 6800000, "go": 4497696, "dart": 11751300, @@ -113,6 +114,7 @@ "_heap_reading_note": "heap_reading_bytes records each measured large-corpus reading so the 0.5x floor is independent of its ceiling. Ceilings use the standard 1.5x allowance for cross-platform and Node heap-accounting differences; re-baseline the reading and ceiling together.", "_kotlin_declared_package_gate": "#2960 replaces Kotlin\u0027s path-suffix cascade with one declared-package/module-binding index. The 32000-file arm measured 4516944 retained bytes (4.31 MiB), with a 6800000-byte ceiling. Correctness and reuse are pinned separately by kotlin-module-resolution.test.ts, external-import-conformance.test.ts, the shared index-reuse contract, and bench/kotlin-import-target.", "heap_reading_bytes": { + "rust": 20853888, "kotlin": 4516944, "go": 2998464, "dart": 7834200, @@ -130,7 +132,6 @@ "heap_bound_bytes": { "cobol": 6200000, "swift": 5200000, - "rust": 1048576, "javascript": 1048576, "typescript": 1048576, "vue": 1048576, @@ -673,6 +674,12 @@ } }, "rust": { + "_lexical_context_3532": "The fifth argument now reaches Rust's lexical module workspace index. Existing scale-arm counts and fingerprints remain unchanged. A declared sibling module resolves locally with context and to the crate-root decoy without it. Retained heap at 8000/32000 files measured 5186248/20853888 bytes (linear ratio 1.005); promote Rust from bounded to budgeted with the standard 1.5x ceiling and the existing floor/ratio gates.", + "context": { + "target": "local::X", + "with_context": "src/parent/local.rs", + "without_context": "src/local.rs" + }, "small": { "files": 400, "imports": 3200, diff --git a/gitnexus/bench/import-target/measure.mjs b/gitnexus/bench/import-target/measure.mjs index 2690e9e48..60a73ffee 100644 --- a/gitnexus/bench/import-target/measure.mjs +++ b/gitnexus/bench/import-target/measure.mjs @@ -604,6 +604,8 @@ const HEAP_BUDGETED = [ 'go', 'cpp', 'objc', + // Lexical module lookup retains a workspace index over parsed Rust files. + 'rust', ]; // javascript, typescript and vue were budgeted here until #2953 and are now // BOUNDED, which is a demotion in gate strength and a promotion in what the @@ -632,7 +634,7 @@ const HEAP_BUDGETED = [ * parameters cannot observe a context and are not listed, so their timed * numbers stay on the three-argument shape. */ -const CONTEXT_LANGS = ['php', 'java', 'kotlin', 'python', 'swift', 'c', 'cpp']; +const CONTEXT_LANGS = ['php', 'java', 'kotlin', 'python', 'swift', 'rust', 'c', 'cpp']; /** * Needs `node --expose-gc` to force collection for a clean delta; without it @@ -1890,7 +1892,15 @@ function resolveOne(lang, from, target, pass) { { fromFile: from, allFilePaths, parsedFiles: pass.parsedFiles }, ); } - if (lang === 'rust') return resolveRustImportTarget(target, from, allFilePaths, undefined); + if (lang === 'rust') { + return resolveRustImportTarget( + target, + from, + allFilePaths, + pass.config, + contextFor(pass, { kind: 'named', localName: 'X', importedName: 'X', targetRaw: target }), + ); + } // Quotes already stripped — `configs/zig.ts` strips them before this call in // production too. `null` build config: this arm pins the path walk alone // (see the header); the config legs are gated by their own unit tests. @@ -2300,6 +2310,37 @@ function measureHeap(lang) { * argument at all. */ const CONTEXT_PROBE = { + // A declared sibling module outranks a crate-root file with the same name. + // Empty module tables on the scaling corpora keep their existing target set; + // this probe witnesses the lexical branch that reads the workspace index. + rust: { + from: 'src/parent/mod.rs', + target: 'local::X', + parsedFiles: (() => { + const from = probeFile('src/parent/mod.rs', [['Namespace', 'local']]); + const def = from.localDefs[0]; + return [ + { + ...from, + scopes: [ + { + id: from.moduleScope, + parent: null, + kind: 'Module', + filePath: from.filePath, + range: { startLine: 1, startCol: 0, endLine: 2, endCol: 0 }, + bindings: new Map([['local', [{ def, origin: 'local' }]]]), + ownedDefs: [def], + imports: [], + typeBindings: new Map(), + }, + ], + }, + probeFile('src/parent/local.rs', [['Struct', 'X']]), + probeFile('src/local.rs', [['Struct', 'X']]), + ]; + })(), + }, /** * `use function App\Ns0\Dup;` where the CLASS `Dup` lives in `Dup.php` and * the FUNCTION `Dup` lives in `Helpers.php`. PHP keeps the two in separate diff --git a/gitnexus/bench/python-scope/baseline-fingerprint.txt b/gitnexus/bench/python-scope/baseline-fingerprint.txt index 9bd636da6..2251b2ed8 100644 --- a/gitnexus/bench/python-scope/baseline-fingerprint.txt +++ b/gitnexus/bench/python-scope/baseline-fingerprint.txt @@ -1 +1 @@ -b6754e5362716211a348f3aafda8bbc00082e3fb0e45cb3aa79a1be5e3df1a8b +95a98cf485746489f3ecf32e9a6bc7ee40604fe156d4cdd3ce36bc40581f1629 diff --git a/gitnexus/bench/python-scope/measure.mjs b/gitnexus/bench/python-scope/measure.mjs index 7254c3543..5c4c4602d 100644 --- a/gitnexus/bench/python-scope/measure.mjs +++ b/gitnexus/bench/python-scope/measure.mjs @@ -155,7 +155,9 @@ function computeFingerprint() { const perFixtureDigests = []; for (const { key, absPath } of collectPythonFixtures()) { const src = fs.readFileSync(absPath, 'utf8'); - const matches = emitPythonScopeCaptures(src, absPath); + // Lexical policies carry scope IDs containing the input path. Keep the + // fingerprint independent of the checkout's absolute location. + const matches = emitPythonScopeCaptures(src, key); groups += matches.length; perFixtureDigests.push(`${key}\t${matches.length}\t${digestCaptures(matches)}`); } diff --git a/gitnexus/bench/receiver-resolution/baseline.json b/gitnexus/bench/receiver-resolution/baseline.json index f40307044..33396439e 100644 --- a/gitnexus/bench/receiver-resolution/baseline.json +++ b/gitnexus/bench/receiver-resolution/baseline.json @@ -199,11 +199,11 @@ } }, "countArm": { - "callDrops": 132, - "totalDropsAllKinds": 178, + "callDrops": 125, + "totalDropsAllKinds": 163, "bySiteKind": { - "call": 132, - "read": 27, + "call": 125, + "read": 19, "write": 19 }, "callDropsByExtension": { @@ -212,7 +212,6 @@ ".ts": 15, ".zig": 11, ".cs": 8, - ".cpp": 7, ".tsx": 6, ".go": 5, ".php": 4, @@ -223,15 +222,15 @@ ".swift": 1 }, "callDropsByShape": { - "chain-field": 60, - "chain-call": 35, + "chain-field": 56, + "chain-call": 32, "no-chain": 23, "<>": 11, "chain-mixed": 2, "chain-unwrap": 1 }, "callDropsByOrigin": { - "in-program": 62, + "in-program": 55, "external": 44, "unknown": 26 } diff --git a/gitnexus/bench/scope-capture/baselines.json b/gitnexus/bench/scope-capture/baselines.json index 1c2992688..addb18a9c 100644 --- a/gitnexus/bench/scope-capture/baselines.json +++ b/gitnexus/bench/scope-capture/baselines.json @@ -38,7 +38,8 @@ "_rebaselined": "#1919 open-language coverage: new lang-resolution fixtures + intended capture additions (F5/F9 c-cpp, F26/F28/F29 dart, F47/F48/F49/F51/F52 kotlin, F75/F79 swift). Fingerprint-only drift; scaling_ratio ~1.0 (linear, no perf regression)." }, "cpp": { - "fingerprint": "3aaee42523f02718795ba348782d78ec1418f35eff2e1b602422b192544fed96", + "_lexical_ownership_3532": "#3532: lexical ownership captures and fixture updates; fixture identities are relative to lang-resolution so serialized scope IDs are independent of checkout location. Full capture content remains hashed; scaling_budget is unchanged.", + "fingerprint": "f18beeadd100e5479c2997876bc541412822cf3a4ae33b0d4a6ea6c35cae7b0d", "scaling_budget": 1.5, "_rebaselined_2833_qualified_member_fields": "#2833 follow-up: the six per-qualifier-depth `field_declaration` type-binding rules for a QUALIFIED generic member are replaced by three depth-agnostic ones that match the outer `qualified_identifier` itself, with the qualifier reduced to its top-level tail in `interpret.ts` (`cppQualifiedTail`). This is a CAPTURE-LOGIC change and it moves the fingerprint in two places at once. (1) A qualified NON-generic member (`ns::Address addr;`, `std::string name;`) was captured by nothing at all and now binds \u2014 that is the whole +24 on the fixture corpus, every one of them a `std::string` member. (2) Qualifier depth is no longer enumerated, so `a::b::c::Repo` (depth 3+) is captured where the old rules stopped at 2. Capture-name histogram, cpp-* corpus (278 files): `@type-binding.field` 8 -> 32, `@type-binding.name` and `@type-binding.type` 401 -> 425; synthetic DAO-20: `@type-binding.field` 40 -> 60, `@type-binding.name` and `@type-binding.type` 61 -> 81 (= 20 entities x the one `std::string name;` member the DAO unit already declared). NO OTHER TAG MOVED in either set \u2014 not one `@declaration.*`, `@scope.*` or `@reference.*` count \u2014 which is the property that says three rules replaced six without widening what a field_declaration matches. Measured over the 13 cpp-* fixture repos whose sources gained a binding, the distinct CALLS edge set is byte-identical before and after (32 edges): a reduced tail that names no workspace class binds nothing. Prior bd47c82d09a83cbf0ac857f41876fa31d22304043735582e913bccde06cf2c1a -> db1156d81b3e3341faf5e938a4a34417f4fd246588b6150b4686481823262529; scaling 1.04 < 1.5.", "_rebaselined_2833_generic_member_fields": "#2833 review follow-up: the cpp DAO generator's unit gains two GENERIC member fields \u2014 `Repo repo;` (bare template_type) and `std::vector items;` (qualified_identifier wrapping a template_type) \u2014 plus the header declaring `template class Repo`. CORPUS CHANGE, NOT A CAPTURE-LOGIC CHANGE: no extractor edit accompanies it. It exists because the corpus had ZERO template-typed member fields and, across 279 cpp-* fixtures, not one qualified generic member either, so BOTH rounds of new `field_declaration` type-binding rules landed with a byte-identical cpp fingerprint \u2014 the gate was structurally blind to the exact thing being changed. Measured under the new corpus, the three states now differ: pre-#2833 query 0e7cbda71360b7ff35dd76091c77f288d6af6a5cfa9185ad85a372aae8c85191 (4521 groups) -> the three template_type field rules de07d8b5300ed867b460918e16b4d80259c7eb6efc1034d32bebe9ff7cab126d (4541) -> the six qualified rules bd47c82d09a83cbf0ac857f41876fa31d22304043735582e913bccde06cf2c1a (4561); under the OLD corpus all three were 856d02f3f9d22cb973877211100aee8e052d4bc545922f78704b1a21ce49ddcc. Capture-name histogram over the synthetic DAO-20: `@type-binding.field` 0 -> 40, `@declaration.field` 40 -> 80, `@type-binding.type`/`@type-binding.name` 20 -> 61, `@declaration.name` 104 -> 147 \u2014 40 = 20 entities x 2 fields, with the residual +1/+2/+3 attributable to the one-off header declaration; every `@reference.*` count is unchanged. Prior 856d02f3f9d22cb973877211100aee8e052d4bc545922f78704b1a21ce49ddcc -> bd47c82d09a83cbf0ac857f41876fa31d22304043735582e913bccde06cf2c1a; scaling 1.058 < 1.5. `c` is unaffected (3418cded..., unchanged).", @@ -77,7 +78,8 @@ "_rebaselined_1432_member_call_callee_name": "#1432 (Zig): the shared callable-flow reader no longer names a callee by simple name for a MEMBER call (`@callable-flow.direct-callee-name` requires a direct designator: `f(x)`, `ns.f(x)`), and a member call is a field-stored-callable invoke only when a MEMBER store (`o.f = handler`) or a declared callable-typed field is visible - a same-named plain binding no longer gates it. CAPTURE-EMISSION CHANGE, not fixture growth (fixture_count unchanged). Only drift: `string.Join(\", \", args)` (csharp-variadic-resolution/Utils/Logger.cs) loses `direct-callee-name|Join`; the argument fact itself is unchanged, capture_groups_fp 2657 (unchanged). Prior 2930ef49fdce984a4c051409880bddfe8445e30e1c6bf802bd90a0a0f8f6b094 -> 9c4d2ca55707a03185ea45e554462e81e6d83c9f8d0ea3cd63db5c362ad1bfec." }, "rust": { - "fingerprint": "e61653008ff2de506cfd47f905fa9eb22d82fbbfe94d2a1d8190c358211b57b7", + "_lexical_ownership_3532": "#3532: lexical ownership captures and fixture updates; fixture identities are relative to lang-resolution so serialized scope IDs are independent of checkout location. Full capture content remains hashed; scaling_budget is unchanged.", + "fingerprint": "8c9f271349ab0a35a32e6f064303d63e191e019daa6dab8c5fc80784717a3321", "scaling_budget": 1.5, "_rebaselined_generic_instantiation_2912": "#2912: RUST_SCOPE_QUERY tags trait-impl heritage with the instantiation the impl was written with (`impl Validator for V`), so interface dispatch can prune implementors of an instantiation the receiver cannot hold. Additive capture text on existing impl matches \u2014 the same matches are minted, carrying one more field \u2014 so this is digest drift, not a capture-set change: capture_groups_fp (3556) and fixture_count (202) are both unchanged, which is the check that no match appeared or vanished. Prior 116a971fee0004f340477aff69fa110a1d92bd8ba882d7c926483c6b1e8ca2b9 -> e61653008ff2de506cfd47f905fa9eb22d82fbbfe94d2a1d8190c358211b57b7; scaling 1.018 < 1.5. Only rust and dart move; the other 13 languages are byte-identical.", "_rebaselined_mod_node_identity_2745_review": "#2745 review: added rust-2742-mod-members, rust-2742-nested-mods and rust-2742-type-vs-module under lang-resolution for the container/owner-edge fix, nested inline modules, and the imported-type-vs-module precedence. emitRustScopeCaptures is unchanged \u2014 verified by removing ONLY those three fixture dirs and re-running, which reproduces the prior fingerprint exactly, so the shift is purely corpus growth (fixture_count 196 -> 202, capture_groups_fp 3432 -> 3556). Prior 90fda086a4e13aa069a5981f63ed58ab1c71f1ed3da5e1480a080e1992b0d3e5 -> 05acbaca48427e0d9e0793bcd0ce4057712d3716b5e7868189c12e05ef8dd300; scaling 1.022 local / 1.057 CI < 1.5. NOTE for the next fixture author: a new rust-* fixture drifts BOTH this bench baseline and the rust-captures-golden snapshot. Updating only the golden is how this reached CI red.", @@ -197,7 +199,8 @@ "capture_groups_fp": 680 }, "typescript": { - "fingerprint": "a7972d87abd253583dafa37443bee8ddf5c635d41b4dfbb0c831efda71d82b47", + "_lexical_ownership_3532": "#3532: lexical ownership captures and fixture updates; fixture identities are relative to lang-resolution so serialized scope IDs are independent of checkout location. Full capture content remains hashed; scaling_budget is unchanged.", + "fingerprint": "bccca22b59d4c4d8aa4766c69092f29912fa035f08f037a639de3beaa725eaf4", "_rebaselined_3446_mcp_tools_fixture": "#3446 adds typescript-mcp-tools/src/{handlers,server,tools}.ts: corpus growth only, with fixture_count 171 -> 174 and capture_groups_fp 2753 -> 2894. Excluding only that fixture directory restores the prior fingerprint 77c9b4ea654123a64972db8348190ec250b669472b150db65ea8c7f20b355467 exactly. The scope emitter, synthetic capture counts (4503/14403), scaling budget, and other language baselines are unchanged.", "_rebaselined_3190": "Capture matches now retain explicit ESM export/private evidence, including synthesized default HOCs; CommonJS surfaces remain undecided. Capture group counts unchanged. Scaling budget unchanged.", "scaling_budget": 1.5, @@ -226,7 +229,8 @@ "_rebaselined_3354_callable_alternatives": "#3354 callable alternatives: a callable chosen by a value-selecting source now flows every branch it can yield (`a ?? b`, `a || b`, `a or b`, `c ? a : b`, statement `if`, elvis), and an operator branch (`x == f || g`) stays opaque instead of seeding a qualified name. Verified by running the BASE (merge-base 233ca2849) and HEAD emitters over the SAME HEAD fixture corpus: every added or removed match is an `@callable-flow.*` match on one of those sources, and the pre-existing corpus is byte-identical (all other languages: zero delta). The rest of the drift is corpus growth from this PR's regression fixture, which this bench globs. Corpus growth: typescript-callable-alternatives/{index,compare,worker,sweep/index}.ts (+4 files, +288 groups). Emitter delta: +18 / -14: `??` / `||` / `&&` / ternary seeds and copies now name each operand (`&&` its right operand only); the 14 removed matches are the whole-expression seeds and copies carrying a qualified name of the compound source, including the comparison branches in compare.ts that now stay opaque. capture_groups_fp 2465 -> 2753, fixture_count 167 -> 171; synthetic counts unchanged; scaling 0.963 < 1.5. Prior 60e75bbe846f7200005f12c4d95c32e82d9d3feae9f2fd594e484670762b82b0 -> 77c9b4ea654123a64972db8348190ec250b669472b150db65ea8c7f20b355467." }, "javascript": { - "fingerprint": "b916c7072b30d09b4949803810604830ea1aca1cfdda0d9d9312f7cb22f7a10a", + "_lexical_ownership_3532": "#3532: lexical ownership captures and fixture updates; fixture identities are relative to lang-resolution so serialized scope IDs are independent of checkout location. Full capture content remains hashed; scaling_budget is unchanged.", + "fingerprint": "857b1257a001f8741412ea08a4f43d9253cc4addfa63421f3013b3ff77a390c9", "_rebaselined_3190": "Capture matches now retain explicit ESM export/private evidence, including synthesized default HOCs; CommonJS surfaces remain undecided. Capture group counts unchanged. Scaling budget unchanged.", "scaling_budget": 1.5, "_rebaselined_callable_flow_2522_review": "PR #2522 review hardening: callable operands retain expression/qualified identity and formals retain signature metadata. Prior b59fe8135b6a31a12bc3f872b224054b16592588153ae3661d03958d787c76f3 -> 479927409bbdd9852a36172c8260aa56df260e99129a7a9c20a0d1903dd5538b; scaling 1.050 < 1.5.", @@ -268,7 +272,8 @@ "_rebaselined_3354_callable_alternatives": "#3354 callable alternatives: a callable chosen by a value-selecting source now flows every branch it can yield (`a ?? b`, `a || b`, `a or b`, `c ? a : b`, statement `if`, elvis), and an operator branch (`x == f || g`) stays opaque instead of seeding a qualified name. Verified by running the BASE (merge-base 233ca2849) and HEAD emitters over the SAME HEAD fixture corpus: every added or removed match is an `@callable-flow.*` match on one of those sources, and the pre-existing corpus is byte-identical (all other languages: zero delta). The rest of the drift is corpus growth from this PR's regression fixture, which this bench globs. Corpus growth: kotlin-callable-alternatives/App.kt (+1 file, +71 groups). Emitter delta: +8 / -2: elvis and `if` expression (bare and braced) seeds and copies now name each operand; the 2 removed matches are the whole-expression seed and copy carrying a qualified name of the compound source; the multi-statement `if` stays opaque. capture_groups_fp 2563 -> 2634, fixture_count 141 -> 142; synthetic counts unchanged; scaling 1.17 < 1.5. Prior a9d3f0db7547ff47856159debf15d2a6f427efca97a6af27b4004174ed432132 -> f3fefe058484823761b52df5715f41d65fa4ff0223843bcc86893d7b0a6b3de2." }, "typescript-deep-chain": { - "fingerprint": "c4dd97ad7be50a554930237f79035993ffa67957de6dc8c1d6606518071204b2", + "_lexical_ownership_3532": "#3532: lexical ownership captures and fixture updates; fixture identities are relative to lang-resolution so serialized scope IDs are independent of checkout location. Full capture content remains hashed; scaling_budget is unchanged.", + "fingerprint": "3f41a3c6d9161f95c0c906cc979e6ef853c25b39052c5cab274dba2aabf270ba", "scaling_budget": 1.5, "_added": "#3354 review: one TS `handler || w === \"k0\" || \u2026` chain, operand count = entity count (250 -> 800). Guards callable-flow value-alternative expansion (valueAlternatives + per-leaf visibility walks) against per-leaf cost that grows with chain depth: before the fix this measured 7.5-8.0 (super-linear; 8000 operands overflowed the stack), after it 1.11-1.22. Synthetic-only fingerprint (no fixture corpus); identical before and after the fix." } diff --git a/gitnexus/bench/scope-capture/measure.mjs b/gitnexus/bench/scope-capture/measure.mjs index faf60e357..b54bdefe4 100644 --- a/gitnexus/bench/scope-capture/measure.mjs +++ b/gitnexus/bench/scope-capture/measure.mjs @@ -389,7 +389,8 @@ function measureLang(lang) { const fixtures = lang.fixturePrefix === undefined ? [] : collectFixtures(lang.fixturePrefix, lang.exts); for (const { key, absPath } of fixtures) { - const matches = lang.emit(fs.readFileSync(absPath, 'utf8'), absPath); + // Scope policies contain file identities; hash stable fixture paths. + const matches = lang.emit(fs.readFileSync(absPath, 'utf8'), key); groups += matches.length; perFixture.push(`${key}\t${matches.length}\t${digestCaptures(matches)}`); } diff --git a/gitnexus/bench/scope-emission/baselines.json b/gitnexus/bench/scope-emission/baselines.json index e9f0fff8d..b704d10cc 100644 --- a/gitnexus/bench/scope-emission/baselines.json +++ b/gitnexus/bench/scope-emission/baselines.json @@ -1,8 +1,8 @@ { - "_comment": "Baselines for bench/scope-emission/measure.mjs --check (#2699), one entry per language. `scopes` is an EXACT count over a synthetic corpus fixed in measure.mjs — a correctness gate, not a timing one, so drift means the emitted scope set moved and must be explained, never re-baselined to make CI green. The two emit-side filters this guards (function-body blocks, and blocks that declare no binding) cut block scopes 19389 -> 5331 on a 762-file TypeScript corpus and took the block-scope overhead from ~+10% to ~+2% of analyze wall time. `@scope.block` = 400 is 2 per module: only the two `if`/`else` branches that declare `const chosen`. If that number jumps, the filters regressed and every scope-chain walk in every function got deeper. BOTH languages are baselined because the filters are implemented twice — FUNCTION_BODY_OWNER_TYPES in typescript/captures.ts and JS_FUNCTION_BODY_OWNER_TYPES in javascript/captures.ts, each with its own blockDeclaresBinding — so a TypeScript-only gate would let a JavaScript-only regression ship green. The two agree exactly on this corpus; that is a measured result, not an invariant the gate depends on. `emit_ms_budget` carries deliberate headroom for shared CI runners and exists to catch an order-of-magnitude regression, not a few percent.", + "_comment": "Baselines for bench/scope-emission/measure.mjs --check (#2699, #3532). Exact scope counts over the fixed synthetic corpus guard function-body suppression and removal of blocks without bindings. Each module has four Block scopes: two const-chosen branches, the lexical for environment, and the catch environment. Lookup-policy and name-claims captures decorate scopes and are excluded from this scope census; bench/scope-capture pins their full content. TypeScript and JavaScript retain separate baselines. emit_ms_budget is unchanged and carries headroom for shared CI runners.", "typescript": { "scopes": { - "@scope.block": 400, + "@scope.block": 800, "@scope.class": 200, "@scope.function": 1400, "@scope.module": 200 @@ -11,7 +11,7 @@ }, "javascript": { "scopes": { - "@scope.block": 400, + "@scope.block": 800, "@scope.class": 200, "@scope.function": 1400, "@scope.module": 200 diff --git a/gitnexus/bench/scope-emission/measure.mjs b/gitnexus/bench/scope-emission/measure.mjs index 078bdcd8f..aaf6247b7 100644 --- a/gitnexus/bench/scope-emission/measure.mjs +++ b/gitnexus/bench/scope-emission/measure.mjs @@ -24,7 +24,8 @@ * `blockDeclaresBinding` and its own `BLOCK_BINDING_CHILD_TYPES` — so a * TypeScript-only bench would let a JavaScript-only regression ship green. * - * On this corpus the two currently agree exactly (2 blocks per module, 2200 + * Loop and catch environments also carry lexical ownership. On this corpus + * the two currently agree exactly (4 blocks per module, 2600 * scopes). That is a measured result, not a required invariant: the fixtures * are structurally parallel and the TS-only syntax they drop carries no extra * scopes. Each language is still gated against its OWN baseline, because the @@ -54,7 +55,8 @@ const { emitJsScopeCaptures } = * * Deliberately mixes the shapes the filters discriminate between: * - function/method/arrow bodies → block scope must be SUPPRESSED - * - `if`/`else`/`for`/`while`/`try` → suppressed when they declare nothing + * - empty `if`/`else`/`while`/`try` → suppressed + * - lexical `for` and `catch` → environment scope REQUIRED * - blocks declaring `let`/`const` → block scope REQUIRED (shadowing) * - a block declaring only `var` → suppressed (`var` hoists past it) */ @@ -164,6 +166,8 @@ const measure = ({ ext, emit, moduleSource }) => { for (const { path, source } of corpus) { for (const match of emit(source, path)) { for (const key of Object.keys(match)) { + // These decorate an existing scope; they do not create scopes. + if (key === '@scope.lookup-policy' || key === '@scope.name-claims') continue; if (key.startsWith('@scope.')) counts.set(key, (counts.get(key) ?? 0) + 1); } } diff --git a/gitnexus/scripts/cross-platform-tests.ts b/gitnexus/scripts/cross-platform-tests.ts index c949b2f44..e7dc5dc6e 100644 --- a/gitnexus/scripts/cross-platform-tests.ts +++ b/gitnexus/scripts/cross-platform-tests.ts @@ -292,6 +292,10 @@ const WORKER_THREADS = [ 'test/unit/community-processor.test.ts', 'test/integration/worker-pool.test.ts', 'test/integration/parse-impl-quarantine-cache-skip.test.ts', + // Real parse workers and both on-disk parse stores must retain lexical + // local-import ownership on each platform's native grammar/runtime pair. + 'test/integration/local-import-worker-parity.test.ts', + 'test/integration/resolvers/local-import-persistence.test.ts', ]; // Tree-sitter native addon smoke tests — verify that native grammars diff --git a/gitnexus/src/core/ingestion/cfg/collect.ts b/gitnexus/src/core/ingestion/cfg/collect.ts index 4e43fbeea..0c4e0ce35 100644 --- a/gitnexus/src/core/ingestion/cfg/collect.ts +++ b/gitnexus/src/core/ingestion/cfg/collect.ts @@ -83,11 +83,17 @@ function shiftCfgLines(cfg: FunctionCfg, offset: number): FunctionCfg { }; } -function remapCfgLines(cfg: FunctionCfg, mapLine: (row: number) => number): FunctionCfg { +function remapCfgLines( + cfg: FunctionCfg, + mapLine: (row: number) => number, + siteCoordinates: 'source' | 'parse-buffer', +): FunctionCfg { // CFG visitors store 1-based source lines; `mapLine` maps 0-based tree-sitter rows. const map1 = (line: number): number => mapLine(line - 1) + 1; const mapSite = (site: SiteRecord): SiteRecord => - site.at !== undefined ? { ...site, at: [map1(site.at[0]), site.at[1]] } : site; + site.at !== undefined && siteCoordinates === 'source' + ? { ...site, at: [map1(site.at[0]), site.at[1]] } + : site; return { ...cfg, functionStartLine: map1(cfg.functionStartLine), @@ -115,6 +121,8 @@ export function collectFunctionCfgs( maxFunctionLines = 0, lineOffset = 0, mapLine?: (row: number) => number, + // Site anchors must match the coordinate space of scope reference captures. + siteCoordinates: 'source' | 'parse-buffer' = 'source', ): CollectedCfgs { const cfgs: FunctionCfg[] = []; let tooManyLines = 0; @@ -136,7 +144,11 @@ export function collectFunctionCfgs( try { const cfg = visitor.buildFunctionCfg(node, filePath); if (cfg) { - cfgs.push(mapLine ? remapCfgLines(cfg, mapLine) : shiftCfgLines(cfg, lineOffset)); + cfgs.push( + mapLine + ? remapCfgLines(cfg, mapLine, siteCoordinates) + : shiftCfgLines(cfg, lineOffset), + ); } } catch (err) { if (err instanceof CfgNestingDepthError) tooDeeplyNested++; diff --git a/gitnexus/src/core/ingestion/language-provider.ts b/gitnexus/src/core/ingestion/language-provider.ts index 718d8693d..71089cb53 100644 --- a/gitnexus/src/core/ingestion/language-provider.ts +++ b/gitnexus/src/core/ingestion/language-provider.ts @@ -299,6 +299,9 @@ interface LanguageProviderConfig { */ readonly preprocessSource?: (sourceText: string, filePath: string) => string; + /** Select the native grammar for a pre-extracted embedded script. */ + readonly selectEmbeddedGrammar?: (scriptLanguage: string) => object; + /** * Runtime/compiler identity reconciliation for framework metadata. The * central importer owns ambiguity handling; providers only supply aliases @@ -870,6 +873,12 @@ interface LanguageProviderConfig { */ sourceMeta?: { readonly sourceKind?: 'full-file' | 'pre-extracted-script'; + /** Explicit language of an embedded script, interpreted by its provider. */ + readonly scriptLanguage?: string; + /** Row offset from an embedded parse buffer to graph-source positions. */ + readonly lineOffset?: number; + /** Original zero-based source row for each embedded buffer row. */ + readonly sourceLineMap?: readonly number[]; /** Python `.ipynb` only: JSON line segments for the pre-extracted buffer. */ readonly notebookSegments?: readonly NotebookLineSegment[]; }, diff --git a/gitnexus/src/core/ingestion/languages/cpp/adl.ts b/gitnexus/src/core/ingestion/languages/cpp/adl.ts index 4c9efb744..1c967c2ff 100644 --- a/gitnexus/src/core/ingestion/languages/cpp/adl.ts +++ b/gitnexus/src/core/ingestion/languages/cpp/adl.ts @@ -295,7 +295,14 @@ function buildAdlIndex( if (parentQName !== '' && parentQName !== qName) keys.push(parentQName); } } - for (const def of scope.ownedDefs) { + // Function definitions own their body scope structurally, while their + // ordinary declaration binds in this namespace. Include those lexical + // bindings alongside prototypes owned directly by the namespace. + const namespaceDefs = new Map(scope.ownedDefs.map((def) => [def.nodeId, def])); + for (const bindings of scope.bindings.values()) { + for (const binding of bindings) namespaceDefs.set(binding.def.nodeId, binding.def); + } + for (const def of namespaceDefs.values()) { if (def.type === 'Function' || def.type === 'Method') { const sn = adlSimpleName(def); pushFlat(idx.nsFunctionsBySimple, sn, def); diff --git a/gitnexus/src/core/ingestion/languages/cpp/callable-bindings.ts b/gitnexus/src/core/ingestion/languages/cpp/callable-bindings.ts new file mode 100644 index 000000000..3352055d7 --- /dev/null +++ b/gitnexus/src/core/ingestion/languages/cpp/callable-bindings.ts @@ -0,0 +1,167 @@ +/** Ordinary unqualified callable lookup and ADL barriers owned by C++. */ +import type { BindingRef, NameLookupOptions, ScopeId, SymbolDefinition } from 'gitnexus-shared'; +import type { ScopeResolutionIndexes } from '../../model/scope-resolution-indexes.js'; +import { lookupBindingsAt, lookupNameClaim } from '../../scope-resolution/scope/walkers.js'; +import { definitionIdPosition } from '../../scope-resolution/utils/definition-id.js'; + +/** + * ISO C++ `[basic.lookup.unqual]` §7: ADL is suppressed when ordinary + * unqualified lookup finds: + * - a name that is NOT a function or function template, OR + * - a block-scope function declaration that is NOT a using-declaration. + * + * Combined walker that stops at the **nearest scope** where `name` has any + * binding (callable or non-callable) and returns: + * - `callables`: Function/Method/Constructor defs found at that scope + * - `nonCallableFound`: a non-function binding was present (variable, class, etc.) + * - `blockScopeDeclFound`: a callable was found at a Function or Block scope + * (block-scope function declaration that blocks ADL) + * + * One pass, one stop — no divergence between callable collection and blocker + * detection. + */ +export function findCallableBindingsAndAdlBlocker( + startScope: ScopeId, + name: string, + scopes: ScopeResolutionIndexes, + options?: NameLookupOptions, +): { + callables: readonly SymbolDefinition[]; + nonCallableFound: boolean; + blockScopeDeclFound: boolean; +} { + // Use the same ownership and provenance selection as every other consumer. + // Deduplicating finalized and augmented refs before selection can discard the + // using-declaration range that proves a callable belongs to this scope. + const claim = lookupNameClaim(startScope, name, scopes, { + purpose: 'value', + ...options, + expandLocalBindings: (scope, local, indexed) => + (scope.kind === 'Module' || scope.kind === 'Namespace') && + local.every((binding) => binding.def.type === 'Function') + ? indexed.filter( + (binding) => + binding.def.type === 'Function' && binding.via?.kind === 'wildcard-expanded', + ) + : [], + }); + const isCallable = (binding: BindingRef): boolean => + binding.def.type === 'Function' || + binding.def.type === 'Method' || + binding.def.type === 'Constructor'; + const callables = new Map(); + for (const binding of normalizeIncludedCallableDeclarations(claim.bindings, scopes)) { + if (isCallable(binding)) callables.set(binding.def.nodeId, binding.def); + } + // C++ constructor-form calls share the class spelling. Local class lookup + // takes precedence over include refs, but its own constructor compatibility + // refs still belong to that selected entity. Defaulted C++ constructors can + // retain the parser's Function label; exact class ownership is authoritative. + const classOwners = new Set( + claim.bindings + .filter( + (binding) => + binding.def.type === 'Class' || + binding.def.type === 'Struct' || + binding.def.type === 'Record', + ) + .map((binding) => binding.def.nodeId), + ); + if (claim.status === 'resolved' && claim.scope !== undefined && classOwners.size > 0) { + for (const { def } of lookupBindingsAt(claim.scope.id, name, scopes)) { + if ( + (def.type === 'Constructor' || def.type === 'Method' || def.type === 'Function') && + def.ownerId !== undefined && + classOwners.has(def.ownerId) + ) + callables.set(def.nodeId, def); + } + } + return { + callables: [...callables.values()], + nonCallableFound: + claim.status === 'blocked' || claim.bindings.some((binding) => !isCallable(binding)), + // Imported functions introduced by using declarations do not suppress ADL. + blockScopeDeclFound: + (claim.scope?.kind === 'Function' || claim.scope?.kind === 'Block') && + claim.bindings.some((binding) => binding.origin === 'local' && isCallable(binding)), + }; +} + +const callableDefinitionScopes = new WeakMap< + ScopeResolutionIndexes['scopeTree'], + { + readonly files: ReadonlySet; + readonly anchors: ReadonlySet; + } +>(); + +/** Coalesce a literal-include prototype with its unique local definition. */ +function normalizeIncludedCallableDeclarations( + bindings: readonly BindingRef[], + scopes: ScopeResolutionIndexes, +): readonly BindingRef[] { + if (!bindings.some((binding) => binding.via?.kind === 'wildcard-expanded')) return bindings; + let definitions = callableDefinitionScopes.get(scopes.scopeTree); + if (definitions === undefined) { + const files = new Set(); + const anchors = new Set(); + for (const scope of scopes.scopeTree.byId.values()) { + files.add(scope.filePath); + if (scope.kind === 'Function') { + anchors.add(`${scope.filePath}\0${scope.range.startLine}\0${scope.range.startCol}`); + } + } + definitions = { files, anchors }; + callableDefinitionScopes.set(scopes.scopeTree, definitions); + } + const definitionIndex = definitions; + const hasBody = (def: SymbolDefinition): boolean => { + const position = definitionIdPosition(def.nodeId, def.filePath); + return ( + position !== undefined && + definitionIndex.anchors.has(`${def.filePath}\0${position.line}\0${position.column}`) + ); + }; + const localBySignature = new Map>(); + for (const binding of bindings) { + if (binding.origin !== 'local' || !hasBody(binding.def)) continue; + const signature = callableRedeclarationSignature(binding.def); + if (signature === undefined) continue; + const ids = localBySignature.get(signature) ?? new Set(); + ids.add(binding.def.nodeId); + localBySignature.set(signature, ids); + } + return bindings.filter((binding) => { + if ( + binding.via?.kind !== 'wildcard-expanded' || + !definitionIndex.files.has(binding.def.filePath) || + hasBody(binding.def) + ) + return true; + const signature = callableRedeclarationSignature(binding.def); + return signature === undefined || localBySignature.get(signature)?.size !== 1; + }); +} + +function callableRedeclarationSignature(def: SymbolDefinition): string | undefined { + if ( + def.type !== 'Function' || + def.ownerId !== undefined || + def.qualifiedName === undefined || + def.parameterCount === undefined || + def.parameterTypes === undefined || + def.parameterTypes.length !== def.parameterCount || + def.parameterTypes.some((type) => type === '') || + (def.typeParameters?.length ?? 0) > 0 || + def.templateConstraints !== undefined + ) + return undefined; + return JSON.stringify([ + def.namespacePrefix ?? '', + def.qualifiedName, + def.parameterCount, + def.parameterTypes, + def.parameterTypeClasses, + ]); +} diff --git a/gitnexus/src/core/ingestion/languages/cpp/capture-side-channel.ts b/gitnexus/src/core/ingestion/languages/cpp/capture-side-channel.ts index ad9766d46..f46d7f764 100644 --- a/gitnexus/src/core/ingestion/languages/cpp/capture-side-channel.ts +++ b/gitnexus/src/core/ingestion/languages/cpp/capture-side-channel.ts @@ -27,6 +27,7 @@ */ import type { ParsedFile } from 'gitnexus-shared'; +import { collectCppUsingDeclarations, type CppUsingDeclaration } from './using-bindings.js'; import { collectCppAdlSideChannel, applyCppAdlSideChannel, type CppAdlSideChannel } from './adl.js'; import { collectCppInlineNamespaceSideChannel, @@ -68,6 +69,7 @@ export interface CppCaptureSideChannel { readonly fileLocal: CppFileLocalSideChannel; readonly twoPhase: CppTwoPhaseSideChannel; readonly memberLookup: CppMemberLookupSideChannel; + readonly usingDeclarations?: readonly CppUsingDeclaration[]; } /** @@ -81,6 +83,7 @@ export function collectCppCaptureSideChannel(filePath: string): CppCaptureSideCh const fileLocal = collectCppFileLocalSideChannel(filePath); const twoPhase = collectCppTwoPhaseSideChannel(filePath); const memberLookup = collectCppMemberLookupSideChannel(filePath); + const usingDeclarations = collectCppUsingDeclarations(filePath); const isEmpty = adl.argInfoBySite.length === 0 && @@ -91,10 +94,19 @@ export function collectCppCaptureSideChannel(filePath: string): CppCaptureSideCh twoPhase.dependentBases.length === 0 && twoPhase.dependentPackBaseClasses.length === 0 && memberLookup.baseEdges.length === 0 && - memberLookup.memberUsings.length === 0; + memberLookup.memberUsings.length === 0 && + usingDeclarations.length === 0; if (isEmpty) return undefined; - return { kind: 'cpp', adl, inlineNamespaceRanges, fileLocal, twoPhase, memberLookup }; + return { + kind: 'cpp', + adl, + inlineNamespaceRanges, + fileLocal, + twoPhase, + memberLookup, + usingDeclarations, + }; } /** diff --git a/gitnexus/src/core/ingestion/languages/cpp/captures.ts b/gitnexus/src/core/ingestion/languages/cpp/captures.ts index ce1ed009c..0cfcccba6 100644 --- a/gitnexus/src/core/ingestion/languages/cpp/captures.ts +++ b/gitnexus/src/core/ingestion/languages/cpp/captures.ts @@ -24,6 +24,11 @@ import { markCppInlineNamespaceRange } from './inline-namespaces.js'; import { extractCppTemplateConstraints } from './constraint-extractor.js'; import { captureCppMemberLookupFacts } from './member-lookup.js'; import { CPP_BRACED_INIT_TYPE_PREFIX } from './conversion-rank.js'; +import { + attachCppUsingClaims, + recordCppUsingDeclaration, + resetCppUsingDeclarations, +} from './using-bindings.js'; import { logger } from '../../../logger.js'; import { synthesizeReceiverChainCapture } from '../../utils/receiver-chain-captures.js'; import { @@ -171,6 +176,7 @@ export function emitCppScopeCaptures( const rawMatches = getCppScopeQuery().matches(tree.rootNode); const out: CaptureMatch[] = []; + resetCppUsingDeclarations(filePath); // #2432: reset the per-file lookup index. The identifier-argument type // lookups below used to re-walk the AST per identifier (full-tree DFS in @@ -238,6 +244,7 @@ export function emitCppScopeCaptures( if (usingNode !== null) { const split = splitCppUsingDecl(usingNode); if (split !== null) { + recordCppUsingDeclaration(filePath, split); out.push(split); continue; } @@ -610,6 +617,16 @@ export function emitCppScopeCaptures( } } + const parameterNode = nodeMap['@type-binding.parameter']; + const parameterCapture = grouped['@type-binding.parameter']; + if (parameterNode !== undefined && parameterCapture !== undefined) { + // Parameter names in a prototype or nested function-pointer type are + // not locals in the containing function. Arity metadata still comes + // from the declaration AST, independently of these local type facts. + if (!isCppLexicalParameter(parameterNode)) continue; + grouped['@type-binding.lexical-parameter'] = parameterCapture; + } + // Structural receiver chain for a call whose receiver is itself an // expression, so resolution can type it by folding over structure // instead of re-parsing the receiver's source text. Self-gating: a @@ -638,9 +655,25 @@ export function emitCppScopeCaptures( captureCppMemberLookupFacts(tree.rootNode, filePath); out.push(...synthesizeCallableFlowCaptures(tree.rootNode, CPP_CALLABLE_CAPTURE_OPTIONS)); + attachCppUsingClaims(out, filePath); return out; } +/** Prototype and nested function-pointer parameter names do not bind the body. */ +function isCppLexicalParameter(node: SyntaxNode): boolean { + for (let parent = node.parent; parent !== null; parent = parent.parent) { + if (parent.type === 'function_definition' || parent.type === 'lambda_expression') return true; + if ( + parent.type === 'declaration' || + parent.type === 'field_declaration' || + parent.type === 'parameter_declaration' || + parent.type === 'optional_parameter_declaration' + ) + return false; + } + return false; +} + function cppFunctionDeclaratorSignature(node: SyntaxNode): CallableCaptureSignature | undefined { const declarator = cppFindDescendantOfType(node, 'function_declarator'); const parameters = declarator?.childForFieldName('parameters'); diff --git a/gitnexus/src/core/ingestion/languages/cpp/file-local-linkage.ts b/gitnexus/src/core/ingestion/languages/cpp/file-local-linkage.ts index e578b20d4..e11f32b8d 100644 --- a/gitnexus/src/core/ingestion/languages/cpp/file-local-linkage.ts +++ b/gitnexus/src/core/ingestion/languages/cpp/file-local-linkage.ts @@ -172,7 +172,7 @@ export function populateCppAnonymousNamespaceScopes(parsed: { * scope is reflected before any cross-file resolution pass consults the * set. * - * A def is "not globally visible" when its nearest structurally enclosing + * A def is "not globally visible" when its nearest lexical binding * scope is a `Namespace` or `Class` — those require qualification * (`ns::name`, `Class::method`) for cross-file unqualified lookup. * Module-scoped defs remain globally visible. @@ -183,6 +183,10 @@ export function populateCppNonGloballyVisible(parsed: { readonly id: ScopeId; readonly kind: string; readonly ownedDefs: readonly { readonly nodeId: string }[]; + readonly bindings?: ReadonlyMap< + string, + readonly { readonly def: { readonly nodeId: string } }[] + >; }[]; }): void { let set = nonGloballyVisibleNodeIds.get(parsed.filePath); @@ -204,8 +208,15 @@ export function populateCppNonGloballyVisible(parsed: { // unqualified lookup that does not go through #include, so dropping // the structural visibility exclusion here is safe. if (scope.kind === 'Namespace' && anonymousNamespaceScopeIds.has(scope.id)) continue; - for (const def of scope.ownedDefs) { - set.add(def.nodeId); + // Scope-creating definitions own themselves structurally. Their lexical + // binding is what distinguishes a namespace function from a file function. + if (scope.bindings !== undefined) { + for (const bindings of scope.bindings.values()) { + for (const binding of bindings) set.add(binding.def.nodeId); + } + } else { + // Compatibility for manually constructed records without binding facts. + for (const def of scope.ownedDefs) set.add(def.nodeId); } } } @@ -226,8 +237,7 @@ export function isCppDefGloballyVisible(filePath: string, nodeId: string): boole } /** - * Return the names visible through a C++ wildcard import (`#include` or - * `using namespace`). + * Return the names visible through a C++ header include. * * ## Contract * @@ -247,10 +257,9 @@ export function isCppDefGloballyVisible(filePath: string, nodeId: string): boole * only dot-qualifies `qualifiedName` for `Class` scopes. Namespace-nested * defs (`namespace ns { void foo(); }`) arrive in `localDefs` with * `qualifiedName === 'foo'` and `ownerId === undefined`, indistinguishable - * from a top-level free function. The structural truth lives in - * `Scope.ownedDefs`: each scope lists what it structurally owns; the - * Module scope owns only top-level symbols. We look the def up by - * `nodeId` against the scope tree to identify its owning kind. + * from a top-level free function. `Scope.bindings` records lexical ownership; + * a function's own body scope structurally owns its definition. Prefer the + * binding owner and retain ownedDefs for older records without binding facts. * * ## `localDefs` consumer survey (recorded for future maintainers) * @@ -306,16 +315,21 @@ export function expandCppWildcardNames( const target = moduleScopeIndex(parsedFiles).get(targetModuleScope); if (target === undefined) return []; - // Build nodeId → owning Scope map from the structural scope tree. - // `Scope.ownedDefs` is the canonical source of structural ownership; - // `localDefs` is its flattened union, which is why the original code - // leaked: walking only `localDefs` discards the owning-scope context. + // Build nodeId → owning Scope, retaining structural facts as a fallback + // before lexical binding ownership refines the definition's visibility. const ownerScopeByNodeId = new Map(); for (const scope of target.scopes) { for (const ownedDef of scope.ownedDefs) { ownerScopeByNodeId.set(ownedDef.nodeId, scope); } } + // Prefer lexical ownership over a definition's own body scope. A function + // inside a namespace has an owning Function scope and a Namespace binding. + for (const scope of target.scopes) { + for (const bindings of scope.bindings?.values() ?? []) { + for (const binding of bindings) ownerScopeByNodeId.set(binding.def.nodeId, scope); + } + } const seen = new Set(); const names: string[] = []; diff --git a/gitnexus/src/core/ingestion/languages/cpp/import-decomposer.ts b/gitnexus/src/core/ingestion/languages/cpp/import-decomposer.ts index 4b0820589..b290b6c06 100644 --- a/gitnexus/src/core/ingestion/languages/cpp/import-decomposer.ts +++ b/gitnexus/src/core/ingestion/languages/cpp/import-decomposer.ts @@ -77,6 +77,7 @@ export function splitCppUsingDecl(node: SyntaxNode): CaptureMatch | null { // lookup set; it is not a namespace import. The C++ member-lookup sidecar // captures it separately, so suppress import decomposition here. for (let parent = node.parent; parent !== null; parent = parent.parent) { + if (parent.type === 'function_definition' || parent.type === 'lambda_expression') break; if (parent.type === 'class_specifier' || parent.type === 'struct_specifier') return null; } @@ -109,6 +110,7 @@ export function splitCppUsingDecl(node: SyntaxNode): CaptureMatch | null { '@import.kind': syntheticCapture('@import.kind', node, 'wildcard'), '@import.source': syntheticCapture('@import.source', node, namespaceName), '@import.using-namespace': syntheticCapture('@import.using-namespace', node, 'true'), + '@import.using-decl': nodeToCapture('@import.using-decl', node), }; } @@ -124,16 +126,18 @@ export function splitCppUsingDecl(node: SyntaxNode): CaptureMatch | null { if (qualId === null) return null; // Extract the imported name (last identifier) and source (namespace part) - const nameNode = qualId.childForFieldName?.('name') ?? null; - const scopeNode = qualId.childForFieldName?.('scope') ?? null; - - const importedName = nameNode?.text ?? qualId.text.split('::').pop() ?? ''; - const source = scopeNode?.text ?? qualId.text.replace(new RegExp('::' + importedName + '$'), ''); + // Qualified identifiers nest on their name side in tree-sitter-cpp. Taking + // only the outer `scope` loses intermediate namespaces in a::b::member. + const separator = qualId.text.lastIndexOf('::'); + if (separator < 0) return null; + const importedName = qualId.text.slice(separator + 2).trim(); + const source = qualId.text.slice(0, separator).trim() || '::'; return { '@import.statement': nodeToCapture('@import.statement', node), '@import.kind': syntheticCapture('@import.kind', node, 'named'), '@import.source': syntheticCapture('@import.source', node, source), '@import.name': syntheticCapture('@import.name', node, importedName), + '@import.using-decl': nodeToCapture('@import.using-decl', node), }; } diff --git a/gitnexus/src/core/ingestion/languages/cpp/interpret.ts b/gitnexus/src/core/ingestion/languages/cpp/interpret.ts index 635b09264..774080965 100644 --- a/gitnexus/src/core/ingestion/languages/cpp/interpret.ts +++ b/gitnexus/src/core/ingestion/languages/cpp/interpret.ts @@ -3,15 +3,21 @@ import type { CaptureMatch, ParsedImport, ParsedTypeBinding, TypeRef } from 'git /** * Interpret a C++ import capture into a ParsedImport. * - * C++ has three import forms: - * 1. #include "file.h" → wildcard import (all symbols from header) - * 2. using namespace X; → wildcard import (all symbols from namespace X) - * 3. using X::name; → named import (single symbol from namespace X) + * Header includes become wildcard file imports. Namespace using captures + * are resolved separately against exact namespace members and their lexical + * declaration positions by populateCppUsingBindings. * * Angle `#include <...>` sets `isSystem`. The resolver searches header * paths for that form and does not use the basename index. */ export function interpretCppImport(captures: CaptureMatch): ParsedImport | null { + // Namespace using facts are linked by populateCppUsingBindings. They are + // not file imports: suffix-matching a namespace against a header is wrong. + if ( + captures['@import.using-decl'] !== undefined || + captures['@import.using-namespace'] !== undefined + ) + return null; const source = captures['@import.source']?.text; if (source === undefined) return null; @@ -19,13 +25,13 @@ export function interpretCppImport(captures: CaptureMatch): ParsedImport | null const kind = captures['@import.kind']?.text; if (kind === 'named') { - // using X::name — named import + // Compatibility for synthetic named-import captures without a using marker. const importedName = captures['@import.name']?.text; if (importedName === undefined) return null; return { kind: 'named', targetRaw: source, localName: importedName, importedName }; } - // #include or using namespace — wildcard import + // #include — wildcard file import return { kind: 'wildcard', targetRaw: source, isSystem }; } diff --git a/gitnexus/src/core/ingestion/languages/cpp/scope-resolver.ts b/gitnexus/src/core/ingestion/languages/cpp/scope-resolver.ts index e88581063..b3feeac8d 100644 --- a/gitnexus/src/core/ingestion/languages/cpp/scope-resolver.ts +++ b/gitnexus/src/core/ingestion/languages/cpp/scope-resolver.ts @@ -35,6 +35,7 @@ import { isCppDependentBaseMember, } from './two-phase-lookup.js'; import { populateCppAssociatedNamespaces, clearCppAdlState, pickCppAdlCandidates } from './adl.js'; +import { findCallableBindingsAndAdlBlocker } from './callable-bindings.js'; import { applyCppCaptureSideChannel } from './capture-side-channel.js'; import { clearCppInlineNamespaces, @@ -54,6 +55,11 @@ import { } from './member-lookup.js'; import { stripCppSpecifiers } from './interpret.js'; import { perFileSet } from '../../import-resolvers/per-file-set.js'; +import { + clearCppUsingDeclarations, + populateCppIncludedUsingClaims, + populateCppUsingBindings, +} from './using-bindings.js'; /** A pointee worth binding: a bare identifier, not `T**`, `T[]`, `A::B` or a * template spelling. Hoisted — a literal here would mint a fresh RegExp on @@ -115,6 +121,7 @@ export const cppScopeResolver: ScopeResolver = { clearCppInlineNamespaces(); clearCppUserDefinedConversions(); clearCppMemberLookupState(); + clearCppUsingDeclarations(); return loadCFamilyResolutionConfig(repoPath, CPP_HEADER_EXTENSIONS); }, @@ -207,6 +214,21 @@ export const cppScopeResolver: ScopeResolver = { populateCppDependentBases(parsedFiles); }, + populateWorkspaceReferences: (parsedFiles, context) => { + const allFilePaths = new Set(parsedFiles.map((parsed) => parsed.filePath)); + populateCppIncludedUsingClaims(parsedFiles, (parsed, imported) => + cppScopeResolver.resolveImportTarget( + imported.targetRaw, + parsed.filePath, + allFilePaths, + context.resolutionConfig, + { parsedFiles, parsedImport: imported }, + ), + ); + }, + + populateNamespaceSiblings: populateCppUsingBindings, + // Simple `isSuperReceiver` returns false for C++. Real super // classification is caller-context-dependent and lives in // `isSuperReceiverInContext` below — without scope context the @@ -356,43 +378,8 @@ export const cppScopeResolver: ScopeResolver = { // specializations with explicit type arguments contribute associated // namespaces. Function-pointer args and full conversion-ranking remain // excluded. - resolveAdlCandidates: (site, callerParsed, scopes, parsedFiles) => { - // `using ns::name;` introduces `name` into ordinary unqualified lookup. - // For template-class method bodies, lexical scope walks can miss this - // named-using visibility; recover by resolving the imported namespace - // member directly when the local call name matches a named using import. - const usingNamedHits: SymbolDefinition[] = []; - const seenUsing = new Set(); - for (const imp of callerParsed.parsedImports) { - if (imp.kind !== 'named') continue; - if (imp.localName !== site.name) continue; - const member = resolveCppQualifiedNamespaceMember( - imp.targetRaw, - imp.importedName, - parsedFiles, - scopes, - ); - if (member === undefined || member === 'ambiguous') continue; - if (seenUsing.has(member.nodeId)) continue; - seenUsing.add(member.nodeId); - usingNamedHits.push(member); - } - const adlHits = pickCppAdlCandidates(site, callerParsed, scopes, parsedFiles); - if (usingNamedHits.length === 0) return adlHits; - if (adlHits === undefined || adlHits.length === 0) return usingNamedHits; - const merged: SymbolDefinition[] = []; - const seen = new Set(); - for (const hit of usingNamedHits) { - seen.add(hit.nodeId); - merged.push(hit); - } - for (const hit of adlHits) { - if (seen.has(hit.nodeId)) continue; - seen.add(hit.nodeId); - merged.push(hit); - } - return merged; - }, + resolveAdlCandidates: pickCppAdlCandidates, + resolveOrdinaryCallables: findCallableBindingsAndAdlBlocker, // C++ qualified namespace-member resolution (U5 of plan 2026-05-13-001). // Handles `outer::foo()` where `outer` is a namespace (not a class). diff --git a/gitnexus/src/core/ingestion/languages/cpp/using-bindings.ts b/gitnexus/src/core/ingestion/languages/cpp/using-bindings.ts new file mode 100644 index 000000000..5874c116f --- /dev/null +++ b/gitnexus/src/core/ingestion/languages/cpp/using-bindings.ts @@ -0,0 +1,514 @@ +import type { + BindingRef, + Capture, + CaptureMatch, + NameClaim, + ParsedFile, + ParsedImport, + Range, + Scope, + ScopeId, +} from 'gitnexus-shared'; +import type { ScopeResolutionIndexes } from '../../model/scope-resolution-indexes.js'; +import { isCppInlineNamespaceScope } from './inline-namespaces.js'; + +/** Namespace lookup facts, deliberately distinct from preprocessor file imports. */ +export interface CppUsingDeclaration { + readonly namespace: string; + readonly name?: string; + readonly range: Range; +} + +const declarationsByFile = new Map(); + +export function clearCppUsingDeclarations(): void { + declarationsByFile.clear(); +} + +export function resetCppUsingDeclarations(filePath: string): void { + declarationsByFile.delete(filePath); +} + +export function recordCppUsingDeclaration(filePath: string, capture: CaptureMatch): void { + const namespace = capture['@import.source']?.text; + const range = capture['@import.statement']?.range; + if (namespace === undefined || range === undefined) return; + const name = capture['@import.name']?.text; + const declarations = declarationsByFile.get(filePath) ?? []; + declarations.push({ namespace, range, ...(name === undefined ? {} : { name }) }); + declarationsByFile.set(filePath, declarations); +} + +export function collectCppUsingDeclarations(filePath: string): readonly CppUsingDeclaration[] { + return declarationsByFile.get(filePath) ?? []; +} + +const compare = ( + a: Pick, + b: Pick, +) => a.startLine - b.startLine || a.startCol - b.startCol; +const end = (range: Range) => ({ startLine: range.endLine, startCol: range.endCol }); +const contains = (outer: Range, inner: Range): boolean => + compare(outer, inner) <= 0 && compare(end(inner), end(outer)) <= 0; + +/** Claims attach to the same innermost capture range as the shared extractor. */ +export function attachCppUsingClaims(captures: CaptureMatch[], filePath: string): void { + const scopes = captures.flatMap((match) => + Object.values(match) + .filter((capture) => + [ + '@scope.module', + '@scope.namespace', + '@scope.class', + '@scope.function', + '@scope.block', + ].includes(capture.name), + ) + .map((capture) => ({ match, capture })), + ); + const byOwner = new Map(); + const add = (claim: NameClaim, functionOnly = false, callable = false): void => { + let owner: (typeof scopes)[number] | undefined; + for (const candidate of scopes) { + if (functionOnly && candidate.capture.name !== '@scope.function') continue; + if ( + callable && + candidate.capture.name === '@scope.function' && + compare(candidate.capture.range, claim.range) === 0 && + compare(end(candidate.capture.range), end(claim.range)) === 0 + ) + continue; + if (!contains(candidate.capture.range, claim.range)) continue; + if (owner === undefined || contains(owner.capture.range, candidate.capture.range)) + owner = candidate; + } + if (owner === undefined) return; + const claims = byOwner.get(owner.match) ?? []; + // A namespace using-declaration extends the existing overload set, while a + // block using-declaration still hides functions declared in outer scopes. + if (callable && !claims.some((existing) => existing.name === claim.name && existing.merge)) + return; + claims.push(claim); + byOwner.set(owner.match, claims); + }; + for (const declaration of collectCppUsingDeclarations(filePath)) { + if (declaration.name === undefined) continue; + add({ + name: declaration.name, + range: declaration.range, + kind: 'binding', + merge: true, + availableFrom: end(declaration.range), + inactive: 'outer', + }); + } + for (const match of captures) { + const variable = match['@declaration.variable']; + const parameter = match['@type-binding.lexical-parameter']; + const callable = match['@declaration.function']; + const anchor = variable ?? parameter ?? callable; + const name = parameter !== undefined ? match['@type-binding.name'] : match['@declaration.name']; + if (anchor === undefined || name === undefined) continue; + add( + { + name: name.text, + range: anchor.range, + kind: 'binding', + availableFrom: end(name.range), + inactive: 'outer', + ...(callable === undefined ? {} : { merge: true }), + }, + parameter !== undefined, + callable !== undefined, + ); + } + for (const [match, claims] of byOwner) { + const anchor = Object.values(match).find((capture) => capture.name.startsWith('@scope.'))!; + (match as Record)['@scope.name-claims'] = { + name: '@scope.name-claims', + range: anchor.range, + text: JSON.stringify(claims), + }; + } +} + +function owningScope(parsed: ParsedFile, range: Range): Scope | undefined { + let owner: Scope | undefined; + for (const scope of parsed.scopes) { + if (contains(scope.range, range) && (owner === undefined || contains(owner.range, scope.range))) + owner = scope; + } + return owner; +} + +interface NamespaceNode { + readonly scope: Scope; + readonly inlineChildren: NamespaceNode[]; +} + +function usingDeclarations(parsed: ParsedFile): readonly CppUsingDeclaration[] { + return ( + ( + parsed.captureSideChannel as + | { usingDeclarations?: readonly CppUsingDeclaration[] } + | undefined + )?.usingDeclarations ?? collectCppUsingDeclarations(parsed.filePath) + ); +} + +function createNamespaceLookup( + parsedFiles: readonly ParsedFile[], + getScope: (id: ScopeId) => Scope | undefined, +) { + const paths = new Map(); + const nodes = new Map(); + const namespaces = new Map(); + const namespacePath = (scope: Scope): string => { + const cached = paths.get(scope.id); + if (cached !== undefined) return cached; + const parts: string[] = []; + let current: Scope | undefined = scope; + while (current !== undefined) { + if (current.kind === 'Namespace') { + const name = current.ownedDefs.find((def) => def.type === 'Namespace')?.qualifiedName; + if (name !== undefined) parts.unshift(name.split('.').pop()!); + } + current = current.parent === null ? undefined : getScope(current.parent); + } + const result = parts.join('.'); + paths.set(scope.id, result); + return result; + }; + for (const parsed of parsedFiles) { + for (const scope of parsed.scopes) { + if (scope.kind !== 'Namespace' && scope.kind !== 'Module') continue; + const path = namespacePath(scope); + if (path === '' && scope.kind !== 'Module') continue; + const node: NamespaceNode = { scope, inlineChildren: [] }; + nodes.set(scope.id, node); + const group = namespaces.get(path) ?? []; + group.push(node); + namespaces.set(path, group); + } + } + for (const node of nodes.values()) { + if (node.scope.parent !== null && isCppInlineNamespaceScope(node.scope.id)) { + nodes.get(node.scope.parent)?.inlineChildren.push(node); + } + } + return { namespacePath, namespaces }; +} + +function resolveUsingMembers( + declaration: CppUsingDeclaration, + owner: Scope, + filePath: string, + visible: ReadonlySet, + { namespacePath, namespaces }: ReturnType, +) { + const raw = declaration.namespace.replaceAll('::', '.').replace(/^\./, ''); + let prefix = declaration.namespace.startsWith('::') ? '' : namespacePath(owner); + let target: string | undefined; + while (true) { + const candidate = prefix === '' ? raw : `${prefix}.${raw}`; + if (namespaces.has(candidate)) { + target = candidate; + break; + } + if (prefix === '') break; + prefix = prefix.includes('.') ? prefix.slice(0, prefix.lastIndexOf('.')) : ''; + } + if (target === undefined) return undefined; + const members = new Map(); + const pending = [...(namespaces.get(target) ?? [])]; + while (pending.length > 0) { + const node = pending.pop()!; + if (!visible.has(node.scope.filePath)) continue; + pending.push(...node.inlineChildren); + for (const [name, refs] of node.scope.bindings) { + if (declaration.name !== undefined && declaration.name !== name) continue; + const bucket = members.get(name) ?? []; + for (const ref of refs) { + if (ref.def.type === 'Namespace') continue; + // A named using snapshots the declarations visible at that point. + if ( + declaration.name !== undefined && + ref.def.filePath === filePath && + ref.declarationRange !== undefined && + compare(ref.declarationRange, declaration.range) > 0 + ) + continue; + if (!bucket.some((entry) => entry.def.nodeId === ref.def.nodeId)) bucket.push(ref); + } + members.set(name, bucket); + } + } + return { target, members }; +} + +function createIncludeClosure(targetsOf: (filePath: string) => readonly string[]) { + const visibleByFile = new Map>(); + return (filePath: string): Set => { + const cached = visibleByFile.get(filePath); + if (cached !== undefined) return cached; + const visible = new Set(); + const pending = [filePath]; + while (pending.length > 0) { + const file = pending.pop()!; + if (visible.has(file)) continue; + visible.add(file); + pending.push(...targetsOf(file)); + } + visibleByFile.set(filePath, visible); + return visible; + }; +} + +/** Preserve unresolved header declarations in the ordinary pre-finalize scope facts. */ +export function populateCppIncludedUsingClaims( + parsedFiles: ParsedFile[], + resolveInclude: (parsed: ParsedFile, imported: ParsedImport) => string | readonly string[] | null, +): void { + const namedByFile = new Map< + string, + { owner: Scope; declarations: readonly CppUsingDeclaration[] } + >(); + for (const parsed of parsedFiles) { + const declarations = usingDeclarations(parsed).filter( + (declaration) => + declaration.name !== undefined && + owningScope(parsed, declaration.range)?.id === parsed.moduleScope, + ); + if (declarations.length === 0) continue; + const owner = parsed.scopes.find((scope) => scope.id === parsed.moduleScope); + if (owner !== undefined) namedByFile.set(parsed.filePath, { owner, declarations }); + } + if (namedByFile.size === 0) return; + + const scopes = new Map( + parsedFiles.flatMap((parsed) => parsed.scopes.map((scope) => [scope.id, scope] as const)), + ); + const namespaceLookup = createNamespaceLookup(parsedFiles, (id) => scopes.get(id)); + const includes = new Map(); + for (const parsed of parsedFiles) { + const edges: { target: string; range: Range }[] = []; + for (const imported of parsed.parsedImports) { + if (imported.kind !== 'wildcard' || imported.atRange === undefined) continue; + const target = resolveInclude(parsed, imported); + if (typeof target === 'string') edges.push({ target, range: imported.atRange }); + } + includes.set(parsed.filePath, edges); + } + const visibleFiles = createIncludeClosure((filePath) => + (includes.get(filePath) ?? []).map((edge) => edge.target), + ); + for (let index = 0; index < parsedFiles.length; index++) { + const parsed = parsedFiles[index]; + const added: NameClaim[] = []; + const seen = new Set(); + for (const edge of includes.get(parsed.filePath) ?? []) { + for (const includedFile of visibleFiles(edge.target)) { + if (includedFile === parsed.filePath) continue; + const named = namedByFile.get(includedFile); + if (named === undefined) continue; + for (const declaration of named.declarations) { + const name = declaration.name!; + const resolved = resolveUsingMembers( + declaration, + named.owner, + includedFile, + visibleFiles(parsed.filePath), + namespaceLookup, + ); + if ((resolved?.members.get(name)?.length ?? 0) > 0) continue; + const key = `${name}:${edge.range.startLine}:${edge.range.startCol}`; + if (seen.has(key)) continue; + seen.add(key); + added.push({ + name, + kind: 'blocked', + merge: true, + range: edge.range, + availableFrom: end(edge.range), + inactive: 'outer', + }); + } + } + } + if (added.length === 0) continue; + const module = parsed.scopes.find((scope) => scope.id === parsed.moduleScope); + if (module === undefined) continue; + const claims = [...(module.nameClaims ?? [])]; + // An inactive include claim must not erase earlier local declarations of + // the same name. Give those bindings their existing declaration evidence; + // the blocked include claim takes effect only at its later source position. + for (const name of new Set(added.map((claim) => claim.name))) { + for (const binding of module.bindings.get(name) ?? []) { + const range = binding.declarationRange; + if ( + range === undefined || + claims.some((claim) => claim.name === name && contains(claim.range, range)) + ) + continue; + claims.push({ + name, + kind: 'binding', + merge: true, + range, + availableFrom: { startLine: range.startLine, startCol: range.startCol }, + inactive: 'outer', + }); + } + } + // Extraction snapshots are frozen and may also live in the parse cache. + // Replace only affected records before registries and ScopeTree are built. + parsedFiles[index] = { + ...parsed, + scopes: parsed.scopes.map((scope) => + scope.id === parsed.moduleScope ? { ...scope, nameClaims: [...claims, ...added] } : scope, + ), + }; + } +} + +/** Materialize namespace identities once, then augment only the using's owner. */ +export function populateCppUsingBindings( + parsedFiles: readonly ParsedFile[], + indexes: ScopeResolutionIndexes, +): void { + if (!parsedFiles.some((parsed) => usingDeclarations(parsed).length > 0)) return; + const namespaceLookup = createNamespaceLookup(parsedFiles, (id) => + indexes.scopeTree.getScope(id), + ); + const { namespacePath } = namespaceLookup; + const byFile = new Map(parsedFiles.map((parsed) => [parsed.filePath, parsed])); + const visibleFiles = createIncludeClosure((filePath) => { + const parsed = byFile.get(filePath); + if (parsed === undefined) return []; + return (indexes.imports.get(parsed.moduleScope) ?? []).flatMap((edge) => + edge.targetFile !== null && edge.linkStatus !== 'unresolved' ? [edge.targetFile] : [], + ); + }); + const augmentations = indexes.bindingAugmentations as Map>; + const append = ( + scope: Scope, + name: string, + refs: readonly BindingRef[], + declaration: CppUsingDeclaration, + ): void => { + let bindings = augmentations.get(scope.id); + if (bindings === undefined) { + bindings = new Map(); + augmentations.set(scope.id, bindings); + } + const bucket = bindings.get(name) ?? []; + for (const ref of refs) { + bucket.push({ + def: ref.def, + origin: 'namespace', + declarationRange: declaration.range, + availableFrom: end(declaration.range), + }); + } + bindings.set(name, bucket); + }; + for (const parsed of parsedFiles) { + const facts = usingDeclarations(parsed); + if (facts.length === 0) continue; + const visible = visibleFiles(parsed.filePath); + for (const declaration of facts) { + const owner = owningScope(parsed, declaration.range); + if (owner === undefined) continue; + const resolved = resolveUsingMembers( + declaration, + owner, + parsed.filePath, + visible, + namespaceLookup, + ); + if (resolved === undefined) continue; + const { target, members } = resolved; + for (const [name, refs] of members) { + if (declaration.name !== undefined) { + append(owner, name, refs, declaration); + continue; + } + // A directive contributes at the nearest common namespace. A nearer + // ordinary declaration hides it; declarations in that namespace join + // its overload/ambiguity set instead of being silently replaced. + let current: Scope | undefined = owner; + let hidden = false; + const candidates = [...refs]; + while (current !== undefined) { + const currentPath = namespacePath(current); + const common = + current.kind === 'Module' || + (current.kind === 'Namespace' && + (target === currentPath || target.startsWith(`${currentPath}.`))); + const ordinary = (current.bindings.get(name) ?? []).filter( + (ref) => + ref.declarationRange === undefined || + compare(ref.declarationRange, declaration.range) <= 0, + ); + if (common) { + candidates.push(...ordinary); + break; + } + const typedLocal = + current.typeBindings.has(name) && + current.nameClaims?.some( + (claim) => + claim.name === name && + compare(claim.availableFrom ?? claim.range, declaration.range) <= 0, + ); + if (ordinary.length > 0 || typedLocal) { + hidden = true; + break; + } + current = + current.parent === null ? undefined : indexes.scopeTree.getScope(current.parent); + } + if (!hidden) append(owner, name, candidates, declaration); + } + } + } + + // A module-scope using written in a header is part of the including + // translation unit. Snapshot before propagation: include cycles and input + // file order must not change the surface copied to another file. + const moduleUsings = new Map>(); + for (const parsed of parsedFiles) { + const bindings = augmentations.get(parsed.moduleScope); + if (bindings !== undefined) { + moduleUsings.set( + parsed.moduleScope, + new Map([...bindings].map(([name, refs]) => [name, [...refs]])), + ); + } + } + if (moduleUsings.size === 0) return; + for (const parsed of parsedFiles) { + const owner = indexes.scopeTree.getScope(parsed.moduleScope); + if (owner === undefined) continue; + const seenIncludes = new Set(); + for (const edge of indexes.imports.get(parsed.moduleScope) ?? []) { + if ( + edge.targetFile === null || + edge.linkStatus === 'unresolved' || + edge.atRange === undefined + ) + continue; + const key = `${edge.targetFile}:${edge.atRange.startLine}:${edge.atRange.startCol}`; + if (seenIncludes.has(key)) continue; + seenIncludes.add(key); + for (const includedFile of visibleFiles(edge.targetFile)) { + if (includedFile === parsed.filePath) continue; + const included = byFile.get(includedFile); + if (included === undefined) continue; + for (const [name, refs] of moduleUsings.get(included.moduleScope) ?? []) { + append(owner, name, refs, { namespace: '', range: edge.atRange }); + } + } + } + } +} diff --git a/gitnexus/src/core/ingestion/languages/javascript/captures.ts b/gitnexus/src/core/ingestion/languages/javascript/captures.ts index a98771b11..21f7ca22e 100644 --- a/gitnexus/src/core/ingestion/languages/javascript/captures.ts +++ b/gitnexus/src/core/ingestion/languages/javascript/captures.ts @@ -52,6 +52,7 @@ import { reducesToContainedType } from '../typescript/interpret.js'; const JS_CLASS_FIELD_DEFINITION_TYPES: ReadonlySet = new Set(['field_definition']); import { hasKeyword } from '../../field-extractors/configs/helpers.js'; import { synthesizeCjsModuleExports } from '../typescript/cjs-module-exports.js'; +import { synthesizeTsLocalImports } from '../typescript/local-loaders.js'; import { isShadowedCjsExportAssignment, isUnexportedMemberAssignmentValue, @@ -248,104 +249,6 @@ function inferArgType(argNode: SyntaxNode): string { } } -// ─── CJS require() decomposition ───────────────────────────────────────── - -/** - * Walk the AST and synthesize `@import.*` captures for CJS `require()` calls: - * - * - `const { X, Y } = require('./m')` → one match per destructured name, - * `@import.kind = 'named'`, `@import.name = X / Y`. - * - `const X = require('./m')` → `@import.kind = 'namespace'`, - * `@import.alias = X` (the whole module is bound to X). - * - `require('./m')` as a bare expression-statement → side-effect. - * - * CJS named-alias form (`const { X: alias } = require('./m')`) emits - * `@import.kind = 'named-alias'` with `@import.name = X` and - * `@import.alias = alias`. - * - * The synthesized markers are identical to those produced by - * `splitImportStatement` for ESM, so `interpretJsImport` can delegate - * unchanged to `interpretTsImport` for all cases. - */ -function synthesizeCjsImports(root: SyntaxNode, out: CaptureMatch[]): void { - const stack: SyntaxNode[] = [root]; - for (;;) { - const node = stack.pop(); - if (node === undefined) break; - for (const child of node.namedChildren) { - if (child !== null) stack.push(child); - } - - if (node.type !== 'call_expression') continue; - - // Require call: function must be bare identifier "require". - const fn = node.childForFieldName('function'); - if (fn === null || fn.type !== 'identifier' || fn.text !== 'require') continue; - - const argsNode = node.childForFieldName('arguments'); - if (argsNode === null) continue; - - // Source must be a string literal. - const firstArg = argsNode.namedChild(0); - if (firstArg === null || firstArg.type !== 'string') continue; - const rawSource = firstArg.text; // includes surrounding quotes - const source = firstArg.namedChild(0)?.text ?? rawSource.slice(1, -1); - - const parent = node.parent; - - // Case 1: const { X } = require('./m') OR const X = require('./m') - if (parent?.type === 'variable_declarator') { - const nameNode = parent.childForFieldName('name'); - if (nameNode === null) continue; - - if (nameNode.type === 'object_pattern') { - // Destructured: emit one match per specifier. - for (const field of nameNode.namedChildren) { - if (field === null) continue; - if (field.type === 'shorthand_property_identifier_pattern') { - const name = field.text; - out.push({ - '@import.statement': syntheticCapture('@import.statement', node, rawSource), - '@import.kind': syntheticCapture('@import.kind', node, 'named'), - '@import.name': syntheticCapture('@import.name', field, name), - '@import.source': syntheticCapture('@import.source', firstArg, source), - }); - } else if (field.type === 'pair_pattern') { - const key = field.childForFieldName('key'); - const value = field.childForFieldName('value'); - if (key === null || value === null || value.type !== 'identifier') continue; - out.push({ - '@import.statement': syntheticCapture('@import.statement', node, rawSource), - '@import.kind': syntheticCapture('@import.kind', node, 'named-alias'), - '@import.name': syntheticCapture('@import.name', key, key.text), - '@import.alias': syntheticCapture('@import.alias', value, value.text), - '@import.source': syntheticCapture('@import.source', firstArg, source), - }); - } - } - } else if (nameNode.type === 'identifier') { - // Namespace-style: const X = require('./m') → bind whole module to X. - out.push({ - '@import.statement': syntheticCapture('@import.statement', node, rawSource), - '@import.kind': syntheticCapture('@import.kind', node, 'namespace'), - '@import.alias': syntheticCapture('@import.alias', nameNode, nameNode.text), - '@import.source': syntheticCapture('@import.source', firstArg, source), - }); - } - continue; - } - - // Case 2: bare require('./m') — side-effect import. - if (parent?.type === 'expression_statement') { - out.push({ - '@import.statement': syntheticCapture('@import.statement', node, rawSource), - '@import.kind': syntheticCapture('@import.kind', node, 'side-effect'), - '@import.source': syntheticCapture('@import.source', firstArg, source), - }); - } - } -} - // ─── JSDoc type binding synthesis ──────────────────────────────────────── interface JsDocParam { @@ -973,18 +876,20 @@ export function emitJsScopeCaptures( sourceText: string, filePath: string, cachedTree?: unknown, + sourceMeta?: { scriptLanguage?: string }, ): readonly CaptureMatch[] { + const grammarPath = sourceMeta?.scriptLanguage === 'jsx' ? `${filePath}.jsx` : filePath; let tree = cachedTree as ReturnType['parse']> | undefined; if (tree !== undefined && !jsCachedTreeMatchesGrammar(tree)) { tree = undefined; } if (tree === undefined) { - tree = parseSourceSafe(getJsParser(filePath), sourceText, undefined, { + tree = parseSourceSafe(getJsParser(grammarPath), sourceText, undefined, { bufferSize: getTreeSitterBufferSize(sourceText), }); } - const rawMatches = getJsScopeQuery(filePath).matches(tree.rootNode); + const rawMatches = getJsScopeQuery(grammarPath).matches(tree.rootNode); // Export evidence, read once per file (see `ts-js-export-marker.ts`). const exportEvidence = collectEsmExportEvidence(tree.rootNode, filePath); const out: CaptureMatch[] = []; @@ -1257,8 +1162,8 @@ export function emitJsScopeCaptures( } // Post-query synthesis passes. - synthesizeCjsImports(tree.rootNode, out); - synthesizeCjsModuleExports(tree.rootNode, filePath, out); + const validRequireCalls = synthesizeTsLocalImports(tree.rootNode, filePath, out); + synthesizeCjsModuleExports(tree.rootNode, filePath, out, validRequireCalls); synthesizeJsDocBindings(tree.rootNode, out); synthesizeConstructorFieldBindings(tree.rootNode, out); synthesizeDestructuringBindings(tree.rootNode, out); diff --git a/gitnexus/src/core/ingestion/languages/javascript/index.ts b/gitnexus/src/core/ingestion/languages/javascript/index.ts index 15411f504..76cb97a79 100644 --- a/gitnexus/src/core/ingestion/languages/javascript/index.ts +++ b/gitnexus/src/core/ingestion/languages/javascript/index.ts @@ -30,15 +30,13 @@ * and `@type {T}` on variable declarations are synthesized. `@typedef` * is not yet synthesized (tracked in #1646). * 2. **CJS chained destructuring** — `const { X: { Y } } = require(...)` - * (nested destructuring) emits only the outer `X` binding; `Y` is not + * (nested destructuring) creates no imported alias; `Y` is not * resolved. * 3. **Dynamic require** — `require(computedPath)` is skipped (non-literal * argument — cannot statically resolve the target). - * 4. **CommonJS `require()` in TypeScript** — `require()` decomposition is a - * JavaScript-emitter concern, so a `.ts` file's `const m = require('./m')` - * is not decomposed into an import. The EXPORT side of a `.ts` CommonJS - * module is now declared (shared with the JS emitter), but an importer - * written in TypeScript still cannot resolve through it. + * 4. **Indirect loaders** — computed sources, unawaited import promises, + * default/rest binding targets and indirect loader chains are not module + * aliases. TS/TSX, JS/JSX and Vue share the static local-loader pass. * 5. **`.cjs` module-level `this`** — the CommonJS/ESM gate consults the file * extension where it can, but `provider.labelOverride` receives no file * path, so a `.cjs` file whose only export is a module-level `this.X = fn` diff --git a/gitnexus/src/core/ingestion/languages/javascript/query.ts b/gitnexus/src/core/ingestion/languages/javascript/query.ts index 8442b40e4..a4bcc3a71 100644 --- a/gitnexus/src/core/ingestion/languages/javascript/query.ts +++ b/gitnexus/src/core/ingestion/languages/javascript/query.ts @@ -139,6 +139,12 @@ export const JAVASCRIPT_SCOPE_QUERY = ` name: (identifier) @declaration.name value: (function_expression) @declaration.function)) +;; Generator expressions have the same callable binding and scope anchor. +(lexical_declaration + (variable_declarator + name: (identifier) @declaration.name + value: (generator_function) @declaration.function)) + (export_statement declaration: (lexical_declaration (variable_declarator @@ -161,6 +167,11 @@ export const JAVASCRIPT_SCOPE_QUERY = ` name: (identifier) @declaration.name value: (function_expression) @declaration.function)) +(variable_declaration + (variable_declarator + name: (identifier) @declaration.name + value: (generator_function) @declaration.function)) + ;; CJS property-assignment exports (#2723): \`exports.foo = function () {}\`, ;; \`module.exports.foo = (a) => a\`. The graph node for these comes from ;; TYPESCRIPT/JAVASCRIPT_QUERIES; this block is the other half — without a @@ -728,8 +739,7 @@ export const JAVASCRIPT_SCOPE_QUERY = ` (for_in_statement left: (identifier) @type-binding.name - right: (member_expression - property: (property_identifier) @type-binding.type)) @type-binding.alias + right: (member_expression) @type-binding.type) @type-binding.alias ;; ── References ──────────────────────────────────────────────────────────── @@ -951,7 +961,7 @@ export function getJsScopeQuery(filePath?: string): Parser.Query { /** Validate that a cached Tree was produced by the JS grammar. */ export function jsCachedTreeMatchesGrammar(tree: unknown): boolean { // eslint-disable-next-line @typescript-eslint/no-explicit-any - const lang = (tree as any)?.getLanguage?.(); + const lang = (tree as any)?.getLanguage?.() ?? (tree as any)?.language; if (lang === undefined || lang === null) return true; return lang === JS_GRAMMAR; } diff --git a/gitnexus/src/core/ingestion/languages/javascript/scope-resolver.ts b/gitnexus/src/core/ingestion/languages/javascript/scope-resolver.ts index 280fd9929..7f4b5a48f 100644 --- a/gitnexus/src/core/ingestion/languages/javascript/scope-resolver.ts +++ b/gitnexus/src/core/ingestion/languages/javascript/scope-resolver.ts @@ -51,6 +51,8 @@ const javascriptScopeResolver: ScopeResolver = { language: SupportedLanguages.JavaScript, languageProvider: javascriptProvider, importEdgeReason: 'javascript-scope: import', + importsBindAtLexicalScope: true, + ownedMembersBindAtModuleScope: false, resolveImportTarget: makeJsResolveImportTarget(), diff --git a/gitnexus/src/core/ingestion/languages/python/captures.ts b/gitnexus/src/core/ingestion/languages/python/captures.ts index 3a224d188..b7dfbde70 100644 --- a/gitnexus/src/core/ingestion/languages/python/captures.ts +++ b/gitnexus/src/core/ingestion/languages/python/captures.ts @@ -42,6 +42,7 @@ import { parseSourceSafe } from '../../../tree-sitter/safe-parse.js'; import { isPythonGlobalDeclaration, pythonFunctionDefinitionLabel } from './simple-hooks.js'; import { synthesizeCallableFlowCaptures } from '../../utils/callable-flow-captures.js'; import { synthesizeReceiverChainCapture } from '../../utils/receiver-chain-captures.js'; +import { applyPythonLexicalBindings } from './lexical-bindings.js'; import { beginPythonSubtypeDispatchCapture, recordPythonSimplePositionalCall, @@ -286,10 +287,16 @@ export function emitPythonScopeCaptures( out.push(...synthesizePythonInheritanceReferences(tree.rootNode)); out.push(...synthesizeCallableFlowCaptures(tree.rootNode, PYTHON_CALLABLE_CAPTURE_OPTIONS)); + const lexicalCaptures = applyPythonLexicalBindings( + out, + tree.rootNode, + filePath, + notebookSegments === undefined ? undefined : (range) => remapRange(range, notebookSegments), + ); if (notebookSegments !== undefined) { - return out.map((match) => remapCaptureMatch(match, notebookSegments)); + return lexicalCaptures.map((match) => remapCaptureMatch(match, notebookSegments)); } - return out; + return lexicalCaptures; } function resolveNotebookCaptureSource( diff --git a/gitnexus/src/core/ingestion/languages/python/depends-references.ts b/gitnexus/src/core/ingestion/languages/python/depends-references.ts index 4982b7dce..debcdda35 100644 --- a/gitnexus/src/core/ingestion/languages/python/depends-references.ts +++ b/gitnexus/src/core/ingestion/languages/python/depends-references.ts @@ -51,6 +51,7 @@ export function synthesizeDependsReferences(fnNode: SyntaxNode): readonly Captur results.push({ '@reference.call.free': nodeToCapture('@reference.call.free', firstArg), '@reference.name': nodeToCapture('@reference.name', firstArg), + '@reference.caller-function': nodeToCapture('@reference.caller-function', fnNode), }); continue; } @@ -63,6 +64,7 @@ export function synthesizeDependsReferences(fnNode: SyntaxNode): readonly Captur '@reference.call.member': nodeToCapture('@reference.call.member', attrName), '@reference.name': nodeToCapture('@reference.name', attrName), '@reference.receiver': nodeToCapture('@reference.receiver', obj), + '@reference.caller-function': nodeToCapture('@reference.caller-function', fnNode), }); } } diff --git a/gitnexus/src/core/ingestion/languages/python/lexical-bindings.ts b/gitnexus/src/core/ingestion/languages/python/lexical-bindings.ts new file mode 100644 index 000000000..17c3d56f1 --- /dev/null +++ b/gitnexus/src/core/ingestion/languages/python/lexical-bindings.ts @@ -0,0 +1,491 @@ +/** Compiler binding ownership, kept separate from successful symbol extraction. */ +import { + makeScopeId, + type Capture, + type CaptureMatch, + type NameClaim, + type Range, + type ScopeKind, +} from 'gitnexus-shared'; +import { nodeToCapture, syntheticCapture, type SyntaxNode } from '../../utils/ast-helpers.js'; +import { splitImportStatement } from './import-decomposer.js'; +import { interpretPythonImport } from './interpret.js'; + +const COMPREHENSIONS = new Set([ + 'list_comprehension', + 'set_comprehension', + 'dictionary_comprehension', + 'generator_expression', +]); +const CONDITIONAL = new Set([ + 'if_statement', + 'for_statement', + 'while_statement', + 'try_statement', + 'match_statement', +]); +interface Environment { + node: SyntaxNode; + kind: ScopeKind; + parent?: Environment; + claims: NameClaim[]; + directives: Map; + imports: Map>; +} +interface Event { + env: Environment; + node: SyntaxNode; + name: string; + kind: NameClaim['kind']; + declaration?: boolean; + annotationOnly?: boolean; + deletion?: boolean; + hoisted?: boolean; + conditional?: boolean; + identity?: string; + packageRoot?: string; +} + +/** Attach plain-data facts only: no syntax nodes survive worker transport. */ +export function applyPythonLexicalBindings( + matches: readonly CaptureMatch[], + root: SyntaxNode, + filePath: string, + mapRange: (range: Range) => Range = (range) => range, +): CaptureMatch[] { + const environments: Environment[] = []; + const environmentsByRange = new Map(); + const byNode = new Map(); + const byRange = new Map(); + const events: Event[] = []; + const annotationOnly = (node: SyntaxNode): boolean => + node.type === 'assignment' && + node.childForFieldName('type') !== null && + node.childForFieldName('right') === null; + const range = (node: SyntaxNode): Range => ({ + startLine: node.startPosition.row + 1, + startCol: node.startPosition.column, + endLine: node.endPosition.row + 1, + endCol: node.endPosition.column, + }); + const key = (r: Range): string => `${r.startLine}:${r.startCol}:${r.endLine}:${r.endCol}`; + const id = (env: Environment) => + makeScopeId({ filePath, kind: env.kind, range: mapRange(range(env.node)) }); + const mark = (node: SyntaxNode, env: Environment): void => { + byNode.set(node.id, env); + byRange.set(key(range(node)), node); + }; + const create = (node: SyntaxNode, kind: ScopeKind, parent?: Environment): Environment => { + const env: Environment = { + node, + kind, + parent, + claims: [], + directives: new Map(), + imports: new Map(), + }; + environments.push(env); + environmentsByRange.set(`${kind}:${key(range(node))}`, env); + mark(node, env); + return env; + }; + const bind = ( + env: Environment, + node: SyntaxNode | null, + kind: NameClaim['kind'] = 'binding', + hoisted = false, + conditional = false, + anchor?: SyntaxNode, + ): void => { + if (node === null) return; + if (node.type === 'identifier') { + events.push({ env, node: anchor ?? node, name: node.text, kind, hoisted, conditional }); + return; + } + if (['attribute', 'subscript', 'type'].includes(node.type)) return; + for (const child of node.namedChildren) bind(env, child, kind, hoisted, conditional, anchor); + }; + const pattern = (env: Environment, node: SyntaxNode): void => { + if (node.type === 'dotted_name') { + if (node.namedChildCount === 1 && node.text !== '_') bind(env, node.firstNamedChild); + return; + } + if (node.type === 'identifier') { + if (node.text !== '_') bind(env, node); + return; + } + if (node.type === 'class_pattern') { + for (const child of node.namedChildren.slice(1)) pattern(env, child); + return; + } + if (node.type === 'keyword_pattern') { + for (const child of node.namedChildren.slice(1)) pattern(env, child); + return; + } + if (node.type === 'dict_pattern') { + for (const child of node.childrenForFieldName('value')) pattern(env, child); + for (const child of node.namedChildren) + if (child.type === 'splat_pattern') pattern(env, child); + return; + } + if (['string', 'concatenated_string', 'attribute'].includes(node.type)) return; + for (const child of node.namedChildren) pattern(env, child); + }; + const visit = (node: SyntaxNode, env: Environment, conditional = false): void => { + mark(node, env); + if ( + node.type === 'function_definition' || + node.type === 'lambda' || + node.type === 'class_definition' + ) { + const isClass = node.type === 'class_definition'; + if (node.type !== 'lambda') { + const name = node.childForFieldName('name'); + if (name) + events.push({ + env, + node, + name: name.text, + kind: 'binding', + declaration: true, + conditional, + }); + } + const childEnv = create(node, isClass ? 'Class' : 'Function', env); + const parameters = node.childForFieldName('parameters'); + if (parameters) { + mark(parameters, childEnv); + for (const parameter of parameters.namedChildren) { + mark(parameter, childEnv); + const target = + parameter.childForFieldName('name') ?? + (parameter.type === 'identifier' ? parameter : parameter.firstNamedChild); + if (target && !['positional_separator', 'keyword_separator'].includes(parameter.type)) + bind(childEnv, target, 'binding', true); + const defaultValue = parameter.childForFieldName('value'); + if (defaultValue) visit(defaultValue, env, conditional); + const annotation = parameter.childForFieldName('type'); + if (annotation) visit(annotation, env, conditional); + } + } + const body = node.childForFieldName('body'); + for (const child of node.namedChildren) { + if (child.id === parameters?.id || child.id === node.childForFieldName('name')?.id) + continue; + visit( + child, + child.id === body?.id ? childEnv : env, + child.id === body?.id ? false : conditional, + ); + } + return; + } + if (COMPREHENSIONS.has(node.type)) { + const childEnv = create(node, 'Expression', env); + const firstFor = node.namedChildren.find((child) => child.type === 'for_in_clause'); + const firstIterable = firstFor?.childForFieldName('right'); + for (const child of node.namedChildren) { + if (child.type === 'for_in_clause') { + mark(child, childEnv); + bind(childEnv, child.childForFieldName('left'), 'binding', true); + for (const part of child.namedChildren) + visit(part, part.id === firstIterable?.id ? env : childEnv, conditional); + } else visit(child, childEnv, conditional); + } + return; + } + if (node.type === 'global_statement' || node.type === 'nonlocal_statement') { + for (const child of node.namedChildren) { + if (env.kind !== 'Module' && child.type === 'identifier') + env.directives.set( + child.text, + node.type === 'global_statement' ? 'module' : 'enclosing-function', + ); + } + return; + } + if (node.type === 'import_statement' || node.type === 'import_from_statement') { + for (const match of splitImportStatement(node)) { + const imp = interpretPythonImport(match); + if (imp && 'localName' in imp && imp.localName !== '') { + events.push({ + env, + node, + name: imp.localName, + kind: 'import', + conditional, + identity: JSON.stringify([ + imp.kind, + imp.targetRaw, + 'importedName' in imp ? imp.importedName : '', + 'explicitAlias' in imp && imp.explicitAlias, + ]), + ...(imp.kind === 'namespace' && !imp.explicitAlias + ? { packageRoot: imp.localName } + : {}), + }); + } + } + return; + } + if (node.type === 'assignment' || node.type === 'augmented_assignment') { + const first = events.length; + bind(env, node.childForFieldName('left'), 'binding', false, conditional, node); + if (annotationOnly(node)) + for (let i = first; i < events.length; i++) events[i]!.annotationOnly = true; + } + if (node.type === 'for_statement') bind(env, node.childForFieldName('left')); + if (node.type === 'delete_statement') { + const first = events.length; + for (const child of node.namedChildren) bind(env, child, 'blocked'); + for (let i = first; i < events.length; i++) events[i]!.deletion = true; + } + if (node.type === 'named_expression') { + let owner = env; + while (COMPREHENSIONS.has(owner.node.type) && owner.parent) owner = owner.parent; + bind(owner, node.childForFieldName('name')); + } + if (node.type === 'as_pattern' && node.childForFieldName('alias')) + bind(env, node.childForFieldName('alias')); + if (node.type === 'case_clause') + for (const child of node.namedChildren) + if (child.type === 'case_pattern') pattern(env, child); + if (node.type === 'type_alias_statement') { + const first = node.firstNamedChild; + let target = first?.firstNamedChild; + if (target?.type === 'generic_type') target = target.firstNamedChild; + if (target?.type === 'identifier') bind(env, target, 'blocked'); + } + const childConditional = conditional || CONDITIONAL.has(node.type); + for (const child of node.namedChildren) visit(child, env, childConditional); + }; + const module = create(root, 'Module'); + for (const child of root.namedChildren) visit(child, module); + + const localNames = new Map>(); + const storedNames = new Map>(); + const definiteDeclarations = new Set(); + for (const event of events) { + const names = localNames.get(event.env) ?? new Set(); + names.add(event.name); + localNames.set(event.env, names); + if (!event.annotationOnly) { + const stores = storedNames.get(event.env) ?? new Set(); + stores.add(event.name); + storedNames.set(event.env, stores); + } + } + const ownerFor = (env: Environment, name: string): Environment => { + const redirect = env.directives.get(name); + if (redirect === 'module') return module; + if (redirect === 'enclosing-function') { + for (let parent = env.parent; parent; parent = parent.parent) { + if ( + parent.kind === 'Function' && + !parent.directives.has(name) && + localNames.get(parent)?.has(name) + ) + return parent; + } + } + return env; + }; + for (const env of environments) { + for (const [name, redirect] of env.directives) + env.claims.push({ + name, + kind: 'blocked', + hoisted: true, + range: mapRange(range(env.node)), + redirect, + }); + } + for (const event of events) { + // An annotation records a compiler local, but never overwrites a value. + // Keep existing imports/declarations when that name has a runtime store. + if (event.annotationOnly && storedNames.get(event.env)?.has(event.name)) continue; + const owner = ownerFor(event.env, event.name); + const eventRange = mapRange(range(event.node)); + const claim: NameClaim = { + name: event.name, + // Imports under guards (including TYPE_CHECKING) still provide static + // references. The identity/mixed-binding pass below blocks ambiguity. + kind: event.annotationOnly && owner.kind === 'Class' ? 'blocked' : event.kind, + range: eventRange, + ...(event.hoisted || event.annotationOnly + ? { hoisted: true } + : { availableFrom: { startLine: eventRange.endLine, startCol: eventRange.endCol } }), + inactive: owner.kind === 'Class' ? 'module' : 'blocked', + ...((event.deletion || event.annotationOnly) && owner.kind === 'Class' + ? { redirect: 'module' } + : {}), + }; + const redirected = owner !== event.env; + if (redirected) + event.env.claims.push({ ...claim, redirect: event.env.directives.get(event.name) }); + const ownerClaim: NameClaim = + redirected && !event.declaration ? { ...claim, kind: 'blocked' } : claim; + owner.claims.push(ownerClaim); + if (event.declaration && !event.conditional) definiteDeclarations.add(ownerClaim); + if (event.identity) { + const imports = owner.imports.get(event.name) ?? []; + imports.push({ identity: event.identity, packageRoot: event.packageRoot, claim: ownerClaim }); + owner.imports.set(event.name, imports); + } + } + for (const env of environments) { + for (const [name, imports] of env.imports) { + const first = imports[0]!; + const compatible = imports.every( + (imp) => + imp.identity === first.identity || + (first.packageRoot !== undefined && imp.packageRoot === first.packageRoot), + ); + const mixed = env.claims.some( + (claim) => + claim.name === name && + claim.redirect === undefined && + !imports.some((imp) => imp.claim === claim), + ); + // Conflicting imports remain uncertain, but a definite def/class store + // still supplies its own local target after the declaration executes. + env.claims = env.claims.map((claim) => + claim.name !== name || claim.redirect !== undefined || definiteDeclarations.has(claim) + ? claim + : !compatible || mixed + ? { ...claim, kind: 'blocked' } + : { ...claim, merge: true }, + ); + } + } + const environmentAt = (node: SyntaxNode): Environment => { + for (let current: SyntaxNode | null = node; current; current = current.parent) { + const env = byNode.get(current.id); + if (env) return env; + } + return module; + }; + const policyFor = (env: Environment) => ({ + callerScopeIsAuthoritative: true, + ...(env.kind === 'Class' ? { skipFromChildren: true } : {}), + ...(env.kind === 'Function' || env.node.type === 'generator_expression' + ? { deferParentActivation: true } + : {}), + ...(env.parent ? { parentScope: id(env.parent) } : {}), + }); + const decorateScope = (match: CaptureMatch, env: Environment): CaptureMatch => ({ + ...match, + '@scope.name-claims': syntheticCapture( + '@scope.name-claims', + env.node, + JSON.stringify(env.claims), + ), + '@scope.lookup-policy': syntheticCapture( + '@scope.lookup-policy', + env.node, + JSON.stringify(policyFor(env)), + ), + }); + const out = matches + .filter((match) => { + const declaration = match['@declaration.variable']; + const node = declaration && byRange.get(key(declaration.range)); + const name = match['@declaration.name']?.text; + // Keep standalone annotations as structural fields/type declarations. + // They must not replace an import in the local-over-import merge: an + // annotation with no initializer does not store a new runtime value. + return ( + node === undefined || + !annotationOnly(node) || + name === undefined || + !environmentAt(node).imports.has(name) + ); + }) + .map((match): CaptureMatch => { + const scopeAnchor = + match['@scope.module'] ?? match['@scope.class'] ?? match['@scope.function']; + if (scopeAnchor) { + // A file consisting of exactly one definition can share its range with + // that child. Physical extent alone does not identify an environment. + const kind = match['@scope.module'] + ? 'Module' + : match['@scope.class'] + ? 'Class' + : 'Function'; + const env = environmentsByRange.get(`${kind}:${key(scopeAnchor.range)}`); + if (env) return decorateScope(match, env); + } + // A framework dependency is invoked for the handler, but its callable + // is evaluated in the default expression's enclosing environment. + // Keep the synthetic caller marker out of lexical anchor selection. + const { '@reference.caller-function': callerFunction, ...referenceMatch } = match; + const entries = Object.values(referenceMatch); + const anchor = entries.reduce( + (best, cap) => + !best || + cap.range.endLine - cap.range.startLine > best.range.endLine - best.range.startLine || + (cap.range.endLine - cap.range.startLine === best.range.endLine - best.range.startLine && + cap.range.endCol - cap.range.startCol > best.range.endCol - best.range.startCol) + ? cap + : best, + undefined, + ); + if (!anchor) return referenceMatch; + const node = byRange.get(key(anchor.range)); + if (!node) return referenceMatch; + const env = environmentAt(node); + const extra: Record = {}; + if (callerFunction) { + const caller = environmentsByRange.get(`Function:${key(callerFunction.range)}`); + if (caller) + extra['@reference.caller-scope'] = syntheticCapture( + '@reference.caller-scope', + node, + id(caller), + ); + } + if (entries.some((cap) => cap.name.startsWith('@reference.'))) + extra['@reference.lookup-scope'] = syntheticCapture( + '@reference.lookup-scope', + node, + id(env), + ); + const declarationName = match['@declaration.name']?.text; + const typeName = match['@type-binding.name']?.text; + if (declarationName) { + const declarationEnv = env.node.id === node.id && env.parent ? env.parent : env; + extra['@binding.scope'] = syntheticCapture( + '@binding.scope', + node, + id(ownerFor(declarationEnv, declarationName)), + ); + } else if ( + typeName && + !match['@type-binding.instance-field'] && + !match['@type-binding.self'] && + !match['@type-binding.cls'] + ) { + let owner = match['@type-binding.return'] && env.parent ? env.parent : env; + if (node.type === 'named_expression') + while (COMPREHENSIONS.has(owner.node.type) && owner.parent) owner = owner.parent; + extra['@binding.scope'] = syntheticCapture( + '@binding.scope', + node, + id(ownerFor(owner, typeName)), + ); + const lookupEnv = + (match['@type-binding.parameter'] || match['@type-binding.return']) && env.parent + ? env.parent + : env; + extra['@type.lookup-scope'] = syntheticCapture('@type.lookup-scope', node, id(lookupEnv)); + } + return { ...referenceMatch, ...extra }; + }); + for (const env of environments) { + if (COMPREHENSIONS.has(env.node.type)) + out.push( + decorateScope({ '@scope.expression': nodeToCapture('@scope.expression', env.node) }, env), + ); + } + return out; +} diff --git a/gitnexus/src/core/ingestion/languages/python/receiver-binding.ts b/gitnexus/src/core/ingestion/languages/python/receiver-binding.ts index 584f09445..3b0cb76c0 100644 --- a/gitnexus/src/core/ingestion/languages/python/receiver-binding.ts +++ b/gitnexus/src/core/ingestion/languages/python/receiver-binding.ts @@ -13,14 +13,18 @@ import type { CaptureMatch } from 'gitnexus-shared'; import { nodeToCapture, syntheticCapture, type SyntaxNode } from '../../utils/ast-helpers.js'; import { decoratorNames, isBuiltinDescriptor } from './builtin-descriptors.js'; -import { isPythonGlobalDeclaration } from './simple-hooks.js'; +import { isPythonGlobalDeclaration, isPythonNonlocalDeclaration } from './simple-hooks.js'; /** Walk up to the enclosing `class_definition`, ignoring the immediate * `decorated_definition` wrapper. Returns `null` when the function is * free, lambda-bodied, or nested inside another function. */ function findEnclosingClassDefinition(node: SyntaxNode): SyntaxNode | null { const name = node.childForFieldName('name')?.text; - if (name !== undefined && isPythonGlobalDeclaration(node, name)) return null; + if ( + name !== undefined && + (isPythonGlobalDeclaration(node, name) || isPythonNonlocalDeclaration(node, name)) + ) + return null; let cur: SyntaxNode | null = node.parent; while (cur !== null) { if (cur.type === 'class_definition') return cur; diff --git a/gitnexus/src/core/ingestion/languages/python/simple-hooks.ts b/gitnexus/src/core/ingestion/languages/python/simple-hooks.ts index 94f6bcfa9..3b46bc323 100644 --- a/gitnexus/src/core/ingestion/languages/python/simple-hooks.ts +++ b/gitnexus/src/core/ingestion/languages/python/simple-hooks.ts @@ -22,6 +22,7 @@ import { walkToScope } from '../../utils/scope-tree-walk.js'; * in the cache so repeated declaration/receiver classification scans each tree * once, without retaining its AST after the tree is released. */ const globalsByTree = new WeakMap>>(); +const nonlocalsByTree = new WeakMap>>(); function enclosingPythonCodeBlock(node: SyntaxNode): SyntaxNode | null { let enclosing = node.parent; @@ -64,6 +65,28 @@ export function isPythonGlobalDeclaration(node: SyntaxNode, name: string): boole return globals.get(owner.id)?.has(name) ?? false; } +/** A class suite may store a declaration in an enclosing function cell. */ +export function isPythonNonlocalDeclaration(node: SyntaxNode, name: string): boolean { + const owner = enclosingPythonCodeBlock(node); + if (owner === null || owner.type === 'module') return false; + let nonlocals = nonlocalsByTree.get(node.tree); + if (nonlocals === undefined) { + const byBlock = new Map>(); + for (const statement of node.tree.rootNode.descendantsOfType('nonlocal_statement')) { + const block = enclosingPythonCodeBlock(statement); + if (block === null || block.type === 'module') continue; + const names = byBlock.get(block.id) ?? new Set(); + for (const identifier of statement.namedChildren) { + if (identifier.type === 'identifier') names.add(identifier.text); + } + byBlock.set(block.id, names); + } + nonlocals = byBlock; + nonlocalsByTree.set(node.tree, nonlocals); + } + return nonlocals.get(owner.id)?.has(name) ?? false; +} + export function pythonFunctionDefinitionLabel( functionNode: SyntaxNode, defaultLabel: NodeLabel, @@ -71,7 +94,9 @@ export function pythonFunctionDefinitionLabel( if (defaultLabel !== 'Function') return defaultLabel; if (enclosingPythonCodeBlock(functionNode)?.type !== 'class_definition') return 'Function'; const name = functionNode.childForFieldName('name')?.text; - return name !== undefined && isPythonGlobalDeclaration(functionNode, name) + return name !== undefined && + (isPythonGlobalDeclaration(functionNode, name) || + isPythonNonlocalDeclaration(functionNode, name)) ? 'Function' : 'Method'; } @@ -93,6 +118,8 @@ export function pythonBindingScopeFor( if (decl['@type-binding.instance-field'] !== undefined) { return walkToScope(innermost, tree, 'Class'); } + const explicitOwner = decl['@binding.scope']?.text; + if (explicitOwner !== undefined && tree.has(explicitOwner)) return explicitOwner; return null; } @@ -103,10 +130,29 @@ export function pythonBindingScopeFor( * `class A: import x` makes `x` a class attribute) attach to the class. * Module-level imports delegate to the central default. */ export function pythonImportOwningScope( - _imp: ParsedImport, + imp: ParsedImport, innermost: Scope, - _tree: ScopeTree, + tree: ScopeTree, ): ScopeId | null { + const name = 'localName' in imp ? imp.localName : undefined; + const redirect = + name === undefined + ? undefined + : innermost.nameClaims?.find( + (claim) => claim.name === name && claim.hoisted === true && claim.redirect !== undefined, + )?.redirect; + if (redirect === 'module') return walkToScope(innermost, tree, 'Module'); + if (redirect === 'enclosing-function') { + let parent = innermost.parent === null ? undefined : tree.getScope(innermost.parent); + while (parent !== undefined) { + if ( + parent.kind === 'Function' && + parent.nameClaims?.some((claim) => claim.name === name && claim.redirect === undefined) + ) + return parent.id; + parent = parent.parent === null ? undefined : tree.getScope(parent.parent); + } + } if (innermost.kind === 'Function' || innermost.kind === 'Class') return innermost.id; return null; } diff --git a/gitnexus/src/core/ingestion/languages/rust/captures.ts b/gitnexus/src/core/ingestion/languages/rust/captures.ts index d6685dde3..09ae8952c 100644 --- a/gitnexus/src/core/ingestion/languages/rust/captures.ts +++ b/gitnexus/src/core/ingestion/languages/rust/captures.ts @@ -15,6 +15,7 @@ import { getTreeSitterBufferSize } from '../../constants.js'; import { parseSourceSafe } from '../../../tree-sitter/safe-parse.js'; import { synthesizeCallableFlowCaptures } from '../../utils/callable-flow-captures.js'; import { synthesizeReceiverChainCapture } from '../../utils/receiver-chain-captures.js'; +import { collectRustNameClaims } from './lexical-bindings.js'; const RUST_CALLABLE_CAPTURE_OPTIONS = { functionNodeTypes: new Set(['function_item', 'closure_expression']), @@ -45,6 +46,7 @@ export function emitRustScopeCaptures( const rawMatches = getRustScopeQuery().matches(tree.rootNode); const out: CaptureMatch[] = []; + const nameClaims = collectRustNameClaims(tree.rootNode); for (const m of rawMatches) { const grouped: Record = {}; @@ -62,6 +64,19 @@ export function emitRustScopeCaptures( } if (Object.keys(grouped).length === 0) continue; + const scopeTag = Object.keys(nodeMap).find((tag) => tag.startsWith('@scope.')); + if (scopeTag !== undefined) { + const scopeNode = nodeMap[scopeTag]; + const claims = nameClaims.get(scopeNode.id); + if (claims !== undefined) { + grouped['@scope.name-claims'] = syntheticCapture( + '@scope.name-claims', + scopeNode, + JSON.stringify(claims), + ); + } + } + // Decompose use declarations into individual import captures if (grouped['@import.statement'] !== undefined) { const useNode = nodeIfType(nodeMap['@import.statement'], 'use_declaration'); diff --git a/gitnexus/src/core/ingestion/languages/rust/cargo-module-files.ts b/gitnexus/src/core/ingestion/languages/rust/cargo-module-files.ts index c73ede641..a8dbd134d 100644 --- a/gitnexus/src/core/ingestion/languages/rust/cargo-module-files.ts +++ b/gitnexus/src/core/ingestion/languages/rust/cargo-module-files.ts @@ -2,6 +2,12 @@ import path from 'node:path'; import type Parser from 'tree-sitter'; import { splitRustUseDeclaration } from './import-decomposer.js'; +export interface RustModuleLink { + readonly module: string; + readonly file: string; + readonly inline: boolean; +} + // Built-in attributes cannot expand to new module declarations. cfg is a union: // visiting both alternatives is conservative; cfg_attr may change a path. const NON_EXPANDING_ATTRIBUTES = new Set([ @@ -140,6 +146,7 @@ export function rustModuleFiles( files: ReadonlySet, missingFiles?: Set, isCrateRoot = false, + moduleLinks?: RustModuleLink[], ): readonly { file: string; ownsDirectory: boolean }[] | undefined { if (root.hasError) return undefined; // Built-in spellings are not proof when an import/local macro can shadow @@ -216,7 +223,7 @@ export function rustModuleFiles( ownsDirectory || path.posix.basename(file) === 'mod.rs' ? fileDir : file.slice(0, -'.rs'.length); - const pending = [{ node: root, moduleDir, attributeDir: fileDir }]; + const pending = [{ node: root, moduleDir, attributeDir: fileDir, module: '', lexical: false }]; const result: { file: string; ownsDirectory: boolean }[] = []; let unresolved = false; while (pending.length) { @@ -291,7 +298,13 @@ export function rustModuleFiles( node.type !== 'token_tree' && node.namedChildCount > 0 ) { - pending.push({ ...context, node }); + pending.push({ + ...context, + node, + lexical: + context.lexical || + ['function_item', 'closure_expression', 'block'].includes(node.type), + }); } continue; } @@ -308,12 +321,20 @@ export function rustModuleFiles( return undefined; } const body = node.childForFieldName('body'); + const module = [context.module, name].filter(Boolean).join('::'); if (body) { const dir = override === undefined ? path.posix.join(context.moduleDir, name) : path.posix.join(context.attributeDir, override); - pending.push({ node: body, moduleDir: dir, attributeDir: dir }); + if (!context.lexical) moduleLinks?.push({ module, file, inline: true }); + pending.push({ + node: body, + moduleDir: dir, + attributeDir: dir, + module, + lexical: context.lexical, + }); continue; } const candidates = @@ -334,6 +355,9 @@ export function rustModuleFiles( // #[path] makes the loaded file own its containing directory, just like // a crate root; its children do NOT acquire the file stem as a prefix. result.push(...existing.map((file) => ({ file, ownsDirectory: override !== undefined }))); + if (!context.lexical) { + moduleLinks?.push(...existing.map((file) => ({ module, file, inline: false }))); + } } } return unresolved ? undefined : result; diff --git a/gitnexus/src/core/ingestion/languages/rust/cargo-targets.ts b/gitnexus/src/core/ingestion/languages/rust/cargo-targets.ts index 95377f032..7173220e3 100644 --- a/gitnexus/src/core/ingestion/languages/rust/cargo-targets.ts +++ b/gitnexus/src/core/ingestion/languages/rust/cargo-targets.ts @@ -8,13 +8,39 @@ import { SupportedLanguages } from 'gitnexus-shared'; import { getLanguageGrammar } from '../../../tree-sitter/parser-loader.js'; import { parseSourceSafe } from '../../../tree-sitter/safe-parse.js'; import { readRepoControlFile } from '../../../../config/repo-control-file.js'; -import { rustModuleFiles, rustPublicUses } from './cargo-module-files.js'; +import { rustModuleFiles, rustPublicUses, type RustModuleLink } from './cargo-module-files.js'; const MAX_FILES = 100_000; type Table = Record; const table = (value: unknown): value is Table => value !== null && typeof value === 'object' && !Array.isArray(value); +function cargoPackageEdition( + manifest: string, + data: Table, + workspaceEditions?: ReadonlyMap, +): string | undefined { + if (!table(data.package)) return undefined; + const pkg = data.package; + const dir = path.posix.dirname(manifest); + let edition: unknown = pkg.edition ?? '2015'; + if (table(edition) && edition.workspace === true) { + let workspace = + typeof pkg.workspace === 'string' ? path.posix.join(dir, pkg.workspace, '.') : dir; + while ( + !workspaceEditions?.has(workspace) && + typeof pkg.workspace !== 'string' && + workspace !== '.' + ) { + workspace = path.posix.dirname(workspace); + } + edition = workspaceEditions?.get(workspace); + } + return typeof edition === 'string' && ['2015', '2018', '2021', '2024'].includes(edition) + ? edition + : undefined; +} + /** A target is identified by its entry file, not its package directory. */ export function cargoTargetRoots( manifest: string, @@ -34,21 +60,8 @@ export function cargoTargetRoots( if (pkg.build !== undefined && typeof pkg.build !== 'string' && typeof pkg.build !== 'boolean') return undefined; const dir = path.posix.dirname(manifest); - let edition: unknown = pkg.edition ?? '2015'; - if (table(edition) && edition.workspace === true) { - let workspace = - typeof pkg.workspace === 'string' ? path.posix.join(dir, pkg.workspace, '.') : dir; - while ( - !workspaceEditions?.has(workspace) && - typeof pkg.workspace !== 'string' && - workspace !== '.' - ) { - workspace = path.posix.dirname(workspace); - } - edition = workspaceEditions?.get(workspace); - } - if (typeof edition !== 'string' || !['2015', '2018', '2021', '2024'].includes(edition)) - return undefined; + const edition = cargoPackageEdition(manifest, data, workspaceEditions); + if (edition === undefined) return undefined; const relative = (file: string): string => path.posix.normalize(path.posix.join(dir, file)); const roots = new Set(); for (const [kind, folder] of [ @@ -115,14 +128,106 @@ export function cargoTargetRoots( return [...roots]; } +export interface RustModuleLocation { + readonly file: string; + /** Namespace within this source file; external file modules start empty. */ + readonly module: string; +} + +const moduleKey = (file: string, module: string): string => JSON.stringify([file, module]); + class RustCargoTargets { constructor( readonly targetsByFile: ReadonlyMap>, readonly rootImports: ReadonlyMap>>, readonly publicUsesByFile: ReadonlyMap>, + readonly relativeUseByFile: ReadonlyMap, + readonly moduleChildren: ReadonlyMap, + readonly moduleParents: ReadonlyMap, ) {} } +/** Resolve only module topology proven by the already-loaded Cargo ASTs. */ +export function rustCargoImportedModules( + config: unknown, + caller: string, + importedModule: readonly string[], + ownerModule: string, + absolute: boolean, +): readonly RustModuleLocation[] | undefined { + if (!(config instanceof RustCargoTargets)) return undefined; + const targets = config.targetsByFile.get(caller); + if (targets === undefined) return undefined; + const parts = [...importedModule]; + const head = parts.shift(); + if (head === undefined) return undefined; + const roots = (): RustModuleLocation[] => [...targets].map((file) => ({ file, module: '' })); + let locations: readonly RustModuleLocation[]; + if (head === 'crate' || head === '$crate') { + locations = roots(); + } else if (head === 'self' || head === 'super') { + locations = [{ file: caller, module: ownerModule }]; + if (head === 'super') parts.unshift(head); + while (parts[0] === 'super') { + parts.shift(); + locations = locations.flatMap((location) => + location.module !== '' + ? [{ file: location.file, module: location.module.split('::').slice(0, -1).join('::') }] + : (config.moduleParents.get(location.file) ?? []), + ); + } + } else { + const local = + !absolute && + config.moduleChildren.get(moduleKey(caller, [ownerModule, head].filter(Boolean).join('::'))); + const mode = config.relativeUseByFile.get(caller); + const importedRoots = new Set(); + for (const target of targets) { + for (const file of config.rootImports.get(target)?.get(head) ?? []) importedRoots.add(file); + } + const external = [...importedRoots].map((file) => ({ file, module: '' })); + if (!absolute && mode === null) { + const legacy = roots().flatMap( + (root) => config.moduleChildren.get(moduleKey(root.file, head)) ?? [], + ); + locations = [...(local || external), ...legacy, ...external]; + if (locations.length === 0) return undefined; + } else if (local && mode !== false) { + locations = local; + } else { + if (importedRoots.size > 0) { + locations = external; + } else if (!absolute && mode === false) { + locations = roots(); + parts.unshift(head); + } else { + return undefined; + } + } + } + for (const part of parts) { + locations = locations.flatMap( + (location) => + config.moduleChildren.get( + moduleKey(location.file, [location.module, part].filter(Boolean).join('::')), + ) ?? [], + ); + } + return [ + ...new Map( + locations.map((location) => [moduleKey(location.file, location.module), location]), + ).values(), + ]; +} + +/** null retains mixed-edition ownership; undefined means no Cargo edition proof. */ +export function rustUsesRelativeImportPaths( + config: unknown, + file: string, +): boolean | null | undefined { + return config instanceof RustCargoTargets ? config.relativeUseByFile.get(file) : undefined; +} + /** Exact public-use evidence, independent of the capture's coarse reexport kind. */ export function rustCargoPubliclyReexports( config: unknown, @@ -443,17 +548,39 @@ export async function loadRustCargoTargets(repoPath: string): Promise { await discover(paths.map((file) => path.posix.join(path.posix.dirname(manifest), file))); } const roots = new Set(); + const relativeUseByTarget = new Map(); const targetsByManifest = new Map(); for (const [manifest, content] of contents) { const targets = cargoTargetRoots(manifest, content, files, workspaceEditions); if (targets === undefined) return undefined; targetsByManifest.set(manifest, targets); - for (const target of targets) roots.add(target); + const relative = + cargoPackageEdition(manifest, manifestData.get(manifest)!, workspaceEditions) !== '2015'; + for (const target of targets) { + roots.add(target); + const previous = relativeUseByTarget.get(target); + relativeUseByTarget.set( + target, + previous === undefined || previous === relative ? relative : null, + ); + } } const parser = new Parser(); parser.setLanguage(getLanguageGrammar(SupportedLanguages.Rust)); const targetsByFile = new Map>(); const publicUsesByFile = new Map>(); + const relativeUseByFile = new Map(); + const moduleChildren = new Map>(); + const moduleParents = new Map>(); + const addLocation = ( + index: Map>, + key: string, + location: RustModuleLocation, + ): void => { + let entries = index.get(key); + if (entries === undefined) index.set(key, (entries = new Map())); + entries.set(moduleKey(location.file, location.module), location); + }; // A file may be reached conventionally AND through #[path]. Those have // different submodule bases, so cache and visit both contexts separately. const childrenByFile = new Map>>(); @@ -470,10 +597,17 @@ export async function loadRustCargoTargets(repoPath: string): Promise { let owners = targetsByFile.get(file); if (!owners) targetsByFile.set(file, (owners = new Set())); owners.add(target); + const relative = relativeUseByTarget.get(target)!; + const previous = relativeUseByFile.get(file); + relativeUseByFile.set( + file, + previous === undefined || previous === relative ? relative : null, + ); let children = childrenByFile.get(key); if (!children) { const tree = parseSourceSafe(parser, await read(file)); const missing = new Set(); + const moduleLinks: RustModuleLink[] = []; let result = rustModuleFiles( tree.rootNode, file, @@ -481,9 +615,11 @@ export async function loadRustCargoTargets(repoPath: string): Promise { files, missing, file === target, + moduleLinks, ); if (result === undefined && missing.size > 0) { await discover(missing); + moduleLinks.length = 0; result = rustModuleFiles( tree.rootNode, file, @@ -491,12 +627,25 @@ export async function loadRustCargoTargets(repoPath: string): Promise { files, undefined, file === target, + moduleLinks, ); } if (result === undefined) return undefined; publicUsesByFile.set(file, rustPublicUses(tree.rootNode)); children = result; childrenByFile.set(key, children); + for (const link of moduleLinks) { + addLocation(moduleChildren, moduleKey(file, link.module), { + file: link.file, + module: link.inline ? link.module : '', + }); + if (!link.inline) { + addLocation(moduleParents, link.file, { + file, + module: link.module.split('::').slice(0, -1).join('::'), + }); + } + } } pending.push(...children); } @@ -505,6 +654,9 @@ export async function loadRustCargoTargets(repoPath: string): Promise { targetsByFile, cargoRootImports(manifestData, targetsByManifest), publicUsesByFile, + relativeUseByFile, + new Map([...moduleChildren].map(([key, entries]) => [key, [...entries.values()]])), + new Map([...moduleParents].map(([key, entries]) => [key, [...entries.values()]])), ); } catch { // I/O, parse or containment failure cannot establish target separation. diff --git a/gitnexus/src/core/ingestion/languages/rust/import-target.ts b/gitnexus/src/core/ingestion/languages/rust/import-target.ts index ef634a247..50e6ed589 100644 --- a/gitnexus/src/core/ingestion/languages/rust/import-target.ts +++ b/gitnexus/src/core/ingestion/languages/rust/import-target.ts @@ -1,3 +1,301 @@ +import type { ParsedFile, ParsedImport, Scope, SymbolDefinition } from 'gitnexus-shared'; +import type { ImportResolutionContext } from '../../scope-resolution/contract/scope-resolver.js'; +import { + rustCargoImportedModules, + rustIsExclusiveCargoRoot, + rustUsesRelativeImportPaths, +} from './cargo-targets.js'; + +interface ImportWorkspace { + readonly files: ReadonlyMap; + readonly scopes: ReadonlyMap; + readonly definitionsByFile: ReadonlyMap>; + readonly moduleDefinitions: ReadonlySet; + readonly importsByScope: ReadonlyMap>; + readonly reexportsByFile: ReadonlyMap>; +} + +interface ScopedReexport { + readonly module: string; + readonly imported: Extract; +} + +const importWorkspace = new WeakMap(); + +function workspaceFor(context: ImportResolutionContext): ImportWorkspace { + let workspace = importWorkspace.get(context.parsedFiles); + if (workspace !== undefined) return workspace; + const definitionsByFile = new Map>(); + const moduleDefinitions = new Set(); + const importsByScope = new Map>(); + const reexportsByFile = new Map>(); + const scopes = new Map( + context.parsedFiles.flatMap((file) => file.scopes.map((scope) => [scope.id, scope] as const)), + ); + const moduleByScope = new Map(); + const moduleAt = (scopeId: string): string => { + const cached = moduleByScope.get(scopeId); + if (cached !== undefined) return cached; + const scope = scopes.get(scopeId); + if (scope === undefined) return ''; + const parent = scope.parent === null ? '' : moduleAt(scope.parent); + const own = + scope.kind === 'Namespace' + ? scope.ownedDefs + .find((def) => def.type === 'Namespace') + ?.qualifiedName?.split('.') + .pop() + : undefined; + const module = [parent, own].filter(Boolean).join('::'); + moduleByScope.set(scopeId, module); + return module; + }; + for (const file of context.parsedFiles) { + const byName = new Map(); + const reexports = new Map(); + definitionsByFile.set(file.filePath, byName); + reexportsByFile.set(file.filePath, reexports); + for (const def of file.localDefs) { + const name = def.qualifiedName?.split('.').pop(); + if (!name) continue; + const entries = byName.get(name) ?? []; + entries.push(def); + byName.set(name, entries); + } + for (const scope of file.scopes) { + if (scope.kind !== 'Module' && scope.kind !== 'Namespace') continue; + for (const bindings of scope.bindings.values()) { + for (const binding of bindings) { + if (binding.origin === 'local') moduleDefinitions.add(binding.def.nodeId); + } + } + } + for (const imp of file.parsedImports) { + if (!('localName' in imp)) continue; + const scope = imp.declaredAtScope ?? file.moduleScope; + let names = importsByScope.get(scope); + if (names === undefined) importsByScope.set(scope, (names = new Map())); + const entries = names.get(imp.localName) ?? []; + entries.push(imp); + names.set(imp.localName, entries); + const owner = scopes.get(scope); + if (imp.kind === 'reexport' && (owner?.kind === 'Module' || owner?.kind === 'Namespace')) { + const modules = reexports.get(imp.localName) ?? []; + modules.push({ module: moduleAt(scope), imported: imp }); + reexports.set(imp.localName, modules); + } + } + } + workspace = { + files: new Map(context.parsedFiles.map((file) => [file.filePath, file])), + scopes, + definitionsByFile, + moduleDefinitions, + importsByScope, + reexportsByFile, + }; + importWorkspace.set(context.parsedFiles, workspace); + return workspace; +} + +/** Named imports can use Cargo's exact root and #[path] topology. */ +function cargoNamedTarget( + segments: readonly string[], + targetRaw: string, + fromFile: string, + allFilePaths: ReadonlySet, + config: unknown, + context: ImportResolutionContext, + visited: ReadonlySet, +): { readonly target: string | readonly string[] | null } | undefined { + const imported = context.parsedImport; + if ( + imported === undefined || + !['named', 'alias', 'reexport'].includes(imported.kind) || + !('importedName' in imported) + ) + return undefined; + if (rustUsesRelativeImportPaths(config, fromFile) === undefined) return undefined; + const workspace = workspaceFor(context); + const parsed = workspace.files.get(fromFile); + if (parsed === undefined) return undefined; + if ( + !targetRaw.startsWith('::') && + !['crate', '$crate', 'self', 'super'].includes(segments[0] ?? '') + ) { + let lexical = workspace.scopes.get(imported.declaredAtScope ?? parsed.moduleScope); + while (lexical !== undefined) { + // A lexical alias owns this path head even when its target is unknown. + // Cargo's extern-prelude name must not bypass that binding. + if ( + workspace.importsByScope + .get(lexical.id) + ?.get(segments[0]) + ?.some((imp) => imp !== imported) + ) { + return { target: null }; + } + const localTypes = (lexical.bindings.get(segments[0]) ?? []).filter( + (binding) => + binding.origin === 'local' && + ['Namespace', 'Class', 'Struct', 'Enum', 'Trait', 'TypeAlias', 'Union'].includes( + binding.def.type, + ), + ); + if (localTypes.length > 0) { + if (localTypes.some((binding) => binding.def.type !== 'Namespace')) return { target: null }; + // Function/block-local modules are not published in Cargo's module tree. + if (lexical.kind !== 'Module' && lexical.kind !== 'Namespace') return undefined; + break; + } + if (lexical.kind === 'Namespace') break; + lexical = lexical.parent === null ? undefined : workspace.scopes.get(lexical.parent); + } + } + const owner: string[] = []; + let scope = workspace.scopes.get(imported.declaredAtScope ?? parsed.moduleScope); + while (scope !== undefined) { + if (scope.kind === 'Namespace') { + const name = scope.ownedDefs + .find((def) => def.type === 'Namespace') + ?.qualifiedName?.split('.') + .pop(); + if (name) owner.unshift(name); + } + scope = scope.parent === null ? undefined : workspace.scopes.get(scope.parent); + } + const modules = rustCargoImportedModules( + config, + fromFile, + segments.slice(0, -1), + owner.join('::'), + targetRaw.startsWith('::'), + ); + if (modules === undefined) return undefined; + const candidates = new Set(); + for (const location of modules) { + if (!allFilePaths.has(location.file)) continue; + const definitions = + workspace.definitionsByFile.get(location.file)?.get(imported.importedName) ?? []; + // The shared finalizer matches a file-wide simple name. A precise module + // receipt must not let it pick a different namespace or a same-file homonym. + if (definitions.length > 1) return { target: null }; + if ( + definitions.length === 1 && + (!workspace.moduleDefinitions.has(definitions[0]!.nodeId) || + (definitions[0]!.namespacePrefix ?? '').replaceAll('.', '::') !== location.module) + ) + continue; + if (definitions.length === 0) { + const reexports = + workspace.reexportsByFile.get(location.file)?.get(imported.importedName) ?? []; + // The closure is file-wide too. A sole explicit re-export in the named + // module is the identity evidence it needs; other namespaces or competing + // re-exports cannot safely be collapsed to one file-wide name. + if (reexports.length !== 1) return { target: null }; + const reexport = reexports[0]!; + if (reexport.module !== location.module) continue; + if (location.module !== '') { + // Namespace-scoped exports deliberately do not enter the shared + // file-level closure. Follow this exact receipt instead. A file-only + // target cannot communicate an alias's changed symbol spelling. + if (reexport.imported.importedName !== imported.importedName) return { target: null }; + const target = resolveRustImportTarget( + reexport.imported.targetRaw, + location.file, + allFilePaths, + config, + { parsedFiles: context.parsedFiles, parsedImport: reexport.imported }, + visited, + ); + if (target === null) return { target: null }; + for (const file of typeof target === 'string' ? [target] : target) candidates.add(file); + continue; + } + } + candidates.add(location.file); + } + const files = [...candidates]; + return { target: files.length === 0 ? null : files.length === 1 ? files[0] : files }; +} + +/** A declared module is evidence; a similarly named file elsewhere is not. */ +function declaredRelativeTarget( + segments: readonly string[], + fromFile: string, + allFilePaths: ReadonlySet, + resolutionConfig: unknown, + context: ImportResolutionContext, +): { readonly target: string | readonly string[] | null } | undefined { + const workspace = workspaceFor(context); + const parsed = workspace.files.get(fromFile); + if (parsed === undefined) return undefined; + let scope = workspace.scopes.get(context.parsedImport?.declaredAtScope ?? parsed.moduleScope); + const visited = new Set(); + while (scope !== undefined && !visited.has(scope.id)) { + visited.add(scope.id); + const ownNamespace = + scope.kind === 'Namespace' + ? scope.ownedDefs.find((def) => def.type === 'Namespace')?.nodeId + : undefined; + const declarations = new Set( + (scope.bindings.get(segments[0]) ?? []) + .filter( + (binding) => + binding.origin === 'local' && + binding.def.type === 'Namespace' && + binding.def.nodeId !== ownNamespace, + ) + .map((binding) => binding.def.nodeId), + ); + if (declarations.size > 0) { + if (declarations.size !== 1) return { target: null }; + const inline: string[] = []; + let owner: Scope | undefined = scope; + while (owner !== undefined) { + if (owner.kind === 'Namespace') { + const name = owner.ownedDefs + .find((def) => def.type === 'Namespace') + ?.qualifiedName?.split('.') + .pop(); + if (name !== undefined) inline.unshift(name); + } + owner = owner.parent === null ? undefined : workspace.scopes.get(owner.parent); + } + const normalized = fromFile.replace(/\\/g, '/'); + const slash = normalized.lastIndexOf('/'); + const directory = slash === -1 ? '' : normalized.slice(0, slash); + const filename = normalized.slice(slash + 1); + const ownsDirectory = + ['lib.rs', 'main.rs', 'mod.rs'].includes(filename) || + rustIsExclusiveCargoRoot(resolutionConfig, fromFile); + const base = ownsDirectory ? directory : normalized.replace(/\.rs$/, ''); + const modulePath = [...inline, ...segments]; + const target = resolveModulePath(modulePath, base, allFilePaths); + if (ownsDirectory || rustUsesRelativeImportPaths(resolutionConfig, fromFile) !== undefined) { + return { target }; + } + // A standalone source may be passed directly to rustc as a custom crate + // root. Without Cargo membership, both its module layout and that root + // layout are possible; keep ambiguity when both contain the module. + const rootTarget = resolveModulePath(modulePath, directory, allFilePaths); + const candidates = [ + ...new Set( + [target, rootTarget].flatMap((candidate) => + candidate === null ? [] : typeof candidate === 'string' ? [candidate] : candidate, + ), + ), + ]; + return { + target: + candidates.length === 0 ? null : candidates.length === 1 ? candidates[0] : candidates, + }; + } + scope = scope.parent === null ? undefined : workspace.scopes.get(scope.parent); + } + return undefined; +} + /** * Resolve a Rust `use` import path to a repo-relative file path. * @@ -11,13 +309,33 @@ export function resolveRustImportTarget( targetRaw: string, fromFile: string, allFilePaths: ReadonlySet, - _resolutionConfig?: unknown, + resolutionConfig?: unknown, + context?: ImportResolutionContext, + visited: ReadonlySet = new Set(), ): string | readonly string[] | null { if (!targetRaw) return null; const segments = targetRaw.split('::').filter(Boolean); if (segments.length === 0) return null; + if (context !== undefined) { + // Inline re-export chains are a provider-only supplement to the shared + // SCC closure. Cycles and excessive depth conservatively stay unresolved. + const imported = context.parsedImport; + if (imported !== undefined && (visited.has(imported) || visited.size >= 64)) return null; + const nextVisited = imported === undefined ? visited : new Set([...visited, imported]); + const cargo = cargoNamedTarget( + segments, + targetRaw, + fromFile, + allFilePaths, + resolutionConfig, + context, + nextVisited, + ); + if (cargo !== undefined) return cargo.target; + } + const fromNormalized = fromFile.replace(/\\/g, '/'); const fromDir = fromNormalized.includes('/') ? fromNormalized.slice(0, fromNormalized.lastIndexOf('/')) @@ -39,13 +357,30 @@ export function resolveRustImportTarget( return resolveModulePath(restPath, fromDir, allFilePaths); } + const relativeMode = rustUsesRelativeImportPaths(resolutionConfig, fromFile); + const relative = + relativeMode !== false && !targetRaw.startsWith('::') && context !== undefined + ? declaredRelativeTarget(segments, fromFile, allFilePaths, resolutionConfig, context) + : undefined; + // Standalone sources have no edition receipt. Prefer a declared local module, + // matching modern Rust; context-free callers keep their legacy resolution. + if (relative !== undefined && relativeMode !== null) return relative.target; + // External crate — try workspace-level resolution const workspaceResult = resolveWorkspaceCrate(segments, allFilePaths); - if (workspaceResult !== null) return workspaceResult; - - // Fallback: treat as implicit crate-relative (Rust 2015 edition or - // when the first segment matches a sibling module name). - return resolveModulePath(segments, findSrcRoot(fromNormalized), allFilePaths); + const legacyTarget = + workspaceResult ?? resolveModulePath(segments, findSrcRoot(fromNormalized), allFilePaths); + if (relative === undefined) return legacyTarget; + // A file shared by 2015 and modern targets has both interpretations. Keep + // every possible target instead of selecting an edition by visitation order. + const candidates = [ + ...new Set( + [relative.target, legacyTarget].flatMap((target) => + target === null ? [] : typeof target === 'string' ? [target] : target, + ), + ), + ]; + return candidates.length === 0 ? null : candidates.length === 1 ? candidates[0] : candidates; } function findSrcRoot(filePath: string): string { diff --git a/gitnexus/src/core/ingestion/languages/rust/lexical-bindings.ts b/gitnexus/src/core/ingestion/languages/rust/lexical-bindings.ts new file mode 100644 index 000000000..5fada0b77 --- /dev/null +++ b/gitnexus/src/core/ingestion/languages/rust/lexical-bindings.ts @@ -0,0 +1,144 @@ +import type { NameClaim } from 'gitnexus-shared'; +import { nodeToCapture, walkNamedTree, type SyntaxNode } from '../../utils/ast-helpers.js'; +import { splitRustUseDeclaration } from './import-decomposer.js'; + +const SCOPE_TYPES = new Set([ + 'source_file', + 'function_item', + 'function_signature_item', + 'closure_expression', + 'block', + 'if_expression', + 'match_expression', + 'match_arm', + 'for_expression', + 'while_expression', + 'loop_expression', + 'mod_item', + 'struct_item', + 'trait_item', + 'impl_item', + 'enum_item', + 'union_item', +]); + +/** Pattern identifiers own values even when no useful target/type was indexed. */ +function patternNames(pattern: SyntaxNode): string[] { + if (pattern.type === 'identifier' || pattern.type === 'shorthand_field_identifier') { + return [pattern.text]; + } + if ( + ['scoped_identifier', 'scoped_type_identifier', 'type_identifier', 'generic_type'].includes( + pattern.type, + ) + ) { + return []; + } + if (pattern.type === 'field_pattern') { + const nested = pattern.childForFieldName('pattern'); + if (nested !== null) return patternNames(nested); + const name = pattern.childForFieldName('name'); + return name?.type === 'shorthand_field_identifier' ? [name.text] : []; + } + const type = pattern.childForFieldName('type'); + return pattern.namedChildren.flatMap((child) => + child.id === type?.id ? [] : patternNames(child), + ); +} + +function ownerOf(node: SyntaxNode | null): SyntaxNode | null { + while (node !== null && !SCOPE_TYPES.has(node.type)) node = node.parent; + return node; +} + +/** One AST pass. Captures carry these plain facts through serial and worker parsing. */ +export function collectRustNameClaims(root: SyntaxNode): ReadonlyMap { + const byScope = new Map(); + const add = ( + owner: SyntaxNode | null, + names: readonly string[], + anchor: SyntaxNode, + extra: Partial = {}, + ) => { + if (owner === null || names.length === 0) return; + const claims = byScope.get(owner.id) ?? []; + for (const name of names) { + if (name === '_') continue; + claims.push({ + name, + range: nodeToCapture('', anchor).range, + kind: 'binding', + purpose: 'value', + ...extra, + }); + } + byScope.set(owner.id, claims); + }; + walkNamedTree(root, (node) => { + if (node.type === 'use_declaration') { + const names = splitRustUseDeclaration(node) + .filter((match) => match['@import.kind']?.text !== 'wildcard') + .flatMap((match) => + match['@import.name'] === undefined ? [] : [match['@import.name'].text], + ); + add(ownerOf(node.parent), names, node, { kind: 'import', purpose: 'both', hoisted: true }); + } else if (node.type === 'let_declaration') { + const pattern = node.childForFieldName('pattern'); + if (pattern !== null) + add(ownerOf(node.parent), patternNames(pattern), node, { + availableFrom: { startLine: node.endPosition.row + 1, startCol: node.endPosition.column }, + inactive: 'outer', + }); + } else if (node.type === 'parameter') { + const pattern = node.childForFieldName('pattern'); + if (pattern !== null) add(ownerOf(node.parent), patternNames(pattern), node); + } else if (node.type === 'closure_expression') { + const parameters = node.childForFieldName('parameters'); + for (const parameter of parameters?.namedChildren ?? []) { + if (parameter.type !== 'parameter') add(node, patternNames(parameter), parameter); + } + } else if (node.type === 'for_expression') { + const pattern = node.childForFieldName('pattern'); + if (pattern !== null) add(node.childForFieldName('body'), patternNames(pattern), pattern); + } else if (node.type === 'match_arm') { + const pattern = node.childForFieldName('pattern'); + if (pattern !== null) add(node, patternNames(pattern), pattern); + } else if (node.type === 'let_condition') { + const pattern = node.childForFieldName('pattern'); + let control = node.parent; + while (control !== null && !['if_expression', 'while_expression'].includes(control.type)) + control = control.parent; + if (pattern !== null && control !== null) { + add( + control.childForFieldName('consequence') ?? control.childForFieldName('body'), + patternNames(pattern), + pattern, + ); + } + } else if ( + [ + 'function_item', + 'const_item', + 'static_item', + 'struct_item', + 'enum_item', + 'union_item', + 'trait_item', + 'type_item', + 'mod_item', + ].includes(node.type) + ) { + const name = node.childForFieldName('name'); + if (name !== null) + add(ownerOf(node.parent), [name.text], node, { + hoisted: true, + purpose: ['function_item', 'const_item', 'static_item'].includes(node.type) + ? 'value' + : ['trait_item', 'type_item'].includes(node.type) + ? 'type' + : 'both', + }); + } + }); + return byScope; +} diff --git a/gitnexus/src/core/ingestion/languages/rust/query.ts b/gitnexus/src/core/ingestion/languages/rust/query.ts index ba20a436c..02b9dde2a 100644 --- a/gitnexus/src/core/ingestion/languages/rust/query.ts +++ b/gitnexus/src/core/ingestion/languages/rust/query.ts @@ -15,6 +15,7 @@ const RUST_SCOPE_QUERY = ` (block) @scope.block (if_expression) @scope.block (match_expression) @scope.block +(match_arm) @scope.block (for_expression) @scope.block (while_expression) @scope.block (loop_expression) @scope.block diff --git a/gitnexus/src/core/ingestion/languages/rust/range-binding.ts b/gitnexus/src/core/ingestion/languages/rust/range-binding.ts index 593716cf9..56fbec352 100644 --- a/gitnexus/src/core/ingestion/languages/rust/range-binding.ts +++ b/gitnexus/src/core/ingestion/languages/rust/range-binding.ts @@ -1,11 +1,12 @@ -import type { ParsedFile, Scope, ScopeId, TypeRef } from 'gitnexus-shared'; +import type { ParsedFile, Scope, TypeRef } from 'gitnexus-shared'; +import { lookupLexicalName } from 'gitnexus-shared'; import type { ScopeResolutionIndexes } from '../../model/scope-resolution-indexes.js'; import { getRustParser } from './query.js'; import { getTreeSitterBufferSize } from '../../constants.js'; import { parseSourceSafe, ParseTimeoutError } from '../../../tree-sitter/safe-parse.js'; import type { SyntaxNode } from '../../utils/ast-helpers.js'; import { logger } from '../../../logger.js'; -import { lookupBindingsAt } from '../../scope-resolution/scope/walkers.js'; +import { lookupBindingsAt, lookupNameClaim } from '../../scope-resolution/scope/walkers.js'; /** * Populate type bindings for patterns and iterators that the tree-sitter @@ -19,6 +20,57 @@ import { lookupBindingsAt } from '../../scope-resolution/scope/walkers.js'; */ type RustTree = ReturnType['parse']>; +interface ProducerReturn { + readonly rawType: string; + readonly typeRef: TypeRef; +} + +function nodeRangeKey(node: SyntaxNode): string { + return `${node.startPosition.row + 1}:${node.startPosition.column}-${node.endPosition.row + 1}:${node.endPosition.column}`; +} + +function indexScopeRanges(parsed: ParsedFile): ReadonlyMap { + return new Map( + parsed.scopes.map((scope) => [ + `${scope.range.startLine}:${scope.range.startCol}-${scope.range.endLine}:${scope.range.endCol}`, + scope, + ]), + ); +} + +function lexicalScopeForNode( + node: SyntaxNode, + scopeRanges: ReadonlyMap, +): Scope | undefined { + let current: SyntaxNode | null = node; + while (current !== null) { + const scope = scopeRanges.get(nodeRangeKey(current)); + if (scope !== undefined) return scope; + current = current.parent; + } + return undefined; +} + +/** null means claimed but unusable; undefined alone permits the legacy fallback. */ +function producerAt( + func: SyntaxNode, + scopeRanges: ReadonlyMap, + indexes: ScopeResolutionIndexes, + returnTypes: ReadonlyMap, +): ProducerReturn | null | undefined { + const scope = lexicalScopeForNode(func, scopeRanges); + if (scope === undefined || !indexes.scopeTree.has(scope.id)) return undefined; + const claim = lookupNameClaim(scope.id, func.text, indexes, { + position: { startLine: func.startPosition.row + 1, startCol: func.startPosition.column }, + purpose: 'value', + }); + if (claim.status === 'absent') return undefined; + if (claim.status === 'blocked') return null; + const defs = new Map(claim.bindings.map((binding) => [binding.def.nodeId, binding.def])); + if (defs.size !== 1) return null; + return returnTypes.get(defs.keys().next().value!) ?? null; +} + /** * Hold parsed trees for reuse across both prepass loops only when the whole * Rust source fits this budget. Trees are much larger than their source, so a @@ -72,6 +124,11 @@ export function populateRustRangeBindings( ): void { const parser = getRustParser(); const allReturnTypes = new Map(); + const producerReturns = new Map(); + const fieldsByDefinition = new Map>(); + const scopeRangesByFile = new Map( + parsedFiles.map((parsed) => [parsed.filePath, indexScopeRanges(parsed)]), + ); const ambiguousReturnTypes = new Set(); const allFieldTypes = new Map>(); const ambiguousFieldTypes = new Set(); @@ -94,6 +151,8 @@ export function populateRustRangeBindings( for (const parsed of parsedFiles) { const tree = getOrParseTree(parser, parsed.filePath, ctx, treeStore); if (tree === null) continue; + const scopeRanges = scopeRangesByFile.get(parsed.filePath)!; + const scopeMap = new Map(parsed.scopes.map((scope) => [scope.id, scope])); for (const fn of tree.rootNode.descendantsOfType('function_item')) { const nameNode = fn.childForFieldName('name'); @@ -111,6 +170,33 @@ export function populateRustRangeBindings( } // Full-generic record per defining file for import-disambiguated lookup. recordByFile(returnTypeByFile, parsed.filePath, name, retType.text); + const declarationScope = scopeRanges.get(nodeRangeKey(fn)); + if (declarationScope !== undefined) { + const declaredReturn = + declarationScope.parent === null + ? undefined + : scopeMap.get(declarationScope.parent)?.typeBindings.get(name); + for (const def of declarationScope.ownedDefs) { + if (def.type !== 'Function' && def.type !== 'Method') continue; + producerReturns.set(def.nodeId, { + rawType: retType.text, + typeRef: { + rawName: + declaredReturn?.source === 'return-annotation' + ? declaredReturn.rawName + : normalizeFieldType(retType.text), + declaredSpelling: retType.text, + declaredAtScope: declarationScope.id, + lookupPosition: { + startLine: retType.startPosition.row + 1, + startCol: retType.startPosition.column, + }, + lookupPurpose: 'type', + source: 'annotation', + }, + }); + } + } } } @@ -119,11 +205,26 @@ export function populateRustRangeBindings( const body = structNode.childForFieldName('body'); if (nameNode === null || body === null) continue; const fields = new Map(); + const fieldRefs = new Map(); + const declarationScope = scopeRanges.get(nodeRangeKey(structNode)); for (const field of body.descendantsOfType('field_declaration')) { const fieldName = field.childForFieldName('name'); const fieldType = field.childForFieldName('type'); if (fieldName !== null && fieldType !== null) { fields.set(fieldName.text, normalizeFieldType(fieldType.text)); + if (declarationScope !== undefined) { + fieldRefs.set(fieldName.text, { + rawName: normalizeFieldType(fieldType.text), + declaredSpelling: fieldType.text, + declaredAtScope: declarationScope.id, + lookupPosition: { + startLine: fieldType.startPosition.row + 1, + startCol: fieldType.startPosition.column, + }, + lookupPurpose: 'type', + source: 'annotation', + }); + } } } if (fields.size > 0) { @@ -138,13 +239,15 @@ export function populateRustRangeBindings( } // Full-generic record per defining file for import-disambiguated lookup. recordByFile(fieldTypeByFile, parsed.filePath, name, fields); + for (const def of declarationScope?.ownedDefs ?? []) { + if (def.type === 'Struct') fieldsByDefinition.set(def.nodeId, fieldRefs); + } } } // Publish per-type member bindings for the whole workspace before resolving // assignments. Otherwise an importer processed before its defining file can // miss a field or identity-method type solely because of file order. - const scopeMap = new Map(parsed.scopes.map((scope) => [scope.id, scope])); processFieldTypeBindings(tree.rootNode, parsed, scopeMap); processIdentityMethodBindings(parsed); } @@ -154,6 +257,7 @@ export function populateRustRangeBindings( if (tree === null) continue; const scopeMap = new Map(parsed.scopes.map((s) => [s.id, s])); + const scopeRanges = scopeRangesByFile.get(parsed.filePath)!; const moduleScope = parsed.scopes.find((s) => s.kind === 'Module'); if (moduleScope === undefined) continue; @@ -165,8 +269,10 @@ export function populateRustRangeBindings( allReturnTypes, indexes, returnTypeByFile, + producerReturns, + scopeRanges, ); - processPatternBindings(tree.rootNode, parsed, scopeMap, moduleScope); + processPatternBindings(tree.rootNode, parsed, scopeMap, moduleScope, scopeRanges); processStructDestructuring( tree.rootNode, parsed, @@ -175,6 +281,8 @@ export function populateRustRangeBindings( allFieldTypes, indexes, fieldTypeByFile, + fieldsByDefinition, + scopeRanges, ); processPendingAssignments( tree.rootNode, @@ -183,6 +291,9 @@ export function populateRustRangeBindings( scopeMap, moduleScope, allReturnTypes, + indexes, + producerReturns, + scopeRanges, ); } } @@ -322,6 +433,8 @@ function processForLoops( allReturnTypes: ReadonlyMap, indexes: ScopeResolutionIndexes, returnTypeByFile: ReadonlyMap>, + producerReturns: ReadonlyMap, + scopeRanges: ReadonlyMap, ): void { for (const forNode of root.descendantsOfType('for_expression')) { const patternNode = forNode.childForFieldName('pattern'); @@ -339,10 +452,13 @@ function processForLoops( allReturnTypes, indexes, returnTypeByFile, + producerReturns, + scopeRanges, ); if (elementType === null) continue; - const targetScope = findEnclosingFunctionScope(forNode, scopeMap) ?? moduleScope; + const targetScope = + lexicalScopeForNode(forNode.childForFieldName('body') ?? forNode, scopeRanges) ?? moduleScope; injectTypeBinding(targetScope, varName, elementType); } } @@ -352,6 +468,7 @@ function processPatternBindings( parsed: ParsedFile, scopeMap: ReadonlyMap, moduleScope: Scope, + scopeRanges: ReadonlyMap, ): void { for (const nodeType of ['let_condition', 'match_arm'] as const) { for (const node of root.descendantsOfType(nodeType)) { @@ -362,12 +479,12 @@ function processPatternBindings( if (patternNode === null) continue; if (patternNode.type === 'captured_pattern') { - processCapturedPattern(patternNode, node, parsed, scopeMap, moduleScope); + processCapturedPattern(patternNode, node, parsed, scopeMap, moduleScope, scopeRanges); continue; } if (patternNode.type === 'tuple_struct_pattern') { - processTupleStructPattern(patternNode, node, parsed, scopeMap, moduleScope); + processTupleStructPattern(patternNode, node, parsed, scopeMap, moduleScope, scopeRanges); } } } @@ -377,8 +494,9 @@ function processCapturedPattern( patternNode: SyntaxNode, contextNode: SyntaxNode, _parsed: ParsedFile, - scopeMap: ReadonlyMap, + _scopeMap: ReadonlyMap, moduleScope: Scope, + scopeRanges: ReadonlyMap, ): void { const varNode = patternNode.namedChildren.find((c) => c.type === 'identifier'); const structPatternNode = patternNode.namedChildren.find((c) => c.type === 'struct_pattern'); @@ -387,7 +505,7 @@ function processCapturedPattern( const typeName = structPatternNode.childForFieldName('type')?.text; if (typeName === undefined) return; - const targetScope = findEnclosingFunctionScope(contextNode, scopeMap) ?? moduleScope; + const targetScope = patternBindingScope(contextNode, scopeRanges) ?? moduleScope; injectTypeBinding(targetScope, varNode.text, typeName); } @@ -397,6 +515,7 @@ function processTupleStructPattern( parsed: ParsedFile, scopeMap: ReadonlyMap, moduleScope: Scope, + scopeRanges: ReadonlyMap, ): void { const wrapperNode = patternNode.childForFieldName('type'); if (wrapperNode === null) return; @@ -424,7 +543,14 @@ function processTupleStructPattern( const sourceVarName = sourceVarNode.type === 'identifier' ? sourceVarNode.text : null; if (sourceVarName === null) return; - const sourceType = lookupTypeInScopes(sourceVarName, contextNode, parsed, scopeMap, moduleScope); + const sourceType = lookupTypeInScopes( + sourceVarName, + contextNode, + parsed, + scopeMap, + moduleScope, + scopeRanges, + ); if (sourceType === null) return; let resolvedType: string | null = null; @@ -444,10 +570,26 @@ function processTupleStructPattern( if (resolvedType === null) return; - const targetScope = findEnclosingFunctionScope(contextNode, scopeMap) ?? moduleScope; + const targetScope = patternBindingScope(contextNode, scopeRanges) ?? moduleScope; injectTypeBinding(targetScope, varName, resolvedType); } +function patternBindingScope( + contextNode: SyntaxNode, + scopeRanges: ReadonlyMap, +): Scope | undefined { + if (contextNode.type === 'match_arm') return lexicalScopeForNode(contextNode, scopeRanges); + let control = contextNode.parent; + while ( + control !== null && + control.type !== 'if_expression' && + control.type !== 'while_expression' + ) + control = control.parent; + const body = control?.childForFieldName('consequence') ?? control?.childForFieldName('body'); + return body == null ? undefined : lexicalScopeForNode(body, scopeRanges); +} + function processStructDestructuring( root: SyntaxNode, parsed: ParsedFile, @@ -456,6 +598,8 @@ function processStructDestructuring( allFieldTypes: ReadonlyMap>, indexes: ScopeResolutionIndexes, fieldTypeByFile: ReadonlyMap>>, + fieldsByDefinition: ReadonlyMap>, + scopeRanges: ReadonlyMap, ): void { for (const letNode of root.descendantsOfType('let_declaration')) { const patternNode = letNode.childForFieldName('pattern'); @@ -467,7 +611,22 @@ function processStructDestructuring( const valueNode = letNode.childForFieldName('value'); if (valueNode === null) continue; - const targetScope = findEnclosingFunctionScope(letNode, scopeMap) ?? moduleScope; + const targetScope = lexicalScopeForNode(letNode, scopeRanges) ?? moduleScope; + const claim = indexes.scopeTree.has(targetScope.id) + ? lookupNameClaim(targetScope.id, typeName, indexes, { + position: { + startLine: patternNode.startPosition.row + 1, + startCol: patternNode.startPosition.column, + }, + purpose: 'type', + }) + : undefined; + const claimedDefs = new Set(claim?.bindings.map((binding) => binding.def.nodeId)); + const claimedFields = + claimedDefs.size === 1 + ? fieldsByDefinition.get(claimedDefs.values().next().value!) + : undefined; + if (claim !== undefined && claim.status !== 'absent' && claimedFields === undefined) continue; for (const fieldNode of patternNode.namedChildren) { let fieldName: string | undefined; @@ -477,6 +636,15 @@ function processStructDestructuring( fieldName = fieldNode.childForFieldName('name')?.text; } if (fieldName === undefined) continue; + const fieldPattern = fieldNode.childForFieldName('pattern'); + const boundName = fieldPattern === null ? fieldName : extractVarName(fieldPattern); + if (boundName === null) continue; + + if (claim !== undefined && claim.status !== 'absent') { + const fieldRef = claimedFields?.get(fieldName); + if (fieldRef !== undefined) injectTypeBinding(targetScope, boundName, fieldRef); + continue; + } let fieldType = lookupFieldType(typeName, fieldName, parsed, scopeMap, moduleScope); if (fieldType === null) { @@ -489,7 +657,7 @@ function processStructDestructuring( fieldType = fields?.get(fieldName) ?? null; } if (fieldType !== null) { - injectTypeBinding(targetScope, fieldName, fieldType); + injectTypeBinding(targetScope, boundName, fieldType); } } } @@ -527,6 +695,9 @@ function processPendingAssignments( scopeMap: ReadonlyMap, moduleScope: Scope, allReturnTypes: ReadonlyMap, + indexes: ScopeResolutionIndexes, + producerReturns: ReadonlyMap, + scopeRanges: ReadonlyMap, ): void { for (let pass = 0; pass < 3; pass++) { for (const letNode of root.descendantsOfType('let_declaration')) { @@ -535,16 +706,51 @@ function processPendingAssignments( const varName = extractVarName(patternNode); if (varName === null) continue; - const targetScope = findEnclosingFunctionScope(letNode, scopeMap) ?? moduleScope; - if (targetScope.typeBindings.has(varName)) continue; - const valueNode = letNode.childForFieldName('value'); if (valueNode === null) continue; + const bindingScope = lexicalScopeForNode(letNode, scopeRanges) ?? moduleScope; + const func = + valueNode.type === 'call_expression' ? valueNode.childForFieldName('function') : null; + if (func?.type === 'identifier' && letNode.childForFieldName('type') === null) { + const producer = producerAt(func, scopeRanges, indexes, producerReturns); + if (producer !== undefined) { + const bindings = bindingScope.typeBindings as Map; + if (producer === null) bindings.delete(varName); + else + bindings.set(varName, { + ...producer.typeRef, + bindingRange: { + startLine: letNode.startPosition.row + 1, + startCol: letNode.startPosition.column, + endLine: letNode.endPosition.row + 1, + endCol: letNode.endPosition.column, + }, + }); + continue; + } + } + + const targetScope = bindingScope; + const existing = targetScope.typeBindings.get(varName); + if ( + existing !== undefined && + existing.source !== 'constructor-inferred' && + existing.source !== 'assignment-inferred' + ) + continue; + if (valueNode.type === 'identifier') { - const rhsType = lookupTypeInScopes(valueNode.text, letNode, parsed, scopeMap, moduleScope); + const rhsType = lookupTypeInScopes( + valueNode.text, + letNode, + parsed, + scopeMap, + moduleScope, + scopeRanges, + ); if (rhsType !== null) { - injectTypeBinding(targetScope, varName, rhsType); + injectTypeBinding(targetScope, varName, rhsType, true); continue; } } @@ -559,11 +765,12 @@ function processPendingAssignments( parsed, scopeMap, moduleScope, + scopeRanges, ); if (receiverType !== null) { const fieldType = findFieldTypeAcrossFiles(receiverType, field.text, allParsedFiles); if (fieldType !== null) { - injectTypeBinding(targetScope, varName, fieldType); + injectTypeBinding(targetScope, varName, fieldType, true); } } } @@ -581,6 +788,7 @@ function processPendingAssignments( parsed, scopeMap, moduleScope, + scopeRanges, ); if (receiverType !== null) { const retType = findMethodReturnTypeAcrossFiles( @@ -589,7 +797,7 @@ function processPendingAssignments( allParsedFiles, ); if (retType !== null) { - injectTypeBinding(targetScope, varName, retType); + injectTypeBinding(targetScope, varName, retType, true); } } } @@ -598,7 +806,7 @@ function processPendingAssignments( if (func !== null && func.type === 'identifier') { const rawReturn = allReturnTypes.get(func.text); if (rawReturn !== undefined) { - injectTypeBinding(targetScope, varName, normalizeFieldType(rawReturn)); + injectTypeBinding(targetScope, varName, normalizeFieldType(rawReturn), true); } } } @@ -614,14 +822,23 @@ function resolveIterableElementType( allReturnTypes: ReadonlyMap, indexes: ScopeResolutionIndexes, returnTypeByFile: ReadonlyMap>, -): string | null { + producerReturns: ReadonlyMap, + scopeRanges: ReadonlyMap, +): string | TypeRef | null { let iterableNode = valueNode; if (iterableNode.type === 'reference_expression') { iterableNode = iterableNode.firstNamedChild ?? iterableNode; } if (iterableNode.type === 'identifier') { - const rawType = lookupTypeInScopes(iterableNode.text, valueNode, parsed, scopeMap, moduleScope); + const rawType = lookupTypeInScopes( + iterableNode.text, + valueNode, + parsed, + scopeMap, + moduleScope, + scopeRanges, + ); if (rawType !== null) return unwrapGeneric(rawType); } @@ -632,12 +849,29 @@ function resolveIterableElementType( if (func.type === 'field_expression') { const receiver = func.childForFieldName('value'); if (receiver !== null && receiver.type === 'identifier') { - const rawType = lookupTypeInScopes(receiver.text, valueNode, parsed, scopeMap, moduleScope); + const rawType = lookupTypeInScopes( + receiver.text, + valueNode, + parsed, + scopeMap, + moduleScope, + scopeRanges, + ); if (rawType !== null) return unwrapGeneric(rawType); } } if (func.type === 'identifier') { + const producer = producerAt(func, scopeRanges, indexes, producerReturns); + if (producer === null) return null; + if (producer !== undefined) { + const elementType = unwrapGeneric(producer.rawType); + return { + ...producer.typeRef, + rawName: normalizeFieldType(elementType), + declaredSpelling: elementType, + }; + } const crossFileReturn = allReturnTypes.get(func.text); if (crossFileReturn !== undefined) return unwrapGeneric(crossFileReturn); const rawReturn = lookupRawFunctionReturnType(func.text, valueNode); @@ -733,20 +967,25 @@ function lookupRawParameterType(paramName: string, contextNode: SyntaxNode): str function lookupTypeInScopes( name: string, contextNode: SyntaxNode, - parsed: ParsedFile, + _parsed: ParsedFile, scopeMap: ReadonlyMap, moduleScope: Scope, + scopeRanges: ReadonlyMap, ): string | null { - const fnScope = findEnclosingFunctionScope(contextNode, scopeMap); - if (fnScope !== null) { - const tb = fnScope.typeBindings.get(name); - if (tb !== undefined) return tb.rawName; - } - - const mtb = moduleScope.typeBindings.get(name); - if (mtb !== undefined) return mtb.rawName; - - return null; + const scope = lexicalScopeForNode(contextNode, scopeRanges) ?? moduleScope; + const claim = lookupLexicalName( + scope.id, + name, + { scopes: { getScope: (id) => scopeMap.get(id) } }, + { + position: { + startLine: contextNode.startPosition.row + 1, + startCol: contextNode.startPosition.column, + }, + purpose: 'value', + }, + ); + return claim.typeBinding?.rawName ?? null; } function lookupReturnTypeInScopes( @@ -851,34 +1090,29 @@ function extractVarName(node: SyntaxNode): string | null { return null; } -function injectTypeBinding(scope: Scope, name: string, typeName: string): void { - if (scope.typeBindings.has(name)) return; - (scope.typeBindings as Map).set(name, { - rawName: typeName, - declaredAtScope: scope.id, - source: 'annotation', - }); -} - -function findEnclosingFunctionScope( - node: SyntaxNode, - scopeMap: ReadonlyMap, -): Scope | null { - let current: SyntaxNode | null = node as SyntaxNode; - while (current !== null) { - if (current.type === 'function_item') { - for (const scope of scopeMap.values()) { - if ( - scope.kind === 'Function' && - scope.range.startLine === current.startPosition.row + 1 && - scope.range.startCol === current.startPosition.column - ) { - return scope; +function injectTypeBinding( + scope: Scope, + name: string, + typeName: string | TypeRef, + replaceInferred = false, +): void { + const existing = scope.typeBindings.get(name); + if ( + existing !== undefined && + !( + replaceInferred && + (existing.source === 'constructor-inferred' || existing.source === 'assignment-inferred') + ) + ) + return; + (scope.typeBindings as Map).set( + name, + typeof typeName === 'string' + ? { + rawName: typeName, + declaredAtScope: scope.id, + source: 'annotation', } - } - break; - } - current = current.parent; - } - return null; + : typeName, + ); } diff --git a/gitnexus/src/core/ingestion/languages/rust/scope-resolver.ts b/gitnexus/src/core/ingestion/languages/rust/scope-resolver.ts index 4beda7c03..f1cab711e 100644 --- a/gitnexus/src/core/ingestion/languages/rust/scope-resolver.ts +++ b/gitnexus/src/core/ingestion/languages/rust/scope-resolver.ts @@ -160,8 +160,8 @@ export const rustScopeResolver: ScopeResolver = { importEdgeReason: 'rust-scope: use', loadResolutionConfig: loadRustCargoTargets, - resolveImportTarget: (targetRaw, fromFile, allFilePaths, resolutionConfig) => - resolveRustImportTarget(targetRaw, fromFile, allFilePaths, resolutionConfig), + resolveImportTarget: (targetRaw, fromFile, allFilePaths, resolutionConfig, context) => + resolveRustImportTarget(targetRaw, fromFile, allFilePaths, resolutionConfig, context), mergeBindings: (existing, incoming, scopeId) => rustMergeBindings(existing, incoming, scopeId), diff --git a/gitnexus/src/core/ingestion/languages/typescript/captures.ts b/gitnexus/src/core/ingestion/languages/typescript/captures.ts index af388aed3..ef57dc985 100644 --- a/gitnexus/src/core/ingestion/languages/typescript/captures.ts +++ b/gitnexus/src/core/ingestion/languages/typescript/captures.ts @@ -49,6 +49,7 @@ import { getTreeSitterBufferSize } from '../../constants.js'; import { parseSourceSafe } from '../../../tree-sitter/safe-parse.js'; import { synthesizeCallableFlowCaptures } from '../../utils/callable-flow-captures.js'; import { synthesizeCjsModuleExports } from './cjs-module-exports.js'; +import { synthesizeTsLocalImports } from './local-loaders.js'; import { synthesizeReceiverChainCapture } from '../../utils/receiver-chain-captures.js'; import { deriveDefaultExportHocName, @@ -362,7 +363,9 @@ export function emitTsScopeCaptures( sourceText: string, filePath: string, cachedTree?: unknown, + sourceMeta?: { scriptLanguage?: string }, ): readonly CaptureMatch[] { + const grammarPath = sourceMeta?.scriptLanguage === 'tsx' ? `${filePath}.tsx` : filePath; // Reuse a pre-parsed Tree when the caller passes one via `cachedTree`; a // miss re-parses. (The cache is currently always empty — its only producer, // the sequential parser, was removed — so this re-parses in practice.) The @@ -377,11 +380,11 @@ export function emitTsScopeCaptures( // fresh parse if they disagree (e.g. a worker-mode parse landed // with the wrong grammar pinned). let tree = cachedTree as ReturnType['parse']> | undefined; - if (tree !== undefined && !tsCachedTreeMatchesGrammar(tree, filePath)) { + if (tree !== undefined && !tsCachedTreeMatchesGrammar(tree, grammarPath)) { tree = undefined; } if (tree === undefined) { - tree = parseSourceSafe(getTsParser(filePath), sourceText, undefined, { + tree = parseSourceSafe(getTsParser(grammarPath), sourceText, undefined, { bufferSize: getTreeSitterBufferSize(sourceText), }); recordCacheMiss(); @@ -389,7 +392,7 @@ export function emitTsScopeCaptures( recordCacheHit(); } - const rawMatches = getTsScopeQuery(filePath).matches(tree.rootNode); + const rawMatches = getTsScopeQuery(grammarPath).matches(tree.rootNode); // Export evidence, read once per file (see `ts-js-export-marker.ts`). const exportEvidence = collectEsmExportEvidence(tree.rootNode, filePath); const out: CaptureMatch[] = []; @@ -733,7 +736,8 @@ export function emitTsScopeCaptures( // emitter: a `.ts` file in a CommonJS package uses the same forms, and // without this the default-export NODE was emitted with nothing declaring it // — the "found, zero callers" state this work exists to remove (#2729 F7). - synthesizeCjsModuleExports(tree.rootNode, filePath, out); + const validRequireCalls = synthesizeTsLocalImports(tree.rootNode, filePath, out); + synthesizeCjsModuleExports(tree.rootNode, filePath, out, validRequireCalls); return out; } @@ -791,7 +795,7 @@ function synthesizeTsInheritanceReferences(root: SyntaxNode, out: CaptureMatch[] for (const child of node.namedChildren) { if (child === null || child.type !== 'extends_type_clause') continue; for (const base of child.namedChildren) { - emitTsInheritanceBase(base, out); + emitTsInheritanceBase(base, out, 'type'); } } continue; @@ -819,11 +823,11 @@ function synthesizeTsInheritanceReferences(root: SyntaxNode, out: CaptureMatch[] // `extends Foo` / `extends Foo` — the base is the `value:` field // (an identifier; generics live in a sibling `type_arguments`). const value = clause.childForFieldName('value') ?? clause.firstNamedChild; - emitTsInheritanceBase(value, out); + emitTsInheritanceBase(value, out, 'value'); } else if (clause.type === 'implements_clause') { // `implements IFoo, IBar` — each base type is a direct named child. for (const base of clause.namedChildren) { - emitTsInheritanceBase(base, out); + emitTsInheritanceBase(base, out, 'type'); } } } @@ -833,13 +837,18 @@ function synthesizeTsInheritanceReferences(root: SyntaxNode, out: CaptureMatch[] /** Emit one `@reference.inherits` match for a TS heritage base, normalizing * the lookup name to its bare simple identifier. No-ops on null / non-type * nodes or when the bare name can't be derived. */ -function emitTsInheritanceBase(base: SyntaxNode | null, out: CaptureMatch[]): void { +function emitTsInheritanceBase( + base: SyntaxNode | null, + out: CaptureMatch[], + purpose: 'value' | 'type', +): void { if (base === null) return; const nameNode = terminalTsTypeNameNode(base); if (nameNode === null) return; out.push({ '@reference.inherits': nodeToCapture('@reference.inherits', base), '@reference.name': nodeToCapture('@reference.name', nameNode), + '@reference.lookup-purpose': syntheticCapture('@reference.lookup-purpose', base, purpose), }); } diff --git a/gitnexus/src/core/ingestion/languages/typescript/cjs-module-exports.ts b/gitnexus/src/core/ingestion/languages/typescript/cjs-module-exports.ts index 012815a09..7f40a4f14 100644 --- a/gitnexus/src/core/ingestion/languages/typescript/cjs-module-exports.ts +++ b/gitnexus/src/core/ingestion/languages/typescript/cjs-module-exports.ts @@ -39,7 +39,11 @@ import { defaultExportNameCollides } from './cjs-export-assignment.js'; * resolve through it, and synthesizing a second binding would re-create the * ambiguity the shadow guard exists to prevent. */ -export function synthesizeCjsReExports(root: SyntaxNode, out: CaptureMatch[]): void { +export function synthesizeCjsReExports( + root: SyntaxNode, + out: CaptureMatch[], + validRequireCalls?: ReadonlyMap>, +): void { // Namespace and named bindings introduced by require(), by local name. const namespaceSources = new Map(); const namedSources = new Map(); @@ -51,6 +55,8 @@ export function synthesizeCjsReExports(root: SyntaxNode, out: CaptureMatch[]): v const value = declarator.childForFieldName('value'); if (value === null || value.type !== 'call_expression') continue; if (value.childForFieldName('function')?.text !== 'require') continue; + if (validRequireCalls !== undefined && !validRequireCalls.has(value.id)) continue; + const blockedNames = validRequireCalls?.get(value.id); const arg = value.childForFieldName('arguments')?.namedChild(0); if (arg === undefined || arg === null || arg.type !== 'string') continue; const source = arg.namedChild(0)?.text ?? arg.text.slice(1, -1); @@ -58,15 +64,22 @@ export function synthesizeCjsReExports(root: SyntaxNode, out: CaptureMatch[]): v const nameNode = declarator.childForFieldName('name'); if (nameNode === null) continue; if (nameNode.type === 'identifier') { + if (blockedNames?.has(nameNode.text)) continue; namespaceSources.set(nameNode.text, { source, node: arg }); } else if (nameNode.type === 'object_pattern') { for (const field of nameNode.namedChildren) { if (field.type === 'shorthand_property_identifier_pattern') { + if (blockedNames?.has(field.text)) continue; namedSources.set(field.text, { source, name: field.text, node: arg }); } else if (field.type === 'pair_pattern') { const key = field.childForFieldName('key'); const local = field.childForFieldName('value'); - if (key !== null && local !== null && local.type === 'identifier') + if ( + key !== null && + local !== null && + local.type === 'identifier' && + !blockedNames?.has(local.text) + ) namedSources.set(local.text, { source, name: key.text, node: arg }); } } @@ -190,7 +203,8 @@ export function synthesizeCjsModuleExports( root: SyntaxNode, filePath: string, out: CaptureMatch[], + validRequireCalls?: ReadonlyMap>, ): void { - synthesizeCjsReExports(root, out); + synthesizeCjsReExports(root, out, validRequireCalls); synthesizeCjsDefaultExport(root, filePath, out); } diff --git a/gitnexus/src/core/ingestion/languages/typescript/index.ts b/gitnexus/src/core/ingestion/languages/typescript/index.ts index 120566f48..0915f6db7 100644 --- a/gitnexus/src/core/ingestion/languages/typescript/index.ts +++ b/gitnexus/src/core/ingestion/languages/typescript/index.ts @@ -39,10 +39,9 @@ * The TypeScript registry-primary path intentionally does NOT resolve * the following. Each is a conscious trade-off at migration time. * - * 1. **Type-only import / export separation** — `import type { X }` - * and `import { X }` produce the same `ParsedImport` shape today; - * `def.type` on the resolved symbol is the only discriminator. - * Parity with the legacy path is preserved. Tracking in #927. + * 1. **Indirect local loaders** — literal require and immediately awaited + * import bindings are supported. Computed sources, promise chains and + * default/rest destructuring do not establish imported aliases. * 2. **Declaration merging for imports** — when `import { Foo }` * brings in a symbol that is BOTH a class and a namespace in the * source module, we currently surface a single binding per the diff --git a/gitnexus/src/core/ingestion/languages/typescript/interpret.ts b/gitnexus/src/core/ingestion/languages/typescript/interpret.ts index 30236f95f..ad26309f7 100644 --- a/gitnexus/src/core/ingestion/languages/typescript/interpret.ts +++ b/gitnexus/src/core/ingestion/languages/typescript/interpret.ts @@ -15,7 +15,13 @@ * `@type-binding.type`. */ -import type { CaptureMatch, ParsedImport, ParsedTypeBinding, TypeRef } from 'gitnexus-shared'; +import type { + CaptureMatch, + ParsedImport, + ParsedTypeBinding, + ScopeId, + TypeRef, +} from 'gitnexus-shared'; /** Shared empty result for the non-type-only path — see `typeOnly` below. */ const NO_TYPE_ONLY: { typeOnly?: true } = Object.freeze({}); @@ -33,6 +39,8 @@ export function interpretTsImport(captures: CaptureMatch): ParsedImport | null { const nameCap = captures['@import.name']; const aliasCap = captures['@import.alias']; const sourceCap = captures['@import.source']; + const owner = captures['@import.lookup-scope']; + const lexicalOwner = owner ? { declaredAtScope: owner.text as ScopeId } : {}; const kind = kindCap?.text; if (kind === undefined) return null; @@ -55,6 +63,7 @@ export function interpretTsImport(captures: CaptureMatch): ParsedImport | null { if (sourceCap === undefined || aliasCap === undefined) return null; return { kind: 'alias', + ...lexicalOwner, localName: aliasCap.text, importedName: 'default', alias: aliasCap.text, @@ -67,6 +76,7 @@ export function interpretTsImport(captures: CaptureMatch): ParsedImport | null { if (sourceCap === undefined || nameCap === undefined) return null; return { kind: 'named', + ...lexicalOwner, localName: nameCap.text, importedName: nameCap.text, targetRaw: sourceCap.text, @@ -80,6 +90,7 @@ export function interpretTsImport(captures: CaptureMatch): ParsedImport | null { } return { kind: 'alias', + ...lexicalOwner, localName: aliasCap.text, importedName: nameCap.text, alias: aliasCap.text, @@ -92,6 +103,7 @@ export function interpretTsImport(captures: CaptureMatch): ParsedImport | null { if (sourceCap === undefined || aliasCap === undefined) return null; return { kind: 'namespace', + ...lexicalOwner, localName: aliasCap.text, importedName: sourceCap.text, targetRaw: sourceCap.text, diff --git a/gitnexus/src/core/ingestion/languages/typescript/local-loaders.ts b/gitnexus/src/core/ingestion/languages/typescript/local-loaders.ts new file mode 100644 index 000000000..37b53093b --- /dev/null +++ b/gitnexus/src/core/ingestion/languages/typescript/local-loaders.ts @@ -0,0 +1,602 @@ +/** Lexical names and statically named loaders shared by TS, JS, and Vue scripts. */ +import { + makeScopeId, + type Capture, + type CaptureMatch, + type NameClaim, + type Range, + type ScopeKind, +} from 'gitnexus-shared'; +import { nodeToCapture, syntheticCapture, type SyntaxNode } from '../../utils/ast-helpers.js'; + +const FUNCTION_TYPES = new Set([ + 'function_declaration', + 'generator_function_declaration', + 'function_signature', + 'function_expression', + 'generator_function', + 'arrow_function', + 'method_definition', + 'method_signature', + 'abstract_method_signature', +]); +const SCOPE_TAGS: Readonly> = { + '@scope.module': 'Module', + '@scope.namespace': 'Namespace', + '@scope.class': 'Class', + '@scope.function': 'Function', + '@scope.block': 'Block', + '@scope.object': 'Object', +}; +interface LexicalScope { + readonly anchor: Capture; + readonly kind: ScopeKind; + readonly matchIndex: number; + node?: SyntaxNode; + parent?: LexicalScope; + readonly claims: NameClaim[]; + readonly names: Set; +} +interface LocatedNode { + readonly node: SyntaxNode; + readonly scope: LexicalScope; +} +const rangeKey = (range: Range) => + `${range.startLine}:${range.startCol}-${range.endLine}:${range.endCol}`; +const scopeKey = (range: Range, kind: ScopeKind) => `${kind}:${rangeKey(range)}`; +const rangeOf = (node: SyntaxNode): Range => { + const start = node.startPosition; + const end = node.endPosition; + return { + startLine: start.row + 1, + startCol: start.column, + endLine: end.row + 1, + endCol: end.column, + }; +}; +const atEnd = (node: SyntaxNode) => ({ + startLine: node.endPosition.row + 1, + startCol: node.endPosition.column, +}); + +/** Binding patterns, not arbitrary descendants: object keys and default RHSs do not bind. */ +function boundNames(node: SyntaxNode | null): SyntaxNode[] { + if (node === null) return []; + switch (node.type) { + case 'identifier': + case 'type_identifier': + case 'shorthand_property_identifier_pattern': + return [node]; + case 'required_parameter': + case 'optional_parameter': + return boundNames(node.childForFieldName('pattern')); + case 'pair_pattern': + return boundNames(node.childForFieldName('value')); + case 'assignment_pattern': + case 'object_assignment_pattern': + return boundNames(node.childForFieldName('left')); + case 'rest_pattern': + return boundNames(node.firstNamedChild); + case 'object_pattern': + case 'array_pattern': + case 'formal_parameters': + return node.namedChildren.flatMap(boundNames); + default: + return []; + } +} + +function functionOwner(scope: LexicalScope): LexicalScope { + let current = scope; + while ( + current.parent && + current.kind !== 'Function' && + current.kind !== 'Module' && + current.kind !== 'Namespace' + ) + current = current.parent; + return current; +} +function bindingOwner(scope: LexicalScope, name: string): LexicalScope | undefined { + for (let current: LexicalScope | undefined = scope; current; current = current.parent) { + if (current.names.has(name)) return current; + } + return undefined; +} +function extraBlock(node: SyntaxNode): boolean { + return ( + node.type === 'catch_clause' || + node.type === 'switch_body' || + (node.type === 'for_in_statement' && + ['let', 'const'].includes(node.childForFieldName('kind')?.text ?? '')) || + (node.type === 'for_statement' && + node.childForFieldName('initializer')?.type === 'lexical_declaration') + ); +} +function nodeScopeKind(node: SyntaxNode): ScopeKind | undefined { + if (FUNCTION_TYPES.has(node.type)) return 'Function'; + switch (node.type) { + case 'program': + return 'Module'; + case 'internal_module': + return 'Namespace'; + case 'class': + case 'class_declaration': + case 'abstract_class_declaration': + case 'interface_declaration': + case 'enum_declaration': + case 'type_alias_declaration': + return 'Class'; + case 'object': + return 'Object'; + case 'statement_block': + return 'Block'; + default: + return extraBlock(node) ? 'Block' : undefined; + } +} +/** A hoisted var initialized conditionally cannot prove a namespace value. */ +function conditionalVar(node: SyntaxNode, owner: LexicalScope): boolean { + if (node.parent?.type !== 'variable_declaration') return false; + for ( + let current = node.parent.parent; + current && current.id !== owner.node?.id; + current = current.parent + ) { + if ( + [ + 'if_statement', + 'switch_case', + 'switch_default', + 'for_statement', + 'for_in_statement', + 'while_statement', + 'do_statement', + 'catch_clause', + 'try_statement', + ].includes(current.type) + ) + return true; + } + return false; +} + +/** + * Add scope facts and imports in one AST pass. Returned identities and blocked + * binding names let CommonJS forwarding reject shadowed loaders and written handles. + * Computed sources and indirect promise/loader chains never become aliases. + */ +export function synthesizeTsLocalImports( + root: SyntaxNode, + filePath: string, + out: CaptureMatch[], +): ReadonlyMap> { + const scopes = new Map(); + const scopesAt = new Map(); + const calls: LocatedNode[] = []; + const writes: LocatedNode[] = []; + const bindingWrites: Array = []; + const uninitializedVarClaims = new Set(); + const declarations: LocatedNode[] = []; + const equalsImports: LocatedNode[] = []; + for (let i = 0; i < out.length; i++) { + for (const [tag, kind] of Object.entries(SCOPE_TAGS)) { + const anchor = out[i][tag]; + if (anchor) + scopes.set(scopeKey(anchor.range, kind), { + anchor, + kind, + matchIndex: i, + claims: [], + names: new Set(), + }); + } + } + const module = scopes.get(scopeKey(rangeOf(root), 'Module')); + if (!module) return new Map(); + + const add = ( + scope: LexicalScope, + node: SyntaxNode, + names: readonly SyntaxNode[], + extra: Partial = {}, + ) => { + for (const name of names) { + if (extra.purpose !== 'type') scope.names.add(name.text); + scope.claims.push({ + name: name.text, + range: rangeOf(node), + kind: 'binding', + purpose: 'value', + ...extra, + }); + } + }; + const pending: Array<{ node: SyntaxNode; parent: LexicalScope }> = [ + { node: root, parent: module }, + ]; + while (pending.length) { + const { node, parent } = pending.pop()!; + const range = rangeOf(node); + const key = rangeKey(range); + const kind = nodeScopeKind(node); + let scope = kind ? scopes.get(scopeKey(range, kind)) : undefined; + if (!scope && extraBlock(node)) { + const anchor = nodeToCapture('@scope.block', node); + scope = { anchor, kind: 'Block', matchIndex: out.length, claims: [], names: new Set() }; + scopes.set(scopeKey(range, 'Block'), scope); + out.push({ '@scope.block': anchor }); + } + if (scope && scope !== parent) scope.parent = parent; + if (scope) scope.node = node; + const current = scope ?? parent; + if (node.type === 'import_statement') scopesAt.set(key, current); + + if (FUNCTION_TYPES.has(node.type)) { + // JSDoc parameter type facts are anchored on the enclosing function. + add( + current, + node, + boundNames(node.childForFieldName('parameters') ?? node.childForFieldName('parameter')), + { hoisted: true }, + ); + const name = node.childForFieldName('name'); + if ( + name && + ['function_declaration', 'generator_function_declaration', 'function_signature'].includes( + node.type, + ) + ) { + add(parent, node, [name], { hoisted: true }); + } else if (name && ['function_expression', 'generator_function'].includes(node.type)) + add(current, node, [name], { hoisted: true }); + } + if (node.type === 'variable_declarator') { + const owner = node.parent?.type === 'variable_declaration' ? functionOwner(current) : current; + // Declaration queries anchor values on the whole declaration (and JS + // also captures its export wrapper). The claim must contain those exact + // declaration ranges while activation still follows this initializer. + const declaration = node.parent ?? node; + const anchor = + declaration.parent?.type === 'export_statement' ? declaration.parent : declaration; + const uninitializedVar = + node.parent?.type === 'variable_declaration' && node.childForFieldName('value') === null; + const claimStart = owner.claims.length; + add( + owner, + anchor, + boundNames(node.childForFieldName('name')), + uninitializedVar ? { hoisted: true } : { availableFrom: atEnd(node) }, + ); + if (uninitializedVar) { + for (const claim of owner.claims.slice(claimStart)) uninitializedVarClaims.add(claim); + } + declarations.push({ node, scope: owner }); + } else if (node.type === 'catch_clause') { + add( + current, + node.childForFieldName('parameter') ?? node, + boundNames(node.childForFieldName('parameter')), + ); + } else if (node.type === 'for_in_statement') { + const kind = node.childForFieldName('kind')?.text; + if (kind) + add( + kind === 'var' ? functionOwner(current) : current, + node.childForFieldName('left') ?? node, + boundNames(node.childForFieldName('left')), + ); + } else if (node.type === 'class') { + const name = node.childForFieldName('name'); + if (name) add(current, node, [name], { purpose: 'both', hoisted: true }); + } else if (node.type === 'class_declaration' || node.type === 'abstract_class_declaration') { + const name = node.childForFieldName('name'); + if (name) { + add(parent, node, [name], { availableFrom: atEnd(node) }); + add(parent, node, [name], { purpose: 'type', hoisted: true }); + } + } else if (node.type === 'enum_declaration' || node.type === 'internal_module') { + const name = node.childForFieldName('name'); + if (name) + add(parent, node, [name], { + purpose: 'both', + availableFrom: { + startLine: node.startPosition.row + 1, + startCol: node.startPosition.column, + }, + }); + } else if (node.type === 'interface_declaration' || node.type === 'type_alias_declaration') { + const name = node.childForFieldName('name'); + if (name) add(scope ? parent : current, node, [name], { purpose: 'type', hoisted: true }); + } + if (node.type === 'call_expression') calls.push({ node, scope: current }); + if ( + node.type === 'assignment_expression' || + node.type === 'augmented_assignment_expression' || + node.type === 'update_expression' + ) + writes.push({ node, scope: current }); + if ( + node.type === 'import_statement' && + node.namedChildren.some((child) => child.type === 'import_require_clause') + ) + equalsImports.push({ node, scope: current }); + for (let i = node.namedChildCount - 1; i >= 0; i--) { + const child = node.namedChild(i); + if (child) pending.push({ node: child, parent: current }); + } + } + + // Static imports establish loader identity before any require call is inspected. + for (const match of out) { + const anchor = match['@import.statement']; + const kind = match['@import.kind']?.text; + if ( + !anchor || + !kind || + kind.startsWith('reexport') || + kind === 'dynamic' || + kind === 'side-effect' + ) + continue; + const name = match['@import.alias']?.text ?? match['@import.name']?.text; + const scope = scopesAt.get(rangeKey(anchor.range)); + if (!name || !scope) continue; + scope.names.add(name); + const claim: NameClaim = { + name, + range: anchor.range, + kind: 'import', + purpose: 'both', + hoisted: true, + }; + if (match['@import.type-only']) { + scope.claims.push( + { ...claim, purpose: 'type' }, + { ...claim, kind: 'blocked', purpose: 'value' }, + ); + } else scope.claims.push(claim); + } + for (const { node, scope } of equalsImports) { + const clause = node.namedChildren.find((child) => child.type === 'import_require_clause')!; + const name = clause.namedChildren.find((child) => child.type === 'identifier'); + if (name) scope.names.add(name.text); + } + + // An assignment to an actual local shadow does not mutate the ambient loader. + // A write to the ambient loader makes its identity unknown for this file. + let ambientLoaderWritten = false; + for (const { node, scope } of writes) { + const names = boundNames(node.childForFieldName('left') ?? node.childForFieldName('argument')); + for (const name of names) { + const owner = bindingOwner(scope, name.text); + if (name.text === 'require' && !owner) ambientLoaderWritten = true; + if (owner) bindingWrites.push({ node, scope: owner, name: name.text }); + } + } + + const validRequireCalls = new Map>(); + const declarationByValue = new Map( + declarations.map((entry) => [entry.node.childForFieldName('value')?.id, entry]), + ); + const emit = ( + at: SyntaxNode, + sourceNode: SyntaxNode, + source: string, + kind: string, + scope: LexicalScope, + name?: SyntaxNode, + imported?: string, + available?: ReturnType, + typeOnly = false, + ) => { + const match: Record = { + '@import.statement': nodeToCapture('@import.statement', at), + '@import.kind': syntheticCapture('@import.kind', at, kind), + '@import.source': syntheticCapture('@import.source', sourceNode, source), + '@import.lookup-scope': syntheticCapture( + '@import.lookup-scope', + at, + makeScopeId({ filePath, range: scope.anchor.range, kind: scope.kind }), + ), + }; + if (typeOnly) match['@import.type-only'] = syntheticCapture('@import.type-only', at, 'true'); + if (name) { + match['@import.alias'] = nodeToCapture('@import.alias', name); + if (imported) match['@import.name'] = syntheticCapture('@import.name', name, imported); + scope.names.add(name.text); + // A loader initializer supplies the variable's value; its ordinary + // declaration must not compete with the imported target at this site. + for (let i = scope.claims.length - 1; i >= 0; i--) { + const claim = scope.claims[i]; + if ( + claim.name === name.text && + claim.kind === 'binding' && + claim.availableFrom?.startLine === available?.startLine && + claim.availableFrom?.startCol === available?.startCol + ) + scope.claims.splice(i, 1); + } + const claim: NameClaim = { + name: name.text, + range: match['@import.statement'].range, + kind: 'import', + purpose: 'both', + ...(available ? { availableFrom: available } : { hoisted: true }), + }; + if (typeOnly) + scope.claims.push( + { ...claim, purpose: 'type' }, + { ...claim, kind: 'blocked', purpose: 'value' }, + ); + else scope.claims.push(claim); + } + out.push(match); + }; + for (const { node, scope } of calls) { + const fn = node.childForFieldName('function'); + const requireCall = fn?.type === 'identifier' && fn.text === 'require'; + const importCall = fn?.type === 'import'; + if (!requireCall && !importCall) continue; + if (requireCall && (ambientLoaderWritten || bindingOwner(scope, 'require'))) continue; + const args = + node + .childForFieldName('arguments') + ?.namedChildren.filter((child) => child.type !== 'comment') ?? []; + const sourceNode = args[0]; + if (!sourceNode || sourceNode.type !== 'string' || (requireCall && args.length !== 1)) continue; + const source = sourceNode.text.slice(1, -1); + if (requireCall) validRequireCalls.set(node.id, new Set()); + const value = importCall && node.parent?.type === 'await_expression' ? node.parent : node; + const declaration = declarationByValue.get(value.id); + if (importCall && value === node) continue; // A promise is not a module namespace. + if (!declaration) { + if (requireCall) emit(node, sourceNode, source, 'side-effect', scope); + continue; + } + const name = declaration.node.childForFieldName('name'); + const available = atEnd(declaration.node); + if (name?.type === 'identifier') + emit(node, sourceNode, source, 'namespace', declaration.scope, name, undefined, available); + else if (name?.type === 'object_pattern') { + for (const field of name.namedChildren) { + if (field.type === 'shorthand_property_identifier_pattern') { + emit(node, sourceNode, source, 'named', declaration.scope, field, field.text, available); + } else if (field.type === 'pair_pattern') { + const key = field.childForFieldName('key'); + const local = field.childForFieldName('value'); + if ( + key && + ['property_identifier', 'identifier', 'string'].includes(key.type) && + local?.type === 'identifier' + ) { + emit( + node, + sourceNode, + source, + 'named-alias', + declaration.scope, + local, + key.type === 'string' ? key.text.slice(1, -1) : key.text, + available, + ); + } + } + } + } + } + for (const { node, scope } of equalsImports) { + const clause = node.namedChildren.find((child) => child.type === 'import_require_clause')!; + const name = clause.namedChildren.find((child) => child.type === 'identifier'); + const source = clause.childForFieldName('source'); + const typeOnly = node.children.some((child) => child.type === 'type'); + if ( + scope.kind === 'Module' && + name && + source?.type === 'string' && + (typeOnly || (!ambientLoaderWritten && !bindingOwner(scope, 'require'))) + ) { + emit( + node, + source, + source.text.slice(1, -1), + 'namespace', + scope, + name, + undefined, + undefined, + typeOnly, + ); + } + } + // Writes invalidate an imported handle's module proof, including a var loader + // that shares a name with a function declaration. Ordinary locals retain their + // type and callable-value inference; an assignment does not erase that evidence. + for (const { node, scope, name } of bindingWrites) { + if ( + !scope.claims.some( + (claim) => claim.name === name && claim.kind === 'import' && claim.purpose !== 'type', + ) + ) { + continue; + } + scope.claims.push({ + name, + range: rangeOf(node), + kind: 'blocked', + purpose: 'value', + availableFrom: atEnd(node), + }); + } + for (const { node, scope } of declarations) { + if (!conditionalVar(node, scope)) continue; + const end = atEnd(node); + const names = new Set(boundNames(node.childForFieldName('name')).map((name) => name.text)); + for (let i = 0; i < scope.claims.length; i++) { + const claim = scope.claims[i]; + if ( + claim.kind === 'import' && + names.has(claim.name) && + claim.availableFrom?.startLine === end.startLine && + claim.availableFrom.startCol === end.startCol + ) { + scope.claims[i] = { ...claim, kind: 'blocked' }; + } + } + } + // Forwarding exports require a stable handle as well as a genuine loader. + // Keep the original import facts for calls before a write; remove forwarding + // proof only for each written handle, preserving unchanged destructured siblings. + for (const { node, scope } of declarations) { + const value = node.childForFieldName('value'); + const blockedNames = value && validRequireCalls.get(value.id); + if (!blockedNames) continue; + const names = new Set(boundNames(node.childForFieldName('name')).map((name) => name.text)); + for (const claim of scope.claims) { + if (claim.kind === 'blocked' && names.has(claim.name)) blockedNames.add(claim.name); + } + } + for (const scope of scopes.values()) { + // `var x;` creates ownership but performs no store. A same-scope parameter, + // function, or initialized declaration already owns that binding's value. + const valueNames = new Set( + scope.claims + .filter( + (claim) => + !uninitializedVarClaims.has(claim) && + claim.kind !== 'blocked' && + claim.purpose !== 'type', + ) + .map((claim) => claim.name), + ); + const claims = scope.claims.filter( + (claim) => !uninitializedVarClaims.has(claim) || !valueNames.has(claim.name), + ); + const selfName = + scope.node?.type === 'class' ? scope.node.childForFieldName('name')?.text : undefined; + out[scope.matchIndex] = { + ...out[scope.matchIndex], + '@scope.name-claims': { + ...scope.anchor, + name: '@scope.name-claims', + text: JSON.stringify(claims), + }, + '@scope.lookup-policy': { + ...scope.anchor, + name: '@scope.lookup-policy', + text: JSON.stringify({ + // An assignment can update a binding, but cannot declare a local. + nameClaimsComplete: true, + ...(scope.kind === 'Function' ? { deferParentActivation: true } : {}), + ...(scope.kind === 'Class' + ? { + skipFromChildren: true, + ...(selfName ? { visibleNamesFromChildren: [selfName] } : {}), + } + : {}), + }), + }, + }; + } + return validRequireCalls; +} diff --git a/gitnexus/src/core/ingestion/languages/typescript/query.ts b/gitnexus/src/core/ingestion/languages/typescript/query.ts index 15f344a37..66437cdbb 100644 --- a/gitnexus/src/core/ingestion/languages/typescript/query.ts +++ b/gitnexus/src/core/ingestion/languages/typescript/query.ts @@ -220,6 +220,12 @@ export const TYPESCRIPT_SCOPE_QUERY = ` name: (identifier) @declaration.name value: (function_expression) @declaration.function)) +;; Generator expressions have the same callable binding and scope anchor. +(lexical_declaration + (variable_declarator + name: (identifier) @declaration.name + value: (generator_function) @declaration.function)) + (variable_declaration (variable_declarator name: (identifier) @declaration.name @@ -230,6 +236,11 @@ export const TYPESCRIPT_SCOPE_QUERY = ` name: (identifier) @declaration.name value: (function_expression) @declaration.function)) +(variable_declaration + (variable_declarator + name: (identifier) @declaration.name + value: (generator_function) @declaration.function)) + ;; CJS property-assignment exports (#2723) — see the matching block in ;; \`languages/javascript/query.ts\` for the rationale. Mirrored here because ;; \`.ts\` files in a CommonJS package use the same form, and because the JS @@ -1096,12 +1107,11 @@ export const TYPESCRIPT_SCOPE_QUERY = ` function: (member_expression) @type-binding.type)) @type-binding.alias ;; Type bindings — for-of member-access iterable: \`for (const u of this.users)\`. -;; Bind u to \`users\` (the attribute name); chain-follow resolves users -;; via the enclosing class's field binding. +;; Preserve the explicit receiver so class fields are resolved as members, +;; without treating them as lexical names visible in method bodies. (for_in_statement left: (identifier) @type-binding.name - right: (member_expression - property: (property_identifier) @type-binding.type)) @type-binding.alias + right: (member_expression) @type-binding.type) @type-binding.alias ;; Type bindings — class field annotation: \`private city: City\`. (public_field_definition @@ -1639,17 +1649,17 @@ export function getTsScopeQuery(filePath?: string): Parser.Query { /** * Validate that a cached `Tree` was produced by the grammar matching - * `filePath` (TSX vs TypeScript). The runtime tree-sitter `Tree` exposes - * `getLanguage()` (returning the grammar object the parser was bound - * to); the .d.ts is incomplete, so we reach via a cast. Identity + * `filePath` (TSX vs TypeScript). Native tree-sitter exposes `language`; + * other Tree implementations expose `getLanguage()`. The .d.ts is + * incomplete, so we reach via a cast. Identity * comparison against `TSX_GRAMMAR` / `TS_GRAMMAR` is exact: the same - * module instance produces both. If `getLanguage` is unavailable for + * module instance produces both. If neither grammar accessor is available for * any reason, return true to keep behavior backwards-compatible (the * original code never validated grammar at all). */ export function tsCachedTreeMatchesGrammar(tree: unknown, filePath: string): boolean { // eslint-disable-next-line @typescript-eslint/no-explicit-any - const lang = (tree as any)?.getLanguage?.(); + const lang = (tree as any)?.getLanguage?.() ?? (tree as any)?.language; if (lang === undefined || lang === null) return true; return isTsxFile(filePath) ? lang === TSX_GRAMMAR : lang === TS_GRAMMAR; } diff --git a/gitnexus/src/core/ingestion/languages/typescript/scope-resolver.ts b/gitnexus/src/core/ingestion/languages/typescript/scope-resolver.ts index b0d12a3e8..40de35d32 100644 --- a/gitnexus/src/core/ingestion/languages/typescript/scope-resolver.ts +++ b/gitnexus/src/core/ingestion/languages/typescript/scope-resolver.ts @@ -87,6 +87,8 @@ const typescriptScopeResolver: ScopeResolver = { language: SupportedLanguages.TypeScript, languageProvider: typescriptProvider, importEdgeReason: 'typescript-scope: import', + importsBindAtLexicalScope: true, + ownedMembersBindAtModuleScope: false, resolveImportTarget: makeTsResolveImportTarget(), @@ -297,6 +299,8 @@ function hasLocalBindingInScopeChain( visited.add(cursor); const scope = indexes.scopeTree.getScope(cursor); if (!scope) return false; + if (scope.nameClaims?.some((claim) => claim.name === name && claim.purpose !== 'type')) + return true; const localBindings = scope.bindings.get(name); if ( @@ -317,15 +321,6 @@ function hasLocalBindingInScopeChain( // belongs to callable space, so a same-file presence check here cannot // over-suppress a legitimate auto-import. // - // Residual (known limitation): this catches parameters whose annotation the - // TS scope query records as a type-binding (`p: Named`, generics, unions, - // predefined, arrays). Function-typed params (`p: () => void`), untyped - // params, destructured locals (`const { x } = …`), and catch-clause vars - // are captured by NEITHER map — the scope query emits no `@declaration` / - // `@type-binding` for them — so those shadow forms still leak an edge. - // Closing that needs shared TS scope-query/extractor changes that alter call - // resolution beyond Nuxt, so it is deferred to a follow-up rather than fixed - // here. if (scope.filePath === filePath && scope.typeBindings.has(name)) { return true; } diff --git a/gitnexus/src/core/ingestion/languages/typescript/simple-hooks.ts b/gitnexus/src/core/ingestion/languages/typescript/simple-hooks.ts index a88ea5069..ec1d3f96f 100644 --- a/gitnexus/src/core/ingestion/languages/typescript/simple-hooks.ts +++ b/gitnexus/src/core/ingestion/languages/typescript/simple-hooks.ts @@ -56,6 +56,26 @@ export function tsBindingScopeFor( return walkToScope(innermost, tree, 'Class'); } + // JSDoc parameters are anchored on the whole function, unlike inline + // parameter annotations. They still belong to its body, not its parent. + if (decl['@type-binding.parameter'] !== undefined) { + return walkToScope(innermost, tree, 'Function'); + } + + // A for-of alias covers the whole loop. When that loop introduces a lexical + // block, its element type belongs beside the loop variable's name claim. + // Returning the owner explicitly prevents equal-range auto-hoisting. + const alias = decl['@type-binding.alias']; + if ( + alias !== undefined && + innermost.kind === 'Block' && + alias.range.startLine === innermost.range.startLine && + alias.range.startCol === innermost.range.startCol && + alias.range.endLine === innermost.range.endLine && + alias.range.endCol === innermost.range.endCol + ) + return innermost.id; + // `this.p = new Outer()` binds the FIELD, not a method-local, so the binding // belongs on the class the way an annotated field's does — that is the only // place `typeOfMemberOnClass` reads. Left on the innermost scope it would sit @@ -107,18 +127,11 @@ function isVarDeclaration(captureText: string): boolean { // ─── importOwningScope ──────────────────────────────────────────────────── /** - * TypeScript imports are syntactically top-level: `import_statement` is - * legal only inside `program` (the module root). `namespace X { … }` - * bodies CAN contain imports (`internal_module`), in which case the - * import scopes to the namespace. Dynamic `import()` calls appear - * inside any scope but their runtime effect is still a module-level - * resolution — we attach the `ParsedImport` to the innermost Module / - * Namespace scope so the binding is visible through the full subtree. - * - * Returning `null` delegates to the central default, which walks to - * the nearest enclosing `Module`/`Namespace`. That matches our rule, - * so we only override when we explicitly need a non-default scope - * (we don't). + * Static external imports belong to the module. Local require/await-import + * captures carry an explicit declaredAtScope from their binding declaration: + * var hoists to a function, while let/const remain in their lexical block. + * The extractor honors that explicit owner before consulting this hook. + * Returning null leaves other imports at their syntactic scope. */ export function tsImportOwningScope( _imp: ParsedImport, diff --git a/gitnexus/src/core/ingestion/languages/vue.ts b/gitnexus/src/core/ingestion/languages/vue.ts index d6493a950..d0a21767f 100644 --- a/gitnexus/src/core/ingestion/languages/vue.ts +++ b/gitnexus/src/core/ingestion/languages/vue.ts @@ -2,7 +2,7 @@ * Vue language provider. * * Vue SFCs are preprocessed by extracting the `, + 'vue-jsx.vue': `\n`, + 'vue-ts.vue': `\n`, + 'rust.rs': `mod target; +fn allowed() { run(); use crate::target::run; } +fn blocked() { use crate::missing::run; run(); } +fn typed(value: Item) { use crate::target::Item; value.run(); } +fn sibling() { run(); } +`, + 'cpp.cpp': `#include "target.hpp" +int allowed() { using target::run; return run(); } +int blocked() { using missing::run; return run(); } +int directive() { using namespace target; return run(); } +int typed() { using target::Item; Item value; return value.run(); } +`, + ...(zig.available + ? { + 'zig.zig': `pub fn allowed() void { const ns = @import("target.zig"); ns.run(); } +pub fn blocked() void { ns.run(); const ns = @import("target.zig"); } +pub fn siblings() void { { const ns = @import("a.zig"); ns.run(); } { const ns = @import("b.zig"); ns.run(); } } +`, + } + : {}), +}; + +// Maps/Sets are part of the shared Scope contract. Their entries, and every +// provider fact, must contain only data: never a tree-sitter node or callback. +function assertPlainFacts(value: unknown): void { + if (value === null || value === undefined || typeof value !== 'object') { + expect(['function', 'symbol']).not.toContain(typeof value); + return; + } + if (value instanceof Map) { + for (const [key, item] of value) { + assertPlainFacts(key); + assertPlainFacts(item); + } + } else if (value instanceof Set || Array.isArray(value)) { + for (const item of value) assertPlainFacts(item); + } else { + expect([Object.prototype, null]).toContain(Object.getPrototypeOf(value)); + for (const item of Object.values(value)) assertPlainFacts(item); + } +} + +describe('local-import facts across the real parse worker', () => { + let tempDir: string; + let output: Awaited>; + let workerFiles: Map; + let directFiles: Map; + let witness: Array<{ + type: string; + files?: string[]; + fileCount?: number; + parsedPaths?: string[]; + }>; + + beforeAll(async () => { + tempDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gitnexus-local-import-worker-')); + const repoDir = path.join(tempDir, 'repo'); + fs.mkdirSync(repoDir); + const marker = path.join(tempDir, 'worker.jsonl'); + const worker = path.join(tempDir, 'observed-worker.mjs'); + const realWorker = new URL( + '../../dist/core/ingestion/workers/parse-worker.js', + import.meta.url, + ); + expect(fs.existsSync(realWorker), 'build gitnexus before real-worker parity tests').toBe(true); + fs.writeFileSync( + worker, + ` +import fs from 'node:fs'; +import { parentPort, threadId } from 'node:worker_threads'; +const record = (entry) => fs.appendFileSync(${JSON.stringify(marker)}, JSON.stringify(entry) + '\\n'); +record({ type: 'boot', threadId }); +parentPort.on('message', (message) => { + if (message.type === 'sub-batch') record({ type: 'dispatch', files: message.files.map(file => file.path) }); +}); +const send = parentPort.postMessage.bind(parentPort); +parentPort.postMessage = (message, ...rest) => { + if (message.type === 'result') record({ type: 'result', fileCount: message.data.fileCount, + parsedPaths: message.data.parsedFiles.map(file => file.filePath) }); + return send(message, ...rest); +}; +await import(${JSON.stringify(realWorker.href)}); +`, + ); + directFiles = new Map(); + for (const [filePath, source] of Object.entries(fixtures)) { + fs.writeFileSync(path.join(repoDir, filePath), source); + const provider = getProviderForFile(filePath)!; + const parsed = extractParsedFile(provider, source, filePath); + expect(parsed, `direct extraction for ${filePath}`).toBeDefined(); + const captureSideChannel = provider.collectCaptureSideChannel?.(filePath); + directFiles.set( + filePath, + captureSideChannel === undefined ? parsed! : { ...parsed!, captureSideChannel }, + ); + } + const paths = Object.keys(fixtures); + output = await runChunkedParseAndResolve( + createKnowledgeGraph(), + paths.map((filePath) => ({ + path: filePath, + size: fs.statSync(path.join(repoDir, filePath)).size, + })), + paths, + paths.length, + repoDir, + Date.now(), + () => {}, + { workerUrlForTest: pathToFileURL(worker), workerPoolSize: 1 }, + ); + workerFiles = new Map(output.parsedFiles.map((file) => [file.filePath, file])); + witness = fs + .readFileSync(marker, 'utf8') + .trim() + .split('\n') + .map((line) => JSON.parse(line)); + }, 120_000); + + afterAll(() => { + // runChunkedParseAndResolve owns the pool's native-safe drain handshake. + if (tempDir) + fs.rmSync(tempDir, { recursive: true, force: true, maxRetries: 5, retryDelay: 50 }); + }); + + it('witnesses actual worker dispatch and returned ParsedFiles without a fallback', () => { + expect(output.usedWorkerPool).toBe(true); + expect(output.reparsedFileCount).toBe(Object.keys(fixtures).length); + expect(output.parseCacheHitFileCount).toBe(0); + expect(output.scopeExtractionFailures).toEqual([]); + expect(witness.some((entry) => entry.type === 'boot')).toBe(true); + expect(witness.flatMap((entry) => entry.files ?? []).sort()).toEqual( + Object.keys(fixtures).sort(), + ); + expect(witness.flatMap((entry) => entry.parsedPaths ?? []).sort()).toEqual( + Object.keys(fixtures).sort(), + ); + expect([...workerFiles.keys()].sort()).toEqual(Object.keys(fixtures).sort()); + }); + + it.each(Object.keys(fixtures))('preserves complete direct provider facts for %s', (filePath) => { + const direct = directFiles.get(filePath)!; + const transported = workerFiles.get(filePath)!; + expect(direct.scopes.some((scope) => (scope.nameClaims?.length ?? 0) > 0)).toBe(true); + expect(transported).toEqual(direct); + assertPlainFacts(transported); + }); + + it('retains the deferred Depends caller independently of its lexical lookup scope', () => { + const parsed = workerFiles.get('depends.py')!; + const site = parsed.referenceSites.find( + (reference) => + reference.kind === 'call' && + reference.name === 'dependency' && + reference.callerScope !== undefined, + ); + expect(site).toBeDefined(); + expect(site!.inScope).toBe(parsed.moduleScope); + expect(site!.callerScope).not.toBe(site!.inScope); + expect(parsed.scopes.find((scope) => scope.id === site!.callerScope)?.kind).toBe('Function'); + }); + + it('derives identical finalized ImportEdge positions from transported facts', () => { + const target = extractParsedFile( + getProviderForFile('target.py')!, + 'def run(): return 1\n', + 'target.py', + )!; + const link = (source: ParsedFile) => + finalize( + { + files: [source, target], + workspaceIndex: undefined, + }, + { + importsBindAtLexicalScope: true, + resolveImportTarget: (raw) => (raw === 'target' ? target.filePath : null), + expandsWildcardTo: () => [], + mergeBindings: (existing, incoming) => [...existing, ...incoming], + }, + ); + const direct = link(directFiles.get('python.py')!); + const transported = link(workerFiles.get('python.py')!); + expect(transported.imports).toEqual(direct.imports); + const edges = [...transported.imports.values()].flat(); + expect(edges.length).toBeGreaterThan(0); + expect(edges.every((edge) => edge.atRange !== undefined)).toBe(true); + expect(edges.some((edge) => edge.targetFile === 'target.py' && edge.localName === 'run')).toBe( + true, + ); + }); + + it('exercises ownership, activation, semantic positions, and C++ side-channel data', () => { + const files = [...workerFiles.values()]; + const scopes = files.flatMap((file) => file.scopes); + const claims = scopes.flatMap((scope) => scope.nameClaims ?? []); + expect(claims.some((claim) => claim.availableFrom !== undefined)).toBe(true); + expect(claims.some((claim) => claim.redirect !== undefined)).toBe(true); + expect(claims.some((claim) => claim.inactive !== undefined)).toBe(true); + expect(claims.some((claim) => claim.hoisted === true)).toBe(true); + expect(claims.some((claim) => claim.merge === true)).toBe(true); + const policies = scopes.flatMap((scope) => (scope.lookupPolicy ? [scope.lookupPolicy] : [])); + for (const key of [ + 'skipFromChildren', + 'deferParentActivation', + 'parentScope', + 'visibleNamesFromChildren', + 'callerScopeIsAuthoritative', + 'nameClaimsComplete', + ] as const) { + expect( + policies.some((policy) => policy[key] !== undefined), + key, + ).toBe(true); + } + const bindings = scopes.flatMap((scope) => [...scope.bindings.values()].flat()); + expect(bindings.some((binding) => binding.declarationRange !== undefined)).toBe(true); + // BindingRef.availableFrom is synthesized by the C++ resolution hook, + // after transport; the persistence graph assertions cover its activation. + const types = scopes.flatMap((scope) => [...scope.typeBindings.values()]); + for (const key of ['lookupPosition', 'lookupPurpose', 'bindingRange'] as const) { + expect( + types.some((type) => type[key] !== undefined), + key, + ).toBe(true); + } + const imports = files.flatMap((file) => file.parsedImports); + expect(imports.some((imp) => imp.atRange !== undefined)).toBe(true); + expect(imports.some((imp) => imp.bindsAtLexicalScope === true)).toBe(true); + expect( + files.flatMap((file) => file.referenceSites).some((site) => site.lookupScope !== undefined), + ).toBe(true); + expect( + workerFiles.get('vue-ts.vue')!.localDefs.some((def) => def.graphPosition !== undefined), + ).toBe(true); + expect(workerFiles.get('cpp.cpp')!.captureSideChannel).toEqual( + expect.objectContaining({ + usingDeclarations: expect.arrayContaining([expect.any(Object)]), + }), + ); + }); +}); diff --git a/gitnexus/test/integration/resolvers/cpp-local-using-headers.test.ts b/gitnexus/test/integration/resolvers/cpp-local-using-headers.test.ts new file mode 100644 index 000000000..d32270599 --- /dev/null +++ b/gitnexus/test/integration/resolvers/cpp-local-using-headers.test.ts @@ -0,0 +1,154 @@ +import { afterAll, beforeAll, describe, expect, it } from 'vitest'; +import fs from 'node:fs'; +import os from 'node:os'; +import path from 'node:path'; +import { + getRelationships, + runPipelineFromRepo, + writeFixtureRepo, + type PipelineResult, +} from './helpers.js'; + +describe('C++ namespace using visibility through literal includes', () => { + let repoDir: string; + let result: PipelineResult; + beforeAll(async () => { + repoDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gitnexus-cpp-using-headers-')); + writeFixtureRepo(repoDir, { + 'definitions.hpp': `namespace helpers { +int work() { return 1; } +struct Worker { int run() { return 1; } }; +} +`, + 'named.hpp': '#include "definitions.hpp"\nusing helpers::work;\nusing helpers::Worker;\n', + 'forwarded.hpp': '#include "named.hpp"\n', + 'directive.hpp': '#include "definitions.hpp"\nusing namespace helpers;\n', + 'named.cpp': + '#include "named.hpp"\nint from_named() { return work(); }\nint from_type() { Worker w; return w.run(); }\n', + 'forwarded.cpp': '#include "forwarded.hpp"\nint from_forwarded() { return work(); }\n', + 'directive.cpp': '#include "directive.hpp"\nint from_directive() { return work(); }\n', + 'before.cpp': 'int before_include() { return work(); }\n#include "named.hpp"\n', + }); + result = await runPipelineFromRepo(repoDir, () => {}); + }, 60_000); + afterAll(() => { + if (repoDir !== undefined) fs.rmSync(repoDir, { recursive: true, force: true }); + }); + + it.each(['from_named', 'from_forwarded', 'from_directive'])( + 'resolves %s through the included namespace using', + (caller) => { + const calls = getRelationships(result, 'CALLS').filter((edge) => edge.source === caller); + expect(calls.map((edge) => [edge.target, edge.targetFilePath])).toEqual([ + ['work', 'definitions.hpp'], + ]); + expect(calls.every((edge) => edge.rel.reason !== 'global-name-fallback')).toBe(true); + }, + ); + it('retains the imported class identity', () => { + const calls = getRelationships(result, 'CALLS').filter( + (edge) => edge.source === 'from_type' && edge.target === 'run', + ); + expect(calls.map((edge) => edge.targetFilePath)).toEqual(['definitions.hpp']); + expect(calls.every((edge) => edge.rel.reason !== 'global-name-fallback')).toBe(true); + }); + it('does not make a later include visible in an earlier function definition', () => { + expect( + getRelationships(result, 'CALLS').filter((edge) => edge.source === 'before_include'), + ).toEqual([]); + }); +}); + +describe('C++ unresolved named using through a literal include', () => { + let repoDir: string; + let result: PipelineResult; + beforeAll(async () => { + repoDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gitnexus-cpp-unresolved-using-header-')); + writeFixtureRepo(repoDir, { + 'unresolved.hpp': 'using absent::work;\n', + 'caller.cpp': '#include "unresolved.hpp"\nint from_unresolved_header() { return work(); }\n', + 'forwarded.hpp': '#include "unresolved.hpp"\n', + 'forwarded.cpp': + '#include "forwarded.hpp"\nint from_forwarded_unresolved_header() { return work(); }\n', + 'missing-export.hpp': 'namespace present {}\nusing present::work;\n', + 'missing-export.cpp': + '#include "missing-export.hpp"\nint from_missing_export_header() { return work(); }\n', + 'before.cpp': + 'int before_unresolved_include() { return work(); }\n#include "unresolved.hpp"\n', + 'local-before.hpp': 'using absent::local_work;\n', + 'local-before.cpp': + 'int local_work() { return 1; }\nint before_local_unresolved_include() { return local_work(); }\n#include "local-before.hpp"\nint after_local_unresolved_include() { return local_work(); }\n', + 'unrelated.cpp': 'int work() { return 9; }\n', + }); + result = await runPipelineFromRepo(repoDir, () => {}); + }, 60_000); + afterAll(() => { + if (repoDir !== undefined) fs.rmSync(repoDir, { recursive: true, force: true }); + }); + + it('does not fall back to an unrelated workspace function after including an unresolved named using', () => { + expect( + getRelationships(result, 'CALLS').filter((edge) => edge.source === 'from_unresolved_header'), + ).toEqual([]); + }); + it.each([ + 'from_forwarded_unresolved_header', + 'from_missing_export_header', + 'after_local_unresolved_include', + ])('preserves the unresolved header claim in %s', (caller) => { + expect(getRelationships(result, 'CALLS').filter((edge) => edge.source === caller)).toEqual([]); + }); + it('keeps global fallback available before an unresolved header include', () => { + const calls = getRelationships(result, 'CALLS').filter( + (edge) => edge.source === 'before_unresolved_include', + ); + expect(calls.map((edge) => [edge.target, edge.targetFilePath, edge.rel.reason])).toEqual([ + ['work', 'unrelated.cpp', 'global-name-fallback'], + ]); + }); + it('retains the local declaration before an unresolved header include', () => { + const calls = getRelationships(result, 'CALLS').filter( + (edge) => edge.source === 'before_local_unresolved_include', + ); + expect(calls.map((edge) => [edge.target, edge.targetFilePath])).toEqual([ + ['local_work', 'local-before.cpp'], + ]); + }); +}); + +describe('C++ included and local overload identity', () => { + let repoDir: string; + let result: PipelineResult; + beforeAll(async () => { + repoDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gitnexus-cpp-include-overloads-')); + writeFixtureRepo(repoDir, { + 'overloads.hpp': 'int choose(int value);\n', + 'main.cpp': + '#include "overloads.hpp"\nint choose(double value) { return 2; }\nint choose_integer() { return choose(1); }\nint choose_double() { return choose(1.5); }\n', + }); + result = await runPipelineFromRepo(repoDir, () => {}); + }, 60_000); + afterAll(() => { + if (repoDir !== undefined) fs.rmSync(repoDir, { recursive: true, force: true }); + }); + it('keeps a distinct included overload beside the local definition', () => { + const calls = getRelationships(result, 'CALLS').filter( + (edge) => edge.source === 'choose_integer', + ); + expect(calls.map((edge) => [edge.target, edge.targetFilePath])).toEqual([ + ['choose', 'overloads.hpp'], + ]); + expect(result.graph.getNode(calls[0].rel.targetId)?.properties.parameterTypes).toEqual(['int']); + }); + it('selects the local overload for its matching argument', () => { + const calls = getRelationships(result, 'CALLS').filter( + (edge) => edge.source === 'choose_double', + ); + expect(calls.map((edge) => [edge.target, edge.targetFilePath])).toEqual([ + ['choose', 'main.cpp'], + ]); + expect(result.graph.getNode(calls[0].rel.targetId)?.properties.parameterTypes).toEqual([ + 'double', + ]); + }); +}); diff --git a/gitnexus/test/integration/resolvers/cpp-local-using.test.ts b/gitnexus/test/integration/resolvers/cpp-local-using.test.ts new file mode 100644 index 000000000..1580a45bc --- /dev/null +++ b/gitnexus/test/integration/resolvers/cpp-local-using.test.ts @@ -0,0 +1,173 @@ +import { afterAll, beforeAll, describe, expect, it } from 'vitest'; +import fs from 'node:fs'; +import os from 'node:os'; +import path from 'node:path'; +import { + getRelationships, + runPipelineFromRepo, + writeFixtureRepo, + type PipelineResult, +} from './helpers.js'; + +describe('C++ lexical using declarations and directives', () => { + let repoDir: string; + let result: PipelineResult; + + beforeAll(async () => { + repoDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gitnexus-cpp-local-using-')); + writeFixtureRepo(repoDir, { + 'alpha.hpp': `namespace alpha { +int work() { return 1; } +int collision() { return 1; } +struct Worker { int run() { return 1; } }; +namespace detail { int nested_work() { return 1; } } +} +int included_helper() { return 1; } +`, + 'beta.hpp': `namespace beta { +int work() { return 2; } +struct Worker { int run() { return 2; } }; +namespace detail { int nested_work() { return 2; } } +} +`, + 'decoy.cpp': 'int work() { return 9; }\n', + 'main.cpp': `#include "alpha.hpp" +#include "beta.hpp" +int collision() { return 0; } +int named_alpha() { using alpha::work; return work(); } +int named_beta() { using beta::work; return work(); } +int before_named() { return work(); using alpha::work; } +int unresolved_named() { using missing::work; return work(); } +int missing_member() { using alpha::absent; return absent(); } +int directive_alpha() { using namespace alpha; return work(); } +int directive_beta() { using namespace beta; return work(); } +int before_directive() { return work(); using namespace alpha; } +int sibling() { return work(); } +int nested_block() { { using alpha::work; work(); } return 0; } +int outside_block() { { using alpha::work; } return work(); } +int ambiguous_directives() { using namespace alpha; using namespace beta; return work(); } +int type_alpha() { using alpha::Worker; Worker worker; return worker.run(); } +int type_beta() { using beta::Worker; Worker worker; return worker.run(); } +int directive_type() { using namespace alpha; Worker worker; return worker.run(); } +int before_type() { Worker worker; using alpha::Worker; return worker.run(); } +int value_shadow() { using namespace alpha; { int work = 0; return work(); } } +int ancestor_value_shadow() { int work = 0; { using namespace alpha; return work(); } } +int global_collision() { using namespace alpha; return collision(); } +int directive_before_value() { using namespace alpha; work(); int work = 0; return 0; } +int prototype_parameter() { using namespace alpha; void unused(int work); return work(); } +int function_pointer_parameter(void (*callback)(int work)) { using namespace alpha; return work(); } +int actual_parameter(int work) { using namespace alpha; return work(); } +int nested_alpha() { using alpha::detail::nested_work; return nested_work(); } +int nested_beta() { using beta::detail::nested_work; return nested_work(); } +struct InlineCaller { int inline_method() { using alpha::work; return work(); } }; +int header_control() { return included_helper(); } +`, + 'overloads.hpp': 'namespace imported { int choose(int value) { return value; } }\n', + 'overloads.cpp': `#include "overloads.hpp" +int choose(double value) { return 1; } +int before_using() { return choose(1.0); } +using imported::choose; +int local_overload() { return choose(1.0); } +int imported_overload() { return choose(1); } +int block_hides_outer() { using imported::choose; return choose(1.0); } +`, + 'nested-overloads.cpp': `#include "overloads.hpp" +namespace nested { +int choose(double value) { return 2; } +using imported::choose; +int nested_local() { return choose(1.0); } +int nested_imported() { return choose(1); } +} +`, + 'absent.cpp': 'int absent() { return 0; }\n', + }); + result = await runPipelineFromRepo(repoDir, () => {}); + }, 60_000); + + afterAll(() => { + if (repoDir !== undefined) fs.rmSync(repoDir, { recursive: true, force: true }); + }); + + it.each([ + ['named_alpha', 'alpha.hpp'], + ['named_beta', 'beta.hpp'], + ['directive_alpha', 'alpha.hpp'], + ['directive_beta', 'beta.hpp'], + ['nested_block', 'alpha.hpp'], + ['inline_method', 'alpha.hpp'], + ['directive_before_value', 'alpha.hpp'], + ['prototype_parameter', 'alpha.hpp'], + ['function_pointer_parameter', 'alpha.hpp'], + ])('resolves %s to its exact namespace member', (caller, targetFile) => { + const calls = getRelationships(result, 'CALLS').filter( + (edge) => edge.source === caller && edge.target === 'work', + ); + expect(calls.map((edge) => edge.targetFilePath)).toEqual([targetFile]); + }); + + it.each([ + ['nested_alpha', 'alpha.hpp'], + ['nested_beta', 'beta.hpp'], + ])('preserves the full namespace path in %s', (caller, targetFile) => { + const calls = getRelationships(result, 'CALLS').filter( + (edge) => edge.source === caller && edge.target === 'nested_work', + ); + expect(calls.map((edge) => edge.targetFilePath)).toEqual([targetFile]); + }); + + it.each(['before_named', 'before_directive', 'sibling', 'outside_block'])( + 'does not leak a namespace using into %s', + (caller) => { + const calls = getRelationships(result, 'CALLS').filter( + (edge) => edge.source === caller && edge.target === 'work', + ); + expect(calls.some((edge) => edge.targetFilePath === 'alpha.hpp')).toBe(false); + expect(calls.some((edge) => edge.targetFilePath === 'beta.hpp')).toBe(false); + }, + ); + + it.each([ + 'unresolved_named', + 'missing_member', + 'ambiguous_directives', + 'value_shadow', + 'ancestor_value_shadow', + 'actual_parameter', + 'global_collision', + 'before_type', + ])('does not guess an unrelated target for %s', (caller) => { + expect(getRelationships(result, 'CALLS').filter((edge) => edge.source === caller)).toEqual([]); + }); + + it.each([ + ['type_alpha', 'alpha.hpp'], + ['type_beta', 'beta.hpp'], + ['directive_type', 'alpha.hpp'], + ])('uses the namespace type identity in %s', (caller, targetFile) => { + const calls = getRelationships(result, 'CALLS').filter( + (edge) => edge.source === caller && edge.target === 'run', + ); + expect(calls.map((edge) => edge.targetFilePath)).toEqual([targetFile]); + }); + + it('preserves compilation-unit include bindings', () => { + const calls = getRelationships(result, 'CALLS').filter( + (edge) => edge.source === 'header_control' && edge.target === 'included_helper', + ); + expect(calls.map((edge) => edge.targetFilePath)).toEqual(['alpha.hpp']); + }); + + it.each([ + ['before_using', 'overloads.cpp'], + ['local_overload', 'overloads.cpp'], + ['imported_overload', 'overloads.hpp'], + ['nested_local', 'nested-overloads.cpp'], + ['nested_imported', 'overloads.hpp'], + ['block_hides_outer', 'overloads.hpp'], + ])('preserves the exact overload selected by %s', (caller, targetFile) => { + const calls = getRelationships(result, 'CALLS').filter( + (edge) => edge.source === caller && edge.target === 'choose', + ); + expect(calls.map((edge) => edge.targetFilePath)).toEqual([targetFile]); + }); +}); diff --git a/gitnexus/test/integration/resolvers/fastapi-depends.test.ts b/gitnexus/test/integration/resolvers/fastapi-depends.test.ts index 93394528e..acd64c9e5 100644 --- a/gitnexus/test/integration/resolvers/fastapi-depends.test.ts +++ b/gitnexus/test/integration/resolvers/fastapi-depends.test.ts @@ -1,6 +1,14 @@ -import { describe, it, expect, beforeAll } from 'vitest'; +import { describe, it, expect, beforeAll, afterAll } from 'vitest'; +import fs from 'node:fs'; +import os from 'node:os'; import path from 'path'; -import { FIXTURES, getRelationships, runPipelineFromRepo, type PipelineResult } from './helpers.js'; +import { + FIXTURES, + getRelationships, + runPipelineFromRepo, + writeFixtureRepo, + type PipelineResult, +} from './helpers.js'; describe('FastAPI Depends() CALLS edge extraction', () => { let result: PipelineResult; @@ -41,3 +49,76 @@ describe('FastAPI Depends() CALLS edge extraction', () => { expect(edge).toBeDefined(); }); }); + +describe('FastAPI dependency caller and lookup ownership', () => { + let repoDir: string; + let result: PipelineResult; + + beforeAll(async () => { + repoDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gitnexus-fastapi-ownership-')); + writeFixtureRepo(repoDir, { + 'outer.py': 'def dependency(): return 1\ndef hidden(): return 1\n', + 'inner.py': 'def dependency(): return 2\ndef hidden(): return 2\n', + 'routes.py': `from fastapi import Depends +from outer import dependency, hidden +import outer as deps + +def handler(value=Depends(dependency)): + from inner import dependency + return dependency() + +def member_handler(value=Depends(deps.dependency)): + import inner as deps + return deps.dependency() + +def wrapper(hidden): + def blocked_handler(value=Depends(hidden)): + from inner import hidden + return blocked_handler + +def ordinary(value=dependency()): + from inner import dependency + return dependency() +`, + }); + result = await runPipelineFromRepo(repoDir, () => {}, { skipGraphPhases: true }); + }, 60000); + + afterAll(() => { + if (repoDir) fs.rmSync(repoDir, { recursive: true, force: true }); + }); + + it.each(['handler', 'member_handler'])( + 'attributes dependencies to %s while resolving outside its local imports', + (caller) => { + expect( + getRelationships(result, 'CALLS') + .filter((edge) => edge.source === caller && edge.target === 'dependency') + .map((edge) => edge.rel.targetId) + .sort(), + ).toEqual(['Function:inner.py:dependency', 'Function:outer.py:dependency']); + }, + ); + + it('keeps an outer parameter barrier for a dependency despite a handler-local import', () => { + expect(getRelationships(result, 'CALLS').filter((edge) => edge.target === 'hidden')).toEqual( + [], + ); + }); + + it('keeps ordinary default execution attributed to the file', () => { + expect( + getRelationships(result, 'CALLS') + .filter( + (edge) => + edge.target === 'dependency' && + (edge.sourceLabel === 'File' || edge.source === 'ordinary'), + ) + .map((edge) => `${edge.rel.sourceId} -> ${edge.rel.targetId}`) + .sort(), + ).toEqual([ + 'File:routes.py -> Function:outer.py:dependency', + 'Function:routes.py:ordinary -> Function:inner.py:dependency', + ]); + }); +}); diff --git a/gitnexus/test/integration/resolvers/generic-field-receiver-matrix.test.ts b/gitnexus/test/integration/resolvers/generic-field-receiver-matrix.test.ts index 8b4da76d9..a130f3a12 100644 --- a/gitnexus/test/integration/resolvers/generic-field-receiver-matrix.test.ts +++ b/gitnexus/test/integration/resolvers/generic-field-receiver-matrix.test.ts @@ -84,7 +84,9 @@ * declaration (the scope chain binds the name; the declaration is in the same * file; the index proves the name is a template family; the file has no * cross-file class channel to be absent from). `py-erased-grounding` pins the - * refusal and `py-generic-grounding-mirrors`, `cpp-csharp-index-channel` and the + * refusal, including a compound receiver head: an unresolved import of `Mapped` + * blocks the unrelated workspace class throughout `m.inner.ping()` as well. + * `py-generic-grounding-mirrors`, `cpp-csharp-index-channel` and the * `-crossfile` rows pin the four shapes that would break if it were stricter. * * ── GAPS THIS FILE ONCE PINNED, NOW CLOSED ─────────────────────────────────── @@ -134,10 +136,6 @@ * - A C++ node id drops the namespace, so two same-named specializations in * ONE file collapse to one node. `cpp-spec-lexical-shadowing` puts its two * `Box` declarations in two FILES for exactly that reason. - * - A chain HEAD whose own type was erased still binds through the - * bare-identifier branch's callable-alias retry: `m.inner.ping()` where - * `m: Mapped[User]` resolves, while the one-segment-shallower `m.save(u)` - * correctly refuses. `py-erased-grounding` / `run_head_chain` pins it. * * ── THE NEGATIVE CONTROLS ───────────────────────────────────────────────────── * @@ -1764,8 +1762,8 @@ def run_no_import_channel(m: Mapped[BUser], u: BUser) -> None: }, { caller: 'run_head_chain', - targets: ['Method:other.py:Inner.ping#0'], - note: 'REMAINING WRONG EDGE, pinned at its measured value so closing it is a visible flip. `m.inner.ping()` binds the unrelated workspace `Mapped` through a route that survives the refusal, while the one-segment-shallower `m.save(u)` (`run_param`, above) correctly declines — same receiver, same declared type, one more segment. The obvious one-line guard in the bare-identifier branch (decline every retry once an erased application failed to ground) was tried and MEASURED not to close it, so the surviving route is elsewhere and this needs its own diagnosis rather than a guess. Deliberately not fixed here: a broader refusal would change chain-head resolution for every language without pinning the shape it is meant to fix.', + targets: [], + note: 'The unresolved external import of Mapped owns the name throughout compound receiver lookup. `m.inner.ping()` cannot borrow the unrelated workspace Mapped class; the same-file and no-import-channel rows retain positive controls.', }, { caller: 'run_param', diff --git a/gitnexus/test/integration/resolvers/local-import-persistence.test.ts b/gitnexus/test/integration/resolvers/local-import-persistence.test.ts new file mode 100644 index 000000000..b10b9e35f --- /dev/null +++ b/gitnexus/test/integration/resolvers/local-import-persistence.test.ts @@ -0,0 +1,426 @@ +import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest'; +import fs from 'node:fs'; +import os from 'node:os'; +import path from 'node:path'; +import { pathToFileURL } from 'node:url'; +import { SupportedLanguages, type ParsedFile } from 'gitnexus-shared'; +import * as scopeBridge from '../../../src/core/ingestion/scope-extractor-bridge.js'; +import { getProviderForFile } from '../../../src/core/ingestion/languages/index.js'; +import { + loadParseCache, + loadParseCacheChunk, + PARSE_CACHE_VERSION, + pruneCache, + saveParseCache, + type ParseCache, +} from '../../../src/storage/parse-cache.js'; +import { + clearParsedFileStore, + durableChunkHasShards, + getDurableParsedFileDir, + loadDurableParsedFileIndex, + loadParsedFilesForPaths, + pruneAndSaveDurableParsedFileStore, +} from '../../../src/storage/parsedfile-store.js'; +import { describeGrammarPresence, optionalGrammarGate } from '../../helpers/optional-grammar.js'; +import { runPipelineFromRepo, writeFixtureRepo, type PipelineResult } from './helpers.js'; + +const zig = optionalGrammarGate(SupportedLanguages.Zig); +describeGrammarPresence(zig); + +interface ExpectedCall { + file: string; + caller: string; + targetFile: string; + target: string; +} + +describe('local-import graph targets across both persistent parse stores', () => { + let tempDir: string; + let repoDir: string; + let storageDir: string; + let marker: string; + let worker: URL; + let files: Record; + let expected: ExpectedCall[]; + const noOp = () => {}; + + beforeAll(() => { + tempDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gitnexus-local-import-persistence-')); + repoDir = path.join(tempDir, 'repo'); + storageDir = path.join(tempDir, 'storage'); + marker = path.join(tempDir, 'worker.jsonl'); + const workerPath = path.join(tempDir, 'observed-worker.mjs'); + const actual = new URL('../../../dist/core/ingestion/workers/parse-worker.js', import.meta.url); + expect(fs.existsSync(actual), 'build gitnexus before real-worker persistence tests').toBe(true); + fs.writeFileSync( + workerPath, + ` +import fs from 'node:fs'; +import { parentPort, threadId } from 'node:worker_threads'; +const record = (entry) => fs.appendFileSync(${JSON.stringify(marker)}, JSON.stringify(entry) + '\\n'); +record({ type: 'boot', threadId }); +parentPort.on('message', (message) => { + if (message.type === 'sub-batch') record({ type: 'dispatch', files: message.files.map(file => file.path) }); +}); +const send = parentPort.postMessage.bind(parentPort); +parentPort.postMessage = (message, ...rest) => { + if (message.type === 'result') record({ type: 'result', fileCount: message.data.fileCount }); + return send(message, ...rest); +}; +await import(${JSON.stringify(actual.href)}); +`, + ); + worker = pathToFileURL(workerPath); + expected = []; + files = { + 'py/main.py': `from a import run +def allowed(): + from a import run + return run() +def denied(): + run() + from a import run +`, + 'py/a.py': 'def run(): return 1\n', + 'py/b.py': 'def run(): return 2\n', + 'py/metadata.py': `from a import run +from fastapi import Depends +def dependency(): return 1 +def handler(value=Depends(dependency)): return value +def redirected(): + global run + from a import run +class Parent: pass +class Child(Parent): + value = 1 +def comprehension(values): + return [value for value in values] +`, + 'metadata.ts': `class Parent {} +class Child extends Parent {} +function typed(value: Child): Parent { return value; } +const Named = class Hidden { method() { return Hidden; } }; +`, + 'Cargo.toml': + '[package]\nname = "local-import-persistence"\nversion = "0.1.0"\nedition = "2021"\n', + 'src/lib.rs': `mod target; +pub fn allowed() { run(); use crate::target::run; } +pub fn denied() { use crate::missing::run; run(); } +`, + 'src/target.rs': 'pub fn run() {}\n', + 'cpp/main.cpp': `#include "target.hpp" +int allowed() { using target::run; return run(); } +int denied() { using missing::run; return run(); } +int before() { return run(); using target::run; } +int outside() { { using target::run; } return run(); } +`, + 'cpp/target.hpp': 'namespace target { int run() { return 1; } }\n', + }; + expected.push( + { file: 'py/main.py', caller: 'allowed', targetFile: 'py/a.py', target: 'run' }, + { file: 'src/lib.rs', caller: 'allowed', targetFile: 'src/target.rs', target: 'run' }, + { file: 'cpp/main.cpp', caller: 'allowed', targetFile: 'cpp/target.hpp', target: 'run' }, + ); + for (const extension of ['ts', 'tsx', 'js', 'jsx']) { + const dir = extension; + const targetExtension = extension.startsWith('ts') ? 'ts' : 'js'; + files[`${dir}/main.${extension}`] = + `export function allowed() { const ns = require('./a'); return ns.run(); } +export function denied(require) { const ns = require('./a'); return ns.run(); } +`; + files[`${dir}/a.${targetExtension}`] = 'export function run() { return 1; }\n'; + files[`${dir}/b.${targetExtension}`] = 'export function run() { return 2; }\n'; + expected.push({ + file: `${dir}/main.${extension}`, + caller: 'allowed', + targetFile: `${dir}/a.${targetExtension}`, + target: 'run', + }); + } + for (const language of ['js', 'jsx', 'ts']) { + const dir = `vue-${language}`; + const targetExtension = language === 'ts' ? 'ts' : 'js'; + files[`${dir}/App.vue`] = `\n`; + files[`${dir}/a.${targetExtension}`] = 'export function run() { return 1; }\n'; + expected.push({ + file: `${dir}/App.vue`, + caller: 'allowed', + targetFile: `${dir}/a.${targetExtension}`, + target: 'run', + }); + } + if (zig.available) { + files['zig/main.zig'] = `pub fn allowed() void { const ns = @import("target.zig"); ns.run(); } +pub fn denied() void { ns.run(); const ns = @import("target.zig"); } +`; + files['zig/target.zig'] = 'pub fn run() void {}\n'; + expected.push({ + file: 'zig/main.zig', + caller: 'allowed', + targetFile: 'zig/target.zig', + target: 'run', + }); + } + writeFixtureRepo(repoDir, files); + }); + + afterAll(() => { + if (tempDir) + fs.rmSync(tempDir, { recursive: true, force: true, maxRetries: 5, retryDelay: 50 }); + }); + + const run = (parseCache: ParseCache) => + runPipelineFromRepo(repoDir, noOp, { + skipGraphPhases: true, + workerPoolSize: 1, + workerUrlForTest: worker, + parseCache, + }); + + async function persist(cache: ParseCache): Promise { + pruneCache(cache, cache.usedKeys); + const keys = await saveParseCache(storageDir, cache); + await pruneAndSaveDurableParsedFileStore( + getDurableParsedFileDir(storageDir), + PARSE_CACHE_VERSION, + new Set(keys), + ); + } + + async function durableRecords(): Promise> { + await clearParsedFileStore(storageDir); + const index = await loadDurableParsedFileIndex( + getDurableParsedFileDir(storageDir), + PARSE_CACHE_VERSION, + ); + expect(index.size).toBeGreaterThan(0); + const paths = new Set(); + for (const [key, wanted] of index) { + expect(await durableChunkHasShards(storageDir, key, wanted)).toBe(true); + for (const filePath of wanted) paths.add(filePath); + } + // This is the same validated durable-to-run snapshot and reader used by + // warm scope resolution, after the previous pipeline cleared its run store. + const loaded = await loadParsedFilesForPaths(storageDir, paths); + expect([...loaded.keys()].sort()).toEqual([...paths].sort()); + return loaded; + } + + function graphEdges(result: PipelineResult): string[] { + return [...result.graph.iterRelationships()] + .filter((edge) => edge.type === 'CALLS' || edge.type === 'IMPORTS') + .map((edge) => `${edge.type} ${edge.sourceId} -> ${edge.targetId}`) + .sort(); + } + + function assertExactCalls(result: PipelineResult, wanted: ExpectedCall[]): void { + const sourceFiles = new Set(expected.map((item) => item.file)); + const nodeId = (filePath: string, name: string) => { + const nodes = [...result.graph.nodes.values()].filter( + (node) => + node.properties.filePath === filePath && + node.properties.name === name && + (node.label === 'Function' || node.label === 'Method'), + ); + expect(nodes, `one callable ${filePath}:${name}`).toHaveLength(1); + return nodes[0]!.id; + }; + const calls = [...result.graph.iterRelationships()].filter( + (edge) => + edge.type === 'CALLS' && + sourceFiles.has(result.graph.getNode(edge.sourceId)?.properties.filePath ?? ''), + ); + expect(calls.map((edge) => `${edge.sourceId} -> ${edge.targetId}`).sort()).toEqual( + wanted + .map( + (item) => `${nodeId(item.file, item.caller)} -> ${nodeId(item.targetFile, item.target)}`, + ) + .sort(), + ); + // The exact set rejects both sibling-target leakage and every denied call. + } + + it('reuses genuine cold output, retains exact targets, and invalidates changed ownership', async () => { + const parsedPaths = Object.keys(files).filter( + (filePath) => getProviderForFile(filePath) !== null, + ); + const extract = vi.spyOn(scopeBridge, 'extractParsedFile'); + try { + const coldCache = await loadParseCache(storageDir); + const cold = await run(coldCache); + expect(cold.usedWorkerPool).toBe(true); + expect(cold.reparsedFileCount).toBe(parsedPaths.length); + expect(cold.parseCacheHitFileCount).toBe(0); + expect(extract).not.toHaveBeenCalled(); + const observed = fs + .readFileSync(marker, 'utf8') + .trim() + .split('\n') + .map((line) => JSON.parse(line) as { type: string; files?: string[]; fileCount?: number }); + expect(observed.some((entry) => entry.type === 'boot')).toBe(true); + expect(observed.flatMap((entry) => entry.files ?? []).sort()).toEqual(parsedPaths.sort()); + expect( + observed + .filter((entry) => entry.type === 'result') + .reduce((sum, entry) => sum + (entry.fileCount ?? 0), 0), + ).toBe(parsedPaths.length); + assertExactCalls(cold, expected); + await persist(coldCache); + const coldRecords = await durableRecords(); + expect([...coldRecords.keys()].sort()).toEqual(parsedPaths.sort()); + expect(coldRecords.get('cpp/main.cpp')!.captureSideChannel).toEqual( + expect.objectContaining({ + usingDeclarations: expect.arrayContaining([expect.any(Object)]), + }), + ); + expect( + coldRecords + .get('py/main.py')! + .scopes.some((scope) => + scope.nameClaims?.some( + (claim) => + claim.name === 'run' && + claim.kind === 'import' && + claim.availableFrom !== undefined, + ), + ), + ).toBe(true); + expect( + coldRecords.get('vue-ts/App.vue')!.localDefs.some((def) => def.graphPosition !== undefined), + ).toBe(true); + const scopes = [...coldRecords.values()].flatMap((file) => file.scopes); + const claims = scopes.flatMap((scope) => scope.nameClaims ?? []); + for (const key of [ + 'name', + 'range', + 'kind', + 'purpose', + 'availableFrom', + 'inactive', + 'redirect', + 'hoisted', + 'merge', + ] as const) { + expect( + claims.some((claim) => claim[key] !== undefined), + `durable NameClaim.${key}`, + ).toBe(true); + } + const policies = scopes.flatMap((scope) => (scope.lookupPolicy ? [scope.lookupPolicy] : [])); + for (const key of [ + 'skipFromChildren', + 'deferParentActivation', + 'parentScope', + 'visibleNamesFromChildren', + 'callerScopeIsAuthoritative', + 'nameClaimsComplete', + ] as const) { + expect( + policies.some((policy) => policy[key] !== undefined), + `durable Scope.lookupPolicy.${key}`, + ).toBe(true); + } + const types = scopes.flatMap((scope) => [...scope.typeBindings.values()]); + for (const key of ['lookupPosition', 'lookupPurpose', 'bindingRange'] as const) { + expect( + types.some((type) => type[key] !== undefined), + `durable TypeRef.${key}`, + ).toBe(true); + } + expect( + scopes + .flatMap((scope) => [...scope.bindings.values()].flat()) + .some((binding) => binding.declarationRange !== undefined), + ).toBe(true); + const imports = [...coldRecords.values()].flatMap((file) => file.parsedImports); + for (const key of ['atRange', 'declaredAtScope', 'bindsAtLexicalScope'] as const) { + expect( + imports.some((imp) => imp[key] !== undefined), + `durable ParsedImport.${key}`, + ).toBe(true); + } + const references = [...coldRecords.values()].flatMap((file) => file.referenceSites); + expect(references.some((site) => site.lookupPurpose !== undefined)).toBe(true); + expect(references.some((site) => site.lookupScope !== undefined)).toBe(true); + const dependency = coldRecords + .get('py/metadata.py')! + .referenceSites.find( + (site) => + site.kind === 'call' && site.name === 'dependency' && site.callerScope !== undefined, + ); + expect(dependency).toBeDefined(); + expect(dependency!.inScope).toBe(coldRecords.get('py/metadata.py')!.moduleScope); + expect(dependency!.callerScope).not.toBe(dependency!.inScope); + expect( + coldRecords + .get('py/metadata.py')! + .scopes.find((scope) => scope.id === dependency!.callerScope)?.kind, + ).toBe('Function'); + + const warmCache = await loadParseCache(storageDir); + expect(warmCache.entries.size).toBe(0); + expect(warmCache.onDiskKeys!.size).toBeGreaterThan(0); + let persistedFileCount = 0; + for (const key of warmCache.onDiskKeys!) { + const chunk = await loadParseCacheChunk(warmCache, key); + expect(chunk).toBeDefined(); + persistedFileCount += chunk!.reduce((sum, part) => sum + part.fileCount, 0); + expect(chunk!.flatMap((part) => part.parsedFiles ?? [])).toEqual([]); + } + expect(persistedFileCount).toBe(parsedPaths.length); + fs.rmSync(marker); + const warm = await run(warmCache); + expect(warm.usedWorkerPool).toBe(false); + expect(warm.reparsedFileCount).toBe(0); + expect(warm.parseCacheHitFileCount).toBe(parsedPaths.length); + expect(fs.existsSync(marker)).toBe(false); + expect(extract).not.toHaveBeenCalled(); + assertExactCalls(warm, expected); + expect(graphEdges(warm)).toEqual(graphEdges(cold)); + await persist(warmCache); + expect(await durableRecords()).toEqual(coldRecords); + + // One changed local target plus one loader that ceases to be an import. + writeFixtureRepo(repoDir, { + 'ts/main.ts': files['ts/main.ts']!.replace("require('./a')", "require('./b')"), + 'js/main.js': files['js/main.js']!.replace('allowed()', 'allowed(require)'), + }); + const editedCache = await loadParseCache(storageDir); + const edited = await run(editedCache); + expect(edited.usedWorkerPool).toBe(true); + expect(edited.reparsedFileCount).toBeGreaterThanOrEqual(2); + expect(edited.parseCacheHitFileCount).toBeGreaterThan(0); + expect(edited.reparsedFileCount! + edited.parseCacheHitFileCount!).toBe(parsedPaths.length); + expect(fs.existsSync(marker)).toBe(true); + expect(extract).not.toHaveBeenCalled(); + const changedExpected = expected + .filter((item) => item.file !== 'js/main.js') + .map((item) => (item.file === 'ts/main.ts' ? { ...item, targetFile: 'ts/b.ts' } : item)); + assertExactCalls(edited, changedExpected); + expect(graphEdges(edited)).not.toEqual(graphEdges(cold)); + await persist(editedCache); + const editedRecords = await durableRecords(); + expect(editedRecords.get('js/main.js')!.parsedImports).toEqual([]); + expect(editedRecords.get('ts/main.ts')!.parsedImports.map((imp) => imp.targetRaw)).toEqual([ + './b', + ]); + expect(editedRecords.get('py/main.py')).toEqual(coldRecords.get('py/main.py')); + + fs.rmSync(marker); + const editedWarm = await run(await loadParseCache(storageDir)); + expect(editedWarm.usedWorkerPool).toBe(false); + expect(editedWarm.reparsedFileCount).toBe(0); + expect(editedWarm.parseCacheHitFileCount).toBe(parsedPaths.length); + expect(fs.existsSync(marker)).toBe(false); + expect(extract).not.toHaveBeenCalled(); + assertExactCalls(editedWarm, changedExpected); + expect(graphEdges(editedWarm)).toEqual(graphEdges(edited)); + } finally { + extract.mockRestore(); + } + }, 180_000); +}); diff --git a/gitnexus/test/integration/resolvers/rust-cargo-review-regressions.test.ts b/gitnexus/test/integration/resolvers/rust-cargo-review-regressions.test.ts index 7a56418ec..430e44a84 100644 --- a/gitnexus/test/integration/resolvers/rust-cargo-review-regressions.test.ts +++ b/gitnexus/test/integration/resolvers/rust-cargo-review-regressions.test.ts @@ -153,7 +153,7 @@ describe('Cargo review regressions (#3294)', () => { }, ); - it('keeps uncertainty for an unknown macro that can share both files', async () => { + it('keeps an explicit unresolved import blocked when a macro makes membership unknown', async () => { const { calls } = await check( { 'src/lib.rs': 'use crate::helper; pub fn caller() { helper(); }', @@ -161,10 +161,10 @@ describe('Cargo review regressions (#3294)', () => { 'src/main.rs': 'macro_rules! share { () => { include!("../tests/helper.rs"); #[path="lib.rs"] mod library; } } share!(); fn main() {}', }, - 1, + 0, false, ); - expect(calls[0]?.rel).toMatchObject({ reason: 'global-name-fallback', confidence: 0.5 }); + expect(calls).toEqual([]); }); it('preserves a labeled guess for an unmodeled extern-crate alias, not an alias-resolution claim', async () => { diff --git a/gitnexus/test/integration/resolvers/rust-cargo-target-fallback.test.ts b/gitnexus/test/integration/resolvers/rust-cargo-target-fallback.test.ts index 87d59d441..3b210ba5c 100644 --- a/gitnexus/test/integration/resolvers/rust-cargo-target-fallback.test.ts +++ b/gitnexus/test/integration/resolvers/rust-cargo-target-fallback.test.ts @@ -6,6 +6,109 @@ import { getRelationships, runPipelineFromRepo, writeFixtureRepo } from './helpe import { loadRustCargoTargets } from '../../../src/core/ingestion/languages/rust/cargo-targets.js'; describe('Rust Cargo target boundaries in name fallback (#3253)', () => { + it.each([ + [ + 'renamed symbol', + { + 'src/lib.rs': + 'pub mod source; pub mod unrelated; pub mod wanted { pub use crate::source::actual as helper; }', + 'src/source.rs': 'pub fn actual() {}', + }, + 'demo::wanted::helper', + ], + [ + 're-export cycle', + { + 'src/lib.rs': 'pub mod a; pub mod b; pub mod unrelated;', + 'src/a.rs': 'pub mod wanted { pub use crate::b::wanted::helper; }', + 'src/b.rs': 'pub mod wanted { pub use crate::a::wanted::helper; }', + }, + 'demo::a::wanted::helper', + ], + ] as const)( + 'abstains from an inline %s without exact binding identity', + async (_label, sources, imported) => { + const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'gn-rust-cargo-reexport-abstention-')); + try { + writeFixtureRepo(dir, { + 'Cargo.toml': '[package]\nname="demo"\nedition="2021"\n', + ...sources, + 'src/unrelated.rs': 'pub fn helper() {}', + 'tests/caller.rs': `use ${imported}; pub fn caller() { helper(); }`, + }); + const result = await runPipelineFromRepo(dir, () => {}); + expect( + getRelationships(result, 'CALLS').filter((edge) => edge.source === 'caller'), + ).toEqual([]); + } finally { + fs.rmSync(dir, { recursive: true, force: true, maxRetries: 5, retryDelay: 50 }); + } + }, + ); + + it.each(['other', 'wanted'] as const)( + 'keeps named imports within the explicit re-export module: %s', + async (exportedFrom) => { + const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'gn-rust-cargo-reexport-identity-')); + try { + const reexport = 'pub use crate::source::helper;'; + writeFixtureRepo(dir, { + 'Cargo.toml': '[package]\nname="demo"\nedition="2021"\n', + 'src/lib.rs': `pub mod source; pub mod other { ${exportedFrom === 'other' ? reexport : ''} } pub mod wanted { ${exportedFrom === 'wanted' ? reexport : ''} }`, + 'src/source.rs': 'pub fn helper() {}', + 'tests/caller.rs': 'use demo::wanted::helper; pub fn caller() { helper(); }', + }); + const result = await runPipelineFromRepo(dir, () => {}); + const calls = getRelationships(result, 'CALLS').filter( + (edge) => edge.source === 'caller' && edge.target === 'helper', + ); + expect(calls.map((edge) => edge.targetFilePath)).toEqual( + exportedFrom === 'wanted' ? ['src/source.rs'] : [], + ); + } finally { + fs.rmSync(dir, { recursive: true, force: true, maxRetries: 5, retryDelay: 50 }); + } + }, + ); + + it('does not choose a same-file homonym when a named inline import cannot carry exact identity', async () => { + const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'gn-rust-cargo-inline-homonym-')); + try { + writeFixtureRepo(dir, { + 'Cargo.toml': '[package]\nname="demo"\nedition="2021"\n', + 'src/lib.rs': 'pub fn helper() {} pub mod nested { pub fn helper() {} }', + 'tests/caller.rs': 'use demo::nested::helper; pub fn caller() { helper(); }', + }); + const result = await runPipelineFromRepo(dir, () => {}); + expect( + getRelationships(result, 'CALLS').filter( + (edge) => edge.source === 'caller' && edge.target === 'helper', + ), + ).toEqual([]); + } finally { + fs.rmSync(dir, { recursive: true, force: true, maxRetries: 5, retryDelay: 50 }); + } + }); + + it('does not bypass a local alias with a same-named Cargo library', async () => { + const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'gn-rust-cargo-named-shadow-')); + try { + writeFixtureRepo(dir, { + 'Cargo.toml': '[package]\nname="demo"\nedition="2021"\n', + 'src/lib.rs': 'pub fn helper() {}', + 'tests/caller.rs': 'pub fn caller() { use missing as demo; use demo::helper; helper(); }', + }); + const result = await runPipelineFromRepo(dir, () => {}); + expect( + getRelationships(result, 'CALLS').filter( + (edge) => edge.source === 'caller' && edge.target === 'helper', + ), + ).toEqual([]); + } finally { + fs.rmSync(dir, { recursive: true, force: true, maxRetries: 5, retryDelay: 50 }); + } + }); + it.each([ ['use demo::*;', false], ['use demo::helper;', false], @@ -327,7 +430,7 @@ describe('Rust Cargo target boundaries in name fallback (#3253)', () => { '[package]\nname="unknown"\nversion="0.1.0"\nedition="2021"\n', 'extern crate self as api;', ], - ])('preserves a labeled guess with %s', async (_name, manifest, prefix) => { + ])('does not bypass an explicit unresolved import with %s', async (_name, manifest, prefix) => { const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'gn-rust-cargo-unknown-')); try { writeFixtureRepo(dir, { @@ -340,9 +443,7 @@ describe('Rust Cargo target boundaries in name fallback (#3253)', () => { const calls = getRelationships(result, 'CALLS').filter( (edge) => edge.source === 'caller' && edge.target === 'helper', ); - expect(calls).toHaveLength(1); - expect(calls[0]!.rel.reason).toBe('global-name-fallback'); - expect(calls[0]!.rel.confidence).toBe(0.5); + expect(calls).toEqual([]); } finally { fs.rmSync(dir, { recursive: true, force: true, maxRetries: 5, retryDelay: 50 }); } diff --git a/gitnexus/test/integration/resolvers/rust-import-scope.test.ts b/gitnexus/test/integration/resolvers/rust-import-scope.test.ts index 44b2d39fb..8926c8098 100644 --- a/gitnexus/test/integration/resolvers/rust-import-scope.test.ts +++ b/gitnexus/test/integration/resolvers/rust-import-scope.test.ts @@ -24,6 +24,97 @@ async function callsFor( } describe('Rust import scope through full ingestion', () => { + it('respects a local type import when inferring a destructured receiver', async () => { + const calls = await callsFor( + ` +mod target; +pub fn allowed(source: target::Container) { + use crate::target::Container; + let Container { item } = source; + item.save(); +} +pub fn denied(source: target::Container) { + use crate::missing::Container; + let Container { item } = source; + item.save(); +} +`, + 'pub struct User {}\nimpl User { pub fn save(&self) {} }\npub struct Container { pub item: User }\n', + 'save', + ); + expect(calls.filter((edge) => edge.source === 'denied')).toEqual([]); + expect( + calls + .filter((edge) => edge.source === 'allowed') + .map((edge) => [edge.targetFilePath, edge.targetLabel, edge.target]), + ).toEqual([['src/target.rs', 'Function', 'save']]); + }); + + it('does not fabricate receiver calls from an unresolved local factory or factory parameter', async () => { + const calls = await callsFor( + ` +mod target; +use crate::target::make_user; +pub fn unresolved() { + use crate::missing::make_user; + let user = make_user(); + user.save(); + for entry in make_user() { entry.save(); } +} +pub fn parameter(make_user: F) { + let user = make_user(); + user.save(); + for entry in make_user() { entry.save(); } +} +pub fn allowed() { + for entry in make_user() { entry.save(); } +} +`, + 'pub struct User {}\nimpl User { pub fn save(&self) {} }\npub fn make_user() -> Vec { vec![] }\n', + 'save', + ); + expect(calls.filter((edge) => edge.source === 'unresolved')).toEqual([]); + expect(calls.filter((edge) => edge.source === 'parameter')).toEqual([]); + expect( + calls.filter((edge) => edge.source === 'allowed').map((edge) => edge.targetFilePath), + ).toEqual(['src/target.rs']); + }); + + it('keeps sibling local factories with equal type names attached to their own declarations', async () => { + const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'gn-rust-factory-scope-')); + try { + writeFixtureRepo(dir, { + 'Cargo.toml': + '[package]\nname = "factory-scope-test"\nversion = "0.1.0"\nedition = "2021"\n', + 'src/lib.rs': ` +mod left; +mod right; +pub fn use_left() { use crate::left::make; let item = make(); item.save(); } +pub fn use_right() { use crate::right::make; let item = make(); item.save(); } +pub fn iterate_left() { use crate::left::all; for item in all() { item.save(); } } +pub fn iterate_right() { use crate::right::all; for item in all() { item.save(); } } +`, + 'src/left.rs': + 'pub struct Item {}\nimpl Item { pub fn save(&self) {} }\npub fn make() -> Item { Item {} }\npub fn all() -> Vec { vec![] }\n', + 'src/right.rs': + 'pub struct Item {}\nimpl Item { pub fn save(&self) {} }\npub fn make() -> Item { Item {} }\npub fn all() -> Vec { vec![] }\n', + }); + const result = await runPipelineFromRepo(dir, () => {}); + const calls = getRelationships(result, 'CALLS').filter((edge) => edge.target === 'save'); + for (const side of ['left', 'right']) { + for (const prefix of ['use', 'iterate']) { + expect( + calls + .filter((edge) => edge.source === `${prefix}_${side}`) + .map((edge) => edge.targetFilePath), + ).toEqual([`src/${side}.rs`]); + } + } + } finally { + fs.rmSync(dir, { recursive: true, force: true, maxRetries: 5, retryDelay: 50 }); + } + }); + it('preserves method calls on the result of a function-local imported factory', async () => { const calls = await callsFor( ` diff --git a/gitnexus/test/integration/resolvers/rust.test.ts b/gitnexus/test/integration/resolvers/rust.test.ts index cab984b8f..7ad1b5e77 100644 --- a/gitnexus/test/integration/resolvers/rust.test.ts +++ b/gitnexus/test/integration/resolvers/rust.test.ts @@ -46,15 +46,26 @@ describe('Rust trait implementation resolution', () => { expect(extends_.length).toBe(0); }); - it('resolves exactly 1 IMPORTS edge: main.rs → button.rs', () => { + it('resolves imports through the declared trait and implementation modules', () => { const imports = getRelationships(result, 'IMPORTS'); - expect(imports.length).toBe(1); - expect(imports[0].source).toBe('main.rs'); - expect(imports[0].target).toBe('button.rs'); + expect(imports.map((edge) => `${edge.sourceFilePath} → ${edge.targetFilePath}`).sort()).toEqual( + [ + 'src/impls/button.rs → src/traits/mod.rs', + 'src/main.rs → src/impls/button.rs', + 'src/traits/mod.rs → src/traits/clickable.rs', + 'src/traits/mod.rs → src/traits/drawable.rs', + ], + ); }); - it('detects 2 modules and functions (trait signatures + impls)', () => { - expect(getNodesByLabel(result, 'Module')).toEqual(['impls', 'traits']); + it('detects 5 modules and functions (trait signatures + impls)', () => { + expect(getNodesByLabel(result, 'Module')).toEqual([ + 'button', + 'clickable', + 'drawable', + 'impls', + 'traits', + ]); const fns = getNodesByLabel(result, 'Function'); // With function_signature_item captured, trait abstract methods AND their // concrete impls both appear (distinct qualified IDs, same name) @@ -192,14 +203,16 @@ describe('Rust ambiguous symbol resolution', () => { }); it('import resolves to src/models/mod.rs (not src/other/mod.rs)', () => { - const imports = getRelationships(result, 'IMPORTS'); - const modelsImport = imports.find((e) => e.targetFilePath.includes('models')); - expect(modelsImport).toBeDefined(); - expect(modelsImport!.targetFilePath).toBe('src/models/mod.rs'); + const imports = getRelationships(result, 'IMPORTS').filter( + (edge) => edge.sourceFilePath === 'src/services/mod.rs', + ); + expect(imports.map((edge) => edge.targetFilePath)).toEqual(['src/models/mod.rs']); }); - it('no import edge to src/other/', () => { - const imports = getRelationships(result, 'IMPORTS'); + it('no services import edge to src/other/', () => { + const imports = getRelationships(result, 'IMPORTS').filter( + (edge) => edge.sourceFilePath === 'src/services/mod.rs', + ); for (const imp of imports) { expect(imp.targetFilePath).not.toMatch(/src\/other\//); } diff --git a/gitnexus/test/integration/resolvers/typescript.test.ts b/gitnexus/test/integration/resolvers/typescript.test.ts index df9041fba..78c93c467 100644 --- a/gitnexus/test/integration/resolvers/typescript.test.ts +++ b/gitnexus/test/integration/resolvers/typescript.test.ts @@ -563,20 +563,22 @@ describe('TypeScript export type re-export chain resolution', () => { expect(getNodesByLabel(result, 'Class')).toEqual(['Repo', 'User']); }); - it('resolves new User() through export type re-export chain to base.ts', () => { + it('does not resolve runtime construction through a type-only re-export chain', () => { const calls = getRelationships(result, 'CALLS'); - const userCtor = calls.find((c) => c.target === 'User' && c.targetLabel === 'Class'); - expect(userCtor).toBeDefined(); - expect(userCtor!.source).toBe('main'); - expect(userCtor!.targetFilePath).toBe('src/base.ts'); + expect(calls.filter((c) => c.source === 'main' && c.targetLabel === 'Class')).toEqual([]); }); - it('resolves user.save() through export type re-export chain to base.ts', () => { + it('does not infer receiver types from invalid type-only construction', () => { const calls = getRelationships(result, 'CALLS'); - const saveCall = calls.find((c) => c.target === 'save'); - expect(saveCall).toBeDefined(); - expect(saveCall!.source).toBe('main'); - expect(saveCall!.targetFilePath).toBe('src/base.ts'); + expect(calls.filter((c) => c.source === 'main')).toEqual([]); + }); + + it('resolves annotated parameter receivers through the type-only re-export chain', () => { + const calls = getRelationships(result, 'CALLS').filter((c) => c.source === 'typed'); + expect(calls.map((c) => [c.target, c.targetFilePath]).sort()).toEqual([ + ['persist', 'src/base.ts'], + ['save', 'src/base.ts'], + ]); }); }); @@ -1846,6 +1848,13 @@ describe('TypeScript member access iterable for-loop', () => { ); expect(repoSave).toBeDefined(); }); + + it('does not borrow a parameter type for a nonexistent receiver field', () => { + const calls = getRelationships(result, 'CALLS'); + expect(calls.filter((c) => c.source === 'processMissingUsers' && c.target === 'save')).toEqual( + [], + ); + }); }); // --------------------------------------------------------------------------- diff --git a/gitnexus/test/integration/resolvers/zig-local-imports.test.ts b/gitnexus/test/integration/resolvers/zig-local-imports.test.ts new file mode 100644 index 000000000..f4e6f75d5 --- /dev/null +++ b/gitnexus/test/integration/resolvers/zig-local-imports.test.ts @@ -0,0 +1,109 @@ +import { describe, expect, it } from 'vitest'; +import fs from 'node:fs'; +import os from 'node:os'; +import path from 'node:path'; +import { SupportedLanguages } from '../../../src/config/supported-languages.js'; +import { optionalGrammarGate } from '../../helpers/optional-grammar.js'; +import { getRelationships, runPipelineFromRepo, writeFixtureRepo } from './helpers.js'; + +const zig = optionalGrammarGate(SupportedLanguages.Zig); + +async function callsFor(source: string) { + const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'gn-zig-local-imports-')); + try { + writeFixtureRepo(dir, { + 'main.zig': source, + 'a.zig': `pub fn same() void {} +pub fn allowed() void {} +pub fn forbidden() void {} +pub const Inner = struct { pub fn run() void {} }; +`, + 'b.zig': `pub fn same() void {} +pub const Inner = struct { pub fn run() void {} }; +`, + }); + const result = await runPipelineFromRepo(dir, () => {}); + return getRelationships(result, 'CALLS').filter((edge) => edge.sourceFilePath === 'main.zig'); + } finally { + fs.rmSync(dir, { recursive: true, force: true, maxRetries: 5, retryDelay: 50 }); + } +} + +describe.skipIf(!zig.available)('Zig block-local imports', () => { + it('resolves legal same-name namespace aliases independently in sibling blocks', async () => { + const calls = await callsFor(`pub fn siblings() void { + { const m = @import("a.zig"); m.same(); } + { const m = @import("b.zig"); m.same(); } +} +`); + expect(calls.map((edge) => edge.rel.targetId).sort()).toEqual([ + 'Function:a.zig:same', + 'Function:b.zig:same', + ]); + }); + + it('does not expose a block alias after leaving its block', async () => { + const calls = await callsFor(`pub fn afterBlock() void { + { const m = @import("a.zig"); m.allowed(); } + m.forbidden(); +} +`); + expect(calls.map((edge) => edge.rel.targetId)).toEqual(['Function:a.zig:allowed']); + }); + + it('does not resolve an alias before its local declaration', async () => { + const calls = await callsFor(`pub fn beforeDeclaration() void { + m.forbidden(); + const m = @import("a.zig"); + m.allowed(); +} +`); + expect(calls.map((edge) => edge.rel.targetId)).toEqual(['Function:a.zig:allowed']); + }); + + it('keeps deep aliases within their declaration block', async () => { + const calls = await callsFor(`pub fn deepSiblings() void { + { const m = @import("a.zig"); const chosen = m.Inner.run; chosen(); } + { const m = @import("b.zig"); const chosen = m.Inner.run; chosen(); } +} +`); + expect(calls.map((edge) => edge.rel.targetId).sort()).toEqual([ + 'Method:a.zig:Inner.run#0', + 'Method:b.zig:Inner.run#0', + ]); + }); + + it('does not borrow an unrelated callable for an unresolved named import', async () => { + const calls = await callsFor(`pub fn missingImport() void { + const forbidden = @import("missing.zig").forbidden; + forbidden(); +} +`); + expect(calls).toEqual([]); + }); + + it('resolves a named member alias through the enclosing block import', async () => { + const calls = await callsFor(`pub fn namedAlias() void { + { const m = @import("a.zig"); const local = m.allowed; local(); } +} +`); + expect(calls.map((edge) => edge.rel.targetId)).toEqual(['Function:a.zig:allowed']); + }); + + it('retains repeated inline receivers in separate functions', async () => { + const calls = await callsFor(`pub fn first() void { @import("a.zig").allowed(); } +pub fn second() void { @import("a.zig").allowed(); } +`); + expect(calls.map((edge) => [edge.source, edge.rel.targetId]).sort()).toEqual([ + ['first', 'Function:a.zig:allowed'], + ['second', 'Function:a.zig:allowed'], + ]); + }); + + it('preserves declaration-order independent module imports', async () => { + const calls = await callsFor(`pub fn moduleImport() void { m.allowed(); } +const m = @import("a.zig"); +`); + expect(calls.map((edge) => edge.rel.targetId)).toEqual(['Function:a.zig:allowed']); + }); +}); diff --git a/gitnexus/test/unit/cfg/vue-visitor.test.ts b/gitnexus/test/unit/cfg/vue-visitor.test.ts index 9301b7f96..4af263b0f 100644 --- a/gitnexus/test/unit/cfg/vue-visitor.test.ts +++ b/gitnexus/test/unit/cfg/vue-visitor.test.ts @@ -40,7 +40,7 @@ describe('Vue CfgVisitor reuse — SFC ` : source; + +describe.each(variants)('$extension local loader extraction', ({ extension, provider }) => { + function extract(source: string) { + return extractParsedFile(provider, wrap(source, extension), `app.${extension}`)!; + } + + it('keeps module and function ownership when their source ranges coincide', () => { + const parsed = extract('function sole(require) { const ns = require("./blocked"); }'); + const module = parsed.scopes.find((scope) => scope.kind === 'Module')!; + const functionScope = parsed.scopes.find((scope) => scope.kind === 'Function')!; + expect(module.nameClaims?.some((claim) => claim.name === 'sole')).toBe(true); + expect(module.nameClaims?.some((claim) => claim.name === 'require')).toBe(false); + expect(functionScope.nameClaims?.some((claim) => claim.name === 'require')).toBe(true); + expect(parsed.parsedImports).toEqual([]); + }); + + it('retains namespace, named, alias and immediately awaited bindings in their function', () => { + const parsed = extract(` + async function owner() { + const ns = require('./a'); + const { run, other: renamed } = require('./b'); + const lazy = await import('./c'); + return ns.run() + run() + renamed() + lazy.run(); + } + `); + const imports = parsed.parsedImports.filter((imp) => 'localName' in imp && imp.localName); + expect(imports.map((imp) => ('localName' in imp ? imp.localName : '')).sort()).toEqual([ + 'lazy', + 'ns', + 'renamed', + 'run', + ]); + for (const imp of imports) { + expect(parsed.scopes.find((scope) => scope.id === imp.declaredAtScope)?.kind).toBe( + 'Function', + ); + } + }); + + it('recognizes the loader binding, including destructured parameters and catch names', () => { + const parsed = extract(` + function parameter(require) { const nope = require('./parameter'); } + function destructured({ require }) { const nope = require('./destructured'); } + function local() { const nope = require('./tdz'); const require = factory; } + function written(require) { require = factory; const nope = require('./written'); } + try { fail(); } catch (require) { const nope = require('./catch'); } + const yes = require('./genuine'); + `); + expect(parsed.parsedImports.map((imp) => imp.targetRaw)).toEqual(['./genuine']); + }); + + it('does not claim namespaces for promises, computed loads or unsupported destructuring', () => { + const parsed = extract(` + const promised = import('./promise'); + const computed = require(specifier); + const { run = fallback, ...rest } = require('./unsupported'); + `); + expect(parsed.parsedImports.filter((imp) => 'localName' in imp && imp.localName)).toEqual([]); + expect(parsed.parsedImports.some((imp) => imp.targetRaw === './promise')).toBe(true); + }); + + it('hoists var imports to the function while keeping lexical imports in the block', () => { + const parsed = extract(`function owner() { + { let marker = 1; var functionLocal = require('./a'); const blockLocal = require('./b'); } + }`); + const imports = parsed.parsedImports.filter((imp) => 'localName' in imp && imp.localName); + expect( + imports + .map((imp) => [ + 'localName' in imp ? imp.localName : '', + parsed.scopes.find((scope) => scope.id === imp.declaredAtScope)?.kind, + ]) + .sort(), + ).toEqual([ + ['blockLocal', 'Block'], + ['functionLocal', 'Function'], + ]); + }); + + it('rejects a reassigned root loader and forwarding exports from a shadowed loader', () => { + const reassigned = extract(`require = custom; const ns = require('./blocked');`); + expect(reassigned.parsedImports).toEqual([]); + const forwarded = extract(`const require = custom; + const ns = require('./blocked'); exports.forwarded = ns.run;`); + expect(forwarded.parsedImports).toEqual([]); + }); + + it('treats a named class expression as a local loader shadow without hiding the real loader', () => { + const parsed = extract(`const Holder = class require { + method() { const ns = require('./shadowed'); return ns.run(); } + }; const real = require('./genuine');`); + expect(parsed.parsedImports.map((imp) => imp.targetRaw)).toEqual(['./genuine']); + }); + + it('does not forward a reassigned CommonJS namespace or named handle', () => { + const parsed = extract(`let ns = require('./a'); ns = custom; exports.forwarded = ns.run; + let { run } = require('./b'); run = custom; exports.named = run;`); + expect(parsed.parsedImports.filter((imp) => imp.kind === 'reexport')).toEqual([]); + expect( + parsed.parsedImports.filter((imp) => imp.kind !== 'reexport').map((imp) => imp.targetRaw), + ).toEqual(['./a', './b']); + }); + + it('keeps unchanged destructured CommonJS exports when sibling handles are reassigned', () => { + const parsed = extract(` + let { changed, stable } = require('./a'); + changed = custom; + exports.changed = changed; + exports.stable = stable; + let { changed: renamedChanged, stable: renamedStable } = require('./b'); + renamedChanged = custom; + exports.renamedChanged = renamedChanged; + exports.renamedStable = renamedStable; + `); + expect(parsed.parsedImports.filter((imp) => imp.kind === 'reexport')).toEqual([ + expect.objectContaining({ targetRaw: './a', importedName: 'stable', localName: 'stable' }), + expect.objectContaining({ + targetRaw: './b', + importedName: 'stable', + localName: 'renamedStable', + }), + ]); + }); + + it('limits reassignment barriers to imported handles, preserving ordinary local values', () => { + const parsed = extract(`function owner(parameter) { + let local = new Service(); + var hoisted = new Service(); + local = new Service(); + hoisted = new Service(); + parameter = new Service(); + let imported = require('./a'); + imported = custom; + }`); + expect( + parsed.scopes + .flatMap((scope) => scope.nameClaims ?? []) + .filter((claim) => claim.kind === 'blocked') + .map((claim) => claim.name), + ).toEqual(['imported']); + }); +}); + +it('recognizes TypeScript import-equals as a module namespace binding', () => { + const parsed = extractParsedFile( + typescriptProvider, + `import tools = require('./tools'); tools.run();`, + 'app.ts', + )!; + expect(parsed.parsedImports).toEqual([ + expect.objectContaining({ + kind: 'namespace', + localName: 'tools', + targetRaw: './tools', + }), + ]); + const owner = parsed.scopes.find((scope) => scope.id === parsed.parsedImports[0].declaredAtScope); + expect(owner?.kind).toBe('Module'); +}); + +it('retains erasure on a type-only import-equals declaration', () => { + const parsed = extractParsedFile( + typescriptProvider, + 'import type tools = require("./tools");', + 'app.ts', + )!; + expect(parsed.parsedImports).toEqual([ + expect.objectContaining({ + kind: 'namespace', + localName: 'tools', + targetRaw: './tools', + typeOnly: true, + }), + ]); +}); + +it('uses the same local-loader facts for full and pre-extracted Vue script', () => { + const source = `function caller() { const ns = require('./target'); ns.run(); }`; + const full = emitVueScopeCaptures(``, 'App.vue'); + const worker = emitVueScopeCaptures(source, 'App.vue', undefined, { + sourceKind: 'pre-extracted-script', + }); + const imports = (captures: typeof full) => + captures + .filter((match) => match['@import.statement']) + .map((match) => [ + match['@import.kind']?.text, + match['@import.alias']?.text, + match['@import.source']?.text, + ]); + expect(imports(full)).toEqual([['namespace', 'ns', './target']]); + expect(imports(worker)).toEqual(imports(full)); +}); + +it('preserves lexical positions while giving full and pre-extracted Vue declarations equal graph positions', () => { + const declaration = + 'function outer() { function inner() { return ns.run(); } const ns = require("./target"); return inner(); }'; + const source = `\n`; + const extracted = extractVueScript(source)!; + const full = extractParsedFile(vueProvider, source, 'App.vue')!; + const worker = extractParsedFile( + vueProvider, + extracted.scriptContent, + 'App.vue', + undefined, + undefined, + 'pre-extracted-script', + undefined, + extracted.lineOffset, + )!; + expect(worker.localDefs).toEqual(full.localDefs); + const inner = full.localDefs.find((def) => def.qualifiedName === 'inner')!; + const innerColumn = declaration.indexOf('function inner'); + expect(inner.nodeId).toBe(`def:App.vue#2:${innerColumn}:Function:inner`); + expect(inner.graphPosition).toEqual({ + startLine: 2 + extracted.lineOffset, + startCol: innerColumn, + }); + expect( + full.scopes.find((scope) => scope.ownedDefs.some((def) => def.nodeId === inner.nodeId))?.range + .startLine, + ).toBe(2); +}); + +describe('local-loader graph targets', () => { + let repo: string; + let result: PipelineResult; + beforeAll(async () => { + repo = fs.mkdtempSync(path.join(os.tmpdir(), 'gitnexus-local-loaders-')); + const files: Record = {}; + for (const { extension } of variants) { + const targetExtension = + extension === 'vue' || extension === 'tsx' ? 'ts' : extension === 'jsx' ? 'js' : extension; + files[`${extension}/a.${targetExtension}`] = 'export function run() { return 1; }'; + files[`${extension}/b.${targetExtension}`] = 'export function run() { return 2; }'; + files[`${extension}/app.${extension}`] = wrap( + ` + import { run } from './a'; + import * as root from './a'; + class LocalService { work() { return 1; } } + ${ + extension === 'js' || extension === 'jsx' + ? '/** @type {LocalService} */ let moduleClient;' + : 'let moduleClient: LocalService;' + } + export function assignModuleClient() { moduleClient = new LocalService(); } + export function readModuleClient() { return moduleClient.work(); } + export function assignedLet() { + let service = new LocalService(); service = new LocalService(); service.work(); + } + export function assignedVar() { + var service = new LocalService(); service = new LocalService(); service.work(); + } + ${ + extension === 'ts' || extension === 'tsx' || extension === 'vue' + ? ` + export function assignedTyped() { + let service: LocalService; service = new LocalService(); service.work(); + } + export function assignedParameter(service: LocalService) { + service = new LocalService(); service.work(); + }` + : '' + } + function replacedLoader() {} + var replacedLoader = require('./a'); + replacedLoader = factory; + export function replacedDeclaration() { return replacedLoader.run(); } + export function first() { const ns = require('./a'); return ns.run(); } + export function second() { const ns = require('./b'); return ns.run(); } + export function named() { const { run: chosen } = require('./b'); return chosen(); } + export async function lazy() { const { run: chosen } = await import('./a'); return chosen(); } + export function varAfter() { { let marker = 1; var ns = require('./a'); } return ns.run(); } + export function varRedeclared() { var ns = require('./a'); var ns; return ns.run(); } + export function varDeclaredBefore() { var ns; var ns = require('./b'); return ns.run(); } + export function functionRedeclared() { function retained() {} var retained; retained(); } + export function varOnly() { run(); var run; } + export function closure() { function inner() { return ns.run(); } const ns = require('./a'); return inner(); } + export function unbound() { return ns.run(); } + export function parameter(ns) { return ns.run(); } + export function parameterCall(run) { return run(); } + export function destructured({ run }) { return run(); } + export function outsideBlock() { { const ns = require('./a'); } return ns.run(); } + export function unresolved() { const root = require('./missing'); return root.run(); } + export function mutated() { let ns = require('./a'); ns = factory; return ns.run(); } + export function conditional() { if (flag) { var ns = require('./a'); } return ns.run(); } + export function before() { ns.run(); const ns = require('./a'); } + export function blocked(require) { const ns = require('./a'); return ns.run(); } + `, + extension, + ); + if (extension === 'ts' || extension === 'js') { + files[`${extension}/forward.${extension}`] = ` + let { run: changed, stable } = require('./exports'); + changed = custom; + exports.changed = changed; + exports.stable = stable; + let { run: renamedChanged, stable: renamedStable } = require('./exports'); + renamedChanged = custom; + exports.renamedChanged = renamedChanged; + exports.renamedStable = renamedStable; + `; + files[`${extension}/exports.${extension}`] = ` + exports.run = function run() { return 1; }; + exports.stable = function stable() { return 2; }; + `; + files[`${extension}/consumer.${extension}`] = ` + const { changed, stable, renamedChanged, renamedStable } = require('./forward'); + export function callChanged() { return changed(); } + export function callStable() { return stable(); } + export function callRenamedChanged() { return renamedChanged(); } + export function callRenamedStable() { return renamedStable(); } + `; + } + } + writeFixtureRepo(repo, files); + result = await runPipelineFromRepo(repo, () => {}, { workerPoolSize: 1 }); + }, 120_000); + afterAll(() => { + if (repo) fs.rmSync(repo, { recursive: true, force: true }); + }); + + it.each(variants)( + '$extension retains method calls after ordinary receiver assignment', + ({ extension }) => { + expect( + getRelationships(result, 'CALLS') + .filter( + (edge) => + edge.sourceFilePath === `${extension}/app.${extension}` && edge.target === 'work', + ) + .map((edge) => [edge.source, edge.targetFilePath]) + .sort(), + ).toEqual( + [ + 'assignedLet', + 'assignedVar', + 'readModuleClient', + ...(extension === 'ts' || extension === 'tsx' || extension === 'vue' + ? ['assignedTyped', 'assignedParameter'] + : []), + ] + .sort() + .map((name) => [name, `${extension}/app.${extension}`]), + ); + }, + ); + + it.each(['ts', 'js'])( + '%s forwards unchanged destructured siblings to their exact definition', + (extension) => { + expect( + getRelationships(result, 'CALLS') + .filter((edge) => edge.sourceFilePath === `${extension}/consumer.${extension}`) + .map((edge) => [edge.source, edge.target, edge.targetFilePath]) + .sort(), + ).toEqual([ + ['callRenamedStable', 'stable', `${extension}/exports.${extension}`], + ['callStable', 'stable', `${extension}/exports.${extension}`], + ]); + }, + ); + + it.each(variants)( + '$extension binds exact targets and excludes sibling, parameter and TDZ leakage', + ({ extension }) => { + const calls = getRelationships(result, 'CALLS') + .filter( + (edge) => + edge.sourceFilePath === `${extension}/app.${extension}` && edge.target === 'run', + ) + .map((edge) => `${edge.source} -> ${edge.targetFilePath}`) + .sort(); + const targetExtension = + extension === 'vue' || extension === 'tsx' ? 'ts' : extension === 'jsx' ? 'js' : extension; + expect(calls).toEqual([ + `first -> ${extension}/a.${targetExtension}`, + `inner -> ${extension}/a.${targetExtension}`, + `lazy -> ${extension}/a.${targetExtension}`, + `named -> ${extension}/b.${targetExtension}`, + `second -> ${extension}/b.${targetExtension}`, + `varAfter -> ${extension}/a.${targetExtension}`, + `varDeclaredBefore -> ${extension}/b.${targetExtension}`, + `varRedeclared -> ${extension}/a.${targetExtension}`, + ]); + expect( + getRelationships(result, 'CALLS') + .filter( + (edge) => + edge.sourceFilePath === `${extension}/app.${extension}` && + edge.source === 'functionRedeclared', + ) + .map((edge) => [edge.target, edge.targetFilePath]), + ).toEqual([['retained', `${extension}/app.${extension}`]]); + }, + ); +}); diff --git a/gitnexus/test/unit/scope-resolution/vue/embedded-scripts.test.ts b/gitnexus/test/unit/scope-resolution/vue/embedded-scripts.test.ts new file mode 100644 index 000000000..50db6e03b --- /dev/null +++ b/gitnexus/test/unit/scope-resolution/vue/embedded-scripts.test.ts @@ -0,0 +1,163 @@ +import { afterAll, beforeAll, describe, expect, it } from 'vitest'; +import fs from 'node:fs'; +import os from 'node:os'; +import path from 'node:path'; +import { vueProvider } from '../../../../src/core/ingestion/languages/vue.js'; +import { getTsParser } from '../../../../src/core/ingestion/languages/typescript/query.js'; +import { extractParsedFile } from '../../../../src/core/ingestion/scope-extractor-bridge.js'; +import { extractVueScript } from '../../../../src/core/ingestion/vue-sfc-extractor.js'; +import { + getRelationships, + runPipelineFromRepo, + writeFixtureRepo, + type PipelineResult, +} from '../../../integration/resolvers/helpers.js'; + +const sources = Object.fromEntries( + ['jsx', 'tsx', 'ts'].map((lang) => [ + lang, + ` + +`, + ]), +); + +describe('Vue embedded script grammar and source positions', () => { + let repo: string; + let result: PipelineResult; + beforeAll(async () => { + repo = fs.mkdtempSync(path.join(os.tmpdir(), 'gitnexus-vue-embedded-')); + writeFixtureRepo(repo, { + 'target.ts': + 'export function run() { return 1; }\nexport function Widget(value) { return value; }', + ...Object.fromEntries( + Object.entries(sources).map(([lang, source]) => [`${lang}.vue`, source]), + ), + }); + result = await runPipelineFromRepo(repo, () => {}, { + workerPoolSize: 1, + skipGraphPhases: true, + pdg: true, + }); + }, 120_000); + afterAll(() => fs.rmSync(repo, { recursive: true, force: true })); + + it.each(['jsx', 'tsx', 'ts'])( + 'maps reordered %s script declarations to original source rows in both paths', + (lang) => { + const source = sources[lang]; + const extracted = extractVueScript(source)!; + expect(extracted.lang).toBe(lang); + expect(extracted.sourceLineMap).toHaveLength(extracted.scriptContent.split('\n').length); + const full = extractParsedFile(vueProvider, source, `${lang}.vue`)!; + const worker = extractParsedFile( + vueProvider, + extracted.scriptContent, + `${lang}.vue`, + undefined, + undefined, + 'pre-extracted-script', + undefined, + extracted.lineOffset, + extracted.lang, + extracted.sourceLineMap, + )!; + expect(worker.localDefs).toEqual(full.localDefs); + for (const name of ['setupCaller', 'normalCaller']) { + const originalRow = source + .split('\n') + .findIndex((line) => line.startsWith(`function ${name}`)); + const definition = full.localDefs.find((def) => def.qualifiedName === name)!; + expect(definition.graphPosition).toEqual({ startLine: originalRow + 1, startCol: 0 }); + const graphNode = result.graph.getNode(`Function:${lang}.vue:${name}`)!; + expect(graphNode.properties.startLine).toBe(originalRow); + const blocks = [...result.graph.iterNodes()].filter( + (node) => + node.label === 'BasicBlock' && + node.properties.filePath === `${lang}.vue` && + node.properties.text?.includes('ns.run()'), + ); + expect(blocks.some((block) => block.properties.startLine === originalRow + 1)).toBe(true); + expect( + blocks.find((block) => block.properties.startLine === originalRow + 1)?.properties + .calleeIds, + ).toBe('Function:target.ts:run'); + expect( + getRelationships(result, 'CALLS') + .filter( + (edge) => + edge.sourceFilePath === `${lang}.vue` && + edge.source === name && + edge.target === 'run', + ) + .map((edge) => edge.rel.targetId), + ).toEqual(['Function:target.ts:run']); + } + }, + ); + + it.each(['jsx', 'tsx'])( + 'retains %s component calls through direct extraction and real workers', + (lang) => { + const parsed = extractParsedFile(vueProvider, sources[lang], `${lang}.vue`)!; + expect( + parsed.referenceSites.some((site) => site.name === 'Widget' && site.kind === 'call'), + ).toBe(true); + expect( + getRelationships(result, 'CALLS') + .filter( + (edge) => + edge.sourceFilePath === `${lang}.vue` && + edge.source === 'render' && + edge.target === 'Widget', + ) + .map((edge) => edge.rel.targetId), + ).toEqual(['Function:target.ts:Widget']); + }, + ); + + it('reparses a cached TypeScript tree for a TSX embedded script', () => { + const extracted = extractVueScript(sources.tsx)!; + const wrongTree = getTsParser('wrong.ts').parse(extracted.scriptContent); + const parsed = extractParsedFile( + vueProvider, + extracted.scriptContent, + 'tsx.vue', + undefined, + wrongTree, + 'pre-extracted-script', + undefined, + extracted.lineOffset, + extracted.lang, + extracted.sourceLineMap, + )!; + expect( + parsed.referenceSites.some((site) => site.name === 'Widget' && site.kind === 'call'), + ).toBe(true); + }); + + it('selects TSX when TypeScript and JSX blocks are combined', () => { + expect( + extractVueScript( + '', + )!.lang, + ).toBe('tsx'); + }); + + it('rejects a native TypeScript tree when extracting JSX scope captures', () => { + const extracted = extractVueScript(sources.jsx)!; + const wrongTree = getTsParser('wrong.ts').parse(extracted.scriptContent); + const parsed = extractParsedFile(vueProvider, sources.jsx, 'jsx.vue', undefined, wrongTree)!; + expect( + parsed.referenceSites.some((site) => site.name === 'Widget' && site.kind === 'call'), + ).toBe(true); + }); +}); diff --git a/gitnexus/test/unit/taint/disk-scope.test.ts b/gitnexus/test/unit/taint/disk-scope.test.ts new file mode 100644 index 000000000..b220cc28e --- /dev/null +++ b/gitnexus/test/unit/taint/disk-scope.test.ts @@ -0,0 +1,103 @@ +import { describe, expect, it } from 'vitest'; +import { readdirSync } from 'node:fs'; +import type { ParsedFile } from 'gitnexus-shared'; +import { createKnowledgeGraph } from '../../../src/core/graph/graph.js'; +import { typescriptProvider } from '../../../src/core/ingestion/languages/typescript.js'; +import { typescriptScopeResolver } from '../../../src/core/ingestion/languages/typescript/scope-resolver.js'; +import { createSemanticModel } from '../../../src/core/ingestion/model/semantic-model.js'; +import { extractParsedFile } from '../../../src/core/ingestion/scope-extractor-bridge.js'; +import { runScopeResolution } from '../../../src/core/ingestion/scope-resolution/pipeline/run.js'; +import { getScopeIndexStoreDir } from '../../../src/storage/scope-index-store.js'; +import { createTempDirPool } from '../../helpers/temp-dir-pool.js'; +import { cfgsOf } from '../../helpers/ts-cfg-harness.js'; + +const stores = createTempDirPool('gn-taint-disk-scope-'); +const code = `import { exec } from 'child_process'; +function command(req, value) { + exec(req.body); exec(value); + eval(req.body); eval(value); +} +function shadowed(req, value, eval, exec) { + exec(req.body); exec(value); + eval(req.body); eval(value); +} +function localCommand(req, value) { + const { exec: run } = require('child_process'); + run(req.body); run(value); +} +function unrelated(req, value) { + run(req.body); run(value); +}`; + +function analyze(disk: boolean) { + const previous = process.env.GITNEXUS_DISK_SCOPE_INDEX; + process.env.GITNEXUS_DISK_SCOPE_INDEX = disk ? '1' : '0'; + try { + const parsed = extractParsedFile(typescriptProvider, code, 'fixture.ts'); + if (parsed === undefined) throw new Error('fixture extraction failed'); + const cfgs = cfgsOf(code); + const preExtracted = new Map([ + [parsed.filePath, { ...parsed, cfgSideChannel: cfgs }], + ]); + const graph = createKnowledgeGraph(); + const model = createSemanticModel(); + const storePath = stores.dir(); + const stats = runScopeResolution( + { + graph, + model, + files: [{ path: parsed.filePath, content: code }], + pdg: true, + preExtractedParsedFiles: preExtracted, + scopeIndexStorePath: storePath, + prebuiltFunctionNodeIndex: new Map([ + [ + parsed.filePath, + new Map(cfgs.map((cfg, i) => [cfg.functionStartLine - 1, [`function:${i}`]])), + ], + ]), + }, + typescriptScopeResolver, + ); + if (disk) { + expect(readdirSync(getScopeIndexStoreDir(storePath)).length).toBeGreaterThan(0); + // The real seal ran and released the caller's resident ParsedFiles. + expect(preExtracted.size).toBe(0); + } + return { + findings: [...graph.iterRelationships()] + .filter((edge) => edge.type === 'TAINTED') + .sort((a, b) => a.id.localeCompare(b.id)), + summaries: stats.functionSummaries, + }; + } finally { + if (previous === undefined) delete process.env.GITNEXUS_DISK_SCOPE_INDEX; + else process.env.GITNEXUS_DISK_SCOPE_INDEX = previous; + } +} + +describe('taint emission after the disk scope seal', () => { + it('preserves exact findings and summaries without losing lexical import/global barriers', () => { + const resident = analyze(false); + const sealed = analyze(true); + // The module import, true global, and function-local import each contribute + // one finding. Shadowed names and the sibling's local import contribute none. + expect(resident.findings).toHaveLength(3); + expect(sealed.findings).toEqual(resident.findings); + expect(sealed.summaries).toEqual(resident.summaries); + const sinks = (functionIndex: number) => + sealed.summaries.find((summary) => summary.fnId === `function:${functionIndex}`)?.paramToSink; + expect(sinks(0)).toEqual([ + { param: 0, sinkKind: 'code-injection' }, + { param: 0, sinkKind: 'command-injection' }, + { param: 1, sinkKind: 'code-injection' }, + { param: 1, sinkKind: 'command-injection' }, + ]); + expect(sinks(1)).toEqual([]); + expect(sinks(2)).toEqual([ + { param: 0, sinkKind: 'command-injection' }, + { param: 1, sinkKind: 'command-injection' }, + ]); + expect(sinks(3)).toEqual([]); + }); +}); diff --git a/gitnexus/test/unit/taint/lexical-imports.test.ts b/gitnexus/test/unit/taint/lexical-imports.test.ts new file mode 100644 index 000000000..fdd6c61a8 --- /dev/null +++ b/gitnexus/test/unit/taint/lexical-imports.test.ts @@ -0,0 +1,284 @@ +import Python from 'tree-sitter-python'; +import { describe, expect, it } from 'vitest'; +import { emitFileCfgs } from '../../../src/core/ingestion/cfg/emit.js'; +import { createKnowledgeGraph } from '../../../src/core/graph/graph.js'; +import { emitFileTaint } from '../../../src/core/ingestion/taint/emit.js'; +import { harvestFileSummaries } from '../../../src/core/ingestion/taint/summary-harvest-driver.js'; +import { createPythonCfgVisitor } from '../../../src/core/ingestion/cfg/visitors/python.js'; +import { pythonProvider } from '../../../src/core/ingestion/languages/python.js'; +import { typescriptProvider } from '../../../src/core/ingestion/languages/typescript.js'; +import { extractParsedFile } from '../../../src/core/ingestion/scope-extractor-bridge.js'; +import { + buildTaintImportIndex, + matchFunctionSites, +} from '../../../src/core/ingestion/taint/match.js'; +import { PYTHON_TAINT_MODEL } from '../../../src/core/ingestion/taint/python-model.js'; +import { TS_JS_TAINT_MODEL } from '../../../src/core/ingestion/taint/typescript-model.js'; +import { hasTaintSafeSites } from '../../../src/core/ingestion/taint/site-safety.js'; +import { makeCfgHarness } from '../../helpers/cfg-harness.js'; +import { cfgsOf } from '../../helpers/ts-cfg-harness.js'; + +const python = makeCfgHarness(Python, createPythonCfgVisitor(), 'fixture.py'); +// A fixture-only sanitizer pins import ownership without changing the built-in model. +const pythonModel = { + ...PYTHON_TAINT_MODEL, + sanitizers: [{ module: 'shlex', name: 'quote', neutralizes: ['command-injection'] as const }], +}; + +function counts(code: string, language: 'python' | 'typescript') { + const isPython = language === 'python'; + const parsed = extractParsedFile( + isPython ? pythonProvider : typescriptProvider, + code, + isPython ? 'fixture.py' : 'fixture.ts', + ); + expect(parsed).toBeDefined(); + const index = buildTaintImportIndex(parsed!.parsedImports, parsed!); + const cfgs = isPython ? python.cfgsOf(code) : cfgsOf(code); + return cfgs.map((cfg) => { + expect(hasTaintSafeSites(cfg)).toBe(true); + const matches = matchFunctionSites(cfg, isPython ? pythonModel : TS_JS_TAINT_MODEL, index); + return { + sinks: matches.statements.flatMap((s) => s.sinks).length, + sanitizers: matches.statements.flatMap((s) => s.sanitizers).length, + }; + }); +} + +describe('taint matches lexical import provenance', () => { + it('preserves true global sinks when no lexical declaration owns the name', () => { + expect(counts('def command(value):\n eval(value)\n', 'python')).toEqual([ + { sinks: 1, sanitizers: 0 }, + ]); + expect(counts('function command(value) { eval(value); }', 'typescript')).toEqual([ + { sinks: 1, sanitizers: 0 }, + ]); + }); + + it('does not select a sanitizer from uncertain conditional imports', () => { + expect( + counts( + `def quoted(flag, value): + if flag: + from shlex import quote + else: + from harmless import quote + return quote(value) +`, + 'python', + ), + ).toEqual([{ sinks: 0, sanitizers: 0 }]); + }); + + it('keeps Python sibling imports separate even when an external module is unresolved', () => { + expect( + counts( + `def command(value): + from os import system as run + run(value) +def unrelated(value): + from harmless import system as run + run(value) +`, + 'python', + ), + ).toEqual([ + { sinks: 1, sanitizers: 0 }, + { sinks: 0, sanitizers: 0 }, + ]); + }); + + it('does not expose a Python function import to a sibling', () => { + expect( + counts( + `def command(value): + from os import system + system(value) +def unrelated(value): + system(value) +`, + 'python', + ), + ).toEqual([ + { sinks: 1, sanitizers: 0 }, + { sinks: 0, sanitizers: 0 }, + ]); + }); + + it('uses columns to distinguish Python calls before and after an import on one line', () => { + expect( + counts( + `def command(value): + system(value); from os import system; system(value) +`, + 'python', + ), + ).toEqual([{ sinks: 1, sanitizers: 0 }]); + }); + + it('blocks a global Python sink shadowed by a module declaration', () => { + expect( + counts( + `eval = custom +def run(value): + eval(value) +`, + 'python', + ), + ).toEqual([{ sinks: 0, sanitizers: 0 }]); + }); + + it('does not borrow a sanitizer from a different Python function', () => { + expect( + counts( + `def quoted(value): + from shlex import quote + return quote(value) +def unrelated(value): + return quote(value) +`, + 'python', + ), + ).toEqual([ + { sinks: 0, sanitizers: 1 }, + { sinks: 0, sanitizers: 0 }, + ]); + }); + + it('resolves module-level CommonJS aliases and renamed destructures', () => { + expect( + counts( + `const cp = require('node:child_process'); +const { exec: run } = require('child_process'); +function command(value) { cp.exec(value); run(value); } +`, + 'typescript', + ), + ).toEqual([{ sinks: 2, sanitizers: 0 }]); + }); + + it('rejects a require call whose loader is a parameter', () => { + expect( + counts( + `function command(require, value) { + const cp = require('child_process'); cp.exec(value); +}`, + 'typescript', + ), + ).toEqual([{ sinks: 0, sanitizers: 0 }]); + }); + + it('does not expose a local CommonJS import to a sibling', () => { + expect( + counts( + `function command(value) { const cp = require('child_process'); cp.exec(value); } +function unrelated(value) { cp.exec(value); }`, + 'typescript', + ), + ).toEqual([ + { sinks: 1, sanitizers: 0 }, + { sinks: 0, sanitizers: 0 }, + ]); + }); + + it('keeps a block shadow local and preserves the outer import after the block', () => { + expect( + counts( + `import { exec } from 'child_process'; +function command(value) { { const exec = custom; exec(value); } exec(value); }`, + 'typescript', + ), + ).toEqual([{ sinks: 1, sanitizers: 0 }]); + }); + + it('does not treat a type-only import as a runtime sink', () => { + expect( + counts( + `import type { exec } from 'child_process'; +function command(value) { exec(value); }`, + 'typescript', + ), + ).toEqual([{ sinks: 0, sanitizers: 0 }]); + }); +}); + +describe('lexical taint integration and incomplete site facts', () => { + it('abstains from module/global matching when a call has no exact position', () => { + const code = `import { exec } from 'child_process'; +function command(value) { exec(value); eval(value); }`; + const parsed = extractParsedFile(typescriptProvider, code, 'fixture.ts')!; + const [harvested] = cfgsOf(code); + const cfg = { + ...harvested, + blocks: harvested.blocks.map((block) => ({ + ...block, + statements: block.statements?.map((statement) => ({ + ...statement, + sites: statement.sites?.map((site) => ({ ...site, at: undefined })), + })), + })), + }; + expect(hasTaintSafeSites(cfg)).toBe(true); + const matches = matchFunctionSites( + cfg, + TS_JS_TAINT_MODEL, + buildTaintImportIndex(parsed.parsedImports, parsed), + ); + expect(matches.hasSink).toBe(false); + }); + + it('carries lexical provenance through emission and function summaries', () => { + const code = `function command(req, value) { + const { exec } = require('child_process'); + exec(req.body); exec(value); +} +function unrelated(req, value) { + exec(req.body); exec(value); +}`; + const parsed = extractParsedFile(typescriptProvider, code, 'fixture.ts')!; + const cfgs = cfgsOf(code); + const graph = createKnowledgeGraph(); + emitFileCfgs(graph, cfgs); + const emitted = emitFileTaint( + graph, + cfgs, + parsed.parsedImports, + TS_JS_TAINT_MODEL, + undefined, + undefined, + undefined, + parsed, + ); + expect(emitted.functionsAnalyzed).toBe(1); + expect(emitted.functionsSkippedNoMatch).toBe(1); + expect(emitted.findingsEmitted).toBe(1); + expect([...graph.iterRelationships()].filter((edge) => edge.type === 'TAINTED')).toHaveLength( + 1, + ); + + // Give the driver unique graph anchors for the real harvested functions. + const fnIndex = new Map([ + [ + parsed.filePath, + new Map(cfgs.map((cfg, i) => [cfg.functionStartLine - 1, [`function:${i}`]] as const)), + ], + ]); + const harvested = harvestFileSummaries( + fnIndex, + cfgs, + parsed.parsedImports, + TS_JS_TAINT_MODEL, + undefined, + undefined, + parsed, + ); + expect(harvested.unresolved).toBe(0); + expect(harvested.gaps).toBe(0); + expect( + harvested.summaries.find((summary) => summary.fnId === 'function:0')?.paramToSink, + ).toContainEqual({ param: 1, sinkKind: 'command-injection' }); + expect( + harvested.summaries.find((summary) => summary.fnId === 'function:1')?.paramToSink, + ).toEqual([]); + }); +}); diff --git a/gitnexus/test/unit/vue-sfc-extractor.test.ts b/gitnexus/test/unit/vue-sfc-extractor.test.ts index b77e0d34f..8f4b62a1b 100644 --- a/gitnexus/test/unit/vue-sfc-extractor.test.ts +++ b/gitnexus/test/unit/vue-sfc-extractor.test.ts @@ -24,8 +24,8 @@ const count = ref(0); expect(result!.isSetup).toBe(true); expect(result!.scriptContent).toContain("import { ref } from 'vue'"); expect(result!.scriptContent).toContain('const count = ref(0)'); - // Line 0-3 is template + blank, line 4 is `; const result = extractVueScript(vue); expect(result).not.toBeNull(); - expect(result!.lang).toBe('js'); + expect(result!.lang).toBe('jsx'); }); - it('ts lang returns empty (only js/jsx triggers JS grammar)', () => { + it('preserves the explicit ts language', () => { const vue = ` `; const result = extractVueScript(vue); expect(result).not.toBeNull(); - expect(result!.lang).toBe(''); + expect(result!.lang).toBe('ts'); }); - it('mixed js + ts blocks return empty lang (TypeScript wins)', () => { + it('selects TypeScript for mixed js and ts blocks', () => { const vue = ` @@ -116,7 +116,7 @@ import { ref } from 'vue'; `; const result = extractVueScript(vue); expect(result).not.toBeNull(); - expect(result!.lang).toBe(''); + expect(result!.lang).toBe('ts'); }); it('isSetup is true when at least one block is setup', () => { @@ -130,8 +130,8 @@ import { ref } from 'vue'; const result = extractVueScript(vue); expect(result).not.toBeNull(); expect(result!.isSetup).toBe(true); - // ts blocks — lang should be empty - expect(result!.lang).toBe(''); + // Both blocks explicitly select TypeScript. + expect(result!.lang).toBe('ts'); }); it('returns null for .vue files with no