From 169aa5d960c88966463cd8d95c1f1ffd01df4290 Mon Sep 17 00:00:00 2001 From: Abhigyan Patwari <126312502+abhigyanpatwari@users.noreply.github.com> Date: Thu, 22 Oct 2026 16:44:28 +0530 Subject: [PATCH] fix(scope): label library-rooted receiver drops as external from provider evidence (#3658) * refactor(scope): expose the position a receiver-chain fold reaches `foldReceiverChain` answers "which class", so a chain whose last step is declared as a type the program does not contain (`Dictionary`, `String`) reads only as a miss. That miss is informative to the receiver-origin classifier: every hop was typed by a program declaration and the last one names a library type. Extract the fold loop into `foldReceiverChainState` and export `foldReceiverChainPosition`, which returns the final position including one whose declared type named no class, and reports nothing to `recordReceiverType`. `foldReceiverChain` keeps its behaviour: it now wraps the same loop and makes the receiver-type report from the final state, as before. Co-Authored-By: Claude Opus 5.5 * fix(scope): classify library-rooted receiver drops from provider evidence `classifyReceiverOrigin` had one source of positive external evidence, `LanguageProvider.isBuiltInName`. That set also filters callee names out of the call graph, so it could not grow to cover host globals, stdlib imports or library-typed values without dropping real edges, and every such drop hedged `impact()` to `lower-bound`. It also read a base name as evidence even where the program bound that name itself: a parameter named `console` with an unresolved type, or an unresolved import of a built-in spelling from the program's own module, came back `external`. Add two provider data hooks read only by the classifier: `externalReceiverNames` (globals, platform roots, library types) and `isExternalModuleSpecifier` (import specifiers reserved for the runtime or stdlib). The classifier now answers `external` when: - the base is unbound at the site and is a built-in or listed name; - the base is bound only by imports that resolved to no program file and all name a library module; - the receiver chain folds, through declarations each owner makes, to a position declared as a library type (`this.data.Keys` with `Dictionary data`), and the program declares nothing by the dropped member's name (an extension method or trait impl could be the real target). The fold skips ancestor-hoisted bindings, which can lend another class's member type to the step. A base the program binds (a parameter, a local, a resolved import, an import from a non-library module) never takes a name-based verdict. The in-program arms are unchanged and still win first. No drop is added or removed and no edge changes. Java supplies its reserved `java` package root for both hooks: the JVM refuses to define classes under `java.*`, so `java.util.Optional.of(...)` and `import java.util.Map` are rooted in the platform. Without the specifier hook, the stricter import rule would have demoted the existing `import java.util.List; List.of(...)` verdict to unknown. Co-Authored-By: Claude Opus 5.5 * fix(typescript): name JavaScript host globals as the program boundary `document.body.appendChild(...)`, `window.history.pushState(...)` and `Buffer.from(raw).toString()` were recorded `unknown`: none of `document`, `window`, `Buffer`, `process` or `globalThis` is in `BUILT_INS`, and they cannot be added there because that set also filters callee names, so `process` would drop every call to a program function named `process`. Each drop hedged `impact()` to `lower-bound` although no program node could have been the target. List the host objects in `EXTERNAL_RECEIVER_NAMES` and supply it to the TypeScript, JavaScript and Vue providers as `externalReceiverNames`. A name counts only where the program binds nothing by it and declares nothing by it, so a `document: Doc` parameter, an untyped `window` parameter, a local `navigator` and a program `class Buffer` keep their verdicts. `location`, `history`, `self` and `crypto` stay out: they are ordinary local names. Corpus: 5 call drops move from unknown to external. `process.env.*` (3 reads) stays in-program there because three fixtures declare a top-level `function process()`, one of them in a global script. Co-Authored-By: Claude Opus 5.5 * fix(python): name standard-library imports as the program boundary `os.environ.setdefault(...)` after `import os`, and `Path(__file__).resolve()` after `from pathlib import Path`, were recorded `unknown`. The imports resolve to no file in the program, but an unresolved import alone cannot be told apart from an in-program module the resolver missed, and nothing said that `os` and `pathlib` belong to the interpreter. Supply `isExternalModuleSpecifier` from a curated set of standard-library modules, matched on the specifier's first segment (`os.path` -> `os`). A relative specifier never matches. The classifier still requires the import to have resolved to no program file, so a local `os.py` that the resolver finds wins, and a parameter named `os` is the program's own. Generic nouns an application plausibly uses for its own modules (`types`, `queue`, `signal`, `secrets`, `string`, `email`, `http`, ...) are left out. Third-party packages are not listed: an unresolved `fastapi` import stays `unknown`. Corpus: 4 call drops move from unknown to external (`os.environ.setdefault` x3, `Path(__file__).resolve()`). Co-Authored-By: Claude Opus 5.5 * fix(zig): name @import("std") and @import("builtin") as the program boundary `std.sort.pdq(...)` and `const Allocator = std.mem.Allocator;` were recorded `unknown`: `std` is bound by `const std = @import("std")`, an import the resolver deliberately leaves external, and nothing told the classifier that the module is the standard library rather than an in-program file it failed to find. Supply `isExternalModuleSpecifier` for the two module names Zig reserves: `std` and the compiler-generated `builtin`. Exact names only, so `@import("std.zig")` (a file path) stays `unknown`, as does `root` (the program's own root module) and any module `build.zig` names. Corpus: 9 drops move from unknown to external: `std.sort.pdq` (2 calls, 2 reads), `std.mem.Allocator` (1 read), and the `zig-qualified-literal` literals `std.Thread.Mutex{}` and `std.mem.Allocator{...}` (2 calls, 2 reads). The last four are that fixture's edge negative control (`useExternal` must emit no edge, which still holds); their receiver is std, so `external` is the right origin. Co-Authored-By: Claude Opus 5.5 * fix(csharp): name the System namespace and Dictionary as the program boundary `System.Console.WriteLine(...)` was recorded `unknown`: `Console` is in `BUILT_INS` but the chain base is `System`, which is not, and `this.data.Keys` with `Dictionary data` was recorded `in-program` because the base `this` resolves to a program class even though the member lives on a BCL type. Supply `externalReceiverNames` with `System` (the BCL root namespace), `Dictionary` (the one collection capture does not reduce to its element, because it takes two type arguments) and `string` (a keyword alias that no program can declare). The base check covers `System.Console`; the chain fold reaches `Dictionary` through the program's own field declaration. A program `class System`, a field of a program type, and an extension method the program declares on `Dictionary` all keep their verdicts. Corpus: 3 call drops move from unknown to external (`System.Console.WriteLine`). The two `this.data.Keys/Values` reads in `csharp-nested-member-foreach` stay in-program there: the fixture's global-namespace `App` is one of several in the merged corpus, so `this` binds another fixture's `App` and the fold declines. Analysed alone, both read `external`. Co-Authored-By: Claude Opus 5.5 * fix(kotlin): name java.lang.System and the platform roots as the program boundary `System.out.println(...)` in Kotlin was recorded `unknown`: Kotlin's `BUILT_INS` lists stdlib function names, not `System`, which Kotlin/JVM imports by default from `java.lang`. Supply `externalReceiverNames` with `System` and the package roots no program can declare into (`java`, refused by the JVM; `kotlin`, reserved by the compiler for the standard library), and `isExternalModuleSpecifier` for imports under those roots. A program `object System` keeps the receiver `in-program`. Corpus: 1 call drop moves from unknown to external (`kotlin-object-literal-scope` `System.out.println`, whose decoy `println` on the enclosing object still gets no edge). Co-Authored-By: Claude Opus 5.5 * test(bench): rebaseline receiver-origin buckets for library receivers The preceding commits label library- and runtime-rooted drops `external` instead of `unknown`. Call drops by origin move from 55 in-program, 44 external and 29 unknown to 55, 62 and 11. Total drops (192), call drops (128), every per-extension and per-shape count, and every shape-arm state are unchanged; a same-corpus A/B against main keeps all 2,838 CALLS/ACCESSES edges byte-identical. Co-Authored-By: Claude Opus 5.5 * test(scope): preserve library member collision hedge * fix(scope): cover more library receiver evidence * fix(vue): classify imported runtime receivers --------- Co-authored-by: Gergo Magyar Co-authored-by: Claude Opus 5.5 --- .../bench/receiver-resolution/BASELINE.md | 9 +- .../bench/receiver-resolution/baseline.json | 4 +- .../src/core/ingestion/language-provider.ts | 37 ++ .../src/core/ingestion/languages/csharp.ts | 26 ++ gitnexus/src/core/ingestion/languages/java.ts | 15 + .../src/core/ingestion/languages/kotlin.ts | 18 + .../src/core/ingestion/languages/python.ts | 112 +++++ .../core/ingestion/languages/typescript.ts | 33 ++ gitnexus/src/core/ingestion/languages/vue.ts | 9 +- gitnexus/src/core/ingestion/languages/zig.ts | 12 + .../passes/compound-receiver.ts | 82 +++- .../passes/receiver-bound-calls.ts | 325 ++++++++++++-- .../scope-resolution/pipeline/run.ts | 11 +- .../scope-resolution/unresolved-receivers.ts | 3 +- .../impact-epistemic-lower-bound.test.ts | 59 +++ .../resolvers/python-module-accessors.test.ts | 15 +- .../receiver-origin-library-boundary.test.ts | 166 +++++++ .../receiver-origin-provider-review.test.ts | 134 ++++++ .../unresolved-receivers.test.ts | 421 +++++++++++++++++- 19 files changed, 1408 insertions(+), 83 deletions(-) create mode 100644 gitnexus/test/integration/resolvers/receiver-origin-library-boundary.test.ts create mode 100644 gitnexus/test/integration/resolvers/receiver-origin-provider-review.test.ts diff --git a/gitnexus/bench/receiver-resolution/BASELINE.md b/gitnexus/bench/receiver-resolution/BASELINE.md index 0eb05b3302..5da1a6d016 100644 --- a/gitnexus/bench/receiver-resolution/BASELINE.md +++ b/gitnexus/bench/receiver-resolution/BASELINE.md @@ -995,9 +995,11 @@ Two incidental corrections from that check, both to KTD6: gap that was really a fixture defect — the same error class as the untyped `$repo` control caught earlier. +### Library-rooted receiver evidence (#3658, #3498) -### Merged main `f6e42092` +After merging main `2b4456553`, receiver totals and all shape states are unchanged: 133 call drops, 160 across all kinds, 27 reads and 0 writes. Provider evidence now classifies 30 calls external, 45 unknown, 56 in-program and 2 unclassified. +The member-name veto spans the workspace graph. A JavaScript receiver through `window.app` stays uncertain when its member is declared only in TypeScript. Direct built-in calls also stay uncertain when a program declaration matches the member name; that precision cost is deliberate. `globalThis` supplies no external evidence, and library imports must be selected by the lexical claim walk. The post-main build, both TypeScript configurations and all 105 classifier, pipeline and CommonJS checks pass. Before the main merge, the mixed-language correction preserves all 5,319 resolution records byte-for-byte; the final corpus check is recorded with the PR validation. Re-measured both benchmark arms after merging the Zig receiver fix from main. Shape states remain unchanged; counts are 132 call drops and 178 drops of all kinds (26 reads, @@ -1017,6 +1019,7 @@ Main `2b4456553` is merged normally. The field lookup retains both the declared Main `2b4456553` is merged normally. The measured baseline has 130 call drops and 156 across all kinds, with 26 reads and zero writes. Main has 133 calls, 27 reads and zero writes (160 total). Every shape state matches main. The fresh build and both TypeScript configurations pass, as do all 68 namespace-boundary, capture, hook and CommonJS merge checks. +After merging main caacc3bac normally, totals remain 130 calls, 156 across all kinds, 26 reads and zero writes. Origin labels are 30 external, 45 unknown, 53 in-program and 2 unclassified. Shape states are unchanged. The 5,328-record corpus is byte-identical to current main. Package build, both pinned TypeScript configurations, 23 CommonJS/C# precedence tests and 15 library-origin boundary tests pass. ### Rust qualified factories after #3655 Main `caacc3bac` is merged normally. The C# capture baseline retains main's fixture additions, and receiver counts are regenerated on the combined source: 132 call drops, 160 total, 28 reads and zero writes. All shape states match the preceding Rust baseline. Fresh build, both TypeScript configurations and 23 Rust factory, capture and CommonJS checks pass. Final hosted CI remains required. @@ -1029,3 +1032,7 @@ The prioritized merge validation passes a fresh package build, both TypeScript c ### Rust qualified factories after #3656 Main `a09f76be3` is merged normally. The shared receiver pass keeps Rust's provider-qualified callee hook and C++'s position-aware qualified-member lookup. Fresh package build, both TypeScript configurations and 302 tests across 20 Rust, C++, Go, CommonJS and sealed-disk-scope files pass. All 16 capture scaling checks pass without changing their budgets. The regenerated receiver check passes with 132 call drops, 160 across all kinds, 28 reads and zero writes; shape states are unchanged. The corpus adds 32 Rust resolution records to main's 5,392, removes none, and totals 5,424. Exact-head hosted CI remains required. + +### External receiver origins after #3657 + +Main `65b101407` is merged normally. Origin evidence retains the qualified-type lookup policy and both pipeline passes retain workspace file paths. Package build, both pinned TypeScript configurations, 425 focused tests across 25 files, and all 16 capture checks pass. The regenerated receiver baseline preserves every shape state and main's 132 call drops, 160 total, 28 reads and zero writes. Origin labels are 30 external, 45 unknown, 55 in-program and two unclassified. The 5,424-record resolution corpus is byte-identical to main. Exact-head hosted CI remains required. diff --git a/gitnexus/bench/receiver-resolution/baseline.json b/gitnexus/bench/receiver-resolution/baseline.json index a70499aa11..b669c1ae82 100644 --- a/gitnexus/bench/receiver-resolution/baseline.json +++ b/gitnexus/bench/receiver-resolution/baseline.json @@ -229,8 +229,8 @@ }, "callDropsByOrigin": { "in-program": 55, - "external": 44, - "unknown": 31, + "unknown": 45, + "external": 30, "<>": 2 } } diff --git a/gitnexus/src/core/ingestion/language-provider.ts b/gitnexus/src/core/ingestion/language-provider.ts index 9f1f61fff4..b8665f1cdf 100644 --- a/gitnexus/src/core/ingestion/language-provider.ts +++ b/gitnexus/src/core/ingestion/language-provider.ts @@ -870,6 +870,43 @@ interface LanguageProviderConfig { * Default: undefined (no language-specific filtering). */ readonly builtInNames?: ReadonlySet; + // ── Program boundary (receiver-origin classification only) ─────────── + /** + * Values, namespaces and types the language's RUNTIME or STANDARD LIBRARY + * supplies without the program declaring them — `document`, `process`, + * `System`, `Dictionary` — matched against a bare identifier: a dropped + * receiver's chain base, or the head of the type a receiver chain ends at. + * + * Read ONLY by `classifyReceiverOrigin`, which labels a call it could not + * resolve `external` (nothing in the program could have been the target) + * instead of `unknown`. Kept apart from `builtInNames` on purpose: that set + * also filters CALLEE names out of the call graph (`parse-worker.ts`, + * `type-env.ts`, the free-call fallback), so `process` there would drop every + * call to an in-program function named `process`. Nothing here can change an + * edge. + * + * A name only counts where the program binds nothing by it at the site and + * declares no type or value by it, so a local `document` parameter or an + * in-program `class System` still wins. The failure is asymmetric — a wrong + * entry hides a real resolver gap behind `epistemic: 'exact'`, a missing one + * only costs a hedge — so a set should under-include. Default: undefined (no + * name is evidence). + */ + readonly externalReceiverNames?: ReadonlySet; + /** + * Does this import SPECIFIER name a module the language reserves for its + * runtime or standard library (Python `os`, Zig `std`, Java `java.util`)? + * + * Read ONLY by `classifyReceiverOrigin`, and only for an import that also + * resolved to no file in the program: a name bound by such an import is + * rooted outside the program however it is spelled locally + * (`const s = @import("std")`). Third-party packages are NOT answered here — + * an unresolved `fastapi` or `express` import cannot be told apart from an + * in-program module the resolver missed, so it stays `unknown`. Default: + * undefined (no import is evidence). + */ + readonly isExternalModuleSpecifier?: (specifier: string) => boolean; + // ══════════════════════════════════════════════════════════════════════════ // Scope-based resolution hooks (RFC #909 — Ring 1 #911) // diff --git a/gitnexus/src/core/ingestion/languages/csharp.ts b/gitnexus/src/core/ingestion/languages/csharp.ts index 6fccd810f9..0467bb72d3 100644 --- a/gitnexus/src/core/ingestion/languages/csharp.ts +++ b/gitnexus/src/core/ingestion/languages/csharp.ts @@ -132,6 +132,31 @@ const BUILT_INS: ReadonlySet = new Set([ 'ThrowIfNull', ]); +/** + * Program-boundary names for `LanguageProvider.externalReceiverNames`, read + * against a dropped receiver's base (`System.Console.WriteLine(...)`) and + * against the type a receiver chain ends at (`this.data.Keys` with + * `Dictionary data`). Kept out of `BUILT_INS`, which also + * filters callee names. + * + * `System` is the BCL's root namespace. These two-argument collection types + * retain their outer name at capture rather than reducing to one element type, + * so they reach a receiver position spelled as themselves. `string` + * is a keyword alias of `System.String` and cannot be declared by a program. + * A program type or binding by any of these names still wins. + */ +const EXTERNAL_RECEIVER_NAMES: ReadonlySet = new Set([ + 'System', + 'Dictionary', + 'SortedDictionary', + 'SortedList', + 'ConcurrentDictionary', + 'ReadOnlyDictionary', + 'IDictionary', + 'IReadOnlyDictionary', + 'string', +]); + export const csharpProvider = defineLanguage({ rename: csharpRenameProvider, id: SupportedLanguages.CSharp, @@ -202,6 +227,7 @@ export const csharpProvider = defineLanguage({ // ── XML doc comments (`///`) → description (issue #2270) ── descriptionExtractor: createLeadingDocDescriptionExtractor(), builtInNames: BUILT_INS, + externalReceiverNames: EXTERNAL_RECEIVER_NAMES, // ── RFC #909 Ring 3: scope-based resolution hooks (RFC §5) ────────── // C# is the second migration after Python. See ./csharp/index.ts for diff --git a/gitnexus/src/core/ingestion/languages/java.ts b/gitnexus/src/core/ingestion/languages/java.ts index 9a308edaae..69be319bbf 100644 --- a/gitnexus/src/core/ingestion/languages/java.ts +++ b/gitnexus/src/core/ingestion/languages/java.ts @@ -126,6 +126,19 @@ const BUILT_INS: ReadonlySet = new Set([ 'Objects', ]); +/** + * The platform's reserved package root, for both program-boundary hooks: + * `java.util.Optional.of(...)` spelled out in full, and `import java.util.List` + * — an import the workspace can never resolve. The JVM refuses to define a + * class in a `java.*` package outside the platform ("Prohibited package name"), + * so nothing a program declares can live under it. `javax` is deliberately + * absent: that root carries no such guarantee. + */ +const JAVA_PLATFORM_ROOT = 'java'; + +const isJavaPlatformModule = (specifier: string): boolean => + specifier === JAVA_PLATFORM_ROOT || specifier.startsWith(`${JAVA_PLATFORM_ROOT}.`); + const orderJavaSameNameTypeCandidates = ({ callSiteFilePath, candidates, @@ -208,6 +221,8 @@ export const javaProvider = defineLanguage({ // ── Javadoc → description (issue #2270) ── descriptionExtractor: createLeadingDocDescriptionExtractor(), builtInNames: BUILT_INS, + externalReceiverNames: new Set([JAVA_PLATFORM_ROOT]), + isExternalModuleSpecifier: isJavaPlatformModule, // ── RFC #909 Ring 3: scope-based resolution hooks ── emitScopeCaptures: emitJavaScopeCaptures, diff --git a/gitnexus/src/core/ingestion/languages/kotlin.ts b/gitnexus/src/core/ingestion/languages/kotlin.ts index bf4b4daf30..3dc87c462c 100644 --- a/gitnexus/src/core/ingestion/languages/kotlin.ts +++ b/gitnexus/src/core/ingestion/languages/kotlin.ts @@ -128,6 +128,22 @@ const BUILT_INS: ReadonlySet = new Set([ 'step', ]); +/** + * Program-boundary names for `LanguageProvider.externalReceiverNames`. + * + * `System` is `java.lang.System`, which Kotlin/JVM imports by default, so + * `System.out.println(...)` reaches the JDK. `java` and `kotlin` are package + * roots no program can declare into — the JVM refuses `java.*`, and the + * compiler reserves `kotlin.*` for the standard library — so a fully qualified + * `java.util.UUID.randomUUID()` is rooted outside too. A name counts only where + * the program binds nothing by it, so an `object System` still wins. + */ +const EXTERNAL_RECEIVER_NAMES: ReadonlySet = new Set(['System', 'java', 'kotlin']); + +/** Imports from those same reserved roots can never resolve in-workspace. */ +const isKotlinPlatformModule = (specifier: string): boolean => + /^(?:java|kotlin)(?:\.|$)/.test(specifier); + export const kotlinProvider = defineLanguage({ rename: kotlinRenameProvider, id: SupportedLanguages.Kotlin, @@ -184,6 +200,8 @@ export const kotlinProvider = defineLanguage({ variableExtractor: createVariableExtractor(kotlinVariableConfig), classExtractor: createClassExtractor(kotlinClassConfig), builtInNames: BUILT_INS, + externalReceiverNames: EXTERNAL_RECEIVER_NAMES, + isExternalModuleSpecifier: isKotlinPlatformModule, // ── KDoc → description (issue #2270) ── descriptionExtractor: createLeadingDocDescriptionExtractor(), diff --git a/gitnexus/src/core/ingestion/languages/python.ts b/gitnexus/src/core/ingestion/languages/python.ts index 23afcf0052..6d29ca2f75 100644 --- a/gitnexus/src/core/ingestion/languages/python.ts +++ b/gitnexus/src/core/ingestion/languages/python.ts @@ -78,6 +78,117 @@ const BUILT_INS: ReadonlySet = new Set([ 'abs', ]); +/** + * Standard-library modules, for `LanguageProvider.isExternalModuleSpecifier`: + * a name bound by `import os` or `from pathlib import Path` — when the import + * resolved to no file in the program — is rooted in the interpreter's library, + * so `os.environ.setdefault(...)` lost no in-program edge. + * + * Matched against the specifier's FIRST segment (`os.path` → `os`); a relative + * specifier (`.compat`) has an empty one and never matches. Third-party + * packages are not listed and cannot be: an unresolved `fastapi` import looks + * exactly like an in-program module the resolver missed. + * + * Under-includes on purpose. The verdict already requires the import to have + * resolved nowhere, so a local `os.py` that the resolver finds wins; what this + * list guards against is a local module the resolver did NOT find. Generic + * nouns an application plausibly names its own modules after — `types`, + * `queue`, `signal`, `secrets`, `string`, `email`, `http`, `html`, `xml`, + * `operator`, `platform`, `test` — are therefore left out even though the + * standard library has them. + */ +const STDLIB_MODULES: ReadonlySet = new Set([ + '__future__', + 'abc', + 'argparse', + 'ast', + 'asyncio', + 'atexit', + 'base64', + 'binascii', + 'bisect', + 'builtins', + 'bz2', + 'cmath', + 'codecs', + 'collections', + 'concurrent', + 'configparser', + 'contextlib', + 'contextvars', + 'copy', + 'csv', + 'ctypes', + 'dataclasses', + 'datetime', + 'decimal', + 'difflib', + 'enum', + 'errno', + 'fnmatch', + 'fractions', + 'functools', + 'gc', + 'getpass', + 'glob', + 'gzip', + 'hashlib', + 'heapq', + 'hmac', + 'importlib', + 'inspect', + 'io', + 'ipaddress', + 'itertools', + 'json', + 'keyword', + 'linecache', + 'locale', + 'logging', + 'lzma', + 'math', + 'mimetypes', + 'multiprocessing', + 'os', + 'pathlib', + 'pickle', + 'pkgutil', + 'pprint', + 'random', + 're', + 'shlex', + 'shutil', + 'socket', + 'sqlite3', + 'ssl', + 'stat', + 'statistics', + 'struct', + 'subprocess', + 'sys', + 'sysconfig', + 'tarfile', + 'tempfile', + 'textwrap', + 'threading', + 'time', + 'timeit', + 'traceback', + 'typing', + 'unicodedata', + 'unittest', + 'urllib', + 'uuid', + 'warnings', + 'weakref', + 'zipfile', + 'zlib', + 'zoneinfo', +]); + +const isPythonStdlibModule = (specifier: string): boolean => + STDLIB_MODULES.has(specifier.split('.', 1)[0] ?? ''); + function pythonDescriptionExtractor( nodeLabel: NodeLabel, _nodeName: string, @@ -136,6 +247,7 @@ export const pythonProvider = defineLanguage({ classExtractor: createClassExtractor(pythonClassConfig), descriptionExtractor: pythonDescriptionExtractor, builtInNames: BUILT_INS, + isExternalModuleSpecifier: isPythonStdlibModule, // Django routing is whole-repo and cross-file (manage.py → settings → // ROOT_URLCONF → root urls.py, then include()s across files), so it runs as // a main-thread pass (see parse-impl's cross-file route extraction) rather diff --git a/gitnexus/src/core/ingestion/languages/typescript.ts b/gitnexus/src/core/ingestion/languages/typescript.ts index 32f2e95cb4..115a9b4860 100644 --- a/gitnexus/src/core/ingestion/languages/typescript.ts +++ b/gitnexus/src/core/ingestion/languages/typescript.ts @@ -7,6 +7,7 @@ */ import { SupportedLanguages } from 'gitnexus-shared'; +import { isBuiltin } from 'node:module'; import type Parser from 'tree-sitter'; import { unboundCallableName } from './typescript/unbound-callables.js'; import type { CaptureMatch, NodeLabel, ParsedFile } from 'gitnexus-shared'; @@ -401,6 +402,34 @@ export const BUILT_INS: ReadonlySet = new Set([ 'valueOf', ]); +/** + * Host objects a JavaScript runtime supplies as globals, for + * `LanguageProvider.externalReceiverNames`: a dropped receiver rooted at one + * (`document.body.appendChild`, `process.env.API_BASE`, + * `Buffer.from(raw).toString()`) reached code no index contains. + * + * Separate from `BUILT_INS`, which also filters CALLEE names out of the call + * graph — `process` there would drop every call to a program function named + * `process`. Here a name counts only when the program binds nothing by it at + * the site and declares nothing by it anywhere, so a `document` parameter or a + * module-level `function process()` keeps the receiver hedged. + * + * Host objects only, and only ones an application does not name its own + * values after: `location`, `history`, `self` and `crypto` are ordinary local + * names, and reaching them through `window` is already covered. + */ +export const EXTERNAL_RECEIVER_NAMES: ReadonlySet = new Set([ + // Browser host. + 'window', + 'document', + 'navigator', + 'localStorage', + 'sessionStorage', + // Node host. + 'process', + 'Buffer', +]); + /** * `this` is the only receiver keyword JavaScript and TypeScript bind, and it * is bound by every function form except an arrow (#2701). The query files @@ -502,6 +531,8 @@ export const typescriptProvider = defineLanguage({ wrapperNodeTypes: ['export_statement'], }), builtInNames: BUILT_INS, + externalReceiverNames: EXTERNAL_RECEIVER_NAMES, + isExternalModuleSpecifier: isBuiltin, // Member-assignment shapes are not free functions (#2723 follow-up). The // capture arrives anchored on the assignment, so the shape is decided from @@ -603,6 +634,8 @@ export const javascriptProvider = defineLanguage({ wrapperNodeTypes: ['export_statement'], }), builtInNames: BUILT_INS, + externalReceiverNames: EXTERNAL_RECEIVER_NAMES, + isExternalModuleSpecifier: isBuiltin, // Member-assignment shapes — see the TypeScript provider above. labelOverride: (functionNode, defaultLabel) => diff --git a/gitnexus/src/core/ingestion/languages/vue.ts b/gitnexus/src/core/ingestion/languages/vue.ts index ca8d49c041..0c421724df 100644 --- a/gitnexus/src/core/ingestion/languages/vue.ts +++ b/gitnexus/src/core/ingestion/languages/vue.ts @@ -12,6 +12,7 @@ */ import { SupportedLanguages } from 'gitnexus-shared'; +import { isBuiltin } from 'node:module'; import { createClassExtractor } from '../class-extractors/generic.js'; import { vueClassConfig } from '../class-extractors/configs/typescript-javascript.js'; import { defineLanguage } from '../language-provider.js'; @@ -19,7 +20,10 @@ import { typeConfig as typescriptConfig } from '../type-extractors/typescript.js import { tsExportChecker } from '../export-detection.js'; import { TYPESCRIPT_QUERIES } from '../tree-sitter-queries.js'; import { typescriptFieldExtractor } from '../field-extractors/typescript.js'; -import { BUILT_INS as TS_BUILT_INS } from './typescript.js'; +import { + BUILT_INS as TS_BUILT_INS, + EXTERNAL_RECEIVER_NAMES as TS_EXTERNAL_RECEIVER_NAMES, +} from './typescript.js'; import { createVariableExtractor } from '../variable-extractors/generic.js'; import { typescriptVariableConfig } from '../variable-extractors/configs/typescript-javascript.js'; import { createCallExtractor } from '../call-extractors/generic.js'; @@ -89,6 +93,9 @@ export const vueProvider = defineLanguage({ variableExtractor: createVariableExtractor(typescriptVariableConfig), classExtractor: vueClassExtractor, builtInNames: VUE_BUILT_INS, + // A ``, + 'RUNTIME_TEST_KEY', + true, + ], + [ + 'unknown import named process in Vue', + 'App.vue', + ``, + 'RUNTIME_TEST_KEY', + false, + ], + [ + 'Node process import in TypeScript', + 'main.ts', + "import process from 'node:process';\nexport const CURRENT = process.env.RUNTIME_TEST_KEY;\n", + 'RUNTIME_TEST_KEY', + true, + ], + [ + 'Node process import in JavaScript', + 'main.js', + "import runtime from 'process';\nexport const CURRENT = runtime.env.RUNTIME_TEST_KEY;\n", + 'RUNTIME_TEST_KEY', + true, + ], + [ + 'unknown import named process', + 'main.ts', + "import process from 'not-node-runtime';\nexport const CURRENT = process.env.RUNTIME_TEST_KEY;\n", + 'RUNTIME_TEST_KEY', + false, + ], + ...[ + 'Dictionary', + 'SortedDictionary', + 'SortedList', + 'ConcurrentDictionary', + 'ReadOnlyDictionary', + 'IDictionary', + 'IReadOnlyDictionary', + ].map( + (type) => + [ + type, + 'main.cs', + `using System.Collections.Generic; +using System.Collections.Concurrent; +using System.Collections.ObjectModel; +class User {} +class Service { + private ${type} data; + public void Run() { foreach (var key in this.data.Keys) {} } +} +`, + 'Keys', + true, + ] as const, + ), +] as const; +it.each(cases)( + '%s uses provider evidence without overriding program claims', + async (_label, file, source, member, external) => { + const root = fs.mkdtempSync(path.join(os.tmpdir(), 'origin-provider-review-')); + try { + writeFixtureRepo(root, { [file]: source }); + const result = await runPipelineFromRepo(root, () => {}); + const drops = getResolutionOutcomes(result).filter( + (o) => o.kind === 'suppressed' && o.reason === 'receiver-unresolved' && o.name === member, + ); + expect(drops.length).toBeGreaterThan(0); + if (external) + expect(drops.every((o) => o.kind === 'suppressed' && o.receiverOrigin === 'external')).toBe( + true, + ); + else + expect(drops.some((o) => o.kind === 'suppressed' && o.receiverOrigin === 'external')).toBe( + false, + ); + } finally { + fs.rmSync(root, { recursive: true, force: true }); + } + }, + 180000, +); + +it('keeps a program collection authoritative', async () => { + const root = fs.mkdtempSync(path.join(os.tmpdir(), 'origin-program-collection-')); + try { + writeFixtureRepo(root, { + 'main.cs': `class SortedDictionary { public string[] Keys; } +class User {} +class Service { + private SortedDictionary data; + public void Run() { foreach (var key in this.data.Keys) {} } +} +`, + }); + const r = await runPipelineFromRepo(root, () => {}); + expect( + getRelationships(r, 'ACCESSES').filter( + (e) => e.target === 'Keys' && e.targetFilePath === 'main.cs', + ).length, + ).toBeGreaterThan(0); + expect( + getResolutionOutcomes(r).some( + (o) => o.kind === 'suppressed' && o.name === 'Keys' && o.receiverOrigin === 'external', + ), + ).toBe(false); + } finally { + fs.rmSync(root, { recursive: true, force: true }); + } +}, 180000); diff --git a/gitnexus/test/unit/scope-resolution/unresolved-receivers.test.ts b/gitnexus/test/unit/scope-resolution/unresolved-receivers.test.ts index 26e5a12190..8efacf1c24 100644 --- a/gitnexus/test/unit/scope-resolution/unresolved-receivers.test.ts +++ b/gitnexus/test/unit/scope-resolution/unresolved-receivers.test.ts @@ -17,6 +17,7 @@ import type { ResolutionOutcome, } from '../../../src/core/ingestion/scope-resolution/resolution-outcome.js'; import { + buildReceiverOriginEvidence, classifyReceiverOrigin, recordSiteMember, type EarlierSiteAtKey, @@ -25,6 +26,10 @@ import { decodeReceiverChain } from '../../../src/core/ingestion/utils/receiver- import { typescriptScopeResolver } from '../../../src/core/ingestion/languages/typescript/scope-resolver.js'; import { goScopeResolver } from '../../../src/core/ingestion/languages/go/scope-resolver.js'; import { javaScopeResolver } from '../../../src/core/ingestion/languages/java/scope-resolver.js'; +import { pythonScopeResolver } from '../../../src/core/ingestion/languages/python/scope-resolver.js'; +import { zigScopeResolver } from '../../../src/core/ingestion/languages/zig/scope-resolver.js'; +import { csharpScopeResolver } from '../../../src/core/ingestion/languages/csharp/scope-resolver.js'; +import { kotlinScopeResolver } from '../../../src/core/ingestion/languages/kotlin/scope-resolver.js'; import { buildScopeModel, type ScopeModelFixture } from '../../helpers/scope-model.js'; const range = { startLine: 1, startCol: 0, endLine: 1, endCol: 1 }; @@ -242,7 +247,7 @@ describe('summarizeUnresolvedReceivers origin routing', () => { /** The provider-hook bag `classifyReceiverOrigin` reads. Derived from the * function so the two cannot drift. */ -type OriginHooks = Parameters[4]; +type OriginHooks = Parameters[5]; /** Classify the receiver of the (unique) reference site invoking `memberName` * under `hooks`, through exactly the arguments the pass threads at its drop @@ -252,26 +257,47 @@ function classifyOriginOf( memberName: string, hooks: OriginHooks, ): ReceiverOrigin { - const site = fixture.sites.find( - (candidate) => candidate.name === memberName && candidate.explicitReceiver !== undefined, - ); - expect(site).toBeDefined(); + const site = + fixture.sites.find( + (candidate) => candidate.name === memberName && candidate.explicitReceiver !== undefined, + ) ?? expect.unreachable(`no receiver site invokes \`${memberName}\``); + const receiver = + site.explicitReceiver ?? expect.unreachable(`\`${memberName}\` has no explicit receiver`); return classifyReceiverOrigin( - decodeReceiverChain(site!.receiverChain), - site!.inScope, - site!.explicitReceiver!.name, + decodeReceiverChain(site.receiverChain), + site.inScope, + receiver.name, + site.name, fixture.scopes, hooks, ); } -/** The normal path: the language's own contract hooks, exactly as the pass - * supplies them. */ +/** The normal path: the language's own contract hooks, built by the same + * factory the pass calls. The fold flags mirror the ones the pass hands + * `resolveCompoundReceiverClass`, read off the resolver. */ function originOf(fixture: ScopeModelFixture, memberName: string): ReceiverOrigin { - return classifyOriginOf(fixture, memberName, { - stripTypePreservingDecoration: fixture.resolver.stripTypePreservingDecoration, - isBuiltInName: fixture.resolver.languageProvider.isBuiltInName, - }); + const { resolver } = fixture; + return classifyOriginOf( + fixture, + memberName, + buildReceiverOriginEvidence({ + parsedFiles: [fixture.parsed], + scopes: fixture.scopes, + index: fixture.index, + foldOptions: { + fieldFallback: resolver.fieldFallbackOnMethodLookup ?? true, + elementTypeOf: resolver.elementTypeOf, + hoistTypeBindingsToModule: resolver.hoistTypeBindingsToModule === true, + stripTypePreservingDecoration: resolver.stripTypePreservingDecoration, + resolveThisViaEnclosingClass: resolver.resolveThisViaEnclosingClass, + }, + stripTypePreservingDecoration: resolver.stripTypePreservingDecoration, + isBuiltInName: resolver.languageProvider.isBuiltInName, + externalReceiverNames: resolver.languageProvider.externalReceiverNames, + isExternalModuleSpecifier: resolver.languageProvider.isExternalModuleSpecifier, + }), + ); } /** The degradation path: the same site with the language's provider hooks @@ -361,6 +387,7 @@ public class Probe { names.iterator(); List.of("a"); Map.entry("a", "b"); + Set.copyOf(names); Helper.assist(); svc.loadUser(); repo.findAll(); @@ -461,13 +488,21 @@ describe('classifyReceiverOrigin', () => { expect(originOf(javaFixture, 'findAll')).toBe('unknown'); }); - it('hedges a platform name deliberately kept OUT of the set', () => { - // `Map.entry(...)` is the exact syntactic twin of the `List.of(...)` above; - // the only thing separating the two verdicts is set membership. `Map` is a - // name applications really do declare (and in Java a same-package type needs - // no import to shadow it), so it stays out and its drops keep hedging. This - // pins the under-inclusion choice as a choice, not an oversight. - expect(originOf(javaFixture, 'entry')).toBe('unknown'); + it('hedges an unimported platform name deliberately kept OUT of the set', () => { + // `Set.copyOf(...)` is the exact syntactic twin of the `List.of(...)` above, + // and nothing in the file imports `Set`. `Set` is a name applications really + // do declare, and in Java an unimported one can only be a type of this + // package, so it stays out and its drops keep hedging. This pins the + // under-inclusion choice as a choice, not an oversight. + expect(originOf(javaFixture, 'copyOf')).toBe('unknown'); + }); + + it('reads an explicit platform import as evidence whatever the set says', () => { + // `import java.util.Map` shadows any same-package `Map` (JLS §6.4.1) and can + // never resolve in-workspace, so the import itself places `Map.entry(...)` + // in the JDK. Before the import counted, this drop hedged on set membership + // alone. + expect(originOf(javaFixture, 'entry')).toBe('external'); }); it('degrades every Java verdict to unknown when the hook is withheld', () => { @@ -476,6 +511,350 @@ describe('classifyReceiverOrigin', () => { }); }); +// --------------------------------------------------------------------------- +// Library and runtime receivers (#3498 triage). +// +// Every row below is a receiver the language's runtime or standard library +// supplies — a host global, a stdlib import, a platform package root, or a +// value whose declared type is a library type — next to a control where the +// program binds the same name itself. The control is the point: positive +// evidence means the program declares nothing by that name at the site, so a +// parameter called `document`, a local `class System` or an unresolved import +// from a non-library module must keep hedging. Built from real extraction for +// the reason given above: what a binding contains is exactly what intuition +// gets wrong. +// --------------------------------------------------------------------------- + +const tsLibraryFixture = buildScopeModel( + typescriptScopeResolver, + `export function viaDocument(): void { + document.body.appendChild(null); +} + +export function viaWindow(): void { + window.history.pushState(null, '', '/'); +} + +export function viaBuffer(token: string): string { + return Buffer.from(token, 'base64').toString(); +} + +export function viaProcess(): string | undefined { + return process.env.HOME; +} + +// The program binds the name at the site: never the host global. +export function typedParameter(document: Doc): void { + document.body.render(); +} + +export function untypedParameter(window): void { + window.history.back(); +} + +export function localValue(): void { + const navigator = makeNavigator(); + navigator.clipboard.writeText('x'); +} +`, + 'main.ts', +); + +// A program class by a host-global name shadows the host global everywhere it +// is visible. Its own file, because the class is visible module-wide. +const tsShadowFixture = buildScopeModel( + typescriptScopeResolver, + `export class Buffer { + static from(raw: string): Buffer { + return new Buffer(); + } +} + +export function viaProgramBuffer(raw: string): void { + Buffer.from(raw).encode(); +} +`, + 'buffer.ts', +); + +// TypeScript stores method return types on the MODULE scope, keyed by method +// name alone. `Status` declares no `name()`; the module's `name` binding is +// `Labels.name`'s. Reading it would type `this.name()` as a library `String` +// and call an in-program receiver external. +const tsHoistedFixture = buildScopeModel( + typescriptScopeResolver, + `export class Labels { + name(): String { + return ''; + } +} + +export class Status { + label(): void { + this.name().toLowerCase(); + } +} +`, + 'status.ts', +); + +const pythonLibraryFixture = buildScopeModel( + pythonScopeResolver, + `import os +from pathlib import Path +from vendorlib import Client +from .compat import json + + +def via_os(): + os.environ.setdefault("A", "b") + + +def via_path(): + return Path(__file__).resolve() + + +def via_third_party(): + Client().session.close() + + +def via_relative(): + json.codec.dumps({}) + + +def via_parameter(os): + os.environ.update() +`, + 'app/main.py', +); + +const zigLibraryFixture = buildScopeModel( + zigScopeResolver, + `const std = @import("std"); +const vendored = @import("std.zig"); + +pub fn sortItems(items: []u32) void { + std.sort.pdq(u32, items, {}, lessThan); +} + +pub fn viaVendored() void { + vendored.sort.insertion(); +} + +fn lessThan(_: void, a: u32, b: u32) bool { + return a < b; +} +`, + 'src/main.zig', +); + +const javaLibraryFixture = buildScopeModel( + javaScopeResolver, + `public class Probe { + public void qualified() { + java.util.Optional.of("x"); + } + + public void shadowed(Holder java) { + java.util.run(); + } +} + +class Holder { + Util util; +} + +class Util { + void run() {} +} +`, + 'src/Probe.java', +); + +const csharpLibraryFixture = buildScopeModel( + csharpScopeResolver, + `using System.Collections.Generic; + +namespace App +{ + public class User { } + + public class Registry + { + public int Count; + } + + public static class DictionaryExtensions + { + public static int Size(this Dictionary d) { return 0; } + } + + public class Probe + { + private Dictionary data = new Dictionary(); + private Registry registry = new Registry(); + + public void Platform() + { + System.Console.WriteLine("x"); + } + + public void ReadKeys() + { + foreach (var key in this.data.Keys) { } + } + + public void ReadProgramField() + { + foreach (var item in this.registry.Items) { } + } + + public void CallExtension() + { + this.data.Size(); + } + } +} +`, + 'App/Probe.cs', +); + +const csharpShadowFixture = buildScopeModel( + csharpScopeResolver, + `namespace App +{ + public class Printer + { + public void Print(string s) { } + } + + public class System + { + public static Printer Console = new Printer(); + } + + public class Probe + { + public void Run() + { + System.Console.WriteLine("x"); + } + } +} +`, + 'App/Shadow.cs', +); + +const kotlinLibraryFixture = buildScopeModel( + kotlinScopeResolver, + `class Probe { + fun platform() { + System.out.println("x") + } +} +`, + 'src/Probe.kt', +); + +const kotlinShadowFixture = buildScopeModel( + kotlinScopeResolver, + `class Printer { + fun write(s: String) {} +} + +object System { + val out = Printer() +} + +class Probe { + fun run() { + System.out.println("x") + } +} +`, + 'src/Shadow.kt', +); + +describe('classifyReceiverOrigin — library and runtime receivers', () => { + it.each([ + [ + 'program global properties', + typescriptScopeResolver, + 'export class Service { save() {} } export function run() { globalThis.service.save(); }', + 'save', + 'caller.ts', + ], + [ + 'program members reached through window', + typescriptScopeResolver, + 'export class Service { save() {} } export function run() { window.service.save(); }', + 'save', + 'caller.ts', + ], + [ + 'a library function returning a program instance', + pythonScopeResolver, + 'import copy\nclass User:\n def save(self): pass\ndef run(user):\n copy.copy(user).save()\n', + 'save', + 'caller.py', + ], + [ + 'a blocked parameter and library import claim', + pythonScopeResolver, + 'def run(os):\n os.child.missing()\n import os\n', + 'missing', + 'caller.py', + ], + ] as const)('keeps %s hedged', (_name, resolver, source, member, file) => { + expect(originOf(buildScopeModel(resolver, source, file), member)).not.toBe('external'); + }); + it.each([ + ['TypeScript document', tsLibraryFixture, 'appendChild'], + ['TypeScript window', tsLibraryFixture, 'pushState'], + ['TypeScript Buffer', tsLibraryFixture, 'toString'], + ['TypeScript process', tsLibraryFixture, 'HOME'], + ['Python stdlib namespace import', pythonLibraryFixture, 'setdefault'], + ['Python stdlib named import', pythonLibraryFixture, 'resolve'], + ['Zig std import', zigLibraryFixture, 'pdq'], + ['Java platform package root', javaLibraryFixture, 'of'], + ['C# System namespace', csharpLibraryFixture, 'WriteLine'], + ['C# field declared as a library type', csharpLibraryFixture, 'Keys'], + ['Kotlin java.lang.System', kotlinLibraryFixture, 'println'], + ] as const)('reports %s as external', (_label, fixture, memberName) => { + expect(originOf(fixture, memberName)).toBe('external'); + }); + + it.each([ + ['a TypeScript parameter named document', 'unknown', tsLibraryFixture, 'render'], + ['an untyped TypeScript parameter named window', 'unknown', tsLibraryFixture, 'back'], + ['a TypeScript local named navigator', 'in-program', tsLibraryFixture, 'writeText'], + ['a TypeScript class named Buffer', 'in-program', tsShadowFixture, 'encode'], + ["a step typed only by another class's member", 'in-program', tsHoistedFixture, 'toLowerCase'], + ['an unresolved third-party Python import', 'unknown', pythonLibraryFixture, 'close'], + ['an unresolved relative Python import', 'unknown', pythonLibraryFixture, 'dumps'], + ['a Python parameter named os', 'unknown', pythonLibraryFixture, 'update'], + ['a Zig path import spelled like std', 'unknown', zigLibraryFixture, 'insertion'], + ['a Java parameter named java', 'in-program', javaLibraryFixture, 'run'], + ['a C# field of a program type', 'in-program', csharpLibraryFixture, 'Items'], + ['a C# extension method the program declares', 'in-program', csharpLibraryFixture, 'Size'], + ['a C# class named System', 'in-program', csharpShadowFixture, 'WriteLine'], + ['a Kotlin object named System', 'in-program', kotlinShadowFixture, 'println'], + ] as const)('keeps %s %s', (_label, expected, fixture, memberName) => { + expect(originOf(fixture, memberName)).toBe(expected); + }); + + // The evidence lives in the provider. A language that supplies none of it + // must never reach an `external` verdict through these routes. + it.each([ + ['TypeScript document', tsLibraryFixture, 'appendChild'], + ['Python stdlib import', pythonLibraryFixture, 'setdefault'], + ['Zig std import', zigLibraryFixture, 'pdq'], + ['C# library-typed field', csharpLibraryFixture, 'Keys'], + ] as const)('degrades %s to non-external without the provider hooks', (_label, fixture, name) => { + expect(originOfWithoutHooks(fixture, name)).not.toBe('external'); + }); +}); + describe('classifyReceiverShape', () => { it('reports no-chain when the site carried no chain', () => { expect(classifyReceiverShape(undefined)).toBe('no-chain');