mirror of
https://github.com/abhigyanpatwari/GitNexus.git
synced 2026-08-28 05:25:25 +00:00
* fix(python): resolve calls through an unaliased dotted namespace import (#2826) `import pkg.db` followed by `pkg.db.session_scope()` emitted no CALLS edge, while all three sibling spellings resolved. In a codebase whose style guide mandates absolute imports this is close to the only cross-module call form used, so `impact()` reported `impactedCount: 0, risk: LOW, epistemic: exact` for functions with dozens of real callers — a dropped caller reading as a verified all-clear. The resolution path was never missing; one map was keyed on the wrong half of the import. `interpretPythonImport`'s plain arm splits `import pkg.db` into `localName: 'pkg'` (the name Python actually binds) and `importedName: 'pkg.db'`, and finalize carries both onto the edge as `localName` / `targetExportedName`. `collectNamespaceTargets` keyed only on `localName`, but the receiver text captured at the call site is the whole dotted path — Python's query binds the attribute's `object` field with a wildcard, so `pkg.db.session_scope()` yields the receiver `pkg.db`. Case 0 declines it (a module is not a class) and falls through, Case 1 looks up `pkg.db` and misses, and Case 1.5 needs `resolveQualifiedReceiverMember`, which only the C++ provider implements. The site drops silently. Key the map on the dotted import path as well — gated on a provider opt-in, not on the edge shape. The shape alone cannot decide it: Swift's `import Foo.Bar` produces the identical pair (`localName: 'Foo'`, `targetExportedName: 'Foo.Bar'`), but there the FIRST segment is the resolved target and `Foo.Bar` names a nested type. Minting a key for it would hand `resolveConstructionExpressionClass` an authoritative namespace — that branch deliberately does not fall through on a miss — and break `Foo.Bar(x)` construction that resolves correctly today. Hence `ScopeResolver.namespaceReceiverIncludesImportPath`, which only Python sets. The root-segment check on the added key does real work: `import pkg.db as pdb` binds only `pdb`, so writing `pkg.db.f()` there is a NameError, and its edge (localName `pdb`, path `pkg.db`) is correctly rejected. Two same-package imports stay separate — `import pkg.db` + `import pkg.cache` key `pkg.db` and `pkg.cache` independently, so neither call can land in the other's module; the shared `pkg` bucket keeps its existing ambiguity rather than gaining any. Tests: five integration rows (the issue's own repro, the three sibling spellings as controls, non-crossing two-package imports, a three-segment receiver, and dotted construction) plus a unit pin on the keying rule that asserts a Swift-shaped edge mints nothing. All five integration rows fail on the pre-fix tree; the controls pass on both, which is what makes them controls. Resolver integration suite 3024 passed / 1 skipped / 0 failed; scope-resolution unit suite 1446 passed. This changes what the resolver produces, not how it is stored — no schema or version constant applies, and an existing index needs a re-analyze to show the new edges. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(resolution): shadow-test a dotted namespace key by its root segment (#2826) `isNamespaceNameShadowed` walks the scope chain looking for a binding, type binding, lexical name, or owned def named exactly `namespaceName`. Once a namespace key can be a dotted import path, that string never matches anything: `import pkg.db` binds `pkg`, so a local `pkg = Decoy()` shadows the import, but the guard was asked about `pkg.db` and answered "not shadowed". The consequence is not a missed edge but a wrong one. The caller treats a verified namespace as authoritative and deliberately does not fall through to the workspace-wide simple-name heuristics, so an unguarded shadowed receiver resolves construction against the imported module instead of the local value. Test the first dot-separated segment instead. Single-segment names are unaffected — their root is themselves — so every pre-existing row keeps its behaviour. This ships with the key that first routes a dotted name into the guard rather than after it: the previous commit is what makes the defect reachable. The new pin fails on the pre-fix guard (verified by reverting the four comparisons and re-running: 1 failed / 5 passed), so it discriminates rather than merely passing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(python): pin the callee name on the dotted-construction row (#2826) The row asserted only that `builds` reached `pkg/db.py`. That module also exports `session_scope`, so a regression that resolved the construction to the wrong member of the right module would have kept the test green — it pinned the file, not the answer. Assert the exact edge set for the caller instead. Verified against the current tree with a scratch probe: `builds -> Model@pkg/db.py` is the only edge the file produces. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs(plans): include the #2826 engineering plan in the PR `.gitignore` keeps `docs/*` local because planning output is normally throwaway. Force-added here at the reviewer's request so the plan travels with the work it drove: it records the evidence chain behind the fix, the two places the plan turned out to be wrong, and the follow-ups deliberately left out of scope. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor(resolution): make the namespace shadow guard shared (#2826) `isNamespaceNameShadowed` lived module-private in `compound-receiver.ts` with a single caller. The namespace map it guards has three consumers, and the next commit adds the guard to a second one, so it moves to `scope/walkers.ts` alongside the other scope-chain primitives rather than being duplicated. Behaviour is unchanged — this is a move plus documentation. Two notes were added because both are easy to get wrong later: - Fails closed on a missing scope or a parent cycle. For every caller, suppressing costs a missing edge while trusting a corrupt scope chain costs a wrong one, so the bias is deliberate. - It reads `scope.bindings` DIRECTLY rather than through `lookupBindingsAt`, which is the opposite of the fix #2745 applied to Rust's `headBoundLocally`. There the question was "is this name bound at all?", so missing finalize's import channels lost real bindings. Here the question is "does something LOCAL shadow the import?", and the import's own finalized binding is exactly what must not count — routing this through `lookupBindingsAt` would find every namespace import shadowing itself and suppress the lot. Verified against a target module carrying a self-named def, which still resolves. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(python): close the three remaining namespace-receiver gaps (#2826) Three defects the first fix left behind. All three were confirmed by probe before being touched, and a fourth suspected gap was disproved the same way. ## 1. Case 1 resolved through an import a local had shadowed `namespaceTargets` is collected per FILE, but Case 1 in `receiver-bound-calls` consulted it with no lexical guard at all, so import pkg.db def f(pkg): # parameter shadows the package return pkg.db.session_scope() emitted an edge to pkg/db.py. That is a WRONG edge, and it predates the dotted key: the single-segment spelling (`import single` + `def f(single)`) failed identically. The compound-receiver construction path has applied this guard since #2770; Case 1 simply never did. Now both use the shared guard. ## 2 + 3. The root key named the leaf module, not the package These read as two gaps and are one. `import a.b.c` binds ONE name — `a` — but makes three attribute paths callable, naming three different files: a → a/__init__.py a.b → a/b/__init__.py a.b.c → a/b/c.py The map keyed only `a`, pointed at the LEAF. So `a.helper()` resolved into a/b/c.py whenever that module happened to export `helper` — silently preferring a decoy over the real definition in the package — and `a.b.mid()` resolved to nothing at all. One wrong edge and one missing edge from a single mis-keying. Fixing it needs per-language knowledge the shared collector cannot have: which prefixes are reachable, and which file each names. The `__init__.py` convention is Python's alone, and the edge shape is ambiguous across languages — Swift's `import Foo.Bar` produces an identical `localName`/`targetExportedName` pair that means the opposite thing. So the previous commit's boolean opt-in is replaced by `ScopeResolver.namespaceReceiverPaths`, which returns every spelling with the file it names; absent or declining, the shared default (bound name → own target) is unchanged for every other language. Prefix files are proposed, not asserted — `moduleFileExists` drops any the workspace never parsed, so a PEP-420 namespace package contributes no key rather than one pointing at a missing file. ## Disproved: C# was not a fourth gap The plan listed C# `using System.Collections.Generic` + `System.Collections.Generic.List` as the same class of bug. It is not: a probe shows `My.Deep.Space.Helpers.Work()` already resolves through the FQN namespace bindings in `walkers.ts`. No change made, and the claim is withdrawn rather than carried forward as a known gap. ## Testing Integration: the shadow block asserts the exact surviving edge set (an absence-only assertion would also pass if the guard over-suppressed and killed the clean rows); the prefix block asserts all three spellings land on their own file, with `helper` defined in BOTH package and leaf so a wrong edge is visible rather than merely possible. Unit: 16 rows on the keying contract, including that a Swift-shaped edge mints nothing and an alias import keys neither the path nor the root. Resolver integration 3024 passed / 1 skipped / 0 failed; scope-resolution unit 1452 passed; tsc clean in both packages. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(python): probe both path separators when resolving a prefix package (#2826) Workspace file paths are not normalized to POSIX at ingestion — `import-target` already re-normalizes at five other comparison points, and `moduleScopeByFile` is keyed by the raw `ParsedFile.filePath`. The prefix probe built only the `/` spelling, so on Windows it would compare `a/b/__init__.py` against an `a\b\__init__.py` key, find nothing, and mint no prefix keys at all. That fails quietly, which is the worst shape for it: `a.b.mid()` simply goes back to unresolved on one platform, with no drop recorded and every test on POSIX still green. Probe both spellings and key whichever the workspace actually holds. The new row is mutation-tested — reverting to the `/`-only probe turns it red (1 failed / 10 passed), so it pins the behaviour rather than passing alongside it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(python): correct three defects a multi-lane review found in this PR (#2826) All three were introduced by this PR's own earlier commits, and none was found by re-reading the diff — each came from a lane attacking an angle the author had not. ## 1. The shadow guard ran BEFORE the map lookup it gates Case 1 evaluated `isNamespaceNameShadowed` unconditionally, then consulted `namespaceTargets`. So every call/read/write site with an explicit receiver, in every language, paid a scope-chain walk (a Set allocation, three Map lookups and a linear `ownedDefs` scan per level) ahead of an O(1) hash miss that was going to decline it anyway. The proof it was an oversight rather than a decision sits in this same PR: the sibling guard in `compound-receiver.ts` reads the map first and only guards on a hit. Two call sites of one shared function, opposite order. Semantics are identical either way — a miss yields `undefined` regardless — which is exactly why it survived several readings. ## 2. Prefix packages were anchored on the import spelling, not the resolved leaf `pythonNamespaceReceiverPaths` built `a/__init__.py` from the dotted path joined at the workspace root, never consulting the file the import actually resolved to. But `resolvePythonImportTarget` resolves off-root in two of its three tiers, so `import utils.db` can land on `libs/common/utils/db.py`. That produced a wrong edge where a same-named `utils/` package exists at the root, and produced NOTHING in a `src/` layout — the prefix feature was inert for the most common Python project shape, silently. Now the prefix directories are derived by walking back from the resolved leaf, which is exact for root, `src/` and off-root layouts alike. It also inherits the leaf's own separator, which subsumes the previous dual-separator probe: that probe was dead code anyway, because `filesystem-walker.ts` normalizes `\` to `/` before a path ever becomes a `ParsedFile.filePath`. Its test row is removed rather than left asserting an unreachable state. ## 3. Keying the root at `__init__.py` INSTEAD of the leaf lost re-exports `findExportedDef` accepts only a binding whose `origin === 'local'`. The canonical Python package re-exports from its submodules — `from .b.c import helper` in `__init__.py` — which is an IMPORT binding, so it is rejected. Keying the prefix solely at the package therefore turned `a.helper()` from a correct edge into no edge at all for the most common package shape. Every fixture in this PR defined its members locally in `__init__.py`, which is precisely the one layout where that mistake is invisible. The prefix now keys the package FIRST and the leaf behind it. A real definition in `__init__.py` still wins over a same-named decoy deeper in the package, and a name merely re-exported there still resolves through the leaf. Ordering is the contract, so the unit rows assert the exact arrays rather than membership. ## Testing New rows: off-root layout with a decoy `utils/` at the root, and a `src/` layout. Both mutation-tested — reverting to the spelling-anchored build turns them red. The re-export case was verified end-to-end with a scratch fixture whose `__init__.py` only re-exports (`uses -> helper@a/b/c.py`). Resolver integration 3131 passed / 1 skipped / 0 failed — unchanged from before these fixes, so they regress nothing. Scope-resolution unit 1459 passed. tsc clean in both packages. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(resolution): stop the namespace shadow guard AT the module scope (#2826) CI caught a regression this PR introduced: `cjs-exports-assignment.test.ts` lost both of its cross-file rows — cross-file require() member call resolves expected [] to deeply equal [ 'handle' ] an `exports` parameter does not hijack the module (UMD factory) expected [] to deeply equal [ 'publicApi' ] — i.e. `const svc = require('./svc'); svc.handle()` stopped resolving in JavaScript. Cause: in CommonJS the namespace import IS a variable declaration. One statement produces both the ImportEdge and a module-scope `const` binding, so the guard, by inspecting the module scope, found the import's own name there and read it as a shadow of itself — suppressing exactly the receivers it exists to enable. The guard's own contract sentence already said the right thing: "a declaration BETWEEN the call site and its module scope". The module scope is the floor of that walk, not a rung on it. It now returns at Module without inspecting it. Nothing is lost on the suppression side: a genuine shadow is a parameter, a local, or a nested declaration, and all of those live in scopes strictly inside the module. The Python rows that pin suppression (`def f(pkg): pkg.db.f()` and its single-segment `import single` twin) still pass, because a parameter is an inner scope. Worth recording for the next reader: two independent review lanes examined this exact scenario and both REFUTED it, reasoning that `require()` yields an ImportEdge in `scope.imports` rather than a local binding. That is true for Python's `import x` and false for CommonJS, where one statement is both. My own probe used a Python fixture and so could not surface it either. Agreement between reviewers was not evidence; the test corpus was. Verified: cjs-exports-assignment 36/36, the #2826 integration rows 7/7, scope-resolution unit 126/126. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
162 lines
5.9 KiB
TypeScript
162 lines
5.9 KiB
TypeScript
import { describe, expect, it } from 'vitest';
|
|
import type { ParsedFile } from 'gitnexus-shared';
|
|
import { pythonScopeResolver } from '../../../../src/core/ingestion/languages/python/scope-resolver.js';
|
|
import { extractParsedFile } from '../../../../src/core/ingestion/scope-extractor-bridge.js';
|
|
import { finalizeScopeModel } from '../../../../src/core/ingestion/finalize-orchestrator.js';
|
|
import { resolveCompoundReceiverClass } from '../../../../src/core/ingestion/scope-resolution/passes/compound-receiver.js';
|
|
import { collectNamespaceTargets } from '../../../../src/core/ingestion/scope-resolution/scope/namespace-targets.js';
|
|
import { buildWorkspaceResolutionIndex } from '../../../../src/core/ingestion/scope-resolution/workspace-index.js';
|
|
|
|
const SOURCES = new Map([
|
|
[
|
|
'pkg/app.py',
|
|
`from pkg import models
|
|
from decoy import Missing
|
|
|
|
def valid():
|
|
return models.User()
|
|
|
|
def missing():
|
|
return models.Missing()
|
|
|
|
def shadowed(models):
|
|
return models.User()
|
|
|
|
def locally_shadowed():
|
|
models = object()
|
|
return models.User()
|
|
`,
|
|
],
|
|
[
|
|
'pkg/dotted.py',
|
|
`import pkg.models
|
|
|
|
def dotted():
|
|
return pkg.models.User()
|
|
|
|
def dotted_root_shadowed():
|
|
pkg = object()
|
|
return pkg.models.User()
|
|
`,
|
|
],
|
|
[
|
|
'pkg/models.py',
|
|
`class User:
|
|
pass
|
|
`,
|
|
],
|
|
['pkg/__init__.py', '# package marker\n'],
|
|
[
|
|
'decoy.py',
|
|
`class Missing:
|
|
pass
|
|
`,
|
|
],
|
|
]);
|
|
|
|
function build() {
|
|
const parsedFiles: ParsedFile[] = [];
|
|
for (const [filePath, source] of SOURCES) {
|
|
const parsed = extractParsedFile(pythonScopeResolver.languageProvider, source, filePath);
|
|
if (parsed === undefined) throw new Error(`scope extraction failed for ${filePath}`);
|
|
pythonScopeResolver.populateOwners(parsed);
|
|
parsedFiles.push(parsed);
|
|
}
|
|
|
|
const allFilePaths = new Set(parsedFiles.map((file) => file.filePath));
|
|
const scopes = finalizeScopeModel(parsedFiles, {
|
|
hooks: {
|
|
resolveImportTarget: (targetRaw, fromFile, _workspaceIndex, parsedImport) =>
|
|
pythonScopeResolver.resolveImportTarget(targetRaw, fromFile, allFilePaths, undefined, {
|
|
parsedFiles,
|
|
parsedImport,
|
|
}),
|
|
isNamespaceImport: (parsedImport, targetFile, fromFile) =>
|
|
pythonScopeResolver.isNamespaceImport?.(parsedImport, targetFile, fromFile) ?? false,
|
|
mergeBindings: (existing, incoming, scopeId) =>
|
|
pythonScopeResolver.mergeBindings(existing, incoming, scopeId),
|
|
},
|
|
});
|
|
const index = buildWorkspaceResolutionIndex(parsedFiles);
|
|
const namespaceTargetsFor = (file: ParsedFile) =>
|
|
collectNamespaceTargets(file, scopes, {
|
|
// Mirror the production wiring in receiver-bound-calls.ts: the extra
|
|
// receiver spellings come from the provider hook, not from a default.
|
|
receiverPaths: pythonScopeResolver.namespaceReceiverPaths,
|
|
moduleFileExists: (filePath) => index.moduleScopeByFile.has(filePath),
|
|
});
|
|
const app = parsedFiles.find((file) => file.filePath === 'pkg/app.py');
|
|
if (app === undefined) throw new Error('missing app fixture');
|
|
const namespaceTargets = namespaceTargetsFor(app);
|
|
|
|
const resolveIn = (functionName: string, expression: string) => {
|
|
const functionScope = app.scopes.find(
|
|
(scope) =>
|
|
scope.kind === 'Function' &&
|
|
scope.ownedDefs.some((def) => def.qualifiedName === functionName),
|
|
);
|
|
if (functionScope === undefined) throw new Error(`missing scope for ${functionName}`);
|
|
return resolveCompoundReceiverClass(expression, functionScope.id, scopes, index, {
|
|
constructionSyntax: { bare: true },
|
|
namespaceTargets,
|
|
});
|
|
};
|
|
|
|
const dotted = parsedFiles.find((file) => file.filePath === 'pkg/dotted.py');
|
|
if (dotted === undefined) throw new Error('missing dotted fixture');
|
|
const dottedNamespaceTargets = namespaceTargetsFor(dotted);
|
|
|
|
const resolveDottedIn = (functionName: string, expression: string) => {
|
|
const functionScope = dotted.scopes.find(
|
|
(scope) =>
|
|
scope.kind === 'Function' &&
|
|
scope.ownedDefs.some((def) => def.qualifiedName === functionName),
|
|
);
|
|
if (functionScope === undefined) throw new Error(`missing scope for ${functionName}`);
|
|
return resolveCompoundReceiverClass(expression, functionScope.id, scopes, index, {
|
|
constructionSyntax: { bare: true },
|
|
namespaceTargets: dottedNamespaceTargets,
|
|
});
|
|
};
|
|
|
|
return { resolveIn, resolveDottedIn };
|
|
}
|
|
|
|
describe('Python module namespace construction', () => {
|
|
const { resolveIn, resolveDottedIn } = build();
|
|
|
|
it('resolves an exported class from the verified module target', () => {
|
|
expect(resolveIn('valid', 'models.User()')).toMatchObject({
|
|
filePath: 'pkg/models.py',
|
|
qualifiedName: 'User',
|
|
});
|
|
});
|
|
|
|
it('fails closed when the module lacks the requested class', () => {
|
|
expect(resolveIn('missing', 'models.Missing()')).toBeUndefined();
|
|
});
|
|
|
|
it('does not reuse a file-level namespace when a parameter shadows it', () => {
|
|
expect(resolveIn('shadowed', 'models.User()')).toBeUndefined();
|
|
});
|
|
|
|
it('does not reuse a file-level namespace when a local shadows it', () => {
|
|
expect(resolveIn('locally_shadowed', 'models.User()')).toBeUndefined();
|
|
});
|
|
|
|
// #2826: `import pkg.models` binds only `pkg`, so the namespace key is the
|
|
// dotted path `pkg.models` while the shadowable name is the root `pkg`.
|
|
it('resolves construction through a dotted import-path namespace', () => {
|
|
expect(resolveDottedIn('dotted', 'pkg.models.User()')).toMatchObject({
|
|
filePath: 'pkg/models.py',
|
|
qualifiedName: 'User',
|
|
});
|
|
});
|
|
|
|
it('does not reuse a dotted namespace when a local shadows its ROOT segment', () => {
|
|
// Fails without the root-segment fix: testing the whole `pkg.models`
|
|
// string against scope bindings never matches, so the guard would pass a
|
|
// shadowed receiver straight through to the authoritative namespace branch.
|
|
expect(resolveDottedIn('dotted_root_shadowed', 'pkg.models.User()')).toBeUndefined();
|
|
});
|
|
});
|