scriptoriumadmin/GitNexus
1
0
Fork 0
mirror of https://github.com/abhigyanpatwari/GitNexus.git synced 2026-08-28 05:25:25 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 9

perf(import-resolvers): index every scanning resolver, consolidate the memo, gate every registered language (#2911)

Browse source 
* perf(import-resolvers): build buildSuffixIndex's dirMap lazily (#2903)

`buildSuffixIndex` eagerly built three maps. `dirMap` is the array-valued one —
one entry per directory suffix per file, so O(files x depth) in entries and
array churn — and only four call sites ever read it, all via `getFilesInDir`:
`import-resolvers/{php,csharp,jvm}.ts` and `import-resolvers/configs/python.ts`.

Ruby (through workspace-file-index), the TypeScript scope resolver, Vue's
import-target and the include-extractor never ask a directory question, and
built it anyway. Since #2880 these indexes are retained for a whole resolution
pass rather than rebuilt per import, so that waste is now resident memory.

Deferring it to the first `getFilesInDir` call is behaviour-identical — same
key, same descending-suffix order, same per-bucket push order, same
`substring(lastIndexOf('.'))` extension clamp. The builder assigns the MAP on
completion, so a repeated miss cannot rebuild it.

Measured on `buildSuffixIndex` alone, 32k paths, index built and
`getFilesInDir` never called:

  C# layout, 13 segments   79,018,680 -> 66,580,488 B   -15.74%
  Ruby layout, 11 segments 60,752,792 -> 48,656,856 B   -19.91%

and on the whole retained WorkspaceFileIndex the bench measures:

  csharp 32k  73.62 -> 61.76 MiB   ruby 32k  55.26 -> 43.69 MiB

When `getFilesInDir` IS called the footprint is unchanged, so the deferral is
never a loss. No new retention: all five construction sites already hold both
input arrays alive beside the index.

The laziness is pinned structurally rather than by timing. The test's corpus is
a `string[]` whose elements are accessor properties, so an indexed read is
observable and the read count IS the pass count: 14 after construction, still
14 after any number of get/getInsensitive, 28 after the first `getFilesInDir`,
28 after five more. Memoizing the decision instead of the map would read 42.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Co58j4au9JLf8dmJpwdF9B

* perf(php): resolve imports from a per-run index, not a scan per import (#2901)

PHP was the last language whose import resolution scanned the workspace per
import. Both `resolvePhpImportTarget` and `resolvePhpImportTargetInternal`
materialized two full arrays from the Set on every call, then passed
`undefined` as the `index` argument — so `resolvePhpImportInternal` fell
through to `suffixResolve`'s linear `findIndex`, once per extension per path
part. Measured at 20,000 files: 96.40 ms per import.

**Handing it the shared SuffixIndex would have moved IMPORTS edges.** All three
index-fed sites answer a different question than the scan they short-circuit,
each found by differential with a concrete witness:

  1. `getInsensitive` — the scan leg is `allFiles.has(path)`, exact whole-path
     with no case-insensitive counterpart; the shared index answers a ci SUFFIX
     probe.
  2. `getFilesInDir` — the scan is root-anchored `startsWith(nsDir + '/')`;
     `dirMap` is keyed on every directory SUFFIX, so a vendor copy can win.
  3. `suffixResolve` — the scan's `endsWith('/' + S)` matches only a PROPER
     suffix; `buildSuffixIndex` indexes j=0, so a root-level `Foo.php` starts
     resolving `use Foo` where it returned null.
  3b. the scan's `endsWith(p) || lower.endsWith(lower(p))` has a second
     disjunct that subsumes the first, so it is purely first-in-Set-order and
     case-insensitive; `get(S) || getInsensitive(S)` lets a case-exact hit
     anywhere beat an earlier ci hit.

So this is not Ruby's #2880 shape. Both sites take `getWorkspaceFileIndex` for
the memoized arrays and hand the internal resolver a PARITY `SuffixIndex`
memoized on the same Set identity: `getInsensitive` disabled, `get`
implementing the scan's real rule via the shared ci lookup plus one O(files)
whole-path correction map, `getFilesInDir` root-anchored in Set order.

  no composer.json    96.40 -> 0.036 ms/import steady state
  with composer.json 100.19 -> 0.068 ms/import steady state

Also closes PHP's last per-import traversal, in `import-resolvers/php.ts`: its
namespace-directory scan ran whenever `getFilesInDir` came back EMPTY, not
merely when no index was supplied — despite the comment above it claiming
"only when SuffixIndex unavailable". An empty bucket is already the answer, so
the scan could only confirm it, at one full pass per import whose namespace
matches a PSR-4 prefix but whose directory has no direct `.php` child
(measured 11 traversals for 10 imports; now 1). Moving it into the `else` is
safe because the bucket is a SUPERSET of what the scan finds — a root-anchored
direct child `nsDir/<x>.php` has its directory exactly equal to `nsDir`, and a
directory is always one of its own suffixes, so both index shapes contain it.

Nine mutations of the new code are caught, including M1 "pass the raw shared
index" (the naive fix) at 23 arms. The adapter guard reads 600 instead of 1
under a defensive `new Set(allFilePaths)` — the #1918 P1 hazard the unit
differential is structurally blind to.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Co58j4au9JLf8dmJpwdF9B

* perf(java): index import resolution instead of scanning per import (#2908)

Java scanned the whole workspace twice per import: once for the three-tier
direct match, and again INSIDE the progressive prefix-stripping loop — so a
single unresolvable import cost one full pass per stripped segment. No WeakMap,
no index, and it is registered in `SCOPE_RESOLVERS`, so it ran in production.

This is byte-for-byte the C# shape #2878 fixed, so Java now reads the same
machinery: `getWorkspaceFileIndex` for `normToRaw` + the segment-suffix index,
and a Java-owned `PackageDirIndex` WeakMap over `buildPackageDirIndex(_, n =>
n.endsWith('.java'))` read through `firstFileDirectlyInPkgDir`. Structure
mirrors C#'s `narrowContext` / `resolveDirectMatch` /
`resolveByProgressiveStripping`.

  20k files, 256 imports, 7-in-8 unresolvable:  8.05 -> 0.62 ms/import
  steady state once the index is built:         0.0036 ms/import

Tie-breaks preserved, and Java's are NOT identical to C#'s:

  - tier 1 `break`s on the exact match, so an exact whole-path hit wins even
    when a suffix or directory-child hit came earlier in iteration order —
    hence `normToRaw.get` before `index.get`, which conflates them;
  - the stripping loop instead returns at the FIRST hit of `f === tailFile ||
    f.endsWith('/' + tailFile)` and only yields its directory child after the
    scan completes, so the conflated `index.get` is the correct lookup THERE.
    Applying tier 1's exact-wins rule inside the loop is a real behaviour
    change (mutation M6);
  - `.*` wildcard stripping stays ahead of everything;
  - `firstFileDirectlyInPkgDir` reproduces Java's at-root/at-nested predicate
    exactly, including the first-`indexOf` rule — proved algebraically rather
    than assumed: the `atRoot` branch matches iff `dir === pathLike`, which is
    `D.indexOf(P) === 0 === D.length - P.length`, and the `atNested` branch's
    first occurrence in `f` is the first occurrence in `D` shifted by one.

Six mutations are caught; a seventh (swapping the two index builds) is a true
equivalence and is recorded as such. Hand-derivation also corrected four cases
where the legacy code resolves and I had predicted null — including
`java.util.List` reaching a local `util/List.java`, because Java has no
in-repo-namespace gate like C#'s #1881. That is preserved here and filed
separately as #2910; the parity test pins it so the fix is visible.

The adapter guard reads 800 instead of 2 under a defensive
`new Set(allFilePaths)`. Two traversals is correct: the workspace index and the
package-dir index are separate WeakMaps and each iterates the Set once, the
same accounting as C#.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Co58j4au9JLf8dmJpwdF9B

* perf(cobol): index COPY resolution instead of two scans per statement (#2908)

`cobolScopeResolver.resolveImportTarget` ran two full workspace scans per
`COPY`, each calling `path.extname` + `path.basename` + `.toUpperCase()` on
every entry: tier 1 over `.cpy`/`.copybook`, tier 2 over `.cbl`/`.cob`/
`.cobol`. No WeakMap, no index, and registered in `SCOPE_RESOLVERS`.

Two uppercased-basename maps, one per tier, filled in a SINGLE pass over the
Set and memoized on Set identity. Lookup is
`copybooks.get(upper) ?? sources.get(upper) ?? null`.

  20k files, 500 COPY operands:  3879-4082 -> 10.5-11.7 us/import  (~350-369x)
  steady state once built:       0.253 us/import

Tie-breaks preserved:

  - TIER ORDER. A `.cpy` match beats a `.cbl` match even when the source file
    appears EARLIER in Set-iteration order. This is the one a naive
    single-map rewrite silently breaks, so it gets its own fixture.
  - Within a tier, first in Set-iteration order wins (`if (!tier.has(...))`,
    mirroring the scans' first-match return).
  - The key is built with the identical call sequence,
    `basename(fp, extname(fp).toLowerCase()).toUpperCase()`, so `Foo.CPY` still
    keys under `FOO.CPY` rather than `FOO`.
  - `path` stays in the loop rather than hand-rolled `/`-slicing, so backslash
    handling is unchanged on every platform — pinned by a `dir\sub\BOOK.cpy`
    case.

All six mutations are caught: collapsing the tiers, within-tier last-wins,
dropping the target uppercase, dropping the extension lowercase, hand-rolled
slicing, and the adapter's defensive copy. The first five are caught by the
differential and are invisible to the adapter guard; the sixth is the reverse,
which is the layering working as intended — the guard reads 600 instead of 1.

`COBOL_SOURCE_EXTENSIONS` was being re-allocated on every call; hoisted to
module scope beside `COPYBOOK_EXTENSIONS`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Co58j4au9JLf8dmJpwdF9B

* perf(csharp): index the csproj leg's namespace-directory scan (#2902)

#2878 moved C#'s no-csproj leg onto memoized indexes; the csproj leg kept a
per-import full scan in `resolveCSharpImportInternal` step 3, measured at
~1.10 ms per import at 50,000 `.cs` files.

**The fix the issue proposed would have moved edges.** It suggested skipping
the fallback when an exhaustive index is available, on the assumption that
step 2's `getFilesInDir` answers the same question. It does not: step 2's
`dirMap` is keyed on segment-aligned directory suffixes, while step 3's
`normalized.indexOf(dirPrefix + '/')` is an UNANCHORED substring match, so
step 3 finds a strict superset — and it runs only when step 2 came back empty,
so those extra hits are observable, not shadowed:

  dirPrefix 'ubModels'  step 2 []  step 3 ['src/SubModels/Widget.cs']
  dirPrefix 'rc/Models' step 2 []  step 3 src/Models/* AND vendor/mysrc/Models/*

So the predicate is kept byte-for-byte and made fast instead. It depends only
on the file's directory (the needle ends with `/`, so every occurrence lies
wholly inside `D + '/'`), which reduces to the `package-dir-index` formula
minus the anchoring leading slash. `PackageDirIndex` itself cannot be reused
for the same reason — its matcher is anchored.

The index is memoized on the `normalizedFileList` array identity and built
lazily at the point step 3 is first reached, so BCL usings — which `continue`
out at the root-namespace gate — never pay for it. Candidates come from an
exact last-segment bucket when `dirPrefix` contains a slash, a last-segment
key sweep when it does not, and `singleSegmentDirs` when it is empty.
Positions rather than paths, merged and sorted when several directories match,
so file-list order survives.

  App.Missing @ {App, src}  1103.0 -> 7.6 us   (145x, and flat in file count:
                                                7.3 @10k, 7.6 @50k, 8.4 @200k)
  App.Missing @ {App, ''}    626.7 -> 108.5 us
  App @ {App, ''}           1077.9 -> 2.0 us   (539x)
  App.Ns8 @ {App, src}         0.6 -> 0.6 us   (step-2 hit, untouched)

`relative === ''` is preserved exactly, including the no-`projectDir` case
where the needle is a bare `/` and the answer is "every `.cs` whose directory
has no slash of its own" — `getFilesInDir('', '.cs')` cannot answer that over
repo-relative paths, so it has its own arm.

13 of 14 mutations are caught, including M1, the naive skip-when-indexed
cleanup, at 9 arms. The survivor drops the empty-prefix fast path and is a
true equivalence. M9 initially survived and exposed a real corpus gap — no
non-`.cs` file lived inside a directory — now covered.

The remaining non-constant term is the slash-free sweep, O(distinct last
segments): 456 us at 200k files on a unique-name layout, but 7.9 us on a
`SrcN/Models` layout, which is how C# repos are actually laid out. Closing the
unique-name case needs a character-suffix map over segments — the
O(files x depth) memory shape `package-dir-index.ts` cites #2649 to avoid — so
it is documented in the code as a design change rather than tuned here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Co58j4au9JLf8dmJpwdF9B

* test(scope-resolution): assert index reuse for every registered language (#2909)

Index reuse was asserted by nine hand-written per-language files, so the
guarantee existed exactly for the languages someone remembered — and #2908 is
the proof that is not good enough: Java and COBOL were registered, quadratic
and unguarded until this branch. `resolveImportTarget` is a required member of
`ScopeResolver` with one signature and 16 registrations, so "calling it N times
against a stable `allFilePaths` must not traverse the set N times" is a
property of the CONTRACT.

`import-target-index-reuse.contract.test.ts` drives every entry of
`SCOPE_RESOLVERS`, modelled on `construction-syntax-wiring.test.ts` — the
established shape here for a property plus a justified inventory. Measured
counts, all memoized:

  c 1  cobol 1  cpp 1  csharp 2  dart 1  go 1  java 2  javascript 2
  kotlin 1  php 1  python 1  ruby 1  rust 0  swift 1  typescript 2  vue 2

**`KNOWN_UNINDEXED` is empty.** The audit that produced it also cleared C, C++,
Rust, Swift, TypeScript, Vue and JavaScript by hand — Rust's memo lives in
`qualified-call.ts::moduleIndexFor`, C's and Swift's loops are inside their
WeakMap builders. The empty map stays as a mechanism: a 17th language cannot
opt out silently, and the inventory arm fails when a registered resolver has no
fixture.

Two things the assertion had to get right:
  - it is `scans(200) === scans(2)`, not `scans === 1`. Per-language counts
    legitimately differ (C# and Java build two indexes), and comparing two
    counts needs no per-language expected value.
  - Rust legitimately scans ZERO times — it answers every leg with
    `allFilePaths.has(candidate)` probes — so the floor is a per-language
    `minimumScans`, 1 for fifteen languages and 0 for Rust with the reason on
    the interface. Paired with a `hitTarget` that must resolve non-null, so the
    property cannot pass vacuously on a resolver that stopped answering.
Miss targets are distinct per import, which defeats the TS/JS/Vue per-target
`resolveCache`.

Also unifies the instrument. Kotlin and Python counted index BUILDS from
production; the other seven count traversals of a `CountingSet`. The build
counter is strictly weaker — a scan added BESIDE a reused index moves no build
count, which is exactly the mutation `baselines.json` `_blind_spot` records as
invisible to every timing arm — and it costs two production modules that ship
in the bundle purely for tests, holding module-global state every test must
`reset()`. Both guards migrate to `CountingSet`, and
`languages/{kotlin,python}/index-stats.ts` plus both call sites are gone, for
-59 lines of shipped source.

(Mechanical note: the two `index-stats.ts` file deletions appear in the #2901
commit rather than this one. They were staged with `git rm` while a concurrent
commit swept the index. The final tree is correct; only that attribution is
off, and rewriting a sibling commit to move them was not worth the risk.)

Coverage went up in the swap: Kotlin's old "rebuilds when the file set is a
different object" arm (3 sets, 3 builds) would have PASSED under a defensive
adapter copy. Its replacement fails, as do all six arms across the two files.

Verified by mutation: `new Set(allFilePaths)` inserted into the kotlin, python
and go adapters fails exactly those three and no others —
`python: 200 imports cost 201 traversals, 2 cost 3`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Co58j4au9JLf8dmJpwdF9B

* test(import-target): gate the four newly-indexed resolvers, retighten heap

The bench covered go/csharp/dart/ruby/kotlin. The four resolvers indexed on
this branch shipped unmeasured, and #2903's memory win was not locked in.

**php, java and cobol join the shared corpus**, each with the two load-bearing
properties the header requires: imports scale with file count, and most imports
MISS so the full cascade runs (resolve rates php 36.0%, java 34.4%,
cobol 36.0%). Java's miss families were measured rather than assumed, since it
has no in-repo-namespace gate (#2910): `java.*` 1041 imports and
`com.google.*` 1006, both resolving 0. COBOL's collide layout repeats a
bookname across BOTH extension tiers, so it reaches the copybook-over-source
tie-break rather than only the basename map.

**`csharp_csproj` is a sixth LANGS entry**, not a new arm dimension — an entry
needs five small additions and inherits all five arms and all seven gates,
where a context axis would have to be threaded through `buildRepo`,
`resolveAll`, `identityPass`, the report shape and every gate. `buildFiles`
aliases it to `csharp`, so the two share one corpus by construction and cannot
drift. Two configs (`{App, 'src'}`, `{Lib, ''}`) produce all three `dirPrefix`
shapes — slashed, slash-free and empty — in five arms instead of ten:

  App.Ns{d}      30.6%  src/Ns{d}        step 2 hit
  App.Missing{n} 25.5%  src/Missing{n}   step 3, last-segment bucket
  Lib            14.0%  (empty)          step 3, singleSegmentDirs
  Lib.Missing{n} 12.0%  Missing{n}       step 3, KEY SWEEP — the one
                                         non-constant path
  BCL / Ghost    12.4%  —                root-namespace-gate control

**2221 of 3200 imports reach the indexed leg**, only 12.4% `continue` out. What
that arm pins is stated plainly rather than overclaimed: step 3 answers null
for all 2221 here (the hits land at step 2), so it gates that leg's COST and
its null answers; its positive tie-breaks stay pinned by the unit parity test.

**Heap ceilings retightened.** #2903 dropped the measured figures, leaving the
1.5x ceilings at ~1.9x — a straight revert to the old size would have passed:

  csharp 116,000,000 -> 98,000,000 B   (measured 61.76 MiB)
  ruby    87,000,000 -> 69,000,000 B   (measured 43.69 MiB)
  php    new 106,000,000 B             (measured 67.29 MiB)
  java   new 154,000,000 B             (measured 97.32 MiB, the largest in the
                                        file — Maven layout is 18 segments)

php and java are gated because both retained NOTHING across imports at BASE and
now retain the O(files x depth) suffix index — the same argument that gates C#.
cobol is not: two `Map<basename, path>`, O(files) with no depth term, and its
retained delta does not clear measurement noise, so a ceiling would gate
nothing. `csharp_csproj` is not: same corpus, same index, a duplicate number —
its one distinguishing footprint, the lazily-built `dirMap` its `getFilesInDir`
forces back, is measured at +20.8% and recorded as a residual instead, because
gating it would licence eager-dirMap everywhere.

csharp's `depth_ratio` also fell 3.318 -> 2.31 (the no-csproj leg never asks a
directory question, so the deep arm stopped paying an eager dirMap build).
Budget 5 -> 3.5, restoring the file's 1.5x convention — and `_arms_note` says
plainly that 3.5 does NOT lock that win in, because locking it needs ~2.9,
which is 1.25x over a 1.05x spread and the kind of tightening `_triage` warns
buys flake rather than signal.

All five pre-existing languages are byte-identical: 25 cells x 5 fields = 125
values, 0 mismatches. The new arms were proven live by a doctored baseline
(cobol ceiling 0.01, php heap 1000 B, java resolved 999) producing three
correctly-worded failures and exit 1.

Wall-clock 10.9 -> 26.1 s, php and csharp_csproj ~11 s of it — both cascades
end in `suffixResolve`'s ~50-extension probe, and both gate the two largest
wins on this branch, so neither is a candidate to drop.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Co58j4au9JLf8dmJpwdF9B

* perf(javascript): build the suffix index JS resolution never had

JavaScript's `PassCache` was TypeScript's minus one field: `index`. So JS
called the shared `resolveTsTarget` with `ctx.index === undefined`, and
`import-resolvers/standard.ts` fell through to `suffixResolve`'s linear
`findIndex` — scanning the materialized path list once per extension (~39)
per path part, per import.

  2000 files   6448.9 -> 28.5 us/import   (TypeScript: 25.0)
  8000 files  25972.6 -> 27.4 us/import   (TypeScript: 27.0)

Per-import scaling over 4x the files: 4.12x -> 1.09x.

**Every instrument on this branch was blind to it.** `CountingSet` counts
traversals of the Set; this walked the array the adapter had already
materialized — the blind spot `counting-file-set.ts` documents in its own
header and `baselines.json` records under `_blind_spot`. Under mutation M1,
which drops `index` and reproduces the shipped defect exactly, the sixteen-
language contract test stays GREEN for javascript, because the pass cache is
still reused and `files.scans` reads 2 either way. Two new arms do catch it: a
`suffixResolve` linear-branch counter that runs the legacy adapter first as its
control (135 entries legacy, 0 now), and a mock-free behavioural assertion that
a repo-root module resolves by bare specifier.

Adding an index moves output, exactly as it did for PHP in #2901, so it was
characterized rather than assumed — 211,200 pairs (400 corpora x 3 importers x
176 targets) plus 184 hand cases. **Two classes move and there is no third:**

  A  null -> repo-root file (108)   `require('config')` with root `config.js`.
     The scan tests `endsWith('/' + suffix)`, so a path with no slash has no
     proper suffix and was unreachable through that leg — while `./config`
     from the root already resolved via the exact `Set.has` branch. JS was
     internally inconsistent.
  B  file -> different file (5679)  `import 'app/main'` was resolving to
     `node_modules/dep0/lib/main.js`; the scan skipped the whole-path candidate
     at the 2-segment suffix and fell through to the 1-segment `/main.js`,
     taking the first such file in Set order.
  C  hit -> null                     ZERO, and impossible: proper-suffix keys
     are a subset of the index's keys.

Both moved classes are JS being wrong. **JS-new agrees with TypeScript on all
211,200 pairs and every corpus case, 0 disagreements** — which is the intended
design, since JS delegates to the TS resolver and differed only by this field.

Also swaps the single-slot `let cached: PassCache | null` in JS, TS and Vue for
a module-level `WeakMap`, matching every other language. Two alternating file
sets rebuilt everything on every call: 12.0 -> 1438.2 ms at 4000 files x 400
imports (120x); after, 11.0 -> 15.7 ms. This is LATENT, not live —
`pipeline/run.ts:673` builds one Set per provider pass and the three are
separate providers — but it is why these were the only languages that could not
carry the standard distinct-set guard. They can now: the arm fails on HEAD for
all three (`expected 42 to be 2`) and passes after.

Six mutations caught, including a global `resolveCache` (M5), which needed a
new arm — `expectDistinctFileSetsGetOwnIndex` builds two IDENTICAL corpora, so
a stale answer carried between them is also the right answer.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Co58j4au9JLf8dmJpwdF9B

* refactor(ingestion): one per-file-set memo primitive, twenty-one call sites

Every language that indexes its import resolution hand-rolled the same memo:
declare a module-level `WeakMap` keyed on the file-set object, `get`,
`if undefined` build and `set`, return. One concept, written twenty-one times,
and this branch had just added five more.

`import-resolvers/per-file-set.ts` exports it once:

    perFileSet<K extends object, T extends object>(build: (key: K) => T): (key: K) => T

Two decisions, both recorded in the file. `T extends object` rather than
`has`-then-`get`: `WeakMap.get` returning `undefined` cannot distinguish "not
built" from "built as undefined", and the `has` form needs a cast or a non-null
assertion, both banned here — the constraint makes the ambiguous case
unrepresentable instead, and a future caller wanting `string | null` gets a
compile error pointing at the decision. A throwing build stores nothing and
runs again next call, so failures are not memoized and a half-filled index is
never published — inert for these pure builders, and the safer direction.

`K extends object` rather than `ReadonlySet<string>` is what lets C#'s
`readonly string[]`-keyed cache share the helper.

Twenty-one sites migrated across `import-resolvers/` and fifteen languages.
Every existing doc comment was re-homed onto the new call rather than deleted —
several record real invariants (the Set-identity contract, the #1918
pass-through rule, why Rust's memo lives on a different hook).

TypeScript, JavaScript and Vue additionally had byte-identical `PassCache`
interfaces and builders. `import-resolvers/pass-cache.ts` now holds the one
builder, taking a single argument — every difference the three have lives in
the CONSUMER (`tsconfigPaths`, the extension list), not the builder. The
builder is shared, the memo deliberately is not: each adapter keeps its own
`perFileSet`, hence its own index and its own `resolveCache`, because the three
disagree about what a specifier resolves to and one shared cache would hand a
language another language's answers. It buys no runtime reuse and the module
says so — each provider pass builds its own `allFilePaths` Set, so the three
are always different keys.

C and C++'s `augmentedFilePaths` was a two-LEVEL memo, and needed no new
abstraction: the outer memo's value is a function and a function is an object,
so `perFileSet(perFileSet(...))` composes. The two instances stay one per file,
and the reason is now in BOTH doc comments rather than only C++'s — cpp
delegates to `resolveCImportTarget`, whose `suffixIndex` is keyed on the
augmented set, so a shared memo would cross the two languages' indexes.

Two sites are deliberately NOT migrated, each with the reason written at the
declaration so the next sweep does not re-litigate them:
  - `configs/swift.ts` is a two-input memo keyed on one. `targets` is not
    derivable from the key; re-keying on `ctx` would force a banned non-null
    assertion or an unreachable fallback inside a memo builder.
  - `rust/qualified-call.ts` `MODULE_SCOPE_CACHE` is three inputs keyed on one,
    and sits ten lines below a `perFileSet` in the same file — the likeliest
    thing to be "fixed" by mistake.

The other ten remaining `WeakMap`s are different concerns and stay: AST-node
caches, worker-pool runtime state, graph metadata, mutable lazily-filled
accumulators, and the C++ ADL / inline-namespace indexes, which are reassigned
by explicit clear functions and epoch-stamped on read — validity rules beyond
key identity that a closure over a private cache cannot express.

Net −20 lines of code, +22 of the two "why not" notes. The primitive's own doc
is where the cost sits: the Set-identity contract and the two design decisions
are written once instead of being twenty-one implicit facts.

Pure refactor: 1764 unit tests, 42 guard tests, all sixteen contract-test
traversal counts unchanged (c 1, cobol 1, cpp 1, csharp 2, dart 1, go 1,
java 2, javascript 2, kotlin 1, php 1, python 1, ruby 1, rust 0, swift 1,
typescript 2, vue 2), 647 C/C++ tests, and every bench fingerprint unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Co58j4au9JLf8dmJpwdF9B

* test(import-target): gate every registered language, not nine of sixteen

The bench pinned output fingerprints and scaling for 9 of the 16 languages in
`SCOPE_RESOLVERS`. The other seven — c, cpp, javascript, python, rust, swift,
typescript, vue — resolve imports in production with nothing pinning their
output or their cost. JavaScript was the sharpest case: the 25,972 us/import
defect fixed earlier on this branch was gated by unit tests alone.

All 16 are now gated, plus the `csharp_csproj` variant: 17 entries.

**The nine existing languages are byte-identical** — 234 committed values
(9 x 5 arms x 5 fields, plus 9 top-level fingerprints), 0 changed, and no
pre-existing budget touched. Measured both before and after the memo
consolidation in e6f15274e, so it doubles as an independent check that the
refactor preserved behaviour.

Corpora keep both load-bearing rules — most imports MISS, and import count
scales with file count — at resolve rates of 26-36%. C and C++ follow the
`csharp_csproj` precedent: a `LANGS` entry carrying its own context (header
paths through `resolutionConfig`) over an aliased corpus, since cpp delegates
into C's `resolveCImportTarget`. Vue threads `tsconfigPaths` so its alias
branch actually runs; ts/js use bare specifiers only, because relative ones
never reach `suffixResolve`.

Two corrections to my own profiling, both verified rather than assumed:
Swift's `byModule` IS depth-scaled (one bucket entry per interior segment, not
O(files)), and Python's index is depth-free while its RESOLVER is quadratic in
depth — `hasRepoCandidate` and `resolveAbsoluteFromFiles` each rebuild one
ancestor prefix per importer directory component, per import. That is why
python's `depth_budget` is 11 against a 3.5 next-highest; the arm is pinning a
real defect rather than a comfortable number, and it is filed separately.

Rust's collide arm was redesigned rather than budgeted away: it is flat on file
count by construction, so a shared-leaf arm would have asserted nothing. Its
collide corpus varies `::` segment count — the axis its cost actually has — and
the linear 1.8 budget asserts the file-count flatness.

Heap: all 8 measured, 3 gated. javascript (44.07 MiB, retained nothing before
its fix), python (7.27 MiB), c (9.55 MiB). Five skipped with their numbers in
`_arms_note` rather than silently: rust 16 B (no index on this hook), swift
reads 3x SMALLER on a 4x corpus so it is below its own noise floor, typescript
288 B on 46 MB, vue +5.4%, cpp 0.04% from c.

Every gate type was proven able to fail: one run with 10 doctored values fired
10 correctly-worded failures across all 8 new languages, covering per-scale
fingerprint, shape/resolved, shape/distinct_outcomes on a non-small arm, depth,
collide scaling, absolute small ms, absolute collide ms, top-level fingerprint
and heap bytes. That proof found two wrong messages, now fixed: the heap
failure claimed a `buildSuffixIndex` cause that is false for python and c, and
the fingerprint failure pointed at a parity harness covering none of the eight.

Wall clock 26 -> 46 s. The ts/js/vue family is 14.6 s of the 18.8 s added,
because `suffixResolve` probes ~39 extensions per path part on a miss — the
real resolver, not something the bench can tune. Per language the bench got
cheaper (2.7 s vs 3.0 s). If it must shrink, `_arms_note` and the CI comment
record the one cut that removes duplicate work rather than coverage — drop
collide for typescript and vue only, -3.9 s, since all three share
`resolveTsTarget` and javascript keeps the arm covering their common axis.
Explicitly NOT `REPS`: it is 15 because `depth_ratio` flaked 1-in-20 at 5, and
lowering it would re-open that for all 17 languages.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Co58j4au9JLf8dmJpwdF9B

* perf(import-resolvers): stop building half of every suffix index

Applies the findings of a four-lane quality review over this branch.

**Half of `buildSuffixIndex` was dead weight for most of its consumers.**
Commit b6ee577e0 on this branch made the THIRD map (`dirMap`) lazy for exactly
this reason and left the two larger ones eager. Tracing every reader: Java and
no-csproj C# call `get` and never `getInsensitive`; PHP calls `getInsensitive`
and never `get`. Measured dead weight at 32k paths: Java 49.98 MiB of a 100.82
MiB index, PHP 34.49 of 69.85.

All three maps are now built on first use, and `lowerMap` is DERIVED from
`exactMap`'s insertion order rather than re-traversed — measured 330 ms against
389 ms today, so it is cheaper even for the two-map consumers. `pass-cache.ts`
hands the builder an already-lowercased list, so for TypeScript, JavaScript and
Vue the derivation is the identity and `getInsensitive` aliases the one map.

  java            80.26 -> 25.61 MiB retained   (-68%)
  csharp no-csproj 57.15 -> 21.52               (-62%)
  javascript       44.07 -> 22.65               (-49%)
  php              60.86 -> 32.09               (-47%)
  build @32k      562.1 -> 119.6 ms  (get-only), 329.5 ms (both)

The derivation is proven, not asserted: keys, values AND insertion order
byte-equal over 968,418 entries across case-colliding, Unicode-adversarial and
pathological corpora, plus 400 seeded-fuzz rounds. Order matters because it is
what makes `getInsensitive` return the first match in file order.

PHP additionally defers `filesByRawDirectory` (statically unreachable unless a
composer.json parses) and `firstProperSuffixMatch` (0 entries and 35.6 ms on
the bench corpus) to the branches that read them.

One suggested micro-optimisation was REJECTED with a counterexample rather than
taken: hoisting `suffixResolve`'s lowercase out of the extension loop assumes
`(s + ext).toLowerCase() === s.toLowerCase() + ext`, which is false for a
segment ending in Greek capital sigma — `("ΑΣ" + ".ts").toLowerCase()` is
`"ασ.ts"`, not `"ας.ts"`, because Final_Sigma is context-sensitive and `.` is
case-ignorable. A file named `ΑΣ.ts` would have stopped resolving. 16
mismatches in 2,171,190 checks, for 8.7%.

**The heap arms had become ceilings over nothing.** `retainedIndexBytes` read
only `index.all.length`, so once the maps went lazy it built none of them and
reported ~0 B — passing every ceiling. All heap arms now route through
`retainedPassBytes`, resolving a real missing import through the real resolver,
so the maps measured are the maps production forces. Two further measurement
defects surfaced while fixing it: PHP reaches the index through a second memo,
so the ephemeron chain needs four GC cycles and was reporting 249,208 B for a
9.3 MB index; and `bytes_large` carried an ~11% rope-flattening bias that made
every ratio read 0.85-0.96 for structures that are linear (now 0.998-1.017).

A `heap_floor_fraction` arm was added — a ceiling can only say "not too big" —
and proven by simulating the exact regression: `16 B at 32000 files < floor
17325000 B — this arm has almost certainly stopped MEASURING`.
`csharp_csproj` is now gated too: its old exclusion as "a duplicate of csharp"
held at +20.8% and is false at 2.47x.

**Three silent-coverage holes in the bench.** `LANGS` was a hand-written
literal claiming to mirror `SCOPE_RESOLVERS` while never importing it — the
seam that let JavaScript ship ungated; it is now derived, with an inventory arm
reconciling both directions. Four per-language budget lookups compared against
a possibly-`undefined` value, so deleting a key deleted the gate. Five
dispatchers ended in bare fallthroughs meaning "ruby" and "csharp", so a
mistyped language would have been benchmarked as Ruby's corpus under C#'s
resolver, forever green.

REPS is now chosen per language (15 below 5 ms, else `clamp(ceil(150/ms),7,15)`)
rather than globally by the noisiest cell: timing phase 39.8 -> 28.7 s, with the
six reduced-N languages showing peak-to-peak 1.008-1.071, no worse than the
eleven that kept 15. Worst headroom across all 85 cells is 0.71 of budget.

`depth_budget` for csharp 3.5 -> 2.2 and java 3.4 -> 2.2: their ratios fell to
1.438/1.402 because the lazy maps stop the deep arm paying for a map it never
reads. The file's own note said 3.5 did not lock that win in; 2.2 does.

Also fixes a raw NUL byte that made `suffix-index-lazy-dir-map.test.ts` BINARY
to git — all 395 lines were invisible to diff, blame and grep. The repo
documents this exact hazard in `route-extractors/dispatch-guard.ts`. That file
now also carries the guard the refactor lacked: eight arms pinning one-map-per
consumer and zero-extra-pass derivation, each proven against four mutations,
including a fused-eager rebuild that moves no total and is caught solely by the
at-construction count.

All 17 bench fingerprints and all 85 per-scale tuples unchanged. 1772 unit
tests, 12 adapter guards, tsc clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Co58j4au9JLf8dmJpwdF9B

* perf(python): memoize the importer's ancestor chain per directory (#2913)

Python's file index was always depth-free; the resolver was not.
`hasRepoCandidate` and `resolveAbsoluteFromFiles` each rebuilt one ancestor
prefix per directory component of the importer on EVERY import, and the
index's own `dirPrefixes` build inserted one entry per component per file.
So an import from `a/b/c/d/e/f/mod.py` did ~6x the prefix work of one from
`a/mod.py` regardless of corpus size — `depth_ratio` 7.239 where the next
worst language sat at 3.446.

The prefixes are a pure function of the importer's DIRECTORY, so they are
memoized per directory inside `getPythonFileIndex` (`ancestorsByDir`), which
is itself already per-file-set. Three smaller cuts came out of profiling the
same delta: the leading segment is rejected up front against a set of nested
directory names, the module and package buckets are consulted before the
walk instead of inside it, and the `dirPrefixes` build stops at the first
ancestor already stored.

Measured over 6 serial runs: depth_ratio 1.748-1.872 against 7.239, and at a
fixed 400 files the per-import cost at 18 directory components drops 6.761 ->
1.065 us. All five python fingerprints are byte-identical, so this is a
hoist; the budget retightening lands in the following commit, because
`_arms_note` is a single JSON line that also carries the heap-gate rewrite.

Also memoizes `pythonFileExportsName`'s `parsedFiles.find`, which was
O(files) for every import whose package probe resolved — the same shape
#2901 removed, keyed on `parsedFiles` rather than on `allFilePaths`.

The new gate is a count, not a timing: `ancestorsByDir.size` after N imports
from D directories must equal D, paired with a reference-identity assertion
so a memo that rebuilds AND re-stores still fails. `CountingSet` cannot see
this defect — the chain derives from the `fromFile` string and a rebuilt
prefix traverses the file set zero extra times.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NtGZN6YSNU8chALSnKYHBp

* fix(import-target): close the eleven findings from the #2911 review

Seven P2s and four P3s. Every one is a gate that could not fail or a
comment that had become false; no shipped behaviour defect was found, and
all 85 per-language fingerprints are unchanged.

GATES THAT COULD NOT FAIL

- The C# namespace-dir memo was keyed on a materialized array, so a
  one-character `[...normalized]` copy at the adapter boundary minted a
  fresh WeakMap key per import while traversing the file set zero extra
  times: 67 tests stayed green and only a timing ratio caught it.
  `resolveCSharpImportInternal` now takes the Set and derives both arrays
  from `getWorkspaceFileIndex`, so there is ONE key shape and ONE
  instrument. Copying the Set now turns three arms red. Established first
  that `configs/csharp.ts` is test-only (`buildImportTargetWorkspace` has
  no production caller) and that both derivations are byte-identical —
  otherwise the rekey would have been a behaviour change, not a hoist.

- The contract test called `resolveImportTarget` with four arguments where
  `pipeline/run.ts:682` passes five, so everything behind `context` was
  ungated for all 16 languages: defeating PHP's `filesByDirectory` memo
  cost 197.0 -> 9,976.2 us/import (50.6x) with 248/248 tests green.
  `CountingSet` provably cannot see it — the builder iterates the
  `parsedFiles` array and touches the Set zero times — so the new gate
  counts own-index reads on `parsedFiles` through a Proxy. Only PHP and
  Python have a context leg; the other fourteen carry the floor anyway.

- Three heap budgets were read with no presence check. `ceiling * undefined`
  is NaN and `bytes < NaN` is false, so deleting `heap_floor_fraction`
  disabled the floor for all eight arms; deleting `heap_ratio_budget` did
  the same; and iterating the baseline's keys dropped a language whose
  ceiling key was deleted out of the gate entirely. All three now fail
  closed with a message naming the broken comparison.

- `HEAP_PROBE_TARGET` decided what each heap arm measured and was compared
  to nothing: repointing csharp_csproj at a non-matching namespace dropped
  it 73.70 -> 59.92 MB with `--check` still exiting 0. The four corpus
  fields are now asserted through the loop the timing scales already use,
  and the floor derives from a recorded reading rather than from a ceiling
  that is itself 1.5x the measurement.

- About 35 of the 86 PHP parity arms were structurally unable to fail:
  both sides called the same production helper, so deleting the `..` guard
  left them green. Every hand case now pins an absolute literal as well as
  the differential. Eight of those literals pin a bug or a documented
  limitation and say so rather than blessing the value.

- The registry inventory arm was weighed and KEPT, against the review's
  suggestion, on a structural number rather than a timing: the benchmarks
  job runs 9m23s against a 12m58s critical path, so its seconds buy no
  merge latency, and moving the arm to vitest would put the registry load
  ON that path while weakening what it reconciles. The "7.3 s" and
  "~46 -> ~42 s" figures it was justified with are corrected, including
  stating that only report mode got faster.

- python's `depth_budget` drops 11 -> 2.6 now that #2913 is in. 1.39x the
  measured maximum rather than the file's usual 1.5x, deliberately: at 2.8
  a revert of the nested-name rejection (2.734) would pass. The two parts
  of that fix this arm cannot gate are named, with the count-based arms
  that do gate them.

COMMENTS THAT HAD BECOME FALSE

- `pass-cache.ts` said it deduplicated "three byte-identical copies".
  JavaScript's had five fields and never called `buildSuffixIndex` — that
  missing field IS this PR's headline defect.
- The per-language census said nine where it is twelve, three of them
  added by this PR. Replaced in seven places with the mechanism that
  enforces it, which cannot go stale.
- `getFilesInDir` handed out the index's live bucket. Now `readonly
  string[]`, so mutation is a compile error; `.slice()` was rejected
  because `configs/python.ts` reads only `.length` and a per-import copy
  would reintroduce the term this PR removes.
- #2910 is the Java in-repo-namespace gap, not the JavaScript index defect.
  13 references corrected, the one correct Java use left in place.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NtGZN6YSNU8chALSnKYHBp

* perf(python,bench): flatten the bare-import walk, measure the context leg

Two follow-ups the #2911 review surfaced but left open.

BARE IMPORTS (`import os`) still walked every ancestor of the importer.
#2913 fixed the dotted tier; this tier lives in `import-resolvers/python.ts`
and no bench arm can reach it, because every python arm here spells its
imports with a dot and returns at the `pathLike.includes('/')` guard.

It also ran TWICE per `from x import y`: `resolvePythonImportTarget` probed
the package with `targetIncludesImportedName: true`, and on null — the
expensive case, having already walked to the workspace root — fell through
to a byte-identical call. Established that the two cannot differ before
collapsing them: the flag's only effect is to skip
`pythonImportedSubmoduleTarget`, so the recursion re-runs the outer frame's
entire tail on the same three references, and reaching the fallthrough means
that tail already returned null.

The walk itself is now a memoized chain plus an O(1) proof of absence
against the index's basename buckets. Its chain is NOT the one #2913
memoized and the difference is semantic, not accidental — no
`filter(Boolean)`, self excluded, workspace root included — so under an
absolute-path workspace the unfiltered chain probes `/abs/a/` where a
filtered one would probe `abs/a/`, a prefix of nothing. Two negative arms
pin that in both directions. The shared index moved to
`import-resolvers/python-file-index.ts` rather than being reached across a
cycle, which also collapsed a standalone memo into the one per-file-set.

12 / 24 / 72 Set probes at depth 1 / 4 / 16 become a flat 2. At 18 path
components, 11.615 -> 0.740 us/import (15.7x) and the depth curve is gone:
7.843 -> 0.925. Gated by probe COUNT, not timing.

THE BENCH CALLED `resolveImportTarget` WITH THREE ARGUMENTS where
`pipeline/run.ts:682` passes five, so no timing arm entered the `context`
leg for any language. Arity checked against the registry rather than the
comment: php and python declare five, every other hook three or four.
`parsedFiles` is built first and `allFilePaths` derived from it, matching
`run.ts`; fresh per pass, because the memos behind that leg key on the
array identity and `fastest()` takes a min.

Python's `parsedFiles` was structurally unreadable, not merely unread: the
arm passed a `namespace` spelling, which makes `pythonImportedSubmoduleTarget`
return null before the context is consulted. The import KIND had to change
too.

No fingerprint moved anywhere — on this corpus PHP's leg returns the same
file the cascade already did — which is exactly why the new `context` arm
asserts with-context against without-context instead. Defeating PHP's
`filesByDirectory` memo now costs 1003.7 ms against a 148 ms budget; before
this the bench could not see it at all.

Re-recorded on a quiet box, maxima over 5 serial runs: php small 27.762 ->
34.023 and heap 37.6 -> 49.6 MB (`filesByDirectory` is now retained for the
pass), python small 1.76 -> 4.358. `depth_budget.python` moves 2.6 -> 2.2,
because the added work is depth-FLAT: absolute cost doubled while the ratio
FELL to 1.563, so the old budget had gone slack. Both lock-in figures were
re-measured under the new call shape rather than carried over — reverting
the ancestor memo scores 2.524, reverting the nested-name rejection 2.553,
so each fails at 2.2 with 13% to spare.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NtGZN6YSNU8chALSnKYHBp

* refactor(import-target): make the key-shape rule a type, drop three censuses

Cleanup pass over the #2911 review-fix commits. No behaviour change: all 85
per-language fingerprints, every `resolved` and every `distinct_outcomes` are
byte-identical, and the targeted suite is 1851/1851.

MEASURED — `byBasename` was 71% empty array slots

`byBasename` holds roughly one bucket per file, and building each with `[]`
followed by `push` makes V8 grow the backing store to its 16-slot minimum, so
every single-file bucket retained 15 empty pointer slots. Constructing the
one-element bucket directly is byte-identical in contents and 5.50 -> 1.60 MiB
at 32000 `.py` paths. The bench arm reads 10543848 -> 6360936 B (-39.7%);
`heap_reading_bytes.python` and its ceiling are re-recorded. The same edit
shares one `{ raw, norm }` between both maps instead of allocating a second
literal for every `__init__.py`.

THE RULE THAT COST A TIMING RATIO TO FIND IS NOW A COMPILE ERROR

`perFileSet`'s key is narrowed from `object` to
`ReadonlySet<string> | readonly ParsedFile[]`. Reintroducing the #2911 defect
shape — a memo keyed on an array materialized from the file set — now fails
with TS2345 instead of silently minting a fresh `WeakMap` key per import while
traversing the Set zero extra times, which every scan-counting guard reads as
green at its correct value.

That also retires the header's hand-maintained roster of `ParsedFile[]`-keyed
call sites, which listed three — this PR added a fourth in `395c707d4` and did
not update it. A census inside a comment warning that censuses go stale, stale
inside one commit. The header now names shapes; the compiler names sites.

Two more claims that had drifted from their code:

- `per-file-set.ts` asserted "No index derived from the file set is keyed on an
  ARRAY materialized from it". `configs/swift.ts` is, deliberately, with its
  reasons written down. Two files in one directory disagreeing is worse than
  either; the rule now states what the type rejects and names the exception.
- `SuffixIndex.getFilesInDir`'s doc explained that it returns the index's own
  bucket by reference. True of `buildSuffixIndex`; the other implementation of
  that interface, in `languages/php/import-target.ts`, returns a filtered copy.
  The interface now carries only the caller-facing contract (`readonly`, do not
  mutate) and the sharing rationale moved onto the implementation it describes.
- The contract test still described Python as having "NO memo on this key".
  `parsedFileByPath` landed in `395c707d4`; the floor of 1 is now its single
  build rather than a per-import scan.

DEDUP

`importerDirOf` replaces four copies of `replace / lastIndexOf / slice` — two in
production, where one was a memo KEY and the other a memo's query argument, so
the two per-directory memos in one index agreed only by inspection. The tests
keep their own verbatim derivation on purpose: importing production's would
make the key lookup agree by construction and hide a regression.

`buildParsedFiles` maps through `probeFile` instead of repeating its 7-field
literal 900 lines away; `requireNumericBudget` and `expectNoOrphanKeys` replace
three and three copies, with every per-arm `why` kept per-arm. The two Python
memo guards collapse onto shared arms in `test/helpers/counting-file-set.ts` —
1847 tests before and after, and both still go red under mutation.

SKIPPED, with reasons: dropping `normSet` for bucket scans (trades O(1) probes
on the hot path for ~1.6 MB against a 6.4 MB reading); measuring heap for all
17 languages (+9 s and a design decision, not a cleanup); `readonly` on the
five sibling resolvers' array parameters and the `getDirMap` slice/join rewrite
(both correct, both outside this diff).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NtGZN6YSNU8chALSnKYHBp

* perf(import-target): rewrite the dirMap build, gate heap for every language

The three items the /simplify pass deferred, plus what measuring them found.

`getDirMap` BUILD — 226.9 ms -> 173.1 ms at 32 000 paths

It built every key with `dirParts.slice(j).join('/')`: one parts array, one
slice array and one joined string per file per directory component, in the map
its own doc calls "by far the most expensive" of the three. Now a
`lastIndexOf` walk slicing substrings out of the original string — the same
rewrite `getExactMap` already records at 357.4 -> 264.5 ms.

The key set is identical, not merely equivalent: 272 956 keys over a 32 000
path corpus carrying absolute paths, leading/interior/trailing doubled
separators, Windows separators, extensionless files, dotfiles, dotted
directories and colons, run both slash-normalized and raw. Zero differences in
keys, in key INSERTION ORDER, in bucket contents, in bucket ORDER, or across
767 732 probes through the real index. Bucket order matters because `php.ts`
reads `[0]`.

READONLY on the per-pass shared arrays

`WorkspaceFileIndex.normalized`/`.all` and the `normalizedFileList`/
`allFileList` parameters of jvm, php, ruby, go and standard are now
`readonly string[]`. This PR already made that argument for one bucket
accessor; these are the two biggest arrays held for a whole pass, and the
blast radius of an in-place sort is larger. Types only — no cast, no copy —
and it let two pre-existing `as string[]` casts in
`languages/typescript/import-target.ts` be deleted rather than added to.

HEAP IS NOW MEASURED FOR ALL SEVENTEEN LANGUAGES, AND THE PROSE WAS WRONG

Nine were excluded on measurements taken once and never re-checked, with the
re-entry condition stated in a comment and watched by nothing. Measuring them:

- go, dart and kotlin had NO stated reason at all — the header said "six of
  seventeen" against a list of eight. kotlin retains 45.85 MiB, the
  second-largest reading in this file, larger than ruby's and java's;
- swift and cobol were recorded as below-noise (0.29 MB, 0 B). They read
  3.29 MB and 2.21 MB and grow the right way. The arm changed under them —
  #2903's real-import probe, then corpus flattening — and nobody re-took it;
- the header quoted javascript at two different values four paragraphs apart.

Only rust's exclusion survived: 16 B at both scales, identical over five runs.

Six of the nine are now FULLY budgeted rather than merely bounded — ceiling,
floor and ratio — because each grows linearly (0.996-1.004 against a 1.25
budget). cobol, swift and rust keep an upper bound and no floor, deliberately:
a floor over a reading at or below its own noise gates the noise. Proven live:
restating kotlin's reading so its floor clears the real measurement fails with
"this arm has almost certainly stopped MEASURING rather than started saving" —
the failure that once left four arms at 0 B under passing ceilings.

Cost: +1.37 s in the heap phase, measured per language rather than asserted.

`normSet` was NOT removed, and the reason is now in the code. It is derivable
from the two buckets, but `byBasename` is keyed on BASENAME: on a 9 000-file
service tree `utils.py` and `models.py` hold 1 000 entries each, so `import
utils` would scan every `utils.py` in the workspace per import — the exact
defect class #2901/#2902/#2908 removed. ~1.6 MB against a 6.4 MB reading buys
both probes staying O(1).

All 85 per-language fingerprints unchanged; 1854 tests pass.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NtGZN6YSNU8chALSnKYHBp

* test(php): drop the impossible undefined comparison from the parity copy

CodeQL (js/comparison-between-incompatible-types, alert 945) flags the
`ctx === undefined` arm of the legacy adapter copy: `WorkspaceIndex` is an
object type at that position, so the comparison can never be true.

Optional chaining expresses the same guard without the type-level clash —
an undefined index still fails the `typeof` test and returns null — so the
copy remains behaviourally verbatim against the shipped adapter, which is
the only property this harness relies on.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017oEY2i74d1HLa5FuGVuPiT

---------

Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Gergő Magyar 2026-08-10 17:22:51 +01:00 • committed by GitHub
parent 4576adfc46
commit 18bc51dfd2
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
60 changed files with 12501 additions and 985 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

112
.github/workflows/ci-tests.yml vendored
View file

@ -529,47 +529,101 @@ jobs:
run: node --import tsx bench/cpp-qualified-ns/measure.mjs --check
working-directory: gitnexus
- name: Import-target resolution guards (#2877/#2878/#2879/#2880/#2872)
# Build-free: runs the Go/C#/Dart/Ruby/Kotlin import-target resolvers
# over ONE shared corpus and asserts each returns an unchanged target
# set (a fingerprint per language AND per arm), that per-import cost
# stays independent of corpus size AND of path depth, that the absolute
# small-arm cost holds — a constant-factor regression that grows both
# scale arms equally passes every ratio — and that the shared
# WorkspaceFileIndex C# and Ruby retain stays within an absolute byte
# ceiling. Each of those resolvers used to scan the whole workspace per
# import (Ruby rebuilt a suffix index per `require`), so resolution was
# O(imports × files); the same corpus shape scores >3.3 against the
# pre-fix implementations. The corpus SHAPE is asserted too — a
# fingerprint alone cannot tell a legitimate resolution change from a
# corpus quietly shrunk below the size the timing arms need.
- name: Import-target resolution guards (every registered language, #2877–#2909, PR #2911)
if: ${{ !cancelled() }}
# Build-free: runs EVERY import-target resolver registered in
# SCOPE_RESOLVERS — plus C# a second time WITH csproj configs, over the
# identical corpus, because the no-csproj arm returns before it can
# reach the leg #2902 indexed. One arm per registered language over ONE
# shared corpus, and no registered language ungated. That is enforced,
# not enumerated: measure.mjs derives its list from a LANG_REGISTRY
# table and its --check inventory arm reconciles that table against
# SCOPE_RESOLVERS in both directions, so a language roster typed out
# here would only be a second copy that can go stale — this one did.
# A C/C++ #include is an import site for this purpose and is gated like
# every other registered language (its headers arrive through
# resolutionConfig rather than allFilePaths, which is the one structural
# difference — see `newPass`).
#
# Asserts each returns an unchanged target set (a fingerprint per
# language AND per arm), that per-import cost stays independent of
# corpus size AND of path depth, that the absolute small-arm cost holds
# — a constant-factor regression that grows both scale arms equally
# passes every ratio — and that the per-pass index eight of them retain
# stays within an absolute byte ceiling. The corpus SHAPE is asserted
# too: a fingerprint alone cannot tell a legitimate resolution change
# from a corpus quietly shrunk below the size the timing arms need.
#
# Several arms exist because an arm that stops MEASURING otherwise
# passes. The heap arms drive real resolvers and carry a FLOOR as well
# as a ceiling: when buildSuffixIndex's suffix maps went lazy, four arms
# that called the builder directly read 0 B, and 0 B is under every
# ceiling. EVERY budget is checked for PRESENCE first, timing and heap
# alike, because `got > undefined` is false and `got < ceiling *
# undefined` is false too, so deleting a budget key deleted its gate —
# and the two heap scalars gate all eight heap arms at once. The heap
# arm's own corpus shape (its two file counts, its path depth and the
# probe it resolves) is asserted by the same loop as the timing arms,
# because those four decide WHAT it measures. And an inventory arm
# reconciles the bench's language table against SCOPE_RESOLVERS itself,
# so a newly registered resolver cannot ship ungated the way JavaScript
# did.
#
# The resolvers gated first were added as their own O(imports × files)
# scans were indexed away (Ruby rebuilt a suffix index per `require`;
# COBOL scanned twice per `COPY`), and the same corpus shape scores >3.3
# against those pre-fix implementations. The rest were ungated until
# this PR, which is not a theoretical gap: PR #2911 found JavaScript
# reaching suffixResolve with no index at all — 25 972 µs per import at
# 8000 files, protected only by unit tests. This step is what stops the
# next one shipping.
#
# SCOPE: "independent of corpus size" holds for UNIQUE-LEAF layouts,
# where no two directories share a last segment and no two files share a
# basename — which is what the small/large/deep arms are, and where
# every index bucket holds exactly one entry. The `collide` arm runs the
# identical workload on the layout these languages are actually written
# in (svcN/internal, SrcN/Models, a repeated basename per package);
# there the bucket grows with the file count by construction and go,
# csharp and dart legitimately score 2.1–3.9, so that arm carries its
# own per-language budget. It is a scope limit, not a regression — the
# indexed code is still faster on that shape than the pre-change scan.
# in (svcN/internal, SrcN/Models, a repeated basename per package, four
# SPM modules instead of fifty); there the bucket grows with the file
# count by construction and go, csharp, dart, java, swift and c/cpp
# legitimately score 2.1–3.9, so that arm carries its own per-language
# budget. It is a scope limit, not a regression — the indexed code is
# still faster on that shape than the pre-change scan. Rust is the one
# language whose collide arm is NOT a shared-leaf layout: it probes
# candidate paths and is provably flat in the file count, so its arm is
# a deep module tree that varies `::` segment count instead — the axis
# its cost actually has.
#
# --expose-gc enables the retained-heap arm; --check REFUSES to run
# without it rather than passing with the memory gate silently skipped.
# ~14 s. REPS is 15 (matching bench/cfg) rather than a cheaper 5 or 7
# because depth_ratio divides two sub-3 ms numbers and at those settings
# it tripped its own budget roughly 1 run in 20 — the estimator was
# fixed instead of the budget widened; distributions in _arms_note.
# ~44–45 s, which is essentially unchanged from the ~46 s it cost
# before: the timing phase did fall from 39.8 s to 28.7 s when the
# min-of-N estimator became per-language, but the inventory arm's one
# dynamic import (pipeline/registry.ts pulls in every registered
# provider) costs 6–10 s depending on the box and consumes almost all of
# that. Report mode, which does not load the registry, is the mode that
# got faster: ~33–35 s. Kept as-is because this job runs minutes clear
# of the sharded coverage job that gates the merge, so the seconds buy
# no merge latency — see COST in the bench header. The ts
# family (javascript/typescript/vue) is still the largest block, 8.8 s,
# because suffixResolve probes ~39 extensions per path part on a miss.
# If this ever has to shrink, drop collide/collide_large for typescript
# and vue (−3.9 s) — the only cut that removes near-duplicate work
# rather than coverage. N is 15 (matching bench/cfg) for every language
# whose cheapest arm is under 5 ms, because depth_ratio divides two
# sub-3 ms numbers and at 5 or 7 it tripped its own budget roughly 1 run
# in 20; the six languages whose cheapest arm is 20-28 ms drop to 7-8,
# where the measured overshoot is at most 6.3%. The estimator was fixed
# rather than the budget widened; distributions in _arms_note.
# The Kotlin arm here is a second corpus, not a replacement for the
# kotlin-import-target bench below, which carries tie-break probes (both
# file-set iteration orders, the four-tier cascade) this one does not.
# A failing step aborts every step after it in this job (#2895), which
# cuts both ways: parking a new gate at the end is not safety, it is the
# slot least likely to execute. This one sits with the other
# resolver-index guards; the estimator fix above is what makes that
# safe, and #2899 carries the `if: ${{ !cancelled() }}` that fixes the
# masking for every step at once.
# It sits with the other resolver-index guards rather than at the end of
# the job: parking a new gate last is not safety, it is the slot least
# likely to execute (#2895 measured the last two guards running zero
# times in 13 runs). #2899 landed the `if: ${{ !cancelled() }}` below,
# which is what makes position irrelevant — a failing step no longer
# aborts the ones after it.
# Rationale, budgets and the measured blind spot: see the header of
# measure.mjs and _blind_spot in baselines.json.
run: node --expose-gc --import tsx bench/import-target/measure.mjs --check

822
gitnexus/bench/import-target/baselines.json
View file

File diff suppressed because one or more lines are too long

2245
gitnexus/bench/import-target/measure.mjs
View file

File diff suppressed because it is too large Load diff

6
gitnexus/src/core/ingestion/import-resolvers/configs/csharp.ts
View file

@ -31,8 +31,10 @@ export const csharpNamespaceStrategy: ImportResolverStrategy = (rawImportPath, _
const resolvedFiles = resolveCSharpImportInternal(
rawImportPath,
csharpConfigs,
ctx.normalizedFileList,
ctx.allFileList,
// The Set, not `ctx.normalizedFileList`/`ctx.allFileList`: the resolver
// derives both from it through the same per-pass memo the ctx's own arrays
// come from, so this is the identical pair by a shorter route.
ctx.allFilePaths,
ctx.index,
evidence,
);

13
gitnexus/src/core/ingestion/import-resolvers/configs/swift.ts
View file

@ -39,6 +39,19 @@ interface SwiftTargetIndex {
* stable reference and the index is built once — not once per import. A
* fresh run produces a fresh array → a fresh index, so cross-run staleness
* is impossible.
*
* DELIBERATELY NOT ON `import-resolvers/per-file-set.ts` (#2909 sweep): this is
* a TWO-input memo keyed on ONE of them. The index is a function of both
* `ctx` (`allFileList` + the index-aligned `normalizedFileList`) and `targets`,
* but the key is only `ctx.allFileList`, and `perFileSet`'s `build: (key) => T`
* hands the builder nothing but the key. It is sound here only because of an
* invariant OUTSIDE the memo — `targets` is `ctx.configs.swiftPackageConfig
* .targets`, so it shares `ctx`'s lifetime and cannot vary while
* `ctx.allFileList` is fixed — and `perFileSet` has no way to express "and this
* other input is pinned by the same lifetime". Re-keying on `ctx` to make
* `targets` derivable from the key would change what the cache is keyed on and
* force an unreachable null-config arm into the builder, so it is a behaviour
* change rather than a consolidation. Leave it hand-rolled.
*/
const SWIFT_TARGET_INDEX_CACHE = new WeakMap<object, SwiftTargetIndex>();

265
gitnexus/src/core/ingestion/import-resolvers/csharp.ts
View file

@ -5,11 +5,225 @@
* This file contains shared helpers for namespace-based resolution.
*/
import { perFileSet } from './per-file-set.js';
import { getWorkspaceFileIndex } from './workspace-file-index.js';
import type { SuffixIndex } from './utils.js';
import { suffixResolve } from './utils.js';
import type { CSharpProjectConfig, CSharpNamespaceEvidence } from '../language-config.js';
import { csharpSuffixFallbackAllowed } from '../csharp-namespace-gate.js';
/**
* Directory index backing the namespace-directory fallback below (step 3).
*
* That fallback used to be a full `normalizedFileList` pass per import, per
* matching csproj config — Θ(files), measured at ~1.08 ms per import over
* 50 000 `.cs` files (#2902). #2878 removed the per-import array REBUILD but
* not the scan itself.
*
* The scan's predicate depends only on the file's DIRECTORY, so it can be
* answered from an index built once per file list. Writing `D` for the
* normalized directory of a `.cs` file and `dirPrefix` for the query:
*
* let H = D + '/', P = dirPrefix + '/'
* match ⟺ H.length >= P.length && H.indexOf(P) === H.length - P.length
*
* Derivation, because both halves are load-bearing:
* - the scan keeps a file only when nothing after the matched occurrence holds
* a slash, so the occurrence's trailing '/' must be the file's LAST slash —
* i.e. `H` ends with `P`;
* - it uses `indexOf`, the FIRST occurrence, so `a/Models/b/Models/x.cs` does
* NOT answer `Models`: the first `Models/` is found and `b/Models/x.cs`
* still contains a slash. Dropping that half moves edges in every repo that
* nests a directory name inside itself.
* - the needle ends with '/', so every occurrence of it lies wholly inside
* `D + '/'` and never reaches into the file name — which is what lets the
* whole test be evaluated on `D` alone.
*
* NOT the same query as `package-dir-index.ts`, and the difference is exactly
* one character on each side: that module tests `'/'+D+'/'` against
* `'/'+pkgPath+'/'`, whose leading slash anchors the match to a segment
* boundary. This scan has no leading slash, so `dirPrefix = 'Models'` also
* matches `src/SubModels/` and `dirPrefix = 'src/Models'` also matches
* `vendor/mysrc/Models/`. Those hits are reachable (step 2 below answers only
* the segment-aligned ones, and step 3 runs precisely when step 2 found
* nothing), so the looser predicate is preserved verbatim rather than
* "cleaned up" into a reuse of `filesDirectlyInPkgDir` — see
* `test/unit/import-resolvers/csharp-csproj-parity.test.ts`.
*
* Candidates are narrowed by the directory's LAST segment, the same
* O(directories) bucket `package-dir-index.ts` uses instead of an
* O(files × depth) suffix map (#2649).
*/
interface CsharpNamespaceDirIndex {
/** Last path segment of a directory → every `.cs` directory ending in it. */
readonly dirsByLastSegment: ReadonlyMap<string, readonly string[]>;
/**
* Directory → positions in `WorkspaceFileIndex.normalized` of the `.cs` files
* directly inside it, ascending.
*
* Positions rather than paths: the emitted value is the RAW path, and the two
* arrays are parallel by construction — `normalized` is `all.map(slash)` — so
* a position is the one key that reads correctly in either. Both arrays come
* from the same `getWorkspaceFileIndex(allFilePaths)` object as this index
* itself, so the pairing cannot drift; it used to be a precondition on the
* caller, who passed the two arrays independently.
*/
readonly positionsByDir: ReadonlyMap<string, readonly number[]>;
/**
* Directories with no slash of their own — the entire answer to an empty
* `dirPrefix`, which is the one query no last-segment bucket expresses.
*/
readonly singleSegmentDirs: readonly string[];
}
/**
* Memoized on the file SET's identity, the same key every other per-file-set
* index in this pipeline uses: the orchestrator builds one Set per pass and
* threads it through every import, so this build runs once.
*
* It used to key on the `normalizedFileList` ARRAY, which was a second key
* shape and — more to the point — one no guard could instrument. Copying an
* array mints a fresh `WeakMap` key while traversing the SET zero extra times,
* so a `[...normalized]` copy at the adapter boundary rebuilt this index once
* per `using` while every scan-counting guard stayed green and only the timing
* bench noticed (#2911 review). Taking the array from
* `getWorkspaceFileIndex(allFilePaths)` inside the builder retires that shape:
* the only way to defeat the memo now is to copy the Set, which is exactly what
* `CountingSet` counts.
*
* It also retires a precondition. The cached positions index `normalized` while
* the emitted value is read from `all`; both now come from the same
* `getWorkspaceFileIndex` object, so the caller can no longer pair a position
* list against a differently-ordered array.
*/
const getCsharpNamespaceDirIndex = perFileSet(
(allFilePaths: ReadonlySet<string>): CsharpNamespaceDirIndex => {
const { normalized: normalizedFileList } = getWorkspaceFileIndex(allFilePaths);
const dirsByLastSegment = new Map<string, string[]>();
const positionsByDir = new Map<string, number[]>();
const singleSegmentDirs: string[] = [];
for (let i = 0; i < normalizedFileList.length; i++) {
const normalized = normalizedFileList[i];
if (!normalized.endsWith('.cs')) continue;
const lastSlash = normalized.lastIndexOf('/');
// A file with no directory can never match: the needle always ends with
// '/', so `indexOf` on a slash-free path is always -1.
if (lastSlash < 0) continue;
const dir = normalized.slice(0, lastSlash);
let positions = positionsByDir.get(dir);
if (positions === undefined) {
positions = [];
positionsByDir.set(dir, positions);
const lastSegment = dir.slice(dir.lastIndexOf('/') + 1);
if (lastSegment === dir) singleSegmentDirs.push(dir);
let dirs = dirsByLastSegment.get(lastSegment);
if (dirs === undefined) {
dirs = [];
dirsByLastSegment.set(lastSegment, dirs);
}
dirs.push(dir);
}
positions.push(i);
}
return { dirsByLastSegment, positionsByDir, singleSegmentDirs };
},
);
/**
* Every directory that could satisfy `dirPrefix`, as a superset — the exact
* test runs in `matchingDirPositions`.
*
* When `dirPrefix` contains a '/', its own slash forces a segment boundary in
* any matching directory: `H` ending with `…/<lastSeg>/` means `D` ends with
* `/<lastSeg>`, so `D`'s last segment IS `lastSeg` and the exact bucket is
* complete. Without a '/', `D`'s last segment only has to END with `dirPrefix`
* (`SubModels` for `Models`), which no single bucket holds, so the last-segment
* KEYS are swept. That is the one term here that is not O(matches), and it is
* O(distinct last segments), not O(directories): C# repos reuse `Models`,
* `Services`, `Controllers` under every project, so the sweep collapses on the
* layouts that actually occur. Measured at 200 000 `.cs` files, 25 000
* directories: 456 µs per import when every directory name is unique, 7.9 µs
* on a `SrcN/Models` layout. Closing the unique-name case needs a character-
* suffix map over the segments, which is the O(files × depth) memory shape
* `package-dir-index.ts` cites #2649 to avoid — a design change, not a tune.
*
* An empty `dirPrefix` would sweep every key and keep every directory, so it is
* answered from `singleSegmentDirs` instead: its needle is a bare '/', which
* only a slash-free directory can carry as its LAST slash.
*/
function* candidateDirs(index: CsharpNamespaceDirIndex, dirPrefix: string): Generator<string> {
if (dirPrefix === '') {
yield* index.singleSegmentDirs;
return;
}
const lastSlash = dirPrefix.lastIndexOf('/');
if (lastSlash >= 0) {
const bucket = index.dirsByLastSegment.get(dirPrefix.slice(lastSlash + 1));
if (bucket !== undefined) yield* bucket;
return;
}
for (const [lastSegment, dirs] of index.dirsByLastSegment) {
if (!lastSegment.endsWith(dirPrefix)) continue;
yield* dirs;
}
}
/** Positions of the `.cs` files in each directory matching `dirPrefix`. */
function* matchingDirPositions(
index: CsharpNamespaceDirIndex,
dirPrefix: string,
): Generator<readonly number[]> {
const needle = dirPrefix + '/';
for (const dir of candidateDirs(index, dirPrefix)) {
const haystack = dir + '/';
// The length guard is not redundant: for a shorter `haystack`, `indexOf`
// returns -1 and `haystack.length - needle.length` can also be -1, which
// would report a bogus match.
if (haystack.length < needle.length) continue;
if (haystack.indexOf(needle) !== haystack.length - needle.length) continue;
const positions = index.positionsByDir.get(dir);
if (positions !== undefined) yield positions;
}
}
/**
* Append every `.cs` file directly inside a directory matching `dirPrefix`, in
* `normalizedFileList` order — the order the single-pass scan emitted, which
* this function's callers return as the whole edge target list.
*/
function pushFilesDirectlyInNamespaceDir(
index: CsharpNamespaceDirIndex,
dirPrefix: string,
allFileList: readonly string[],
results: string[],
): void {
// One matching directory is the overwhelmingly common case, and its positions
// are already ascending, so the first bucket is held by reference. A second
// one promotes it to a real accumulator that is appended to from then on —
// never re-spread per directory, which would cost O(files × dirs²) copies in
// a monorepo carrying the same namespace directory under many projects.
let first: readonly number[] | null = null;
let merged: number[] | null = null;
for (const positions of matchingDirPositions(index, dirPrefix)) {
if (first === null) {
first = positions;
continue;
}
if (merged === null) merged = [...first];
for (const position of positions) merged.push(position);
}
if (first === null) return;
if (merged === null) {
for (const position of first) results.push(allFileList[position]);
return;
}
merged.sort((a, b) => a - b);
for (const position of merged) results.push(allFileList[position]);
}
/**
* Resolve a C# using-directive import path to matching .cs files (low-level helper).
* Tries single-file match first, then directory match for namespace imports.
@ -17,15 +231,23 @@ import { csharpSuffixFallbackAllowed } from '../csharp-namespace-gate.js';
* The final unanchored suffix fallback is gated on `evidence` so BCL usings
* (e.g. `System.Threading.Tasks`) can't match a coincidentally-named local
* file (#1881). When `evidence` is omitted the fallback stays permissive.
*
* Takes the file SET, not the two materialized lists it used to take: both are
* derived here from the per-pass `getWorkspaceFileIndex` memo, which is where
* every caller already got them. That leaves one key shape for the indexes
* below and makes the `normalized`/`all` pairing structural rather than a
* contract the caller has to honour. `index` stays a parameter — the parity
* harness drives this resolver with and without one, and the no-index legs are
* a tested dimension, not a degenerate case.
*/
export function resolveCSharpImportInternal(
importPath: string,
csharpConfigs: CSharpProjectConfig[],
normalizedFileList: string[],
allFileList: string[],
allFilePaths: ReadonlySet<string>,
index?: SuffixIndex,
evidence?: CSharpNamespaceEvidence,
): string[] {
const { normalized: normalizedFileList, all: allFileList } = getWorkspaceFileIndex(allFilePaths);
const namespacePath = importPath.replace(/\./g, '/');
const results: string[] = [];
@ -75,21 +297,30 @@ export function resolveCSharpImportInternal(
if (results.length > 0) return results;
}
// 3. Linear scan fallback for directory matching
if (results.length === 0) {
const dirTrail = dirPrefix + '/';
for (let i = 0; i < normalizedFileList.length; i++) {
const normalized = normalizedFileList[i];
if (!normalized.endsWith('.cs')) continue;
const prefixIdx = normalized.indexOf(dirTrail);
if (prefixIdx < 0) continue;
const afterDir = normalized.substring(prefixIdx + dirTrail.length);
if (!afterDir.includes('/')) {
results.push(allFileList[i]);
}
}
if (results.length > 0) return results;
}
// 3. Directory matching, UNANCHORED.
//
// Not redundant with step 2, and not skippable when `index` is present:
// `getFilesInDir` is keyed on SEGMENT suffixes of a directory, while this
// leg's predicate is an unanchored substring one, so it additionally
// answers `Models` with `src/SubModels/` and `src/Models` with
// `vendor/mysrc/Models/`. It is also the only leg that answers an empty
// `dirPrefix` — the `relative = ''` branch above (the import IS the root
// namespace) with no `projectDir` to stand in for it — because
// `buildSuffixIndex` emits an empty directory suffix only for a path that
// BEGINS with '/', so over repo-relative paths `getFilesInDir('', '.cs')`
// is always empty. See `CsharpNamespaceDirIndex` above for the index that
// replaced the per-import Θ(files) scan this used to be (#2902).
//
// `results` is provably empty here: step 2 returns as soon as it pushes
// anything, and so does this leg, so every iteration of the config loop
// starts empty.
pushFilesDirectlyInNamespaceDir(
getCsharpNamespaceDirIndex(allFilePaths),
dirPrefix,
allFileList,
results,
);
if (results.length > 0) return results;
}
// Fallback: suffix matching without namespace stripping (single file).

4
gitnexus/src/core/ingestion/import-resolvers/go.ts
View file

@ -25,8 +25,8 @@ export function resolveGoPackageDir(importPath: string, goModule: GoModuleConfig
export function resolveGoPackage(
importPath: string,
goModule: GoModuleConfig,
normalizedFileList: string[],
allFileList: string[],
normalizedFileList: readonly string[],
allFileList: readonly string[],
): string[] {
if (!importPath.startsWith(goModule.modulePath)) return [];

8
gitnexus/src/core/ingestion/import-resolvers/jvm.ts
View file

@ -31,8 +31,8 @@ export const appendKotlinWildcard = (importPath: string, importNode: SyntaxNode)
*/
export function resolveJvmWildcard(
importPath: string,
normalizedFileList: string[],
allFileList: string[],
normalizedFileList: readonly string[],
allFileList: readonly string[],
extensions: readonly string[],
index?: SuffixIndex,
): string[] {
@ -90,8 +90,8 @@ export function resolveJvmWildcard(
*/
export function resolveJvmMemberImport(
importPath: string,
normalizedFileList: string[],
allFileList: string[],
normalizedFileList: readonly string[],
allFileList: readonly string[],
extensions: readonly string[],
index?: SuffixIndex,
): string | null {

79
gitnexus/src/core/ingestion/import-resolvers/pass-cache.ts Normal file
View file

@ -0,0 +1,79 @@
import { buildSuffixIndex, type SuffixIndex } from './utils.js';
/**
* Everything the standard `resolveTsTarget` path derives from one workspace
* file set: the file list, the lower-cased file list, the suffix index and the
* per-pass `resolveCache`.
*
* Without this memoization the resolver re-derived `allFileList` and
* `normalizedFileList` (both O(N_files)), rebuilt the index and threw away the
* `resolveCache` on every import — O(N_files × N_imports) total work for what
* should be O(N_files + N_imports).
*/
export interface ImportPassCache {
readonly allFilePaths: Set<string>;
readonly allFileList: readonly string[];
readonly normalizedFileList: readonly string[];
readonly index: SuffixIndex;
readonly resolveCache: Map<string, string | null>;
}
/**
* Build that state. Shared by every adapter whose resolution runs through
* `resolveTsTarget`.
*
* Not a dedup of identical copies, and the difference is the point. At
* 49c5b7d81 each of those adapters carried this record inline and they did NOT
* agree: `languages/typescript/scope-resolver.ts` and
* `languages/vue/import-target.ts` held six byte-identical fields built around
* `index: buildSuffixIndex(normalizedFileList, allFileList)`, while
* `languages/javascript/import-target.ts` held five and never called
* `buildSuffixIndex` at all. That one missing field IS the O(imports × files)
* defect PR #2911 fixed — `resolveTsTarget` fell back to `suffixResolve`'s
* linear scan for every JavaScript import — and the header of
* `languages/javascript/import-target.ts` carries the measurements. Hoisting
* the builder is what makes a fourth adapter unable to omit it again: `index`
* is not optional on `ImportPassCache`.
*
* The BUILDER is shared; the MEMO deliberately is not. Each adapter wraps this
* in its own `perFileSet(...)`, so each gets its own `WeakMap`, its own index
* instance and — the one that would be a behaviour change — its own
* `resolveCache`. The languages disagree about what a specifier resolves to
* (`tsconfigPaths` is read from config for TypeScript and Vue, pinned to `null`
* for JavaScript, and the tried extension list differs), so one shared resolve
* cache across them would hand a language another language's answers.
*
* Sharing the builder is a code dedup and nothing more: it buys no runtime
* reuse, because there is none to buy. Each provider pass builds its own
* `allFilePaths` Set (`scope-resolution/pipeline/run.ts`, per provider), so
* TypeScript's set and JavaScript's set are different objects and therefore
* different `WeakMap` keys even where the two memos are the same code.
*/
export function buildImportPassCache(allFilePaths: ReadonlySet<string>): ImportPassCache {
const allFileList = Array.from(allFilePaths);
// LOWERCASED, not slash-normalized — unlike every other caller of
// `buildSuffixIndex`. That is what `alreadyLowercased` below records.
const normalizedFileList = allFileList.map((f) => f.toLowerCase());
return {
// Copied ONCE per file set, not once per import: `TsResolveContext` wants a
// mutable `Set` and the orchestrator hands us a `ReadonlySet`. The copy is
// not the #1918 hazard because the cache KEY is the caller's original Set.
allFilePaths: new Set(allFilePaths),
allFileList,
normalizedFileList,
// Every suffix of an all-lowercase path is itself lowercase, so the index's
// case-folded map came out a byte-for-byte copy of its exact map — same
// keys, same values, same insertion order — one per `ImportPassCache`, so
// once per adapter per pass. Measured 14.00 MiB at 32 000 paths, 29.8% of
// the retained `ImportPassCache`. The flag drops the copy; it does not change
// what `getInsensitive` answers, because the copy was the identity (see
// `SuffixIndexOptions`). Checked, not assumed: over 474 524 probes on four
// mixed-case corpora — Vue PascalCase plus alias specifiers, case-colliding
// twins, a 600-file deep monorepo, and Unicode paths carrying final sigma,
// dotted-I and sharp-S — the two maps came out byte-identical, the exact
// map was the sole answerer 0 times, and `get(s) || getInsensitive(s)`
// returned the same file 474 524 times out of 474 524.
index: buildSuffixIndex(normalizedFileList, allFileList, { alreadyLowercased: true }),
resolveCache: new Map(),
};
}

83
gitnexus/src/core/ingestion/import-resolvers/per-file-set.ts Normal file
View file

@ -0,0 +1,83 @@
/**
* The one memo every per-file-set index in this pipeline is built on.
*
* The scope-resolution orchestrator builds ONE file-set object per provider
* pass and threads that same object through every `resolveImportTarget` call in
* the pass, so anything derived from it — a suffix index, a package-directory
* map, a basename bucket — can be built once and read by every import instead
* of rebuilt per import. Keying on the object's IDENTITY is what makes that
* work, and it is equally the contract callers must keep: the set is passed
* THROUGH, never copied. A defensive `new Set(allFilePaths)` at an adapter
* boundary hands a fresh key per import and silently restores
* O(imports × files) — the bug PR #1918 shipped and had to fix in review (P1).
* The guards are `test/integration/<lang>-import-index-reuse.test.ts` and, for
* every registered language at once,
* `test/unit/scope-resolution/import-target-index-reuse.contract.test.ts`,
* whose inventory arm fails when an entry of `SCOPE_RESOLVERS` has no fixture.
* That arm is why no language is named here: the registry is the census, and a
* hand-copied list of languages goes stale the release after it is written.
*
* A `WeakMap` rather than a `Map`: the entry is reclaimed with the file set it
* was derived from, so a pass can never read a previous pass's index and memory
* does not grow across runs. There is no invalidation rule to get wrong because
* there is nothing to invalidate — a new file set is a new key.
*
* The KEY TYPE is constrained rather than described, because which object is
* the key decides whether the guards above can see the memo fail, and a prose
* list of call sites is the thing this file elsewhere tells you not to write.
* `K` admits exactly the two shapes the orchestrator keeps stable for a pass:
*
* - `ReadonlySet<string>`, the pass's file set — every index derived from it,
* including the derived header-closure sets that `languages/{c,cpp}/
* scope-resolver.ts` memoize inside an outer per-file-set memo. Defeating
* one of these means copying the SET, which re-traverses it, which the
* `CountingSet` instrument (`test/helpers/counting-file-set.ts`) reads as a
* scan count rising with the import count.
* - `readonly ParsedFile[]`, the pass's parsed-file array. Not derived from
* the file set at all, so the file-set guards do not reach them; these key
* on the array the orchestrator already threads through the pass, and their
* contract is that same pass-through discipline. The instrument that CAN see
* them counts element reads on that array — `countedParsedFiles`, beside
* `CountingSet`, driven by the contract test's `minimumParsedFileReads`.
*
* A THIRD shape — an array materialized from the file set — is what the type
* exists to reject. `import-resolvers/csharp.ts` used one until #2911, and it
* is worth a compile error rather than a rule: copying an array mints a fresh
* `WeakMap` key while traversing the Set zero extra times, so every
* scan-counting guard stays green at its correct value while the index rebuilds
* once per import. That failure is invisible to the whole instrument family
* above and was caught only by a timing ratio in `bench/import-target/`. Derive
* the array inside the builder from `getWorkspaceFileIndex(allFilePaths)`
* instead. `string[]` is not assignable to `K`, so the shape cannot come back
* silently — `configs/swift.ts` keeps the one hand-rolled `WeakMap` on
* `ctx.allFileList` in the tree, deliberately and with its reasons written
* down, and it is deliberately NOT on this primitive.
*
* `T extends object` is deliberate, chosen over probing `has` before `get`.
* `WeakMap.get` returning `undefined` cannot distinguish "not built yet" from
* "built, and the value is `undefined`"; constraining the value to an object
* makes the second case unrepresentable rather than paying a second lookup on
* every import, and it needs no cast to type-check. Every index memoized here
* is a record, `Map` or `Set`, so the constraint costs nothing today — and a
* later caller wanting to memoize a `string | null` gets a compile error
* pointing at this line instead of a memo that silently rebuilds on every miss.
*
* A `build` that THROWS stores nothing, so the next call for that key runs it
* again: failures are not memoized, and a half-filled index is never published.
* Inert for the builders here — each is a pure, total pass over the file set —
* and the safer of the two behaviours if that ever stops being true.
*/
import type { ParsedFile } from 'gitnexus-shared';
export function perFileSet<K extends ReadonlySet<string> | readonly ParsedFile[], T extends object>(
build: (key: K) => T,
): (key: K) => T {
const cache = new WeakMap<K, T>();
return (key) => {
const cached = cache.get(key);
if (cached !== undefined) return cached;
const built = build(key);
cache.set(key, built);
return built;
};
}

45
gitnexus/src/core/ingestion/import-resolvers/php.ts
View file

@ -37,8 +37,8 @@ export function resolvePhpImportInternal(
importPath: string,
composerConfig: ComposerConfig | null,
allFiles: Set<string>,
normalizedFileList: string[],
allFileList: string[],
normalizedFileList: readonly string[],
allFileList: readonly string[],
index?: SuffixIndex,
): string | null {
// Normalize: replace backslashes with forward slashes
@ -67,21 +67,38 @@ export function resolvePhpImportInternal(
const lastSlash = remainder.lastIndexOf('/');
const nsDir = lastSlash >= 0 ? dirPrefix + '/' + remainder.slice(0, lastSlash) : dirPrefix;
// Prefer SuffixIndex directory lookup (O(log n + matches)) over linear scan
// Prefer SuffixIndex directory lookup (O(log n + matches)) over linear scan.
//
// An EMPTY bucket is a final answer, not a miss to retry with the scan
// below — which is what the `else` restores, and what this comment
// always claimed. Re-scanning on empty was the last per-import
// workspace traversal left in PHP resolution after #2901: any `use`
// matching a PSR-4 prefix whose directory holds no direct `.php` child
// (`App\Legacy\Ghost`) paid a full pass, measured at 201 traversals for
// 200 imports.
//
// The bucket is a superset of what the scan can find, for BOTH index
// shapes that reach here. A root-anchored direct child `nsDir/<x>.php`
// has its directory exactly equal to `nsDir`, and `nsDir` is always one
// of that directory's own suffixes — so the shared `dirMap` (keyed on
// every directory suffix) necessarily contains it, as does the
// root-anchored parity index `languages/php/import-target.ts` builds.
// Empty superset therefore implies empty scan, and control falls
// through to the next PSR-4 prefix exactly as before.
if (index) {
const candidates = index.getFilesInDir(nsDir, '.php');
if (candidates.length > 0) return candidates[0];
}
// Fallback: linear scan (only when SuffixIndex unavailable)
const nsDirPrefix = nsDir.endsWith('/') ? nsDir : nsDir + '/';
for (const f of allFiles) {
if (
f.startsWith(nsDirPrefix) &&
f.endsWith('.php') &&
!f.slice(nsDirPrefix.length).includes('/')
) {
return f;
} else {
// Linear scan, only when a SuffixIndex is genuinely unavailable.
const nsDirPrefix = nsDir.endsWith('/') ? nsDir : nsDir + '/';
for (const f of allFiles) {
if (
f.startsWith(nsDirPrefix) &&
f.endsWith('.php') &&
!f.slice(nsDirPrefix.length).includes('/')
) {
return f;
}
}
}
}

371
gitnexus/src/core/ingestion/import-resolvers/python-file-index.ts Normal file
View file

@ -0,0 +1,371 @@
/**
* The one per-file-set index behind Python import resolution, plus the two
* importer-chain memos that ride inside it.
*
* ## Why this is its own module
*
* Everything here is derived from `allFilePaths` and nothing here is specific
* to either CALLER, and there are two of them on opposite sides of a layer
* boundary: `import-resolvers/python.ts` resolves the single-segment bare tier
* and `languages/python/import-target.ts` resolves the dotted tiers. The second
* imports the first, so the index could not live in either without the other
* reaching back through a cycle — it used to live in `import-target.ts`, which
* is why the bare tier had no O(1) proof of absence and probed the whole
* ancestor chain for every `import os`.
*
* The shape is the one `workspace-file-index.ts` and `package-dir-index.ts`
* already use in this directory: an interface, one `perFileSet` builder, and
* query functions taking the index.
*/
import { perFileSet } from './per-file-set.js';
/**
* The importer's ancestor directories, CLOSEST FIRST and excluding the
* workspace root — `["backend/routers", "backend"]` for `backend/routers/x.py`
* — memoized per importer DIRECTORY for the lifetime of the pass.
*
* This is the #2913 fix. Both consumers used to rebuild the chain inline, one
* `dirParts.slice(0, i).join('/')` per component, on EVERY import: a per-import
* cost proportional to the importer's path depth, and quadratic in characters,
* on a file index that is itself depth-free. Real Python layouts are deep
* (`src/pkg/sub/feature/impl/mod.py` is ordinary), so the resolver was 6.8x
* slower on a deep corpus than on a shallow one holding the file count fixed,
* where every other language sat between 1.0x and 3.4x.
*
* A directory's ancestors are a pure function of the directory, and a pass
* resolves many imports per file, so one entry serves every import issued from
* anywhere in that directory.
*
* ## Lifetime and memory
*
* The Map lives INSIDE the per-file-set index, so it is reclaimed with the file
* set it was reached through (`perFileSet` is a `WeakMap`): it cannot leak
* across passes or repos, and there is no invalidation rule to get wrong. It is
* filled lazily, so it holds one entry per directory that actually ISSUES a
* Python import, never one per file and never one per directory in the repo —
* the bound #2649 (kernel-scale OOM) asks for. Each entry's strings are
* `slice`s of the longest one, so a chain costs pointers rather than a copy of
* the path per component.
*
* The derived key is the importer's directory exactly as the old inline code
* computed it — `norm.split('/').slice(0, -1).join('/')`, which for a path
* without a separator is `''` (a root-level importer, whose chain is empty).
*/
/**
* The importer's own directory, normalized — the key BOTH per-directory memos
* below are stored under.
*
* One exported derivation rather than one per accessor: the two memos live in
* the same index and must agree on what "the importer's directory" is, and a
* caller that already holds the directory (the bare-import tier computes it for
* its own proximity check) should not pay for it twice. It was three copies of
* `replace / lastIndexOf / slice` across two modules before, byte-identical by
* inspection and by nothing else.
*/
export function importerDirOf(fromFile: string): string {
const norm = fromFile.replace(/\\/g, '/');
const lastSlash = norm.lastIndexOf('/');
return lastSlash === -1 ? '' : norm.slice(0, lastSlash);
}
export function importerAncestors(index: PythonFileIndex, importerDir: string): readonly string[] {
const memoized = index.ancestorsByDir.get(importerDir);
if (memoized !== undefined) return memoized;
const built = buildImporterAncestors(importerDir);
index.ancestorsByDir.set(importerDir, built);
return built;
}
/**
* `["a/b/c", "a/b", "a"]` for `a/b/c`. Empty components are dropped first, so
* an absolute `/a/b` yields `["a/b", "a"]` — matching the `filter(Boolean)` the
* two inline walks did, and with it the absolute-path gating pinned by
* `python-import-target-parity.test.ts` (PR #1918 review P3a).
*/
function buildImporterAncestors(importerDir: string): readonly string[] {
const chain: string[] = [];
const parts = importerDir.split('/').filter(Boolean);
if (parts.length === 0) return chain;
chain.push(parts.join('/'));
for (let i = 1; i < parts.length; i++) {
const child = chain[i - 1];
chain.push(child.slice(0, child.lastIndexOf('/')));
}
return chain;
}
/**
* Per-file-set index for Python import resolution, memoized on the
* `allFilePaths` Set object (the same Set is passed for every import in a run,
* so the index is built once and reused). Replaces the per-import O(files)
* scans in `resolveAbsoluteFromFiles` (suffix match) and `hasRepoCandidate`
* (package-existence gate) with O(1)/O(bucket) lookups.
*
* - `normSet`: every file path, normalized to forward slashes (for the exact
* `f === rootFile|initFile` membership checks). It IS derivable from the two
* buckets below — both probes could be a `.some(c => c.norm === …)` over
* `byBasename.get(rootFile)` / `byInitParent.get(initFile)` — and it is kept
* anyway, deliberately. `byBasename` is keyed on the BASENAME, so its bucket
* for a common Python file name is not small and grows with the repo: on a
* 9 000-file service tree, `utils.py`, `models.py` and `views.py` hold 1 000
* entries each. `import utils` would then scan every `utils.py` in the
* workspace on every import — a per-import cost proportional to corpus size,
* which is the exact defect class #2901/#2902/#2908 removed. The Set trades
* ~1.6 MB at 32 000 files, against a 6.4 MB reading, to keep both probes
* O(1). Do not "simplify" it away without re-measuring that bucket.
* - `byBasename`: last path component (e.g. `models.py`, `__init__.py`) ->
* all `{ raw, norm }` candidates, so suffix matches can be gathered from the
* relevant bucket and the exact tie-break applied across ALL of them.
* - `byInitParent`: `__init__.py` files keyed by their last TWO components
* (`<parentDir>/__init__.py`). The package suffix lookup (`pkg.sub` ->
* `…/sub/__init__.py`) targets only same-named package dirs via this map
* instead of scanning every `__init__.py` in the repo — the common
* multi-segment import path no longer scales with package count
* (PR #1918 review P2b). `__init__.py` files stay in `byBasename` too, for
* the rarer explicit `pkg.__init__` import that resolves via the module
* (`…<lastSeg>.py`) lookup.
* - `dirPrefixes`: every directory prefix of a `.py` file, trailing-slashed
* (`a/b/c.py` -> `a/`, `a/b/`), for "is there a .py file under `<dir>/`".
* - `nestedDirNames`: the NAME of every such directory that has a non-empty
* parent (`a/b/c.py` -> `b`, not `a`), which is exactly the set of segments
* `hasRepoCandidate`'s ancestor walk can ever match — so a segment absent
* from it settles the walk in one lookup (#2913).
* - `ancestorsByDir`: the per-importer-directory ancestor-chain memo behind
* `importerAncestors`. The one structure here that is NOT derived from the
* file set: it is filled lazily, from the importer paths the pass actually
* resolves against, and lives here so it dies with the pass.
* - `bareImportPrefixesByDir`: the same idea for the OTHER chain — the
* sys.path-style prefixes `resolvePythonImportInternal`'s single-segment
* walk probes. A different sequence, not a different spelling: see
* `importerBarePrefixes`. Two memos in one index rather than two indexes,
* because they are keyed on the same thing and must die together.
*
* Exported for `test/unit/scope-resolution/python/python-importer-ancestors.test.ts`
* and `test/unit/import-resolvers/python-importer-prefixes.test.ts`, which read
* the two memos after driving the production adapters. No counter ships for
* either — the Map IS the memo, and its SIZE is the assertion: one entry per
* importer directory, however many imports were resolved. Everything else about
* the index stays internal.
*/
export interface PythonFileIndex {
readonly normSet: Set<string>;
readonly byBasename: Map<string, { raw: string; norm: string }[]>;
readonly byInitParent: Map<string, { raw: string; norm: string }[]>;
readonly dirPrefixes: Set<string>;
readonly nestedDirNames: Set<string>;
readonly ancestorsByDir: Map<string, readonly string[]>;
readonly bareImportPrefixesByDir: Map<string, readonly string[]>;
}
export const getPythonFileIndex = perFileSet(
(allFilePaths: ReadonlySet<string>): PythonFileIndex => {
// Runs on a cache miss only. That it happens once per run and not once per
// import is asserted by counting traversals of the Set itself, in
// `test/integration/python-import-index-reuse.test.ts` — the PR #1918 review
// P1 guard (#2909).
const normSet = new Set<string>();
const byBasename = new Map<string, { raw: string; norm: string }[]>();
const byInitParent = new Map<string, { raw: string; norm: string }[]>();
const dirPrefixes = new Set<string>();
const nestedDirNames = new Set<string>();
for (const raw of allFilePaths) {
const norm = raw.replace(/\\/g, '/');
// Python import resolution only ever queries `.py` paths: module `<seg>.py`
// and package `<seg>/__init__.py` membership (normSet), `<lastSeg>.py` /
// `__init__.py` basename buckets (byBasename), and `.py` directory prefixes
// (dirPrefixes). Non-`.py` files can never match any of those, so skip them
// — they were dead weight in every structure on polyglot monorepos
// (PR #1918 review P3b; dirPrefixes was already `.py`-gated).
if (!norm.endsWith('.py')) continue;
normSet.add(norm);
// ONE entry object per file, shared by both buckets below: a package file
// lands in `byBasename` and `byInitParent`, and two literals for the same
// `(raw, norm)` pair cost ~40 B each on every `__init__.py`.
const entry = { raw, norm };
const lastSlash = norm.lastIndexOf('/');
const base = lastSlash >= 0 ? norm.slice(lastSlash + 1) : norm;
// `set(base, [entry])` rather than `set(base, [])` then `push`: an empty
// array literal that is immediately pushed to makes V8 grow the backing
// store to its 16-slot minimum, so every bucket holding ONE file retains
// 15 empty pointer slots — 128 B — for the whole pass. `byBasename` has
// roughly one bucket per file, which made that the dominant term in this
// index: measured 5.50 MiB against 1.60 MiB for the one-element form at
// 32 000 `.py` paths, byte-identical contents. Same shape as
// `languages/php/import-target.ts`'s directory buckets.
const bucket = byBasename.get(base);
if (bucket === undefined) byBasename.set(base, [entry]);
else bucket.push(entry);
// Package files also get a parent-keyed bucket so a `pkg.sub` lookup hits
// only `…/sub/__init__.py` candidates, not every `__init__.py` (P2b).
if (base === '__init__.py' && lastSlash >= 0) {
const dir = norm.slice(0, lastSlash);
const parentSlash = dir.lastIndexOf('/');
const parentName = parentSlash >= 0 ? dir.slice(parentSlash + 1) : dir;
if (parentName) {
const initKey = `${parentName}/__init__.py`;
const ib = byInitParent.get(initKey);
if (ib === undefined) byInitParent.set(initKey, [entry]);
else ib.push(entry);
}
}
// Directory prefixes: every slash-terminated prefix of the path (every
// index just past a '/', up to and including the file's own directory).
// Scanning the FULL normalized path — including any leading '/' for
// absolute paths — makes `dirPrefixes.has(X)` match exactly when the old
// gate's `f.startsWith(X)` (X always ends in '/') matched. The previous
// split+`filter(Boolean)` dropped the leading empty component, so an
// absolute file `/repo/svc/x.py` yielded `repo/svc/` (no leading slash) and
// gate-passed where `"/repo/svc/x.py".startsWith("repo/svc/")` is false
// (PR #1918 review P3a). For relative paths the set is identical.
//
// The walk runs from the DEEPEST prefix outward and stops at the first
// one already recorded. Every prefix is added together with all of its
// own ancestors, so a hit proves the rest of the chain is already there —
// which makes the second and later files of a directory cost ONE lookup
// instead of one insert per path component. This build was the last part
// of Python's resolution that still scaled with path depth (#2913): the
// same 400-file corpus moved sixteen directories down went from 800
// inserts to 7200, for the same ~120 distinct prefixes.
//
// `nestedDirNames` rides the same walk. A directory prefix has the shape
// `<parent>/<name>/` — the only shape `hasRepoCandidate`'s check (3)
// probes — exactly when another slash precedes it at index > 0. Index 0
// is excluded on purpose: `a/` and `/` name a directory whose parent is
// empty, which check (2) already answers and which the ancestor walk
// (non-empty ancestors only) never probes.
for (let i = lastSlash; i >= 0; i--) {
if (norm[i] !== '/') continue;
const dirPrefix = norm.slice(0, i + 1);
if (dirPrefixes.has(dirPrefix)) break;
dirPrefixes.add(dirPrefix);
const parentSlash = i > 0 ? norm.lastIndexOf('/', i - 1) : -1;
if (parentSlash > 0) nestedDirNames.add(norm.slice(parentSlash + 1, i));
}
}
return {
normSet,
byBasename,
byInitParent,
dirPrefixes,
nestedDirNames,
ancestorsByDir: new Map<string, readonly string[]>(),
bareImportPrefixesByDir: new Map<string, readonly string[]>(),
};
},
);
/**
* The sys.path-style prefixes `resolvePythonImportInternal`'s single-segment
* bare-import walk probes, in order, for an importer sitting in `importerDir` —
* memoized per DIRECTORY for the lifetime of the pass, in the same index and
* for the same reasons as `importerAncestors`.
*
* ## Why this is not `ancestorsByDir`
*
* A DIFFERENT SEQUENCE, not a different spelling. For `backend/routers/cron.py`:
*
* importerAncestors ["backend/routers", "backend"]
* importerBarePrefixes ["backend/", ""]
*
* Three differences, each load-bearing:
*
* 1. `importerAncestors` opens with the importer's OWN directory; this walk
* does not, because its proximity check has already probed that directory.
* 2. This walk ENDS at the workspace root (`""`, which probes `<module>.py`
* unprefixed); `importerAncestors` stops short of it, because
* `resolveAbsoluteFromFiles` probes the root before its walk instead.
* 3. `importerAncestors` drops empty components (`filter(Boolean)`); this walk
* keeps them, and the difference decides real resolutions — for
* `/abs/a/b/mod.py` this walk probes `/abs/a/`, `/abs/`, `""`, `""` where a
* filtered chain would probe `abs/a/b/`, `abs/a/`, `abs/`, none of which is
* a prefix of any file in an absolute-path workspace.
*
* So the two cannot share one chain without changing which files resolve. They
* do share the index, the key and the lifetime, which is what actually matters
* for #2649: both are filled lazily, hold one entry per directory that ISSUES
* an import, and die with the pass because the index does.
*/
export function importerBarePrefixes(
index: PythonFileIndex,
importerDir: string,
): readonly string[] {
const memoized = index.bareImportPrefixesByDir.get(importerDir);
if (memoized !== undefined) return memoized;
const built = buildImporterBarePrefixes(importerDir);
index.bareImportPrefixesByDir.set(importerDir, built);
return built;
}
/**
* `["a/b/", "a/", ""]` for `a/b/c` — every proper ancestor of `importerDir`,
* closest first, slash-terminated, ending at the workspace root.
*
* Cutting the string at each `lastIndexOf('/')` walks the same ancestors the
* pre-#2913-followup `dirParts.slice(0, i).join('/')` produced, INCLUDING the
* empty components a `filter(Boolean)` would have dropped: `/abs/a/b` yields
* `["/abs/a/", "/abs/", "", ""]`, the second `""` being the `i === 0` step that
* followed the leading empty component. Byte-identical sequences, duplicates
* kept, so the probes this feeds are unchanged in content, order and count.
*/
function buildImporterBarePrefixes(importerDir: string): readonly string[] {
const prefixes: string[] = [];
let dir = importerDir;
let slash = dir.lastIndexOf('/');
while (slash !== -1) {
dir = dir.slice(0, slash);
prefixes.push(dir === '' ? '' : `${dir}/`);
slash = dir.lastIndexOf('/');
}
prefixes.push('');
return prefixes;
}
/**
* "No file anywhere in the workspace can be `<X>/<segment>.py` or
* `<X>/<segment>/__init__.py`, for ANY prefix `<X>`" — in two Map lookups.
*
* This is a PROOF OF ABSENCE, not a heuristic filter, and it is what lets the
* single-segment bare walk skip itself entirely. Both shapes it rules out are
* the only two shapes that walk probes: a probe `${prefix}${segment}.py` that
* is a member of the file set is a path with no backslash (the prefix comes
* from a normalized importer and the guard below rejects a segment carrying
* one), so it equals its own normalized form and its basename is exactly
* `${segment}.py` — which puts it in `byBasename`. A probe
* `${prefix}${segment}/__init__.py` that is a member likewise has parent
* directory name exactly `segment`, non-empty, which puts it in `byInitParent`
* whether or not `prefix` is empty. So a miss in both buckets means every probe
* the walk would issue is guaranteed to miss.
*
* Two inputs cannot be proven absent and get `false` — walk as before:
*
* - the EMPTY segment (a target spelled with a trailing dot).
* `byInitParent` skips `__init__.py` files whose parent directory name is
* empty, so its absence proves nothing. Same carve-out
* `resolveAbsoluteFromFiles` makes for `lastSeg === ''`.
* - a segment containing a BACKSLASH. The buckets are keyed on normalized
* paths, so a raw `a\b.py` is filed under basename `b.py`; a probe for the
* segment `a\b` would look up `a\b.py`, miss, and wrongly conclude absence
* while `allFilePaths.has('a\\b.py')` is true. Not reachable from a Python
* import statement, but this function is a proof and a proof has no
* unstated preconditions.
*
* The dotted tier in `languages/python/import-target.ts` asks the same question
* of the same two buckets and is deliberately NOT routed through here: it needs
* the candidate ARRAYS for its suffix fallback, so it does the two `get`s it
* already needs and derives the answer, rather than paying two extra `has`
* lookups per import to share four lines.
*/
export function pythonSegmentAbsent(index: PythonFileIndex, segment: string): boolean {
if (segment === '' || segment.includes('\\')) return false;
if (index.byBasename.has(`${segment}.py`)) return false;
if (index.byInitParent.has(`${segment}/__init__.py`)) return false;
return true;
}

36
gitnexus/src/core/ingestion/import-resolvers/python.ts
View file

@ -6,6 +6,12 @@
* This file contains the shared internal helper used by the strategy and tests.
*/
import {
getPythonFileIndex,
importerBarePrefixes,
importerDirOf,
pythonSegmentAbsent,
} from './python-file-index.js';
import { tryResolveWithExtensions } from './utils.js';
/**
@ -51,8 +57,24 @@ export function resolvePythonImportInternal(
const pathLike = importPath.replace(/\./g, '/');
if (pathLike.includes('/')) return null;
// Normalize for Windows backslashes
const importerDir = currentFile.replace(/\\/g, '/').split('/').slice(0, -1).join('/');
// O(1) proof of absence, before any probing. Every probe below — the two
// proximity probes and the two per ancestor step — has the shape
// `<X>/<pathLike>.py` or `<X>/<pathLike>/__init__.py`, and
// `pythonSegmentAbsent` answers "no file in the workspace has EITHER shape,
// for any prefix" in two Map lookups on the index the dotted tiers already
// build. That is `true` for `os`, `sys`, `django` and every other
// distribution the repo does not vendor — i.e. for most imports in most
// Python repos — and it retires the whole walk for them instead of running
// it to the workspace root. It is exact, not a filter: a miss here means
// every probe the walk would have issued was guaranteed to miss.
const index = getPythonFileIndex(allFiles);
if (pythonSegmentAbsent(index, pathLike)) return null;
// One derivation, shared with the index's other per-directory memo — see
// `importerDirOf`. It replaced `split('/').slice(0, -1).join('/')`: identical
// for every input (a path with no separator has no directory, which is `''`
// both ways) without the per-import array of one element per path component.
const importerDir = importerDirOf(currentFile);
// Proximity check — only applies when the importer lives in a subdirectory.
// Root-level importers (importerDir === '') skip straight to the ancestor
@ -68,10 +90,12 @@ export function resolvePythonImportInternal(
// importer's directory to find the module in an ancestor, preferring the closest match.
// This prevents cross-language misresolution (e.g., Python `from middleware import X`
// resolving to a TypeScript middleware.ts via suffix matching). Issue #417.
const dirParts = importerDir.split('/');
for (let i = dirParts.length - 1; i >= 0; i--) {
const ancestorDir = dirParts.slice(0, i).join('/');
const prefix = ancestorDir ? `${ancestorDir}/` : '';
//
// The prefixes come from `importerBarePrefixes`, built ONCE per importer
// directory per pass and stored in the same index consulted above. Rebuilding
// them here — `dirParts.slice(0, i).join('/')`, one array and one string per
// path component — was the last per-import ancestor walk left after #2913.
for (const prefix of importerBarePrefixes(index, importerDir)) {
if (allFiles.has(`${prefix}${pathLike}/__init__.py`)) return `${prefix}${pathLike}/__init__.py`;
if (allFiles.has(`${prefix}${pathLike}.py`)) return `${prefix}${pathLike}.py`;
}

4
gitnexus/src/core/ingestion/import-resolvers/ruby.ts
View file

@ -16,8 +16,8 @@ import { suffixResolve } from './utils.js';
*/
export function resolveRubyImportInternal(
importPath: string,
normalizedFileList: string[],
allFileList: string[],
normalizedFileList: readonly string[],
allFileList: readonly string[],
index?: SuffixIndex,
): string | null {
const pathParts = importPath.replace(/^\.\//, '').split('/').filter(Boolean);

4
gitnexus/src/core/ingestion/import-resolvers/standard.ts
View file

@ -29,8 +29,8 @@ export const resolveImportPath = (
currentFile: string,
importPath: string,
allFiles: Set<string>,
allFileList: string[],
normalizedFileList: string[],
allFileList: readonly string[],
normalizedFileList: readonly string[],
resolveCache: Map<string, string | null>,
language: SupportedLanguages,
tsconfigPaths: TsconfigPaths | null,

313
gitnexus/src/core/ingestion/import-resolvers/utils.ts
View file

@ -79,66 +79,295 @@ export function tryResolveWithExtensions(
* etc.
*/
export interface SuffixIndex {
/** Exact suffix lookup (case-sensitive) */
/**
* Exact suffix lookup (case-sensitive).
*
* The map behind this is built on the FIRST call and memoized — see
* `buildSuffixIndex`. All three maps are deferred; a consumer pays only for
* the questions it actually asks.
*/
get(suffix: string): string | undefined;
/** Case-insensitive suffix lookup */
/**
* Case-insensitive suffix lookup.
*
* Deferred like `get`, and — when `get` was asked first — DERIVED from that
* map rather than traversed for a second time. See `buildSuffixIndex`.
*/
getInsensitive(suffix: string): string | undefined;
/** Get all files in a directory suffix */
getFilesInDir(dirSuffix: string, extension: string): string[];
/**
* Get all files in a directory suffix.
*
* `readonly` is the CONTRACT, and it is the contract for every implementation
* of this interface, not a description of any one of them: an implementation
* is free to return its own bucket by reference, so callers must treat the
* result as shared and never `sort`/`splice` it in place. The compiler now
* refuses that at the call site. Whether a given implementation shares or
* copies is its own business and documented where it is built —
* `buildSuffixIndex` shares, the root-anchored parity index in
* `languages/php/import-target.ts` returns a filtered copy.
*
* Implementations that memoize should note the directory map behind this may
* be built on the FIRST call rather than up front, so a caller that never
* asks a directory question never pays for it — see `buildSuffixIndex`.
*/
getFilesInDir(dirSuffix: string, extension: string): readonly string[];
}
export function buildSuffixIndex(normalizedFileList: string[], allFileList: string[]): SuffixIndex {
// Map: normalized suffix -> original file path
const exactMap = new Map<string, string>();
// Map: lowercase suffix -> original file path
const lowerMap = new Map<string, string>();
// Map: directory suffix -> list of file paths in that directory
const dirMap = new Map<string, string[]>();
export interface SuffixIndexOptions {
/**
* Promise from the caller that `normalizedFileList[i] === normalizedFileList[i].toLowerCase()`
* for every `i` — i.e. the "normalized" list is a LOWERCASED file list, not
* merely a slash-normalized one.
*
* `import-resolvers/pass-cache.ts` is the one caller that can make it: it
* builds `normalizedFileList` as `allFileList.map((f) => f.toLowerCase())`.
* Every suffix of an all-lowercase path is itself lowercase, so
* `suffix.toLowerCase() === suffix` and the case-folded map came out a
* byte-identical copy of the exact one — same keys, same values, same
* insertion order. Measured 14.00 MiB at 32 000 paths, 29.8% of the retained
* `ImportPassCache` — and one `ImportPassCache` is built per ts-family
* adapter per pass, so the waste was carried once for each of them.
*
* With this set, `getInsensitive` reads the exact map directly instead. It is
* the same map the derivation below would have produced, so this is a skipped
* copy and not a second lookup rule — see `getLowerMap`.
*
* Setting it over a list that is NOT all-lowercase is a behaviour change, not
* an optimization: `getInsensitive` would then answer case-sensitively.
*/
readonly alreadyLowercased?: boolean;
}
for (let i = 0; i < normalizedFileList.length; i++) {
const normalized = normalizedFileList[i];
const original = allFileList[i];
const parts = normalized.split('/');
export function buildSuffixIndex(
normalizedFileList: readonly string[],
allFileList: readonly string[],
options?: SuffixIndexOptions,
): SuffixIndex {
const alreadyLowercased = options?.alreadyLowercased === true;
// Index all suffixes: "a/b/c.java" -> ["c.java", "b/c.java", "a/b/c.java"]
for (let j = parts.length - 1; j >= 0; j--) {
const suffix = parts.slice(j).join('/');
// Only store first match (longest path wins for ambiguous suffixes)
if (!exactMap.has(suffix)) {
exactMap.set(suffix, original);
/**
* Map: normalized suffix -> original file path.
*
* DEFERRED, like `dirMap` below and for the same reason (#2903 extended to
* the two suffix maps). Several consumers on the ScopeResolver path ask only
* ONE of the two suffix questions and were paying for both:
*
* - `languages/java/import-target.ts` and the no-csproj leg of
* `languages/csharp/import-target.ts` call `get` and never
* `getInsensitive` — measured 49.98 MiB dead of a 100.82 MiB Java index
* at 32 000 paths (49.6%), against a gated ceiling of 146.9 MiB;
* - `languages/php/import-target.ts` calls `getInsensitive` and never `get`
* — 34.49 MiB of 69.85 MiB (49.4%).
*
* Ruby, the csproj leg of C#, `group/extractors/include-extractor.ts` and
* `suffixResolve` below read both, and all four read `get` FIRST (they are
* written `get(s) || getInsensitive(s)`), which is what makes the derivation
* in `getLowerMap` the cheap order rather than the expensive one.
*/
let exactMap: Map<string, string> | null = null;
const getExactMap = (): Map<string, string> => {
if (exactMap !== null) return exactMap;
const built = new Map<string, string>();
for (let i = 0; i < normalizedFileList.length; i++) {
const normalized = normalizedFileList[i];
const original = allFileList[i];
// Index all suffixes: "a/b/c.java" -> ["c.java", "b/c.java", "a/b/c.java"].
//
// Walked as slash offsets into `normalized` rather than as
// `normalized.split('/')` + `parts.slice(j).join('/')`: the slice of the
// ORIGINAL string is byte-identical to the re-joined parts (no separator
// is invented or dropped — verified over 361 865 suffix strings including
// leading, doubled and trailing slashes), and it allocates one string
// instead of a parts array, a slice array and a joined string per suffix.
// Measured 357.4 ms -> 264.5 ms at 32 000 paths.
let slash = normalized.lastIndexOf('/');
while (slash >= 0) {
const suffix = normalized.slice(slash + 1);
// Only store first match (longest path wins for ambiguous suffixes)
if (!built.has(suffix)) built.set(suffix, original);
// A path may begin with '/', whose suffix is the whole string below.
if (slash === 0) break;
slash = normalized.lastIndexOf('/', slash - 1);
}
const lower = suffix.toLowerCase();
if (!lowerMap.has(lower)) {
lowerMap.set(lower, original);
// j = 0 — the whole path, which the slash walk cannot emit.
if (!built.has(normalized)) built.set(normalized, original);
}
exactMap = built;
return built;
};
/**
* Map: lowercase suffix -> original file path.
*
* Deferred, and when the exact map already exists DERIVED from it instead of
* traversed for: one pass over that map's DISTINCT keys rather than a second
* pass over every (file × depth) suffix. Measured 330.3 ms total (200.6 build
* + 129.7 derive) against 388.8 ms for the single fused traversal that built
* both eagerly — so the two-map consumers get cheaper too, which per-map
* laziness on its own does not (407.1 ms, a second full traversal).
*
* The derivation is EQUAL, not approximate, and the argument is short. Let
* the fused loop's global order be the pairs (suffix, file) it visited. For a
* lowercase key L, let p be the first position whose suffix lowercases to L —
* the entry today's `lowerMap` keeps. Nothing before p carries that suffix
* spelled ANY way, so p is also the first occurrence of its exact spelling
* and is therefore in the exact map, holding that same file. Exact-map
* insertion order is by first-occurrence position, so among the exact keys
* folding to L, p's is reached first and first-wins keeps it. Insertion order
* of the derived map is the order of those p's, which is the order today's
* `lowerMap` inserts L. Verified rather than only argued: byte-equal keys,
* values and order over 968 418 entries across bench-shaped, PascalCase,
* case-colliding, deep-monorepo, Unicode-adversarial and 400 seeded-fuzz
* corpora.
*
* When `getInsensitive` is asked FIRST (PHP), there is nothing to derive
* from, so it is built straight — one traversal, one map, which is the point.
* Asking `get` afterwards would then cost the second traversal; no consumer
* does, and the fallback stays correct if one ever starts.
*/
let lowerMap: Map<string, string> | null = null;
const getLowerMap = (): Map<string, string> => {
// Over an already-lowercased file list the derivation is the identity, so
// the exact map IS the case-folded map. Skip the copy.
if (alreadyLowercased) return getExactMap();
if (lowerMap !== null) return lowerMap;
const built = new Map<string, string>();
if (exactMap !== null) {
for (const [suffix, original] of exactMap) {
const lower = suffix.toLowerCase();
if (!built.has(lower)) built.set(lower, original);
}
lowerMap = built;
return built;
}
// Index directory membership
const lastSlash = normalized.lastIndexOf('/');
if (lastSlash >= 0) {
// Build all directory suffixes
const dirParts = parts.slice(0, -1);
const fileName = parts[parts.length - 1];
const ext = fileName.substring(fileName.lastIndexOf('.'));
for (let i = 0; i < normalizedFileList.length; i++) {
const normalized = normalizedFileList[i];
const original = allFileList[i];
let slash = normalized.lastIndexOf('/');
while (slash >= 0) {
const lower = normalized.slice(slash + 1).toLowerCase();
if (!built.has(lower)) built.set(lower, original);
if (slash === 0) break;
slash = normalized.lastIndexOf('/', slash - 1);
}
const whole = normalized.toLowerCase();
if (!built.has(whole)) built.set(whole, original);
}
lowerMap = built;
return built;
};
for (let j = dirParts.length - 1; j >= 0; j--) {
const dirSuffix = dirParts.slice(j).join('/');
const key = `${dirSuffix}:${ext}`;
let list = dirMap.get(key);
/**
* Map: `${directory suffix}:${extension}` -> file paths in that directory.
*
* DEFERRED, not dropped (#2903). This is the array-valued map of the three
* and by far the most expensive: one entry — and one array push — per file
* per directory component, so O(files × depth) in entries AND in array
* churn. Measured on the 32k-path arms of `bench/import-target/`, it is
* ~15% of the retained C# index and ~19% of the retained Ruby one.
*
* Only `getFilesInDir` reads it, and only four call sites reach that:
* `import-resolvers/{php,csharp,jvm}.ts` and `import-resolvers/configs/
* python.ts`. Every other consumer of this index — `workspace-file-index.ts`
* serving Ruby, `languages/typescript/scope-resolver.ts`,
* `languages/vue/import-target.ts`, `group/extractors/include-extractor.ts`
* — asks only suffix questions and was paying the whole footprint for a map
* it never touched. Since these indexes are now retained for a whole
* resolution pass rather than rebuilt per import (#2877-#2880), that is
* retained memory against the #2649 kernel-scale OOM constraint.
*
* `null` until the first `getFilesInDir`; the MAP is memoized, not the
* decision to build it, so a repeated miss cannot rebuild it. Building it
* later is behaviour-identical because it is a pure function of
* `normalizedFileList` / `allFileList`, and it retains nothing new: every
* production caller already holds both arrays alive alongside the index
* (`WorkspaceFileIndex.normalized`/`.all`, the TS and Vue `PassCache`s,
* `IncludeExtractor.extract`'s locals).
*/
let dirMap: Map<string, string[]> | null = null;
const getDirMap = (): Map<string, string[]> => {
if (dirMap !== null) return dirMap;
const built = new Map<string, string[]>();
for (let i = 0; i < normalizedFileList.length; i++) {
const normalized = normalizedFileList[i];
const original = allFileList[i];
const lastSlash = normalized.lastIndexOf('/');
// A file at the repo root is in no directory suffix.
if (lastSlash < 0) continue;
// The file name from its last '.', or the WHOLE file name when it carries
// none — `substring(-1)` clamps to 0, which is what the `parts` form
// (`fileName.substring(fileName.lastIndexOf('.'))`) spelled. A '.' in a
// DIRECTORY is not an extension, hence `dot > lastSlash` rather than
// `dot >= 0`.
const dot = normalized.lastIndexOf('.');
const ext = dot > lastSlash ? normalized.slice(dot) : normalized.slice(lastSlash + 1);
// Every directory suffix of `normalized.slice(0, lastSlash)`, shortest
// first — the order `for (j = dirParts.length - 1; j >= 0; j--)` emitted,
// and load-bearing: `php.ts` returns `candidates[0]` of a bucket, so a
// reordered bucket is a behaviour change, not a wash.
//
// Walked as slash offsets into `normalized`, the same rewrite `getExactMap`
// above documents and for the same reason — a slice of the ORIGINAL string
// is byte-identical to the re-joined parts, and it allocates one string per
// suffix instead of a parts array, a slice array and a joined string per
// suffix. This is the map where it pays most: one entry, one array push AND
// one key per file per directory component, the "by far the most expensive"
// of the three. Measured 226.9 ms -> 173.1 ms at 32 000 paths averaging
// ~10 directory components (min of 9, both loops alternating in one
// process). Verified rather than argued, over a 32 000-path corpus
// carrying absolute paths, doubled separators (`a//b`), backslash paths,
// root-level and extensionless files, dotted directories and trailing
// separators: 272 956 keys and 329 361 bucket entries came out with
// identical key sets in identical INSERTION order and identical buckets
// element-for-element, and 767 732 probes of the built index — every
// emitted (directory, extension) pair plus a wrong-extension and a
// one-level-deeper miss for each — answered exactly as the `parts` form's
// map did. 0 differences.
//
// `slash < 0` is the whole directory, which no slash search can emit and
// the only suffix a one-component directory has.
let start = lastSlash;
while (start >= 0) {
const slash = start > 0 ? normalized.lastIndexOf('/', start - 1) : -1;
const key = `${normalized.slice(slash + 1, lastSlash)}:${ext}`;
let list = built.get(key);
if (!list) {
list = [];
dirMap.set(key, list);
built.set(key, list);
}
list.push(original);
start = slash;
}
}
}
dirMap = built;
return built;
};
return {
get: (suffix: string) => exactMap.get(suffix),
getInsensitive: (suffix: string) => lowerMap.get(suffix.toLowerCase()),
get: (suffix: string) => getExactMap().get(suffix),
getInsensitive: (suffix: string) => getLowerMap().get(suffix.toLowerCase()),
// THIS implementation shares: it hands back `dirMap`'s own bucket rather
// than a copy. The map is built on first query and then held for the whole
// pass, so the window in which a mutating caller could corrupt later
// imports is the whole pass — which is why the interface makes the result
// `readonly` and the compiler refuses the mutation at the call site.
//
// Sharing beats copying because no caller keeps the array: two only measure
// it and two build a fresh array from it, so a defensive copy would
// allocate a whole bucket per import on the path this index exists to keep
// flat. `package-dir-index.ts` reached the same conclusion the same way —
// read-only containers, plus one copy where a bucket genuinely LEAVES
// (`sortedRootFiles`), which is the case `configs/swift.ts` is in.
getFilesInDir: (dirSuffix: string, extension: string) => {
return dirMap.get(`${dirSuffix}:${extension}`) || [];
return getDirMap().get(`${dirSuffix}:${extension}`) || [];
},
};
}
@ -148,8 +377,8 @@ export function buildSuffixIndex(normalizedFileList: string[], allFileList: stri
*/
export function suffixResolve(
pathParts: string[],
normalizedFileList: string[],
allFileList: string[],
normalizedFileList: readonly string[],
allFileList: readonly string[],
index?: SuffixIndex,
): string | null {
if (index) {

90
gitnexus/src/core/ingestion/import-resolvers/workspace-file-index.ts
View file

@ -1,6 +1,6 @@
/**
* Per-file-set workspace index for the import-target resolvers that need the
* shared `SuffixIndex` (C#, Ruby).
* shared `SuffixIndex` (C#, Java, PHP, Ruby).
*
* The scope-resolution orchestrator passes the SAME `allFilePaths` Set object to
* every `resolveImportTarget` call in a pass (`pipeline/run.ts` builds it once),
@ -12,28 +12,55 @@
* per call and silently restores the O(imports × files) behaviour — the exact
* bug PR #1918 shipped and had to fix in review (P1).
*
* Two layers guard that, and they guard different things:
* Three layers guard that, and they guard different things:
* - ADAPTER BOUNDARY, where the defensive-copy hazard actually lives:
* `test/integration/<lang>-import-index-reuse.test.ts` (csharp and ruby for
* this index; go, dart, kotlin and python for the sibling ones) resolves
* through `<lang>ScopeResolver.resolveImportTarget` — the orchestrator
* adapter — and asserts the file set is traversed once per run (twice for
* C#, which builds two indexes). Kotlin and Python instead count index
* BUILDS from production (`languages/<lang>/index-stats.ts`); either way, a
* copy inserted in an adapter fails these.
* `test/integration/*-import-index-reuse.test.ts` resolves through
* `<lang>ScopeResolver.resolveImportTarget` — the orchestrator adapter — and
* pins the EXACT number of times a run traverses the file set, one file per
* covered language over that language's own corpus. (The expected count is
* per language and legitimately differs: it is however many times the
* adapter derives something from the Set — two indexes, or an index plus the
* mutable copy the ts-family context wants.) All of them count traversals
* of a `CountingSet` (`test/helpers/counting-file-set.ts`): one instrument,
* no production surface, and it catches both the per-import rebuild and a
* scan reintroduced beside a reused index (#2909).
* - EVERY REGISTERED LANGUAGE, at the same boundary but as one property rather
* than one corpus per language: `test/unit/scope-resolution/import-target-index-reuse.contract.test.ts`
* drives each entry of `SCOPE_RESOLVERS` and asserts the traversal count for
* many imports equals the count for two. A new language cannot skip it, and
* the enforcement is a test rather than a roster anyone maintains: that
* file's inventory arm compares `SCOPE_RESOLVERS`' keys against its own
* fixture table and fails on a registered resolver that has neither a
* fixture nor an exemption, and its next arm pins the exemption map empty.
* - RESOLVER LEVEL: `test/unit/scope-resolution/import-target-index-parity.test.ts`
* calls the resolvers directly, so it never crosses the adapter boundary and
* a copy there leaves it green. What it catches is a rescan reintroduced
* INSIDE a resolver, by counting how many times the Set is iterated.
*/
import { perFileSet } from './per-file-set.js';
import { buildSuffixIndex, type SuffixIndex } from './utils.js';
/**
* `normalized` and `all` are `readonly string[]`, and — like
* `SuffixIndex.getFilesInDir` — that is the CONTRACT rather than a description
* of the arrays: they are built once and then held for the whole pass, so an
* in-place `sort`/`splice`/`reverse` would corrupt every later import in that
* pass, and these two are the largest shared arrays here (one element per file,
* read by C#, Java, PHP and Ruby). `readonly` on the field is what makes the
* compiler refuse the mutation at the call site instead of leaving it to a
* comment. `ImportPassCache` (`pass-cache.ts`) states the same contract the
* same way for the ts-family lists.
*
* The positional pairing is load-bearing too and depends on it: `csharp.ts`
* caches POSITIONS into `normalized` and reads the answer out of `all`, so a
* reordering of either array alone silently re-points every cached position.
*/
export interface WorkspaceFileIndex {
/** Every path, backslashes normalized to `/`. Parallel to `all`. */
readonly normalized: string[];
readonly normalized: readonly string[];
/** Every path, exactly as it appears in the Set. Parallel to `normalized`. */
readonly all: string[];
readonly all: readonly string[];
/** Segment-suffix → first file (in Set iteration order) carrying that suffix. */
readonly index: SuffixIndex;
/**
@ -46,27 +73,22 @@ export interface WorkspaceFileIndex {
readonly normToRaw: Map<string, string>;
}
const WORKSPACE_FILE_INDEX_CACHE = new WeakMap<ReadonlySet<string>, WorkspaceFileIndex>();
export const getWorkspaceFileIndex = perFileSet(
(allFilePaths: ReadonlySet<string>): WorkspaceFileIndex => {
const all = [...allFilePaths];
const normalized = all.map((f) => f.replace(/\\/g, '/'));
const normToRaw = new Map<string, string>();
for (let i = 0; i < normalized.length; i++) {
// First wins, mirroring the `for (const raw of allFilePaths)` scans this
// replaces: they returned on the first match in iteration order.
if (!normToRaw.has(normalized[i])) normToRaw.set(normalized[i], all[i]);
}
export function getWorkspaceFileIndex(allFilePaths: ReadonlySet<string>): WorkspaceFileIndex {
const cached = WORKSPACE_FILE_INDEX_CACHE.get(allFilePaths);
if (cached !== undefined) return cached;
const all = [...allFilePaths];
const normalized = all.map((f) => f.replace(/\\/g, '/'));
const normToRaw = new Map<string, string>();
for (let i = 0; i < normalized.length; i++) {
// First wins, mirroring the `for (const raw of allFilePaths)` scans this
// replaces: they returned on the first match in iteration order.
if (!normToRaw.has(normalized[i])) normToRaw.set(normalized[i], all[i]);
}
const built: WorkspaceFileIndex = {
normalized,
all,
index: buildSuffixIndex(normalized, all),
normToRaw,
};
WORKSPACE_FILE_INDEX_CACHE.set(allFilePaths, built);
return built;
}
return {
normalized,
all,
index: buildSuffixIndex(normalized, all),
normToRaw,
};
},
);

29
gitnexus/src/core/ingestion/languages/c/import-target.ts
View file

@ -1,4 +1,5 @@
import { dirname, join } from 'path';
import { perFileSet } from '../../import-resolvers/per-file-set.js';
/**
* A workspace file path pre-decomposed for the suffix-match fallback:
@ -28,26 +29,20 @@ interface CSuffixCandidate {
* `WeakMap`-keyed so it is reclaimed with the pass (no cross-pass staleness).
* Shared by C and C++ (`resolveCppImportTarget` delegates here).
*/
const suffixIndexByPaths = new WeakMap<ReadonlySet<string>, Map<string, CSuffixCandidate[]>>();
function suffixIndex(allFilePaths: ReadonlySet<string>): Map<string, CSuffixCandidate[]> {
let index = suffixIndexByPaths.get(allFilePaths);
if (index === undefined) {
index = new Map<string, CSuffixCandidate[]>();
for (const original of allFilePaths) {
const normalized = original.replace(/\\/g, '/');
const basename = normalized.slice(normalized.lastIndexOf('/') + 1);
let bucket = index.get(basename);
if (bucket === undefined) {
bucket = [];
index.set(basename, bucket);
}
bucket.push({ original, normalized, depth: normalized.split('/').length });
const suffixIndex = perFileSet((allFilePaths: ReadonlySet<string>) => {
const index = new Map<string, CSuffixCandidate[]>();
for (const original of allFilePaths) {
const normalized = original.replace(/\\/g, '/');
const basename = normalized.slice(normalized.lastIndexOf('/') + 1);
let bucket = index.get(basename);
if (bucket === undefined) {
bucket = [];
index.set(basename, bucket);
}
suffixIndexByPaths.set(allFilePaths, index);
bucket.push({ original, normalized, depth: normalized.split('/').length });
}
return index;
}
});
/**
* Resolve a C #include path to a file in the workspace.

42
gitnexus/src/core/ingestion/languages/c/scope-resolver.ts
View file

@ -8,6 +8,7 @@ import { cArityCompatibility, cMergeBindings, resolveCImportTarget } from './ind
import { scanHeaderFiles } from './header-scan.js';
import { expandCWildcardNames, isStaticName, clearStaticNames } from './static-linkage.js';
import { applyCStaticLinkageSideChannel } from './capture-side-channel.js';
import { perFileSet } from '../../import-resolvers/per-file-set.js';
/**
* Per-pass memo of the augmented `#include`-resolution file set
@ -19,31 +20,26 @@ import { applyCStaticLinkageSideChannel } from './capture-side-channel.js';
* handing it a new set identity each time. Both `allFilePaths` (built once in
* scope-resolution `run.ts`) and the header set (`loadResolutionConfig`
* result) are stable per pass, so the union is built once and reused.
* `WeakMap`-keyed → reclaimed with the pass (no cross-pass staleness).
* Reclaimed with the pass (no cross-pass staleness).
*
* Two inputs, so two levels of `perFileSet` composed rather than a second
* primitive: the outer memo's value is the inner memo, and a function is an
* object, which is all `T extends object` asks for.
*
* The MEMO stays private to this file even though the C++ resolver's twin is
* byte-identical. The augmented set's IDENTITY is load-bearing downstream —
* C++ delegates to `resolveCImportTarget`, whose `suffixIndex` memo is keyed on
* exactly this set — so one memo shared across the two languages would hand
* each the other's index. Same builder-shared/memo-separate rule as
* `import-resolvers/pass-cache.ts`.
*/
const augmentedPathsByPass = new WeakMap<
ReadonlySet<string>,
WeakMap<ReadonlySet<string>, ReadonlySet<string>>
>();
function augmentedFilePaths(
allFilePaths: ReadonlySet<string>,
headerPaths: ReadonlySet<string>,
): ReadonlySet<string> {
let byHeaders = augmentedPathsByPass.get(allFilePaths);
if (byHeaders === undefined) {
byHeaders = new WeakMap();
augmentedPathsByPass.set(allFilePaths, byHeaders);
}
let augmented = byHeaders.get(headerPaths);
if (augmented === undefined) {
const augmentedFilePathsFor = perFileSet((allFilePaths: ReadonlySet<string>) =>
perFileSet((headerPaths: ReadonlySet<string>): ReadonlySet<string> => {
const set = new Set(allFilePaths);
for (const h of headerPaths) set.add(h);
augmented = set;
byHeaders.set(headerPaths, augmented);
}
return augmented;
}
return set;
}),
);
/**
* C `ScopeResolver` registered in `SCOPE_RESOLVERS` and consumed by
@ -94,7 +90,7 @@ export const cScopeResolver: ScopeResolver = {
return resolveCImportTarget(
targetRaw,
fromFile,
augmentedFilePaths(allFilePaths, headerPaths),
augmentedFilePathsFor(allFilePaths)(headerPaths),
);
}
return resolveCImportTarget(targetRaw, fromFile, allFilePaths);

21
gitnexus/src/core/ingestion/languages/c/static-linkage.ts
View file

@ -1,4 +1,5 @@
import type { ParsedFile, ScopeId, SymbolDefinition } from 'gitnexus-shared';
import { perFileSet } from '../../import-resolvers/per-file-set.js';
/**
* Per-file set of function names declared with `static` storage class.
@ -59,27 +60,23 @@ export function clearStaticNames(): void {
* thousands of resolved includes) that is ~10^10+ comparisons on a single
* thread — the dominant term in the scope-resolution finalize grind.
*
* Building the lookup once collapses it to O(R_include + F). `WeakMap`-keyed
* on the array so the index is reclaimed with the pass — no cross-pass
* Building the lookup once collapses it to O(R_include + F). `perFileSet` keys
* on the array identity so the index is reclaimed with the pass — no cross-pass
* staleness (mirrors the {@link clearStaticNames} discipline for server-mode
* / multi-repo reuse), and a fresh array transparently rebuilds.
*/
const moduleScopeIndexByPass = new WeakMap<readonly ParsedFile[], Map<ScopeId, ParsedFile>>();
function moduleScopeIndex(parsedFiles: readonly ParsedFile[]): Map<ScopeId, ParsedFile> {
let index = moduleScopeIndexByPass.get(parsedFiles);
if (index === undefined) {
index = new Map<ScopeId, ParsedFile>();
const moduleScopeIndex = perFileSet(
(parsedFiles: readonly ParsedFile[]): Map<ScopeId, ParsedFile> => {
const index = new Map<ScopeId, ParsedFile>();
// First-wins to preserve `Array.find` semantics (returns the first match).
// `moduleScope` is unique per file in practice, so collisions are absent;
// the guard only formalises identical behaviour to the prior `.find`.
for (const p of parsedFiles) {
if (!index.has(p.moduleScope)) index.set(p.moduleScope, p);
}
moduleScopeIndexByPass.set(parsedFiles, index);
}
return index;
}
return index;
},
);
/**
* Return the names visible through a C wildcard import (`#include`).

78
gitnexus/src/core/ingestion/languages/cobol/scope-resolver.ts
View file

@ -12,12 +12,71 @@
import path from 'node:path';
import type { ParsedFile } from 'gitnexus-shared';
import { SupportedLanguages } from 'gitnexus-shared';
import { perFileSet } from '../../import-resolvers/per-file-set.js';
import { populateClassOwnedMembers } from '../../scope-resolution/scope/walkers.js';
import type { ScopeResolver } from '../../scope-resolution/contract/scope-resolver.js';
import { cobolProvider } from '../cobol.js';
// Copybook file extensions for COPY name resolution
const COPYBOOK_EXTENSIONS = new Set(['.cpy', '.copybook']);
// COBOL source files, searched only after every copybook has missed.
const COBOL_SOURCE_EXTENSIONS = new Set(['.cbl', '.cob', '.cobol']);
/**
* Uppercased-basename → first file carrying it, one map PER TIER, memoized on
* the `allFilePaths` Set identity (#2908).
*
* `resolveImportTarget` used to run two full workspace scans per `COPY` — one
* for the copybook tier, one for the source tier — each calling `path.extname`
* + `path.basename` + `toUpperCase` on every entry. A `COPY` of a member that
* lives outside the repo (the common case: vendor and system copybooks) missed
* in both, so both scans always ran to completion, making resolution
* O(copies × files). The orchestrator passes the SAME Set to every import in a
* pass (`pipeline/run.ts` builds it once), so a `WeakMap` keyed on that Set
* turns the scans into one build per run.
*
* Two tiers rather than one map is the tie-break, not a stylistic choice: a
* `.cpy`/`.copybook` hit beats a `.cbl`/`.cob`/`.cobol` hit even when the source
* file comes FIRST in Set-iteration order, which is exactly what collapsing the
* tiers into a single first-wins map would silently discard. Within a tier the
* first file in Set-iteration order wins, mirroring the `return` on first match
* in the scans this replaces.
*
* The per-file key is derived with the same `path.extname(fp).toLowerCase()` →
* `path.basename(fp, ext)` → `toUpperCase()` sequence the scans used, including
* its quirk: `path.basename` strips the suffix only on an exact, case-sensitive
* match, so `Foo.CPY` indexes under `FOO.CPY` rather than `FOO`. Node's `path`
* stays in the loop for the same reason — on POSIX it does not treat `\` as a
* separator, and hand-rolled slicing on `/` would start resolving backslash
* paths the scans never resolved.
*/
interface CobolCopyIndex {
/** `.cpy` / `.copybook` files — tier 1. */
readonly copybooks: ReadonlyMap<string, string>;
/** `.cbl` / `.cob` / `.cobol` files — tier 2. */
readonly sources: ReadonlyMap<string, string>;
}
const getCobolCopyIndex = perFileSet((allFilePaths: ReadonlySet<string>): CobolCopyIndex => {
const copybooks = new Map<string, string>();
const sources = new Map<string, string>();
// One pass builds both tiers: the two scans walked the same files and
// classified each by the same extension test.
for (const fp of allFilePaths) {
const ext = path.extname(fp).toLowerCase();
const tier = COPYBOOK_EXTENSIONS.has(ext)
? copybooks
: COBOL_SOURCE_EXTENSIONS.has(ext)
? sources
: undefined;
if (tier === undefined) continue;
const basename = path.basename(fp, ext).toUpperCase();
// First in Set-iteration order wins, as the scans' first-match `return` did.
if (!tier.has(basename)) tier.set(basename, fp);
}
return { copybooks, sources };
});
const cobolScopeResolver: ScopeResolver = {
language: SupportedLanguages.Cobol,
@ -27,22 +86,9 @@ const cobolScopeResolver: ScopeResolver = {
// ── Resolve COPY bookname to file path ─────────────────────────────
resolveImportTarget: (targetRaw, _fromFile, allFilePaths) => {
const upper = targetRaw.toUpperCase();
// Check copybook files first
for (const fp of allFilePaths) {
const ext = path.extname(fp).toLowerCase();
if (!COPYBOOK_EXTENSIONS.has(ext)) continue;
const basename = path.basename(fp, ext).toUpperCase();
if (basename === upper) return fp;
}
// Also search COBOL source files (.cbl, .cob, .cobol)
const COBOL_SOURCE_EXTS = new Set(['.cbl', '.cob', '.cobol']);
for (const fp of allFilePaths) {
const ext = path.extname(fp).toLowerCase();
if (!COBOL_SOURCE_EXTS.has(ext)) continue;
const basename = path.basename(fp, ext).toUpperCase();
if (basename === upper) return fp;
}
return null;
const index = getCobolCopyIndex(allFilePaths);
// Copybooks first, then COBOL sources — the tier order IS the tie-break.
return index.copybooks.get(upper) ?? index.sources.get(upper) ?? null;
},
// COBOL has no binding-merge rules beyond the default (local-first-then-imports).

23
gitnexus/src/core/ingestion/languages/cpp/file-local-linkage.ts
View file

@ -1,4 +1,5 @@
import type { ParsedFile, Scope, ScopeId, SymbolDefinition } from 'gitnexus-shared';
import { perFileSet } from '../../import-resolvers/per-file-set.js';
import { isCppInlineNamespaceScope } from './inline-namespaces.js';
/**
@ -283,24 +284,20 @@ export function isCppDefGloballyVisible(filePath: string, nodeId: string): boole
* `parsedFiles` reference; the old `parsedFiles.find(...)` was therefore O(F)
* per edge → O(R·F) overall (at kernel scale the ~25–30k `.h` headers are
* classified C++, so this fires hard — the C twin in `c/static-linkage.ts`).
* Building the lookup once collapses it to O(R+F). `WeakMap`-keyed so it is
* reclaimed with the pass (no cross-pass staleness; mirrors
* {@link clearFileLocalNames}).
* Building the lookup once collapses it to O(R+F). `perFileSet` keys on the
* array identity so it is reclaimed with the pass (no cross-pass staleness;
* mirrors {@link clearFileLocalNames}).
*/
const moduleScopeIndexByPass = new WeakMap<readonly ParsedFile[], Map<ScopeId, ParsedFile>>();
function moduleScopeIndex(parsedFiles: readonly ParsedFile[]): Map<ScopeId, ParsedFile> {
let index = moduleScopeIndexByPass.get(parsedFiles);
if (index === undefined) {
index = new Map<ScopeId, ParsedFile>();
const moduleScopeIndex = perFileSet(
(parsedFiles: readonly ParsedFile[]): Map<ScopeId, ParsedFile> => {
const index = new Map<ScopeId, ParsedFile>();
// First-wins to preserve `Array.find` semantics (returns the first match).
for (const p of parsedFiles) {
if (!index.has(p.moduleScope)) index.set(p.moduleScope, p);
}
moduleScopeIndexByPass.set(parsedFiles, index);
}
return index;
}
return index;
},
);
export function expandCppWildcardNames(
targetModuleScope: ScopeId,

42
gitnexus/src/core/ingestion/languages/cpp/scope-resolver.ts
View file

@ -48,6 +48,7 @@ import {
resolveCppReceiverMember,
} from './member-lookup.js';
import { stripCppSpecifiers } from './interpret.js';
import { perFileSet } from '../../import-resolvers/per-file-set.js';
/** A pointee worth binding: a bare identifier, not `T**`, `T[]`, `A::B` or a
* template spelling. Hoisted — a literal here would mint a fresh RegExp on
@ -61,32 +62,25 @@ const CPP_SIMPLE_POINTEE_RE = /^[A-Za-z_]\w*$/;
* a fresh ~F-entry `Set` on every call AND defeated the shared
* `resolveCImportTarget` suffix-index memo (in `c/import-target.ts`) by handing
* it a new set identity each time. Both inputs are stable per pass, so the
* union is built once and reused. `WeakMap`-keyed → reclaimed with the pass.
* (Twin of the C resolver's `augmentedFilePaths`.)
* union is built once and reused. Reclaimed with the pass.
*
* Two inputs, so two levels of `perFileSet` composed rather than a second
* primitive: the outer memo's value is the inner memo, and a function is an
* object, which is all `T extends object` asks for.
*
* (Twin of the C resolver's `augmentedFilePathsFor`.) The two memos stay
* SEPARATE deliberately. C++ delegates to `resolveCImportTarget`, whose
* `suffixIndex` memo is keyed on the augmented set, so a single memo shared
* with C would hand each language the other's index — same
* builder-shared/memo-separate rule as `import-resolvers/pass-cache.ts`.
*/
const augmentedPathsByPass = new WeakMap<
ReadonlySet<string>,
WeakMap<ReadonlySet<string>, ReadonlySet<string>>
>();
function augmentedFilePaths(
allFilePaths: ReadonlySet<string>,
headerPaths: ReadonlySet<string>,
): ReadonlySet<string> {
let byHeaders = augmentedPathsByPass.get(allFilePaths);
if (byHeaders === undefined) {
byHeaders = new WeakMap();
augmentedPathsByPass.set(allFilePaths, byHeaders);
}
let augmented = byHeaders.get(headerPaths);
if (augmented === undefined) {
const augmentedFilePathsFor = perFileSet((allFilePaths: ReadonlySet<string>) =>
perFileSet((headerPaths: ReadonlySet<string>): ReadonlySet<string> => {
const set = new Set(allFilePaths);
for (const h of headerPaths) set.add(h);
augmented = set;
byHeaders.set(headerPaths, augmented);
}
return augmented;
}
return set;
}),
);
/**
* C++ `ScopeResolver` registered in `SCOPE_RESOLVERS` and consumed by
@ -128,7 +122,7 @@ export const cppScopeResolver: ScopeResolver = {
return resolveCppImportTarget(
targetRaw,
fromFile,
augmentedFilePaths(allFilePaths, headerPaths),
augmentedFilePathsFor(allFilePaths)(headerPaths),
);
}
return resolveCppImportTarget(targetRaw, fromFile, allFilePaths);

19
gitnexus/src/core/ingestion/languages/csharp/import-target.ts
View file

@ -31,6 +31,7 @@ import {
firstFileDirectlyInPkgDir,
type PackageDirIndex,
} from '../../import-resolvers/package-dir-index.js';
import { perFileSet } from '../../import-resolvers/per-file-set.js';
import { csharpSuffixFallbackAllowed } from '../../csharp-namespace-gate.js';
export interface CsharpResolveContext {
@ -46,15 +47,10 @@ export interface CsharpResolveContext {
* `import-resolvers/package-dir-index.ts`), which the no-csproj path calls once
* for the direct match and then up to once per stripped namespace prefix.
*/
const csharpDirIndexCache = new WeakMap<ReadonlySet<string>, PackageDirIndex>();
function getCsharpDirIndex(allFilePaths: ReadonlySet<string>): PackageDirIndex {
const cached = csharpDirIndexCache.get(allFilePaths);
if (cached) return cached;
const built = buildPackageDirIndex(allFilePaths, (normalized) => normalized.endsWith('.cs'));
csharpDirIndexCache.set(allFilePaths, built);
return built;
}
const getCsharpDirIndex = perFileSet(
(allFilePaths: ReadonlySet<string>): PackageDirIndex =>
buildPackageDirIndex(allFilePaths, (normalized) => normalized.endsWith('.cs')),
);
export function resolveCsharpImportTarget(
parsedImport: ParsedImport,
@ -69,12 +65,11 @@ export function resolveCsharpImportTarget(
const csharpConfigs = ctx.csharpConfigs ?? [];
if (csharpConfigs.length > 0) {
const { normalized, all, index } = getWorkspaceFileIndex(ctx.allFilePaths);
const { index } = getWorkspaceFileIndex(ctx.allFilePaths);
const fromCsproj = resolveCSharpImportInternal(
targetRaw,
[...csharpConfigs],
normalized,
all,
ctx.allFilePaths,
index,
evidence,
);

13
gitnexus/src/core/ingestion/languages/dart/import-target.ts
View file

@ -13,6 +13,7 @@
* `targetRaw` arrives already quote-stripped from `interpretDartImport`.
*/
import { perFileSet } from '../../import-resolvers/per-file-set.js';
import { DART_HERITAGE_PREFIX } from './interpret.js';
/**
@ -35,11 +36,7 @@ interface DartFileIndex {
readonly byBasename: Map<string, string[]>;
}
const DART_FILE_INDEX_CACHE = new WeakMap<ReadonlySet<string>, DartFileIndex>();
function getDartFileIndex(allFilePaths: ReadonlySet<string>): DartFileIndex {
const cached = DART_FILE_INDEX_CACHE.get(allFilePaths);
if (cached !== undefined) return cached;
const getDartFileIndex = perFileSet((allFilePaths: ReadonlySet<string>): DartFileIndex => {
const byBasename = new Map<string, string[]>();
for (const fp of allFilePaths) {
const base = fp.slice(fp.lastIndexOf('/') + 1);
@ -50,10 +47,8 @@ function getDartFileIndex(allFilePaths: ReadonlySet<string>): DartFileIndex {
}
bucket.push(fp);
}
const built: DartFileIndex = { byBasename };
DART_FILE_INDEX_CACHE.set(allFilePaths, built);
return built;
}
return { byBasename };
});
/** First file (in Set-iteration order) that IS `candidate` or ends with
* `/<candidate>` — the exact predicate of the scans this replaces. */

24
gitnexus/src/core/ingestion/languages/go/import-target.ts
View file

@ -5,6 +5,7 @@ import {
sortedRootFiles,
type PackageDirIndex,
} from '../../import-resolvers/package-dir-index.js';
import { perFileSet } from '../../import-resolvers/per-file-set.js';
/**
* Resolve a Go import path to ALL .go files in the matching package directory.
@ -56,6 +57,11 @@ export function resolveGoImportTarget(
return null;
}
/** Go packages exclude `_test.go` files: they are a separate package. */
function isGoPackageFile(normalized: string): boolean {
return normalized.endsWith('.go') && !normalized.endsWith('_test.go');
}
/**
* Package index over the file set, memoized on the Set's identity (#2877).
*
@ -69,20 +75,10 @@ export function resolveGoImportTarget(
* is built once per run. `resolveGoImportTarget` must therefore never copy the
* Set before this point — see `import-resolvers/workspace-file-index.ts`.
*/
const GO_PACKAGE_INDEX_CACHE = new WeakMap<ReadonlySet<string>, PackageDirIndex>();
/** Go packages exclude `_test.go` files: they are a separate package. */
function isGoPackageFile(normalized: string): boolean {
return normalized.endsWith('.go') && !normalized.endsWith('_test.go');
}
function getGoPackageIndex(allFilePaths: ReadonlySet<string>): PackageDirIndex {
const cached = GO_PACKAGE_INDEX_CACHE.get(allFilePaths);
if (cached !== undefined) return cached;
const built = buildPackageDirIndex(allFilePaths, isGoPackageFile);
GO_PACKAGE_INDEX_CACHE.set(allFilePaths, built);
return built;
}
const getGoPackageIndex = perFileSet(
(allFilePaths: ReadonlySet<string>): PackageDirIndex =>
buildPackageDirIndex(allFilePaths, isGoPackageFile),
);
function findRootPackageFiles(allFilePaths: ReadonlySet<string>): string[] {
return sortedRootFiles(getGoPackageIndex(allFilePaths));

194
gitnexus/src/core/ingestion/languages/java/import-target.ts
View file

@ -8,27 +8,79 @@
* 4. Progressive prefix stripping for non-standard layouts
*
* Returns `null` for unresolvable / JDK imports.
*
* ## Why the scans are gone (#2908)
*
* Every leg above used to be answered by `for (const raw of ctx.allFilePaths)`,
* and the stripping loop ran that scan again per stripped segment — so one
* unresolvable `import a.b.c.D;` (the COMMON case: JDK and third-party imports
* run the whole cascade to completion) cost four full workspace passes. This is
* byte-for-byte the shape C# carried until #2878; both now read the same two
* per-file-set indexes, memoized on the Set's identity:
*
* - `getWorkspaceFileIndex` — `normToRaw` (whole-path lookup) and `index`
* (segment-suffix lookup);
* - `getJavaDirIndex` — `firstFileDirectlyInPkgDir`'s package-directory index.
*
* ## The tie-breaks the scans encoded, and where they now live
*
* 1. The first pass `break`s on an exact whole-path hit but keeps scanning
* otherwise, then returns `exactFile ?? suffixFile ?? directoryChild`. So an
* exact match wins over a suffix or directory-child match found EARLIER in
* iteration order — hence `normToRaw` before `index`, which conflates the
* two (see `resolveDirectMatch`).
* 2. The stripping loop instead `return`s mid-scan on `f === tailFile ||
* f.endsWith(tailSuffix)`, i.e. at the first hit of EITHER, and only returns
* its directory child after the scan completes. So file/suffix beats
* directory child within one `skip` level regardless of order, and the
* conflated `index.get` is the CORRECT lookup there (see
* `resolveByProgressiveStripping`).
* 3. Wildcard imports drop their trailing `.*` before resolution, so
* `com.example.*` resolves as the package directory.
* 4. `.java` filter and backslash normalization, with the RAW path returned:
* the indexes normalize for their keys and hand back the raw Set member, and
* only a `.java` file can carry a `…/<name>.java` suffix key, so the
* extension filter is implied on the file/suffix legs and explicit in the
* directory index's `accept`.
* 5. The directory-child leg matched on the FIRST `'/' + pathLike + '/'`
* occurrence, so `com/example/com/example/Deep.java` does NOT answer
* `com.example`. `firstFileDirectlyInPkgDir` encodes exactly that rule (see
* the header of `import-resolvers/package-dir-index.ts`).
*/
import type { ParsedImport, WorkspaceIndex } from 'gitnexus-shared';
import {
getWorkspaceFileIndex,
type WorkspaceFileIndex,
} from '../../import-resolvers/workspace-file-index.js';
import {
buildPackageDirIndex,
firstFileDirectlyInPkgDir,
type PackageDirIndex,
} from '../../import-resolvers/package-dir-index.js';
import { perFileSet } from '../../import-resolvers/per-file-set.js';
export interface JavaResolveContext {
readonly fromFile: string;
readonly allFilePaths: ReadonlySet<string>;
}
/**
* Package-directory index over the `.java` files, memoized on the Set's
* identity. Feeds `firstFileDirectlyInPkgDir`, which is called once for the
* direct match and then up to once per stripped package prefix.
*/
const getJavaDirIndex = perFileSet(
(allFilePaths: ReadonlySet<string>): PackageDirIndex =>
buildPackageDirIndex(allFilePaths, (normalized) => normalized.endsWith('.java')),
);
export function resolveJavaImportTarget(
parsedImport: ParsedImport,
workspaceIndex: WorkspaceIndex,
): string | null {
const ctx = workspaceIndex as JavaResolveContext | undefined;
if (
ctx === undefined ||
typeof (ctx as { fromFile?: unknown }).fromFile !== 'string' ||
!((ctx as { allFilePaths?: unknown }).allFilePaths instanceof Set)
) {
return null;
}
const ctx = narrowContext(workspaceIndex);
if (ctx === null) return null;
if (parsedImport.kind === 'dynamic-unresolved') return null;
if (parsedImport.targetRaw === null || parsedImport.targetRaw === '') return null;
@ -40,69 +92,87 @@ export function resolveJavaImportTarget(
// Package path: `com.example.User` → `com/example/User`
const pathLike = target.replace(/\./g, '/');
const suffix = `/${pathLike}`;
let exactFile: string | null = null;
let suffixFile: string | null = null;
let directoryChild: string | null = null;
const dirPrefix = `${pathLike}/`;
const suffixDirPrefix = `/${dirPrefix}`;
const ws = getWorkspaceFileIndex(ctx.allFilePaths);
const dirs = getJavaDirIndex(ctx.allFilePaths);
for (const raw of ctx.allFilePaths) {
const f = raw.replace(/\\/g, '/');
if (!f.endsWith('.java')) continue;
if (f === `${pathLike}.java`) {
exactFile = raw;
break;
}
if (suffixFile === null && f.endsWith(`${suffix}.java`)) {
suffixFile = raw;
}
if (directoryChild === null) {
const atRoot = f.startsWith(dirPrefix);
const atNested = f.includes(suffixDirPrefix);
if (atRoot || atNested) {
const idx = atRoot ? 0 : f.indexOf(suffixDirPrefix) + 1;
const after = f.slice(idx + dirPrefix.length);
if (after.length > 0 && !after.includes('/')) {
directoryChild = raw;
}
}
}
}
if (exactFile !== null) return exactFile;
if (suffixFile !== null) return suffixFile;
if (directoryChild !== null) return directoryChild;
const direct = resolveDirectMatch(ws, dirs, pathLike);
if (direct !== null) return direct;
// Progressive prefix stripping — handles `import com.example.User;`
// in a repo laid out `User.java` (no `com/example/` prefix).
return resolveByProgressiveStripping(ws, dirs, pathLike);
}
/**
* `WorkspaceIndex` is an opaque `unknown` placeholder in the shared contract;
* the orchestrator hands us a `JavaResolveContext`-shaped object. Narrow
* structurally rather than via a cast chain so unexpected shapes fail cleanly.
*/
function narrowContext(workspaceIndex: WorkspaceIndex): JavaResolveContext | null {
const ctx = workspaceIndex as JavaResolveContext | undefined;
if (
ctx === undefined ||
typeof (ctx as { fromFile?: unknown }).fromFile !== 'string' ||
!((ctx as { allFilePaths?: unknown }).allFilePaths instanceof Set)
) {
return null;
}
return ctx;
}
/**
* First-pass resolution against the full package path:
* exact whole-path file > nested suffix file > first `.java` directly inside
* the package directory.
*/
function resolveDirectMatch(
ws: WorkspaceFileIndex,
dirs: PackageDirIndex,
pathLike: string,
): string | null {
const exactName = `${pathLike}.java`;
// The scan `break`s here, so an exact whole-path match wins even when a
// `…/<exactName>` suffix match appeared EARLIER in iteration order. The two
// lookups therefore stay separate: `index.get` conflates them and would
// return the earlier suffix hit.
const exact = ws.normToRaw.get(exactName);
if (exact !== undefined) return exact;
// No whole-path file exists, so every segment-suffix hit is a `/<exactName>`
// match and `index.get` yields the first one in iteration order — exactly the
// `suffixFile` the scan kept. Only a `.java` file can carry a `.java` suffix
// key, so the old `endsWith('.java')` filter is implied.
const suffixFile = ws.index.get(exactName);
if (suffixFile !== undefined) return suffixFile;
// First `.java` file living directly inside the package directory `pathLike`
// (at repo root or nested under a source-root prefix), not deeper — the leg
// wildcard imports land on.
return firstFileDirectlyInPkgDir(dirs, pathLike);
}
/**
* Try each suffix of the package path against `.java` files and directories,
* stripping leading segments one at a time. Models `import com.example.User;`
* resolving to `User.java` in a repo laid out without the `com/example/` prefix.
*/
function resolveByProgressiveStripping(
ws: WorkspaceFileIndex,
dirs: PackageDirIndex,
pathLike: string,
): string | null {
const segments = pathLike.split('/').filter(Boolean);
for (let skip = 1; skip < segments.length; skip++) {
const tail = segments.slice(skip).join('/');
if (tail === '') continue;
const tailFile = `${tail}.java`;
const tailSuffix = `/${tailFile}`;
const tailDir = `${tail}/`;
const tailSuffixDir = `/${tailDir}`;
let tailDirectChild: string | null = null;
for (const raw of ctx.allFilePaths) {
const f = raw.replace(/\\/g, '/');
if (!f.endsWith('.java')) continue;
if (f === tailFile) return raw;
if (f.endsWith(tailSuffix)) return raw;
if (tailDirectChild === null) {
const atRoot = f.startsWith(tailDir);
const atNested = f.includes(tailSuffixDir);
if (atRoot || atNested) {
const idx = atRoot ? 0 : f.indexOf(tailSuffixDir) + 1;
const after = f.slice(idx + tailDir.length);
if (after.length > 0 && !after.includes('/')) tailDirectChild = raw;
}
}
}
if (tailDirectChild !== null) return tailDirectChild;
// `f === tailFile || f.endsWith('/' + tailFile)`, first in iteration order —
// the scan returned at the first hit of EITHER, with no exact-wins rule,
// so here the conflated suffix lookup is the right one.
const tailFileMatch = ws.index.get(`${tail}.java`);
if (tailFileMatch !== undefined) return tailFileMatch;
// Collected mid-scan but returned only after it, so the file/suffix hit
// above beats it even when this one came first in iteration order.
const child = firstFileDirectlyInPkgDir(dirs, tail);
if (child !== null) return child;
}
return null;
}

88
gitnexus/src/core/ingestion/languages/javascript/import-target.ts
View file

@ -18,25 +18,81 @@
* tsconfig-based aliases alongside JavaScript can still resolve via the
* standard extension-suffix fallback; the alias branch is a no-op when
* `tsconfigPaths` is null.
*
* ## The suffix index changes bare-specifier answers (PR #2911)
*
* Supplying `index` is not only a speed-up: `suffixResolve` answers a different
* question with one than without. Without an index it tests
* `filePath.endsWith('/' + suffix)`, so only a PROPER suffix can match; with
* one it reads `buildSuffixIndex`, which indexes `j = 0` and therefore matches
* WHOLE paths too. Two classes of answer move, both only on the bare/absolute
* specifier leg (relative imports resolve by exact `Set.has` and never reach
* it), and both toward what TypeScript and Vue have always answered:
*
* 1. a repo-root file becomes reachable at all — `require('config')` now
* finds `config.js`, where before no proper suffix existed and the answer
* was null;
* 2. a whole-path candidate outranks a proper-suffix candidate found at a
* SHORTER path suffix or a later extension — `import 'app/main'` resolved
* to `node_modules/dep/lib/main.js` (the first `/main.js` in file order)
* and now resolves to `app/main.js`.
*
* Measured over 211 200 old-vs-new pairs there is no third class: the index
* never loses a match the scan found, and its answer is never matched at a less
* specific (path-part, extension) position. `test/unit/scope-resolution/
* javascript-import-target-parity.test.ts` is that differential, and pins both
* classes by witness.
*/
import { SupportedLanguages } from 'gitnexus-shared';
import { resolveTsTarget, type TsResolveContext } from '../typescript/import-target.js';
import { buildImportPassCache } from '../../import-resolvers/pass-cache.js';
import { perFileSet } from '../../import-resolvers/per-file-set.js';
export type JsResolveContext = TsResolveContext;
type PassCache = {
readonly key: ReadonlySet<string>;
readonly allFilePaths: Set<string>;
readonly allFileList: readonly string[];
readonly normalizedFileList: readonly string[];
readonly resolveCache: Map<string, string | null>;
};
/**
* Everything `resolveTsTarget` derives from one workspace file set, built once
* per set rather than once per import.
*
* `index` is not optional, and its absence was the defect (PR #2911). The
* TypeScript adapter has carried a `SuffixIndex` since #1918; this one did not,
* so every JavaScript import reached `suffixResolve` with `index === undefined`
* and took its linear-`findIndex` fallback — one pass over `normalizedFileList`
* per path part per extension, and `EXTENSIONS` has ~39 entries. Measured on
* mostly-missing bare specifiers (imports scaling with files, as in
* `bench/import-target/`): 6448.9 µs per import at 2000 files and 25972.6 µs at
* 8000 — 4.12x the per-import cost for 4x the files, which is O(imports ×
* files) — against 25.0 / 27.0 µs for TypeScript over the identical corpus.
* With the index it is 28.5 / 27.4 µs and the scaling factor is 1.09x.
*
* No instrument on the #2901-#2909 branch could see it: `CountingSet` counts
* traversals of the SET, and this scan walks the materialized array behind it.
* See `test/integration/javascript-import-index-reuse.test.ts` for the guard
* that can.
*
* Memoized on the `allFilePaths` Set identity, like every other language's
* import index (`import-resolvers/workspace-file-index.ts` and friends).
*
* A single-slot `let cached` keyed on `cached.key !== allFilePaths` — what this
* adapter used before — is correct for one file set and degenerate for two:
* alternating calls across two sets rebuild everything every time. Measured on
* the TypeScript adapter at 4000 files × 400 imports: 12.0 ms for one set,
* 1438.2 ms alternating between two (120x). A `WeakMap` has no such state to
* thrash, which is also what lets this adapter carry the standard
* `expectDistinctFileSetsGetOwnIndex` guard every other indexed adapter
* carries.
*
* The Set must be passed THROUGH by the caller, never copied: a defensive
* `new Set(allFilePaths)` at the adapter boundary hands a fresh key per import
* and restores the per-import rebuild (PR #1918 review P1).
*/
const passCacheFor = perFileSet(buildImportPassCache);
/**
* Build a memoized `resolveImportTarget` adapter for JavaScript.
* Caches the derived arrays and per-pass resolve cache across
* `resolveImportTarget` calls within a single workspace pass.
* Caches the derived arrays, the suffix index and the per-pass resolve cache
* across `resolveImportTarget` calls over one workspace file set.
*/
export function makeJsResolveImportTarget(): (
targetRaw: string,
@ -44,19 +100,8 @@ export function makeJsResolveImportTarget(): (
allFilePaths: ReadonlySet<string>,
resolutionConfig?: unknown,
) => string | readonly string[] | null {
let cached: PassCache | null = null;
return (targetRaw, fromFile, allFilePaths) => {
if (cached === null || cached.key !== allFilePaths) {
const allFileList = Array.from(allFilePaths);
cached = {
key: allFilePaths,
allFilePaths: new Set(allFilePaths),
allFileList,
normalizedFileList: allFileList.map((f) => f.toLowerCase()),
resolveCache: new Map(),
};
}
const cached = passCacheFor(allFilePaths);
const ws: JsResolveContext = {
fromFile,
@ -64,6 +109,7 @@ export function makeJsResolveImportTarget(): (
allFilePaths: cached.allFilePaths,
allFileList: cached.allFileList,
normalizedFileList: cached.normalizedFileList,
index: cached.index,
resolveCache: cached.resolveCache,
tsconfigPaths: null,
};

20
gitnexus/src/core/ingestion/languages/kotlin/import-target.ts
View file

@ -1,6 +1,6 @@
import type { ParsedImport, WorkspaceIndex } from 'gitnexus-shared';
import { KOTLIN_EXTENSIONS } from '../../import-resolvers/jvm.js';
import { recordKotlinFileIndexBuild } from './index-stats.js';
import { perFileSet } from '../../import-resolvers/per-file-set.js';
export interface KotlinResolveContext {
readonly fromFile: string;
@ -180,14 +180,10 @@ interface KotlinFileIndex {
readonly dirChildren: Map<string, readonly string[]>;
}
const KOTLIN_FILE_INDEX_CACHE = new WeakMap<ReadonlySet<string>, KotlinFileIndex>();
function getKotlinFileIndex(allFilePaths: ReadonlySet<string>): KotlinFileIndex {
const cached = KOTLIN_FILE_INDEX_CACHE.get(allFilePaths);
if (cached !== undefined) return cached;
// Cache miss: materialize a fresh index. Counted so a test can assert this
// happens once per run, not once per import.
recordKotlinFileIndexBuild();
const getKotlinFileIndex = perFileSet((allFilePaths: ReadonlySet<string>): KotlinFileIndex => {
// Runs on a cache miss only. That it happens once per run and not once per
// import is asserted by counting traversals of the Set itself, in
// `test/integration/kotlin-import-index-reuse.test.ts` (#2909).
const exactByStem = new Map<string, string>();
const suffixByStem = new Map<string, string>();
@ -255,10 +251,8 @@ function getKotlinFileIndex(allFilePaths: ReadonlySet<string>): KotlinFileIndex
// future mutation is a loud TypeError instead of a silent edge move.
for (const bucket of dirChildren.values()) Object.freeze(bucket);
const index: KotlinFileIndex = { exactByStem, suffixByStem, dirChildren };
KOTLIN_FILE_INDEX_CACHE.set(allFilePaths, index);
return index;
}
return { exactByStem, suffixByStem, dirChildren };
});
function addChild(dirChildren: Map<string, string[]>, dir: string, raw: string): void {
const bucket = dirChildren.get(dir);

29
gitnexus/src/core/ingestion/languages/kotlin/index-stats.ts
View file

@ -1,29 +0,0 @@
/**
* Build counter for the per-file-set Kotlin import-resolution index
* (`getKotlinFileIndex` in `import-target.ts`).
*
* A "build" is a `WeakMap` cache MISS that materializes a fresh
* `KotlinFileIndex` (O(files)). Mirrors `../python/index-stats.ts`: the counter
* is always live rather than gated behind a profiling env var, because an index
* build happens at most once per resolution run, so the single increment is
* negligible and an unconditional counter avoids env-var load-order fragility
* in tests.
*
* Used by `test/integration/kotlin-import-index-reuse.test.ts` to assert the
* index is reused across imports (built once per run) rather than rebuilt per
* import — the regression guard for the quadratic resolution this replaced.
*/
let INDEX_BUILDS = 0;
export function recordKotlinFileIndexBuild(): void {
INDEX_BUILDS++;
}
export function getKotlinFileIndexBuildCount(): number {
return INDEX_BUILDS;
}
export function resetKotlinFileIndexBuildCount(): void {
INDEX_BUILDS = 0;
}

245
gitnexus/src/core/ingestion/languages/php/import-target.ts
View file

@ -18,6 +18,9 @@
import type { ParsedFile, ParsedImport, WorkspaceIndex } from 'gitnexus-shared';
import type { ImportResolutionContext } from '../../scope-resolution/contract/scope-resolver.js';
import { resolvePhpImportInternal } from '../../import-resolvers/php.js';
import type { SuffixIndex } from '../../import-resolvers/utils.js';
import { perFileSet } from '../../import-resolvers/per-file-set.js';
import { getWorkspaceFileIndex } from '../../import-resolvers/workspace-file-index.js';
import type { ComposerConfig } from '../../language-config.js';
import { readFileSync } from 'node:fs';
import { join } from 'node:path';
@ -72,12 +75,6 @@ function namespaceDirectories(
return [...directories];
}
// A scope-resolution pass shares one stable parsedFiles array across imports.
const phpDirectoryIndexCache = new WeakMap<
readonly ParsedFile[],
ReadonlyMap<string, readonly ParsedFile[]>
>();
function parentDirectory(filePath: string): string {
const normalizedPath = normalizePhpPath(filePath);
const separator = normalizedPath.lastIndexOf('/');
@ -98,24 +95,210 @@ function directoryAliases(filePath: string): string[] {
return [...aliases];
}
function filesByDirectory(
parsedFiles: readonly ParsedFile[],
): ReadonlyMap<string, readonly ParsedFile[]> {
const cached = phpDirectoryIndexCache.get(parsedFiles);
if (cached) return cached;
const mutable = new Map<string, ParsedFile[]>();
for (const parsed of parsedFiles) {
for (const directory of directoryAliases(parsed.filePath)) {
const files = mutable.get(directory) ?? [];
files.push(parsed);
mutable.set(directory, files);
/**
* Directory alias → the files under it, built once per pass.
*
* A scope-resolution pass shares one stable `parsedFiles` array across imports,
* so the array identity is the memo key — see `perFileSet`.
*/
const filesByDirectory = perFileSet(
(parsedFiles: readonly ParsedFile[]): ReadonlyMap<string, readonly ParsedFile[]> => {
const mutable = new Map<string, ParsedFile[]>();
for (const parsed of parsedFiles) {
for (const directory of directoryAliases(parsed.filePath)) {
const files = mutable.get(directory) ?? [];
files.push(parsed);
mutable.set(directory, files);
}
}
}
phpDirectoryIndexCache.set(parsedFiles, mutable);
return mutable;
return mutable;
},
);
// ─── workspace index (#2901) ───────────────────────────────────────────────
/**
* PHP's view of the shared per-file-set workspace index.
*
* Both adapters below used to materialize `[...allFilePaths]` twice per import
* and then hand `resolvePhpImportInternal` an `index` of `undefined`, which
* dropped it onto `suffixResolve`'s linear `findIndex` — one full pass over
* every file per path-part × per extension (≈50 extensions). That is the 98 ms
* per import measured at 20k files, and the arrays were the small half of it.
*
* PASSING THE SHARED `SuffixIndex` STRAIGHT THROUGH IS NOT A HOIST — IT MOVES
* IMPORTS EDGES. `resolvePhpImportInternal` reads the index at three sites, and
* all three answer a DIFFERENT question than the scan they short-circuit
* (measured, one example each):
*
* 1. `index.getInsensitive(filePath)` on the PSR-4 class-style leg has no
* no-index counterpart at all — that leg is `allFiles.has(filePath)`, an
* exact whole-path test. The index turns it into a case-insensitive SUFFIX
* probe, so `App\Models\User` under `psr-4: {"App\\": "src"}` would start
* matching `vendor/x/src/models/user.php`.
* 2. `index.getFilesInDir(nsDir, '.php')` is keyed on every directory SUFFIX,
* while the scan it replaces is anchored at the repo root
* (`f.startsWith(nsDir + '/')`). With `app/Models/Aaa.php` and
* `vendor/pkg/app/Models/Zed.php` present, `use function App\Models\getUser`
* resolves to the former today and to the latter with the raw index.
* 3. `suffixResolve` with an index probes `index.get(S) || index.getInsensitive(S)`,
* which matches WHOLE paths too (`buildSuffixIndex` indexes the `j = 0`
* suffix); the scan compares `endsWith('/' + S)` and so can only match a
* PROPER suffix. Root-level `Foo.php` is unresolvable for `use Foo;` today
* and resolvable with the raw index; and where both match,
* `App/Models/User.php` (whole path, later in iteration order) would beat
* `vendor/x/Models/User.php` (proper suffix, earlier), which is the file the
* scan returns.
*
* So this builds a PARITY view instead: the same memoized arrays, and a
* `SuffixIndex` whose three methods reproduce the no-index answers exactly.
* - `getInsensitive` returns `undefined` unconditionally, which makes site 1 a
* no-op and falls through exactly as `index === undefined` did. It is safe to
* hollow out because `suffixResolve` reads it only as
* `get(S) || getInsensitive(S)`, so `get` can carry both halves — see below.
* - `getFilesInDir` answers from a root-anchored raw-path directory bucket, so
* site 2 returns what the scan returned, in the same order.
* - `get` answers site 3, defined as "first file in Set order whose normalized
* path has `S` as a proper segment suffix, compared case-insensitively".
* That single rule IS the scan: its predicate is
* `endsWith(p) || toLowerCase().endsWith(p.toLowerCase())`, whose first
* disjunct is subsumed by the second, so a case-sensitive hit never outranks
* an earlier case-insensitive one the way `get() || getInsensitive()` does.
*
* `get` is built on the shared `index.getInsensitive`, which is that same rule
* plus the whole-path (`j = 0`) entries. The correction needs one extra map, and
* only O(files) of it: the shared lookup can only over-match when `S` IS some
* file's whole normalized path, so `firstProperSuffixMatch` is keyed on exactly
* those strings. (Whole-string vs per-segment lowercasing agree here: no case
* mapping in Unicode produces or consumes `/`, so `lower(p).split('/')` and
* `p.split('/').map(lower)` are the same list.)
*
* `index.getInsensitive` is the ONLY shared-index method this file calls — it
* never asks the case-sensitive question — which is why `buildSuffixIndex`
* defers its two suffix maps rather than fusing them: PHP builds and retains
* one of the pair instead of both (34.49 MiB of 69.85 MiB at 32 000 paths).
*
* The two maps built HERE are deferred for the same reason and are each cheap
* only in ENTRIES, not in the walk that fills them — see the notes on
* `getFirstProperSuffixMatch` (O(paths × depth) to fill, typically zero entries)
* and `getFilesByRawDirectory` (unreachable without a `composer.json`).
*/
interface PhpWorkspaceIndex {
/** Every path, backslashes normalized to `/`. Parallel to `all`. */
readonly normalized: readonly string[];
/** Every path, exactly as it appears in the Set. Parallel to `normalized`. */
readonly all: readonly string[];
/** Scan-equivalent `SuffixIndex` for `resolvePhpImportInternal`. */
readonly suffixIndex: SuffixIndex;
}
/** Memoized on the `allFilePaths` Set identity, like `getWorkspaceFileIndex`. */
const getPhpWorkspaceIndex = perFileSet((allFilePaths: ReadonlySet<string>): PhpWorkspaceIndex => {
// The Set is passed THROUGH to the shared cache, never copied — a defensive
// `new Set(...)` here or in `scope-resolver.ts` would hand both WeakMaps a
// fresh key per import and silently restore O(imports × files) (#1918 P1).
const { normalized, all, index } = getWorkspaceFileIndex(allFilePaths);
/**
* Whole-path-lowercase → the first PROPER-suffix match, the correction `get`
* applies to a whole-path hit from the shared index.
*
* DEFERRED, and deferred all the way to the branch that reads it rather than
* to the first `get`. The builder walks every slash of every path and
* lowercases a slice at each, so it is O(paths × depth) in both time and
* allocation — measured 46.0 ms at 32 000 paths on the PHP arm of
* `bench/import-target/`, filling a map that held ZERO entries, because it
* can only hold one when some file's whole path is also a proper suffix of
* another's. Most repos never produce that, and the ones that do reach this
* branch only for the imports that actually hit a whole path. Pure function
* of `normalized`/`all`, both of which the returned object already retains,
* so building it late is behaviour-identical and retains nothing new.
*
* `wholePathLower` is a scratch set of the builder, not state: nothing reads
* it afterwards, so deferring the map defers it too.
*/
let firstProperSuffixMatch: Map<string, string> | null = null;
const getFirstProperSuffixMatch = (): Map<string, string> => {
if (firstProperSuffixMatch !== null) return firstProperSuffixMatch;
const wholePathLower = new Set<string>();
for (const path of normalized) wholePathLower.add(path.toLowerCase());
// Only the suffixes that a whole path can shadow are worth storing; see the
// header. Built from `normalized`, so it costs no traversal of the Set.
const built = new Map<string, string>();
for (let i = 0; i < normalized.length; i++) {
const lower = normalized[i].toLowerCase();
for (let slash = lower.indexOf('/'); slash >= 0; slash = lower.indexOf('/', slash + 1)) {
const suffix = lower.slice(slash + 1);
if (!wholePathLower.has(suffix)) continue;
if (!built.has(suffix)) built.set(suffix, all[i]);
}
}
firstProperSuffixMatch = built;
return built;
};
/**
* Raw directory → the files directly in it, for `getFilesInDir`.
*
* DEFERRED for the same reason as the shared `dirMap` (#2903), and here the
* case is stronger: `getFilesInDir` has exactly one caller,
* `import-resolvers/php.ts`'s PSR-4 function/constant fallback, and that
* caller sits inside `if (composerConfig) { … }`. `resolvePhpImportTarget`
* hard-codes `composerConfig: null`, so on the LanguageProvider path the map
* is statically unreachable; on the ScopeResolver path it is reachable only
* in a repo that has a parseable `composer.json` with `autoload.psr-4`.
* Measured 6.8 ms / 3.56 MiB at 32 000 paths, paid by every PHP repo without
* one. Pure function of `all`, which the returned object retains.
*/
let filesByRawDirectory: Map<string, string[]> | null = null;
const getFilesByRawDirectory = (): Map<string, string[]> => {
if (filesByRawDirectory !== null) return filesByRawDirectory;
// Raw paths, not normalized: the scan this replaces tests `f.startsWith(...)`
// against the Set's own strings, so a backslash path is a miss there and must
// stay a miss here. Insertion order is Set order, so `[0]` is the file the
// scan would have returned first.
const built = new Map<string, string[]>();
for (const raw of all) {
const separator = raw.lastIndexOf('/');
if (separator < 0) continue;
const directory = raw.slice(0, separator);
const bucket = built.get(directory);
if (bucket === undefined) built.set(directory, [raw]);
else bucket.push(raw);
}
filesByRawDirectory = built;
return built;
};
const suffixIndex: SuffixIndex = {
get: (suffix: string): string | undefined => {
const hit = index.getInsensitive(suffix);
if (hit === undefined) return undefined;
const lower = suffix.toLowerCase();
// A proper-suffix hit is already the scan's answer: the shared map holds
// the first file matching EITHER way, so nothing earlier matched at all.
if (hit.replace(/\\/g, '/').toLowerCase() !== lower) return hit;
// Whole-path hit — invisible to `endsWith('/' + S)`. The scan keeps going.
// The only branch that needs the correction map, hence the only one that
// builds it.
return getFirstProperSuffixMatch().get(lower);
},
// Site 1 must stay a no-op, and `suffixResolve` folds this into `get`.
getInsensitive: (): undefined => undefined,
getFilesInDir: (dirSuffix: string, extension: string): string[] => {
// `nsDirPrefix` is `nsDir` when it already ends in `/`, else `nsDir + '/'`
// — either way the directory is `nsDir` minus one trailing slash.
const directory = dirSuffix.endsWith('/') ? dirSuffix.slice(0, -1) : dirSuffix;
const bucket = getFilesByRawDirectory().get(directory);
if (bucket === undefined) return [];
return bucket.filter((file) => file.endsWith(extension));
},
};
return { normalized, all, suffixIndex };
});
// ─── loadResolutionConfig ──────────────────────────────────────────────────
/**
@ -181,17 +364,17 @@ export function resolvePhpImportTarget(
if (parsedImport.kind === 'dynamic-unresolved') return null;
if (parsedImport.targetRaw === null || parsedImport.targetRaw === '') return null;
// Cast, not copy: `getPhpWorkspaceIndex` memoizes on this exact Set object.
const allFiles = ctx.allFilePaths as Set<string>;
const normalizedFileList = [...allFiles].map((f) => f.replace(/\\/g, '/'));
const allFileList = [...allFiles];
const { normalized, all, suffixIndex } = getPhpWorkspaceIndex(allFiles);
return resolvePhpImportInternal(
parsedImport.targetRaw,
null, // composerConfig not available through LanguageProvider path
allFiles,
normalizedFileList,
allFileList,
undefined,
normalized,
all,
suffixIndex,
);
}
@ -216,17 +399,17 @@ export function resolvePhpImportTargetInternal(
? (resolutionConfig as ComposerConfig)
: null;
// Cast, not copy: `getPhpWorkspaceIndex` memoizes on this exact Set object.
const allFiles = allFilePaths as Set<string>;
const normalizedFileList = [...allFiles].map((f) => f.replace(/\\/g, '/'));
const allFileList = [...allFiles];
const { normalized, all, suffixIndex } = getPhpWorkspaceIndex(allFiles);
const resolved = resolvePhpImportInternal(
targetRaw,
composerConfig,
allFiles,
normalizedFileList,
allFileList,
undefined,
normalized,
all,
suffixIndex,
);
const parsedImport = context?.parsedImport;

248
gitnexus/src/core/ingestion/languages/python/import-target.ts
View file

@ -11,8 +11,13 @@
*/
import type { ParsedFile, ParsedImport, WorkspaceIndex } from 'gitnexus-shared';
import { perFileSet } from '../../import-resolvers/per-file-set.js';
import {
getPythonFileIndex,
importerAncestors,
importerDirOf,
} from '../../import-resolvers/python-file-index.js';
import { resolvePythonImportInternal } from '../../import-resolvers/python.js';
import { recordPythonFileIndexBuild } from './index-stats.js';
export interface PythonResolveContext {
readonly fromFile: string;
@ -82,7 +87,35 @@ export function resolvePythonImportTarget(
workspaceIndex,
);
if (submodule !== null) return submodule;
if (packageTarget !== null) return packageTarget;
// `return packageTarget`, not `if (packageTarget !== null) return …` —
// falling through when it is null RE-RAN THE ENTIRE TAIL BELOW, a second
// time, with byte-identical arguments.
//
// `packageTarget` IS this function's tail for this import. The recursion
// above differs from the outer frame in exactly one field,
// `targetIncludesImportedName`, whose only effect is to make
// `pythonImportedSubmoduleTarget` return null and so skip this branch: the
// spread preserves `kind` (still `named`/`alias`, so the
// `dynamic-unresolved` guard cannot fire) and `targetRaw` (which already
// passed the null/empty guard), and `workspaceIndex` is the same object, so
// `ctx.fromFile`, `ctx.allFilePaths` and `ctx.parsedFiles` are the same
// references. The recursion therefore ran `resolvePythonImportInternal` →
// relative gate → `hasRepoCandidate` → `resolveAbsoluteFromFiles` on
// exactly the inputs the fallthrough would use.
//
// That tail is a pure function of (`fromFile`, `targetRaw`,
// `allFilePaths`): it only reads the Set and indexes memoized on the Set,
// and the `submodule` probe in between is equally read-only, so nothing can
// have changed the answer. Reaching this line means the tail already
// returned null; running it again returns null again, after another
// proximity probe and another full ancestor walk to the workspace root.
//
// Measured before this change, `from x import y` at four directory
// components: 24 `allFilePaths.has` probes per import, of which probes
// 12-23 were byte-identical repeats of 0-11. `python-import-probe-count
// .test.ts` is the gate.
return packageTarget;
}
// PEP-328 relative + single-segment proximity bare imports.
@ -144,6 +177,13 @@ export function resolvePythonImportTarget(
* that classification is what open issue #2882 is about, so it belongs with
* that fix rather than bolted on here. Not a regression: both halves behave
* exactly as they did before #2864.
*
* The `parsedFiles.find` this used to open with was the same O(imports x files)
* shape #2913 removes on the path Set, keyed on the other collection the
* orchestrator threads: every import whose package probe resolves scanned the
* whole parsed workspace, and on a repo where `from pkg import X` usually
* resolves that is most imports. `parsedFileByPath` replaces it with one pass
* per pass.
*/
function pythonFileExportsName(
targetFile: string,
@ -151,7 +191,7 @@ function pythonFileExportsName(
parsedFiles: readonly ParsedFile[] | undefined,
): boolean {
if (parsedFiles === undefined) return false;
const parsed = parsedFiles.find((file) => file.filePath === targetFile);
const parsed = parsedFileByPath(parsedFiles).get(targetFile);
if (parsed === undefined) return false;
return parsed.localDefs.some((def) => {
const qualifiedName = def.qualifiedName;
@ -161,6 +201,27 @@ function pythonFileExportsName(
});
}
/**
* `filePath -> ParsedFile`, memoized on the identity of the pass's
* `parsedFiles` array — the second stable object the orchestrator threads
* through `resolveImportTarget`, beside the path Set.
*
* FIRST WINS on a duplicated path, which is what `Array.prototype.find`
* returned, so the answer is unchanged for a workspace that somehow parsed one
* path twice. Values are references to the array's own elements: the Map costs
* one pointer per parsed file and, living in a `WeakMap` keyed on the array,
* is reclaimed with the pass rather than accumulating across runs (#2649).
*/
const parsedFileByPath = perFileSet(
(parsedFiles: readonly ParsedFile[]): Map<string, ParsedFile> => {
const byPath = new Map<string, ParsedFile>();
for (const file of parsedFiles) {
if (!byPath.has(file.filePath)) byPath.set(file.filePath, file);
}
return byPath;
},
);
/**
* Resolve `package/sub/module` style paths (already dot-flattened) to a
* concrete file in `allFilePaths`. Tries the exact path first, then walks
@ -196,19 +257,44 @@ function resolveAbsoluteFromFiles(
if (allFilePaths.has(directFile)) return directFile;
if (allFilePaths.has(directPkg)) return directPkg;
// Both remaining tiers — the ancestor walk and the suffix fallback — can only
// ever land on a file whose basename is `<lastSeg>.py`, or on an `__init__.py`
// whose parent directory is named `<lastSeg>`. The two buckets the suffix
// fallback already needs therefore also decide, in O(1) and before the walk,
// whether the walk can hit at all: neither bucket present means no tier below
// can match, and one bucket absent removes that tier's probe from EVERY step
// of the walk. On the deep corpus that is half the walk's probes (#2913).
//
// `pythonSegmentAbsent` states this same rule for the single-segment bare
// tier. It is deliberately not called here: that tier needs only the answer,
// this one needs the candidate ARRAYS for the suffix fallback below, so
// sharing would mean two extra `has` lookups per import to save four lines.
const index = getPythonFileIndex(allFilePaths);
const lastSeg = pathLike.slice(pathLike.lastIndexOf('/') + 1);
const moduleCandidates = index.byBasename.get(`${lastSeg}.py`);
const packageCandidates = index.byInitParent.get(`${lastSeg}/__init__.py`);
const mayBeModule = moduleCandidates !== undefined;
// `byInitParent` skips `__init__.py` files whose parent directory name is
// empty (a doubled separator), so an empty `<lastSeg>` — a target spelled
// with a trailing dot — cannot use the bucket as proof of absence and keeps
// probing exactly as before.
const mayBePackage = packageCandidates !== undefined || lastSeg === '';
if (!mayBeModule && !mayBePackage) return null;
// Ancestor walk — match the single-segment resolver's behavior at
// multi-segment granularity. Closest match wins. Stop at `i > 0` because
// `i === 0` would re-check the workspace-root candidates already covered
// by the direct check above.
const importerDir = fromFile.replace(/\\/g, '/').split('/').slice(0, -1).join('/');
if (importerDir) {
const dirParts = importerDir.split('/').filter(Boolean);
for (let i = dirParts.length; i > 0; i--) {
const ancestor = dirParts.slice(0, i).join('/');
const prefix = `${ancestor}/`;
const candidateFile = `${prefix}${directFile}`;
const candidatePkg = `${prefix}${directPkg}`;
// multi-segment granularity. Closest match wins. The chain stops short of the
// workspace root because the root candidates are the direct check above.
//
// The chain comes from `importerAncestors`, which builds it ONCE per importer
// directory per pass. Rebuilding it here — one `slice(0, i).join('/')` per
// path component, on every import — was half of the depth quadratic in #2913.
for (const ancestor of importerAncestors(index, importerDirOf(fromFile))) {
if (mayBeModule) {
const candidateFile = `${ancestor}/${directFile}`;
if (allFilePaths.has(candidateFile)) return candidateFile;
}
if (mayBePackage) {
const candidatePkg = `${ancestor}/${directPkg}`;
if (allFilePaths.has(candidatePkg)) return candidatePkg;
}
}
@ -237,17 +323,15 @@ function resolveAbsoluteFromFiles(
// shared buildSuffixIndex is deliberately NOT used: it keeps only one
// path per suffix (longest wins) and so cannot reproduce this exact
// fewest-segments-then-lexicographic tie-break across all candidates.
const index = getPythonFileIndex(allFilePaths);
const lastSeg = pathLike.slice(pathLike.lastIndexOf('/') + 1);
const matches: { raw: string; norm: string }[] = [];
for (const cand of index.byBasename.get(`${lastSeg}.py`) ?? []) {
for (const cand of moduleCandidates ?? []) {
if (cand.norm.endsWith(suffixFile)) matches.push(cand);
}
// Package form: only `__init__.py` files whose parent dir is named `<lastSeg>`
// can match `…/<lastSeg>/__init__.py` — look them up by parent key (P2b) and
// confirm the full suffix. Same final candidate set as the old `__init__.py`
// scan, just without iterating unrelated packages.
for (const cand of index.byInitParent.get(`${lastSeg}/__init__.py`) ?? []) {
for (const cand of packageCandidates ?? []) {
if (cand.norm.endsWith(suffixPkg)) matches.push(cand);
}
if (matches.length === 0) return null;
@ -293,131 +377,33 @@ function hasRepoCandidate(
const rootFile = `${leadingSegment}.py`;
const initFile = `${leadingSegment}/__init__.py`;
// Build importer-ancestor prefixes: for `backend/routers/cron.py`,
// produces `["backend/routers/services/", "backend/services/"]` for
// segment `services` (closest first, root excluded — covered above).
const importerDir = fromFile.replace(/\\/g, '/').split('/').slice(0, -1).join('/');
const dirParts = importerDir ? importerDir.split('/').filter(Boolean) : [];
const ancestorPrefixes: string[] = [];
for (let i = dirParts.length; i > 0; i--) {
ancestorPrefixes.push(`${dirParts.slice(0, i).join('/')}/${leadingSegment}/`);
}
// Indexed equivalents of the old O(files) scan:
// (1) `f === rootFile || f === initFile` -> normalized-path membership.
// (2) `f.startsWith(`${seg}/`) && f.endsWith('.py')` -> some .py file lives
// under directory `${seg}/`, i.e. `${seg}/` is a known .py dir prefix.
// (3) ancestor namespace case -> `${ancestor}/${seg}/` is a known .py dir
// prefix.
// prefix, for some ancestor of the importer's directory.
const index = getPythonFileIndex(allFilePaths);
if (index.normSet.has(rootFile) || index.normSet.has(initFile)) return true;
if (index.dirPrefixes.has(prefix)) return true;
for (const ap of ancestorPrefixes) {
if (index.dirPrefixes.has(ap)) return true;
// (3) used to MATERIALIZE one `${ancestor}/${seg}/` string per component of
// the importer's directory, eagerly, before checks (1) and (2) had even run —
// O(depth^2) characters on every import, and the other half of #2913. Two
// things replace that: `nestedDirNames` answers "is `seg` the name of any
// directory sitting under a non-empty parent?" in O(1), which is `false` for
// every external import (`os`, `django`, an unknown distribution) and skips
// the walk outright; and what remains walks the per-directory ancestor chain,
// built once per pass, closest first, so the common in-repo hit exits after a
// step or two. `nestedDirNames` is exact, not a filter: `${A}/${seg}/` can
// only be a directory prefix if `seg` names a directory under the non-empty
// parent `A`, so a miss here means the old loop would have missed too.
if (!index.nestedDirNames.has(leadingSegment)) return false;
for (const ancestor of importerAncestors(index, importerDirOf(fromFile))) {
if (index.dirPrefixes.has(`${ancestor}/${prefix}`)) return true;
}
return false;
}
/**
* Per-file-set index for Python import resolution, memoized on the
* `allFilePaths` Set object (the same Set is passed for every import in a run,
* so the index is built once and reused). Replaces the per-import O(files)
* scans in `resolveAbsoluteFromFiles` (suffix match) and `hasRepoCandidate`
* (package-existence gate) with O(1)/O(bucket) lookups.
*
* - `normSet`: every file path, normalized to forward slashes (for the exact
* `f === rootFile|initFile` membership checks).
* - `byBasename`: last path component (e.g. `models.py`, `__init__.py`) ->
* all `{ raw, norm }` candidates, so suffix matches can be gathered from the
* relevant bucket and the exact tie-break applied across ALL of them.
* - `byInitParent`: `__init__.py` files keyed by their last TWO components
* (`<parentDir>/__init__.py`). The package suffix lookup (`pkg.sub` ->
* `…/sub/__init__.py`) targets only same-named package dirs via this map
* instead of scanning every `__init__.py` in the repo — the common
* multi-segment import path no longer scales with package count
* (PR #1918 review P2b). `__init__.py` files stay in `byBasename` too, for
* the rarer explicit `pkg.__init__` import that resolves via the module
* (`…<lastSeg>.py`) lookup.
* - `dirPrefixes`: every directory prefix of a `.py` file, trailing-slashed
* (`a/b/c.py` -> `a/`, `a/b/`), for "is there a .py file under `<dir>/`".
*/
interface PythonFileIndex {
readonly normSet: Set<string>;
readonly byBasename: Map<string, { raw: string; norm: string }[]>;
readonly byInitParent: Map<string, { raw: string; norm: string }[]>;
readonly dirPrefixes: Set<string>;
}
const PYTHON_FILE_INDEX_CACHE = new WeakMap<ReadonlySet<string>, PythonFileIndex>();
function getPythonFileIndex(allFilePaths: ReadonlySet<string>): PythonFileIndex {
const cached = PYTHON_FILE_INDEX_CACHE.get(allFilePaths);
if (cached !== undefined) return cached;
// Cache miss: materialize a fresh index. Counted so a test can assert this
// happens once per run, not once per import (PR #1918 review P1 guard).
recordPythonFileIndexBuild();
const normSet = new Set<string>();
const byBasename = new Map<string, { raw: string; norm: string }[]>();
const byInitParent = new Map<string, { raw: string; norm: string }[]>();
const dirPrefixes = new Set<string>();
for (const raw of allFilePaths) {
const norm = raw.replace(/\\/g, '/');
// Python import resolution only ever queries `.py` paths: module `<seg>.py`
// and package `<seg>/__init__.py` membership (normSet), `<lastSeg>.py` /
// `__init__.py` basename buckets (byBasename), and `.py` directory prefixes
// (dirPrefixes). Non-`.py` files can never match any of those, so skip them
// — they were dead weight in every structure on polyglot monorepos
// (PR #1918 review P3b; dirPrefixes was already `.py`-gated).
if (!norm.endsWith('.py')) continue;
normSet.add(norm);
const lastSlash = norm.lastIndexOf('/');
const base = lastSlash >= 0 ? norm.slice(lastSlash + 1) : norm;
let bucket = byBasename.get(base);
if (bucket === undefined) {
bucket = [];
byBasename.set(base, bucket);
}
bucket.push({ raw, norm });
// Package files also get a parent-keyed bucket so a `pkg.sub` lookup hits
// only `…/sub/__init__.py` candidates, not every `__init__.py` (P2b).
if (base === '__init__.py' && lastSlash >= 0) {
const dir = norm.slice(0, lastSlash);
const parentSlash = dir.lastIndexOf('/');
const parentName = parentSlash >= 0 ? dir.slice(parentSlash + 1) : dir;
if (parentName) {
const initKey = `${parentName}/__init__.py`;
let ib = byInitParent.get(initKey);
if (ib === undefined) {
ib = [];
byInitParent.set(initKey, ib);
}
ib.push({ raw, norm });
}
}
// Directory prefixes: every slash-terminated prefix of the path (every
// index just past a '/', up to and including the file's own directory).
// Scanning the FULL normalized path — including any leading '/' for
// absolute paths — makes `dirPrefixes.has(X)` match exactly when the old
// gate's `f.startsWith(X)` (X always ends in '/') matched. The previous
// split+`filter(Boolean)` dropped the leading empty component, so an
// absolute file `/repo/svc/x.py` yielded `repo/svc/` (no leading slash) and
// gate-passed where `"/repo/svc/x.py".startsWith("repo/svc/")` is false
// (PR #1918 review P3a). For relative paths the set is identical.
for (let i = 0; i <= lastSlash; i++) {
if (norm[i] === '/') dirPrefixes.add(norm.slice(0, i + 1));
}
}
const index: PythonFileIndex = { normSet, byBasename, byInitParent, dirPrefixes };
PYTHON_FILE_INDEX_CACHE.set(allFilePaths, index);
return index;
}
function pythonImportedSubmoduleTarget(parsedImport: ParsedImport): string | null {
if (parsedImport.kind !== 'named' && parsedImport.kind !== 'alias') return null;
if (parsedImport.targetIncludesImportedName === true) return null;

29
gitnexus/src/core/ingestion/languages/python/index-stats.ts
View file

@ -1,29 +0,0 @@
/**
* Build counter for the per-file-set Python import-resolution index
* (`getPythonFileIndex` in `import-target.ts`).
*
* A "build" is a `WeakMap` cache MISS that materializes a fresh
* `PythonFileIndex` (O(files)). Unlike `cache-stats.ts` (which gates its
* counters behind `PROF_SCOPE_RESOLUTION` because they sit on the per-capture
* hot path), this counter is always live: an index build happens at most once
* per resolution run, so the single increment is negligible and an unconditional
* counter avoids env-var load-order fragility in tests.
*
* Used by `test/integration/python-import-index-reuse.test.ts` to assert the
* index is reused across imports (built once per run) rather than rebuilt per
* import — the regression guard for PR #1918 review finding P1.
*/
let INDEX_BUILDS = 0;
export function recordPythonFileIndexBuild(): void {
INDEX_BUILDS++;
}
export function getPythonFileIndexBuildCount(): number {
return INDEX_BUILDS;
}
export function resetPythonFileIndexBuildCount(): void {
INDEX_BUILDS = 0;
}

23
gitnexus/src/core/ingestion/languages/rust/qualified-call.ts
View file

@ -33,6 +33,7 @@
*/
import type { ParsedFile, Scope, ScopeId, SymbolDefinition } from 'gitnexus-shared';
import { perFileSet } from '../../import-resolvers/per-file-set.js';
import { isOverloadableCallable } from '../../utils/callable-labels.js';
import { lookupBindingsAt } from '../../scope-resolution/scope/walkers.js';
import type { ScopeResolutionIndexes } from '../../model/scope-resolution-indexes.js';
@ -53,16 +54,9 @@ import {
* The hook is invoked per call site; rebuilding the index each time would make
* qualified-call resolution O(sites x files).
*/
const MODULE_INDEX_CACHE = new WeakMap<ReadonlySet<string>, RustModuleIndex>();
function moduleIndexFor(allFilePaths: ReadonlySet<string>): RustModuleIndex {
let index = MODULE_INDEX_CACHE.get(allFilePaths);
if (index === undefined) {
index = buildRustModuleIndex(allFilePaths);
MODULE_INDEX_CACHE.set(allFilePaths, index);
}
return index;
}
const moduleIndexFor = perFileSet(
(allFilePaths: ReadonlySet<string>): RustModuleIndex => buildRustModuleIndex(allFilePaths),
);
export function resolveRustQualifiedFreeCall(
site: { readonly name: string; readonly rawQualifiedName?: string; readonly inScope: ScopeId },
@ -488,6 +482,15 @@ interface PassModuleIndex {
readonly inlineModuleKeys: ReadonlySet<string>;
}
/**
* DELIBERATELY NOT ON `import-resolvers/per-file-set.ts` (#2909 sweep), unlike
* {@link moduleIndexFor} above. {@link passIndexFor} takes THREE inputs —
* `workspaceIndex`, `index` and `scopes` — and keys on the first alone; the
* builder reads `scopes.defs.byId` and `index`, neither of which is derivable
* from the key, and `perFileSet`'s `build: (key) => T` hands the builder
* nothing but the key. Sound here only because all three share the resolution
* pass's lifetime, which is an invariant the primitive cannot express.
*/
const MODULE_SCOPE_CACHE = new WeakMap<WorkspaceResolutionIndex, PassModuleIndex>();
function moduleKey(module: RustModule): string {

14
gitnexus/src/core/ingestion/languages/swift/import-target.ts
View file

@ -25,6 +25,7 @@
*/
import type { ParsedImport, WorkspaceIndex } from 'gitnexus-shared';
import { perFileSet } from '../../import-resolvers/per-file-set.js';
export interface SwiftResolveContext {
readonly fromFile: string;
@ -39,12 +40,7 @@ interface SwiftModuleIndex {
readonly byModule: Map<string, string[]>;
}
const SWIFT_MODULE_INDEX_CACHE = new WeakMap<ReadonlySet<string>, SwiftModuleIndex>();
function getSwiftModuleIndex(allFilePaths: ReadonlySet<string>): SwiftModuleIndex {
const cached = SWIFT_MODULE_INDEX_CACHE.get(allFilePaths);
if (cached !== undefined) return cached;
const getSwiftModuleIndex = perFileSet((allFilePaths: ReadonlySet<string>): SwiftModuleIndex => {
const byModule = new Map<string, string[]>();
for (const raw of allFilePaths) {
const norm = raw.replace(/\\/g, '/');
@ -66,10 +62,8 @@ function getSwiftModuleIndex(allFilePaths: ReadonlySet<string>): SwiftModuleInde
}
}
const index: SwiftModuleIndex = { byModule };
SWIFT_MODULE_INDEX_CACHE.set(allFilePaths, index);
return index;
}
return { byModule };
});
export function resolveSwiftImportTarget(
parsedImport: ParsedImport,

4
gitnexus/src/core/ingestion/languages/typescript/import-target.ts
View file

@ -82,8 +82,8 @@ export function resolveTsTarget(targetRaw: string, ctx: TsResolveContext): strin
ctx.fromFile,
targetRaw,
ctx.allFilePaths,
allFileList as string[],
normalizedFileList as string[],
allFileList,
normalizedFileList,
resolveCache,
language,
ctx.tsconfigPaths ?? null,

56
gitnexus/src/core/ingestion/languages/typescript/scope-resolver.ts
View file

@ -22,7 +22,8 @@ import { simpleKey } from '../../scope-resolution/graph-bridge/node-lookup.js';
import type { ScopeResolutionIndexes } from '../../model/scope-resolution-indexes.js';
import { typescriptProvider } from '../typescript.js';
import { loadTsconfigPaths, type TsconfigPaths } from '../../language-config.js';
import { buildSuffixIndex, type SuffixIndex } from '../../import-resolvers/utils.js';
import { buildImportPassCache } from '../../import-resolvers/pass-cache.js';
import { perFileSet } from '../../import-resolvers/per-file-set.js';
import { indexOnlyElementType } from '../../type-extractors/shared.js';
import {
typescriptArityCompatibility,
@ -55,42 +56,31 @@ const TYPESCRIPT_TYPE_ONLY_BINDING_TYPES = new Set<NodeLabel>([
]);
/**
* Build a `resolveImportTarget` adapter that memoizes the workspace
* file list, the lower-cased file list, and the per-pass `resolveCache`
* across every import lookup in a single workspace pass. The
* orchestrator passes the same `ReadonlySet` reference for every call
* within a pass — we use that identity to detect when the workspace
* changes and recompute the derived state lazily.
* Memoized on the `allFilePaths` Set identity, like every other language's
* import index (`import-resolvers/workspace-file-index.ts` and friends).
*
* Without this memoization, `resolveTsTarget` re-derived
* `allFileList` and `normalizedFileList` (both O(N_files)) and threw
* away the `resolveCache` on every import — O(N_files × N_imports)
* total work for what should be O(N_files + N_imports).
* This used to be a single-slot `let cached` invalidated by
* `cached.key !== allFilePaths` — correct for one file set and degenerate for
* two: alternating calls across two sets rebuilt everything every time.
* Measured here at 4000 files × 400 imports: 12.0 ms for one set, 1438.2 ms
* alternating between two (120x). A `WeakMap` has no such state to thrash, and
* it is what lets this adapter carry the standard
* `expectDistinctFileSetsGetOwnIndex` guard the other languages carry
* (`test/integration/typescript-import-index-reuse.test.ts`).
*
* The Set must be passed THROUGH by the caller, never copied: a defensive
* `new Set(allFilePaths)` at the adapter boundary hands a fresh key per import
* and restores the per-import rebuild (PR #1918 review P1).
*/
const tsPassCacheFor = perFileSet(buildImportPassCache);
/**
* Build a `resolveImportTarget` adapter that reads the memoized per-file-set
* state above rather than re-deriving it on every import lookup.
*/
function makeTsResolveImportTarget(): ScopeResolver['resolveImportTarget'] {
interface PassCache {
readonly key: ReadonlySet<string>;
readonly allFilePaths: Set<string>;
readonly allFileList: readonly string[];
readonly normalizedFileList: readonly string[];
readonly index: SuffixIndex;
readonly resolveCache: Map<string, string | null>;
}
let cached: PassCache | null = null;
return (targetRaw, fromFile, allFilePaths, resolutionConfig) => {
if (cached === null || cached.key !== allFilePaths) {
const allFileList = Array.from(allFilePaths);
const normalizedFileList = allFileList.map((f) => f.toLowerCase());
cached = {
key: allFilePaths,
allFilePaths: new Set(allFilePaths),
allFileList,
normalizedFileList,
index: buildSuffixIndex(normalizedFileList, allFileList),
resolveCache: new Map(),
};
}
const cached = tsPassCacheFor(allFilePaths);
const cfg = resolutionConfig as TypescriptResolutionConfig | undefined;
const ws: TsResolveContext = {

49
gitnexus/src/core/ingestion/languages/vue/import-target.ts
View file

@ -14,27 +14,39 @@
* logic fires.
*
* Memoization mirrors the TypeScript adapter: workspace file-list
* arrays, the suffix index, and the per-pass resolve cache are rebuilt
* lazily when `allFilePaths` reference changes (once per workspace pass).
* arrays, the suffix index and the per-pass resolve cache are built
* once per `allFilePaths` Set and memoized on that Set's identity.
*/
import { SupportedLanguages } from 'gitnexus-shared';
import { resolveTsTarget, type TsResolveContext } from '../typescript/import-target.js';
import { buildSuffixIndex, type SuffixIndex } from '../../import-resolvers/utils.js';
import { buildImportPassCache } from '../../import-resolvers/pass-cache.js';
import { perFileSet } from '../../import-resolvers/per-file-set.js';
import type { TsconfigPaths } from '../../language-config.js';
interface VueResolutionConfig {
readonly tsconfigPaths: TsconfigPaths | null;
}
interface PassCache {
readonly key: ReadonlySet<string>;
readonly allFilePaths: Set<string>;
readonly allFileList: readonly string[];
readonly normalizedFileList: readonly string[];
readonly index: SuffixIndex;
readonly resolveCache: Map<string, string | null>;
}
/**
* Memoized on the `allFilePaths` Set identity, like every other language's
* import index (`import-resolvers/workspace-file-index.ts` and friends).
*
* This used to be a single-slot `let cached` invalidated by
* `cached.key !== allFilePaths` — correct for one file set and degenerate for
* two: alternating calls across two sets rebuilt everything every time.
* Measured on the identical TypeScript adapter at 4000 files × 400 imports:
* 12.0 ms for one set, 1438.2 ms alternating between two (120x). A `WeakMap`
* has no such state to thrash, and it is what lets this adapter carry the
* standard
* `expectDistinctFileSetsGetOwnIndex` guard the other languages carry
* (`test/integration/vue-import-index-reuse.test.ts`).
*
* The Set must be passed THROUGH by the caller, never copied: a defensive
* `new Set(allFilePaths)` at the adapter boundary hands a fresh key per import
* and restores the per-import rebuild (PR #1918 review P1).
*/
const passCacheFor = perFileSet(buildImportPassCache);
/**
* Build a memoized `resolveImportTarget` adapter for Vue SFCs.
@ -49,21 +61,8 @@ export function makeVueResolveImportTarget(): (
allFilePaths: ReadonlySet<string>,
resolutionConfig?: unknown,
) => string | readonly string[] | null {
let cached: PassCache | null = null;
return (targetRaw, fromFile, allFilePaths, resolutionConfig) => {
if (cached === null || cached.key !== allFilePaths) {
const allFileList = Array.from(allFilePaths);
const normalizedFileList = allFileList.map((f) => f.toLowerCase());
cached = {
key: allFilePaths,
allFilePaths: new Set(allFilePaths),
allFileList,
normalizedFileList,
index: buildSuffixIndex(normalizedFileList, allFileList),
resolveCache: new Map(),
};
}
const cached = passCacheFor(allFilePaths);
const cfg = resolutionConfig as VueResolutionConfig | undefined;
const ws: TsResolveContext = {

347
gitnexus/test/helpers/counting-file-set.ts
View file

@ -1,16 +1,18 @@
/**
* A `Set<string>` that counts how many times it is TRAVERSED in full — the
* measuring instrument behind the import-target index-reuse guards
* (`test/unit/scope-resolution/import-target-index-parity.test.ts` and the
* per-language `test/integration/<lang>-import-index-reuse.test.ts` files).
* A `Set<string>` that counts how many times it is TRAVERSED in full — the one
* measuring instrument behind every import-target index-reuse guard
* (`test/unit/scope-resolution/import-target-index-parity.test.ts`,
* `test/unit/scope-resolution/import-target-index-reuse.contract.test.ts`, and
* the per-language `test/integration/<lang>-import-index-reuse.test.ts` files).
*
* ## Why a counting Set rather than a production build counter
*
* Kotlin and Python count index BUILDS from production (`languages/<lang>/
* index-stats.ts`). That catches the per-import rebuild, but it is blind to a
* scan added BESIDE a reused index: the cache still hits, the build count still
* reads 1. Counting traversals of the file set instead needs no production
* surface at all and catches both failures with one number:
* Kotlin and Python used to count index BUILDS, through a counter module that
* shipped in production for no reason but this observation (deleted in #2909).
* A build count catches the per-import rebuild, but it is blind to a scan added
* BESIDE a reused index: the cache still hits, the count still reads 1. Counting
* traversals of the file set instead needs no production surface at all and
* catches both failures with one number:
*
* - an adapter that copies the set (`new Set(allFilePaths)`) hands a fresh
* `WeakMap` key per import, so the count rises to the import count;
@ -40,6 +42,14 @@
* Guarding that would mean either instrumenting production or proxying an index
* internal; see the header of the parity test for why neither is in place.
*
* It is equally blind to the OTHER per-file-set key the orchestrator threads —
* `ImportResolutionContext.parsedFiles`, the fifth argument of
* `resolveImportTarget`. PHP's `filesByDirectory` memo (`languages/php/
* import-target.ts`) is keyed on that array, not on this Set, so defeating it
* rebuilds a `Map<dirAlias, ParsedFile[]>` per import at O(files × depth)
* without moving this counter by one. `countedParsedFiles` below is the
* instrument for that channel.
*
* `instanceof Set` still holds, which matters: C#'s `narrowContext` rejects a
* workspace context whose `allFilePaths` is not a `Set`, so a plain object with
* a counter would silently resolve nothing and every assertion would pass on
@ -47,9 +57,14 @@
*
* `expectDistinctFileSetsGetOwnIndex` below is the one arm of those guards that
* is identical in every language once the four values that differ are named, so
* it lives here beside the instrument it reads rather than in each guard.
* it lives here beside the instrument it reads rather than in each guard. The
* `ChainMemoArm` section at the bottom applies the same rule to the guards that
* watch a MEMO instead of a scan count — the two Python importer-chain guards,
* which this instrument provably cannot see (their headers say why) and which
* were arm-for-arm the same suite written twice.
*/
import { expect } from 'vitest';
import type { ParsedFile, ParsedImport } from 'gitnexus-shared';
import type { ScopeResolver } from '../../src/core/ingestion/scope-resolution/contract/scope-resolver.js';
export class CountingSet extends Set<string> {
@ -85,6 +100,81 @@ export class CountingSet extends Set<string> {
}
}
/**
* The `CountingSet` of the OTHER per-file-set key: the `parsedFiles` array the
* orchestrator passes as `resolveImportTarget`'s fifth argument
* (`scope-resolution/pipeline/run.ts`). PHP memoizes `filesByDirectory` on that
* array's identity and Python reads it in `pythonFileExportsName`, and neither
* touches the path Set while doing so — so without this the whole `context`
* channel is unmeasured.
*
* ## Element reads, not traversal entry points
*
* `CountingSet` can override the five ways a `Set` is walked and be done. An
* array has no such closed list: `for…of`, `forEach`, `map`, `filter`,
* `flatMap`, `reduce`, `find`, `some`, `every`, `indexOf` and a bare
* `for (let i = 0; i < a.length; i++)` all walk the same elements, and the last
* one goes through no method at all. Overriding a chosen subset would build in
* exactly the blind spot this instrument exists to remove — PHP's builder is a
* `for…of` today and one refactor away from an index loop.
*
* So the trap is on the read of an own indexed element. Every route above goes
* through it, including the index loop, and nothing else does: `length`,
* method lookups and `Symbol.iterator` are not counted. A full pass over N
* files therefore reads exactly N, and the number is a function of the file
* count and the number of passes — never of wall time.
*
* ## It counts THIS array only
*
* Reads of arrays DERIVED from it — the `ParsedFile[]` buckets inside PHP's
* directory index, the `candidateFiles` list filtered per import — are
* invisible, and deliberately so. That per-import work is bounded by the
* candidate set rather than by the workspace, so counting it would make the
* count grow with the import count for correct code and there would be no
* property left to assert.
*
* The `ParsedFile`s are minimal on purpose: `filePath` is the only field either
* consumer reads to build its index, and empty `localDefs` keeps both languages
* on their fallback answer, so the fixture measures the index and changes no
* resolution result. A test that needs the declaration legs to FIRE wants
* `php-import-target-parity.test.ts`, which carries defs.
*/
export interface CountedFileList {
/** Pass as `ImportResolutionContext.parsedFiles`. Stable identity, so it is
* a usable `perFileSet` key for the whole run. */
readonly parsedFiles: readonly ParsedFile[];
/** Reads of an own indexed element of `parsedFiles`, by any route. */
readonly reads: () => number;
}
/** Own array indices — `'0'`, `'12'`; not `'length'`, `'-1'` or `'01'`. */
const ARRAY_INDEX = /^(?:0|[1-9][0-9]*)$/;
/**
* A counted `parsedFiles` workspace for `filePaths`, one minimal `ParsedFile`
* each, in order. Build a FRESH one per run: the indexes are memoized on the
* array's identity, so two runs sharing one would have the second read the
* first's index and report zero.
*/
export function countedParsedFiles(filePaths: readonly string[]): CountedFileList {
const backing: ParsedFile[] = filePaths.map((filePath) => ({
filePath,
moduleScope: `module:${filePath}`,
scopes: [],
parsedImports: [],
localDefs: [],
referenceSites: [],
}));
let reads = 0;
const counting = new Proxy(backing, {
get(target, key, receiver): unknown {
reads += typeof key === 'string' && ARRAY_INDEX.test(key) ? 1 : 0;
return Reflect.get(target, key, receiver);
},
});
return { parsedFiles: counting, reads: () => reads };
}
/**
* Everything that differs between the per-language spellings of the
* distinct-file-set arm. Nothing else about that arm varies, which is why it is
@ -159,3 +249,240 @@ export function expectDistinctFileSetsGetOwnIndex(arm: DistinctFileSetArm): void
expect(a.scans).toBe(arm.expectedScans);
expect(b.scans).toBe(arm.expectedScans);
}
// ─── Python import shapes ────────────────────────────────────────────────────
/**
* `from <targetRaw> import Widget`. The shape that makes
* `resolvePythonImportTarget` run the package-attribute probe
* (`pythonFileExportsName`, the `context.parsedFiles` reader) ahead of the
* submodule fallback — so it is the shape that re-enters the resolver and pays
* the importer's chain TWICE, and the shape `pythonImportedSubmoduleTarget`
* fires for.
*
* The default the adapter synthesizes when `context` is absent is a `namespace`
* import, and that shape never reaches the probe. A guard that means to measure
* either leg therefore has to pass this one explicitly.
*/
export const pythonNamedImport = (targetRaw: string): ParsedImport => ({
kind: 'named',
localName: 'Widget',
importedName: 'Widget',
targetRaw,
});
/** `import <targetRaw>` — the single-walk shape, and the adapter's default. */
export const pythonNamespaceImport = (targetRaw: string): ParsedImport => ({
kind: 'namespace',
localName: '_',
importedName: '_',
targetRaw,
});
/**
* ONE array per file that uses it, never a fresh `[]` per call:
* `parsedFileByPath` memoizes on its identity, and a new array per import would
* mint a `WeakMap` key per import for a channel these guards are not measuring
* (`countedParsedFiles` above is the instrument for that one). Empty, so
* `pythonFileExportsName` answers false and the package-vs-submodule precedence
* never fires — the walk, not the precedence, is what the numbers measure.
*/
export const NO_PARSED_FILES: readonly ParsedFile[] = [];
// ─── the Python importer-chain memo guards ───────────────────────────────────
/**
* What one resolution answered, as the chain-memo arms read it: a path, a path
* list (the `ScopeResolver` signature allows one), or `null`. The two values
* the non-vacuity pairing rule counts are `arm.hitResult` and `null`.
*/
export type ChainMemoResult = string | readonly string[] | null;
/**
* Everything that differs between the two Python importer-chain memo guards:
* `test/unit/import-resolvers/python-importer-prefixes.test.ts`
* (`bareImportPrefixesByDir`) and
* `test/unit/scope-resolution/python/python-importer-ancestors.test.ts`
* (`ancestorsByDir`).
*
* The two memos hold DIFFERENT SEQUENCES under the same key — self included or
* not, workspace root included or not, empty components kept or dropped; see
* `importerBarePrefixes`'s header for why neither guard can be deleted in
* favour of the other. But each guard is the same four arms over the same
* importer corpus once these four values are named, so the arms live here and
* each guard supplies its own four.
*/
export interface ChainMemoArm {
/** The memo under test, read off the pass's per-file-set index. */
readonly memoOf: (files: ReadonlySet<string>) => ReadonlyMap<string, readonly string[]>;
/**
* Drives a production surface `perImporter` times from `fromFile`, with
* spellings that reach the memo, and answers what each call resolved to.
* Exactly one call per invocation must answer `arm.hitResult`, and at least
* one must answer `null`. Must pass `files` THROUGH: both memos are keyed on
* its identity, so a copy here would measure nothing.
*/
readonly drive: (
files: Set<string>,
fromFile: string,
perImporter: number,
) => readonly ChainMemoResult[];
/**
* The verbatim pre-change chain builder, which is the specification: the memo
* agreeing with it is what makes the change a hoist rather than a behaviour
* change.
*/
readonly legacyChain: (fromFile: string) => readonly string[];
/** What the one must-resolve spelling in `drive` answers, once per importer. */
readonly hitResult: string;
}
/** Every file that issues an import in the chain-memo arms. */
export const CHAIN_MEMO_IMPORTERS: readonly string[] = [
'svc/a/one.py',
'svc/a/two.py',
'svc/b/one.py',
'deep/x/y/z/one.py',
'root.py',
];
/** Four directories for those five importers — `svc/a` holds two of them. */
export const CHAIN_MEMO_IMPORTER_DIRS: readonly string[] = ['svc/a', 'svc/b', 'deep/x/y/z', ''];
/** The directory two importers share, which is where identity is measured. */
const SHARED_DIR = 'svc/a';
const SHARED_DIR_IMPORTERS: readonly string[] = ['svc/a/one.py', 'svc/a/two.py'];
/** Imports one directory issues before its chain's identity is re-read. */
const CHAIN_IDENTITY_REPEATS = 40;
/** A sorted copy, so a key set is compared without depending on fill order. */
export const sortedStrings = (values: Iterable<string>): string[] => [...values].sort();
/**
* The importer directory both memos are keyed on, derived exactly as the
* pre-change inline code derived it — `''` for a path with no separator.
*/
const importerDirOf = (fromFile: string): string =>
fromFile.replace(/\\/g, '/').split('/').slice(0, -1).join('/');
/**
* The path-shape space an importer chain has to be correct over, as ONE table
* both guards run: they enumerate the same space and nothing kept the two
* copies in lockstep.
*
* Each `why` names the SHAPE, not what either chain does with it, because the
* two chains do different things with several of these rows — the bare-prefix
* chain KEEPS the empty component an absolute path or a doubled separator
* produces and the ancestor chain drops it, and that difference decides real
* resolutions. Which is why every arm below compares against the guard's own
* `legacyChain` rather than against a shared expectation.
*/
export const IMPORTER_PATH_SHAPES: readonly { readonly fromFile: string; readonly why: string }[] =
[
{ fromFile: 'svc/a/one.py', why: 'a two-component directory' },
{ fromFile: 'deep/x/y/z/one.py', why: 'a four-component directory' },
{ fromFile: 'root.py', why: 'a workspace-root importer' },
{ fromFile: '/abs/svc/a/one.py', why: 'an absolute path (leading empty component)' },
{ fromFile: 'svc//a/one.py', why: 'a doubled separator (empty component)' },
{ fromFile: 'svc\\a\\one.py', why: 'Windows separators' },
{ fromFile: 'trailing/', why: 'a path ending in a separator' },
];
/**
* The gate: N imports from five importers over four directories leave FOUR
* entries, for every N. That is "the chain work is O(1) amortized after the
* first import from a given directory", stated as a number. A chain rebuilt per
* import cannot be memoized at all (size 0); a chain keyed on the importing
* FILE reads five.
*
* Paired with the non-vacuity assertions every guard in this family states: a
* perfect memo count is equally true of an adapter that resolves nothing.
*/
export function expectOneChainPerImporterDir(
arm: ChainMemoArm,
files: Set<string>,
perImporter: number,
): void {
const resolved: ChainMemoResult[] = [];
for (const fromFile of CHAIN_MEMO_IMPORTERS) {
resolved.push(...arm.drive(files, fromFile, perImporter));
}
expect(arm.memoOf(files).size).toBe(CHAIN_MEMO_IMPORTER_DIRS.length);
expect(sortedStrings(arm.memoOf(files).keys())).toEqual(sortedStrings(CHAIN_MEMO_IMPORTER_DIRS));
expect(resolved.filter((value) => value === arm.hitResult)).toHaveLength(
CHAIN_MEMO_IMPORTERS.length,
);
expect(resolved.filter((value) => value === null).length).toBeGreaterThan(0);
}
/**
* The stored-object arm: a memo that stores a FRESH chain on every import posts
* a perfect size while doing all of the work again, so the size gate above is
* paired with reference identity across many later imports from the same
* directory — issued from BOTH files in it, so a chain keyed on the importing
* file would be replaced rather than reused.
*
* Contents are asserted FIRST: `toBe` against an absent entry would pass on
* `undefined === undefined` if the memo were deleted outright.
*/
export function expectSameChainObjectReused(arm: ChainMemoArm, files: Set<string>): void {
const [firstImporter] = SHARED_DIR_IMPORTERS;
arm.drive(files, firstImporter, 1);
const first = arm.memoOf(files).get(SHARED_DIR);
expect(first).toEqual(arm.legacyChain(firstImporter));
for (const fromFile of SHARED_DIR_IMPORTERS) {
arm.drive(files, fromFile, CHAIN_IDENTITY_REPEATS);
}
expect(arm.memoOf(files).get(SHARED_DIR)).toBe(first);
}
/**
* The legacy-equality arm for one path shape: what the memo stored under
* `fromFile`'s directory is what the pre-change inline code built for it.
*
* Returns the memoized chain, so a guard whose memo feeds a SECOND consumer can
* go on to assert that consumer's derived form of it.
*/
export function expectMemoizedChainMatchesLegacy(
arm: ChainMemoArm,
files: Set<string>,
fromFile: string,
): readonly string[] {
arm.drive(files, fromFile, 1);
const chain = arm.memoOf(files).get(importerDirOf(fromFile));
expect(chain).toEqual(arm.legacyChain(fromFile));
return chain ?? [];
}
/**
* The distinct-file-set arm: two independently built file sets each get their
* own memo — equal in content, never the same object, neither leaking into the
* other. The two are driven interleaved, so a memo keyed on anything but the
* Set's identity shows up here as a SHARED entry rather than as a stale one.
*/
export function expectDistinctFileSetsGetOwnChainMemo(
arm: ChainMemoArm,
a: Set<string>,
b: Set<string>,
perImporter: number,
): void {
for (const fromFile of CHAIN_MEMO_IMPORTERS) {
arm.drive(a, fromFile, perImporter);
arm.drive(b, fromFile, perImporter);
}
const memoA = arm.memoOf(a);
const memoB = arm.memoOf(b);
expect(memoA).not.toBe(memoB);
expect(memoA.get(SHARED_DIR)).not.toBe(memoB.get(SHARED_DIR));
expect(memoA.get(SHARED_DIR)).toEqual(memoB.get(SHARED_DIR));
expect(memoA.size).toBe(CHAIN_MEMO_IMPORTER_DIRS.length);
expect(memoB.size).toBe(CHAIN_MEMO_IMPORTER_DIRS.length);
}

133
gitnexus/test/integration/cobol-import-index-reuse.test.ts Normal file
View file

@ -0,0 +1,133 @@
/**
* Production-path regression guard for the COBOL `COPY`-target index (#2908).
*
* The two-tier basename index (`getCobolCopyIndex` in
* `languages/cobol/scope-resolver.ts`) is memoized on the `allFilePaths` Set
* identity via a WeakMap, so the file set must be passed THROUGH from the
* orchestrator, never copied. A defensive `new Set(allFilePaths)` in the
* adapter hands a fresh WeakMap key per call and rebuilds the index on every
* `COPY`, restoring the O(copies × files) scans this replaced — the exact bug
* PR #1918 shipped for Python and had to fix in review (P1).
*
* COBOL is the language where that copy costs the most: every `COPY` used to
* run TWO full scans, and mainframe repos are copybook-dense — one program can
* carry dozens of `COPY` statements.
*
* Unlike the other languages in this family, COBOL has no separate
* `resolve<Lang>ImportTarget` function; the adapter IS the resolver. The unit
* parity test (`test/unit/scope-resolution/cobol-import-target-parity.test.ts`)
* therefore reaches the same entry point — but it says nothing about Set
* identity, so a copy inserted there leaves every one of its arms green. This
* file is what notices, by counting traversals of the set.
*
* The traversal-count assertions are the perf guard. They are paired with
* result assertions on purpose: a count of 1 is equally true of an adapter that
* has stopped resolving anything at all, so counting alone would stay green
* while every COBOL COPY edge disappeared.
*
* Expected count is 1: both tiers are filled in a single pass over the set.
*/
import { describe, it, expect } from 'vitest';
import { cobolScopeResolver } from '../../src/core/ingestion/languages/cobol/scope-resolver.js';
import { CountingSet, expectDistinctFileSetsGetOwnIndex } from '../helpers/counting-file-set.js';
const { resolveImportTarget } = cobolScopeResolver;
const FROM_FILE = 'src/PROG.cbl';
/**
* A synthetic mainframe checkout: many copybooks under `copybooks/`, plus the
* three files the arms below address — a copybook, a program reachable only
* through the SOURCE tier, and a program that a copybook of the same name must
* beat despite coming first in Set-iteration order.
*/
function buildWorkspace(fileCount: number): CountingSet {
const files: string[] = [];
// Inserted before the copybook below so the tier-order arm is a real
// tie-break rather than an artefact of ordering.
files.push('src/CUSTREC.cbl');
for (let i = 0; i < fileCount; i++) {
files.push(`copybooks/BOOK${String(i).padStart(5, '0')}.cpy`);
}
files.push('copybooks/CUSTREC.cpy');
files.push('src/PAYROLL.cbl');
files.push('src/PROG.cbl');
return new CountingSet(files);
}
describe('COBOL COPY resolution — index reuse across imports (#2908)', () => {
it('builds the file index once for many COPY statements over a stable file set', () => {
const files = buildWorkspace(300);
const resolved: (string | readonly string[] | null)[] = [];
for (let i = 0; i < 200; i++) {
// Three shapes: a copybook hit, a hit that only the SOURCE tier answers,
// and a member that is not in the repo at all — the last is the common
// case in real COBOL (vendor and system copybooks) and the one that used
// to cost TWO full workspace scans per statement.
resolved.push(resolveImportTarget('CUSTREC', FROM_FILE, files, undefined));
resolved.push(resolveImportTarget('PAYROLL', FROM_FILE, files, undefined));
resolved.push(resolveImportTarget(`VENDOR${i}`, FROM_FILE, files, undefined));
}
expect(files.scans).toBe(1);
// Paired result assertions — a count of 1 must not be the count of an
// adapter that resolves nothing. The first also pins the tier order: the
// `.cbl` twin was inserted FIRST.
expect(resolved[0]).toBe('copybooks/CUSTREC.cpy');
expect(resolved[1]).toBe('src/PAYROLL.cbl');
expect(resolved[2]).toBeNull();
});
it('a distinct file set gets its own index (no stale cross-run reuse)', () => {
expectDistinctFileSetsGetOwnIndex({
resolveImportTarget,
buildWorkspace: () => buildWorkspace(20),
targetRaw: 'CUSTREC',
fromFile: FROM_FILE,
resolutionConfig: undefined,
expected: 'copybooks/CUSTREC.cpy',
expectedScans: 1,
});
});
it('still resolves real COPY statements correctly (the perf test is not vacuous)', () => {
const files = new CountingSet([
'src/CUSTREC.cbl',
'copybooks/CUSTREC.cpy',
'copybooks/custrec-lower.copybook',
'copybooks/Mixed.CPY',
'src/PAYROLL.cob',
'src/TAXCALC.cobol',
'docs/CUSTREC.txt',
'copybooks/NOEXT',
]);
// Tier order: the copybook wins over the `.cbl` inserted before it.
expect(resolveImportTarget('CUSTREC', FROM_FILE, files, undefined)).toBe(
'copybooks/CUSTREC.cpy',
);
// Case: the COPY operand and the file's stem are both upper-cased.
expect(resolveImportTarget('custrec', FROM_FILE, files, undefined)).toBe(
'copybooks/CUSTREC.cpy',
);
expect(resolveImportTarget('CUSTREC-LOWER', FROM_FILE, files, undefined)).toBe(
'copybooks/custrec-lower.copybook',
);
// Source tier, reached only after every copybook missed.
expect(resolveImportTarget('PAYROLL', FROM_FILE, files, undefined)).toBe('src/PAYROLL.cob');
expect(resolveImportTarget('TAXCALC', FROM_FILE, files, undefined)).toBe('src/TAXCALC.cobol');
// `path.basename(fp, '.cpy')` will not strip `.CPY`, so the stem keeps it.
expect(resolveImportTarget('MIXED.CPY', FROM_FILE, files, undefined)).toBe(
'copybooks/Mixed.CPY',
);
expect(resolveImportTarget('MIXED', FROM_FILE, files, undefined)).toBeNull();
// Neither tier: a `.txt`, a file with no extension, and an absent member.
expect(resolveImportTarget('NOEXT', FROM_FILE, files, undefined)).toBeNull();
expect(resolveImportTarget('ABSENT', FROM_FILE, files, undefined)).toBeNull();
// One traversal covered all of it.
expect(files.scans).toBe(1);
});
});

126
gitnexus/test/integration/csharp-import-index-reuse.test.ts
View file

@ -27,35 +27,95 @@
* result assertions on purpose: a count of 2 is equally true of an adapter that
* has stopped resolving anything at all, so counting alone would stay green
* while every C# IMPORTS edge disappeared.
*
* ## The csproj leg is guarded by the SAME instrument (#2911 review)
*
* With `.csproj` configs present the adapter takes a different branch entirely
* — `resolveCSharpImportInternal` — and that branch used to be unguarded here:
* no arm supplied `csharpConfigs`, so no counting Set ever entered it. Worse,
* its namespace-directory index was keyed on the `normalizedFileList` ARRAY, a
* shape no scan count can instrument — a `[...normalized]` copy at the adapter
* boundary rebuilt the index once per `using` while traversing the Set exactly
* zero extra times. Reproduced against this PR's tree: the copy left all 67
* tests of the four import-index guards green and only the timing bench
* noticed (`csharp_csproj scaling 3.556 > 1.8`).
*
* #2911 rekeyed that index onto the Set, so the array shape is gone and the
* only remaining way to defeat the memo — copying the Set — is what
* `CountingSet` already counts. The csproj arms below therefore read the same
* one number as the arms above, with no second instrument.
*/
import { describe, it, expect } from 'vitest';
import { csharpScopeResolver } from '../../src/core/ingestion/languages/csharp/scope-resolver.js';
import type { CsharpResolutionConfig } from '../../src/core/ingestion/languages/csharp/resolution-config.js';
import { CountingSet, expectDistinctFileSetsGetOwnIndex } from '../helpers/counting-file-set.js';
const { resolveImportTarget } = csharpScopeResolver;
const FROM_FILE = 'App/Program.cs';
/** Where a workspace's padded filler files go, and what is appended after them. */
interface WorkspaceLayout {
/** Directory the filler files live in. */
readonly dir: string;
/** Basename stem the filler files are numbered from. */
readonly stem: string;
/** The files the resolutions actually target, appended in order. */
readonly extras: readonly string[];
}
/**
* `fileCount` filler files under `layout.dir`, then `layout.extras`. The filler
* is what makes a traversal expensive enough for a per-`using` rebuild to be a
* different number rather than a different constant; the counting instrument
* reads the traversals either way.
*/
function buildWorkspace(fileCount: number, layout: WorkspaceLayout): CountingSet {
const files: string[] = [];
for (let i = 0; i < fileCount; i++) {
files.push(`${layout.dir}/${layout.stem}${String(i).padStart(5, '0')}.cs`);
}
return new CountingSet([...files, ...layout.extras]);
}
/**
* A synthetic C# solution with no `.csproj` discovered, which is the leg #2878
* moved onto the indexes. `App/Models/User.cs` answers the whole-path lookup,
* `App/Services/` answers the namespace-directory lookup, and `Domain/Order.cs`
* is reachable only after progressive prefix stripping.
*/
function buildWorkspace(fileCount: number): CountingSet {
const files: string[] = [];
for (let i = 0; i < fileCount; i++) {
files.push(`App/Services/Service${String(i).padStart(5, '0')}.cs`);
}
files.push('App/Models/User.cs');
files.push('Domain/Order.cs');
files.push('App/Program.cs');
return new CountingSet(files);
}
const NO_CSPROJ_LAYOUT: WorkspaceLayout = {
dir: 'App/Services',
stem: 'Service',
extras: ['App/Models/User.cs', 'Domain/Order.cs', 'App/Program.cs'],
};
/** The one `.csproj` config that puts the adapter on the csproj leg. */
const CSPROJ_CONFIG: CsharpResolutionConfig = {
csharpConfigs: [{ rootNamespace: 'App', projectDir: 'App' }],
};
/**
* A workspace whose `App.Models` resolution reaches the namespace-DIRECTORY
* index, which is the only thing on the csproj leg keyed on the array.
*
* That takes a layout the first two legs both miss. `src/MyApp/Models/` answers
* `dirPrefix = 'App/Models'` under the unanchored substring rule ('MyApp/'
* supplies the 'App/'), and under nothing weaker: no file is named
* `App/Models.cs` or `Models.cs`, so the single-file leg misses, and no
* directory has the SEGMENT suffix `App/Models`, so `getFilesInDir` misses too.
* A layout where the first two legs answer would leave the index unbuilt and
* the build count blind to the very copy it is here to catch.
*/
const CSPROJ_LAYOUT: WorkspaceLayout = {
dir: 'src/MyApp/Models',
stem: 'Entity',
extras: ['App/Program.cs'],
};
describe('C# import resolution — index reuse across usings (#2878)', () => {
it('builds each index once for many usings over a stable file set', () => {
const files = buildWorkspace(300);
const files = buildWorkspace(300, NO_CSPROJ_LAYOUT);
const resolved: (string | readonly string[] | null)[] = [];
for (let i = 0; i < 200; i++) {
@ -82,7 +142,7 @@ describe('C# import resolution — index reuse across usings (#2878)', () => {
it('a distinct file set gets its own indexes (no stale cross-run reuse)', () => {
expectDistinctFileSetsGetOwnIndex({
resolveImportTarget,
buildWorkspace: () => buildWorkspace(20),
buildWorkspace: () => buildWorkspace(20, NO_CSPROJ_LAYOUT),
targetRaw: 'App.Models.User',
fromFile: FROM_FILE,
resolutionConfig: undefined,
@ -94,7 +154,7 @@ describe('C# import resolution — index reuse across usings (#2878)', () => {
});
it('still resolves real usings correctly (the perf test is not vacuous)', () => {
const files = buildWorkspace(5);
const files = buildWorkspace(5, NO_CSPROJ_LAYOUT);
// Whole-path match on the namespace path.
expect(resolveImportTarget('App.Models.User', FROM_FILE, files, undefined)).toBe(
@ -114,3 +174,43 @@ describe('C# import resolution — index reuse across usings (#2878)', () => {
expect(resolveImportTarget('Vendor.Ghost.Missing', FROM_FILE, files, undefined)).toBeNull();
});
});
describe('C# import resolution — index reuse on the csproj leg (#2911)', () => {
it('builds each index once for many usings over a stable file set', () => {
const files = buildWorkspace(300, CSPROJ_LAYOUT);
const resolved: (string | readonly string[] | null)[] = [];
for (let i = 0; i < 200; i++) {
// A namespace-directory hit and a miss, both reaching the array-keyed
// index — the miss under a fresh namespace each time so no upstream
// string-level memo can stand in for the index being reused.
resolved.push(resolveImportTarget('App.Models', FROM_FILE, files, CSPROJ_CONFIG));
resolved.push(resolveImportTarget(`App.Ghost${i}`, FROM_FILE, files, CSPROJ_CONFIG));
}
// One traversal for 400 usings. One, not two: `getCsharpDirIndex` belongs to
// the no-csproj leg, and the namespace-directory index this branch DOES
// build reads its file list from the same `getWorkspaceFileIndex` memo
// rather than re-walking the Set. A defensive copy of the Set at the
// adapter boundary reads 400 here.
expect(files.scans).toBe(1);
// Paired result assertions — the count must not be the count of an adapter
// that resolves nothing.
expect(resolved[0]).toBe('src/MyApp/Models/Entity00000.cs');
expect(resolved[1]).toBeNull();
});
it('a distinct file set gets its own indexes (no stale cross-run reuse)', () => {
expectDistinctFileSetsGetOwnIndex({
resolveImportTarget,
buildWorkspace: () => buildWorkspace(20, CSPROJ_LAYOUT),
targetRaw: 'App.Models',
fromFile: FROM_FILE,
resolutionConfig: CSPROJ_CONFIG,
expected: 'src/MyApp/Models/Entity00000.cs',
// One, not two: see the scan-count comment above.
expectedScans: 1,
});
});
});

6
gitnexus/test/integration/go-import-index-reuse.test.ts
View file

@ -12,11 +12,11 @@
* replaced. Python hit exactly that (PR #1918 review P1), and the parity test
* cannot see it: it never crosses the adapter.
*
* Kotlin and Python count index BUILDS from production (`index-stats.ts`).
* These four use `CountingSet` (`test/helpers/counting-file-set.ts`) instead,
* Every one of these guards uses `CountingSet` (`test/helpers/counting-file-set.ts`),
* which counts full traversals of the file set and so catches BOTH the
* per-import rebuild and a scan reintroduced beside a reused index — with no
* production surface added for a test-only observation.
* production surface added for a test-only observation. Kotlin and Python
* counted index BUILDS from production until #2909 moved them onto this one.
*
* The traversal-count assertions are the perf guard. They are paired with
* result assertions on purpose: a count of 1 is equally true of an adapter that

130
gitnexus/test/integration/java-import-index-reuse.test.ts Normal file
View file

@ -0,0 +1,130 @@
/**
* Production-path regression guard for the Java import-resolution indexes
* (#2908).
*
* `resolveJavaImportTarget` reads TWO per-file-set indexes, each memoized on
* the `allFilePaths` Set identity via its own WeakMap: the shared
* `getWorkspaceFileIndex` (`import-resolvers/workspace-file-index.ts`, which
* answers the whole-path and segment-suffix legs) and `getJavaDirIndex`
* (`languages/java/import-target.ts`, the package-directory index behind
* `firstFileDirectlyInPkgDir`). Before the hoist every leg was a full
* `allFilePaths` scan, and the progressive-stripping loop re-ran that scan once
* per stripped segment — so a four-segment `import` that resolves to nothing,
* which is what every JDK and third-party import does, cost four full passes.
*
* Resolution reaches both indexes through `javaScopeResolver.resolveImportTarget`
* — the orchestrator adapter — not by calling `resolveJavaImportTarget` directly
* the way the unit parity test does. The adapter must therefore pass the Set
* THROUGH; a defensive copy (`new Set(allFilePaths)`) would hand a fresh WeakMap
* key per call and rebuild BOTH indexes on every import, restoring the
* O(imports × files) behaviour this replaced. Python hit exactly that (PR #1918
* review P1), and `test/unit/scope-resolution/java-import-target-parity.test.ts`
* cannot see it: it never crosses the adapter.
*
* The counting instrument has to be a real `Set` subclass: `narrowContext`
* rejects a workspace context whose `allFilePaths` fails `instanceof Set`, and a
* rejected context resolves nothing — every assertion would then pass on
* `null === null`.
*
* The traversal-count assertions are the perf guard. They are paired with result
* assertions on purpose: a count of 2 is equally true of an adapter that has
* stopped resolving anything at all, so counting alone would stay green while
* every Java IMPORTS edge disappeared.
*/
import { describe, it, expect } from 'vitest';
import { javaScopeResolver } from '../../src/core/ingestion/languages/java/scope-resolver.js';
import { CountingSet, expectDistinctFileSetsGetOwnIndex } from '../helpers/counting-file-set.js';
const { resolveImportTarget } = javaScopeResolver;
const FROM_FILE = 'src/main/java/com/example/App.java';
/**
* A synthetic Java source tree covering all four legs of the cascade:
* `com/example/model/User.java` answers the whole-path lookup,
* `src/main/java/com/example/service/` answers the package-directory lookup a
* wildcard import lands on, `src/main/java/com/example/util/Strings.java`
* answers the nested-suffix lookup, and `domain/Order.java` is reachable only
* after progressive prefix stripping.
*/
function buildWorkspace(fileCount: number): CountingSet {
const files: string[] = [];
for (let i = 0; i < fileCount; i++) {
files.push(`src/main/java/com/example/service/Service${String(i).padStart(5, '0')}.java`);
}
files.push('com/example/model/User.java');
files.push('src/main/java/com/example/util/Strings.java');
files.push('domain/Order.java');
files.push(FROM_FILE);
return new CountingSet(files);
}
describe('Java import resolution — index reuse across imports (#2908)', () => {
it('builds each index once for many imports over a stable file set', () => {
const files = buildWorkspace(300);
const resolved: (string | readonly string[] | null)[] = [];
for (let i = 0; i < 200; i++) {
// A whole-path hit, a nested-suffix hit, a package-directory hit via a
// wildcard, and a miss that runs the full progressive-stripping cascade —
// the case that used to re-scan the workspace once per stripped prefix.
resolved.push(resolveImportTarget('com.example.model.User', FROM_FILE, files, undefined));
resolved.push(resolveImportTarget('com.example.util.Strings', FROM_FILE, files, undefined));
resolved.push(resolveImportTarget('com.example.service.*', FROM_FILE, files, undefined));
resolved.push(
resolveImportTarget(`vendor${i}.ghost.deep.Missing`, FROM_FILE, files, undefined),
);
}
// Two passes: the shared workspace/suffix index and the package-dir index.
// Not one: they are separate WeakMaps and `buildPackageDirIndex` takes the
// Set, so each iterates it once — the same accounting as C# (#2878).
expect(files.scans).toBe(2);
// Paired result assertions — a count of 2 must not be the count of an
// adapter that resolves nothing.
expect(resolved[0]).toBe('com/example/model/User.java');
expect(resolved[1]).toBe('src/main/java/com/example/util/Strings.java');
expect(resolved[2]).toBe('src/main/java/com/example/service/Service00000.java');
expect(resolved[3]).toBeNull();
});
it('a distinct file set gets its own indexes (no stale cross-run reuse)', () => {
expectDistinctFileSetsGetOwnIndex({
resolveImportTarget,
buildWorkspace: () => buildWorkspace(20),
targetRaw: 'com.example.model.User',
fromFile: FROM_FILE,
resolutionConfig: undefined,
expected: 'com/example/model/User.java',
// Two, not one: the shared workspace/suffix index and the package-dir
// index are separate WeakMaps over the same Set.
expectedScans: 2,
});
});
it('still resolves real imports correctly (the perf test is not vacuous)', () => {
const files = buildWorkspace(5);
// Whole-path match on the package path.
expect(resolveImportTarget('com.example.model.User', FROM_FILE, files, undefined)).toBe(
'com/example/model/User.java',
);
// Nested suffix match under the source root.
expect(resolveImportTarget('com.example.util.Strings', FROM_FILE, files, undefined)).toBe(
'src/main/java/com/example/util/Strings.java',
);
// Wildcard: `.*` is stripped and the package directory answers with its
// first `.java` child in file-set order.
expect(resolveImportTarget('com.example.service.*', FROM_FILE, files, undefined)).toBe(
'src/main/java/com/example/service/Service00000.java',
);
// Progressive prefix stripping: the repo has no `com/shop/` prefix.
expect(resolveImportTarget('com.shop.domain.Order', FROM_FILE, files, undefined)).toBe(
'domain/Order.java',
);
// Unknown packages resolve to nothing.
expect(resolveImportTarget('vendor.ghost.Missing', FROM_FILE, files, undefined)).toBeNull();
});
});

175
gitnexus/test/integration/javascript-import-index-reuse.test.ts Normal file
View file

@ -0,0 +1,175 @@
/**
* Production-path regression guard for the JavaScript import-resolution index
* (#2910).
*
* `makeJsResolveImportTarget`'s `PassCache` was the TypeScript one minus its
* `index` field, so every JavaScript import reached `suffixResolve` with
* `index === undefined` and took the linear-`findIndex` fallback: one pass over
* `normalizedFileList` per path part per extension, ~39 extensions. 6448.9 µs
* per import at 2000 files and 25972.6 µs at 8000 — 4.12x the per-import cost
* for 4x the files, which is O(imports × files) — against 25.0 / 27.0 µs for
* TypeScript over the identical corpus. With the index it is 28.5 / 27.4 µs and
* the scaling factor is 1.09x.
*
* ## Why the existing guards were blind to it
*
* `CountingSet` counts traversals of the SET, and this scan walked the array
* the adapter had already materialized from it (`test/helpers/counting-file-set.ts`
* says so under "What it does NOT see"). The pass cache was reused correctly,
* so the traversal count read 2 with the defect and reads 2 without it — the
* sixteen-language contract test scored `javascript` a clean pass throughout.
*
* So the arm that would have caught this is not a count of Set traversals but
* `resolves a repo-root module by bare specifier` below: without an index a
* repo-root file is unreachable through this leg, because the scan tests
* `endsWith('/' + suffix)` and a root-level path has no `/`. It is a behaviour
* assertion, it is deterministic, and it fails the moment `index` leaves the
* cache. The direct instrument — counting entries into `suffixResolve`'s linear
* branch, with the pre-index adapter as its control — lives beside the
* differential in `test/unit/scope-resolution/javascript-import-target-parity.test.ts`.
*
* ## What the traversal counts here do guard
*
* Resolution reaches the cache through `javascriptScopeResolver.resolveImportTarget`
* — the orchestrator adapter — which must pass the Set THROUGH: a defensive
* `new Set(allFilePaths)` hands a fresh `WeakMap` key per import and restores
* the per-import rebuild (PR #1918 review P1). Two traversals per file set, not
* one: the adapter materializes `allFileList` and then keeps one mutable copy
* of the Set, because `TsResolveContext.allFilePaths` is a `Set`, not a
* `ReadonlySet`.
*
* The counts are paired with result assertions on purpose: a count of 2 is
* equally true of an adapter that has stopped resolving anything at all.
*/
import { describe, it, expect } from 'vitest';
import { javascriptScopeResolver } from '../../src/core/ingestion/languages/javascript/scope-resolver.js';
import { CountingSet, expectDistinctFileSetsGetOwnIndex } from '../helpers/counting-file-set.js';
const { resolveImportTarget } = javascriptScopeResolver;
const FROM_FILE = 'src/main.js';
/**
* A synthetic CommonJS/ESM app covering the legs the resolver takes: a relative
* import answered by exact `Set.has`, a bare specifier answered by path suffix,
* a `node_modules` package, a directory `index.js`, and `config.js` at the repo
* root — the one shape only the index can reach.
*/
function buildWorkspace(fileCount: number): CountingSet {
const files: string[] = [];
for (let i = 0; i < fileCount; i++) {
files.push(`src/components/Widget${String(i).padStart(5, '0')}.js`);
}
files.push('src/util.js');
files.push('src/models/index.js');
files.push('lib/esm.mjs');
files.push('node_modules/dep/index.js');
files.push('config.js');
files.push('bootstrap.cjs');
files.push(FROM_FILE);
return new CountingSet(files);
}
describe('JavaScript import resolution — index reuse across imports (#2910)', () => {
it('builds the pass cache once for many imports over a stable file set', () => {
const files = buildWorkspace(300);
const resolved: (string | readonly string[] | null)[] = [];
for (let i = 0; i < 200; i++) {
// A relative hit, a bare-specifier suffix hit, a repo-root hit, and a
// bare specifier that misses. The miss is the expensive case: it runs
// every path part × every extension before returning null, which is the
// loop that used to scan the whole file list each time round.
resolved.push(resolveImportTarget('./util', FROM_FILE, files, undefined));
resolved.push(resolveImportTarget('src/models', FROM_FILE, files, undefined));
resolved.push(resolveImportTarget('config', FROM_FILE, files, undefined));
resolved.push(resolveImportTarget(`@vendor/ghost${i}/deep`, FROM_FILE, files, undefined));
}
// Two: `Array.from(allFilePaths)` and the one mutable `Set` copy the
// resolver context requires. Both happen once per file set.
expect(files.scans).toBe(2);
// Paired result assertions — a count of 2 must not be the count of an
// adapter that resolves nothing.
expect(resolved[0]).toBe('src/util.js');
expect(resolved[1]).toBe('src/models/index.js');
expect(resolved[2]).toBe('config.js');
expect(resolved[3]).toBeNull();
});
it('a distinct file set gets its own index (no stale cross-run reuse)', () => {
expectDistinctFileSetsGetOwnIndex({
resolveImportTarget,
buildWorkspace: () => buildWorkspace(20),
targetRaw: 'src/models',
fromFile: FROM_FILE,
resolutionConfig: undefined,
expected: 'src/models/index.js',
expectedScans: 2,
});
});
/**
* The per-file-set index and `resolveCache` must not become global. The arm
* above cannot see that: `expectDistinctFileSetsGetOwnIndex` builds two
* IDENTICAL corpora, so a stale answer carried across them is also the right
* answer, and only its traversal counts would notice. These two workspaces
* answer the same specifier differently, and they are resolved alternately.
*/
it('two different workspaces answer the same specifier differently', () => {
const a = new Set(['src/util.js', FROM_FILE]);
const b = new Set(['vendor/util.js', FROM_FILE]);
expect(resolveImportTarget('util', FROM_FILE, a, undefined)).toBe('src/util.js');
expect(resolveImportTarget('util', FROM_FILE, b, undefined)).toBe('vendor/util.js');
expect(resolveImportTarget('util', FROM_FILE, a, undefined)).toBe('src/util.js');
expect(resolveImportTarget('util', FROM_FILE, b, undefined)).toBe('vendor/util.js');
});
/**
* The arm that fails without the suffix index, and the reason it is here
* rather than in the counting arms above: a repo-root file has no `/`, so
* `suffixResolve`'s scan — which tests `endsWith('/' + suffix)` — can never
* match it, while `buildSuffixIndex` indexes the whole path and can.
* Dropping `index` from the pass cache turns every one of these back to null
* while leaving `files.scans` at 2.
*/
it('resolves a repo-root module by bare specifier — impossible without the index', () => {
const files = buildWorkspace(5);
expect(resolveImportTarget('config', FROM_FILE, files, undefined)).toBe('config.js');
expect(resolveImportTarget('bootstrap', FROM_FILE, files, undefined)).toBe('bootstrap.cjs');
// Not `'config.js'`, and that is unchanged by the index: a specifier with
// no `/` has its dots turned into slashes before the suffix cascade
// (`resolveImportPath`), so `config.js` is looked up as `config/js`.
// TypeScript answers null here too — it is the same code path.
expect(resolveImportTarget('config.js', FROM_FILE, files, undefined)).toBeNull();
});
it('still resolves real imports correctly (the perf test is not vacuous)', () => {
const files = buildWorkspace(5);
// Relative, with and without an extension.
expect(resolveImportTarget('./util', FROM_FILE, files, undefined)).toBe('src/util.js');
expect(resolveImportTarget('./util.js', FROM_FILE, files, undefined)).toBe('src/util.js');
// Directory index.
expect(resolveImportTarget('./models', FROM_FILE, files, undefined)).toBe(
'src/models/index.js',
);
// Bare specifier resolved by path suffix, and an ESM extension.
expect(resolveImportTarget('components/Widget00000', FROM_FILE, files, undefined)).toBe(
'src/components/Widget00000.js',
);
expect(resolveImportTarget('lib/esm', FROM_FILE, files, undefined)).toBe('lib/esm.mjs');
// A package in node_modules.
expect(resolveImportTarget('dep', FROM_FILE, files, undefined)).toBe(
'node_modules/dep/index.js',
);
// Nothing in the repo answers these.
expect(resolveImportTarget('./nowhere', FROM_FILE, files, undefined)).toBeNull();
expect(resolveImportTarget('@vendor/ghost/deep', FROM_FILE, files, undefined)).toBeNull();
});
});

103
gitnexus/test/integration/kotlin-import-index-reuse.test.ts
View file

@ -10,17 +10,31 @@
* call and rebuild the index on every import, restoring the O(imports × files)
* behaviour this replaced. Python hit exactly that (PR #1918 review P1).
*
* The build-count assertions are the perf guard. They are paired with result
* assertions on purpose: a build count of 1 is equally true of an adapter that
* has stopped resolving anything at all, so counting alone would stay green
* while every Kotlin IMPORTS edge disappeared.
* ## Why this counts TRAVERSALS and not index builds (#2909)
*
* This guard used to read a build counter that shipped in production purely so
* a test could read it (now deleted). `CountingSet`
* (`test/helpers/counting-file-set.ts`) replaces it, and the swap is not a
* wash:
*
* - STRICTLY MORE COVERAGE. A scan added BESIDE a reused index moves no build
* count — the cache still hits, the counter still reads 1 — but it does move
* the traversal count. That mutation is the one `bench/import-target/`
* provably cannot see either: `baselines.json` `_blind_spot` records a full
* workspace scan on 1-in-32 imports passing every timing arm.
* - LESS PRODUCTION SURFACE. ~30 lines shipped in the bundle whose only caller
* outside a cache miss was this file.
* - PARALLEL-SAFE. The counter lives on the instance the test built, so there
* is no module-global to `reset()` and no ordering hazard between tests.
*
* The traversal-count assertions are the perf guard. They are paired with result
* assertions on purpose: a count of 1 is equally true of an adapter that has
* stopped resolving anything at all, so counting alone would stay green while
* every Kotlin IMPORTS edge disappeared.
*/
import { describe, it, expect } from 'vitest';
import { kotlinScopeResolver } from '../../src/core/ingestion/languages/kotlin/scope-resolver.js';
import {
getKotlinFileIndexBuildCount,
resetKotlinFileIndexBuildCount,
} from '../../src/core/ingestion/languages/kotlin/index-stats.js';
import { CountingSet, expectDistinctFileSetsGetOwnIndex } from '../helpers/counting-file-set.js';
// `resolveImportTarget` is a required member of `ScopeResolver`, so this is a
// plain read — no optional call, and no `toBeDefined()` guarding a branch that
@ -32,13 +46,13 @@ const { resolveImportTarget } = kotlinScopeResolver;
* over one shared package namespace, so a package is reachable only as a path
* suffix and never at the workspace root.
*/
function buildWorkspace(fileCount: number): Set<string> {
const files = new Set<string>();
function buildWorkspace(fileCount: number): CountingSet {
const files: string[] = [];
for (let i = 0; i < fileCount; i++) {
files.add(`lib${String(i).padStart(5, '0')}/src/main/kotlin/com/example/widget/Widget${i}.kt`);
files.push(`lib${String(i).padStart(5, '0')}/src/main/kotlin/com/example/widget/Widget${i}.kt`);
}
files.add('common/src/main/kotlin/com/example/common/Util.kt');
return files;
files.push('common/src/main/kotlin/com/example/common/Util.kt');
return new CountingSet(files);
}
const FROM_FILE = 'common/src/main/kotlin/com/example/common/Util.kt';
@ -46,29 +60,44 @@ const FROM_FILE = 'common/src/main/kotlin/com/example/common/Util.kt';
describe('Kotlin import resolution — index reuse across imports', () => {
it('builds the file index once for many imports over a stable file set', () => {
const files = buildWorkspace(300);
resetKotlinFileIndexBuildCount();
const resolved: (string | readonly string[] | null)[] = [];
for (let i = 0; i < 200; i++) {
// Alternates the two tiers that dominate real Kotlin source: a named type
// (tier 1, reached by path suffix) and a top-level function, which has no
// file named after it and so falls through to the package fan-out
for (let i = 0; i < 100; i++) {
// Both tiers that dominate real Kotlin source, every iteration: a named
// type (tier 1, reached by path suffix) and a top-level function, which
// has no file named after it and so falls through to the package fan-out
// (#1759). Driving one tier only would leave the other unmeasured here.
const target =
i % 2 === 0 ? `com.example.widget.Widget${i}` : `com.example.widget.someTopLevelFun${i}`;
resolveImportTarget(target, FROM_FILE, files);
resolved.push(
resolveImportTarget(`com.example.widget.Widget${i}`, FROM_FILE, files, undefined),
);
resolved.push(
resolveImportTarget(`com.example.widget.someTopLevelFun${i}`, FROM_FILE, files, undefined),
);
}
// An import that matches nothing at all, which runs the whole cascade —
// every tier misses and the progressive prefix strip walks to the end.
resolved.push(resolveImportTarget('org.absent.pkg.Missing', FROM_FILE, files, undefined));
expect(getKotlinFileIndexBuildCount()).toBe(1);
expect(files.scans).toBe(1);
// Paired result assertions — a count of 1 must not be the count of an
// adapter that resolves nothing. Tier 1 by path suffix, tier 3 fanning out
// over the package directory, and the total miss.
expect(resolved[0]).toBe('lib00000/src/main/kotlin/com/example/widget/Widget0.kt');
expect(resolved[1]).toHaveLength(300);
expect(resolved[200]).toBeNull();
});
it('rebuilds when the file set is a different object', () => {
resetKotlinFileIndexBuildCount();
for (let i = 0; i < 3; i++) {
resolveImportTarget('com.example.common.Util', 'a/B.kt', buildWorkspace(5));
}
expect(getKotlinFileIndexBuildCount()).toBe(3);
it('a distinct file set gets its own index (no stale cross-run reuse)', () => {
expectDistinctFileSetsGetOwnIndex({
resolveImportTarget,
buildWorkspace: () => buildWorkspace(5),
targetRaw: 'com.example.common.Util',
fromFile: 'a/B.kt',
resolutionConfig: undefined,
expected: 'common/src/main/kotlin/com/example/common/Util.kt',
expectedScans: 1,
});
});
it('still resolves real imports correctly (the perf test is not vacuous)', () => {
@ -76,7 +105,7 @@ describe('Kotlin import resolution — index reuse across imports', () => {
// Tier 1 through the adapter. The package sits under a module source root,
// so this resolves by path suffix, not by an exact workspace-rooted match.
expect(resolveImportTarget('com.example.widget.Widget7', FROM_FILE, files)).toBe(
expect(resolveImportTarget('com.example.widget.Widget7', FROM_FILE, files, undefined)).toBe(
'lib00007/src/main/kotlin/com/example/widget/Widget7.kt',
);
@ -84,11 +113,19 @@ describe('Kotlin import resolution — index reuse across imports', () => {
// it, so the stripped path resolves to the package directory and fans out
// to every file in it. The finalize pass then picks the one whose localDefs
// export the name (#1759).
const fanOut = resolveImportTarget('com.example.widget.someTopLevelFun', FROM_FILE, files);
const fanOut = resolveImportTarget(
'com.example.widget.someTopLevelFun',
FROM_FILE,
files,
undefined,
);
expect(fanOut).toHaveLength(20);
expect(fanOut).toContain('lib00000/src/main/kotlin/com/example/widget/Widget0.kt');
// An import that matches nothing in the workspace resolves to null.
expect(resolveImportTarget('org.absent.pkg.Missing', FROM_FILE, files)).toBeNull();
expect(resolveImportTarget('org.absent.pkg.Missing', FROM_FILE, files, undefined)).toBeNull();
// All of it off one traversal.
expect(files.scans).toBe(1);
});
});

147
gitnexus/test/integration/php-import-index-reuse.test.ts Normal file
View file

@ -0,0 +1,147 @@
/**
* Production-path regression guard for the PHP import-resolution index (#2901).
*
* PHP was the last language resolving imports with a full workspace scan per
* import. Both adapters in `languages/php/import-target.ts` materialized
* `[...allFilePaths]` twice per import and handed `resolvePhpImportInternal` an
* `index` of `undefined`, dropping it onto `suffixResolve`'s linear `findIndex`
* — a pass over every file per path-part × per extension, 98 ms per import at
* 20k files. They now read the shared `getWorkspaceFileIndex`
* (`import-resolvers/workspace-file-index.ts`), memoized on the `allFilePaths`
* Set identity via a WeakMap, through a PHP-specific parity view that keeps the
* three index-fed fast paths answering exactly what the scans answered (see the
* `#2901` header in `import-target.ts` — passing the raw shared index straight
* through MOVES IMPORTS edges, and `test/unit/scope-resolution/
* php-import-target-parity.test.ts` is the differential that proves this one
* does not).
*
* Resolution reaches that index through `phpScopeResolver.resolveImportTarget`
* — the orchestrator adapter — not by calling `resolvePhpImportTargetInternal`
* directly the way the unit parity test does. The adapter must therefore pass
* the Set THROUGH; a defensive copy (`new Set(allFilePaths)`) would hand a
* fresh WeakMap key per call and restore the per-import rebuild. Python hit
* exactly that (PR #1918 review P1), and the parity test cannot see it: it
* never crosses the adapter.
*
* The traversal-count assertions are the perf guard. They are paired with
* result assertions on purpose: a count of 1 is equally true of an adapter that
* has stopped resolving anything at all, so counting alone would stay green
* while every PHP IMPORTS edge disappeared.
*
* On the one traversal PHP still pays per import in a specific case — a PSR-4
* namespace whose directory has no direct `.php` children — see the pinned
* residual arm at the bottom of the unit parity test. It lives in
* `import-resolvers/php.ts`, which #2901 does not touch, so the corpora here
* resolve through the legs that do reach the index.
*/
import { describe, it, expect } from 'vitest';
import { phpScopeResolver } from '../../src/core/ingestion/languages/php/scope-resolver.js';
import type { ComposerConfig } from '../../src/core/ingestion/language-config.js';
import { CountingSet, expectDistinctFileSetsGetOwnIndex } from '../helpers/counting-file-set.js';
const { resolveImportTarget } = phpScopeResolver;
const FROM_FILE = 'app/Main.php';
/** The `composer.json` PSR-4 map `loadPhpComposerConfig` would have produced. */
const COMPOSER: ComposerConfig = { psr4: new Map([['App', 'app']]) };
/**
* A synthetic PSR-4 app: many service classes, plus the shapes the three
* index-fed legs answer — `app/Models/User.php` for the class-style whole-path
* hit, the populated `app/Models/` directory for the function-import fallback,
* and `lib/Legacy/Helper.php` for the suffix fallback that runs when no PSR-4
* prefix matches.
*/
function buildWorkspace(fileCount: number): CountingSet {
const files: string[] = [];
for (let i = 0; i < fileCount; i++) {
files.push(`app/Services/Service${String(i).padStart(5, '0')}.php`);
}
files.push('app/Models/User.php');
files.push('app/Models/functions.php');
files.push('lib/Legacy/Helper.php');
files.push('index.php');
files.push(FROM_FILE);
return new CountingSet(files);
}
describe('PHP import resolution — index reuse across use-statements (#2901)', () => {
it('builds the workspace index once for many imports over a stable file set', () => {
const files = buildWorkspace(300);
const resolved: (string | readonly string[] | null)[] = [];
for (let i = 0; i < 200; i++) {
// A PSR-4 class hit, a function import that falls back to the namespace
// directory, and a third-party namespace that misses. The miss is the
// expensive case: it matches no PSR-4 prefix and so walks every suffix ×
// every extension before returning null.
resolved.push(resolveImportTarget('App\\Models\\User', FROM_FILE, files, COMPOSER));
resolved.push(resolveImportTarget('App\\Models\\getUser', FROM_FILE, files, COMPOSER));
resolved.push(resolveImportTarget(`Psr\\Log\\Missing${i}`, FROM_FILE, files, COMPOSER));
}
expect(files.scans).toBe(1);
// Paired result assertions — a count of 1 must not be the count of an
// adapter that resolves nothing.
expect(resolved[0]).toBe('app/Models/User.php');
expect(resolved[1]).toBe('app/Models/User.php');
expect(resolved[2]).toBeNull();
});
it('builds the workspace index once with no composer.json at all', () => {
const files = buildWorkspace(300);
const resolved: (string | readonly string[] | null)[] = [];
// `loadResolutionConfig` returns null when the repo has no composer.json,
// which skips the PSR-4 block entirely and leaves `suffixResolve` — the leg
// that used to cost a `findIndex` pass per extension — as the only path.
for (let i = 0; i < 200; i++) {
resolved.push(resolveImportTarget('Legacy\\Helper', FROM_FILE, files, null));
resolved.push(resolveImportTarget(`Psr\\Log\\Missing${i}`, FROM_FILE, files, null));
}
expect(files.scans).toBe(1);
expect(resolved[0]).toBe('lib/Legacy/Helper.php');
expect(resolved[1]).toBeNull();
});
it('a distinct file set gets its own index (no stale cross-run reuse)', () => {
expectDistinctFileSetsGetOwnIndex({
resolveImportTarget,
buildWorkspace: () => buildWorkspace(20),
targetRaw: 'App\\Models\\User',
fromFile: FROM_FILE,
resolutionConfig: COMPOSER,
expected: 'app/Models/User.php',
expectedScans: 1,
});
});
it('still resolves real use-statements correctly (the perf test is not vacuous)', () => {
const files = buildWorkspace(5);
// PSR-4 class-style: `App\Models\User` → `app/Models/User.php`.
expect(resolveImportTarget('App\\Models\\User', FROM_FILE, files, COMPOSER)).toBe(
'app/Models/User.php',
);
expect(resolveImportTarget('App\\Services\\Service00000', FROM_FILE, files, COMPOSER)).toBe(
'app/Services/Service00000.php',
);
// Suffix fallback: no PSR-4 prefix matches `Legacy`, so `suffixResolve`
// answers from the longest matching proper path suffix.
expect(resolveImportTarget('Legacy\\Helper', FROM_FILE, files, COMPOSER)).toBe(
'lib/Legacy/Helper.php',
);
// A root-level file is NOT reachable as a proper suffix — the pre-#2901
// behaviour the parity view preserves, and the single most likely thing a
// raw `getWorkspaceFileIndex().index` hand-off would have changed.
expect(resolveImportTarget('index', FROM_FILE, files, COMPOSER)).toBeNull();
// Third-party namespaces have no file in the repo.
expect(resolveImportTarget('Psr\\Log\\LoggerInterface', FROM_FILE, files, COMPOSER)).toBeNull();
});
});

106
gitnexus/test/integration/python-import-index-reuse.test.ts
View file

@ -1,7 +1,8 @@
/**
* Production-path regression guard for PR #1918 review finding P1.
*
* The Python file index (`getPythonFileIndex` in `import-target.ts`) is
* The Python file index (`getPythonFileIndex` in
* `import-resolvers/python-file-index.ts`) is
* memoized on the `allFilePaths` Set identity via a WeakMap. The registry-
* primary path reaches it through `pythonScopeResolver.resolveImportTarget`
* (the orchestrator adapter) — NOT by calling `resolvePythonImportTarget`
@ -10,15 +11,38 @@
* WeakMap key per call so the index rebuilt every import (O(imports × files)).
*
* This test drives the adapter exactly as the orchestrator does and asserts the
* index is built ONCE across many imports on a stable set. It fails (build
* count == number of imports) if the per-import copy is reintroduced.
* file set is traversed ONCE across many imports on a stable set. It fails
* (one traversal per import) if the per-import copy is reintroduced.
*
* ## Why this counts TRAVERSALS and not index builds (#2909)
*
* This guard used to read a build counter that shipped in production purely so
* a test could read it (now deleted). `CountingSet`
* (`test/helpers/counting-file-set.ts`) replaces it, and the swap is not a
* wash:
*
* - STRICTLY MORE COVERAGE. A scan added BESIDE a reused index moves no build
* count — the cache still hits, the counter still reads 1 — but it does move
* the traversal count. That mutation is the one `bench/import-target/`
* provably cannot see either: `baselines.json` `_blind_spot` records a full
* workspace scan on 1-in-32 imports passing every timing arm.
* - LESS PRODUCTION SURFACE. ~30 lines shipped in the bundle whose only caller
* outside a cache miss was this file.
* - PARALLEL-SAFE. The counter lives on the instance the test built, so there
* is no module-global to `reset()` and no ordering hazard between tests.
*
* The traversal-count assertions are the perf guard. They are paired with result
* assertions on purpose: a count of 1 is equally true of an adapter that has
* stopped resolving anything at all, so counting alone would stay green while
* every Python IMPORTS edge disappeared.
*/
import { describe, it, expect } from 'vitest';
import { pythonScopeResolver } from '../../src/core/ingestion/languages/python/scope-resolver.js';
import {
getPythonFileIndexBuildCount,
resetPythonFileIndexBuildCount,
} from '../../src/core/ingestion/languages/python/index-stats.js';
import { CountingSet, expectDistinctFileSetsGetOwnIndex } from '../helpers/counting-file-set.js';
const { resolveImportTarget } = pythonScopeResolver;
const FROM_FILE = 'app/main.py';
/**
* A synthetic workspace: a real package (`realpkg/__init__.py`, so the
@ -26,62 +50,60 @@ import {
* below are multi-segment and miss every fast path, so each call reaches both
* `hasRepoCandidate` and `resolveAbsoluteFromFiles` — the two index consumers.
*/
function buildWorkspace(fileCount: number): Set<string> {
const files = new Set<string>();
function buildWorkspace(fileCount: number): CountingSet {
const files: string[] = [];
for (let i = 0; i < fileCount; i++) {
files.add(`pkg/sub/mod${String(i).padStart(5, '0')}.py`);
files.push(`pkg/sub/mod${String(i).padStart(5, '0')}.py`);
}
files.add('realpkg/__init__.py');
files.add('realpkg/widget.py');
return files;
files.push('realpkg/__init__.py');
files.push('realpkg/widget.py');
return new CountingSet(files);
}
describe('Python import resolution — index reuse across imports (PR #1918 P1)', () => {
it('builds the file index once for many imports over a stable file set', () => {
const allFilePaths = buildWorkspace(300);
const fromFile = 'app/main.py';
const importCount = 300;
const files = buildWorkspace(300);
const resolved: (string | readonly string[] | null)[] = [];
resetPythonFileIndexBuildCount();
for (let i = 0; i < importCount; i++) {
for (let i = 0; i < 300; i++) {
// Multi-segment, candidate-passing, suffix-miss → reaches the index.
pythonScopeResolver.resolveImportTarget(`realpkg.ghost${i}`, fromFile, allFilePaths);
resolved.push(resolveImportTarget(`realpkg.ghost${i}`, FROM_FILE, files, undefined));
}
resolved.push(resolveImportTarget('realpkg.widget', FROM_FILE, files, undefined));
// The whole point of PR #1918: O(imports + files), not O(imports × files).
// Pre-fix this was 300 (one rebuild per import via the adapter's Set copy).
expect(getPythonFileIndexBuildCount()).toBe(1);
expect(files.scans).toBe(1);
// Paired result assertions — a count of 1 must not be the count of an
// adapter that resolves nothing.
expect(resolved[0]).toBeNull();
expect(resolved[300]).toBe('realpkg/widget.py');
});
it('rebuilds once per distinct file set (per-run isolation, no stale reuse)', () => {
const fromFile = 'app/main.py';
resetPythonFileIndexBuildCount();
const setA = buildWorkspace(50);
for (let i = 0; i < 20; i++) {
pythonScopeResolver.resolveImportTarget(`realpkg.ghost${i}`, fromFile, setA);
}
expect(getPythonFileIndexBuildCount()).toBe(1);
// A different Set instance is a different logical workspace → one more build.
const setB = buildWorkspace(50);
for (let i = 0; i < 20; i++) {
pythonScopeResolver.resolveImportTarget(`realpkg.ghost${i}`, fromFile, setB);
}
expect(getPythonFileIndexBuildCount()).toBe(2);
it('a distinct file set gets its own index (per-run isolation, no stale reuse)', () => {
expectDistinctFileSetsGetOwnIndex({
resolveImportTarget,
buildWorkspace: () => buildWorkspace(50),
targetRaw: 'realpkg.widget',
fromFile: FROM_FILE,
resolutionConfig: undefined,
expected: 'realpkg/widget.py',
expectedScans: 1,
});
});
it('still resolves real imports correctly (the perf test is not vacuous)', () => {
const allFilePaths = buildWorkspace(20);
const fromFile = 'app/main.py';
const files = buildWorkspace(20);
// Suffix-fallback hit through the adapter: realpkg.widget → realpkg/widget.py.
expect(pythonScopeResolver.resolveImportTarget('realpkg.widget', fromFile, allFilePaths)).toBe(
expect(resolveImportTarget('realpkg.widget', FROM_FILE, files, undefined)).toBe(
'realpkg/widget.py',
);
// Gated-out / unresolvable import returns null.
expect(
pythonScopeResolver.resolveImportTarget('realpkg.ghost', fromFile, allFilePaths),
).toBeNull();
expect(resolveImportTarget('realpkg.ghost', FROM_FILE, files, undefined)).toBeNull();
// All of it off one traversal.
expect(files.scans).toBe(1);
});
});

184
gitnexus/test/integration/typescript-import-index-reuse.test.ts Normal file
View file

@ -0,0 +1,184 @@
/**
* Production-path regression guard for the TypeScript import-resolution pass
* cache (#2910).
*
* `makeTsResolveImportTarget` has carried a `SuffixIndex` since #1918, so the
* per-import rebuild this file's siblings were written for never applied here.
* What did apply is the OTHER failure mode of the memo it used: a single slot,
* invalidated by `cached.key !== allFilePaths`. One file set is memoized
* perfectly; two alternating file sets rebuild the arrays, the index and the
* `resolveCache` on every single call. Measured at 4000 files × 400 imports:
* 12.0 ms for one set, 1438.2 ms alternating between two — 120x, and the same
* O(imports × files) shape the per-file-set index hoists removed.
*
* That is why this adapter could not carry `expectDistinctFileSetsGetOwnIndex`,
* the one arm every other language's guard has: the arm alternates two sets by
* construction, and the single-slot cache posts 42 traversals against the 2 it
* posts now. The cache is a `WeakMap<ReadonlySet<string>, PassCache>` keyed on
* the Set, like every other language's index, and the arm below is the proof.
*
* Whether the thrash was reachable in production: `pipeline/run.ts` builds one
* `allFilePaths` Set per provider pass and TypeScript, JavaScript and Vue are
* separate providers with separate caches, so within one analyze it was latent
* rather than live. It was one refactor — an interleaved or re-entrant pass, a
* second workspace, a caller resolving against a filtered file set — away from
* live, and the `WeakMap` is strictly simpler than the slot it replaces.
*
* Resolution goes through `typescriptScopeResolver.resolveImportTarget`, the
* orchestrator adapter, which must pass the Set THROUGH: a defensive
* `new Set(allFilePaths)` hands a fresh `WeakMap` key per import and restores
* the per-import rebuild (PR #1918 review P1). Two traversals per file set, not
* one: the adapter materializes `allFileList` and then keeps one mutable copy
* of the Set, because `TsResolveContext.allFilePaths` is a `Set`, not a
* `ReadonlySet`.
*
* The counts are paired with result assertions on purpose: a count of 2 is
* equally true of an adapter that has stopped resolving anything at all.
*/
import { describe, it, expect } from 'vitest';
import { typescriptScopeResolver } from '../../src/core/ingestion/languages/typescript/scope-resolver.js';
import type { TsconfigPaths } from '../../src/core/ingestion/language-config.js';
import { CountingSet, expectDistinctFileSetsGetOwnIndex } from '../helpers/counting-file-set.js';
const { resolveImportTarget } = typescriptScopeResolver;
const FROM_FILE = 'src/main.ts';
/** What `loadTsconfigPaths` produces for `"@/*": ["src/*"]` under `baseUrl: "."`. */
const TSCONFIG_PATHS: TsconfigPaths = { aliases: new Map([['@/', 'src/']]), baseUrl: '.' };
/** The shape `loadResolutionConfig` returns for a non-Nuxt TypeScript repo. */
const RESOLUTION_CONFIG = { tsconfigPaths: TSCONFIG_PATHS, nuxtAutoImports: null };
/**
* A synthetic TypeScript app covering the legs the resolver takes: a relative
* import answered by exact `Set.has`, an ESM `.js` specifier that must strip to
* `.ts`, a bare specifier answered by path suffix, a directory `index.ts`, and
* a `@/`-aliased path.
*/
function buildWorkspace(fileCount: number): CountingSet {
const files: string[] = [];
for (let i = 0; i < fileCount; i++) {
files.push(`src/services/Service${String(i).padStart(5, '0')}.ts`);
}
files.push('src/util.ts');
files.push('src/models/index.ts');
files.push('src/components/Widget.tsx');
files.push('node_modules/dep/index.d.ts');
files.push(FROM_FILE);
return new CountingSet(files);
}
describe('TypeScript import resolution — index reuse across imports (#2910)', () => {
it('builds the pass cache once for many imports over a stable file set', () => {
const files = buildWorkspace(300);
const resolved: (string | readonly string[] | null)[] = [];
for (let i = 0; i < 200; i++) {
// A relative hit, an ESM `.js` specifier stripped back to `.ts`, an
// aliased path, a bare-specifier suffix hit, and a bare specifier that
// misses — the expensive case, which runs every path part × every
// extension before returning null.
resolved.push(resolveImportTarget('./util', FROM_FILE, files, RESOLUTION_CONFIG));
resolved.push(resolveImportTarget('./util.js', FROM_FILE, files, RESOLUTION_CONFIG));
resolved.push(resolveImportTarget('@/models', FROM_FILE, files, RESOLUTION_CONFIG));
resolved.push(resolveImportTarget('components/Widget', FROM_FILE, files, RESOLUTION_CONFIG));
resolved.push(
resolveImportTarget(`@vendor/ghost${i}/deep`, FROM_FILE, files, RESOLUTION_CONFIG),
);
}
// Two: `Array.from(allFilePaths)` and the one mutable `Set` copy the
// resolver context requires. Both happen once per file set.
expect(files.scans).toBe(2);
// Paired result assertions — a count of 2 must not be the count of an
// adapter that resolves nothing.
expect(resolved[0]).toBe('src/util.ts');
expect(resolved[1]).toBe('src/util.ts');
expect(resolved[2]).toBe('src/models/index.ts');
expect(resolved[3]).toBe('src/components/Widget.tsx');
expect(resolved[4]).toBeNull();
});
/**
* The arm the single-slot cache could not pass. `expectDistinctFileSetsGetOwnIndex`
* alternates two file sets 20 times; the slot was invalidated on every one of
* those calls, so each set posted 42 traversals instead of 2.
*/
it('a distinct file set gets its own index (no stale cross-run reuse)', () => {
expectDistinctFileSetsGetOwnIndex({
resolveImportTarget,
buildWorkspace: () => buildWorkspace(20),
targetRaw: '@/models',
fromFile: FROM_FILE,
resolutionConfig: RESOLUTION_CONFIG,
expected: 'src/models/index.ts',
expectedScans: 2,
});
});
/**
* The per-file-set index and `resolveCache` must not become global. The arm
* above cannot see that: `expectDistinctFileSetsGetOwnIndex` builds two
* IDENTICAL corpora, so a stale answer carried across them is also the right
* answer, and only its traversal counts would notice. These two workspaces
* answer the same specifier differently, and they are resolved alternately.
*/
it('two different workspaces answer the same specifier differently', () => {
const a = new Set(['src/util.ts', FROM_FILE]);
const b = new Set(['vendor/util.ts', FROM_FILE]);
expect(resolveImportTarget('util', FROM_FILE, a, RESOLUTION_CONFIG)).toBe('src/util.ts');
expect(resolveImportTarget('util', FROM_FILE, b, RESOLUTION_CONFIG)).toBe('vendor/util.ts');
expect(resolveImportTarget('util', FROM_FILE, a, RESOLUTION_CONFIG)).toBe('src/util.ts');
expect(resolveImportTarget('util', FROM_FILE, b, RESOLUTION_CONFIG)).toBe('vendor/util.ts');
});
it('still resolves real imports correctly (the perf test is not vacuous)', () => {
const files = buildWorkspace(5);
// Relative, extensionless and with the ESM `.js` spelling.
expect(resolveImportTarget('./util', FROM_FILE, files, RESOLUTION_CONFIG)).toBe('src/util.ts');
expect(resolveImportTarget('./util.js', FROM_FILE, files, RESOLUTION_CONFIG)).toBe(
'src/util.ts',
);
// Directory index.
expect(resolveImportTarget('./models', FROM_FILE, files, RESOLUTION_CONFIG)).toBe(
'src/models/index.ts',
);
// tsconfig alias.
expect(
resolveImportTarget('@/services/Service00000', FROM_FILE, files, RESOLUTION_CONFIG),
).toBe('src/services/Service00000.ts');
// Bare specifier resolved by path suffix.
expect(resolveImportTarget('components/Widget', FROM_FILE, files, RESOLUTION_CONFIG)).toBe(
'src/components/Widget.tsx',
);
// Nothing in the repo answers these.
expect(resolveImportTarget('./nowhere', FROM_FILE, files, RESOLUTION_CONFIG)).toBeNull();
expect(
resolveImportTarget('@vendor/ghost/deep', FROM_FILE, files, RESOLUTION_CONFIG),
).toBeNull();
});
/**
* No `tsconfig.json` at all, which is the config the orchestrator threads for
* a repo without one. It skips the alias branch entirely and leaves the
* suffix cascade as the only path to the index.
*/
it('builds the pass cache once with no tsconfig paths', () => {
const files = buildWorkspace(300);
const resolved: (string | readonly string[] | null)[] = [];
for (let i = 0; i < 200; i++) {
resolved.push(resolveImportTarget('components/Widget', FROM_FILE, files, undefined));
resolved.push(resolveImportTarget(`@vendor/ghost${i}/deep`, FROM_FILE, files, undefined));
}
expect(files.scans).toBe(2);
expect(resolved[0]).toBe('src/components/Widget.tsx');
expect(resolved[1]).toBeNull();
});
});

168
gitnexus/test/integration/vue-import-index-reuse.test.ts Normal file
View file

@ -0,0 +1,168 @@
/**
* Production-path regression guard for the Vue import-resolution pass cache
* (#2910).
*
* `makeVueResolveImportTarget` is the TypeScript adapter with the language
* pinned to TypeScript, and it inherited the same memo: a single slot,
* invalidated by `cached.key !== allFilePaths`. One file set is memoized
* perfectly; two alternating file sets rebuild the arrays, the suffix index and
* the `resolveCache` on every call. Measured on the identical TypeScript
* adapter at 4000 files × 400 imports: 12.0 ms for one set, 1438.2 ms
* alternating between two — 120x, and the same O(imports × files) shape the
* per-file-set index hoists removed.
*
* That is why this adapter could not carry `expectDistinctFileSetsGetOwnIndex`,
* the one arm every other language's guard has: the arm alternates two sets by
* construction, and the single-slot cache posts 42 traversals against the 2 it
* posts now. The cache is a `WeakMap<ReadonlySet<string>, PassCache>` keyed on
* the Set, like every other language's index, and the arm below is the proof.
*
* Whether the thrash was reachable in production: `pipeline/run.ts` builds one
* `allFilePaths` Set per provider pass and Vue, TypeScript and JavaScript are
* separate providers with separate caches, so within one analyze it was latent
* rather than live — one refactor away from live, and the `WeakMap` is strictly
* simpler than the slot it replaces.
*
* Resolution goes through `vueScopeResolver.resolveImportTarget`, the
* orchestrator adapter, which must pass the Set THROUGH: a defensive
* `new Set(allFilePaths)` hands a fresh `WeakMap` key per import and restores
* the per-import rebuild (PR #1918 review P1). Two traversals per file set, not
* one: the adapter materializes `allFileList` and then keeps one mutable copy
* of the Set, because `TsResolveContext.allFilePaths` is a `Set`, not a
* `ReadonlySet`.
*
* The counts are paired with result assertions on purpose: a count of 2 is
* equally true of an adapter that has stopped resolving anything at all.
*/
import { describe, it, expect } from 'vitest';
import { vueScopeResolver } from '../../src/core/ingestion/languages/vue/scope-resolver.js';
import type { TsconfigPaths } from '../../src/core/ingestion/language-config.js';
import { CountingSet, expectDistinctFileSetsGetOwnIndex } from '../helpers/counting-file-set.js';
const { resolveImportTarget } = vueScopeResolver;
const FROM_FILE = 'src/App.vue';
/** What `loadTsconfigPaths` produces for `"@/*": ["src/*"]` under `baseUrl: "."`. */
const TSCONFIG_PATHS: TsconfigPaths = { aliases: new Map([['@/', 'src/']]), baseUrl: '.' };
/** The shape `loadResolutionConfig` returns for a Vue repo with a tsconfig. */
const RESOLUTION_CONFIG = { tsconfigPaths: TSCONFIG_PATHS };
/**
* A synthetic Vue SFC project: many components, a `.ts` composable reached by
* bare specifier, a `@/`-aliased path, and a directory `index.ts`. `.vue`
* imports carry their extension, so they land on the exact-path branch.
*/
function buildWorkspace(fileCount: number): CountingSet {
const files: string[] = [];
for (let i = 0; i < fileCount; i++) {
files.push(`src/components/Widget${String(i).padStart(5, '0')}.vue`);
}
files.push('src/composables/useUser.ts');
files.push('src/stores/index.ts');
files.push('src/util.ts');
files.push(FROM_FILE);
return new CountingSet(files);
}
describe('Vue import resolution — index reuse across imports (#2910)', () => {
it('builds the pass cache once for many imports over a stable file set', () => {
const files = buildWorkspace(300);
const resolved: (string | readonly string[] | null)[] = [];
for (let i = 0; i < 200; i++) {
// A relative `.vue` hit, a relative `.ts` composable, an aliased path, a
// bare-specifier suffix hit, and a bare specifier that misses — the
// expensive case, which runs every path part × every extension before
// returning null.
resolved.push(
resolveImportTarget('./components/Widget00000.vue', FROM_FILE, files, RESOLUTION_CONFIG),
);
resolved.push(
resolveImportTarget('./composables/useUser', FROM_FILE, files, RESOLUTION_CONFIG),
);
resolved.push(resolveImportTarget('@/stores', FROM_FILE, files, RESOLUTION_CONFIG));
resolved.push(
resolveImportTarget('composables/useUser', FROM_FILE, files, RESOLUTION_CONFIG),
);
resolved.push(
resolveImportTarget(`@vendor/ghost${i}/deep`, FROM_FILE, files, RESOLUTION_CONFIG),
);
}
// Two: `Array.from(allFilePaths)` and the one mutable `Set` copy the
// resolver context requires. Both happen once per file set.
expect(files.scans).toBe(2);
// Paired result assertions — a count of 2 must not be the count of an
// adapter that resolves nothing.
expect(resolved[0]).toBe('src/components/Widget00000.vue');
expect(resolved[1]).toBe('src/composables/useUser.ts');
expect(resolved[2]).toBe('src/stores/index.ts');
expect(resolved[3]).toBe('src/composables/useUser.ts');
expect(resolved[4]).toBeNull();
});
/**
* The arm the single-slot cache could not pass. `expectDistinctFileSetsGetOwnIndex`
* alternates two file sets 20 times; the slot was invalidated on every one of
* those calls, so each set posted 42 traversals instead of 2.
*/
it('a distinct file set gets its own index (no stale cross-run reuse)', () => {
expectDistinctFileSetsGetOwnIndex({
resolveImportTarget,
buildWorkspace: () => buildWorkspace(20),
targetRaw: '@/stores',
fromFile: FROM_FILE,
resolutionConfig: RESOLUTION_CONFIG,
expected: 'src/stores/index.ts',
expectedScans: 2,
});
});
/**
* The per-file-set index and `resolveCache` must not become global. The arm
* above cannot see that: `expectDistinctFileSetsGetOwnIndex` builds two
* IDENTICAL corpora, so a stale answer carried across them is also the right
* answer, and only its traversal counts would notice. These two workspaces
* answer the same specifier differently, and they are resolved alternately.
*/
it('two different workspaces answer the same specifier differently', () => {
const a = new Set(['src/util.ts', FROM_FILE]);
const b = new Set(['vendor/util.ts', FROM_FILE]);
expect(resolveImportTarget('util', FROM_FILE, a, RESOLUTION_CONFIG)).toBe('src/util.ts');
expect(resolveImportTarget('util', FROM_FILE, b, RESOLUTION_CONFIG)).toBe('vendor/util.ts');
expect(resolveImportTarget('util', FROM_FILE, a, RESOLUTION_CONFIG)).toBe('src/util.ts');
expect(resolveImportTarget('util', FROM_FILE, b, RESOLUTION_CONFIG)).toBe('vendor/util.ts');
});
it('still resolves real SFC imports correctly (the perf test is not vacuous)', () => {
const files = buildWorkspace(5);
// `.vue` imports are written with their extension and hit the exact-path
// branch.
expect(
resolveImportTarget('./components/Widget00000.vue', FROM_FILE, files, RESOLUTION_CONFIG),
).toBe('src/components/Widget00000.vue');
// A composable, relative and extensionless.
expect(resolveImportTarget('./composables/useUser', FROM_FILE, files, RESOLUTION_CONFIG)).toBe(
'src/composables/useUser.ts',
);
// Directory index behind a tsconfig alias.
expect(resolveImportTarget('@/stores', FROM_FILE, files, RESOLUTION_CONFIG)).toBe(
'src/stores/index.ts',
);
// Bare specifier resolved by path suffix.
expect(
resolveImportTarget('components/Widget00000.vue', FROM_FILE, files, RESOLUTION_CONFIG),
).toBe('src/components/Widget00000.vue');
// Nothing in the repo answers these.
expect(resolveImportTarget('./Missing.vue', FROM_FILE, files, RESOLUTION_CONFIG)).toBeNull();
expect(
resolveImportTarget('@vendor/ghost/deep', FROM_FILE, files, RESOLUTION_CONFIG),
).toBeNull();
});
});

620
gitnexus/test/unit/import-resolvers/csharp-csproj-parity.test.ts Normal file
View file

@ -0,0 +1,620 @@
/**
* Differential harness for the C# **csproj** leg of `resolveCSharpImportInternal`
* (#2902).
*
* #2878 moved C#'s no-csproj leg onto memoized indexes. The csproj leg kept a
* per-import, per-matching-config Θ(files) scan — step 3, "linear scan fallback
* for directory matching" — measured at ~1.08 ms per import over 50 000 `.cs`
* files. This PR answers that leg from a per-file-list directory index instead.
*
* WHY A VERBATIM COPY AND NOT A DELETION. The obvious cleanup is "step 2 already
* asks the index the same question, so skip step 3 whenever an index exists".
* That is wrong, and this file is the proof. Step 2 filters
* `index.getFilesInDir(dirPrefix, '.cs')`, whose buckets are keyed on
* SEGMENT-aligned directory suffixes; step 3 runs an UNANCHORED
* `normalized.indexOf(dirPrefix + '/')`. Step 3 therefore answers strictly more:
*
* - `dirPrefix = 'ubModels'` matches `src/SubModels/` (character suffix of a
* segment, not a segment);
* - `dirPrefix = 'rc/Models'` matches BOTH `src/Models/` and
* `vendor/mysrc/Models/`;
* - `dirPrefix = ''` — the "the import IS the root namespace and the config
* has no projectDir" case — matches every `.cs` file exactly one directory
* deep. `buildSuffixIndex` emits an empty directory suffix only for a path
* that BEGINS with '/', so over repo-relative paths `getFilesInDir('',
* '.cs')` is always empty and step 3 is the only implementation that case
* has ever had. (The leading-slash shape has its own arm below, kept off
* the main corpus precisely because step 2 DOES answer it.)
*
* Step 3 also runs only when step 2 found nothing, so those extra hits are
* observable rather than shadowed. `skips step 3 when the index is present`
* below pins that: it drives the naive cleanup and asserts it CHANGES answers.
*
* The arms all assert the full `string[]` and its order — this resolver returns
* every match, not one, and `configs/csharp.ts` turns a multi-file result into a
* `kind: 'package'` edge set whose order reaches the graph.
*/
import { describe, expect, it } from 'vitest';
import {
resolveCSharpImportInternal,
resolveCSharpNamespaceDir,
} from '../../../src/core/ingestion/import-resolvers/csharp.js';
import {
buildSuffixIndex,
suffixResolve,
type SuffixIndex,
} from '../../../src/core/ingestion/import-resolvers/utils.js';
import { csharpSuffixFallbackAllowed } from '../../../src/core/ingestion/csharp-namespace-gate.js';
import { CountingSet } from '../../helpers/counting-file-set.js';
import type {
CSharpProjectConfig,
CSharpNamespaceEvidence,
} from '../../../src/core/ingestion/language-config.js';
// ─── verbatim pre-change implementation ──────────────────────────────────────
// `git show HEAD~:gitnexus/src/core/ingestion/import-resolvers/csharp.ts`, body
// copied unchanged. Its helpers (`suffixResolve`, `csharpSuffixFallbackAllowed`,
// `SuffixIndex`) are imported from production because this PR does not touch
// them; only the function below changed.
function legacyResolveCSharpImportInternal(
importPath: string,
csharpConfigs: CSharpProjectConfig[],
normalizedFileList: string[],
allFileList: string[],
index?: SuffixIndex,
evidence?: CSharpNamespaceEvidence,
): string[] {
const namespacePath = importPath.replace(/\./g, '/');
const results: string[] = [];
for (const config of csharpConfigs) {
const nsPath = config.rootNamespace.replace(/\./g, '/');
let relative: string;
if (namespacePath.startsWith(nsPath + '/')) {
relative = namespacePath.slice(nsPath.length + 1);
} else if (namespacePath === nsPath) {
// The import IS the root namespace — resolve to all .cs files in project root
relative = '';
} else {
continue;
}
const dirPrefix = config.projectDir
? relative
? config.projectDir + '/' + relative
: config.projectDir
: relative;
// 1. Try as single file: relative.cs (e.g., "Models/DlqMessage.cs")
if (relative) {
const candidate = dirPrefix + '.cs';
if (index) {
const result = index.get(candidate) || index.getInsensitive(candidate);
if (result) return [result];
}
// Also try suffix match
const suffixResult = index?.get(relative + '.cs') || index?.getInsensitive(relative + '.cs');
if (suffixResult) return [suffixResult];
}
// 2. Try as directory: all .cs files directly inside (namespace import)
if (index) {
const dirFiles = index.getFilesInDir(dirPrefix, '.cs');
for (const f of dirFiles) {
const normalized = f.replace(/\\/g, '/');
// Check it's a direct child by finding the dirPrefix and ensuring no deeper slashes
const prefixIdx = normalized.indexOf(dirPrefix + '/');
if (prefixIdx < 0) continue;
const afterDir = normalized.substring(prefixIdx + dirPrefix.length + 1);
if (!afterDir.includes('/')) {
results.push(f);
}
}
if (results.length > 0) return results;
}
// 3. Linear scan fallback for directory matching
if (results.length === 0) {
const dirTrail = dirPrefix + '/';
for (let i = 0; i < normalizedFileList.length; i++) {
const normalized = normalizedFileList[i];
if (!normalized.endsWith('.cs')) continue;
const prefixIdx = normalized.indexOf(dirTrail);
if (prefixIdx < 0) continue;
const afterDir = normalized.substring(prefixIdx + dirTrail.length);
if (!afterDir.includes('/')) {
results.push(allFileList[i]);
}
}
if (results.length > 0) return results;
}
}
// Fallback: suffix matching without namespace stripping (single file).
// Gated on in-repo declared-namespace evidence (#1881).
if (!csharpSuffixFallbackAllowed(importPath, evidence)) {
return [];
}
const pathParts = namespacePath.split('/').filter(Boolean);
const fallback = suffixResolve(pathParts, normalizedFileList, allFileList, index);
return fallback ? [fallback] : [];
}
/**
* The naive cleanup this PR deliberately did NOT do: keep step 3 only as an
* un-indexed fallback. Same body as the legacy copy with step 3 gated on
* `index === undefined`. Driven by one arm below, which asserts it diverges.
*/
function skipStep3WhenIndexed(
importPath: string,
csharpConfigs: CSharpProjectConfig[],
normalizedFileList: string[],
allFileList: string[],
index?: SuffixIndex,
evidence?: CSharpNamespaceEvidence,
): string[] {
const namespacePath = importPath.replace(/\./g, '/');
const results: string[] = [];
for (const config of csharpConfigs) {
const nsPath = config.rootNamespace.replace(/\./g, '/');
let relative: string;
if (namespacePath.startsWith(nsPath + '/')) {
relative = namespacePath.slice(nsPath.length + 1);
} else if (namespacePath === nsPath) {
relative = '';
} else {
continue;
}
const dirPrefix = config.projectDir
? relative
? config.projectDir + '/' + relative
: config.projectDir
: relative;
if (relative) {
const candidate = dirPrefix + '.cs';
if (index) {
const result = index.get(candidate) || index.getInsensitive(candidate);
if (result) return [result];
}
const suffixResult = index?.get(relative + '.cs') || index?.getInsensitive(relative + '.cs');
if (suffixResult) return [suffixResult];
}
if (index) {
const dirFiles = index.getFilesInDir(dirPrefix, '.cs');
for (const f of dirFiles) {
const normalized = f.replace(/\\/g, '/');
const prefixIdx = normalized.indexOf(dirPrefix + '/');
if (prefixIdx < 0) continue;
const afterDir = normalized.substring(prefixIdx + dirPrefix.length + 1);
if (!afterDir.includes('/')) {
results.push(f);
}
}
if (results.length > 0) return results;
continue;
}
const dirTrail = dirPrefix + '/';
for (let i = 0; i < normalizedFileList.length; i++) {
const normalized = normalizedFileList[i];
if (!normalized.endsWith('.cs')) continue;
const prefixIdx = normalized.indexOf(dirTrail);
if (prefixIdx < 0) continue;
const afterDir = normalized.substring(prefixIdx + dirTrail.length);
if (!afterDir.includes('/')) {
results.push(allFileList[i]);
}
}
if (results.length > 0) return results;
}
if (!csharpSuffixFallbackAllowed(importPath, evidence)) {
return [];
}
const pathParts = namespacePath.split('/').filter(Boolean);
const fallback = suffixResolve(pathParts, normalizedFileList, allFileList, index);
return fallback ? [fallback] : [];
}
// ─── corpus ──────────────────────────────────────────────────────────────────
/**
* Hand-built so every tie-break the scan expressed through `indexOf` positions
* and file-list order has a witness. Order matters: the resolver emits in
* file-list order, so the interleavings below (`src/Models/Late.cs` after
* `other/Models/Thing.cs`, `src/Extra.cs` after `Models/TopLevel.cs`) are what
* make a directory-at-a-time emit distinguishable from a merged one.
*/
const RAW_FILES: readonly string[] = [
// Repo-root files: no directory at all, so no `dirPrefix + '/'` can ever hit.
'Root.cs',
'notes.txt',
// The `src` project.
'src/Program.cs',
'src/Startup.cs',
'src/Models/User.cs',
'src/Models/Order.cs',
'src/Models/Deep/Nested.cs',
// Character suffix of a segment, NOT a segment: answers `ubModels`, and
// answers `Models` only when no segment-aligned `Models` directory does.
'src/SubModels/Widget.cs',
'src/Services/UserService.cs',
'src/Services/Sub/Inner.cs',
// A second directory sharing the `Models` last segment, minted BEFORE
// `src/Models/Late.cs` so multi-directory answers have to interleave.
'other/Models/Thing.cs',
// Character suffix across a segment boundary: answers `rc/Models`.
'vendor/mysrc/Models/Vendored.cs',
'src/Models/Late.cs',
// `Models` nested inside `Models`: the FIRST `indexOf` occurrence is the
// outer one, whose remainder still holds a slash, so this answers nothing.
'nest/Models/inner/Models/Ignored.cs',
// Single-segment directory, so it answers the empty `dirPrefix`.
'Models/TopLevel.cs',
// Backslash separators: `allFileList` keeps them, the predicate runs on the
// normalized form, and the emitted value is the RAW one.
'win\\Models\\Win.cs',
'win\\Deep\\Models\\Deeper\\Skip.cs',
// Second project root, plus a case-only twin for the case-insensitive leg.
'lib/Core/Widgets/Widget.cs',
'lib/Core/Widgets.cs',
'lib/Core/widgets/Lower.cs',
// Second single-segment-directory file, after `Models/TopLevel.cs`.
'src/Extra.cs',
// Non-`.cs` files INSIDE directories that answer queries, so dropping the
// extension filter is visible rather than shadowed by the root-level
// `notes.txt` (which no `dirPrefix + '/'` can reach anyway).
'src/notes.md',
'Models/schema.json',
];
const ALL_FILE_LIST: string[] = [...RAW_FILES];
const NORMALIZED_FILE_LIST: string[] = ALL_FILE_LIST.map((f) => f.replace(/\\/g, '/'));
const SUFFIX_INDEX: SuffixIndex = buildSuffixIndex(NORMALIZED_FILE_LIST, ALL_FILE_LIST);
// One Set per corpus, built once: `resolveCSharpImportInternal` now derives its
// normalized/raw lists from `getWorkspaceFileIndex(allFilePaths)`, whose memo is
// keyed on this object's identity. `RAW_FILES` is duplicate-free, so the derived
// pair is `NORMALIZED_FILE_LIST`/`ALL_FILE_LIST` element for element — which is
// what keeps the differential below a like-for-like comparison against the
// frozen legacy implementation, which still takes the two arrays.
const ALL_FILE_PATHS: ReadonlySet<string> = new Set(ALL_FILE_LIST);
const CONFIG_SHAPES: ReadonlyArray<readonly [string, CSharpProjectConfig[]]> = [
['no configs at all', []],
['projectDir=src', [{ rootNamespace: 'App', projectDir: 'src' }]],
// `projectDir` is a required `string`, so "without projectDir" is the empty
// string the `config.projectDir ? …` ternary treats as absent.
['no projectDir', [{ rootNamespace: 'App', projectDir: '' }]],
['dotted root namespace', [{ rootNamespace: 'Acme.App', projectDir: 'src' }]],
['nested projectDir', [{ rootNamespace: 'Lib', projectDir: 'lib/Core' }]],
// Unanchored projectDirs: neither is a directory in the corpus, so both fall
// through to step 3 and match by character suffix.
['unanchored projectDir', [{ rootNamespace: 'App', projectDir: 'rc' }]],
// A projectDir that already starts with '/' makes `dirPrefix` one character
// LONGER than a directory it shares a last segment with, the one shape where
// `indexOf` and `haystack.length - needle.length` both come out -1.
['absolute projectDir', [{ rootNamespace: 'App', projectDir: '/Models' }]],
[
'two configs, both match',
[
{ rootNamespace: 'App', projectDir: 'nope' },
{ rootNamespace: 'App', projectDir: 'src' },
],
],
[
'two configs, second root namespace extends the first',
[
{ rootNamespace: 'App', projectDir: 'src' },
{ rootNamespace: 'App.Models', projectDir: 'other' },
],
],
[
'two configs, the matching one is second and has no projectDir',
[
{ rootNamespace: 'Zzz', projectDir: 'src' },
{ rootNamespace: 'App', projectDir: '' },
],
],
['no config matches', [{ rootNamespace: 'Zzz', projectDir: 'src' }]],
];
const IMPORTS: readonly string[] = [
// Root-namespace-equals-import, against every projectDir shape.
'App',
'Acme.App',
'Lib',
// Directories that exist, segment-aligned.
'App.Models',
'App.Services',
'App.Services.Sub',
'App.Models.Deep',
'App.SubModels',
'Acme.App.Models',
'Lib.Widgets',
// Case-only variants (step 1's `getInsensitive` legs).
'Lib.widgets',
'App.models',
// Character-suffix-only directories: segment-aligned lookups find nothing.
'App.ubModels',
'App.odels',
'App.Models.Late',
// A namespace with no matching directory anywhere — the issue's trigger.
'App.Missing',
'App.Missing.Deeper',
'Acme.App.Missing',
// Single files rather than directories.
'App.Program',
'App.Root',
'Lib.Core.Widgets',
// Imports that match no configured root namespace at all (BCL usings).
'System',
'System.Threading.Tasks',
'Models',
'Models.TopLevel',
];
/** Every (config shape, import) pair, plus both index modes. */
const PAIRS: ReadonlyArray<{
readonly key: string;
readonly configs: CSharpProjectConfig[];
readonly importPath: string;
readonly index: SuffixIndex | undefined;
}> = CONFIG_SHAPES.flatMap(([shape, configs]) =>
IMPORTS.flatMap((importPath) =>
[SUFFIX_INDEX, undefined].map((index) => ({
key: `${shape} | ${importPath} | index=${index === undefined ? 'absent' : 'present'}`,
configs,
importPath,
index,
})),
),
);
function runCurrent(pair: (typeof PAIRS)[number]): string[] {
return resolveCSharpImportInternal(pair.importPath, pair.configs, ALL_FILE_PATHS, pair.index);
}
function runLegacy(pair: (typeof PAIRS)[number]): string[] {
return legacyResolveCSharpImportInternal(
pair.importPath,
pair.configs,
NORMALIZED_FILE_LIST,
ALL_FILE_LIST,
pair.index,
);
}
/** `label -> joined result`, so a mismatch prints the pair AND both answers. */
function table(run: (pair: (typeof PAIRS)[number]) => string[]): Record<string, string> {
const out: Record<string, string> = {};
for (const pair of PAIRS) out[pair.key] = run(pair).join(' , ');
return out;
}
describe('C# csproj leg — directory index vs the pre-change linear scan (#2902)', () => {
it('returns byte-identical results, in order, for every config shape and import', () => {
expect(table(runCurrent)).toEqual(table(runLegacy));
});
it('agrees on the `#1881` evidence gate too (the suffix fallback is downstream of step 3)', () => {
const evidence: CSharpNamespaceEvidence = {
declaredNamespaces: new Set(['App', 'App.Models', 'Lib.Core']),
rootNamespaces: new Set(['App', 'Lib']),
truncated: false,
};
const current: Record<string, string> = {};
const legacy: Record<string, string> = {};
for (const pair of PAIRS) {
current[pair.key] = resolveCSharpImportInternal(
pair.importPath,
pair.configs,
ALL_FILE_PATHS,
pair.index,
evidence,
).join(' , ');
legacy[pair.key] = legacyResolveCSharpImportInternal(
pair.importPath,
pair.configs,
NORMALIZED_FILE_LIST,
ALL_FILE_LIST,
pair.index,
evidence,
).join(' , ');
}
expect(current).toEqual(legacy);
});
it('leaves `resolveCSharpNamespaceDir` — the sibling that shares the dirPrefix maths — alone', () => {
const dirs: Record<string, string | null> = {};
for (const [shape, configs] of CONFIG_SHAPES) {
for (const importPath of IMPORTS) {
dirs[`${shape} | ${importPath}`] = resolveCSharpNamespaceDir(importPath, configs);
}
}
expect(dirs['projectDir=src | App.Models']).toBe('/src/Models/');
expect(dirs['no projectDir | App']).toBeNull();
expect(dirs['no projectDir | App.Models']).toBe('/Models/');
expect(dirs['no config matches | App.Models']).toBeNull();
});
});
describe('C# csproj leg — the answers only step 3 can give (#2902)', () => {
const withIndex = (configs: CSharpProjectConfig[], importPath: string): string[] =>
resolveCSharpImportInternal(importPath, configs, ALL_FILE_PATHS, SUFFIX_INDEX);
it('`relative === ""` with no projectDir gives dirPrefix "" — every .cs one level deep, merged', () => {
// `getFilesInDir('', '.cs')` is empty for every file set, so step 2 cannot
// answer this at all. Two directories match (`src`, `Models`) and their
// files interleave, so a directory-at-a-time emit reorders this.
expect(withIndex([{ rootNamespace: 'App', projectDir: '' }], 'App')).toEqual([
'src/Program.cs',
'src/Startup.cs',
'Models/TopLevel.cs',
'src/Extra.cs',
]);
});
it('`relative === ""` WITH a projectDir gives dirPrefix = projectDir, answered by step 2', () => {
expect(withIndex([{ rootNamespace: 'App', projectDir: 'src' }], 'App')).toEqual([
'src/Program.cs',
'src/Startup.cs',
'src/Extra.cs',
]);
});
it('matches a directory by CHARACTER suffix of a segment, which no segment bucket holds', () => {
expect(withIndex([{ rootNamespace: 'App', projectDir: '' }], 'App.ubModels')).toEqual([
'src/SubModels/Widget.cs',
]);
});
it('matches a character suffix ACROSS a segment boundary, over several directories', () => {
expect(withIndex([{ rootNamespace: 'App', projectDir: 'rc' }], 'App.Models')).toEqual([
'src/Models/User.cs',
'src/Models/Order.cs',
'vendor/mysrc/Models/Vendored.cs',
'src/Models/Late.cs',
]);
});
it('keeps the FIRST-occurrence tie-break: a directory nested inside a same-named one loses', () => {
// `nest/Models/inner/Models/Ignored.cs` is absent: `indexOf('odels/')` finds
// the outer `Models/`, and `inner/Models/Ignored.cs` still has a slash.
expect(withIndex([{ rootNamespace: 'App', projectDir: '' }], 'App.odels')).toEqual([
'src/Models/User.cs',
'src/Models/Order.cs',
'src/SubModels/Widget.cs',
'other/Models/Thing.cs',
'vendor/mysrc/Models/Vendored.cs',
'src/Models/Late.cs',
'Models/TopLevel.cs',
'win\\Models\\Win.cs',
]);
});
it('emits the RAW path for backslash-separated files while matching on the normalized one', () => {
expect(withIndex([{ rootNamespace: 'App', projectDir: 'win' }], 'App.Models')).toEqual([
'win\\Models\\Win.cs',
]);
});
it('a leading-slash dirPrefix cannot bogus-match a shorter directory', () => {
// `dirPrefix = '/Models'` (projectDir used verbatim, since the import IS
// the root namespace): `'Models/'` is SHORTER than `'/Models/'`, and both
// `indexOf` and `haystack.length - needle.length` come out -1 without a
// length guard, so `Models/TopLevel.cs` would join the answer.
expect(withIndex([{ rootNamespace: 'App', projectDir: '/Models' }], 'App')).toEqual([
'src/Models/User.cs',
'src/Models/Order.cs',
'other/Models/Thing.cs',
'vendor/mysrc/Models/Vendored.cs',
'src/Models/Late.cs',
'win\\Models\\Win.cs',
]);
});
it('an import with no matching directory resolves to nothing (the issue trigger)', () => {
expect(withIndex([{ rootNamespace: 'App', projectDir: 'src' }], 'App.Missing')).toEqual([]);
});
it('skipping step 3 when the index is present CHANGES answers — the fallback is load-bearing', () => {
const divergent = PAIRS.filter(
(pair) =>
pair.index !== undefined &&
runCurrent(pair).join(' , ') !==
skipStep3WhenIndexed(
pair.importPath,
pair.configs,
NORMALIZED_FILE_LIST,
ALL_FILE_LIST,
pair.index,
).join(' , '),
).map((pair) => pair.key);
expect(divergent).toContain('no projectDir | App | index=present');
expect(divergent).toContain('no projectDir | App.ubModels | index=present');
expect(divergent).toContain('unanchored projectDir | App.Models | index=present');
expect(divergent.length).toBeGreaterThan(10);
});
it('the parity arms are not vacuous: most pairs resolve, and step 3 answers many of them', () => {
const nonEmpty = PAIRS.filter((pair) => runCurrent(pair).length > 0);
const multiFile = PAIRS.filter((pair) => runCurrent(pair).length > 1);
expect(nonEmpty.length).toBeGreaterThan(PAIRS.length / 3);
expect(multiFile.length).toBeGreaterThan(20);
});
});
describe('C# csproj leg — the directory index is built once per file set (#2902)', () => {
it('resolves many imports with a single pass over the file set', () => {
// `CountingSet`, not a counting ARRAY. This arm used to proxy
// `normalizedFileList` and count reads of `[0]`, because the index was keyed
// on that array; #2911 rekeyed it onto the Set, so the file set is now the
// only thing a rebuild has to re-traverse and the one instrument every other
// import-index guard already uses covers this leg too.
const files = new CountingSet(ALL_FILE_LIST);
const index = buildSuffixIndex([...NORMALIZED_FILE_LIST], ALL_FILE_LIST);
const configs: CSharpProjectConfig[] = [{ rootNamespace: 'App', projectDir: 'src' }];
for (let i = 0; i < 40; i++) {
// Every one of these misses steps 1 and 2 and reaches step 3, so the
// directory index is genuinely consulted 40 times.
resolveCSharpImportInternal(`App.Missing${i % 4}`, configs, files, index);
}
expect(files.scans).toBe(1);
});
it('a path that BEGINS with a slash keeps parity (its directory is the empty string)', () => {
// Kept off the main corpus on purpose: `buildSuffixIndex` DOES emit an
// empty directory suffix for such a path, so step 2 would answer the empty
// `dirPrefix` here and short-circuit the very leg the arms above pin.
const raw = ['/Rooted.cs', 'src/Nested.cs', '/Other.cs'];
const normalized = raw.map((f) => f.replace(/\\/g, '/'));
// Duplicate-free, so the resolver derives exactly `normalized`/`raw` from
// it and the two sides of the differential still see the same corpus.
const rootedPaths: ReadonlySet<string> = new Set(raw);
const index = buildSuffixIndex([...normalized], [...raw]);
const shapes: CSharpProjectConfig[][] = [
[{ rootNamespace: 'App', projectDir: '' }],
[{ rootNamespace: 'App', projectDir: 'src' }],
[{ rootNamespace: 'App', projectDir: '/' }],
];
const current: string[][] = [];
const legacy: string[][] = [];
for (const configs of shapes) {
for (const importPath of ['App', 'App.Nested', 'App.Rooted']) {
for (const withIndex of [index, undefined]) {
current.push(resolveCSharpImportInternal(importPath, configs, rootedPaths, withIndex));
legacy.push(
legacyResolveCSharpImportInternal(importPath, configs, [...normalized], raw, withIndex),
);
}
}
}
expect(current).toEqual(legacy);
// Not vacuous: the un-indexed empty-dirPrefix query reaches `dir === ''`.
expect(
resolveCSharpImportInternal(
'App',
[{ rootNamespace: 'App', projectDir: '' }],
rootedPaths,
undefined,
),
).toEqual(['/Rooted.cs', 'src/Nested.cs', '/Other.cs']);
});
it('a distinct file set gets its own index (no stale cross-run reuse)', () => {
const other: ReadonlySet<string> = new Set(['App2/Models/Only.cs']);
const configs: CSharpProjectConfig[] = [{ rootNamespace: 'App', projectDir: 'App2' }];
expect(resolveCSharpImportInternal('App.Models', configs, other, undefined)).toEqual([
'App2/Models/Only.cs',
]);
expect(resolveCSharpImportInternal('App.Models', configs, ALL_FILE_PATHS, undefined)).toEqual(
[],
);
});
});

426
gitnexus/test/unit/import-resolvers/python-importer-prefixes.test.ts Normal file
View file

@ -0,0 +1,426 @@
/**
* Gate for the single-segment bare-import ancestor walk in
* `import-resolvers/python.ts`: Python bare-import resolution must not scale
* with the importer's path depth.
*
* #2913 memoized the ancestor chains inside `languages/python/import-target.ts`
* and left this one behind, because its chain is a DIFFERENT SEQUENCE (self
* excluded, workspace root included, empty components kept) and no bench arm
* can reach it — `bench/import-target/measure.mjs` spells every Python import
* with a dot, and this walk runs only for a spelling with none. So it kept
* rebuilding `dirParts.slice(0, i).join('/')` per path component per import,
* and for `from x import y` / `import x as y` it did so TWICE per import:
* `resolvePythonImportTarget` probes the package with
* `targetIncludesImportedName` first and, when that misses, falls through to
* the identical call. Measured on a 400-file corpus, 3200 named single-segment
* imports: 24 `allFilePaths.has` probes per import at four directory
* components, the second twelve byte-identical to the first.
*
* ## Why this is a count and not a timing budget
*
* `test/helpers/counting-file-set.ts` is the house instrument for import-target
* reuse guards and it cannot see this defect, for the reason the #2913 gate
* states: the chain is derived from the `fromFile` STRING, and a rebuilt prefix
* traverses the file set zero extra times and issues the same `has` probes with
* the same arguments in the same order. A hoist is invisible to any instrument
* watching the resolver's inputs. So this file watches the memo, which is the
* one place the hoist is observable.
*
* The gate: `prefixMemo(files).size` after N imports from D
* importer directories must be D, for every N. That is "the prefix work is O(1)
* amortized after the first import from a given directory", stated as a number.
* It is paired with a reference-identity assertion, because a memo that stores
* a FRESH array on every import posts the same size while doing all of the work
* again, and with a non-vacuity assertion, because a perfect count is equally
* true of a resolver that stopped resolving anything.
*
* Both production surfaces are driven, not the helper: `pythonScopeResolver
* .resolveImportTarget` (the scope-resolution orchestrator's adapter, the one
* that pays the walk twice) and `pythonImportStrategy` (the import-resolver
* pipeline's, via `ImportTargetWorkspace`'s shared `ResolveCtx`). They thread
* different objects around the same Set, and the memo is keyed on the Set.
*
* `legacyPrefixes` is a verbatim copy of the pre-change inline code, in the
* house style of `python-importer-ancestors.test.ts`: it is the specification,
* and the memo agreeing with it is what makes this a hoist rather than a
* behaviour change.
*
* The four arms themselves live in `test/helpers/counting-file-set.ts`, beside
* the other import-target scaffolding: this guard and the #2913 one are the
* same suite over the same importer corpus once four values are named (the
* memo, the drive, the legacy builder, the hit), and they were previously
* written out twice. The two chains still DIFFER — this one keeps the empty
* components an absolute path or a doubled separator produces, and #2913's
* drops them — which is why `legacyChain` is per-guard and the shared path-shape
* table names shapes rather than expectations.
*/
import { describe, expect, it } from 'vitest';
import type { ParsedImport } from 'gitnexus-shared';
import type { ImportResolutionContext } from '../../../src/core/ingestion/scope-resolution/contract/scope-resolver.js';
import { pythonScopeResolver } from '../../../src/core/ingestion/languages/python/scope-resolver.js';
import { resolvePythonImportInternal } from '../../../src/core/ingestion/import-resolvers/python.js';
import { getPythonFileIndex } from '../../../src/core/ingestion/import-resolvers/python-file-index.js';
/** The bare-import prefix memo, which lives inside the shared per-file-set index. */
const prefixMemo = (files: ReadonlySet<string>): ReadonlyMap<string, readonly string[]> =>
getPythonFileIndex(files).bareImportPrefixesByDir;
import { pythonImportStrategy } from '../../../src/core/ingestion/import-resolvers/configs/python.js';
import { buildSuffixIndex } from '../../../src/core/ingestion/import-resolvers/utils.js';
import type {
ImportResult,
ResolveCtx,
} from '../../../src/core/ingestion/import-resolvers/types.js';
import {
IMPORTER_PATH_SHAPES,
NO_PARSED_FILES,
expectDistinctFileSetsGetOwnChainMemo,
expectMemoizedChainMatchesLegacy,
expectOneChainPerImporterDir,
expectSameChainObjectReused,
pythonNamedImport,
pythonNamespaceImport,
type ChainMemoArm,
type ChainMemoResult,
} from '../../helpers/counting-file-set.js';
const { resolveImportTarget } = pythonScopeResolver;
// ─── verbatim pre-change implementation ──────────────────────────────────────
/** The prefix sequence the inline walk materialized on every import. */
function legacyPrefixes(currentFile: string): string[] {
const importerDir = currentFile.replace(/\\/g, '/').split('/').slice(0, -1).join('/');
const prefixes: string[] = [];
const dirParts = importerDir.split('/');
for (let i = dirParts.length - 1; i >= 0; i--) {
const ancestorDir = dirParts.slice(0, i).join('/');
prefixes.push(ancestorDir ? `${ancestorDir}/` : '');
}
return prefixes;
}
// ─── surfaces ────────────────────────────────────────────────────────────────
/**
* A `ResolveCtx` for the import-resolver pipeline surface. `allFilePaths` is
* the caller's Set passed THROUGH — the memo is keyed on its identity, so a
* copy here would measure nothing.
*/
function makeResolveCtx(allFilePaths: Set<string>): ResolveCtx {
const allFileList = [...allFilePaths];
const normalizedFileList = allFileList.map((file) => file.replace(/\\/g, '/'));
return {
allFilePaths,
allFileList,
normalizedFileList,
index: buildSuffixIndex(normalizedFileList, allFileList),
resolveCache: new Map<string, string | null>(),
configs: {
tsconfigPaths: null,
goModule: null,
composerConfig: null,
swiftPackageConfig: null,
csharpConfigs: [],
},
};
}
const ctxFor = (parsedImport: ParsedImport): ImportResolutionContext => ({
parsedFiles: NO_PARSED_FILES,
parsedImport,
});
/**
* An `ImportResult` as the arms read it: the resolved path, or `null` for a
* miss. A stop-the-chain result carrying no files joins to `''`, which is
* neither the hit nor a miss — the same distinction the pre-collapse arm drew
* with `value?.kind === 'files' && value.files.join() === HIT_RESULT`.
*/
const resolvedPath = (result: ImportResult): ChainMemoResult =>
result === null ? null : result.files.join();
// ─── the workspace the surfaces are driven against ───────────────────────────
/**
* `shared.py` sits at the workspace root, which is the LAST step of the walk,
* so every importer below reaches it only by running the chain to the end —
* the most expensive path, and the one the memo has to keep correct.
*
* `elsewhere/deep/probe.py` makes `probe` a segment that SURVIVES
* `pythonSegmentAbsent` (a file with that basename exists) while sitting in no
* importer's ancestry, so the walk runs to completion and misses. That
* combination is what a memo-filling miss looks like now: a segment the
* workspace has never heard of is retired in two Map lookups and never reaches
* the walk at all, which is the point of the early-out and the reason a
* `ghost{i}` spelling can no longer drive this memo.
*/
const WORKSPACE: readonly string[] = [
'svc/a/one.py',
'svc/a/two.py',
'svc/b/one.py',
'svc/common.py',
'deep/x/y/z/one.py',
'elsewhere/deep/probe.py',
'shared.py',
'root.py',
];
const HIT_TARGET = 'shared';
const HIT_RESULT = 'shared.py';
/** Survives the absence proof, then walks the whole chain and misses — the
* dotted tier's suffix fallback picks it up as `elsewhere/deep/probe.py`. */
const WALK_TARGET = 'probe';
/** Provably absent: retired before the walk, so it never touches the memo. */
const ABSENT_TARGET = 'ghostmod';
/**
* The spelling sequence BOTH surfaces are driven with, `perImporter` rounds of
* it plus the one spelling that must resolve: one target that walks the whole
* chain and misses, one that the absence proof retires before the walk, and one
* that hits at the workspace root. No spelling is varied per round — the Python
* chain keeps no per-target cache, so a repeated target really is re-resolved.
*
* `resolve` is the only thing that differs between the orchestrator adapter and
* the import-resolver pipeline; everything about the sequence is shared, which
* is why the two used to be the same loop written twice.
*/
function drive<T>(
fromFile: string,
perImporter: number,
resolve: (targetRaw: string, fromFile: string) => T,
): T[] {
const out: T[] = [];
for (let i = 0; i < perImporter; i++) {
for (const target of [WALK_TARGET, ABSENT_TARGET]) out.push(resolve(target, fromFile));
}
out.push(resolve(HIT_TARGET, fromFile));
return out;
}
/** The ORCHESTRATOR ADAPTER — the surface that pays the walk twice. */
const adapterArm = (mkImport: (targetRaw: string) => ParsedImport): ChainMemoArm => ({
memoOf: prefixMemo,
drive: (files, fromFile, perImporter) =>
drive(fromFile, perImporter, (target, from) =>
resolveImportTarget(target, from, files, undefined, ctxFor(mkImport(target))),
),
legacyChain: legacyPrefixes,
hitResult: HIT_RESULT,
});
/**
* The import-resolver pipeline surface. One `ResolveCtx` per drive rather than
* one for the run, on purpose: everything the strategy reads off it is derived
* from the same Set (and `pythonImportStrategy` only ever WRITES
* `resolveCache`), so a fresh ctx around the same Set is exactly the "different
* objects, same Set" case the memo has to survive.
*/
const strategyArm: ChainMemoArm = {
memoOf: prefixMemo,
drive: (files, fromFile, perImporter) => {
const ctx = makeResolveCtx(files);
return drive(fromFile, perImporter, (target, from) =>
resolvedPath(pythonImportStrategy(target, from, ctx)),
);
},
legacyChain: legacyPrefixes,
hitResult: HIT_RESULT,
};
describe('Python bare-import prefix memo', () => {
it.each([
{ perImporter: 1, kind: 'namespace', mkImport: pythonNamespaceImport },
{ perImporter: 40, kind: 'namespace', mkImport: pythonNamespaceImport },
{ perImporter: 1, kind: 'named', mkImport: pythonNamedImport },
{ perImporter: 40, kind: 'named', mkImport: pythonNamedImport },
])(
'holds one chain per importer DIRECTORY, not per import — $perImporter x $kind',
({ perImporter, mkImport }) => {
expectOneChainPerImporterDir(adapterArm(mkImport), new Set(WORKSPACE), perImporter);
},
);
it.each([
{ perImporter: 1, label: 'one import per importer' },
{ perImporter: 40, label: 'forty imports per importer' },
])(
'holds one chain per importer DIRECTORY on the import-resolver surface too — $label',
({ perImporter }) => {
expectOneChainPerImporterDir(strategyArm, new Set(WORKSPACE), perImporter);
},
);
it('reuses the SAME chain object, rather than rebuilding and re-storing it', () => {
expectSameChainObjectReused(adapterArm(pythonNamedImport), new Set(WORKSPACE));
});
it.each(IMPORTER_PATH_SHAPES)(
'memoizes the chain the pre-change code built — $why',
({ fromFile }) => {
expectMemoizedChainMatchesLegacy(
adapterArm(pythonNamespaceImport),
new Set(WORKSPACE),
fromFile,
);
},
);
it.each([
{ perImporter: 2, label: 'two imports per importer' },
{ perImporter: 20, label: 'twenty imports per importer' },
])(
'gives a distinct file set its own memo (no leak across passes) — $label',
({ perImporter }) => {
expectDistinctFileSetsGetOwnChainMemo(
adapterArm(pythonNamedImport),
new Set(WORKSPACE),
new Set(WORKSPACE),
perImporter,
);
},
);
/**
* The memo is filled from the importers a pass actually resolves against, so
* it is bounded by DIRECTORIES THAT IMPORT — never by the file count and
* never by the repo's directory count, which is the bound #2649 asks for.
*/
it.each([
{ dirs: 4, importsPerDir: 1 },
{ dirs: 4, importsPerDir: 50 },
{ dirs: 30, importsPerDir: 7 },
])(
'is bounded by importing directories, not by files or imports — $dirs dirs x $importsPerDir',
({ dirs, importsPerDir }) => {
const paths: string[] = [];
for (let d = 0; d < dirs; d++) {
for (let f = 0; f < 25; f++) paths.push(`pkg${d}/nest/file${f}.py`);
}
paths.push('shared.py');
const files = new Set(paths);
const resolved: ChainMemoResult[] = [];
for (let d = 0; d < dirs; d++) {
for (let i = 0; i < importsPerDir; i++) {
resolved.push(
resolveImportTarget(
HIT_TARGET,
`pkg${d}/nest/file0.py`,
files,
undefined,
ctxFor(pythonNamedImport(HIT_TARGET)),
),
);
}
}
expect(prefixMemo(files).size).toBe(dirs);
expect(resolved.filter((value) => value === HIT_RESULT)).toHaveLength(dirs * importsPerDir);
},
);
});
/**
* Absolute expectations for the walk itself. `test/unit/suffix-index-ambiguity
* .test.ts` covers the proximity tier and the suffix fallback around it; the
* ANCESTOR tier — the thing issue #417 added and the thing this change touches
* — had no absolute coverage at all, in particular none for the two path shapes
* where its chain differs from `ancestorsByDir`'s: absolute paths and doubled
* separators, both of which a `filter(Boolean)` would send to the wrong files.
*/
describe('Python bare-import ancestor walk — resolution', () => {
it.each([
{
why: 'the importer own directory wins over every ancestor',
files: ['app/svc/user.py', 'app/user.py', 'user.py', 'app/svc/auth.py'],
fromFile: 'app/svc/auth.py',
expected: 'app/svc/user.py',
},
{
why: 'a same-directory package beats a same-directory module (PEP 451 §4)',
files: ['app/svc/user/__init__.py', 'app/svc/user.py', 'app/svc/auth.py'],
fromFile: 'app/svc/auth.py',
expected: 'app/svc/user/__init__.py',
},
{
why: 'the CLOSEST ancestor wins (#417)',
files: ['app/user.py', 'user.py', 'app/svc/auth.py'],
fromFile: 'app/svc/auth.py',
expected: 'app/user.py',
},
{
why: 'a package beats a module at the same ancestor step',
files: ['app/user/__init__.py', 'app/user.py', 'app/svc/auth.py'],
fromFile: 'app/svc/auth.py',
expected: 'app/user/__init__.py',
},
{
why: 'the workspace root is the last step of the walk',
files: ['user.py', 'app/svc/auth.py'],
fromFile: 'app/svc/auth.py',
expected: 'user.py',
},
{
why: 'a root-level importer walks the root and nothing else',
files: ['user.py', 'auth.py'],
fromFile: 'auth.py',
expected: 'user.py',
},
{
why: 'an absolute workspace keeps the leading empty component',
files: ['/repo/app/user.py', '/repo/app/svc/auth.py'],
fromFile: '/repo/app/svc/auth.py',
expected: '/repo/app/user.py',
},
{
why: 'a doubled separator keeps the empty component',
files: ['a//user.py', 'a//b/auth.py'],
fromFile: 'a//b/auth.py',
expected: 'a//user.py',
},
{
why: 'Windows separators in the importer normalize before the walk',
files: ['app/user.py', 'app/svc/auth.py'],
fromFile: 'app\\svc\\auth.py',
expected: 'app/user.py',
},
])('$why', ({ files, fromFile, expected }) => {
const set = new Set(files);
// The helper, the scope-resolution adapter and the import-resolver
// strategy must agree: all three reach the same walk.
expect(resolvePythonImportInternal(fromFile, 'user', set)).toBe(expected);
expect(
resolveImportTarget('user', fromFile, set, undefined, ctxFor(pythonNamespaceImport('user'))),
).toBe(expected);
expect(pythonImportStrategy('user', fromFile, makeResolveCtx(set))).toEqual({
kind: 'files',
files: [expected],
});
});
it.each([
{
why: 'a module outside the importer ancestry is not an ancestor hit (#417)',
files: ['other/branch/user.py', 'app/svc/auth.py'],
fromFile: 'app/svc/auth.py',
},
{
why: 'a namespace package (no __init__.py) has no file to resolve to',
files: ['app/user/model.py', 'app/svc/auth.py'],
fromFile: 'app/svc/auth.py',
},
{
why: 'a sibling directory of the importer is not an ancestor',
files: ['app/other/user.py', 'app/svc/auth.py'],
fromFile: 'app/svc/auth.py',
},
{
why: 'an absolute workspace does not answer a de-rooted prefix',
files: ['repo/app/user.py', '/repo/app/svc/auth.py'],
fromFile: '/repo/app/svc/auth.py',
},
])('returns null and lets the caller fall through — $why', ({ files, fromFile }) => {
expect(resolvePythonImportInternal(fromFile, 'user', new Set(files))).toBeNull();
});
});

910
gitnexus/test/unit/import-resolvers/suffix-index-lazy-dir-map.test.ts Normal file
View file

@ -0,0 +1,910 @@
/**
* `buildSuffixIndex` builds NOTHING at construction. Each of its three maps is
* built the first time a question needs it, and one of them is DERIVED from
* another rather than traversed for. (The file name predates the change: #2903
* deferred `dirMap` alone, and the two suffix maps followed.)
*
* What the index now does:
*
* - `get` builds `exactMap` by one pass over the file list;
* - `getInsensitive` after `get` DERIVES `lowerMap` from `exactMap` — one pass
* over that map's distinct keys, not a second pass over the file list;
* - `getInsensitive` first builds `lowerMap` straight off the file list, so a
* case-insensitive-only consumer holds exactly one map. Asking `get`
* afterwards is the documented fallback and does cost the second traversal;
* no consumer uses that order;
* - `getFilesInDir` builds `dirMap`, unchanged since #2903;
* - and over the pre-lowercased list `pass-cache.ts` hands it (TypeScript,
* JavaScript, Vue) the derivation is the identity, so `getInsensitive` reads
* the exact map ITSELF rather than a copy of it — one map, both questions.
*
* All of it is memory. `dirMap` is the array-valued map — one entry and one
* array push per file per directory component, O(files × depth) in entries and
* in churn — and only four call sites ever read it
* (`import-resolvers/{php,csharp,jvm}.ts`, `import-resolvers/configs/
* python.ts`), yet `workspace-file-index.ts` serving Ruby,
* `languages/typescript/scope-resolver.ts`, `languages/vue/import-target.ts`
* and `group/extractors/include-extractor.ts` all built it and never touched
* it: ~15% of the retained C# index and ~19% of the retained Ruby one on
* `bench/import-target/`'s 32k-path arms. The suffix maps are the same story
* one level down — `languages/java/import-target.ts` reads only `get` and was
* carrying 49.98 MiB of dead `lowerMap` at 32k paths, `languages/php/
* import-target.ts` reads only `getInsensitive` and was carrying 34.49 MiB of
* dead `exactMap`. Retained index: Java 80.26 -> 25.61 MiB, PHP 60.86 ->
* 32.09, JavaScript 44.07 -> 22.65. Since #2877-#2880 these indexes live for a
* whole resolution pass rather than being rebuilt per import, so all of that is
* retained memory against the #2649 kernel-scale OOM constraint.
*
* Deferring and deriving are only free if two things hold, and this file
* asserts both:
*
* 1. **Nothing observable moved.** `eagerDirMap` and `eagerSuffixMaps` below
* are verbatim copies of the pre-change loops, and the parity arms compare
* the built-on-demand answers against them over the FULL key space each
* corpus can produce — every suffix in three spellings, every directory
* suffix crossed with every extension, hits and misses alike — plus
* hand-written arms that pin buckets, collisions and their ORDER outright,
* so a parity arm cannot pass by two implementations being wrong together.
* Order is load-bearing twice over: `php.ts` returns `candidates[0]`, and
* the derived `lowerMap` is claimed byte-equal to a freshly built one in
* keys, values AND insertion order. Insertion order is not readable through
* this API, but its one consequence is: which file a case-folded key
* resolves to when several fold together. The parity arms run in BOTH build
* orders — derived and built-direct — over corpora that include
* case-colliding twins and a context-sensitive Greek final sigma.
* 2. **Each map is built at most ONCE, and only if asked for.** The laziness
* arms count index reads of the two input arrays. Every build pass reads
* each element exactly once, so the read count IS the pass count: 0 after
* construction, 1 for a `get`-only consumer however many times it asks, 1
* for a `getInsensitive`-only consumer, still 1 for `get` THEN
* `getInsensitive` because the derivation reads no file, and one more —
* once, forever — for `getFilesInDir`. Memoizing the DECISION rather than
* the MAP, rebuilding whenever a lookup misses, reads 3, 4, 5. This is a
* structural count, not a timing or a memory delta: exact and deterministic
* on any machine.
*
* What the counter cannot see is a map derived from another map, since that
* touches no file: it would catch a `lowerMap` rebuilt from the LIST beside a
* `get`, not one copied from `exactMap`. That is why the derivation is measured
* as costing zero passes rather than assumed absent, and why its contents are
* policed by the parity arms instead.
*
* Every count assertion is paired with a result assertion. A pass count of 0 is
* equally true of an index that has stopped answering — the pairing rule of
* `test/helpers/counting-file-set.ts` and the twelve
* `test/integration/*-import-index-reuse.test.ts` guards.
*/
import { describe, expect, it } from 'vitest';
import {
buildSuffixIndex,
type SuffixIndex,
} from '../../../src/core/ingestion/import-resolvers/utils.js';
// ─── verbatim pre-change implementations ─────────────────────────────────────
/**
* The directory-membership half of `buildSuffixIndex` exactly as it stood
* before #2903, lifted out of the shared loop and otherwise untouched. This is
* the specification the deferred build is measured against.
*/
function eagerDirMap(normalizedFileList: string[], allFileList: string[]): Map<string, string[]> {
const dirMap = new Map<string, string[]>();
for (let i = 0; i < normalizedFileList.length; i++) {
const normalized = normalizedFileList[i];
const original = allFileList[i];
const parts = normalized.split('/');
const lastSlash = normalized.lastIndexOf('/');
if (lastSlash >= 0) {
const dirParts = parts.slice(0, -1);
const fileName = parts[parts.length - 1];
const ext = fileName.substring(fileName.lastIndexOf('.'));
for (let j = dirParts.length - 1; j >= 0; j--) {
const dirSuffix = dirParts.slice(j).join('/');
const key = `${dirSuffix}:${ext}`;
let list = dirMap.get(key);
if (!list) {
list = [];
dirMap.set(key, list);
}
list.push(original);
}
}
}
return dirMap;
}
/** The two suffix questions, however they happen to be answered. */
type SuffixAnswerer = Pick<SuffixIndex, 'get' | 'getInsensitive'>;
/**
* The two suffix maps of `buildSuffixIndex` exactly as they stood before this
* change: ONE fused traversal writing both, suffixes cut with `split('/')` plus
* `slice(j).join('/')`, first spelling winning in each map independently. This
* is the specification both the deferred exact map and the derived case-folded
* map are measured against — and it is also the reference for the suffix-
* cutting rewrite that came with them (the production loop now walks slash
* offsets and slices the original string instead of re-joining parts).
*/
function eagerSuffixMaps(normalizedFileList: string[], allFileList: string[]): SuffixAnswerer {
const exactMap = new Map<string, string>();
const lowerMap = new Map<string, string>();
for (let i = 0; i < normalizedFileList.length; i++) {
const normalized = normalizedFileList[i];
const original = allFileList[i];
const parts = normalized.split('/');
for (let j = parts.length - 1; j >= 0; j--) {
const suffix = parts.slice(j).join('/');
// Only store first match (longest path wins for ambiguous suffixes)
if (!exactMap.has(suffix)) {
exactMap.set(suffix, original);
}
const lower = suffix.toLowerCase();
if (!lowerMap.has(lower)) {
lowerMap.set(lower, original);
}
}
}
return {
get: (suffix: string) => exactMap.get(suffix),
getInsensitive: (suffix: string) => lowerMap.get(suffix.toLowerCase()),
};
}
// ─── corpus ──────────────────────────────────────────────────────────────────
/**
* Raw paths, in index order. Each entry is here for a reason the parity arms
* would not otherwise reach:
*
* - `src/com/{example,other}` — the same basename under two directories that
* share a parent, so `com/example` and `example` must select differently;
* - `app/Models/Legacy/User.php` — a file one level DEEPER than the bucket
* under test, which must not appear in `Models`'s bucket (the map is keyed
* on directory SUFFIX, not prefix);
* - `Makefile` — repo root, no directory at all, and no extension: skipped
* entirely by `dirMap`'s `lastSlash >= 0` guard, while the suffix maps still
* hold it under its whole-path key, which is the one key the slash walk
* cannot emit;
* - `scripts/build` — no extension, but IN a directory. `lastIndexOf('.')` is
* -1 and `substring(-1)` clamps to 0, so the extension is the whole
* filename and the key is `scripts:build`. Odd, long-standing, and pinned
* here so deferring the build cannot quietly "fix" it;
* - `lib/vendor.min.js` before `lib/vendor.js` — multiple dots (extension is
* the LAST one), and a two-entry bucket whose order is not alphabetical, so
* an implementation that sorted or reversed would be caught;
* - `win\pkg\Thing.cs` — a backslash path, so the arms cover the raw-vs-
* normalized split: keys come off the NORMALIZED path, values are the
* ORIGINAL one.
*/
const RAW_FILES: readonly string[] = [
'src/com/example/Foo.java',
'src/com/example/Bar.java',
'src/com/other/Foo.java',
'app/Models/User.php',
'app/Models/Post.php',
'app/Models/Legacy/User.php',
'Makefile',
'scripts/build',
'scripts/deploy.sh',
'lib/vendor.min.js',
'lib/vendor.js',
'a/b/c/d.ts',
'b/c/d.ts',
'win\\pkg\\Thing.cs',
];
const ALL_FILES: string[] = [...RAW_FILES];
const NORMALIZED_FILES: string[] = ALL_FILES.map((f) => f.replace(/\\/g, '/'));
/**
* Two paths that differ only in case, plus a three-way collision. Nothing about
* the exact map is exercised here; the point is the case-folded one, where all
* three spellings collapse to a single key and only ONE file can answer it. The
* file that does is decided by insertion order, so this corpus is what makes
* "the derived map has the same insertion order as a freshly built one" an
* observable claim rather than an internal one.
*/
const CASE_TWIN_FILES: readonly string[] = [
'src/Util/Helper.php',
'src/util/helper.php',
'app/README.md',
'app/ReadMe.md',
'app/readme.md',
'lib/Model/User.php',
'lib/model/USER.PHP',
];
/**
* Paths whose `.toLowerCase()` is not a per-character mapping.
*
* `Σ` folds to `ς` at the end of a word and to `σ` elsewhere, and JS applies
* that context rule: `'ΟΔΟΣ/x'` folds the sigma to `ς` (a slash is not a cased
* letter, so the word ends) while `'ΟΔΟΣ.ts'` folds it to `σ` (`t` is). So
* `src/ΟΔΟΣ/ΟΔΟΣ.ts` contributes the same segment spelling under two DIFFERENT
* folded keys. A derivation that folded the whole path once and sliced the
* result, or that folded per character, would answer differently here.
* `İstanbul` (one code point, two after folding) and `Gruß`/`GRUSS` (which do
* NOT collide under `toLowerCase`, unlike under full case folding) pin the two
* other classic hazards.
*/
const UNICODE_FOLDING_FILES: readonly string[] = [
'src/ΟΔΟΣ/Καλημέρα.ts',
'src/ΟΔΟΣ.ts',
'src/ΟΔΟΣ/ΟΔΟΣ.ts',
'i18n/İstanbul/Page.tsx',
'de/STRASSE/Gruß.ts',
'de/strasse/GRUSS.ts',
];
/**
* Slash spellings where cutting a suffix by slash offsets could disagree with
* `split('/')` + `join('/')`: a leading slash (where the walk stops early and
* the whole-path key is left to the write after the loop), a doubled slash (an
* empty segment mid-path), a trailing slash (an empty final segment), and a
* path with no slash at all, whose only key is that final write.
*/
const ODD_SLASH_FILES: readonly string[] = ['/root.ts', 'a//b/c.ts', 'dir/trailing/', 'plain.ts'];
interface Corpus {
readonly name: string;
readonly raw: readonly string[];
}
const PARITY_CORPORA: readonly Corpus[] = [
{ name: 'base', raw: RAW_FILES },
{ name: 'case-colliding twins', raw: CASE_TWIN_FILES },
{ name: 'unicode folding', raw: UNICODE_FOLDING_FILES },
{ name: 'slash oddities', raw: ODD_SLASH_FILES },
];
function corpusLists(raw: readonly string[]): { all: string[]; normalized: string[] } {
const all = [...raw];
return { all, normalized: all.map((f) => f.replace(/\\/g, '/')) };
}
// ─── probe spaces ────────────────────────────────────────────────────────────
/** Every directory suffix the corpus can produce, in first-seen order. */
function corpusDirSuffixes(normalized: readonly string[]): string[] {
const suffixes: string[] = [];
for (const path of normalized) {
const parts = path.split('/');
const dirParts = parts.slice(0, -1);
for (let j = dirParts.length - 1; j >= 0; j--) {
const suffix = dirParts.slice(j).join('/');
if (!suffixes.includes(suffix)) suffixes.push(suffix);
}
}
return suffixes;
}
/** Every extension the corpus can produce, in first-seen order. */
function corpusExtensions(normalized: readonly string[]): string[] {
const extensions: string[] = [];
for (const path of normalized) {
const fileName = path.slice(path.lastIndexOf('/') + 1);
const ext = fileName.substring(fileName.lastIndexOf('.'));
if (!extensions.includes(ext)) extensions.push(ext);
}
return extensions;
}
/**
* The full `getFilesInDir` probe space: every directory suffix crossed with
* every extension — so the parity arm asserts the misses too, not only the 19
* populated keys — plus spellings that exist nowhere in the corpus at all.
*/
function dirProbeSpace(normalized: readonly string[]): Array<readonly [string, string]> {
const probes: Array<readonly [string, string]> = [];
for (const dirSuffix of corpusDirSuffixes(normalized)) {
for (const ext of corpusExtensions(normalized)) probes.push([dirSuffix, ext]);
}
// Absent entirely: a directory PREFIX (`src`, which no file sits directly
// in), a case variant (the map is case-sensitive, unlike `getInsensitive`), a
// trailing-slash spelling, and a bare miss.
for (const dirSuffix of ['src', 'app', 'models', 'Models/', 'nope']) {
for (const ext of ['.php', '.java', '.nope', '']) probes.push([dirSuffix, ext]);
}
return probes;
}
function probeAllDirs(
probes: ReadonlyArray<readonly [string, string]>,
lookup: (dirSuffix: string, extension: string) => readonly string[],
): Record<string, readonly string[]> {
const answers: Record<string, readonly string[]> = {};
for (const [dirSuffix, ext] of probes)
answers[`${dirSuffix}\u0000${ext}`] = lookup(dirSuffix, ext);
return answers;
}
/**
* The full suffix probe space: every key either suffix map can hold — every
* suffix of every path, which is exactly what the build loops insert — each in
* its own spelling plus its lowercased and uppercased forms, so the folded
* lookups are driven with queries that hit, miss and collide. Plus spellings
* absent from every corpus.
*/
function suffixProbeSpace(normalized: readonly string[]): string[] {
const probes: string[] = [];
const add = (probe: string): void => {
if (!probes.includes(probe)) probes.push(probe);
};
for (const path of normalized) {
const parts = path.split('/');
for (let j = parts.length - 1; j >= 0; j--) {
const suffix = parts.slice(j).join('/');
add(suffix);
add(suffix.toLowerCase());
add(suffix.toUpperCase());
}
}
for (const miss of ['', '/', 'nope.java', 'NOPE.JAVA', 'src', 'Foo', 'Foo.java/']) add(miss);
return probes;
}
/**
* Ask both suffix questions about every probe, `get` FIRST — so the exact map
* exists before the first `getInsensitive` and the folded map is DERIVED.
*/
function answersGetFirst(
probes: readonly string[],
answerer: SuffixAnswerer,
): Record<string, string | null> {
const answers: Record<string, string | null> = {};
for (const probe of probes) {
answers[`exact\u0000${probe}`] = answerer.get(probe) ?? null;
answers[`folded\u0000${probe}`] = answerer.getInsensitive(probe) ?? null;
}
return answers;
}
/**
* The same probes, `getInsensitive` FIRST — PHP's order, where the folded map
* is built straight off the file list and the exact map is the later fallback.
* `undefined` is mapped to `null` in both collectors because `toEqual` treats
* an explicitly-undefined property as an absent one.
*/
function answersInsensitiveFirst(
probes: readonly string[],
answerer: SuffixAnswerer,
): Record<string, string | null> {
const answers: Record<string, string | null> = {};
for (const probe of probes) {
answers[`folded\u0000${probe}`] = answerer.getInsensitive(probe) ?? null;
answers[`exact\u0000${probe}`] = answerer.get(probe) ?? null;
}
return answers;
}
/** Every probe of every corpus, one flat table, so one `toEqual` covers all four. */
function answersAcrossCorpora(
collect: (probes: readonly string[], answerer: SuffixAnswerer) => Record<string, string | null>,
build: (normalized: string[], all: string[]) => SuffixAnswerer,
): Record<string, string | null> {
const answers: Record<string, string | null> = {};
for (const corpus of PARITY_CORPORA) {
const { all, normalized } = corpusLists(corpus.raw);
const collected = collect(suffixProbeSpace(normalized), build(normalized, all));
for (const [key, value] of Object.entries(collected)) {
answers[`${corpus.name}\u0000${key}`] = value;
}
}
return answers;
}
// ─── read-counting file lists ────────────────────────────────────────────────
interface CountingList {
/** A real `string[]`, so `buildSuffixIndex` takes it unmodified. */
readonly list: string[];
/** Element reads so far. Every build pass reads each element once. */
reads: () => number;
}
/**
* A `string[]` whose elements are accessor properties, so every `list[i]` is
* counted. An accessor on a real array rather than a `Proxy` keeps the value a
* genuine `Array` — `.length` and every array method behave normally — and
* counts only indexed reads, never `.length`.
*/
function countingList(paths: readonly string[]): CountingList {
let reads = 0;
const list = new Array<string>(paths.length);
paths.forEach((value, i) => {
Object.defineProperty(list, i, {
enumerable: true,
configurable: true,
get: () => {
reads += 1;
return value;
},
});
});
return { list, reads: () => reads };
}
/** One full pass over the file list reads every element of both arrays once. */
const ONE_PASS = RAW_FILES.length;
/** Lookups driven bare before a count is read — the count must not move. */
const LOOKUP_REPEATS = 20;
/** `import-resolvers/pass-cache.ts` builds exactly this: lowercased, not slash-normalized. */
const LOWERCASED_FILES: string[] = ALL_FILES.map((f) => f.toLowerCase());
// ─── parity: the directory map ───────────────────────────────────────────────
describe('buildSuffixIndex.getFilesInDir — the deferred build is behaviour-identical', () => {
it('answers the full probe space exactly as the eager implementation did', () => {
const index = buildSuffixIndex(NORMALIZED_FILES, ALL_FILES);
const reference = eagerDirMap(NORMALIZED_FILES, ALL_FILES);
const probes = dirProbeSpace(NORMALIZED_FILES);
const deferred = probeAllDirs(probes, (dir, ext) => index.getFilesInDir(dir, ext));
const eager = probeAllDirs(probes, (dir, ext) => reference.get(`${dir}:${ext}`) ?? []);
// A guard on the instrument: an empty or collapsed probe space would make
// the comparison below vacuous.
expect(probes.length).toBe(164);
expect(Object.values(eager).filter((files) => files.length > 0).length).toBe(19);
expect(deferred).toEqual(eager);
});
it('pins each bucket and its order outright, not only against the old code', () => {
const index = buildSuffixIndex(NORMALIZED_FILES, ALL_FILES);
expect({
// Same basename under sibling directories: the deeper suffix disambiguates.
'example:.java': index.getFilesInDir('example', '.java'),
'com/example:.java': index.getFilesInDir('com/example', '.java'),
'src/com/example:.java': index.getFilesInDir('src/com/example', '.java'),
'other:.java': index.getFilesInDir('other', '.java'),
// `Legacy/User.php` is one level deeper and belongs to `Legacy`, not `Models`.
'Models:.php': index.getFilesInDir('Models', '.php'),
'Legacy:.php': index.getFilesInDir('Legacy', '.php'),
'Models/Legacy:.php': index.getFilesInDir('Models/Legacy', '.php'),
// Multiple dots: the extension is the LAST one, and the bucket keeps
// index order (`vendor.min.js` was indexed first) rather than sorting.
'lib:.js': index.getFilesInDir('lib', '.js'),
// No extension at all: `substring(-1)` clamps to 0, so the "extension"
// is the whole filename.
'scripts:build': index.getFilesInDir('scripts', 'build'),
'scripts:.sh': index.getFilesInDir('scripts', '.sh'),
// Keyed on the normalized path, holding the ORIGINAL raw one.
'pkg:.cs': index.getFilesInDir('pkg', '.cs'),
'win/pkg:.cs': index.getFilesInDir('win/pkg', '.cs'),
// Two files in same-named leaf directories at different depths.
'c:.ts': index.getFilesInDir('c', '.ts'),
'b/c:.ts': index.getFilesInDir('b/c', '.ts'),
'a/b/c:.ts': index.getFilesInDir('a/b/c', '.ts'),
// Misses: a repo-root file is in no bucket; `src` is a PREFIX, never a
// directory suffix any file sits directly in; the key is case-sensitive.
':': index.getFilesInDir('', ''),
'src:.java': index.getFilesInDir('src', '.java'),
'models:.php': index.getFilesInDir('models', '.php'),
'Models:.java': index.getFilesInDir('Models', '.java'),
}).toEqual({
'example:.java': ['src/com/example/Foo.java', 'src/com/example/Bar.java'],
'com/example:.java': ['src/com/example/Foo.java', 'src/com/example/Bar.java'],
'src/com/example:.java': ['src/com/example/Foo.java', 'src/com/example/Bar.java'],
'other:.java': ['src/com/other/Foo.java'],
'Models:.php': ['app/Models/User.php', 'app/Models/Post.php'],
'Legacy:.php': ['app/Models/Legacy/User.php'],
'Models/Legacy:.php': ['app/Models/Legacy/User.php'],
'lib:.js': ['lib/vendor.min.js', 'lib/vendor.js'],
'scripts:build': ['scripts/build'],
'scripts:.sh': ['scripts/deploy.sh'],
'pkg:.cs': ['win\\pkg\\Thing.cs'],
'win/pkg:.cs': ['win\\pkg\\Thing.cs'],
'c:.ts': ['a/b/c/d.ts', 'b/c/d.ts'],
'b/c:.ts': ['a/b/c/d.ts', 'b/c/d.ts'],
'a/b/c:.ts': ['a/b/c/d.ts'],
':': [],
'src:.java': [],
'models:.php': [],
'Models:.java': [],
});
});
it('returns the same answers whether or not suffix lookups came first', () => {
const warmed = buildSuffixIndex(NORMALIZED_FILES, ALL_FILES);
warmed.get('Foo.java');
warmed.getInsensitive('USER.PHP');
warmed.getFilesInDir('nope', '.nope');
const cold = buildSuffixIndex(NORMALIZED_FILES, ALL_FILES);
const probes = dirProbeSpace(NORMALIZED_FILES);
expect(probeAllDirs(probes, (d, e) => warmed.getFilesInDir(d, e))).toEqual(
probeAllDirs(probes, (d, e) => cold.getFilesInDir(d, e)),
);
});
it('leaves the suffix answers untouched — building the dir map moves nothing', () => {
const index = buildSuffixIndex(NORMALIZED_FILES, ALL_FILES);
index.getFilesInDir('Models', '.php');
expect({
exact: index.get('example/Foo.java'),
// First path wins for an ambiguous suffix.
ambiguous: index.get('Foo.java'),
insensitive: index.getInsensitive('APP/MODELS/USER.PHP'),
// The suffix maps are built off the normalized path and return the raw one.
backslash: index.get('pkg/Thing.cs'),
miss: index.get('nope.java'),
}).toEqual({
exact: 'src/com/example/Foo.java',
ambiguous: 'src/com/example/Foo.java',
insensitive: 'app/Models/User.php',
backslash: 'win\\pkg\\Thing.cs',
miss: undefined,
});
});
});
// ─── parity: the derived case-folded map ─────────────────────────────────────
describe('buildSuffixIndex suffix maps — derived answers are the eager answers', () => {
it('answers the full suffix key space as the eager fused loop did, in both build orders', () => {
const eager = answersAcrossCorpora(answersGetFirst, eagerSuffixMaps);
// `get` first: the folded map is DERIVED from the exact one.
const derived = answersAcrossCorpora(answersGetFirst, (normalized, all) =>
buildSuffixIndex(normalized, all),
);
// `getInsensitive` first: the folded map is built straight off the list,
// and the exact map is the fallback traversal behind it.
const direct = answersAcrossCorpora(answersInsensitiveFirst, (normalized, all) =>
buildSuffixIndex(normalized, all),
);
// Guards on the instrument: a collapsed probe space, or a reference that
// answered nothing, would make both comparisons vacuous.
expect(Object.keys(eager).length).toBe(430);
expect(Object.values(eager).filter((file) => file !== null).length).toBe(268);
expect(derived).toEqual(eager);
expect(direct).toEqual(eager);
});
it('pins first-in-file-order for case-folded collisions, derived or built direct', () => {
const { all, normalized } = corpusLists(CASE_TWIN_FILES);
const derived = buildSuffixIndex(normalized, all);
// Exact map first, so the folded map below is derived rather than built.
const derivedExact = derived.get('Util/Helper.php');
const direct = buildSuffixIndex(normalized, all);
expect({
derivedExact,
// Two spellings of one path; the FIRST indexed answers both queries and
// `src/util/helper.php` answers neither. Insertion order is the only
// thing that decides this, and it must survive the derivation.
derivedTwin: derived.getInsensitive('HELPER.PHP'),
directTwin: direct.getInsensitive('HELPER.PHP'),
derivedTwinPath: derived.getInsensitive('SRC/UTIL/HELPER.PHP'),
directTwinPath: direct.getInsensitive('SRC/UTIL/HELPER.PHP'),
// Three spellings collapse to one folded key: the first still wins.
derivedThreeWay: derived.getInsensitive('app/readme.md'),
directThreeWay: direct.getInsensitive('app/readme.md'),
// The exact map keeps them apart; only the folded one collapses.
derivedUpper: derived.get('Model/User.php'),
derivedLower: derived.get('model/USER.PHP'),
// `get` after `getInsensitive` is the fallback order: a second traversal,
// the same answers.
directUpper: direct.get('Model/User.php'),
directLower: direct.get('model/USER.PHP'),
directMiss: direct.get('model/User.php'),
}).toEqual({
derivedExact: 'src/Util/Helper.php',
derivedTwin: 'src/Util/Helper.php',
directTwin: 'src/Util/Helper.php',
derivedTwinPath: 'src/Util/Helper.php',
directTwinPath: 'src/Util/Helper.php',
derivedThreeWay: 'app/README.md',
directThreeWay: 'app/README.md',
derivedUpper: 'lib/Model/User.php',
derivedLower: 'lib/model/USER.PHP',
directUpper: 'lib/Model/User.php',
directLower: 'lib/model/USER.PHP',
directMiss: undefined,
});
});
it('pins the context-sensitive folds outright, derived or built direct', () => {
const { all, normalized } = corpusLists(UNICODE_FOLDING_FILES);
const derived = buildSuffixIndex(normalized, all);
const derivedExact = derived.get('ΟΔΟΣ.ts');
const direct = buildSuffixIndex(normalized, all);
expect({
derivedExact,
// `Σ` before `.ts` is not word-final, so it folds to `σ`...
derivedNonFinal: derived.getInsensitive('ΟΔΟΣ.ts'),
directNonFinal: direct.getInsensitive('ΟΔΟΣ.ts'),
// ...and the folded key really is spelled with `σ`, not `ς`.
derivedSigmaKey: derived.getInsensitive('οδοσ.ts'),
derivedFinalSigmaKey: derived.getInsensitive('οδος.ts'),
// Before a slash it IS word-final and folds to `ς` — the same segment
// spelling, a different key, from the same path.
derivedFinal: derived.getInsensitive('ΟΔΟΣ/ΟΔΟΣ.ts'),
directFinal: direct.getInsensitive('ΟΔΟΣ/ΟΔΟΣ.ts'),
derivedFinalTyped: derived.getInsensitive('οδος/οδοσ.ts'),
// One code point folding to two: `İ` -> `i` + U+0307.
derivedDotted: derived.getInsensitive('İSTANBUL/PAGE.TSX'),
directDotted: direct.getInsensitive('İstanbul/Page.tsx'),
// `ß` and `SS` are distinct under `toLowerCase`, unlike full case folding.
derivedSharpS: derived.getInsensitive('STRASSE/GRUß.TS'),
derivedDoubleS: derived.getInsensitive('STRASSE/GRUSS.TS'),
}).toEqual({
derivedExact: 'src/ΟΔΟΣ.ts',
derivedNonFinal: 'src/ΟΔΟΣ.ts',
directNonFinal: 'src/ΟΔΟΣ.ts',
derivedSigmaKey: 'src/ΟΔΟΣ.ts',
derivedFinalSigmaKey: undefined,
derivedFinal: 'src/ΟΔΟΣ/ΟΔΟΣ.ts',
directFinal: 'src/ΟΔΟΣ/ΟΔΟΣ.ts',
derivedFinalTyped: 'src/ΟΔΟΣ/ΟΔΟΣ.ts',
derivedDotted: 'i18n/İstanbul/Page.tsx',
directDotted: 'i18n/İstanbul/Page.tsx',
derivedSharpS: 'de/STRASSE/Gruß.ts',
derivedDoubleS: 'de/strasse/GRUSS.ts',
});
});
});
// ─── laziness ────────────────────────────────────────────────────────────────
describe('buildSuffixIndex — nothing is built at construction, each map once on first use', () => {
it('reads the file list zero times at construction, and one pass for all suffix lookups', () => {
const normalized = countingList(NORMALIZED_FILES);
const all = countingList(ALL_FILES);
const index = buildSuffixIndex(normalized.list, all.list);
const afterBuild = { normalized: normalized.reads(), all: all.reads() };
index.get('Foo.java');
index.get('nope.java');
index.getInsensitive('APP/MODELS/USER.PHP');
index.getInsensitive('nope.java');
expect({
afterBuild,
afterSuffixLookups: { normalized: normalized.reads(), all: all.reads() },
// A count of zero is equally the count of an index that answers nothing.
answer: index.get('Foo.java'),
folded: index.getInsensitive('APP/MODELS/USER.PHP'),
}).toEqual({
afterBuild: { normalized: 0, all: 0 },
afterSuffixLookups: { normalized: ONE_PASS, all: ONE_PASS },
answer: 'src/com/example/Foo.java',
folded: 'app/Models/User.php',
});
});
it('builds ONE map for a get-only consumer — Java never pays for the folded map', () => {
const normalized = countingList(NORMALIZED_FILES);
const all = countingList(ALL_FILES);
const index = buildSuffixIndex(normalized.list, all.list);
const atConstruction = { normalized: normalized.reads(), all: all.reads() };
// Driven bare: asserting inside the loop restates one bit twenty times.
for (let i = 0; i < LOOKUP_REPEATS; i++) {
index.get('Foo.java');
index.get('example/Foo.java');
index.get('nope.java');
}
expect({
// A second map built eagerly BESIDE the exact one shows up HERE — fused
// into the same loop, as it used to be, or in a loop of its own. Fused,
// the total below does not move at all, so this field is the only thing
// that catches it.
atConstruction,
afterManyGets: { normalized: normalized.reads(), all: all.reads() },
exact: index.get('example/Foo.java'),
ambiguous: index.get('Foo.java'),
miss: index.get('nope.java'),
}).toEqual({
atConstruction: { normalized: 0, all: 0 },
afterManyGets: { normalized: ONE_PASS, all: ONE_PASS },
exact: 'src/com/example/Foo.java',
ambiguous: 'src/com/example/Foo.java',
miss: undefined,
});
});
it('builds ONE map for a getInsensitive-only consumer — PHP never pays for the exact map', () => {
const normalized = countingList(NORMALIZED_FILES);
const all = countingList(ALL_FILES);
const index = buildSuffixIndex(normalized.list, all.list);
const atConstruction = { normalized: normalized.reads(), all: all.reads() };
for (let i = 0; i < LOOKUP_REPEATS; i++) {
index.getInsensitive('USER.PHP');
index.getInsensitive('APP/MODELS/USER.PHP');
index.getInsensitive('NOPE.PHP');
}
expect({
atConstruction,
afterManyLookups: { normalized: normalized.reads(), all: all.reads() },
basename: index.getInsensitive('USER.PHP'),
path: index.getInsensitive('APP/MODELS/USER.PHP'),
miss: index.getInsensitive('NOPE.PHP'),
}).toEqual({
atConstruction: { normalized: 0, all: 0 },
afterManyLookups: { normalized: ONE_PASS, all: ONE_PASS },
basename: 'app/Models/User.php',
path: 'app/Models/User.php',
miss: undefined,
});
});
it('derives the folded map from the exact one — the second question costs no pass', () => {
const normalized = countingList(NORMALIZED_FILES);
const all = countingList(ALL_FILES);
const index = buildSuffixIndex(normalized.list, all.list);
const exact = index.get('Foo.java');
const afterGet = { normalized: normalized.reads(), all: all.reads() };
for (let i = 0; i < LOOKUP_REPEATS; i++) {
index.getInsensitive('FOO.JAVA');
index.getInsensitive('APP/MODELS/USER.PHP');
index.getInsensitive('NOPE.JAVA');
}
expect({
afterGet,
// The derivation walks the exact map's keys, never the file list, so this
// must not move. A second full traversal would read a second pass.
afterDerivation: { normalized: normalized.reads(), all: all.reads() },
exact,
folded: index.getInsensitive('FOO.JAVA'),
foldedPath: index.getInsensitive('APP/MODELS/USER.PHP'),
foldedMiss: index.getInsensitive('NOPE.JAVA'),
}).toEqual({
afterGet: { normalized: ONE_PASS, all: ONE_PASS },
afterDerivation: { normalized: ONE_PASS, all: ONE_PASS },
exact: 'src/com/example/Foo.java',
folded: 'src/com/example/Foo.java',
foldedPath: 'app/Models/User.php',
foldedMiss: undefined,
});
});
it('pays the second traversal only in the order no consumer uses — folded, then exact', () => {
const normalized = countingList(NORMALIZED_FILES);
const all = countingList(ALL_FILES);
const index = buildSuffixIndex(normalized.list, all.list);
const folded = index.getInsensitive('FOO.JAVA');
const afterInsensitive = { normalized: normalized.reads(), all: all.reads() };
for (let i = 0; i < LOOKUP_REPEATS; i++) index.get('Foo.java');
expect({
afterInsensitive,
// There is nothing to derive an exact map FROM, so this order costs the
// traversal the other one saves. Documented, unused, and still correct.
afterFallback: { normalized: normalized.reads(), all: all.reads() },
folded,
exact: index.get('Foo.java'),
// Case-sensitive again, which is the point of the fallback being a real
// second map rather than an alias of the folded one.
exactMiss: index.get('FOO.JAVA'),
}).toEqual({
afterInsensitive: { normalized: ONE_PASS, all: ONE_PASS },
afterFallback: { normalized: ONE_PASS * 2, all: ONE_PASS * 2 },
folded: 'src/com/example/Foo.java',
exact: 'src/com/example/Foo.java',
exactMiss: undefined,
});
});
it('takes exactly one more pass on the first getFilesInDir, and none after', () => {
const normalized = countingList(NORMALIZED_FILES);
const all = countingList(ALL_FILES);
const index = buildSuffixIndex(normalized.list, all.list);
const beforeFirst = { normalized: normalized.reads(), all: all.reads() };
index.getFilesInDir('Models', '.php');
const afterFirst = { normalized: normalized.reads(), all: all.reads() };
// Hits, misses and repeats alike: memoizing the DECISION instead of the
// MAP would rebuild on every one of these and the count would climb.
index.getFilesInDir('Models', '.php');
index.getFilesInDir('example', '.java');
index.getFilesInDir('nope', '.nope');
index.getFilesInDir('nope', '.nope');
index.getFilesInDir('', '');
expect({
beforeFirst,
afterFirst,
afterMany: { normalized: normalized.reads(), all: all.reads() },
answer: index.getFilesInDir('Models', '.php'),
}).toEqual({
beforeFirst: { normalized: 0, all: 0 },
afterFirst: { normalized: ONE_PASS, all: ONE_PASS },
afterMany: { normalized: ONE_PASS, all: ONE_PASS },
answer: ['app/Models/User.php', 'app/Models/Post.php'],
});
});
it('defers per index, not per module — a second index starts cold', () => {
const firstNormalized = countingList(NORMALIZED_FILES);
const firstAll = countingList(ALL_FILES);
const first = buildSuffixIndex(firstNormalized.list, firstAll.list);
first.getFilesInDir('Models', '.php');
const secondNormalized = countingList(NORMALIZED_FILES);
const secondAll = countingList(ALL_FILES);
const second = buildSuffixIndex(secondNormalized.list, secondAll.list);
expect({
first: { normalized: firstNormalized.reads(), all: firstAll.reads() },
// Cold even though a fully built index of the same paths exists: the maps
// hang off the closure, not off the module.
second: { normalized: secondNormalized.reads(), all: secondAll.reads() },
// Pairing rule: a count of zero must not be the count of an index that
// answers nothing. The second index still resolves once asked.
secondAnswer: second.getFilesInDir('Models', '.php'),
secondReadsAfterAsking: secondNormalized.reads(),
}).toEqual({
first: { normalized: ONE_PASS, all: ONE_PASS },
second: { normalized: 0, all: 0 },
secondAnswer: ['app/Models/User.php', 'app/Models/Post.php'],
secondReadsAfterAsking: ONE_PASS,
});
});
it('aliases one map for both questions when the caller pre-lowercased the list', () => {
// `pass-cache.ts` (TypeScript, JavaScript, Vue) passes a lowercased list and
// says so, and over such a list the derivation is the identity — so the
// folded map is the exact map, not a copy of it, in either asking order.
const getFirstNormalized = countingList(LOWERCASED_FILES);
const getFirstAll = countingList(ALL_FILES);
const getFirst = buildSuffixIndex(getFirstNormalized.list, getFirstAll.list, {
alreadyLowercased: true,
});
const getFirstExact = getFirst.get('app/models/user.php');
const getFirstFolded = getFirst.getInsensitive('APP/MODELS/USER.PHP');
const foldedFirstNormalized = countingList(LOWERCASED_FILES);
const foldedFirstAll = countingList(ALL_FILES);
const foldedFirst = buildSuffixIndex(foldedFirstNormalized.list, foldedFirstAll.list, {
alreadyLowercased: true,
});
const foldedFirstFolded = foldedFirst.getInsensitive('APP/MODELS/USER.PHP');
const foldedFirstExact = foldedFirst.get('app/models/user.php');
expect({
getFirstReads: { normalized: getFirstNormalized.reads(), all: getFirstAll.reads() },
foldedFirstReads: { normalized: foldedFirstNormalized.reads(), all: foldedFirstAll.reads() },
getFirstExact,
getFirstFolded,
foldedFirstFolded,
foldedFirstExact,
// Values are the ORIGINAL paths; only the keys were lowercased.
backslash: getFirst.getInsensitive('PKG/THING.CS'),
}).toEqual({
getFirstReads: { normalized: ONE_PASS, all: ONE_PASS },
foldedFirstReads: { normalized: ONE_PASS, all: ONE_PASS },
getFirstExact: 'app/Models/User.php',
getFirstFolded: 'app/Models/User.php',
foldedFirstFolded: 'app/Models/User.php',
foldedFirstExact: 'app/Models/User.php',
backslash: undefined,
});
});
});

338
gitnexus/test/unit/scope-resolution/cobol-import-target-parity.test.ts Normal file
View file

@ -0,0 +1,338 @@
/**
* Differential harness for the COBOL `COPY`-target index hoist (#2908).
*
* `cobolScopeResolver.resolveImportTarget` used to answer every `COPY` with TWO
* full `allFilePaths` scans — copybooks first, then COBOL sources — each calling
* `path.extname` + `path.basename` + `toUpperCase` on every entry, so resolution
* cost O(copies × files) and a `COPY` of a member that is not in the repo (the
* common case) ran both scans to completion. Replacing them with a per-run
* two-tier index is a pure performance change ONLY if every implicit tie-break
* survives, and none of them is visible to the type system:
*
* - TIER ORDER: a `.cpy`/`.copybook` hit beats a `.cbl`/`.cob`/`.cobol` hit
* even when the source file comes FIRST in Set-iteration order. Collapsing
* the two tiers into one first-wins map is the "obvious" rewrite and it
* silently inverts this;
* - WITHIN A TIER: the first file in Set-iteration order wins, because the
* scans returned on first match;
* - CASE: the extension is compared LOWER-cased while the basename is
* compared UPPER-cased, and `path.basename(fp, ext)` strips the suffix only
* on an exact, case-sensitive match — so `Foo.CPY` is indexed under
* `FOO.CPY`, not `FOO`, and is unreachable by a `COPY FOO`;
* - `path` SEMANTICS: Node's `path.extname`/`path.basename` are what decide
* where the stem starts, and on POSIX they do not treat `\` as a separator.
* Hand-rolled slicing on `/` would start resolving backslash paths that
* previously returned null.
*
* So this file keeps a VERBATIM copy of the pre-change resolver body
* (`git show HEAD~:…/languages/cobol/scope-resolver.ts`) and asserts the new
* implementation agrees with it on a deterministic generated corpus plus a
* hand-built layout per tie-break. The copy is the specification; if an arm here
* fails, the resolver's OUTPUT moved and COBOL's IMPORTS edges move with it.
*
* Mutation-tested against the new implementation — each of these was inserted,
* confirmed RED here, and reverted: tiers collapsed into one map; within-tier
* first-wins flipped to last-wins; `targetRaw.toUpperCase()` dropped;
* `path.extname(fp).toLowerCase()` left un-lowercased.
*
* This file calls the resolver directly, which for COBOL is also the
* orchestrator adapter — but the arms below say nothing about the Set being
* passed THROUGH, and a defensive `new Set(allFilePaths)` would leave them all
* green while restoring the per-import rebuild. That failure is guarded by
* `test/integration/cobol-import-index-reuse.test.ts`.
*/
import path from 'node:path';
import { describe, expect, it } from 'vitest';
import { cobolScopeResolver } from '../../../src/core/ingestion/languages/cobol/scope-resolver.js';
const { resolveImportTarget } = cobolScopeResolver;
/** COBOL takes no `resolutionConfig` and ignores `fromFile`; both are pinned. */
const FROM_FILE = 'src/PROG.cbl';
function resolve(targetRaw: string, files: ReadonlySet<string>): string | readonly string[] | null {
return resolveImportTarget(targetRaw, FROM_FILE, files, undefined);
}
// ─── verbatim pre-change implementation ──────────────────────────────────────
const LEGACY_COPYBOOK_EXTENSIONS = new Set(['.cpy', '.copybook']);
function legacyResolveCobolImportTarget(
targetRaw: string,
allFilePaths: ReadonlySet<string>,
): string | null {
const upper = targetRaw.toUpperCase();
// Check copybook files first
for (const fp of allFilePaths) {
const ext = path.extname(fp).toLowerCase();
if (!LEGACY_COPYBOOK_EXTENSIONS.has(ext)) continue;
const basename = path.basename(fp, ext).toUpperCase();
if (basename === upper) return fp;
}
// Also search COBOL source files (.cbl, .cob, .cobol)
const COBOL_SOURCE_EXTS = new Set(['.cbl', '.cob', '.cobol']);
for (const fp of allFilePaths) {
const ext = path.extname(fp).toLowerCase();
if (!COBOL_SOURCE_EXTS.has(ext)) continue;
const basename = path.basename(fp, ext).toUpperCase();
if (basename === upper) return fp;
}
return null;
}
// ─── deterministic corpus ────────────────────────────────────────────────────
/** Murmur3 finalizer — a reproducible stand-in for `Math.random()`. */
function mix(n: number): number {
let x = n >>> 0;
x = Math.imul(x ^ (x >>> 16), 0x85ebca6b) >>> 0;
x = Math.imul(x ^ (x >>> 13), 0xc2b2ae35) >>> 0;
return (x ^ (x >>> 16)) >>> 0;
}
/**
* Directory shapes of a typical mainframe checkout, including one whose
* segments are separated by BACKSLASHES — on POSIX that is one long filename,
* which is precisely the `path` semantic the index must not "simplify" away.
*/
const DIRS = [
'',
'copybooks',
'COPYBOOKS',
'src',
'src/copy',
'legacy/cpy',
'jcl/proclib',
'win\\dir',
];
/** Member names in the case mixture a real repo has. */
const STEMS = ['CUSTREC', 'custrec', 'AcctRec', 'PAYROLL', 'BOOK', 'COMMON', 'TAXCALC', 'ERRDEMO'];
/**
* Both tiers, both cases, plus two extensions in NEITHER tier: `.txt` (a
* non-COBOL file that must never answer a `COPY`) and `''` (a file with no
* extension at all, which `path.extname` reports as the empty string and which
* therefore falls out of both extension sets).
*/
const EXTS = ['.cpy', '.copybook', '.CPY', '.cbl', '.cob', '.cobol', '.CBL', '.txt', ''];
function corpus(seed: number, fileCount: number): Set<string> {
const files = new Set<string>();
for (let i = 0; i < fileCount; i++) {
const a = mix(seed * 7919 + i);
const b = mix(a ^ 0x9e3779b9);
const c = mix(b ^ 0x85ebca6b);
const dir = DIRS[a % DIRS.length];
const stem = STEMS[b % STEMS.length];
const rel = `${stem}${EXTS[c % EXTS.length]}`;
files.add(dir === '' ? rel : `${dir}/${rel}`);
}
// Backslash-separated paths, which `path` reads differently per platform and
// a `/`-slicing rewrite would read differently from `path` on POSIX.
files.add('win\\dir\\BOOK.cpy');
files.add('win\\dir\\PAYROLL.cbl');
return files;
}
/**
* `COPY` operands as they appear in source, plus the spellings that reach the
* corpus's awkward files. Lower-case entries are what breaks if the target
* stops being upper-cased; the `.CPY`/`.CBL` suffixed entries are what reaches
* a file whose mixed-case extension `path.basename` refused to strip.
*/
const TARGETS = [
'',
'CUSTREC',
'custrec',
'CustRec',
'ACCTREC',
'PAYROLL',
'payroll',
'BOOK',
'COMMON',
'TAXCALC',
'ERRDEMO',
'MISSING',
'BOOK.CPY',
'CUSTREC.CPY',
'PAYROLL.CBL',
'win\\dir\\BOOK',
'WIN\\DIR\\BOOK',
'win/dir/BOOK',
];
const REPOS = 40;
describe('COBOL COPY-target index hoist — output parity with the pre-change scans (#2908)', () => {
it('agrees with the verbatim pre-change resolver over the generated corpus', () => {
let checked = 0;
for (let repo = 0; repo < REPOS; repo++) {
const files = corpus(repo, 6 + (repo % 25));
for (const target of TARGETS) {
expect(resolve(target, files), `cobol "${target}" repo=${repo}`).toEqual(
legacyResolveCobolImportTarget(target, files),
);
checked++;
}
}
expect(checked).toBe(REPOS * TARGETS.length);
});
it('the corpus actually resolves things (the parity arm is not vacuous)', () => {
// A corpus that resolved nothing would make the arm above pass on
// `null === null` forever. Measured on this corpus: 390 hits.
let hits = 0;
for (let repo = 0; repo < REPOS; repo++) {
const files = corpus(repo, 6 + (repo % 25));
for (const target of TARGETS) {
hits += legacyResolveCobolImportTarget(target, files) === null ? 0 : 1;
}
}
expect(hits).toBeGreaterThan(300);
});
});
// ─── hand-built tie-breaks ───────────────────────────────────────────────────
/**
* `path` decides where the stem of a backslash path starts: on POSIX the whole
* `dir\sub\BOOK` is the stem, on Windows only `BOOK`. Deriving the target
* through the SAME call keeps the two arms below a hit and a miss respectively
* on both platforms, so what they pin is the `path` semantics rather than the
* host — and a rewrite that replaces `path` with slicing on `/` changes what
* they resolve to on Windows.
*/
const BACKSLASH_FILE = 'dir\\sub\\BOOK.cpy';
const BACKSLASH_STEM = path.basename(BACKSLASH_FILE, '.cpy').toUpperCase();
/** The stem's last backslash-delimited segment — a HIT only where `path` splits on `\`. */
const BACKSLASH_LEAF = 'BOOK';
const BACKSLASH_LEAF_EXPECTED = BACKSLASH_STEM === BACKSLASH_LEAF ? BACKSLASH_FILE : null;
interface HandBuilt {
readonly why: string;
/** Insertion order IS the Set-iteration order, and for most arms it IS the tie-break. */
readonly files: readonly string[];
readonly target: string;
/** The one path (or `null`) both implementations must return. */
readonly expected: string | null;
}
const HANDBUILT: readonly HandBuilt[] = [
{
why: 'a copybook beats a COBOL source that comes FIRST in Set order (tier order)',
files: ['src/BOOK.cbl', 'copybooks/BOOK.cpy'],
target: 'BOOK',
expected: 'copybooks/BOOK.cpy',
},
{
why: '.copybook is tier 1 too, and beats an earlier .cob',
files: ['src/BOOK.cob', 'copybooks/BOOK.copybook'],
target: 'BOOK',
expected: 'copybooks/BOOK.copybook',
},
{
why: 'the source tier answers only when every copybook has missed',
files: ['copybooks/OTHER.cpy', 'src/BOOK.cbl'],
target: 'BOOK',
expected: 'src/BOOK.cbl',
},
{
why: 'within the copybook tier, first in Set order wins',
files: ['a/BOOK.cpy', 'b/BOOK.cpy'],
target: 'BOOK',
expected: 'a/BOOK.cpy',
},
{
why: 'within the source tier, first in Set order wins',
files: ['b/BOOK.cbl', 'a/BOOK.cob', 'c/BOOK.cobol'],
target: 'BOOK',
expected: 'b/BOOK.cbl',
},
{
why: 'the basename is compared UPPER-cased, so a lower-case file answers an upper-case COPY',
files: ['copybooks/custrec.cpy'],
target: 'CUSTREC',
expected: 'copybooks/custrec.cpy',
},
{
why: 'the TARGET is upper-cased too, so a lower-case COPY reaches an upper-case file',
files: ['copybooks/CUSTREC.cpy'],
target: 'custrec',
expected: 'copybooks/CUSTREC.cpy',
},
{
why: 'the extension is matched LOWER-cased, so `Foo.CPY` is a copybook at all',
files: ['copybooks/Foo.CPY'],
target: 'FOO.CPY',
expected: 'copybooks/Foo.CPY',
},
{
why: '`path.basename(fp, ext)` strips case-SENSITIVELY, so `Foo.CPY` is NOT reachable as FOO',
files: ['copybooks/Foo.CPY'],
target: 'FOO',
expected: null,
},
{
why: 'a `.CPY` file keyed with its suffix loses `BOOK` to a `.cbl` in the later tier',
files: ['x/BOOK.cbl', 'y/BOOK.CPY'],
target: 'BOOK',
expected: 'x/BOOK.cbl',
},
{
why: 'an uppercase source extension is a source file (`.CBL` → tier 2, keyed with its suffix)',
files: ['src/Pay.CBL'],
target: 'PAY.CBL',
expected: 'src/Pay.CBL',
},
{
why: 'a file with NO extension never answers a COPY',
files: ['copybooks/BOOK'],
target: 'BOOK',
expected: null,
},
{
why: 'a non-COBOL extension never answers a COPY',
files: ['copybooks/BOOK.txt', 'docs/BOOK.md'],
target: 'BOOK',
expected: null,
},
{
why: 'a target matching nothing resolves to null',
files: ['copybooks/BOOK.cpy', 'src/PROG.cbl'],
target: 'NOSUCHBOOK',
expected: null,
},
{
why: 'an empty target matches nothing (no file has an empty stem)',
files: ['copybooks/BOOK.cpy', 'src/PROG.cbl'],
target: '',
expected: null,
},
{
why: 'a backslash path is addressed by the stem `path` reports for it',
files: [BACKSLASH_FILE],
target: BACKSLASH_STEM,
expected: BACKSLASH_FILE,
},
{
why: 'its trailing segment is a hit only where `path` treats `\\` as a separator',
files: [BACKSLASH_FILE],
target: BACKSLASH_LEAF,
expected: BACKSLASH_LEAF_EXPECTED,
},
];
describe('COBOL COPY-target index hoist — hand-built tie-breaks (#2908)', () => {
it.each(HANDBUILT)('$why', ({ files, target, expected }) => {
const set = new Set(files);
// Two assertions, not one: agreeing with the legacy copy proves the hoist
// preserved the behaviour, and pinning the literal proves the behaviour
// being preserved is the one the case is named for — `toEqual(null)` on
// both sides would otherwise satisfy an arm that stopped resolving.
expect(legacyResolveCobolImportTarget(target, set), `legacy: ${target}`).toBe(expected);
expect(resolve(target, set), `new: ${target}`).toBe(expected);
});
});

10
gitnexus/test/unit/scope-resolution/import-target-index-parity.test.ts
View file

@ -30,9 +30,7 @@
* provably cannot see: a full workspace scan on 1-in-32 imports scores 1.458
* against a 1.8 scaling budget and 1.736 ms against a 4 ms ceiling — it passes
* everything — while this counter reads 14 instead of 1. Timing gates catch the
* constant factor; this catches the scan. Kotlin (#2872) is covered there too,
* because its own guard counts index BUILDS and a scan beside a reused index
* moves no build count.
* constant factor; this catches the scan. Kotlin (#2872) is covered there too.
*
* It is NOT the guard for PR #1918 review finding P1. That failure — a
* defensive `new Set(allFilePaths)` in the orchestrator ADAPTER, handing a fresh
@ -793,9 +791,9 @@ describe('import-target index hoist — built once per file set, not once per im
it('kotlin builds one index for many imports (#2872)', () => {
// Kotlin's own guard (`test/integration/kotlin-import-index-reuse.test.ts`)
// counts index BUILDS. That catches the per-import rebuild, but a scan added
// beside a reused index moves no build count — this arm sees it, because it
// counts iterations of the Set rather than cache misses.
// counts the same traversals one layer up, at the adapter. This arm covers
// the resolver function directly, so a rescan reintroduced inside
// `resolveKotlinImportTarget` fails here even if the adapter is untouched.
const files = countingCorpus(7, '.kt');
for (let i = 0; i < 200; i++) {
resolveKotlinImportTarget(

650
gitnexus/test/unit/scope-resolution/import-target-index-reuse.contract.test.ts Normal file
View file

@ -0,0 +1,650 @@
/**
* One property, asserted for EVERY registered `ScopeResolver` (#2909).
*
* Import-target resolution must not re-derive its per-pass workspace structures
* once per import. The per-language guards matching
* `test/integration/*-import-index-reuse.test.ts` say that once each, with their
* own corpus, their own expected traversal count and their own header — and
* there is one only for the languages someone wrote one for, never for the rest,
* because adding a resolver to `SCOPE_RESOLVERS` is two lines
* (`pipeline/registry.ts`), neither of which is a test. This file closes that
* gap without restating its size: the table below is keyed by
* `SupportedLanguages`, and the inventory arm diffs its keys against
* `SCOPE_RESOLVERS` so a registered resolver missing from it fails.
*
* ## Called the way the ORCHESTRATOR calls, with all five arguments
*
* `pipeline/run.ts` passes five: `(targetRaw, fromFile, allFilePaths,
* resolutionConfig, { parsedFiles, parsedImport })`. This file used to pass
* four, and a fifth argument that is never supplied is a channel that is never
* measured — `languages/php/import-target.ts` returns early on `context ===
* undefined`, so everything behind that guard was ungated for every language in
* the table. Replacing PHP's `perFileSet` with an identity wrapper, which
* rebuilds its `Map<dirAlias, ParsedFile[]>` per import at O(files × depth)
* (197.0 µs → 9976.2 µs per import at 8000 files, depth 6), left every arm of
* this file green. Both call sites below now pass a `context`, and the fixtures
* name their `parsedImport` explicitly so no adapter's use of the channel can
* hide behind an omission.
*
* ## Two counters, because there are two per-file-set KEYS
*
* `perFileSet` memoizes on object identity, and the orchestrator threads two
* stable objects per pass: the `allFilePaths` Set and the `parsedFiles` array.
* A `CountingSet` sees only the first, and the readers of the second touch the
* Set nowhere while reading it, so `scans` moves by ZERO for anything that goes
* wrong on that key — measured, with the identity-wrapper mutation above:
* `scans` 1 and 1, `parsedFileReads` 9 and 603. Hence `countedParsedFiles`
* (`test/helpers/counting-file-set.ts`), and hence the same comparison asserted
* twice, once per key.
*
* Two registered resolvers read `context` — PHP (`languages/php/scope-resolver.ts`
* → `resolvePhpImportTargetInternal`) and Python
* (`languages/python/scope-resolver.ts` → `pythonFileExportsName`). Every other
* adapter declares three or four parameters and cannot observe a fifth. Which
* ones those are is not a number to maintain here: the per-language
* `minimumParsedFileReads` floor in the table IS the record, and it is what a
* new reader has to change. Both languages that read this key memoize on it,
* and the two memos fail differently:
*
* - PHP: the `filesByDirectory` memo, `perFileSet`-keyed on the `parsedFiles`
* array. Defeat it and every import rebuilds a `Map<dirAlias,
* ParsedFile[]>`; the arm reads 603 against 9 (proven by mutation).
* - Python: `parsedFileByPath`, keyed the same way, behind
* `pythonFileExportsName`. It is built by the FIRST import whose package
* probe resolves — one pass over the array — and every later one is a
* `Map.get`, so the floor of 1 is that single build and the equality half
* is what proves it does not repeat. Before that memo the same call was a
* `parsedFiles.find` per resolving import, which is the shape #2901
* removed on the file-set key; this arm is why it cannot come back.
* - Every other language: `0 === 0`, recorded as a floor of 0. A new reader
* arrives with that floor already in place and is caught by the equality
* half, which needs no per-language knowledge at all.
*
* Out of reach from here, and stated so it is not mistaken for covered:
* `bench/import-target/measure.mjs` calls the resolvers with THREE arguments,
* so no timing arm in that harness enters the `context` leg either.
*
* ## The assertion is a COMPARISON, not a constant
*
* `scans(200) === scans(2)`, never `scans === 1`. Per-language counts legitimately
* differ — C# and Java each build two indexes over the same Set, TypeScript /
* JavaScript / Vue materialize an array and a copy behind their pass cache, Rust
* never traverses at all — and a table of expected constants would be one entry
* per language to get wrong. Comparing two counts against each other
* needs no per-language knowledge and states the actual property: the traversal
* count is a function of the FILE SET, not of the import count.
*
* ## Paired with non-vacuity, because the comparison alone is trivially true
*
* `scans(200) === scans(2)` holds perfectly for a resolver that returns `null`
* without ever touching the set — which is exactly what an adapter looks like
* after its context narrowing starts rejecting the workspace (`instanceof Set`
* for C# and Java, the `has`/iterator duck-type for Swift). So each case also
* asserts:
*
* - `hitTarget` still resolves to something. This is the same pairing rule the
* `test/integration/*-import-index-reuse.test.ts` guards state in their
* headers, and the reason `CountingSet` is a real `Set` subclass rather than
* a counter object.
* - the count clears `minimumScans`, which proves the counting Set is the
* object the resolver actually indexed rather than a copy made upstream.
*
* ## What it does NOT cover
*
* The fixtures are minimal by design — a handful of files and two import
* spellings per language, enough to reach the index and no more. Output parity,
* tie-breaks and iteration order are the subject of
* `import-target-index-parity.test.ts` and the per-language parity tests; the
* per-language integration guards carry the realistic corpora and the exact
* expected traversal counts. This file only answers "does the work stay flat in
* the number of imports", for every resolver `SCOPE_RESOLVERS` registers.
*
* Neither counter sees inside a structure once it has been built: a scan over
* `WorkspaceFileIndex.normalized`, or over a `ParsedFile[]` bucket in PHP's
* directory index, moves nothing (see `test/helpers/counting-file-set.ts`).
* That is not hypothetical — JavaScript's adapter had no suffix
* index at all until #2910, so every JavaScript import ran `suffixResolve`'s
* linear pass over the materialized `normalizedFileList` (6448.9 µs per import
* at 2000 files, against 25.0 µs for TypeScript), and the `javascript` case
* below scored a clean pass throughout: the pass cache WAS reused, so the
* traversal count read 2 either way. What catches that class of defect is a
* behaviour or call-count assertion, not a traversal count — see
* `test/integration/javascript-import-index-reuse.test.ts` and
* `test/unit/scope-resolution/javascript-import-target-parity.test.ts`.
*/
import { describe, expect, it } from 'vitest';
import { SupportedLanguages } from 'gitnexus-shared';
import type { ParsedImport } from 'gitnexus-shared';
import { SCOPE_RESOLVERS } from '../../../src/core/ingestion/scope-resolution/pipeline/registry.js';
import type { ScopeResolver } from '../../../src/core/ingestion/scope-resolution/contract/scope-resolver.js';
import type { ComposerConfig } from '../../../src/core/ingestion/language-config.js';
import {
CountingSet,
countedParsedFiles,
pythonNamedImport,
} from '../../helpers/counting-file-set.js';
/**
* The minimum a language needs for one `resolveImportTarget` call to reach
* whatever structure it derives from the file set.
*/
interface ImportTargetFixture {
/**
* The whole synthetic workspace. Small on purpose: the count being compared
* is traversals, not their cost.
*
* Also the source of `context.parsedFiles` — one minimal `ParsedFile` per
* path, built by `countedParsedFiles`. NOT a second fixture field, because
* the orchestrator derives the path Set FROM the parsed workspace
* (`new Set(parsedFiles.map((f) => f.filePath))` in `pipeline/run.ts`), so two
* independent lists here could disagree in a way no real pass can.
*/
readonly files: readonly string[];
/** The importing file. */
readonly fromFile: string;
/**
* The resolver's 4th argument. Not optional: the languages that take none
* pass `undefined` in the open, so no call site hides which adapters read
* this channel behind an omission.
*/
readonly resolutionConfig: unknown;
/**
* An import that resolves to NOTHING, spelled differently on every call.
*
* A miss is the expensive case in every resolver here — it runs the cascade to
* completion instead of returning on the first hit — and the distinct spelling
* defeats the per-target `resolveCache` that TypeScript, JavaScript and Vue
* keep, so the resolution path is really re-entered per import rather than
* answered from a memo.
*/
readonly missTarget: (i: number) => string;
/** An import that MUST resolve. The non-vacuity half of the assertion. */
readonly hitTarget: string;
/**
* The `parsedImport` half of the resolver's 5th argument, for the spelling
* being resolved. A function of the spelling, not a constant: PHP reaches its
* `parsedFiles` leg only for `kind: 'named' | 'alias'` carrying an
* `importedSymbolKind` of `function` or `const`, and Python resolves
* `parsedImport.targetRaw` in preference to the `targetRaw` argument — so one
* fixed import would resolve a single spelling 201 times and be answered from
* the per-target memo that the distinct `missTarget` spellings exist to
* defeat.
*
* `undefined` wherever the adapter ignores `context`; that is a statement
* about the resolver, made in the open, for the same reason
* `resolutionConfig` is never omitted.
*/
readonly parsedImport: (targetRaw: string) => ParsedImport | undefined;
/**
* Traversals of one file set that the property permits, as a floor.
*
* One for every language that derives an index from the set. ZERO for Rust,
* which is not an exemption: `resolveRustImportTarget` answers every leg with
* `allFilePaths.has(candidate)` membership probes and never iterates, so there
* is no traversal to hoist and nothing for the counter to see. (Rust's one
* workspace index, `buildRustModuleIndex`, is memoized in
* `qualified-call.ts::moduleIndexFor` and hangs off `resolveQualifiedFreeCall`
* — a different hook, not this one.)
*/
readonly minimumScans: number;
/**
* Element reads of one `context.parsedFiles` array that the property permits,
* as a floor — the `minimumScans` of the second key.
*
* ZERO wherever the adapter never reads `context`, and that zero is a fact
* about the adapter rather than an exemption: the equality half still holds,
* so a resolver that starts reading `parsedFiles` per import fails here with a
* floor of 0 in place. ONE for PHP and Python, which is what proves the leg
* behind `context` was entered at all — an early `return` on
* `context === undefined` posts a perfect zero otherwise, which is precisely
* how every arm of this file passed while measuring nothing on that channel.
*/
readonly minimumParsedFileReads: number;
}
/** The `composer.json` PSR-4 map `loadPhpComposerConfig` would have produced. */
const PHP_COMPOSER: ComposerConfig = { psr4: new Map([['App', 'app']]) };
/** The value `loadGoModulePath` produces for a repo with a `go.mod`. */
const GO_MODULE = { modulePath: 'example.com/mod' };
/**
* The `parsedImport` of an adapter that takes three or four parameters and so
* cannot observe one. Named rather than inlined so a reader scanning the table
* sees at a glance which languages differ.
*/
const IGNORES_CONTEXT = (): undefined => undefined;
/**
* `use function Vendor\Ghost\missing;` — the one PHP import shape that reaches
* `filesByDirectory`. A `type` import (the default for `use X;`) returns before
* the `parsedFiles` leg, so the class-style spelling the other arms use would
* leave `parsedFileReads` at 0.
*/
const PHP_FUNCTION_IMPORT = (targetRaw: string): ParsedImport => ({
kind: 'named',
localName: 'imported',
importedName: 'imported',
targetRaw,
importedSymbolKind: 'function',
});
/**
* `from <targetRaw> import Widget` — a named import, which is what makes
* `resolvePythonImportTarget` run the package-attribute probe
* (`pythonFileExportsName`, the `context.parsedFiles` reader) ahead of the
* submodule fallback. The default the adapter synthesizes when `context` is
* absent is a `namespace` import, and that shape never reaches the probe.
*/
const FIXTURES: ReadonlyMap<SupportedLanguages, ImportTargetFixture> = new Map<
SupportedLanguages,
ImportTargetFixture
>([
[
SupportedLanguages.Python,
{
// `realpkg/__init__.py` makes the package real, so `hasRepoCandidate`
// passes and the miss reaches `resolveAbsoluteFromFiles` — both index
// consumers, not just the gate.
files: ['pkg/sub/mod.py', 'realpkg/__init__.py', 'realpkg/widget.py', 'app/main.py'],
fromFile: 'app/main.py',
resolutionConfig: undefined,
missTarget: (i) => `realpkg.ghost${i}`,
hitTarget: 'realpkg.widget',
parsedImport: pythonNamedImport,
minimumScans: 1,
// The one build of `parsedFileByPath`, triggered by the single import
// whose package probe resolves — the misses never get that far, and
// every later resolver is a `Map.get` rather than another pass.
minimumParsedFileReads: 1,
},
],
[
SupportedLanguages.CSharp,
{
// No `.csproj` in the config, which is the leg that reads both the shared
// workspace index and the namespace-directory index.
files: ['App/Models/User.cs', 'App/Services/Service.cs', 'App/Program.cs'],
fromFile: 'App/Program.cs',
resolutionConfig: undefined,
missTarget: (i) => `Vendor${i}.Ghost.Deep.Missing`,
hitTarget: 'App.Models.User',
parsedImport: IGNORES_CONTEXT,
minimumScans: 1,
minimumParsedFileReads: 0,
},
],
[
SupportedLanguages.TypeScript,
{
files: ['src/util.ts', 'src/models/user.ts', 'src/main.ts'],
fromFile: 'src/main.ts',
resolutionConfig: undefined,
missTarget: (i) => `./ghost${i}`,
hitTarget: './util',
parsedImport: IGNORES_CONTEXT,
minimumScans: 1,
minimumParsedFileReads: 0,
},
],
[
SupportedLanguages.Go,
{
files: ['internal/models/user.go', 'internal/models/user_test.go', 'main.go'],
fromFile: 'main.go',
resolutionConfig: GO_MODULE,
// Third-party: misses the module leg and runs the whole GOPATH suffix
// cascade, which used to cost one full scan per path segment.
missTarget: (i) => `github.com/vendor/dep${i}/sub`,
hitTarget: 'example.com/mod/internal/models',
parsedImport: IGNORES_CONTEXT,
minimumScans: 1,
minimumParsedFileReads: 0,
},
],
[
SupportedLanguages.Java,
{
files: ['com/example/model/User.java', 'src/main/java/com/example/App.java'],
fromFile: 'src/main/java/com/example/App.java',
resolutionConfig: undefined,
// Four segments and no hit: the progressive-stripping loop runs to the
// end, which is what every JDK and third-party import does.
missTarget: (i) => `vendor${i}.ghost.deep.Missing`,
hitTarget: 'com.example.model.User',
parsedImport: IGNORES_CONTEXT,
minimumScans: 1,
minimumParsedFileReads: 0,
},
],
[
SupportedLanguages.C,
{
// `resolutionConfig` is the header set from `loadResolutionConfig`. Left
// undefined so the resolver indexes THIS set: with headers present the
// adapter hands the resolver a memoized union instead, and the union's
// own scan is the only one this counter would see.
files: ['include/util.h', 'src/helper.h', 'src/main.c'],
fromFile: 'src/main.c',
resolutionConfig: undefined,
missTarget: (i) => `ghost${i}.h`,
hitTarget: 'util.h',
parsedImport: IGNORES_CONTEXT,
minimumScans: 1,
minimumParsedFileReads: 0,
},
],
[
SupportedLanguages.CPlusPlus,
{
// Same accounting as C: `resolveCppImportTarget` delegates to the C
// resolver's basename index, keyed on the same Set.
files: ['include/util.hpp', 'src/helper.hpp', 'src/main.cpp'],
fromFile: 'src/main.cpp',
resolutionConfig: undefined,
missTarget: (i) => `ghost${i}.hpp`,
hitTarget: 'util.hpp',
parsedImport: IGNORES_CONTEXT,
minimumScans: 1,
minimumParsedFileReads: 0,
},
],
[
SupportedLanguages.PHP,
{
files: ['app/Models/User.php', 'lib/Legacy/Helper.php', 'app/Main.php'],
fromFile: 'app/Main.php',
resolutionConfig: PHP_COMPOSER,
// Deliberately matches NO PSR-4 prefix. `resolvePhpImportInternal` runs
// its namespace-directory fallback scan unconditionally when
// `getFilesInDir` comes back empty, so a miss UNDER `App\` — say
// `App\Legacy\Ghost`, whose directory does not exist — costs one
// traversal per import: swapping this fixture onto that spelling posts
// 201 traversals for 200 imports against 3 for two (measured). The
// residual is real and is out of this file's reach — it lives in
// `import-resolvers/php.ts`, which the #2901 hoist does not touch — so it
// is pinned by name in `php-import-target-parity.test.ts` and this
// fixture takes the leg that IS indexed rather than restating it.
missTarget: (i) => `Vendor${i}\\Ghost\\Missing`,
hitTarget: 'App\\Models\\User',
parsedImport: PHP_FUNCTION_IMPORT,
minimumScans: 1,
// `filesByDirectory`'s one pass over the parsed workspace, memoized on it.
minimumParsedFileReads: 1,
},
],
[
SupportedLanguages.Rust,
{
files: ['src/lib.rs', 'src/models.rs', 'src/main.rs'],
fromFile: 'src/main.rs',
resolutionConfig: undefined,
missTarget: (i) => `ghost${i}::deep::Missing`,
hitTarget: 'crate::models',
parsedImport: IGNORES_CONTEXT,
// See `minimumScans` on the interface: membership probes only.
minimumScans: 0,
minimumParsedFileReads: 0,
},
],
[
SupportedLanguages.JavaScript,
{
files: ['src/util.js', 'src/models/user.js', 'src/main.js'],
fromFile: 'src/main.js',
resolutionConfig: undefined,
missTarget: (i) => `./ghost${i}`,
hitTarget: './util',
parsedImport: IGNORES_CONTEXT,
minimumScans: 1,
minimumParsedFileReads: 0,
},
],
[
SupportedLanguages.Kotlin,
{
files: [
'lib/src/main/kotlin/com/example/widget/Widget.kt',
'common/src/main/kotlin/com/example/common/Util.kt',
],
fromFile: 'common/src/main/kotlin/com/example/common/Util.kt',
resolutionConfig: undefined,
missTarget: (i) => `org.ghost${i}.deep.Missing`,
// Under a module source root, so this resolves by path suffix rather than
// by a workspace-rooted exact match.
hitTarget: 'com.example.widget.Widget',
parsedImport: IGNORES_CONTEXT,
minimumScans: 1,
minimumParsedFileReads: 0,
},
],
[
SupportedLanguages.Ruby,
{
files: ['lib/app/models/user.rb', 'lib/util.rb', 'lib/main.rb'],
fromFile: 'lib/main.rb',
resolutionConfig: undefined,
// A bare `require`, not a `require_relative`: the relative leg answers
// from `Set.has` and never reaches the index.
missTarget: (i) => `gem${i}/missing`,
hitTarget: 'app/models/user',
parsedImport: IGNORES_CONTEXT,
minimumScans: 1,
minimumParsedFileReads: 0,
},
],
[
SupportedLanguages.Cobol,
{
files: ['copybooks/CUSTREC.cpy', 'src/PAYROLL.cbl', 'src/PROG.cbl'],
fromFile: 'src/PROG.cbl',
resolutionConfig: undefined,
// Vendor and system copybooks live outside the repo, so the common case
// misses both tiers — two full scans per `COPY` before the index.
missTarget: (i) => `VENDOR${i}`,
hitTarget: 'CUSTREC',
parsedImport: IGNORES_CONTEXT,
minimumScans: 1,
minimumParsedFileReads: 0,
},
],
[
SupportedLanguages.Swift,
{
files: ['Sources/Models/User.swift', 'Sources/App/main.swift'],
fromFile: 'Sources/App/main.swift',
resolutionConfig: undefined,
missTarget: (i) => `Ghost${i}`,
hitTarget: 'Models',
parsedImport: IGNORES_CONTEXT,
minimumScans: 1,
minimumParsedFileReads: 0,
},
],
[
SupportedLanguages.Dart,
{
files: ['lib/models.dart', 'tool/generate.dart', 'lib/main.dart'],
fromFile: 'lib/main.dart',
resolutionConfig: undefined,
// An external package: both `lib/<rel>` and bare `<rel>` miss, which is
// the two-scan case.
missTarget: (i) => `package:vendor${i}/ghost.dart`,
hitTarget: 'package:app/models.dart',
parsedImport: IGNORES_CONTEXT,
minimumScans: 1,
minimumParsedFileReads: 0,
},
],
[
SupportedLanguages.Vue,
{
files: ['src/components/Widget.vue', 'src/util.ts', 'src/App.vue'],
fromFile: 'src/App.vue',
resolutionConfig: undefined,
missTarget: (i) => `./ghost${i}.vue`,
hitTarget: './components/Widget.vue',
parsedImport: IGNORES_CONTEXT,
minimumScans: 1,
minimumParsedFileReads: 0,
},
],
]);
/**
* Registered resolvers exempted from the property, each with the open issue
* that will remove the exemption.
*
* EMPTY, and that is the result rather than the starting state: every resolver
* in `SCOPE_RESOLVERS` either memoizes its index on the `allFilePaths` Set
* identity or never traverses the Set at all (#2872, #2877, #2878, #2879, #2880,
* #2901, #2902, #2908 closed the last of them). The map stays because the
* mechanism is the point — the next language must not be able to opt out of the
* property by quietly not appearing in `FIXTURES`. An entry here must
* cite an open issue (`#NNNN`); the arm below enforces the citation, and the
* pinned empty key list means adding one is a visible, reviewed edit rather
* than a line in a table nobody reads.
*/
const KNOWN_UNINDEXED: ReadonlyMap<SupportedLanguages, string> = new Map<
SupportedLanguages,
string
>();
interface ContractCase {
readonly language: SupportedLanguages;
readonly resolver: ScopeResolver;
readonly fixture: ImportTargetFixture;
}
const CASES: readonly ContractCase[] = [...SCOPE_RESOLVERS.entries()].flatMap(
([language, resolver]) => {
const fixture = FIXTURES.get(language);
return fixture === undefined ? [] : [{ language, resolver, fixture }];
},
);
/** Imports driven in the baseline run — the smallest count above one. */
const BASELINE_IMPORTS = 2;
/** Imports driven in the comparison run. A per-import scan shows up as a 100x. */
const MANY_IMPORTS = 200;
interface ImportRun {
/** Full traversals of the run's own file set. */
readonly scans: number;
/** Element reads of the run's own `context.parsedFiles` array. */
readonly parsedFileReads: number;
/** What `hitTarget` resolved to, read after the misses. */
readonly hit: string | readonly string[] | null;
}
/**
* Drive `importCount` missing imports and then one resolvable import through
* the orchestrator ADAPTER — `<lang>ScopeResolver.resolveImportTarget`, the
* surface a defensive `new Set(allFilePaths)` copy breaks and the per-language
* unit parity tests never cross.
*
* Five arguments, the shape `pipeline/run.ts` uses. One `context` object for
* the whole run, because that is what the orchestrator threads: it builds
* `parsedFiles` once per pass, so the array identity PHP's `filesByDirectory`
* memoizes on is stable across every import. Rebuilding it here would hand each
* import a fresh key and turn the fixture itself into the defect.
*
* Fresh instruments per run, for the same reason on both keys: the indexes hang
* off object identity, so two runs sharing a Set or a `parsedFiles` array would
* have the second read the first's index and report zero.
*/
function driveImports(
resolver: ScopeResolver,
fixture: ImportTargetFixture,
importCount: number,
): ImportRun {
const files = new CountingSet(fixture.files);
const workspace = countedParsedFiles(fixture.files);
const contextFor = (targetRaw: string) => ({
parsedFiles: workspace.parsedFiles,
parsedImport: fixture.parsedImport(targetRaw),
});
for (let i = 0; i < importCount; i++) {
const target = fixture.missTarget(i);
resolver.resolveImportTarget(
target,
fixture.fromFile,
files,
fixture.resolutionConfig,
contextFor(target),
);
}
const hit = resolver.resolveImportTarget(
fixture.hitTarget,
fixture.fromFile,
files,
fixture.resolutionConfig,
contextFor(fixture.hitTarget),
);
return { scans: files.scans, parsedFileReads: workspace.reads(), hit };
}
describe('import-target index reuse — the contract every registered resolver holds', () => {
it.each(CASES)(
'$language traverses the file set no more times for many imports than for two',
({ language, resolver, fixture }) => {
const few = driveImports(resolver, fixture, BASELINE_IMPORTS);
const many = driveImports(resolver, fixture, MANY_IMPORTS);
// The property. A per-import scan makes `many` ~100x `few`; a scan
// reintroduced beside a reused index moves both by the same constant and
// is caught instead by the per-language guards' exact counts.
expect(
many.scans,
`${language}: ${MANY_IMPORTS} imports cost ${many.scans} traversals, ${BASELINE_IMPORTS} cost ${few.scans} — the file set is being re-read per import`,
).toBe(few.scans);
// The same property on the other per-file-set key. PHP's
// `filesByDirectory` and Python's `pythonFileExportsName` read
// `context.parsedFiles` and never touch the Set, so the arm above is
// blind to both — measured, not assumed: defeating PHP's `perFileSet`
// leaves `scans` unmoved and takes this count from 9 to 603.
expect(
many.parsedFileReads,
`${language}: ${MANY_IMPORTS} imports read context.parsedFiles ${many.parsedFileReads} times, ${BASELINE_IMPORTS} read it ${few.parsedFileReads} — the parsed workspace is being re-derived per import`,
).toBe(few.parsedFileReads);
// Non-vacuity, one arm per thing the counts could be measuring nothing
// about. Without them a resolver that resolves nothing, or a leg that is
// never entered, posts a perfect score.
expect(
many.scans,
`${language}: the counting file set was never reached — is the adapter copying it?`,
).toBeGreaterThanOrEqual(fixture.minimumScans);
expect(
many.parsedFileReads,
`${language}: context.parsedFiles was never read — did the leg behind it stop being entered?`,
).toBeGreaterThanOrEqual(fixture.minimumParsedFileReads);
expect(
many.hit,
`${language}: '${fixture.hitTarget}' no longer resolves, so the counts above measure nothing`,
).not.toBeNull();
},
);
it('covers every registered scope resolver', () => {
const registered = [...SCOPE_RESOLVERS.keys()].sort();
const accountedFor = [...FIXTURES.keys(), ...KNOWN_UNINDEXED.keys()].sort();
// A new language in `pipeline/registry.ts` lands here first: it is either
// given a fixture in `FIXTURES` or an entry in `KNOWN_UNINDEXED`, and both
// are edits someone has to justify.
expect(accountedFor).toEqual(registered);
});
it('exempts nothing, and would make an exemption cite an issue', () => {
for (const [language, reason] of KNOWN_UNINDEXED) {
expect(reason, `${language}'s exemption must cite an open issue`).toMatch(/#\d+/);
}
expect([...KNOWN_UNINDEXED.keys()]).toEqual([]);
});
});

634
gitnexus/test/unit/scope-resolution/java-import-target-parity.test.ts Normal file
View file

@ -0,0 +1,634 @@
/**
* Differential harness for the Java import-target index hoist (#2908).
*
* `resolveJavaImportTarget` answered its three-tier cascade with a full
* `allFilePaths` scan, and ran that scan AGAIN inside the progressive
* prefix-stripping loop — once per stripped segment. Replacing those scans with
* the per-file-set indexes (`getWorkspaceFileIndex` +
* `buildPackageDirIndex`/`firstFileDirectlyInPkgDir`) is a pure performance
* change ONLY if every implicit tie-break survives, and those tie-breaks are
* expressed through Set-iteration order and `indexOf` positions rather than
* through anything the type system or the existing Java tests can see:
*
* - the first pass `break`s on an exact whole-path hit, so an exact match wins
* over a suffix OR directory-child match found EARLIER in iteration order;
* - the stripping loop instead returns mid-scan at the first hit of
* `f === tailFile || f.endsWith('/' + tailFile)` — no exact-wins rule there
* — while its directory child is collected and returned only after the scan
* completes, so file/suffix beats directory child within one `skip` level
* regardless of order;
* - the directory-child leg takes the FIRST `'/' + pathLike + '/'` occurrence,
* so `com/example/com/example/Deep.java` does NOT answer `com.example`;
* - a wildcard import drops its trailing `.*` before any of that runs;
* - paths are compared normalized (`\` → `/`) but returned RAW.
*
* So this file keeps a VERBATIM copy of the pre-change implementation — the
* `resolveJavaImportTarget` that shipped before #2908, scans and all — and
* asserts the new one agrees with it, both on hand-built corpora built to force
* exactly those cases and on a generated corpus replayed under three insertion
* orders — order being the only channel most of these tie-breaks travel on.
* The copy is the specification; if a future change makes an arm here fail, the
* resolver's OUTPUT moved and Java's IMPORTS edges move with it.
*
* The hand-built arm additionally pins ABSOLUTE expectations. A pure
* differential goes green when old and new agree on `null` everywhere, which is
* also what a corpus that has quietly stopped matching anything looks like.
*
* The last arm counts how often the file Set is iterated, as the deterministic
* guard against a scan reintroduced BESIDE the reused index. It is not the
* guard for a defensive `new Set(allFilePaths)` copy in the orchestrator
* ADAPTER — that lives one layer above every call here, and is guarded by
* `test/integration/java-import-index-reuse.test.ts`.
*/
import { describe, expect, it } from 'vitest';
import type { ParsedImport, WorkspaceIndex } from 'gitnexus-shared';
import { resolveJavaImportTarget } from '../../../src/core/ingestion/languages/java/import-target.js';
import { CountingSet } from '../../helpers/counting-file-set.js';
// ─── verbatim pre-change implementation ──────────────────────────────────────
interface LegacyJavaResolveContext {
readonly fromFile: string;
readonly allFilePaths: ReadonlySet<string>;
}
function legacyResolveJavaImportTarget(
parsedImport: ParsedImport,
workspaceIndex: WorkspaceIndex,
): string | null {
const ctx = workspaceIndex as LegacyJavaResolveContext | undefined;
if (
ctx === undefined ||
typeof (ctx as { fromFile?: unknown }).fromFile !== 'string' ||
!((ctx as { allFilePaths?: unknown }).allFilePaths instanceof Set)
) {
return null;
}
if (parsedImport.kind === 'dynamic-unresolved') return null;
if (parsedImport.targetRaw === null || parsedImport.targetRaw === '') return null;
// Strip trailing `.*` for wildcard imports: `com.example.*` → `com.example`
let target = parsedImport.targetRaw;
if (target.endsWith('.*')) {
target = target.slice(0, -2);
}
// Package path: `com.example.User` → `com/example/User`
const pathLike = target.replace(/\./g, '/');
const suffix = `/${pathLike}`;
let exactFile: string | null = null;
let suffixFile: string | null = null;
let directoryChild: string | null = null;
const dirPrefix = `${pathLike}/`;
const suffixDirPrefix = `/${dirPrefix}`;
for (const raw of ctx.allFilePaths) {
const f = raw.replace(/\\/g, '/');
if (!f.endsWith('.java')) continue;
if (f === `${pathLike}.java`) {
exactFile = raw;
break;
}
if (suffixFile === null && f.endsWith(`${suffix}.java`)) {
suffixFile = raw;
}
if (directoryChild === null) {
const atRoot = f.startsWith(dirPrefix);
const atNested = f.includes(suffixDirPrefix);
if (atRoot || atNested) {
const idx = atRoot ? 0 : f.indexOf(suffixDirPrefix) + 1;
const after = f.slice(idx + dirPrefix.length);
if (after.length > 0 && !after.includes('/')) {
directoryChild = raw;
}
}
}
}
if (exactFile !== null) return exactFile;
if (suffixFile !== null) return suffixFile;
if (directoryChild !== null) return directoryChild;
// Progressive prefix stripping — handles `import com.example.User;`
// in a repo laid out `User.java` (no `com/example/` prefix).
const segments = pathLike.split('/').filter(Boolean);
for (let skip = 1; skip < segments.length; skip++) {
const tail = segments.slice(skip).join('/');
if (tail === '') continue;
const tailFile = `${tail}.java`;
const tailSuffix = `/${tailFile}`;
const tailDir = `${tail}/`;
const tailSuffixDir = `/${tailDir}`;
let tailDirectChild: string | null = null;
for (const raw of ctx.allFilePaths) {
const f = raw.replace(/\\/g, '/');
if (!f.endsWith('.java')) continue;
if (f === tailFile) return raw;
if (f.endsWith(tailSuffix)) return raw;
if (tailDirectChild === null) {
const atRoot = f.startsWith(tailDir);
const atNested = f.includes(tailSuffixDir);
if (atRoot || atNested) {
const idx = atRoot ? 0 : f.indexOf(tailSuffixDir) + 1;
const after = f.slice(idx + tailDir.length);
if (after.length > 0 && !after.includes('/')) tailDirectChild = raw;
}
}
}
if (tailDirectChild !== null) return tailDirectChild;
}
return null;
}
// ─── harness ─────────────────────────────────────────────────────────────────
const FROM_FILE = 'src/main/java/com/example/App.java';
function javaImport(targetRaw: string): ParsedImport {
return { kind: 'named', localName: '_', importedName: '_', targetRaw };
}
/** A file layout plus the import spelling resolved against it. */
interface Case {
readonly label: string;
readonly files: readonly string[];
readonly target: string;
}
/**
* `label => result`, so a divergence names the case instead of printing an
* index into two long arrays. `null` is spelled, not dropped: "resolved
* nothing" is a real answer and must be diffed like any other.
*/
function runAll(
cases: readonly Case[],
resolve: (i: ParsedImport, w: WorkspaceIndex) => string | null,
): string[] {
return cases.map((c) => {
const ws = { fromFile: FROM_FILE, allFilePaths: new Set(c.files) };
return `${c.label} => ${resolve(javaImport(c.target), ws) ?? 'null'}`;
});
}
// ─── hand-built tie-break corpora ────────────────────────────────────────────
/**
* One corpus per tie-break, each as small as the rule it pins. The expectation
* strings are the pre-change behaviour, derived by hand from the scan and
* confirmed against the verbatim copy by the first `it` below.
*/
const HAND_CASES: readonly Case[] = [
{
// Tier 1: the scan `break`s on the exact hit, so it wins over the suffix
// match already found at position 0. `index.get` alone returns position 0.
label: 'exact-beats-earlier-suffix',
files: ['src/main/java/com/example/model/User.java', 'com/example/model/User.java'],
target: 'com.example.model.User',
},
{
// Same rule against the directory-child leg.
label: 'exact-beats-earlier-directory-child',
files: ['com/example/util/Helper/Inner.java', 'com/example/util/Helper.java'],
target: 'com.example.util.Helper',
},
{
label: 'suffix-beats-earlier-directory-child',
files: ['com/example/util/Helper/Inner.java', 'src/com/example/util/Helper.java'],
target: 'com.example.util.Helper',
},
{
label: 'directory-child-is-first-in-set-order',
files: ['com/example/service/Beta.java', 'com/example/service/Alpha.java'],
target: 'com.example.service',
},
{
label: 'directory-child-order-follows-insertion',
files: ['com/example/service/Alpha.java', 'com/example/service/Beta.java'],
target: 'com.example.service',
},
{
// Tie-break 3: `.*` is stripped, so this is the package-directory query.
label: 'wildcard-resolves-as-package-directory',
files: ['com/example/service/Beta.java', 'com/example/service/Alpha.java'],
target: 'com.example.service.*',
},
{
// A file named like the package beats that package's directory child.
label: 'wildcard-exact-file-beats-directory',
files: ['com/example/service/Alpha.java', 'com/example/service.java'],
target: 'com.example.service.*',
},
{
// Tie-break 5: the FIRST `/com/example/` occurrence leaves `com/example/
// Deep.java` after it, which still contains a slash — so no match.
label: 'self-nested-directory-does-not-match-outer',
files: ['com/example/com/example/Deep.java'],
target: 'com.example',
},
{
label: 'self-nested-directory-matches-full-path',
files: ['com/example/com/example/Deep.java'],
target: 'com.example.com.example',
},
{
// Tie-break 2: the directory child is seen first, the suffix hit second,
// and the suffix hit still wins because the scan returns mid-loop.
label: 'stripping-suffix-beats-earlier-directory-child',
files: ['x/models/Order/Part.java', 'y/models/Order.java'],
target: 'com.shop.models.Order',
},
{
label: 'stripping-reaches-root-file',
files: ['Order.java'],
target: 'com.shop.Order',
},
{
// The mirror of tie-break 1: inside the stripping loop the scan returns at
// the first hit of `f === tailFile || f.endsWith('/' + tailFile)`, so the
// suffix hit at position 0 beats the whole-path file behind it. Applying
// tier 1's exact-wins rule here would answer `Order.java`.
label: 'stripping-takes-the-first-hit-not-the-whole-path-one',
files: ['a/Order.java', 'Order.java'],
target: 'com.shop.Order',
},
{
label: 'stripping-reaches-directory-child',
files: ['proj/models/Thing.java'],
target: 'com.shop.models',
},
{
// Tie-break 4: matched on the normalized path, returned RAW.
label: 'backslash-paths-normalize-and-return-raw',
files: ['win\\src\\com\\example\\win\\Windows.java'],
target: 'com.example.win.Windows',
},
{
label: 'duplicate-normalized-path-keeps-first-raw-spelling',
files: ['a/b/Dup.java', 'a\\b\\Dup.java'],
target: 'a.b.Dup',
},
{
label: 'duplicate-normalized-path-keeps-first-raw-spelling-reversed',
files: ['a\\b\\Dup.java', 'a/b/Dup.java'],
target: 'a.b.Dup',
},
{
// Tie-break 4: the `.java` filter, on both the file and the directory legs.
label: 'non-java-sibling-is-skipped',
files: ['com/example/model/User.kt', 'com/example/model/User.java'],
target: 'com.example.model.User',
},
{
label: 'directory-of-non-java-files-is-not-a-package-directory',
files: ['com/example/onlytext/notes.txt', 'com/example/onlytext/README.md'],
target: 'com.example.onlytext',
},
{
label: 'several-directories-share-a-last-segment',
files: ['svc-b/shared/BShared.java', 'svc-a/shared/AShared.java'],
target: 'shared',
},
{
label: 'several-directories-share-a-last-segment-reversed',
files: ['svc-a/shared/AShared.java', 'svc-b/shared/BShared.java'],
target: 'shared',
},
{
label: 'root-file-exact-match',
files: ['src/Loose.java', 'Loose.java'],
target: 'Loose',
},
{
label: 'single-segment-target-has-no-stripping-pass',
files: ['deep/pkg/Loose.java'],
target: 'Loose',
},
{
// `.*` alone strips to the empty package path — the degenerate query the
// directory index answers through its empty-last-segment bucket.
label: 'bare-wildcard-over-absolute-paths',
files: ['/abs/Root.java', '/Top.java'],
target: '.*',
},
{
label: 'bare-wildcard-over-relative-paths',
files: ['pkg/Root.java', 'Top.java'],
target: '.*',
},
{
label: 'empty-segments-are-not-collapsed-in-the-first-pass',
files: ['com/example/User.java'],
target: 'com..example.User',
},
{
// Every tier is case-sensitive, so the package path never matches — but the
// stripping loop reaches the bare basename, which does.
label: 'case-mismatch-falls-through-to-basename-stripping',
files: ['com/Example/Model/Cased.java'],
target: 'com.example.model.Cased',
},
{
label: 'directory-named-like-a-java-file',
files: ['com/example/weird.java/Inside.java'],
target: 'com.example.weird',
},
{
// Java has no in-repo-namespace gate (C#'s #1881), so a JDK import whose
// tail happens to exist locally resolves to it. Pinned, not endorsed.
label: 'jdk-import-strips-into-a-local-lookalike',
files: ['com/example/model/User.java', 'src/main/java/util/List.java'],
target: 'java.util.List',
},
{
label: 'jdk-import-with-no-lookalike-resolves-to-nothing',
files: ['com/example/model/User.java'],
target: 'java.util.List',
},
];
/** Absolute pre-change behaviour, so the differential cannot pass vacuously. */
const HAND_EXPECTED: readonly string[] = [
'exact-beats-earlier-suffix => com/example/model/User.java',
'exact-beats-earlier-directory-child => com/example/util/Helper.java',
'suffix-beats-earlier-directory-child => src/com/example/util/Helper.java',
'directory-child-is-first-in-set-order => com/example/service/Beta.java',
'directory-child-order-follows-insertion => com/example/service/Alpha.java',
'wildcard-resolves-as-package-directory => com/example/service/Beta.java',
'wildcard-exact-file-beats-directory => com/example/service.java',
'self-nested-directory-does-not-match-outer => null',
'self-nested-directory-matches-full-path => com/example/com/example/Deep.java',
'stripping-suffix-beats-earlier-directory-child => y/models/Order.java',
'stripping-reaches-root-file => Order.java',
'stripping-takes-the-first-hit-not-the-whole-path-one => a/Order.java',
'stripping-reaches-directory-child => proj/models/Thing.java',
'backslash-paths-normalize-and-return-raw => win\\src\\com\\example\\win\\Windows.java',
'duplicate-normalized-path-keeps-first-raw-spelling => a/b/Dup.java',
'duplicate-normalized-path-keeps-first-raw-spelling-reversed => a\\b\\Dup.java',
'non-java-sibling-is-skipped => com/example/model/User.java',
'directory-of-non-java-files-is-not-a-package-directory => null',
'several-directories-share-a-last-segment => svc-b/shared/BShared.java',
'several-directories-share-a-last-segment-reversed => svc-a/shared/AShared.java',
'root-file-exact-match => Loose.java',
'single-segment-target-has-no-stripping-pass => deep/pkg/Loose.java',
'bare-wildcard-over-absolute-paths => /Top.java',
// A relative root file has no leading slash, so the empty package path finds
// nothing — unlike the absolute case above.
'bare-wildcard-over-relative-paths => null',
// `filter(Boolean)` drops the empty segment, so stripping recovers the file
// the first pass could not see.
'empty-segments-are-not-collapsed-in-the-first-pass => com/example/User.java',
'case-mismatch-falls-through-to-basename-stripping => com/Example/Model/Cased.java',
'directory-named-like-a-java-file => null',
'jdk-import-strips-into-a-local-lookalike => src/main/java/util/List.java',
'jdk-import-with-no-lookalike-resolves-to-nothing => null',
];
// ─── generated corpus ────────────────────────────────────────────────────────
const SOURCE_ROOTS = ['src/main/java', 'src/test/java', '', 'legacy', 'modules/core/src/main/java'];
const PACKAGE_DIRS = [
'com/example/model',
'com/example/service',
'com/example/util',
'org/acme/api',
'io/gn/core',
];
const TYPE_NAMES = ['User', 'Order', 'Helper', 'Client', 'Registry'];
/**
* A layout where the same package path exists under several source roots AND
* root-relative, so most lookups have a whole-path candidate and one or more
* earlier suffix candidates — the collision tier 1 turns on. The tail adds the
* shapes a regular layout never produces: self-nested packages, directories
* sharing a last segment, non-`.java` neighbours, root files, backslash paths,
* and the stripping-only targets.
*/
function generatedFiles(): string[] {
const files: string[] = [];
for (const pkg of PACKAGE_DIRS) {
for (const type of TYPE_NAMES) {
for (const root of SOURCE_ROOTS) {
files.push(root === '' ? `${pkg}/${type}.java` : `${root}/${pkg}/${type}.java`);
}
}
}
// A file whose whole path IS a package directory used elsewhere.
files.push('com/example/service.java');
files.push('src/main/java/com/example/model.java');
// Packages nested inside themselves.
files.push('com/example/model/com/example/model/Nested.java');
files.push('legacy/io/gn/core/io/gn/core/Legacy.java');
// Directories sharing a last segment across trees.
for (let i = 0; i < 4; i++) {
files.push(`svc${i}/shared/Shared${i}.java`);
files.push(`svc${i}/shared/internal/Deep${i}.java`);
}
// Non-`.java` neighbours, including a directory with none of them accepted.
files.push('com/example/model/User.kt');
files.push('com/example/model/package-info.txt');
files.push('com/example/resources/application.yaml');
files.push('com/example/weird.java/Inside.java');
// Root files and a deep chain.
files.push('Loose.java');
files.push('Order.java');
files.push('a/b/c/d/e/f/Deep6.java');
// Backslash spellings, one of them a duplicate of a forward-slash entry.
files.push('win\\src\\main\\java\\com\\example\\win\\WinUser.java');
files.push('a/b/Dup.java');
files.push('a\\b\\Dup.java');
// Reachable only after progressive prefix stripping, with a directory child
// planted ahead of the suffix hit at the same `skip` level.
files.push('x/models/Order/Part.java');
files.push('bare/models/Order.java');
files.push('bare/models/Invoice.java');
return files;
}
function generatedTargets(): string[] {
const targets: string[] = [];
for (const pkg of PACKAGE_DIRS) {
const dotted = pkg.replace(/\//g, '.');
targets.push(dotted);
targets.push(`${dotted}.*`);
for (const type of TYPE_NAMES) targets.push(`${dotted}.${type}`);
}
targets.push(
// Package prefixes: partial paths that are directories but not packages.
'com',
'com.example',
'com.*',
'org',
'org.acme',
'io',
'io.gn',
'src.main.java.com.example.model.User',
'legacy.com.example.util.Helper',
'modules.core.src.main.java.io.gn.core.Client',
// Self-nesting.
'com.example.model.com.example.model',
'com.example.model.com.example.model.Nested',
'io.gn.core.io.gn.core.Legacy',
// Shared last segments.
'shared',
'shared.*',
'svc0.shared',
'svc2.shared.internal',
'internal',
// Stripping-only.
'com.shop.models.Order',
'com.shop.models',
'com.shop.models.*',
'whatever.bare.models.Invoice',
'nowhere.Loose',
'nowhere.deeply.nested.Order',
// Non-`.java` and odd shapes.
'com.example.resources',
'com.example.weird',
'com.example.model.User.kt',
'a.b.Dup',
'a.b.c.d.e.f.Deep6',
'com.example.win.WinUser',
'.*',
'*',
'com..example.User',
'Loose',
'Order',
// Unresolvable: JDK and third-party, the majority case in real source.
'java.util.List',
'java.util.*',
'java.io.File',
'javax.annotation.Nullable',
'org.junit.jupiter.api.Test',
'org.springframework.boot.SpringApplication',
'com.google.common.collect.ImmutableList',
'com.example.missing.Absent',
);
return targets;
}
/**
* Three insertion orders over the same paths. Set-iteration order IS the
* tie-break channel for every "first match wins" rule here, so replaying the
* same targets under a reversal and a rotation exercises each collision from
* both sides — the as-built order alone would leave half of them one-sided.
*/
function orderedCorpora(): ReadonlyMap<string, readonly string[]> {
const base = generatedFiles();
const reversed = [...base].reverse();
const rotation = 7;
const rotated = [...base.slice(rotation), ...base.slice(0, rotation)];
return new Map([
['as-built', base],
['reversed', reversed],
['rotated', rotated],
]);
}
function generatedCases(): Case[] {
const cases: Case[] = [];
for (const [order, files] of orderedCorpora()) {
for (const target of generatedTargets()) {
cases.push({ label: `${order}|${target}`, files, target });
}
}
return cases;
}
const GENERATED_CASES = generatedCases();
// ─── arms ────────────────────────────────────────────────────────────────────
describe('Java import target — index hoist parity (#2908)', () => {
it('reproduces the pre-change results on the hand-built tie-break corpora', () => {
expect(runAll(HAND_CASES, legacyResolveJavaImportTarget)).toEqual(HAND_EXPECTED);
expect(runAll(HAND_CASES, resolveJavaImportTarget)).toEqual(HAND_EXPECTED);
});
it('reproduces the pre-change results across three insertion orders', () => {
expect(runAll(GENERATED_CASES, resolveJavaImportTarget)).toEqual(
runAll(GENERATED_CASES, legacyResolveJavaImportTarget),
);
});
it('the generated corpus resolves a broad set of distinct targets', () => {
const results = runAll(GENERATED_CASES, resolveJavaImportTarget);
const resolvedFiles = new Set(
results.map((r) => r.split(' => ')[1]).filter((r) => r !== 'null'),
);
// Non-vacuity: a differential is worthless if both sides answer `null`.
// Sized just under the current values so ordinary corpus edits do not trip
// it, while a corpus that stops matching does.
expect(results.filter((r) => !r.endsWith('=> null')).length).toBeGreaterThan(150);
expect(resolvedFiles.size).toBeGreaterThan(40);
// ...and it must keep exercising the unresolvable majority, which is the
// only case that runs the whole cascade.
expect(results.filter((r) => r.endsWith('=> null')).length).toBeGreaterThan(50);
});
it('matches the pre-change guards for unusable inputs', () => {
const files = new Set(['com/example/model/User.java']);
const good = { fromFile: FROM_FILE, allFilePaths: files };
const inputs: readonly (readonly [string, ParsedImport, WorkspaceIndex])[] = [
['undefined context', javaImport('com.example.model.User'), undefined],
['missing fromFile', javaImport('com.example.model.User'), { allFilePaths: files }],
[
'allFilePaths is not a Set',
javaImport('com.example.model.User'),
{ fromFile: FROM_FILE, allFilePaths: ['com/example/model/User.java'] },
],
[
'dynamic-unresolved import',
{ kind: 'dynamic-unresolved', localName: '', targetRaw: 'com.example.model.User' },
good,
],
// `targetRaw: null` is reachable only on `dynamic-unresolved`, which the
// kind check above already refuses, so the resolver's null branch has no
// typeable input of its own.
['empty target', javaImport(''), good],
['wildcard kind', { kind: 'wildcard', targetRaw: 'com.example.model.*' }, good],
];
const legacy = inputs.map(([label, imp, ws]) => {
return `${label} => ${legacyResolveJavaImportTarget(imp, ws) ?? 'null'}`;
});
const current = inputs.map(([label, imp, ws]) => {
return `${label} => ${resolveJavaImportTarget(imp, ws) ?? 'null'}`;
});
expect(current).toEqual(legacy);
// Every one of them refuses, except the last — a well-formed call, so the
// arm cannot pass by refusing everything.
expect(current).toEqual([
'undefined context => null',
'missing fromFile => null',
'allFilePaths is not a Set => null',
'dynamic-unresolved import => null',
'empty target => null',
'wildcard kind => com/example/model/User.java',
]);
});
it('builds each index once per file set rather than once per import', () => {
const files = new CountingSet(generatedFiles());
const ws = { fromFile: FROM_FILE, allFilePaths: files };
const targets = generatedTargets();
const results = targets.map((t) => resolveJavaImportTarget(javaImport(t), ws));
// Two traversals for the whole run: the shared workspace/suffix index and
// the package-directory index, each memoized on this Set's identity. The
// pre-change resolver traversed once per import PLUS once per stripped
// segment.
expect(files.scans).toBe(2);
// Paired result assertion — a count of 2 is equally true of a resolver that
// has stopped resolving anything at all.
expect(results.filter((r) => r !== null).length).toBeGreaterThan(20);
expect(resolveJavaImportTarget(javaImport('com.example.model.User'), ws)).toBe(
'com/example/model/User.java',
);
});
});

710
gitnexus/test/unit/scope-resolution/javascript-import-target-parity.test.ts Normal file
View file

@ -0,0 +1,710 @@
/**
* Differential harness for the JavaScript import-target suffix index (#2910).
*
* JavaScript's `PassCache` was the TypeScript one minus its `index` field, so
* `makeJsResolveImportTarget` handed `resolveTsTarget` a context with
* `index: undefined` and every JavaScript import fell through to
* `suffixResolve`'s linear `findIndex` — one pass over `normalizedFileList` per
* path part per extension, and `EXTENSIONS` has ~39 entries. 6448.9 µs per
* import at 2000 files and 25972.6 µs at 8000, against 25.0 / 27.0 µs for
* TypeScript over the same corpus; 28.5 / 27.4 µs with the index.
*
* Adding the field is NOT a pure hoist. `suffixResolve` answers a different
* question with an index than without:
*
* - without: `filePath.endsWith('/' + s)`, so only a PROPER suffix matches;
* - with: `index.get(s) || index.getInsensitive(s)`, and `buildSuffixIndex`
* indexes `j = 0`, so WHOLE paths match too.
*
* This file holds a verbatim copy of the pre-change adapter
* (`git show HEAD:gitnexus/src/core/ingestion/languages/javascript/import-target.ts`)
* and pins exactly what that difference does. Two classes of answer move and no
* others:
*
* A. `null → repo-root file`. A path with no `/` has no proper suffix at all,
* so the scan could never reach it: `require('config')` was unresolvable
* with `config.js` sitting in the repo root.
* B. `file → different file`, always toward a MORE specific match. The scan
* skips the whole-path candidate and falls through to a shorter path
* suffix or a later extension, where it finds something else:
* `import 'app/main'` resolved to `node_modules/dep0/lib/main.js` — the
* first `/main.js` in file order — and now resolves to `app/main.js`.
*
* Measured over 211 200 old-vs-new pairs (400 generated corpora × 3 importing
* files × 176 targets) there is no third class: the index never loses a match
* the scan found, and its answer is never matched at a less specific
* (path-part, extension) position. Both of those are asserted below as
* universal properties over this corpus rather than as a count.
*
* ## Why the moved answers are JavaScript being fixed, not the index being wrong
*
* TypeScript and Vue have run the indexed path since #1918, over an identically
* built `normalizedFileList` (`allFileList.map(f => f.toLowerCase())`), through
* the same `resolveTsTarget` — and this adapter's whole stated design is "TS
* resolver, JS extensions". So the fix makes JavaScript agree with TypeScript,
* and the arm below asserts that agreement over the entire corpus rather than
* asserting it in prose. Class B's witness settles the direction: resolving
* `'app/main'` into `node_modules` was not a behaviour worth preserving.
*
* ## The scan counter, and its control
*
* The last arm counts entries into `suffixResolve`'s linear branch. That is the
* instrument this defect needed and did not have: `CountingSet` counts
* traversals of the SET, and this scan walks the materialized array behind it,
* which is exactly why the defect survived every index-reuse guard that existed
* and the contract test over `SCOPE_RESOLVERS`. The arm reads the legacy
* adapter first, so a
* count of zero is paired with a demonstration that the counter can be nonzero.
*/
import { describe, expect, it, vi } from 'vitest';
import { SupportedLanguages } from 'gitnexus-shared';
import { makeJsResolveImportTarget } from '../../../src/core/ingestion/languages/javascript/import-target.js';
import { typescriptScopeResolver } from '../../../src/core/ingestion/languages/typescript/scope-resolver.js';
import {
resolveTsTarget,
type TsResolveContext,
} from '../../../src/core/ingestion/languages/typescript/import-target.js';
import { EXTENSIONS } from '../../../src/core/ingestion/import-resolvers/utils.js';
// ─── the linear-fallback counter ─────────────────────────────────────────────
// `suffixResolve` is reached through `import-resolvers/standard.ts`, which
// imports it by a relative specifier that resolves to this same module id.
// Everything else in the module — `buildSuffixIndex`, `EXTENSIONS`,
// `tryResolveWithExtensions` — is passed straight through.
const linearScans = { count: 0 };
vi.mock('../../../src/core/ingestion/import-resolvers/utils.js', async (importOriginal) => {
const actual =
await importOriginal<typeof import('../../../src/core/ingestion/import-resolvers/utils.js')>();
return {
...actual,
suffixResolve: (
pathParts: string[],
normalizedFileList: string[],
allFileList: string[],
index?: import('../../../src/core/ingestion/import-resolvers/utils.js').SuffixIndex,
): string | null => {
linearScans.count += index === undefined ? 1 : 0;
return actual.suffixResolve(pathParts, normalizedFileList, allFileList, index);
},
};
});
// ─── verbatim pre-change implementation ──────────────────────────────────────
// Copied from `git show HEAD:gitnexus/src/core/ingestion/languages/javascript/
// import-target.ts`. Only the names are prefixed; the body is untouched, and in
// particular the `PassCache` below still has no `index` field and the cache is
// still the single slot the WeakMap replaced.
type LegacyJsResolveContext = TsResolveContext;
type LegacyPassCache = {
readonly key: ReadonlySet<string>;
readonly allFilePaths: Set<string>;
readonly allFileList: readonly string[];
readonly normalizedFileList: readonly string[];
readonly resolveCache: Map<string, string | null>;
};
function legacyMakeJsResolveImportTarget(): (
targetRaw: string,
fromFile: string,
allFilePaths: ReadonlySet<string>,
resolutionConfig?: unknown,
) => string | readonly string[] | null {
let cached: LegacyPassCache | null = null;
return (targetRaw, fromFile, allFilePaths) => {
if (cached === null || cached.key !== allFilePaths) {
const allFileList = Array.from(allFilePaths);
cached = {
key: allFilePaths,
allFilePaths: new Set(allFilePaths),
allFileList,
normalizedFileList: allFileList.map((f) => f.toLowerCase()),
resolveCache: new Map(),
};
}
const ws: LegacyJsResolveContext = {
fromFile,
language: SupportedLanguages.JavaScript,
allFilePaths: cached.allFilePaths,
allFileList: cached.allFileList,
normalizedFileList: cached.normalizedFileList,
resolveCache: cached.resolveCache,
tsconfigPaths: null,
};
return resolveTsTarget(targetRaw, ws);
};
}
// ─── corpus ──────────────────────────────────────────────────────────────────
/** One differential case. `files` is emitted in the listed order, and that
* order is the tie-break under test — nothing here is random. */
interface Case {
readonly name: string;
readonly files: readonly string[];
readonly target: string;
readonly fromFile: string;
}
const FROM_FILE = 'src/main.js';
/**
* A deterministic multi-root workspace. Every root carries the same relative
* layout, so a suffix-keyed lookup and a proper-suffix scan disagree about
* which root wins; `node_modules/dep0/lib/main.js` exists so a short suffix has
* somewhere wrong to land; `SRC/Utils/Helper0.JS` differs from
* `src/utils/helper0.js` only in case; and `config.js`, `index.js`, `mod0.js`
* sit at the repo root, where no proper suffix can reach them.
*/
function generatedFiles(): string[] {
const files: string[] = [];
for (let i = 0; i < 10; i++) {
files.push(`src/components/Widget${i}.js`);
files.push(`src/components/widget${i}.jsx`);
files.push(`vendor/pkg${i % 3}/src/utils/helper${i}.js`);
files.push(`src/utils/helper${i}.js`);
files.push(`src/utils/helper${i}/index.js`);
files.push(`lib/mod${i}.mjs`);
files.push(`lib/legacy${i}.cjs`);
files.push(`mod${i}.js`);
files.push(`node_modules/dep${i}/index.js`);
files.push(`node_modules/dep${i}/lib/main.js`);
files.push(`SRC/Utils/Helper${i}.JS`);
files.push(`packages/app${i}/src/index.js`);
}
files.push('config.js');
files.push('index.js');
files.push('src/index.js');
files.push('src/main.js');
files.push('app/main.js');
return files;
}
const GENERATED_FILES = generatedFiles();
/**
* Targets swept across `GENERATED_FILES`: relative hits and misses,
* extensionless and explicit-extension spellings, `index.js` directories, bare
* and `node_modules` specifiers, scoped packages, case-differing paths, and
* plain misses. Most of them miss, which is both the realistic shape and the
* expensive one — a miss runs the cascade to completion.
*/
function generatedTargets(): string[] {
const targets: string[] = [];
for (let i = 0; i < 10; i++) {
targets.push(`./components/Widget${i}`);
targets.push(`./components/Widget${i}.js`);
targets.push(`../src/utils/helper${i}`);
targets.push(`src/utils/helper${i}`);
targets.push(`utils/helper${i}`);
targets.push(`helper${i}`);
targets.push(`mod${i}`);
targets.push(`lib/mod${i}.mjs`);
targets.push(`lib/legacy${i}`);
targets.push(`dep${i}`);
targets.push(`dep${i}/lib/main`);
targets.push(`SRC/Utils/Helper${i}`);
targets.push(`packages/app${i}/src`);
targets.push(`@scope/pkg${i}`);
targets.push(`ghost${i}/missing`);
targets.push(`node_modules/dep${i}`);
}
targets.push('config');
targets.push('index');
targets.push('src');
targets.push('src/main');
targets.push('app/main');
return targets;
}
const GENERATED_CASES: readonly Case[] = generatedTargets().map((target) => ({
name: `generated ${target}`,
files: GENERATED_FILES,
target,
fromFile: FROM_FILE,
}));
/** Hand-built cases, one per shape the index could have moved. */
const HAND_CASES: readonly Case[] = [
// ── relative specifiers: resolved by exact `Set.has`, never reach the index ──
{
name: 'relative hit',
files: ['src/util.js', 'src/main.js'],
target: './util',
fromFile: FROM_FILE,
},
{
name: 'relative hit with an explicit extension',
files: ['src/util.js', 'src/main.js'],
target: './util.js',
fromFile: FROM_FILE,
},
{
name: 'relative parent-directory hit',
files: ['shared/util.js', 'src/main.js'],
target: '../shared/util',
fromFile: FROM_FILE,
},
{
name: 'relative miss',
files: ['src/util.js', 'src/main.js'],
target: './missing',
fromFile: FROM_FILE,
},
{
name: 'relative directory index.js',
files: ['src/util/index.js', 'src/main.js'],
target: './util',
fromFile: FROM_FILE,
},
{
name: 'relative ESM specifier written as .js against a .mjs file',
files: ['src/util.mjs', 'src/main.js'],
target: './util.js',
fromFile: FROM_FILE,
},
// ── class A: repo-root files, unreachable as a proper suffix ────────────────
{
name: 'root-level file by bare specifier',
files: ['config.js', 'src/main.js'],
target: 'config',
fromFile: FROM_FILE,
},
{
name: 'root index.js by bare specifier',
files: ['index.js', 'src/main.js'],
target: 'index',
fromFile: FROM_FILE,
},
{
name: 'root-level file with an explicit extension',
files: ['config.js', 'src/main.js'],
target: 'config.js',
fromFile: FROM_FILE,
},
{
name: 'root-level .mjs by bare specifier',
files: ['esm.mjs'],
target: 'esm',
fromFile: FROM_FILE,
},
{
name: 'root-level .cjs by bare specifier',
files: ['legacy.cjs'],
target: 'legacy',
fromFile: FROM_FILE,
},
{
name: 'root-level .jsx by bare specifier',
files: ['Btn.jsx'],
target: 'Btn',
fromFile: FROM_FILE,
},
// ── class B: whole path vs proper suffix ────────────────────────────────────
{
name: 'whole-path candidate earlier in file order than a proper-suffix one',
files: ['src/util.js', 'vendor/src/util.js'],
target: 'src/util',
fromFile: FROM_FILE,
},
{
name: 'whole-path candidate later in file order than a proper-suffix one',
files: ['vendor/src/util.js', 'src/util.js'],
target: 'src/util',
fromFile: FROM_FILE,
},
{
name: 'whole-path hit at a long suffix vs proper-suffix hit at a short one',
files: ['node_modules/dep/lib/main.js', 'app/main.js'],
target: 'app/main',
fromFile: FROM_FILE,
},
{
name: 'whole-path hit at a long suffix vs proper-suffix hit at a short one, reversed',
files: ['app/main.js', 'node_modules/dep/lib/main.js'],
target: 'app/main',
fromFile: FROM_FILE,
},
{
name: 'whole path is the only candidate, and a proper suffix of it exists',
files: ['src/util.js', 'src/main.js'],
target: 'src/util',
fromFile: FROM_FILE,
},
{
name: 'whole-path directory index.js',
files: ['src/util/index.js', 'src/main.js'],
target: 'src/util',
fromFile: FROM_FILE,
},
{
name: 'whole-path candidate at an earlier extension than the proper-suffix one',
files: ['x/U.js', 'U.jsx'],
target: 'U',
fromFile: FROM_FILE,
},
{
name: 'whole-path candidate at an earlier extension than the proper-suffix one, reversed',
files: ['U.jsx', 'x/U.js'],
target: 'U',
fromFile: FROM_FILE,
},
{
name: 'root .js outranks a nested .mjs',
files: ['lib/mod.mjs', 'mod.js'],
target: 'mod',
fromFile: FROM_FILE,
},
// ── case-differing paths ────────────────────────────────────────────────────
{
name: 'case-differing whole path beats a case-exact proper suffix',
files: ['SRC/Util.js', 'other/src/util.js'],
target: 'src/util',
fromFile: FROM_FILE,
},
{
name: 'case-differing proper suffixes only',
files: ['other/SRC/Util.js', 'zz/deep/src/util.js'],
target: 'src/util',
fromFile: FROM_FILE,
},
{
name: 'case-exact file later in order than a case-differing one',
files: ['a/FOO.js', 'b/Foo.js'],
target: 'Foo',
fromFile: FROM_FILE,
},
{
name: 'case-exact file earlier in order than a case-differing one',
files: ['b/Foo.js', 'a/FOO.js'],
target: 'Foo',
fromFile: FROM_FILE,
},
// ── bare / node_modules specifiers ──────────────────────────────────────────
{
name: 'node_modules package by bare specifier',
files: ['node_modules/dep/index.js', 'src/main.js'],
target: 'dep',
fromFile: FROM_FILE,
},
{
name: 'node_modules deep path',
files: ['node_modules/dep/lib/main.js', 'src/main.js'],
target: 'dep/lib/main',
fromFile: FROM_FILE,
},
{
name: 'scoped package with no file anywhere',
files: ['src/util.js', 'src/main.js'],
target: '@scope/pkg',
fromFile: FROM_FILE,
},
{
name: 'dotted specifier is split on dots',
files: ['a/b.js', 'src/main.js'],
target: 'a.b',
fromFile: FROM_FILE,
},
// ── extension coverage ──────────────────────────────────────────────────────
{
name: 'nested .mjs by bare specifier',
files: ['lib/mod.mjs', 'src/main.js'],
target: 'lib/mod',
fromFile: FROM_FILE,
},
{
name: 'nested .cjs by bare specifier',
files: ['lib/legacy.cjs', 'src/main.js'],
target: 'lib/legacy',
fromFile: FROM_FILE,
},
{
name: 'nested .jsx by bare specifier',
files: ['comp/Btn.jsx', 'src/main.js'],
target: 'comp/Btn',
fromFile: FROM_FILE,
},
// ── degenerate inputs ───────────────────────────────────────────────────────
{ name: 'empty file set', files: [], target: 'anything', fromFile: FROM_FILE },
{ name: 'empty target', files: ['src/util.js'], target: '', fromFile: FROM_FILE },
{
name: 'plain miss',
files: ['src/util.js', 'src/main.js'],
target: 'nowhere/at/all',
fromFile: FROM_FILE,
},
{
name: 'importing file is itself at the repo root',
files: ['config.js', 'main.js'],
target: 'config',
fromFile: 'main.js',
},
];
// ─── runners ─────────────────────────────────────────────────────────────────
type Resolved = string | readonly string[] | null;
/**
* One legacy adapter and one current adapter per corpus, each over its own copy
* of the file set — the legacy single-slot cache and the current WeakMap are
* both keyed on the Set, so sharing one would let each observe the other's
* work.
*/
interface Runners {
readonly legacy: (target: string, fromFile: string) => Resolved;
readonly current: (target: string, fromFile: string) => Resolved;
readonly typescript: (target: string, fromFile: string) => Resolved;
}
function runnersFor(files: readonly string[]): Runners {
const legacyAdapter = legacyMakeJsResolveImportTarget();
const currentAdapter = makeJsResolveImportTarget();
const legacyFiles = new Set(files);
const currentFiles = new Set(files);
const typescriptFiles = new Set(files);
return {
legacy: (target, fromFile) => legacyAdapter(target, fromFile, legacyFiles, undefined),
current: (target, fromFile) => currentAdapter(target, fromFile, currentFiles, undefined),
typescript: (target, fromFile) =>
typescriptScopeResolver.resolveImportTarget(target, fromFile, typescriptFiles, undefined),
};
}
interface Outcome {
readonly name: string;
readonly target: string;
readonly legacy: Resolved;
readonly current: Resolved;
readonly typescript: Resolved;
}
/** Every case, resolved once. Built lazily and shared: the generated corpus is
* one file set across 165 targets, which is the shape a real pass has. */
const OUTCOMES: readonly Outcome[] = (() => {
const generated = runnersFor(GENERATED_FILES);
const handOutcomes = HAND_CASES.map((testCase) => {
const runners = runnersFor(testCase.files);
return {
name: testCase.name,
target: testCase.target,
legacy: runners.legacy(testCase.target, testCase.fromFile),
current: runners.current(testCase.target, testCase.fromFile),
typescript: runners.typescript(testCase.target, testCase.fromFile),
};
});
const generatedOutcomes = GENERATED_CASES.map((testCase) => ({
name: testCase.name,
target: testCase.target,
legacy: generated.legacy(testCase.target, testCase.fromFile),
current: generated.current(testCase.target, testCase.fromFile),
typescript: generated.typescript(testCase.target, testCase.fromFile),
}));
return [...handOutcomes, ...generatedOutcomes];
})();
const DIVERGENT: readonly Outcome[] = OUTCOMES.filter(
(outcome) => outcome.legacy !== outcome.current,
);
function describeDivergence(outcome: Outcome): string {
return `${outcome.name} :: ${JSON.stringify(outcome.legacy)} → ${JSON.stringify(outcome.current)}`;
}
/**
* Where in `suffixResolve`'s two nested loops a result was matched, as
* `pathPartIndex:extensionIndex`. Lower is more specific: a longer path suffix,
* or the same suffix at an earlier extension. Mirrors `resolveImportPath`'s
* own `pathParts` construction (dots become slashes only when the specifier
* carries no slash).
*/
function matchPosition(result: string, target: string): readonly [number, number] {
const pathLike = target.includes('/') ? target : target.replace(/\./g, '/');
const parts = pathLike.split('/').filter(Boolean);
const lower = result.toLowerCase();
const positions = parts.flatMap((_part, i) => {
const suffix = parts.slice(i).join('/').toLowerCase();
return EXTENSIONS.flatMap((ext, e) => {
const candidate = suffix + ext.toLowerCase();
const matches = lower === candidate || lower.endsWith(`/${candidate}`);
return matches ? [[i, e] as const] : [];
});
});
return positions[0] ?? [Number.MAX_SAFE_INTEGER, Number.MAX_SAFE_INTEGER];
}
/**
* Class A — a repo-root file, which has no proper suffix and so was
* unreachable through this leg at all. Every line goes `null → <root file>`,
* and the arm below enforces that shape rather than trusting the grouping.
*/
const CLASS_A_DIVERGENCES: readonly string[] = [
'root-level file by bare specifier :: null → "config.js"',
'root index.js by bare specifier :: null → "index.js"',
'root-level .mjs by bare specifier :: null → "esm.mjs"',
'root-level .cjs by bare specifier :: null → "legacy.cjs"',
'root-level .jsx by bare specifier :: null → "Btn.jsx"',
'importing file is itself at the repo root :: null → "config.js"',
'generated config :: null → "config.js"',
];
/**
* Class B — the scan skipped the whole-path candidate and landed on a shorter
* path suffix or a later extension instead. Every line moves from one file to
* another, toward the more specific match; `never answers at a less specific
* path-part / extension position` is the property behind that claim.
*
* `generated src/main` and `generated app/main` are the witnesses that settle
* the direction: both used to resolve into `node_modules`, because
* `node_modules/dep0/lib/main.js` is the first file in the corpus ending in
* `/main.js` and the scan never tried the two-segment suffix as a whole path.
*/
const CLASS_B_DIVERGENCES: readonly string[] = [
'whole-path candidate earlier in file order than a proper-suffix one :: "vendor/src/util.js" → "src/util.js"',
'whole-path hit at a long suffix vs proper-suffix hit at a short one :: "node_modules/dep/lib/main.js" → "app/main.js"',
'whole-path candidate at an earlier extension than the proper-suffix one :: "x/U.js" → "U.jsx"',
'whole-path candidate at an earlier extension than the proper-suffix one, reversed :: "x/U.js" → "U.jsx"',
'root .js outranks a nested .mjs :: "lib/mod.mjs" → "mod.js"',
'case-differing whole path beats a case-exact proper suffix :: "other/src/util.js" → "SRC/Util.js"',
'generated src/main :: "node_modules/dep0/lib/main.js" → "src/main.js"',
'generated app/main :: "node_modules/dep0/lib/main.js" → "app/main.js"',
'generated mod0 :: "lib/mod0.mjs" → "mod0.js"',
'generated mod1 :: "lib/mod1.mjs" → "mod1.js"',
'generated mod2 :: "lib/mod2.mjs" → "mod2.js"',
'generated mod3 :: "lib/mod3.mjs" → "mod3.js"',
'generated mod4 :: "lib/mod4.mjs" → "mod4.js"',
'generated mod5 :: "lib/mod5.mjs" → "mod5.js"',
'generated mod6 :: "lib/mod6.mjs" → "mod6.js"',
'generated mod7 :: "lib/mod7.mjs" → "mod7.js"',
'generated mod8 :: "lib/mod8.mjs" → "mod8.js"',
'generated mod9 :: "lib/mod9.mjs" → "mod9.js"',
];
/**
* The divergences this corpus produces, pinned old → new. A future edit that
* moves a DIFFERENT answer — or stops moving one of these — fails here rather
* than quietly shipping.
*/
const EXPECTED_DIVERGENCES: readonly string[] = [...CLASS_A_DIVERGENCES, ...CLASS_B_DIVERGENCES];
// ─── the differential ────────────────────────────────────────────────────────
describe('JavaScript import-target parity with the pre-index adapter (#2910)', () => {
it('agrees with the pre-index adapter on every case outside the pinned set', () => {
const pinned = new Set(EXPECTED_DIVERGENCES);
const unexpected = DIVERGENT.map(describeDivergence).filter(
(description) => !pinned.has(description),
);
expect(unexpected).toEqual([]);
});
it('moves exactly the pinned answers, and still moves all of them', () => {
expect(DIVERGENT.map(describeDivergence).sort()).toEqual([...EXPECTED_DIVERGENCES].sort());
});
it('never loses a match the scan found', () => {
const lost = OUTCOMES.filter(
(outcome) => outcome.legacy !== null && outcome.current === null,
).map(describeDivergence);
expect(lost).toEqual([]);
});
it('never answers at a less specific path-part / extension position', () => {
const lessSpecific = DIVERGENT.filter(
(outcome) => typeof outcome.legacy === 'string' && typeof outcome.current === 'string',
)
.map((outcome) => ({
outcome,
was: matchPosition(String(outcome.legacy), outcome.target),
now: matchPosition(String(outcome.current), outcome.target),
}))
.filter(({ was, now }) => now[0] > was[0] || (now[0] === was[0] && now[1] > was[1]))
.map(({ outcome, was, now }) => `${describeDivergence(outcome)} (${was} → ${now})`);
expect(lessSpecific).toEqual([]);
});
it('answers identically to the TypeScript adapter over the whole corpus', () => {
const disagreements = OUTCOMES.filter((outcome) => outcome.current !== outcome.typescript).map(
(outcome) =>
`${outcome.name} :: js=${JSON.stringify(outcome.current)} ts=${JSON.stringify(outcome.typescript)}`,
);
expect(disagreements).toEqual([]);
});
it('both classes are witnessed, and each line has its class’s shape', () => {
// Class A is `null → <a repo-root file>`: no slash in the new answer, which
// is the whole reason the scan could not reach it.
const misfiledA = CLASS_A_DIVERGENCES.filter((line) => !/ :: null → "[^/"]+"$/.test(line));
// Class B moves between two files; neither side is null.
const misfiledB = CLASS_B_DIVERGENCES.filter((line) => line.includes('null'));
expect(misfiledA).toEqual([]);
expect(misfiledB).toEqual([]);
expect(CLASS_A_DIVERGENCES.length).toBeGreaterThan(0);
expect(CLASS_B_DIVERGENCES.length).toBeGreaterThan(0);
});
it('resolves real JavaScript imports (the differential is not vacuous)', () => {
const resolveImportTarget = makeJsResolveImportTarget();
const files = new Set([
'src/main.js',
'src/util.js',
'src/components/Widget.jsx',
'src/models/index.js',
'lib/esm.mjs',
'lib/legacy.cjs',
'node_modules/dep/index.js',
]);
expect(resolveImportTarget('./util', FROM_FILE, files, undefined)).toBe('src/util.js');
expect(resolveImportTarget('./util.js', FROM_FILE, files, undefined)).toBe('src/util.js');
expect(resolveImportTarget('./components/Widget', FROM_FILE, files, undefined)).toBe(
'src/components/Widget.jsx',
);
expect(resolveImportTarget('./models', FROM_FILE, files, undefined)).toBe(
'src/models/index.js',
);
expect(resolveImportTarget('lib/esm', FROM_FILE, files, undefined)).toBe('lib/esm.mjs');
expect(resolveImportTarget('lib/legacy', FROM_FILE, files, undefined)).toBe('lib/legacy.cjs');
expect(resolveImportTarget('./nowhere', FROM_FILE, files, undefined)).toBeNull();
expect(resolveImportTarget('@scope/absent', FROM_FILE, files, undefined)).toBeNull();
});
});
// ─── the guard the defect needed ─────────────────────────────────────────────
describe('JavaScript import resolution never enters the linear suffix scan (#2910)', () => {
/**
* The control runs first and on purpose. `CountingSet` cannot see this defect
* — the scan walks the array the index materialized, not the Set — so an
* assertion of zero is worth nothing unless the same instrument is shown
* reading nonzero against the adapter that had the bug.
*/
it('the pre-index adapter scans linearly once per bare specifier; the current one never does', () => {
const files = new Set(GENERATED_FILES);
const bareTargets = generatedTargets().filter((target) => !target.startsWith('.'));
const legacyAdapter = legacyMakeJsResolveImportTarget();
linearScans.count = 0;
bareTargets.forEach((target) => legacyAdapter(target, FROM_FILE, files, undefined));
const legacyEntries = linearScans.count;
const currentAdapter = makeJsResolveImportTarget();
linearScans.count = 0;
bareTargets.forEach((target) => currentAdapter(target, FROM_FILE, new Set(files), undefined));
const currentEntries = linearScans.count;
expect(legacyEntries).toBe(bareTargets.length);
expect(currentEntries).toBe(0);
});
});

1223
gitnexus/test/unit/scope-resolution/php-import-target-parity.test.ts Normal file
View file

File diff suppressed because it is too large Load diff

259
gitnexus/test/unit/scope-resolution/python/python-import-probe-count.test.ts Normal file
View file

@ -0,0 +1,259 @@
/**
* Gate for the two probe-count defects in Python import resolution: the
* duplicated tail in `resolvePythonImportTarget`, and the missing O(1) proof of
* absence in front of `resolvePythonImportInternal`'s bare-import walk.
*
* ## What is being counted, and why not `CountingSet`
*
* `test/helpers/counting-file-set.ts` counts full TRAVERSALS of the file set.
* Neither defect here traverses it even once: both are made of `Set.has`
* probes, so the house instrument reads the same number before and after and
* cannot see either. This file counts the probes themselves — the one quantity
* both defects move — with a local `Set` subclass. Deterministic: the count is
* a function of the corpus and the spelling, never of wall time, and the same
* run reports the same number on any machine.
*
* ## Defect 1 — the duplicated tail (`named` / `alias` paid twice)
*
* `resolvePythonImportTarget` probes the package first with
* `targetIncludesImportedName: true`. That recursion differs from the outer
* frame in exactly one field, whose only effect is to skip the branch, so it
* runs the outer frame's whole tail — `resolvePythonImportInternal`, the
* relative gate, `hasRepoCandidate`, `resolveAbsoluteFromFiles` — on identical
* inputs. When it returned null the code FELL THROUGH and ran all of it again.
* Measured at four directory components: 24 probes, of which 12 were
* byte-identical repeats.
*
* The gate is that `from x import y` and `import x as y` issue exactly the
* probes `import x` issues. Stated as absolute numbers rather than as
* `named === namespace`, because an equality alone also passes if BOTH kinds
* start paying twice.
*
* ## Defect 2 — no proof of absence in front of the walk
*
* The bare walk probed `<ancestor>/<seg>.py` and `<ancestor>/<seg>/__init__.py`
* at every step from the importer's directory to the workspace root, for every
* single-segment import — including `import os`, `import sys` and every other
* distribution the repo does not vendor, where every probe is guaranteed to
* miss. `pythonSegmentAbsent` answers "no file anywhere can have either shape"
* in two Map lookups on the index the dotted tiers already build.
*
* The gate is that a provably-absent segment costs the SAME at depth 16 as at
* depth 1, paired with the control that a segment which survives the proof
* still walks and still costs more with depth — otherwise a resolver that
* simply stopped working would post a perfect flat line.
*/
import { describe, expect, it } from 'vitest';
import type { ParsedImport } from 'gitnexus-shared';
import { pythonScopeResolver } from '../../../../src/core/ingestion/languages/python/scope-resolver.js';
import { resolvePythonImportInternal } from '../../../../src/core/ingestion/import-resolvers/python.js';
import {
NO_PARSED_FILES,
pythonNamedImport,
pythonNamespaceImport,
} from '../../../helpers/counting-file-set.js';
const { resolveImportTarget } = pythonScopeResolver;
/** Counts `has` probes. `instanceof Set` still holds, which the adapter's
* structural narrowing needs. */
class ProbeCountingSet extends Set<string> {
probes = 0;
override has(value: string): boolean {
this.probes++;
return super.has(value);
}
}
/** `import x as y`. The third kind, and the only one of the three that is not
* shared with the memo guards — it reaches the same package-attribute probe
* `pythonNamedImport` does, and both arms below assert they cost the same. */
const aliasImport = (targetRaw: string): ParsedImport => ({
kind: 'alias',
localName: 'w',
importedName: 'Widget',
alias: 'w',
targetRaw,
});
const DEPTHS: readonly number[] = [1, 2, 4, 8, 16];
/**
* An importer `depth` directory components down.
*
* `far/away/probe.py` is out of the importer's ancestry, so a `probe` walk runs
* to the end and misses — and it makes `probe` a known basename, so the absence
* proof passes it through. `vendor/thing.py` makes `vendor/` a root directory
* prefix, so `hasRepoCandidate('vendor')` passes on its check (2) and the
* dotted target below reaches `resolveAbsoluteFromFiles` instead of being
* gated out.
*/
function corpus(depth: number): { files: readonly string[]; fromFile: string } {
const fromFile = `${Array.from({ length: depth }, (_, i) => `d${i}`).join('/')}/mod.py`;
return {
files: [fromFile, 'zz/keep.py', 'far/away/probe.py', 'vendor/thing.py'],
fromFile,
};
}
function probeCount(
mkImport: (targetRaw: string) => ParsedImport,
depth: number,
targetRaw: string,
): { probes: number; result: string | readonly string[] | null } {
const { files, fromFile } = corpus(depth);
const set = new ProbeCountingSet(files);
const result = resolveImportTarget(targetRaw, fromFile, set, undefined, {
parsedFiles: NO_PARSED_FILES,
parsedImport: mkImport(targetRaw),
});
return { probes: set.probes, result };
}
/** Exists as a basename, so the absence proof passes it through to the walk. */
const PRESENT_TARGET = 'probe';
const PRESENT_RESULT = 'far/away/probe.py';
/** No file has basename `ghostmod.py` and no directory is named `ghostmod`. */
const ABSENT_TARGET = 'ghostmod';
/**
* `2 + 2 x depth` probes in the bare walk (proximity, then two per ancestor
* step including the workspace root), then `2 + depth` in the dotted tier below
* it (two direct root probes, then one per ancestor — only the module form,
* because no `probe/__init__.py` exists anywhere). `4 + 3 x depth`.
*/
const PRESENT_PROBES: readonly number[] = [7, 10, 16, 28, 52];
/** Two: the dotted tier's direct workspace-root probes. The bare walk issues
* NONE — it is retired before the proximity check. */
const ABSENT_PROBES = 2;
/**
* A DOTTED target that passes `hasRepoCandidate` (its leading segment `vendor`
* is a root directory prefix), reaches `resolveAbsoluteFromFiles`, walks the
* whole ancestor chain and still resolves to nothing — because the only
* `probe.py` in the workspace does not end with `/vendor/probe.py`.
*
* This is the shape the duplicated tail actually costs on, and the reason the
* single-segment arm above cannot see it: a single-segment target that survives
* the absence proof is always answered by the suffix fallback, so its
* `packageTarget` is never null and the fallthrough never fires. A dotted one
* can miss, and missing is precisely when the old code ran the tail again.
*/
const DOTTED_TARGET = 'vendor.probe';
/** `2 + depth`: two direct root probes, then one module probe per ancestor. */
const DOTTED_NAMESPACE_PROBES: readonly number[] = [3, 4, 6, 10, 18];
/**
* `named`/`alias` legitimately add TWO — the submodule probe for
* `vendor.probe.Widget`, a different target with its own direct root checks.
* What they must NOT add is a third component: another whole copy of the
* package tail. With the fallthrough restored these read [8, 10, 14, 22, 38].
*/
const DOTTED_SUBMODULE_PROBES: readonly number[] = [5, 6, 8, 12, 20];
describe('Python import probe count', () => {
it.each([
{ kind: 'import x (namespace)', mkImport: pythonNamespaceImport },
{ kind: 'from x import y (named)', mkImport: pythonNamedImport },
{ kind: 'import x as y (alias)', mkImport: aliasImport },
])('costs the same for every import KIND — single-segment, resolving — $kind', ({ mkImport }) => {
const counted = DEPTHS.map((depth) => probeCount(mkImport, depth, PRESENT_TARGET));
expect(counted.map((c) => c.probes)).toEqual(PRESENT_PROBES);
// Non-vacuity: a probe count is equally flattering to a resolver that
// resolves nothing.
expect(counted.map((c) => c.result)).toEqual(DEPTHS.map(() => PRESENT_RESULT));
});
it.each([
{
kind: 'import x (namespace)',
mkImport: pythonNamespaceImport,
expected: DOTTED_NAMESPACE_PROBES,
},
{
kind: 'from x import y (named)',
mkImport: pythonNamedImport,
expected: DOTTED_SUBMODULE_PROBES,
},
{ kind: 'import x as y (alias)', mkImport: aliasImport, expected: DOTTED_SUBMODULE_PROBES },
])(
'runs the package tail ONCE for a dotted target that misses — $kind',
({ mkImport, expected }) => {
const counted = DEPTHS.map((depth) => probeCount(mkImport, depth, DOTTED_TARGET));
// The duplicated-tail gate. Restoring the fallthrough adds a second copy of
// the namespace column to the two submodule rows.
expect(counted.map((c) => c.probes)).toEqual(expected);
expect(counted.map((c) => c.result)).toEqual(DEPTHS.map(() => null));
},
);
it.each([
{ kind: 'import x (namespace)', mkImport: pythonNamespaceImport },
{ kind: 'from x import y (named)', mkImport: pythonNamedImport },
{ kind: 'import x as y (alias)', mkImport: aliasImport },
])('retires a provably absent segment in a CONSTANT probe count — $kind', ({ mkImport }) => {
const counted = DEPTHS.map((depth) => probeCount(mkImport, depth, ABSENT_TARGET));
// The gate: flat in depth. Without the proof of absence this is
// `4 + 4 x depth` for a miss, i.e. 8 at depth 1 and 68 at depth 16.
expect(counted.map((c) => c.probes)).toEqual(DEPTHS.map(() => ABSENT_PROBES));
expect(counted.map((c) => c.result)).toEqual(DEPTHS.map(() => null));
});
it('the counter can see depth — the flat line above is the proof, not the instrument', () => {
// Control for the arm above: the same instrument, the same corpus, the same
// depths, one different spelling — and the count triples across the range.
// So a flat line means the walk was skipped, not that nothing is counted.
const present = DEPTHS.map(
(depth) => probeCount(pythonNamedImport, depth, PRESENT_TARGET).probes,
);
expect(present).toEqual(PRESENT_PROBES);
expect(new Set(present).size).toBe(DEPTHS.length);
});
/**
* The two inputs `pythonSegmentAbsent` refuses to answer for. Both must keep
* probing exactly as before; a proof of absence that fires on either would
* silently stop resolving real files.
*/
it.each([
{
why: 'the EMPTY segment, module form — basename `.py` is indexed normally',
files: ['a/b/.py', 'a/b/mod.py'],
fromFile: 'a/b/mod.py',
importPath: '',
expected: 'a/b/.py',
},
{
// THE reason the empty-segment carve-out exists. The probe for an empty
// segment is `<prefix>/__init__.py`, whose parent directory name is empty
// — exactly the case the `byInitParent` build skips. So the bucket cannot
// witness this file, and its absence is not proof of the file's absence.
why: 'the EMPTY segment, package form under a doubled separator — `byInitParent` skips it',
files: ['a//__init__.py', 'a/b/mod.py'],
fromFile: 'a/b/mod.py',
importPath: '',
expected: 'a//__init__.py',
},
{
why: 'the EMPTY segment, package form at the filesystem root',
files: ['/__init__.py', 'a/b/mod.py'],
fromFile: 'a/b/mod.py',
importPath: '',
expected: '/__init__.py',
},
{
why: 'a segment carrying a BACKSLASH — the buckets are keyed on normalized paths',
files: ['a\\b.py', 'x/mod.py'],
fromFile: 'x/mod.py',
importPath: 'a\\b',
expected: 'a\\b.py',
},
])('still resolves what the proof of absence cannot rule out — $why', (row) => {
expect(resolvePythonImportInternal(row.fromFile, row.importPath, new Set(row.files))).toBe(
row.expected,
);
});
});

291
gitnexus/test/unit/scope-resolution/python/python-importer-ancestors.test.ts Normal file
View file

@ -0,0 +1,291 @@
/**
* Gate for #2913: Python import resolution must not scale with the importer's
* path depth.
*
* `hasRepoCandidate` and `resolveAbsoluteFromFiles` each rebuilt one ancestor
* prefix per component of the importer's directory, on EVERY import — a cost
* proportional to depth (quadratic in characters) on an index that is itself
* depth-free. `importerAncestors` builds that chain once per importer DIRECTORY
* and stores it in `PythonFileIndex.ancestorsByDir`, which lives inside the
* per-file-set value and so dies with the pass.
*
* ## Why this is not `CountingSet`
*
* `test/helpers/counting-file-set.ts` is the house instrument for every other
* import-target reuse guard, and it cannot see this one. It counts TRAVERSALS
* of the file set; the ancestor chain is derived from the `fromFile` STRING and
* touches the set only through `Set.has`, whose argument and count are byte-for-
* byte identical before and after the hoist. The same is true of a `has`-call
* counter: memoizing a string that is then concatenated into the same probe
* changes no probe. A pure hoist is invisible to any instrument that watches
* only the resolver's inputs — so this file watches the memo, which is the one
* place the hoist is observable, and watches it THROUGH the production adapter
* (`pythonScopeResolver.resolveImportTarget`, the surface the orchestrator
* calls) rather than through the resolver function the parity test uses.
*
* The gate is a COUNT, not a timing budget: `ancestorsByDir.size` after N
* imports from D directories must be D, for every N. That is exactly "the
* ancestor-prefix work is O(1) amortized after the first import from a given
* directory", stated as a number a test can assert. It is paired with a
* reference-identity assertion, because a memo that stores a FRESH chain on
* every import posts the same size while doing all of the work again.
*
* Nothing ships for this file to read. `getPythonFileIndex` is the pass's own
* index and `ancestorsByDir` is the memo itself; the export is visibility, not
* a counter — the surface #2909 deleted was ~30 lines of production code whose
* only caller was a test. The module barrel (`languages/python/index.ts`) is
* unchanged, so the index stays out of the package's public API.
*
* The `legacy*` helpers below are verbatim copies of the pre-#2913 inline code,
* in the house style of `import-target-index-parity.test.ts`: they are the
* specification, and the memo agreeing with them is what makes this a hoist
* rather than a behaviour change.
*
* The four memo arms live in `counting-file-set.ts` beside the other
* import-target scaffolding, because this guard and the bare-prefix one
* (`test/unit/import-resolvers/python-importer-prefixes.test.ts`) are the same
* suite over the same importer corpus once four values are named (the memo, the
* drive, the legacy builder, the hit). The two CHAINS still differ — this one
* drops the empty components an absolute path or a doubled separator produces
* and the other keeps them — so `legacyChain` stays per-guard and the shared
* path-shape table names shapes rather than expectations.
*/
import { describe, expect, it } from 'vitest';
import type { ParsedImport } from 'gitnexus-shared';
import { pythonScopeResolver } from '../../../../src/core/ingestion/languages/python/scope-resolver.js';
import { getPythonFileIndex } from '../../../../src/core/ingestion/import-resolvers/python-file-index.js';
import {
IMPORTER_PATH_SHAPES,
countedParsedFiles,
expectDistinctFileSetsGetOwnChainMemo,
expectMemoizedChainMatchesLegacy,
expectOneChainPerImporterDir,
expectSameChainObjectReused,
sortedStrings,
type ChainMemoArm,
type ChainMemoResult,
} from '../../../helpers/counting-file-set.js';
const { resolveImportTarget } = pythonScopeResolver;
// ─── verbatim pre-#2913 implementations ──────────────────────────────────────
/** The `ancestorPrefixes` array `hasRepoCandidate` used to build per import. */
function legacyAncestorPrefixes(fromFile: string, leadingSegment: string): string[] {
const importerDir = fromFile.replace(/\\/g, '/').split('/').slice(0, -1).join('/');
const dirParts = importerDir ? importerDir.split('/').filter(Boolean) : [];
const ancestorPrefixes: string[] = [];
for (let i = dirParts.length; i > 0; i--) {
ancestorPrefixes.push(`${dirParts.slice(0, i).join('/')}/${leadingSegment}/`);
}
return ancestorPrefixes;
}
/** The ancestors `resolveAbsoluteFromFiles`'s walk used to build per import. */
function legacyAncestorChain(fromFile: string): string[] {
const importerDir = fromFile.replace(/\\/g, '/').split('/').slice(0, -1).join('/');
const chain: string[] = [];
const dirParts = importerDir ? importerDir.split('/').filter(Boolean) : [];
for (let i = dirParts.length; i > 0; i--) {
chain.push(dirParts.slice(0, i).join('/'));
}
return chain;
}
/** The forward, no-early-exit `dirPrefixes` build. */
function legacyDirPrefixes(files: readonly string[]): Set<string> {
const dirPrefixes = new Set<string>();
for (const raw of files) {
const norm = raw.replace(/\\/g, '/');
if (!norm.endsWith('.py')) continue;
const lastSlash = norm.lastIndexOf('/');
for (let i = 0; i <= lastSlash; i++) {
if (norm[i] === '/') dirPrefixes.add(norm.slice(0, i + 1));
}
}
return dirPrefixes;
}
/**
* The specification of `nestedDirNames`, read off the legacy prefix set: the
* name of every directory prefix that has a NON-EMPTY parent, which is exactly
* the set of `${ancestor}/${segment}/` shapes the old ancestor loop could ever
* match. A segment outside it made the old loop run to completion and answer
* false; the new code answers false without running it.
*/
function specNestedDirNames(dirPrefixes: ReadonlySet<string>): Set<string> {
const names = new Set<string>();
for (const prefix of dirPrefixes) {
const dir = prefix.slice(0, -1);
const slash = dir.lastIndexOf('/');
if (slash > 0) names.add(dir.slice(slash + 1));
}
return names;
}
// ─── the workspace the adapter is driven against ─────────────────────────────
/**
* `outer/nested/` is what makes `nested` a NESTED directory name without making
* `nested/` a root prefix, so `hasRepoCandidate('nested')` has to reach the
* ancestor walk instead of answering from check (1) or (2). `one.py` repeats
* across three directories so the `outer.one` spelling reaches
* `resolveAbsoluteFromFiles`'s walk too — both memo call sites, one corpus.
*/
const WORKSPACE: readonly string[] = [
'outer/nested/mod.py',
'svc/a/one.py',
'svc/a/two.py',
'svc/b/one.py',
'deep/x/y/z/one.py',
'root.py',
];
/**
* One import that must resolve, so a memo count is never the count of an
* adapter that has stopped resolving anything (the pairing rule every guard in
* this family states). `outer/` is a root directory prefix, so the gate passes
* on check (2) and the direct workspace-root hit answers it.
*/
const HIT_TARGET = 'outer.nested.mod';
const HIT_RESULT = 'outer/nested/mod.py';
/**
* Drives the ORCHESTRATOR ADAPTER `perImporter` times from `fromFile`, with two
* spellings that between them enter the memo from both call sites:
* - `nested.ghost{i}` — reaches `hasRepoCandidate`'s ancestor walk and misses.
* Spelled differently every iteration, so nothing upstream can answer it
* from a per-target memo.
* - `outer.one` — passes the gate on check (2) (`outer/` is a root directory
* prefix), misses the direct workspace-root hit, and reaches
* `resolveAbsoluteFromFiles`'s ancestor walk. NOT varied per iteration:
* `one.py` has to be a real basename somewhere or the walk is skipped
* before it starts, and the Python chain keeps no per-target cache, so a
* repeated spelling really is re-resolved.
* …then the one spelling that must resolve.
*/
function driveImporter(
files: Set<string>,
fromFile: string,
perImporter: number,
): ChainMemoResult[] {
const out: ChainMemoResult[] = [];
for (let i = 0; i < perImporter; i++) {
out.push(resolveImportTarget(`nested.ghost${i}`, fromFile, files, undefined, undefined));
out.push(resolveImportTarget('outer.one', fromFile, files, undefined, undefined));
}
out.push(resolveImportTarget(HIT_TARGET, fromFile, files, undefined, undefined));
return out;
}
const ancestorArm: ChainMemoArm = {
memoOf: (files) => getPythonFileIndex(files).ancestorsByDir,
drive: driveImporter,
legacyChain: legacyAncestorChain,
hitResult: HIT_RESULT,
};
describe('Python importer-ancestor memo (#2913)', () => {
it.each([
{ perImporter: 1, label: 'one import per importer' },
{ perImporter: 40, label: 'forty imports per importer' },
])('holds one chain per importer DIRECTORY, not per import — $label', ({ perImporter }) => {
expectOneChainPerImporterDir(ancestorArm, new Set(WORKSPACE), perImporter);
});
it('reuses the SAME chain object, rather than rebuilding and re-storing it', () => {
expectSameChainObjectReused(ancestorArm, new Set(WORKSPACE));
});
it.each(IMPORTER_PATH_SHAPES)(
'memoizes the chain the pre-#2913 code built — $why',
({ fromFile }) => {
const chain = expectMemoizedChainMatchesLegacy(ancestorArm, new Set(WORKSPACE), fromFile);
// Both consumers' chains, from the one memo: `resolveAbsoluteFromFiles`
// walked these directories, `hasRepoCandidate` walked the same directories
// with `/<segment>/` appended.
expect(chain.map((ancestor) => `${ancestor}/nested/`)).toEqual(
legacyAncestorPrefixes(fromFile, 'nested'),
);
},
);
it.each([
{ why: 'relative paths sharing directories', files: WORKSPACE },
{ why: 'absolute paths', files: ['/repo/pkg/__init__.py', '/repo/vendor/pkg/thing.py'] },
{ why: 'a doubled separator', files: ['a//b/x.py', 'a//b/y.py'] },
{ why: 'Windows separators', files: ['a\\b\\x.py', 'a\\b\\c\\y.py'] },
{ why: 'root-level files only', files: ['x.py', 'y.py'] },
{ why: 'a polyglot corpus', files: ['a/b/x.py', 'a/b/x.ts', 'c/d/e/f/g/h.py', 'c/d/n.go'] },
{ why: 'one deep directory, many files', files: ['a/b/c/d/e/1.py', 'a/b/c/d/e/2.py'] },
])('builds the same prefix set as the pre-#2913 forward scan — $why', ({ files }) => {
const index = getPythonFileIndex(new Set(files));
const legacy = legacyDirPrefixes(files);
// The build now walks separators from the deepest outward and stops at
// the first prefix already present. Skipping the rest is only sound
// because a prefix is always stored with all of its own ancestors.
expect(sortedStrings(index.dirPrefixes)).toEqual(sortedStrings(legacy));
expect(sortedStrings(index.nestedDirNames)).toEqual(sortedStrings(specNestedDirNames(legacy)));
});
/**
* The same defect on the OTHER collection the orchestrator threads.
* `pythonFileExportsName` opened with `parsedFiles.find(...)`, an O(files)
* scan run for every import whose package probe resolves — which on a repo
* where `from pkg import X` usually resolves is most imports.
*
* `import-target-index-reuse.contract.test.ts` measures this channel for
* every language, but its Python fixture has exactly ONE resolving import, so
* its equality arm passes whether the scan is memoized or not. This arm is
* the one that bites: every import resolves through the probe, so a per-import
* `find` makes the read count grow with the import count.
*/
it.each([
{ imports: 2, label: 'two imports' },
{ imports: 200, label: 'two hundred imports' },
])('reads the parsed workspace once per PASS, not once per import — $label', ({ imports }) => {
const modules = Array.from({ length: 30 }, (_, i) => `pkg/m${i}.py`);
const paths = ['pkg/__init__.py', ...modules, 'app/main.py'];
const workspace = countedParsedFiles(paths);
const files = new Set(paths);
const resolved: ChainMemoResult[] = [];
for (let i = 0; i < imports; i++) {
const targetRaw = `pkg.m${i % modules.length}`;
const parsedImport: ParsedImport = {
kind: 'named',
localName: 'Widget',
importedName: 'Widget',
targetRaw,
};
resolved.push(
resolveImportTarget(targetRaw, 'app/main.py', files, undefined, {
parsedFiles: workspace.parsedFiles,
parsedImport,
}),
);
}
// One pass over the parsed workspace, whatever the import count. A `find`
// per import reads 32 for two imports and thousands for two hundred.
expect(workspace.reads()).toBe(paths.length);
// ...and the leg was really entered, so the count is not a perfect zero
// posted by a resolver that returned early.
expect(workspace.reads()).toBeGreaterThan(0);
expect(resolved[0]).toBe('pkg/m0.py');
expect(resolved[resolved.length - 1]).toBe(`pkg/m${(imports - 1) % modules.length}.py`);
});
it('gives a distinct file set its own memo (no leak across passes)', () => {
const a = new Set(WORKSPACE);
const b = new Set(WORKSPACE);
expectDistinctFileSetsGetOwnChainMemo(ancestorArm, a, b, 2);
// The whole per-file-set index, not only the memo inside it: the WeakMap is
// keyed on the Set, so two Sets can never share one index.
expect(getPythonFileIndex(a)).not.toBe(getPythonFileIndex(b));
});
});