scriptoriumadmin/GitNexus
1
0
Fork 0
mirror of https://github.com/abhigyanpatwari/GitNexus.git synced 2026-08-28 05:25:25 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 9

perf(import-target): index the workspace once per run for go/csharp/dart/ruby (#2898)

Browse source 
* perf(import-target): index the workspace once per run for go/csharp/dart/ruby

Four import-target resolvers answered their lookups with a full
`allFilePaths` scan per import, making resolution O(imports x files):

- go (#2877): `findRootPackageFiles` / `findAllFilesInPkgDir`, the latter
  once per path segment on the GOPATH fallback. Most Go imports are
  external, so the whole cascade ran to completion before returning null.
- csharp (#2878): the no-csproj leg took the raw Set past the memoized
  index the csproj leg was already using - up to eight passes for a
  four-segment `using`.
- dart (#2879): one scan per candidate path, and for an external package
  both candidates miss, so both always ran to completion.
- ruby (#2880): a complete `buildSuffixIndex` rebuilt and discarded per
  `require` - every require paid to index every file in the repo.

Each now reads an index memoized on the `allFilePaths` Set identity, the
shape `getPythonFileIndex` (#1918) and csharp's own `getWorkspaceFileIndex`
(#1881) already used. Two shared modules back them:

- `workspace-file-index.ts`: normalized list + `SuffixIndex` + a
  normalized->raw map, for csharp and ruby.
- `package-dir-index.ts`: "which files live directly inside a directory
  ending with <path>", for go and csharp. Candidates are bucketed by the
  directory's last segment rather than by indexing every directory suffix,
  which would cost O(files x depth) entries at kernel scale (#2649).

Behaviour is unchanged, including the tie-breaks that are expressed only
through Set-iteration order and `indexOf` positions: the go root leg stays
sorted and its package leg stays unsorted, the first-occurrence rule that
excludes a directory nested inside a same-named directory is preserved,
csharp's whole-path match still beats an earlier suffix match, and dart
still tries `lib/<rel>` fully before bare `<rel>` and matches raw paths.

Verified two ways. `import-target-index-parity.test.ts` keeps verbatim
copies of the pre-change implementations and diffs against them over a
deterministic corpus plus hand-built layouts for each tie-break; six
mutations of the new code were confirmed to fail it. Separately, the bench
corpus produces byte-identical fingerprints against the pre-change
resolvers at both 400 and 1600 files.

`bench/import-target/measure.mjs` gates both arms in CI: per-language
output fingerprints, a scaling budget (measured 0.98-1.12 here, 3.32-4.10
against the pre-change scans), and the corpus shape, so the corpus cannot
be shrunk below the size the scaling arm needs and still print PASS.

Closes #2877
Closes #2878
Closes #2879
Closes #2880

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BCptYZWRgnnJ821rzebQyj

* perf(import-target): cover kotlin, and add depth + absolute-cost arms

#2872 landed the same index hoist for Kotlin while this branch was open.
Fold it into the shared measures so all five resolvers are gated on one
corpus, and adopt the two arms that PR's review proved a scaling ratio
alone cannot carry.

- `bench/import-target/measure.mjs` gains a kotlin arm: a Gradle-shaped
  corpus with per-module source roots over one package namespace, `.kt`
  and `.kts` stems, a nested same-name package directory, and a share of
  wildcard `.*` imports so the package fan-out tier — the only tier whose
  output is order-bearing — is inside the fingerprint.

- `depth_ratio`: deep arm at a FIXED file count with ~6x the path
  components. `scaling_ratio` divides the file count out, so it is
  scale-invariant and structurally cannot see a cost that grows with path
  depth instead, and `buildSuffixIndex` (C#, Ruby) and Kotlin's
  `suffixByStem` each emit one entry per component. Measured: go 0.98,
  dart 0.88 (depth-free indexes), ruby 1.48, kotlin 2.20, csharp 3.45 —
  which is why the budget is per language. One global budget would have
  to sit at 5.0 and would let Dart go 0.88 -> 4.9 unnoticed.

- `small_ms_ceiling`: an absolute bound at 4x the measured arm, because a
  constant-factor regression that grows both scale arms equally passes
  every ratio.

- The deep arm must resolve exactly what the small arm resolves. Padding
  was supposed to change depth and nothing else; a deep arm that stopped
  resolving would be timing the null path.

The five fingerprints are unchanged by this commit - verified against the
previous baseline before rewriting it, so adding the kotlin arm and the
deep scale did not perturb the four languages' output.

Kotlin joins the Set-iteration counter in
`import-target-index-parity.test.ts` too. Its own guard
(`kotlin-import-index-reuse.test.ts`) counts index BUILDS, which a scan
added beside a reused index does not move.

That counter is also the only DETERMINISTIC guard against a reintroduced
scan, and this commit documents why rather than pretending otherwise: a
full workspace scan on 1-in-32 imports was measured to pass every timing
arm here (dart, 1.458 scaling against a 1.8 budget, 1.736 ms against a
4 ms ceiling) while the counter reads 14 instead of 1. Tightening the
ceilings toward the noise floor to chase that case would only buy flaky
CI.

`bench/kotlin-import-target/` stays: it fingerprints both file-set
iteration orders and probes the four-tier cascade shape by shape, neither
of which this corpus does.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BCptYZWRgnnJ821rzebQyj

* perf(import-target): merge matching package dirs in one pass

`filesDirectlyInPkgDir` re-spread its accumulator once per matching
directory, costing O(files x dirs^2) copies per import. On a Go monorepo
where many services carry the same package directory (`svcN/internal/pkg`,
which Go's GOPATH cascade queries by two-segment tail) that made the index
SLOWER than the scan it replaced: 13.4x at 1600 matching directories.

Append into one array and sort once. Measured against a verbatim copy of
the pre-change scan, output byte-identical at every k:

  k=200   1400 files   old 0.126 ms   was 0.169 ms   now 0.042 ms
  k=800   5600 files   old 0.457 ms   was 3.002 ms   now 0.185 ms
  k=1600 11200 files   old 0.960 ms   was 12.890 ms  now 0.232 ms

The index now beats the scan by 2.5-4.1x on this shape instead of losing
to it by up to 13x.

Also drop the min-`ord` comparison in `firstFileDirectlyInPkgDir`: the
build loop appends a directory to its last-segment bucket the moment it
accepts that directory's first file, so bucket order already IS ascending
first-file-`ord` order and the first hit is the minimum. Differentially
verified at 0 divergences. The invariant, and the build-loop edits that
would silently break it, are now recorded at the early return.

Type the index containers as deeply readonly so Go's deliberate
`[...rootFiles].sort()` copy is compile-enforced rather than
comment-enforced, and correct the header's claim that a polyglot repo
"never pays" -- only the stored index is per-language.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013rLy781K8E3rRHFGEe1khJ

* test(import-target): guard index reuse at the adapter boundary

`workspace-file-index.ts` documented the hazard as "a defensive
`new Set(allFilePaths)` in an ADAPTER" -- the bug #1918 shipped -- and
named the unit parity test as the guard. It is not: that test imports the
resolvers directly, while production reaches them through
`<lang>ScopeResolver.resolveImportTarget`. Inserting the copy at
`go/scope-resolver.ts:31`, `csharp:35`, `dart:189` and `ruby:268` left the
parity test 28/28 green and `measure.mjs --check` PASS in all four cases.
Kotlin and Python already had adapter-level guards; go/csharp/dart/ruby
had none.

Add `test/integration/<lang>-import-index-reuse.test.ts` for the four,
mirroring the Kotlin/Python precedent: resolve through the scope resolver,
assert the file set is traversed once (twice for C#, which builds two
indexes), and pair every count with a result assertion so a count of 1
cannot be the count of an adapter that resolves nothing. Each was proven
to fail under the copy it exists to catch:

  go     expected 600 to be 1     dart   expected 600 to be 1
  ruby   expected 400 to be 1     csharp expected 600 to be 2

`CountingSet` moves to `test/helpers/counting-file-set.ts` and now counts
`forEach`, `values`, `keys` and `entries` as well as `[Symbol.iterator]`.
It missed a rescan spelled `allFilePaths.forEach(...)` entirely; with the
overrides that mutation reads 14 instead of 1.

Four fixtures that pinned the guard next door, each now shown to kill its
mutation:
- the Dart "matched RAW" case used a forward-slash target, so the basename
  bucket missed before the raw comparison was reached and it asserted
  `null === null`. A positive twin carrying the backslash in the TARGET
  catches both half-mutations.
- no C# or Ruby target addressed the corpus's `win\dir\thing` file, so
  deleting the backslash normalization in `workspace-file-index.ts` passed
  both gates. Now 4 failures.
- `normToRaw`'s first-wins rule had no normalization twin in any corpus.
- the Go nested-package fixture was decided by the `endsWith` half and
  never reached the first-occurrence branch its title names; addressing
  the directory as a single segment makes it reach it.

The parity test's own docstring no longer claims the scan count is a
complete census -- it names the three materialized arrays it cannot see.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013rLy781K8E3rRHFGEe1khJ

* test(import-target): assert every scale, add collide and retained-heap arms

Three holes in the gate this PR ships as its own proof.

1. `--check` computed three fingerprints per language, stored all three,
   and compared one. `DEEP_PAD = 16 -> 0` deleted the entire depth arm and
   still printed PASS, because depth padding is count-neutral by design so
   no asserted number moved. Assert `fingerprint` per scale, and assert
   `deep.fingerprint !== small.fingerprint` so the padding's EFFECT is
   pinned, not just its output.

2. The corpus minted per-index directory names (`src/pkg${d}`,
   `src/Ns${d}`, `lib/feature${d}`), so max last-segment bucket and max
   matching dirs were both 1 -- and bucket cardinality is the only
   non-constant term the index has. The `dirCount > 1` merge branch had
   never executed in any arm. Add a `collide` arm on shared-leaf layouts
   with identical files/imports/resolved counts; it reaches 9,269
   multi-directory merges per run, up to 34 directories at once. Go and
   C#/Dart legitimately score above the linear budget there and get their
   own; Ruby and Kotlin stay at 1.8 because their keyed maps are
   collision-immune and that immunity is the assertion.

3. No arm measured memory, while the C# no-csproj leg newly retains an
   O(files x depth) suffix index. Add a retained-heap arm on the
   `bench/cfg` pattern, including its loud failure when `--expose-gc` is
   missing rather than a silent skip. Measured at 32k files:
   csharp 73.62 MiB, ruby 55.26 MiB. Ceiling is 1.5x, NOT the 4x the
   timing arms use -- the measurement is byte-stable to 0.00085% across
   processes, so 4x would be throwing away the gate. `_arms_note` records
   why, so nobody harmonises it back.

`depth_ratio`, added by this PR, flaked ~1-in-20: go peaked at 1.748 and
dart at 2.043 against a 1.6 budget, both ratios of two sub-3 ms minima.
Fixed at the estimator, not the threshold -- REPS 5 -> 15, matching
`bench/cfg`, `schema-pairs` and `callable-value-flow` (5 was the lowest in
the repo; the sibling `kotlin-import-target` uses 7, which was not enough
here). 22/22 PASS, every arm now at 70-78% of its budget with a <=1.26x
swing. No budget was widened; the distributions are recorded in
`_arms_note` so the headroom is visibly earned.

Three copies of the same overclaim corrected: the parity test NARROWS the
1-in-32 blind spot, it does not close it -- it watches the Set while the
resolvers hold materialized arrays. `_floor` no longer claims its ratios
"match" the issues' (different corpora, both quadratic).

The step moves to the END of the benchmarks job and runs with
`--expose-gc`. A failing step aborts every step after it (#2895), so the
newest, least-proven gate must not sit ahead of eight established ones.

All five output fingerprints are byte-identical to before this session --
the proof that every change here was behaviour-preserving.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013rLy781K8E3rRHFGEe1khJ

* docs(import-target): point the reuse contract at the guard that guards it

`workspace-file-index.ts` told callers the unit parity test guards the
adapter-copy hazard. It does not -- it never crosses the adapter. Name
both layers and say which catches what: the per-language
`test/integration/<lang>-import-index-reuse.test.ts` files at the adapter
boundary, the parity test for a rescan reintroduced inside a resolver.

The C# namespace-dir index comment named `findDirectChild`, which this PR
deleted; it feeds `firstFileDirectlyInPkgDir` now.

Drop `GoResolveContext`, dead since the legacy call-resolution DAG was
removed in #942 -- zero importers, and `gitnexus`'s package.json declares
no `main`, `exports` or `types`, so it is not a published surface.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013rLy781K8E3rRHFGEe1khJ

* refactor(import-target): quality pass over the review-round changes

Four cleanup lanes (reuse / simplification / efficiency / altitude) over the
previous four commits. No behaviour change anywhere: all 25 bench cells
(5 languages x 5 arms) are byte-identical on files, imports, resolved,
distinct_outcomes and fingerprint, re-verified after each individual edit.

**Restores a fast path the last commit lost.** Fixing the O(k^2) accumulator
made the SINGLE-directory case — the overwhelmingly common one — copy the
bucket where the original aliased it: measured 1.11x slower at 4 files/dir
rising to 1.72x at 128. Holding the first bucket by reference and promoting to
an accumulator only when a second directory appears is 0.65-0.97x of the
previous code at dirCount=1 and parity at dirCount=64. 176-case differential,
0 divergences.

**`sortedRootFiles` accessor.** `rootFiles` was the only index container read
directly from outside the module. `readonly` is erased at runtime and
`Array.isArray` widens it back, so the copy rule now lives with the code that
owns the invariant instead of at the call site. No `Object.freeze`: V8's
PACKED_FROZEN_ELEMENTS read cost lands on the hot `matchingDirs` path.

**One shared arm for the four reuse guards.** The distinct-file-set test was
copy-pasted four ways, 33-38 identical lines each, and this repo's own helpers
(`mini-repo.ts`, `scope-model.ts`) document extracting at the SECOND verbatim
consumer. `expectDistinctFileSetsGetOwnIndex` takes what actually varies; its
`expected` type excludes `null` so the pairing rule cannot be reinstated as a
hole. The per-language first and third arms stay duplicated on purpose —
corpora and payload shapes genuinely differ. Re-proven: all four still fail
under an adapter-inserted `new Set(allFilePaths)`.

**Bench.** `dirsFor` shared by the two functions that must agree on directory
fan-out (they mint and address the same files). `SCALES` derived from the arm
table, so a future arm cannot be measured, printed and silently never asserted.
Five timing checks with one shape collapsed to a table — the trailing sentence
had already drifted into four wordings. `uniqueTarget`/`collideTarget` as flat
functions, mirroring the `uniqueDir`/`collideDir` split rather than nesting a
second axis four ternaries deep. One `identityPass` replaces two untimed full
resolution passes per cell: -371 ms median.

**CI step moved back where it belongs.** It was parked last "until #2895
lands", but that reasoning was backwards twice over: the flake that motivated
it was fixed at the estimator in the previous commit, and #2895's own audit
measured the last slot as executing zero times in 13 runs. It sits with the
other resolver-index guards; #2899 carries the `if: !cancelled()` that fixes
step masking for every step at once.

Filed rather than fixed here: #2908 (java and cobol still scan the workspace
per import, same shape as #2877-#2880, neither memoized), #2909 (make index
reuse a contract test over SCOPE_RESOLVERS on one instrument).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Co58j4au9JLf8dmJpwdF9B

---------

Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Gergő Magyar 2026-08-09 09:59:37 +01:00 • committed by GitHub
parent c6b24162d9
commit 78ecce1b92
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
16 changed files with 2933 additions and 107 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

46
.github/workflows/ci-tests.yml vendored
View file

@ -508,6 +508,52 @@ jobs:
run: node --import tsx bench/cpp-qualified-ns/measure.mjs --check
working-directory: gitnexus
- name: Import-target resolution guards (#2877/#2878/#2879/#2880/#2872)
# Build-free: runs the Go/C#/Dart/Ruby/Kotlin import-target resolvers
# over ONE shared corpus and asserts each returns an unchanged target
# set (a fingerprint per language AND per arm), that per-import cost
# stays independent of corpus size AND of path depth, that the absolute
# small-arm cost holds — a constant-factor regression that grows both
# scale arms equally passes every ratio — and that the shared
# WorkspaceFileIndex C# and Ruby retain stays within an absolute byte
# ceiling. Each of those resolvers used to scan the whole workspace per
# import (Ruby rebuilt a suffix index per `require`), so resolution was
# O(imports × files); the same corpus shape scores >3.3 against the
# pre-fix implementations. The corpus SHAPE is asserted too — a
# fingerprint alone cannot tell a legitimate resolution change from a
# corpus quietly shrunk below the size the timing arms need.
#
# SCOPE: "independent of corpus size" holds for UNIQUE-LEAF layouts,
# where no two directories share a last segment and no two files share a
# basename — which is what the small/large/deep arms are, and where
# every index bucket holds exactly one entry. The `collide` arm runs the
# identical workload on the layout these languages are actually written
# in (svcN/internal, SrcN/Models, a repeated basename per package);
# there the bucket grows with the file count by construction and go,
# csharp and dart legitimately score 2.1–3.9, so that arm carries its
# own per-language budget. It is a scope limit, not a regression — the
# indexed code is still faster on that shape than the pre-change scan.
#
# --expose-gc enables the retained-heap arm; --check REFUSES to run
# without it rather than passing with the memory gate silently skipped.
# ~14 s. REPS is 15 (matching bench/cfg) rather than a cheaper 5 or 7
# because depth_ratio divides two sub-3 ms numbers and at those settings
# it tripped its own budget roughly 1 run in 20 — the estimator was
# fixed instead of the budget widened; distributions in _arms_note.
# The Kotlin arm here is a second corpus, not a replacement for the
# kotlin-import-target bench below, which carries tie-break probes (both
# file-set iteration orders, the four-tier cascade) this one does not.
# A failing step aborts every step after it in this job (#2895), which
# cuts both ways: parking a new gate at the end is not safety, it is the
# slot least likely to execute. This one sits with the other
# resolver-index guards; the estimator fix above is what makes that
# safe, and #2899 carries the `if: ${{ !cancelled() }}` that fixes the
# masking for every step at once.
# Rationale, budgets and the measured blind spot: see the header of
# measure.mjs and _blind_spot in baselines.json.
run: node --expose-gc --import tsx bench/import-target/measure.mjs --check
working-directory: gitnexus
- name: Kotlin import-resolution identity + scaling guards
# Build-free: asserts resolveKotlinImportTarget resolves an unchanged
# file set (fingerprint, in both file-set iteration orders — every

278
gitnexus/bench/import-target/baselines.json Normal file
View file

@ -0,0 +1,278 @@
{
"_what": "Baselines for bench/import-target/measure.mjs — the Go, C#, Dart, Ruby (#2877/#2878/#2879/#2880) and Kotlin (#2872) import-target resolvers on one shared corpus.",
"_fingerprint_note": "Per-language sha256 over every distinct fromFile|target -> resolved target. A change here is a BEHAVIOUR change: the resolver returned a different target set, and IMPORTS/CALLS edges moved. Explain it, never re-baseline to make CI green. For the four languages this PR changed, the pre-change implementations produce these same values on this corpus at both 400 and 1600 files — that is what makes the index hoist a performance change. The tie-break-level proof lives in test/unit/scope-resolution/import-target-index-parity.test.ts (verbatim copies of the pre-change code, diffed) and, for Kotlin, in test/unit/scope-resolution/kotlin/kotlin-import-target-parity.test.ts.",
"_shape_note": "files/imports/resolved/distinct_outcomes AND the fingerprint are asserted exactly, per scale. A fingerprint alone cannot tell a legitimate resolution change from a corpus quietly shrunk below the size at which the timing arms can see anything; conversely the counts alone cannot see a defect confined to one arm, because the arms differ only in path padding and directory layout and both of those are count-neutral by design. Two cross-arm assertions close the remaining hole: the deep and collide arms must resolve exactly what small resolves (they are the same workload), and each of their fingerprints must DIFFER from small's (they are not the same corpus). Without the second, setting DEEP_PAD to 0 — which deletes the entire depth arm — moves no asserted number and prints PASS; the same is true of a collideDir that forwards to uniqueDir.",
"_arms_note": "Five timing arms, one memory arm and one deterministic arm elsewhere, because none of them gates alone. scaling_ratio (t_large/t_small)/(1600/400) catches cost growing with FILE COUNT — the #2877-#2880 regression itself. depth_ratio (t_deep/t_small at a FIXED file count, ~6x the path components) catches cost growing with path DEPTH, which scaling_ratio divides out and structurally cannot see; buildSuffixIndex (C#, Ruby) and Kotlin suffixByStem emit one entry per component, so they legitimately sit above 1.0 while Go and Dart, whose indexes are depth-free, sit at ~1.0. collide_scaling_ratio is the same measurement on a SHARED-LEAF layout (svcN/internal, SrcN/Models, a repeated mod0.dart/mod0.rb basename in every package) carrying an identical file, import and resolved count: the small/large/deep arms mint one directory name per index, so every index bucket in them holds exactly ONE entry (measured: max last-segment bucket 1 and max matching directories 1 for go and csharp at 400 and 1600 files; max basename bucket 1 for dart and ruby), and bucket cardinality is the only non-constant term the new indexes have. On the shared-leaf shape go, csharp and dart legitimately score 2.1-3.9 because the bucket grows with the file count BY CONSTRUCTION — this is a limit on the SCOPE of the 'independent of corpus size' claim, not a regression (the indexed code is still faster there than the pre-change full scan); their collide budgets say so honestly instead of pretending 1.8. Ruby and Kotlin answer from keyed maps and are collision-immune, so they keep the linear 1.8 budget and that immunity is the assertion. The collide arm is also the only arm that reaches filesDirectlyInPkgDir's dirCount > 1 merge (go: 388 multi-directory calls at 400 files, up to 9 directories; 1517 at 1600 files, up to 34). small_ms_ceiling and collide_ms_ceiling are ABSOLUTE (~4x the measured arm), because a constant-factor regression that grows both scale arms equally passes every ratio. NOISE, measured rather than assumed: depth_ratio divides two sub-3 ms numbers (Dart's are sub-1 ms) and is by far the noisiest arm here, so it set REPS for the whole file. fastest() is a min-of-N estimator, so N is the knob. Over 22 --check runs on an idle box, peak-to-peak: at REPS=5 go ran 0.757-1.748 (2.31x) and tripped its own 1.6 budget about 1 run in 20; at REPS=7 (the kotlin-import-target setting) Dart still ran 0.678-2.043 (3.01x) and tripped once; at REPS=15 (bench/cfg, bench/schema-pairs, bench/callable-value-flow) every language collapsed to a 1.13-1.26x swing with 22/22 passing - go 0.968-1.120, csharp 2.963-3.740, dart 1.031-1.248, ruby 1.406-1.588, kotlin 2.097-2.404, i.e. a maximum sitting at 70-78% of each budget. The budgets were NOT widened; the estimator was fixed instead, which is why the headroom above is real rather than granted. A --check run costs ~14 s. heap_ceiling_bytes is the retained size of the shared WorkspaceFileIndex, the only arm here that can see memory: buildSuffixIndex emits three maps at O(files x depth), the profile package-dir-index.ts cites #2649 to avoid for itself, and C#'s no-csproj leg retained nothing at BASE but now builds it unconditionally. It is measured at 8000 and 32000 files at HEAP_PAD depth rather than at the timing arms' sizes, because the finding is an ABSOLUTE footprint at repository scale. Its ceiling is 1.5x the measured arm, and the DIFFERENCE FROM THE 4x TIMING CONVENTION IS DELIBERATE — do not harmonise it back. 4x exists because runner contention dominates a wall-clock number; this one has essentially no measurement noise (spread 656 B on 77.2 MB across 22 runs, 0.00085%, and identical to the byte across separate processes), so 4x would throw away almost all of the gate's power and sail straight past the regression this arm exists to catch. 1.5x still tolerates ~50% of cross-platform and Node-version drift, which is ~50000x the observed process-to-process spread and far more than a Node major bump plausibly moves heapUsed accounting; it catches a duplicated index (+100%) or a second exactMap-sized suffix map (+~85%). RESIDUAL, stated so nobody assumes otherwise: one additional dirMap-sized map is only +18% of the total (dirMap itself measures 15.7% of the C# index and 20.3% of the Ruby one) and would still pass. heap_ratio_budget, the linear-growth check across the 4x file-count gap, is the orthogonal arm: it sees per-file and per-depth growth but not a constant factor.",
"_triage": "Every ratio and ms ceiling here is a TIMING signal — re-run on an idle machine before investigating; runner contention dominates. depth_ratio is the noisiest of them by a wide margin (it divides two sub-3 ms numbers, and Dart's are sub-1 ms): if exactly one arm fails and it is that one, suspect the machine first. REPS is 15 rather than this bench's original 5 specifically to hold that arm's peak-to-peak swing under 1.26x — see _arms_note for the measured distributions — so a depth_ratio failure that REPRODUCES is a real signal, not noise. The fingerprint, shape and heap arms are the opposite: deterministic (the heap arm reproduces to within 0.001% across processes), a re-run never changes them, and they must never be wished away.",
"_floor": "Measured against the pre-change implementations on THIS corpus at 150/600 files: go 3.36, csharp 4.10, dart 3.32, ruby 3.87. The issues report 4.00 / 3.43 / 4.05 on their own corpora; those are DIFFERENT numbers from different repositories and are not reproduced here — what they and these share is that both independently land in the quadratic band, well clear of the ~1.0 a linear result gives. Note also that this floor was taken at 150/600 while the gate runs at 400/1600, so it is a lower bound on what the pre-change code would score today. Kotlin's own bench measured its pre-index floor at 3.737. The 1.8 budget sits well above the linear result and well below every one of those.",
"scaling_budget": 1.8,
"collide_scaling_budget": {
"go": 5.5,
"csharp": 3.4,
"dart": 3.3,
"ruby": 1.8,
"kotlin": 1.8
},
"depth_budget": {
"go": 1.6,
"csharp": 5,
"dart": 1.6,
"ruby": 2.2,
"kotlin": 3.4
},
"small_ms_ceiling": {
"go": 7,
"csharp": 11,
"dart": 3,
"ruby": 77,
"kotlin": 12
},
"collide_ms_ceiling": {
"go": 28,
"csharp": 22,
"dart": 6,
"ruby": 95,
"kotlin": 12
},
"heap_ceiling_bytes": {
"csharp": 116000000,
"ruby": 87000000
},
"heap_ratio_budget": 1.25,
"languages": {
"go": {
"small": {
"files": 400,
"imports": 3200,
"resolved": 979,
"distinct_outcomes": 2913,
"fingerprint": "c96c4a0adce69f7c0e40fa84f6d2920c160e8a76d594803b4c725666fdcb7edf"
},
"large": {
"files": 1600,
"imports": 12800,
"resolved": 4064,
"distinct_outcomes": 11709,
"fingerprint": "ec4bb401b3465713ad6dedc4f9aa774e586b319f21d406fd79eaad29f4e98861"
},
"deep": {
"files": 400,
"imports": 3200,
"resolved": 979,
"distinct_outcomes": 2913,
"fingerprint": "f1ce3dbe9fa4d03bae9504b644e39cc3c815d99defda70aedcf09fb743575f54"
},
"collide": {
"files": 400,
"imports": 3200,
"resolved": 979,
"distinct_outcomes": 2868,
"fingerprint": "6727beda6df2251260ee89718ea7931fa7ff1e59fa9966caf4df95f7186b6257"
},
"collide_large": {
"files": 1600,
"imports": 12800,
"resolved": 4064,
"distinct_outcomes": 11570,
"fingerprint": "6d844763547b5cad54f41cfa0c0d618b098b214cb58654375fe7d74d229bee98"
},
"fingerprint": "ec4bb401b3465713ad6dedc4f9aa774e586b319f21d406fd79eaad29f4e98861",
"_measured": {
"collide_ms": 6.676,
"collide_scaling_ratio": 3.958,
"depth_ratio": 1.032,
"scaling_ratio": 1.042,
"small_ms": 1.698
}
},
"csharp": {
"small": {
"files": 400,
"imports": 3200,
"resolved": 979,
"distinct_outcomes": 2844,
"fingerprint": "cd1d665d997280a3eeb52069f2a8745f85ef91042085498a9ab546ed45faf10b"
},
"large": {
"files": 1600,
"imports": 12800,
"resolved": 4064,
"distinct_outcomes": 11440,
"fingerprint": "0b46146f5213fea8c07f1f90305a14da5ce31c865c495180f3f3903ddc6b8117"
},
"deep": {
"files": 400,
"imports": 3200,
"resolved": 979,
"distinct_outcomes": 2844,
"fingerprint": "2baee530615a03ac6342d8d330f8b40e619b46e5de40f688ca8fb8a5f8b35027"
},
"collide": {
"files": 400,
"imports": 3200,
"resolved": 979,
"distinct_outcomes": 2844,
"fingerprint": "51d8d08e5195a416a2e7d8e42daf69bf5c5e4882239732dee454bd9c8ecf935e"
},
"collide_large": {
"files": 1600,
"imports": 12800,
"resolved": 4064,
"distinct_outcomes": 11440,
"fingerprint": "6d3a964bdb4ae4a0c64a1e31023fb736c42643f5aeeef704d8e00c31ae12c3af"
},
"fingerprint": "0b46146f5213fea8c07f1f90305a14da5ce31c865c495180f3f3903ddc6b8117",
"_measured": {
"collide_ms": 4.755,
"collide_scaling_ratio": 2.265,
"depth_ratio": 3.318,
"heap_bytes_large": 77200616,
"heap_bytes_small": 19548056,
"heap_mib_large": 73.62,
"heap_ratio": 0.987,
"scaling_ratio": 1.2,
"small_ms": 2.704
}
},
"dart": {
"small": {
"files": 400,
"imports": 3200,
"resolved": 1153,
"distinct_outcomes": 2987,
"fingerprint": "318084f48ffa4eeae4a5b7fc25916d4ad78673d92dba62b7e462d1bb87ca553a"
},
"large": {
"files": 1600,
"imports": 12800,
"resolved": 4681,
"distinct_outcomes": 11875,
"fingerprint": "5151cd2498bd4b7698dc9309e2539977d306f9ba82a388c630c89b51fc4a3187"
},
"deep": {
"files": 400,
"imports": 3200,
"resolved": 1153,
"distinct_outcomes": 2987,
"fingerprint": "79776ec1c22afa619fd31aeb05dcac567723b436d0461a5782693b9e929f6f74"
},
"collide": {
"files": 400,
"imports": 3200,
"resolved": 1153,
"distinct_outcomes": 2999,
"fingerprint": "1b145a4c3b41ffc4efa26f74449c3d44646d6d59728163ac896fc0ee25c6d608"
},
"collide_large": {
"files": 1600,
"imports": 12800,
"resolved": 4681,
"distinct_outcomes": 11948,
"fingerprint": "b7e5303220b8fa64e85c7e17622961018316a10c5ef921a02584864309748b52"
},
"fingerprint": "5151cd2498bd4b7698dc9309e2539977d306f9ba82a388c630c89b51fc4a3187",
"_measured": {
"collide_ms": 1.495,
"collide_scaling_ratio": 2.316,
"depth_ratio": 1.153,
"scaling_ratio": 1.102,
"small_ms": 0.53
}
},
"ruby": {
"small": {
"files": 400,
"imports": 3200,
"resolved": 1153,
"distinct_outcomes": 2936,
"fingerprint": "54abc79cc3fd4bfbf341119f3c511c2d64d55556a0c23984032f34e259283e46"
},
"large": {
"files": 1600,
"imports": 12800,
"resolved": 4681,
"distinct_outcomes": 11786,
"fingerprint": "31804ae9633d51ce7597d886f9c2230fec3448b0aa00d9ab953086e393cd28a9"
},
"deep": {
"files": 400,
"imports": 3200,
"resolved": 1153,
"distinct_outcomes": 2936,
"fingerprint": "a6dac0609e6800571bcc19ee30818ab571549f275d691f98e4c238aaa4fb1362"
},
"collide": {
"files": 400,
"imports": 3200,
"resolved": 1153,
"distinct_outcomes": 2936,
"fingerprint": "065fb3a97e1fa01416396b128b1a03d4ea5b49634cb3ffed4b81a07b251c2f2e"
},
"collide_large": {
"files": 1600,
"imports": 12800,
"resolved": 4681,
"distinct_outcomes": 11786,
"fingerprint": "55a3afc06a48334a6dd2f29c730ae0cfd3a6d54f3013c0853a310af2bbcba277"
},
"fingerprint": "31804ae9633d51ce7597d886f9c2230fec3448b0aa00d9ab953086e393cd28a9",
"_measured": {
"collide_ms": 21.702,
"collide_scaling_ratio": 1.111,
"depth_ratio": 1.517,
"heap_bytes_large": 57939392,
"heap_bytes_small": 14835688,
"heap_mib_large": 55.26,
"heap_ratio": 0.976,
"scaling_ratio": 1.143,
"small_ms": 20.799
}
},
"kotlin": {
"small": {
"files": 400,
"imports": 3200,
"resolved": 1100,
"distinct_outcomes": 2868,
"fingerprint": "59c4287225e765d75518a7ae1531487e0270c9a1ce42a3e1821c301f7cfeb3cc"
},
"large": {
"files": 1600,
"imports": 12800,
"resolved": 4456,
"distinct_outcomes": 11512,
"fingerprint": "003bb2fe82972c6bb6b4b4e569fb49dfcda2d7c61922d68c65b04398cbbde50b"
},
"deep": {
"files": 400,
"imports": 3200,
"resolved": 1100,
"distinct_outcomes": 2868,
"fingerprint": "e541c9daee39c271ccc08b045f5330a98efba8b8dd4ded233e39e174f50d3785"
},
"collide": {
"files": 400,
"imports": 3200,
"resolved": 1100,
"distinct_outcomes": 2775,
"fingerprint": "1e855115befc9a7e972bc3990816c366c9ac1ccb1aa3e50d237ccb76e9a8f99a"
},
"collide_large": {
"files": 1600,
"imports": 12800,
"resolved": 4456,
"distinct_outcomes": 11087,
"fingerprint": "d867aaa39e47ba55df1853c4eaca741977e946a16ad3d99f35c698abe7241ac7"
},
"fingerprint": "003bb2fe82972c6bb6b4b4e569fb49dfcda2d7c61922d68c65b04398cbbde50b",
"_measured": {
"collide_ms": 2.703,
"collide_scaling_ratio": 1.199,
"depth_ratio": 2.287,
"scaling_ratio": 1.224,
"small_ms": 2.821
}
}
},
"_blind_spot": "MEASURED, so nobody has to rediscover it: a full workspace scan reintroduced on 1-in-32 imports passes EVERY arm here — dart scored 1.458 scaling and 1.736 ms against the 1.8 budget and 4 ms ceiling of an earlier revision. At 1-in-8 the scaling arm catches it (2.414). The gate that NARROWS this is not a timing gate at all: test/unit/scope-resolution/import-target-index-parity.test.ts counts iterations of the file-set Set and reads 14 instead of 1 for that same 1-in-32 mutation, deterministically and for all five languages. It does NOT close it. The counter watches the Set, and the resolvers no longer read the Set — they read materialized copies of the same file list: WorkspaceFileIndex.normalized and .all (C#, Ruby), Dart's byBasename buckets, and PackageDirIndex.filesByDir (Go, C#). A 1-in-32 scan over any of those three touches the Set zero extra times, so it passes the parity test AND passes --check. Closing it would take an iteration counter on the materialized arrays themselves. Read the two gates together; tightening these ceilings toward the noise floor to chase that case would only buy flaky CI."
}

756
gitnexus/bench/import-target/measure.mjs Normal file
View file

@ -0,0 +1,756 @@
/**
* Build-free scaling + identity bench for the Go, C#, Dart, Ruby (#2877, #2878,
* #2879, #2880) and Kotlin (#2872) import-target resolvers, over ONE shared
* corpus so the five are directly comparable.
*
* Kotlin also has `bench/kotlin-import-target/`, and this does not replace it:
* that bench fingerprints both file-set iteration orders and probes the
* four-tier cascade shape by shape, which this corpus does not. What Kotlin
* gains here is a second corpus and the arms below that its own bench predates.
*
* Before this PR each of the other four answered its lookups with a full
* `allFilePaths` scan per import, so import resolution cost O(imports × files):
*
* - Go: `findRootPackageFiles` / `findAllFilesInPkgDir`, the latter once per
* path segment on the GOPATH fallback — several full scans per import;
* - C#: the no-csproj leg took the raw Set past the memoized index the csproj
* leg was already using — up to eight passes for a four-segment `using`;
* - Dart: one full scan per candidate path, and for an external package both
* candidates miss, so both always ran to completion;
* - Ruby: a complete `buildSuffixIndex` rebuilt and discarded per `require`.
*
* Two properties of the corpus are load-bearing and must not be "simplified":
*
* 1. **Most imports are unresolvable.** In real source the majority of imports
* name the stdlib or a third-party package, and those run every leg of the
* cascade to completion before returning null — the fast paths never fire.
* A corpus of mostly-resolving imports measures the wrong half of the
* function and would score a reintroduced scan as linear.
* 2. **Import count scales WITH file count.** The regression is quadratic in
* `imports × files`; holding imports fixed while files grow would halve the
* exponent and let a per-import scan pass the budget.
*
* Reports per language and scale:
* - `ms`: fastest of REPS full passes, INCLUDING the one-time index build —
* hiding the build would let an index that is itself quadratic pass;
* - `scaling_ratio` `(t_large/t_small)/(LARGE/SMALL)`: ~1.0 linear, ~4.x
* quadratic at this scale gap;
* - `depth_ratio` `t_deep/t_small` at a FIXED file count with ~6x the path
* components. `scaling_ratio` divides the file count out, so it is
* scale-invariant and structurally cannot see a cost that grows with path
* DEPTH instead — and `buildSuffixIndex` (C#, Ruby) and Kotlin's
* `suffixByStem` each emit one entry per component. Go and Dart, whose
* indexes are depth-free, sit at ~0.9; the others sit legitimately above
* 1.0, which is why the budget is per language;
* - `collide_scaling_ratio`, the same measurement on a corpus whose
* directories SHARE their last segment and whose files share basenames —
* see the `collide` section below;
* - `heap` (C# and Ruby): retained bytes of the shared `WorkspaceFileIndex`
* — see the `heap` section below;
* - a sha256 over every distinct `fromFile | target → result`, as the
* correctness gate. The tie-break-level proof that this PR's index
* reproduces the scans lives in
* `test/unit/scope-resolution/import-target-index-parity.test.ts`, which
* diffs against verbatim copies of the pre-change implementations; this
* fingerprint is the forward guard that keeps the output pinned from here.
* Every scale's fingerprint is asserted, not just `large`'s: the arms
* differ only in layout and padding, so a per-scale-only defect (a resolver
* bug that corrupts deep paths, or a corpus edit that quietly deletes the
* depth padding) moves no asserted count and would otherwise print PASS.
*
* `--check` adds arms that no ratio can carry:
* - the corpus SHAPE (files, imports, resolved, distinct outcomes, per scale),
* so a future edit cannot quietly shrink the corpus below the sizes that
* make the timing arms meaningful and still print PASS. The `deep` and
* `collide` arms must also resolve exactly what `small` resolves — padding
* and re-layout were supposed to change path depth and directory naming and
* nothing else;
* - `deep.fingerprint !== small.fingerprint` and
* `collide.fingerprint !== small.fingerprint`, so those two arms' EFFECT is
* pinned rather than only their output. Both are count-neutral by
* construction, so neutering either one (`DEEP_PAD = 0`, a `collideDir`
* that forwards to `uniqueDir`) leaves every asserted number untouched;
* comparing the arms to `small` is the only thing that notices;
* - `small_ms_ceiling` and `collide_ms_ceiling`, ABSOLUTE bounds, because a
* constant-factor regression that grows both scale arms equally passes
* every ratio.
*
* SCOPE OF THE "independent of corpus size" CLAIM — the `collide` arm.
* `small`/`large`/`deep` mint one directory name per index (`src/pkg7`,
* `src/Ns7`, `lib/feature7`) and one basename per file, so every index bucket
* in them holds exactly ONE entry: measured, max last-segment bucket = 1 and
* max matching directories = 1 for go and csharp at both 400 and 1600 files,
* max basename bucket = 1 for dart and ruby. Bucket cardinality is the only
* non-constant term the new indexes have, so those arms certify the headline
* claim on the one shape where that term cannot appear. `collide` is the same
* workload — identical file, import and resolved counts — laid out the way
* these languages are actually written: `svcN/internal/`, `SrcN/Models/`, a
* `mod0.dart`/`mod0.rb` in every package. Measured on that shape the per-import
* cost is NOT corpus-size-independent for the three resolvers that scan a
* bucket:
*
* - go and csharp walk `PackageDirIndex.dirsByLastSegment[seg]`, which now
* holds every directory;
* - dart walks its basename bucket, which now holds every same-named file;
* - ruby and kotlin answer from keyed maps and are collision-IMMUNE, so their
* collide budgets are the linear ones — that immunity is the assertion.
*
* This is a scope-of-claim limit, not a regression: on the MISS path with a
* shared leaf name the bucket grows with the file count BY CONSTRUCTION, and
* the indexed code is still faster there than the pre-change full scan. The arm
* exists so the real shape is measured and pinned, and so nobody reads the 1.8
* budget as covering it. Narrowing it would mean a reversed-path prefix-range
* structure, which trades against the O(files × depth) memory
* `package-dir-index.ts` cites #2649 to avoid — a design change, not a tune.
*
* MEMORY — the `heap` arm. C# and Ruby both resolve through the shared
* `WorkspaceFileIndex`, and `buildSuffixIndex` under it emits three maps at
* O(files × depth): exactly the profile `package-dir-index.ts` cites #2649 to
* avoid for itself. C# is the reason this is gated rather than noted: at BASE
* `getWorkspaceFileIndex` was reached only from the csproj branch and the
* no-csproj leg scanned the Set and retained nothing, whereas it is now called
* unconditionally. Every other arm here is time or count, and no ratio can see
* a footprint. Measured in ABSOLUTE bytes, not only as a ratio: the finding is
* about the footprint itself, and a ratio alone hides a large constant.
*
* KNOWN BLIND SPOT, measured: a full workspace scan reintroduced on 1-in-32
* imports passes every arm here (dart scored 1.458 scaling, 1.736 ms). The gate
* that NARROWS it is not a timing gate — the parity test above counts
* iterations of the file-set Set and reads 14 instead of 1 for that same
* mutation. It does not CLOSE it: the counter watches the Set, while the
* resolvers hold materialized arrays of the same file list
* (`WorkspaceFileIndex.normalized`/`.all`, Dart's basename buckets,
* `PackageDirIndex.filesByDir`), and a 1-in-32 scan over one of THOSE passes
* both the parity test and `--check`. Chasing it by tightening these ceilings
* toward the noise floor would only buy flaky CI; see `_blind_spot` in
* baselines.json.
*
* Run:
* node --expose-gc --import tsx bench/import-target/measure.mjs # report
* node --expose-gc --import tsx bench/import-target/measure.mjs --check # CI gate
*/
import fs from 'node:fs';
import path from 'node:path';
import crypto from 'node:crypto';
import { fileURLToPath } from 'node:url';
import { resolveGoImportTarget } from '../../src/core/ingestion/languages/go/import-target.ts';
import { resolveDartImportTarget } from '../../src/core/ingestion/languages/dart/import-target.ts';
import { resolveRubyImportTarget } from '../../src/core/ingestion/languages/ruby/import-target.ts';
import { resolveCsharpImportTarget } from '../../src/core/ingestion/languages/csharp/import-target.ts';
import { resolveKotlinImportTarget } from '../../src/core/ingestion/languages/kotlin/import-target.ts';
import { getWorkspaceFileIndex } from '../../src/core/ingestion/import-resolvers/workspace-file-index.ts';
const __dirname = path.dirname(fileURLToPath(import.meta.url));
const BASELINE_PATH = path.resolve(__dirname, 'baselines.json');
const SMALL = 400;
const LARGE = 1600;
const IMPORTS_PER_FILE = 8;
/** Extra directory components prepended in the `deep` arm — see `depth_ratio`. */
const DEEP_PAD = 16;
/** `fastest()` below is a min-of-N estimator, so N is the noise knob: raising it
* lowers and stabilises the minimum. `depth_ratio` divides two sub-3 ms
* measurements, and Dart's are sub-1 ms, so it is by far the noisiest number
* here and it sets N for the whole file. Measured over 22 `--check` runs on an
* idle box: at N=5 it tripped its own budget ~1 run in 20, at N=7 Dart still
* swung 3.0x peak-to-peak and tripped once. N=15 matches `bench/cfg`,
* `bench/schema-pairs` and `bench/callable-value-flow`; the distributions it
* produces are recorded in `_arms_note`. */
const REPS = 15;
const WARMUP = 2;
/** Heap arm (C#, Ruby). Far more files than the timing arms because the finding
* is an ABSOLUTE footprint at repository scale, and 1600 files would report a
* fraction of a MiB — a number no ceiling could usefully bound. `HEAP_PAD`
* keeps the paths at a plausible monorepo depth: `buildSuffixIndex` is
* O(files × depth), so a flat corpus would understate it by ~4x. */
const HEAP_SMALL = 8000;
const HEAP_LARGE = 32000;
const HEAP_PAD = 8;
/** Languages whose resolvers retain the shared `WorkspaceFileIndex`. */
const HEAP_LANGS = ['csharp', 'ruby'];
/** Needs `node --expose-gc` to force collection for a clean delta; without it
* the heap metric is reported as null and its `--check` gate would be skipped,
* which is why `--check` refuses to run without the flag (see below). */
const GC = typeof global.gc === 'function' ? () => (global.gc(), global.gc()) : null;
/** Deterministic 32-bit avalanche (murmur3 finalizer) — no `Math.random()`, so
* the corpus and therefore the fingerprint are byte-reproducible. */
function mix(n) {
let x = n >>> 0;
x = Math.imul(x ^ (x >>> 16), 0x85ebca6b) >>> 0;
x = Math.imul(x ^ (x >>> 13), 0xc2b2ae35) >>> 0;
return (x ^ (x >>> 16)) >>> 0;
}
const GO_MODULE = { modulePath: 'example.com/mod' };
const EXTENSION = { go: '.go', csharp: '.cs', dart: '.dart', ruby: '.rb', kotlin: '.kt' };
/** Directory fan-out. Shared because `buildRepo`'s collide targets address
* files by `j % dirs` / `Math.floor(j / dirs)` and must agree with the layout
* `buildFiles` produced. */
const dirsFor = (fileCount) => Math.max(4, Math.floor(fileCount / 8));
/**
* UNIQUE-LEAF layout: one directory name per index, so no two directories share
* a last segment and no two files share a basename. Every index bucket holds
* exactly one entry. A nested same-name directory in one repo slice is the
* shape whose handling the first-`indexOf` tie-break decides (see
* package-dir-index.ts), and the shape Kotlin's `dirChildren` resolves the same
* way.
*/
function uniqueDir(lang, d) {
if (lang === 'go') return d % 7 === 0 ? `src/pkg${d}/internal/pkg${d}` : `src/pkg${d}`;
if (lang === 'csharp') return d % 7 === 0 ? `src/Ns${d}/Sub/Ns${d}` : `src/Ns${d}`;
if (lang === 'dart') return d % 3 === 0 ? `lib/feature${d}` : `pkg/feature${d}`;
if (lang === 'kotlin') {
return d % 7 === 0
? `mod${d}/src/main/kotlin/com/example/pkg${d}/inner/pkg${d}`
: `mod${d}/src/main/kotlin/com/example/pkg${d}`;
}
return `lib/mod${d}`;
}
/**
* SHARED-LEAF layout: every directory ends in the SAME segment, so one bucket
* holds all of them. The `d % 7` slice keeps the nested same-name directory of
* the unique layout, and Go additionally replicates one package (`internal/
* shared`) across services — the monorepo shape `filesDirectlyInPkgDir`'s merge
* exists for, and the only arm in this bench that reaches `dirCount > 1`.
*
* Each language's local import spelling is chosen so this arm resolves exactly
* as many imports as `small` does (asserted): same workload, different layout.
*/
function collideDir(lang, d) {
if (lang === 'go') {
if (d % 7 === 0) return `svc${d}/internal/sub/internal`;
return d % 5 === 1 ? `svc${d}/internal/shared` : `svc${d}/internal`;
}
if (lang === 'csharp') return d % 7 === 0 ? `Src${d}/Models/Inner/Models` : `Src${d}/Models`;
if (lang === 'dart') return `pkg${d}/lib/src`;
if (lang === 'kotlin') {
return d % 7 === 0
? `mod${d}/src/main/kotlin/com/example/models/inner/models`
: `mod${d}/src/main/kotlin/com/example/models`;
}
return `svc${d}/lib/models`;
}
/**
* The file paths of one synthetic repository. `dirs` grows with the file count
* so directory fan-out is realistic at both scales rather than collapsing onto
* a handful of buckets.
*
* `pad` prepends that many extra directory components to every path. Every
* language's in-repo target resolves through a path SUFFIX (Go's module leg
* against the package dir, C#'s progressive strip, Dart's `lib/<rel>`, Ruby's
* suffix match, Kotlin's `suffixByStem`), so the padding changes path depth
* without changing what resolves — which is what makes the `deep` arm a clean
* depth measurement rather than a different corpus.
*
* Split out from `buildRepo` so the heap arm can build 32k paths without also
* minting 256k import tuples it would never resolve.
*/
function buildFiles(lang, fileCount, pad, shape) {
const dirs = dirsFor(fileCount);
const files = [];
const ext = EXTENSION[lang];
const prefix = pad === 0 ? '' : Array.from({ length: pad }, (_, n) => `d${n}`).join('/') + '/';
for (let i = 0; i < fileCount; i++) {
const d = i % dirs;
const dir = shape === 'collide' ? collideDir(lang, d) : uniqueDir(lang, d);
// In the collide shape Dart and Ruby carry a REPEATED basename — the term
// their indexes bucket on. `i / dirs` is unique within a directory (8 files
// land in each) and identical across directories, which is exactly the
// `models.dart` / `models.rb`-in-every-package convention. Go, C# and
// Kotlin bucket on the directory instead, so their stems stay unique.
const collideStem = lang === 'dart' || lang === 'ruby';
const stem =
shape === 'collide' && collideStem
? `mod${Math.floor(i / dirs)}`
: lang === 'csharp' || lang === 'kotlin'
? `File${i}`
: `file${i}`;
// Go's package leg must exclude `_test.go`; keep a real share of them.
// Kotlin resolves `.kt` and `.kts` through the same stem maps; keep both.
const suffix =
lang === 'go' && i % 6 === 0 ? '_test.go' : lang === 'kotlin' && i % 11 === 0 ? '.kts' : ext;
files.push(`${prefix}${dir}/${stem}${suffix}`);
}
return files;
}
/**
* The import one file issues in the UNIQUE-LEAF layout `uniqueDir` produced.
*
* The TARGET axis is split from the DIRECTORY axis exactly the way `uniqueDir`
* and `collideDir` split it above — two flat functions, selected once — rather
* than a `collide ?` ternary threaded through five languages' `local ? …`
* ladders. `local` picks in-repo vs external; the handful of MISS lines that
* are identical between the two shapes are duplicated on purpose, because the
* alternative is four levels of nesting in a single expression.
*/
function uniqueTarget(lang, { local, r, d, j }) {
if (lang === 'go') {
return local
? `${GO_MODULE.modulePath}/src/pkg${d}`
: (r >>> 3) % 2 === 0
? ['fmt', 'os', 'net/http', 'encoding/json'][(r >>> 4) % 4]
: `github.com/org/repo${(r >>> 4) % 97}/pkg/util`;
}
if (lang === 'csharp') {
return local
? `App.Ns${d}`
: (r >>> 3) % 2 === 0
? ['System', 'System.Threading.Tasks', 'System.Collections.Generic'][(r >>> 4) % 3]
: `Ghost${(r >>> 4) % 97}.Deep.Missing`;
}
if (lang === 'dart') {
return local
? `package:app/feature${d}/file${j}.dart`
: (r >>> 3) % 3 === 0
? ['dart:core', 'dart:async', 'dart:io'][(r >>> 4) % 3]
: `package:ext${(r >>> 4) % 97}/src/thing.dart`;
}
if (lang === 'kotlin') {
// A share of wildcard imports: `.*` lands on the package fan-out tier,
// which returns a LIST and is the only tier whose output is order-bearing.
return local
? (r >>> 3) % 3 === 0
? `com.example.pkg${d}.*`
: `com.example.pkg${d}.File${j}`
: (r >>> 3) % 2 === 0
? ['java.util.List', 'kotlin.collections.Map', 'kotlinx.coroutines.flow.Flow'][
(r >>> 4) % 3
]
: `com.ghost${(r >>> 4) % 97}.deep.Missing`;
}
return local
? `mod${d}/file${j}`
: (r >>> 3) % 2 === 0
? ['json', 'set', 'net/http', 'digest'][(r >>> 4) % 4]
: `gem${(r >>> 4) % 97}/missing/thing`;
}
/**
* The same import in the SHARED-LEAF layout `collideDir` produced. Each
* language's local spelling is chosen so this arm resolves exactly as many
* imports as the unique arm does (asserted): same workload, different layout.
*/
function collideTarget(lang, { local, r, d, j, dirs }) {
if (lang === 'go') {
return local
? // The replicated package is addressed by the path it shares, so the
// module leg matches every service at once (`dirCount > 1`).
d % 5 === 1 && d % 7 !== 0
? `${GO_MODULE.modulePath}/internal/shared`
: `${GO_MODULE.modulePath}/svc${d}/internal`
: (r >>> 3) % 2 === 0
? ['fmt', 'os', 'net/http', 'encoding/json'][(r >>> 4) % 4]
: // Ends in the shared segment, so the GOPATH fallback walks the whole
// bucket three times and still returns null: the MISS path this arm
// exists to measure.
`github.com/org/repo${(r >>> 4) % 97}/internal`;
}
if (lang === 'csharp') {
return local
? // `Vendor` has no directory anywhere, mirroring the unique arm's
// nested-same-name slice, which also resolves to nothing.
d % 7 === 0
? `App.Src${d}.Vendor`
: `App.Src${d}.Models`
: (r >>> 3) % 2 === 0
? ['System', 'System.Threading.Tasks', 'System.Collections.Generic'][(r >>> 4) % 3]
: `Ghost${(r >>> 4) % 97}.Deep.Missing`;
}
if (lang === 'dart') {
return local
? `package:app/pkg${j % dirs}/lib/src/mod${Math.floor(j / dirs)}.dart`
: (r >>> 3) % 3 === 0
? ['dart:core', 'dart:async', 'dart:io'][(r >>> 4) % 3]
: // A repeated basename under a directory nothing carries: both
// candidates walk the whole basename bucket and miss.
`package:ext${(r >>> 4) % 97}/other/mod${(r >>> 4) % 8}.dart`;
}
if (lang === 'kotlin') {
// Same wildcard share as the unique arm; `vendor${d}` is the collide
// layout's spelling of a package that exists nowhere.
return local
? (r >>> 3) % 3 === 0
? d % 7 === 0
? `com.example.vendor${d}.*`
: `com.example.models.*`
: `com.example.models.File${j}`
: (r >>> 3) % 2 === 0
? ['java.util.List', 'kotlin.collections.Map', 'kotlinx.coroutines.flow.Flow'][
(r >>> 4) % 3
]
: `com.ghost${(r >>> 4) % 97}.deep.Missing`;
}
return local
? `svc${j % dirs}/lib/models/mod${Math.floor(j / dirs)}`
: (r >>> 3) % 2 === 0
? ['json', 'set', 'net/http', 'digest'][(r >>> 4) % 4]
: `gem${(r >>> 4) % 97}/missing/thing`;
}
/**
* One synthetic repository per language: the file set plus the import list each
* file issues.
*/
function buildRepo(lang, fileCount, pad = 0, shape = 'unique') {
const dirs = dirsFor(fileCount);
const files = buildFiles(lang, fileCount, pad, shape);
const mintTarget = shape === 'collide' ? collideTarget : uniqueTarget;
const imports = [];
for (let i = 0; i < fileCount; i++) {
const from = files[i];
for (let k = 0; k < IMPORTS_PER_FILE; k++) {
const r = mix(i * 65599 + k);
// ~3 in 8 imports resolve in-repo; the rest are external and run the
// whole cascade to completion (corpus property 1).
const local = r % 8 < 3;
const d = r % dirs;
const j = r % fileCount;
imports.push([from, mintTarget(lang, { local, r, d, j, dirs })]);
}
}
return { files, imports };
}
/** The timed loop. A FRESH Set per pass, so every pass pays exactly one index
* build — reusing one Set across passes would hide the build after the first
* and let a rebuilt-per-import index look free from rep 2 onward. */
function resolveAll(lang, files, imports) {
const allFilePaths = new Set(files);
let sink = 0;
for (const [from, target] of imports) {
const hit = resolveOne(lang, from, target, allFilePaths);
if (hit !== null) sink++;
}
return sink;
}
function resolveOne(lang, from, target, allFilePaths) {
if (lang === 'go') return resolveGoImportTarget(target, from, allFilePaths, GO_MODULE);
if (lang === 'dart') return resolveDartImportTarget(target, from, allFilePaths);
if (lang === 'ruby') return resolveRubyImportTarget(target, from, allFilePaths);
if (lang === 'kotlin') {
return resolveKotlinImportTarget(
{ kind: 'named', localName: 'X', importedName: 'X', targetRaw: target },
{ fromFile: from, allFilePaths },
);
}
return resolveCsharpImportTarget(
{ kind: 'namespace', localName: '_', importedName: '_', targetRaw: target },
{ fromFile: from, allFilePaths },
);
}
/** The single untimed identity pass, producing BOTH non-timing results: the
* distinct `from|target → result` set the fingerprint hashes, and `resolved`
* counted over every import. One resolve per DISTINCT pair — on a fixed file
* set the resolvers are pure, so a repeated pair can only re-derive what the
* first occurrence already recorded, and the memoized `key → wasNull` answers
* the count for the repeat. Merged from two passes that each walked the whole
* corpus; the duplicate resolves measured ~1.15 s of an 11.4 s run.
*
* Deliberately NOT shared with `resolveAll`, which is the TIMED loop: the memo
* that makes this pass cheap is exactly what would hide the cost that loop
* exists to measure. */
function identityPass(lang, files, imports) {
const allFilePaths = new Set(files);
const outcomes = new Set();
const wasNullByKey = new Map();
let resolved = 0;
for (const [from, target] of imports) {
const key = `${from}\u0000${target}`;
let wasNull = wasNullByKey.get(key);
if (wasNull !== undefined) {
if (!wasNull) resolved++;
continue;
}
const hit = resolveOne(lang, from, target, allFilePaths);
wasNull = hit === null;
wasNullByKey.set(key, wasNull);
if (!wasNull) resolved++;
const rendered = wasNull ? '<null>' : Array.isArray(hit) ? hit.join(',') : hit;
outcomes.add(`${key}\u0000${rendered}`);
}
return { outcomes, resolved };
}
/** MIN, not median: both scales are timed in one process and every error source
* (GC, scheduler preemption, a noisy CI neighbour) is additive, so the fastest
* observed pass is the closest estimate of the uncontended cost. */
function fastest(values) {
return Math.min(...values);
}
function timeResolution(lang, files, imports) {
for (let w = 0; w < WARMUP; w++) resolveAll(lang, files, imports);
const samples = [];
for (let r = 0; r < REPS; r++) {
const t0 = performance.now();
resolveAll(lang, files, imports);
samples.push(performance.now() - t0);
}
return fastest(samples);
}
/**
* Retained JS heap of the `WorkspaceFileIndex` built over `files`.
*
* GROWTH form, not the release form `bench/cfg/measure.mjs` uses: the index is
* memoized in a `WeakMap` keyed on the Set, so releasing it means releasing the
* Set too, which would fold the Set's own cost into the delta. Here the Set is
* live across BOTH reads and the `files` array holds the path strings, so the
* delta is the index's own footprint — the arrays, the three `buildSuffixIndex`
* maps and `normToRaw` — and not the paths they point at. A forced double GC
* before each read makes it robust to pre-existing garbage the same way.
*/
function retainedIndexBytes(files) {
const set = new Set(files);
GC();
const before = process.memoryUsage().heapUsed;
const index = getWorkspaceFileIndex(set);
GC();
const after = process.memoryUsage().heapUsed;
// Keeps both live past the second read, and fails loudly if the corpus ever
// stops being one distinct path per file (which would silently shrink it).
if (index.all.length !== files.length || set.size !== files.length) {
throw new Error(`heap arm corpus is not distinct: ${set.size} of ${files.length}`);
}
return Math.max(0, after - before);
}
function measureHeap(lang) {
if (GC === null) return null;
const small = buildFiles(lang, HEAP_SMALL, HEAP_PAD, 'unique');
// The first build in a fresh process reads a few percent low (lazily grown
// spaces, unJITted build loop); discard it.
retainedIndexBytes(small);
const bytesSmall = retainedIndexBytes(small);
const large = buildFiles(lang, HEAP_LARGE, HEAP_PAD, 'unique');
const bytesLarge = retainedIndexBytes(large);
return {
files_small: HEAP_SMALL,
files_large: HEAP_LARGE,
path_segments: small[0].split('/').length,
bytes_small: bytesSmall,
bytes_large: bytesLarge,
mib_large: Number((bytesLarge / 1024 / 1024).toFixed(2)),
ratio: Number((bytesLarge / bytesSmall / (HEAP_LARGE / HEAP_SMALL)).toFixed(3)),
};
}
function fingerprint(outcomes) {
return crypto
.createHash('sha256')
.update([...outcomes].sort().join('\n'))
.digest('hex');
}
const CHECK = process.argv.includes('--check');
// The heap arm is a primary regression detector, but it can only be measured
// with a forced GC. Rather than let `--check` silently PASS with the heap gate
// skipped (a green no-op if someone drops --expose-gc), fail loudly.
if (CHECK && GC === null) {
process.stderr.write(
'[import-target --check] FAIL: the retained-heap arm requires --expose-gc. ' +
'Run: node --expose-gc --import tsx bench/import-target/measure.mjs --check\n',
);
process.exit(1);
}
const LANGS = ['go', 'csharp', 'dart', 'ruby', 'kotlin'];
/** name, file count, depth padding, directory/basename layout. */
const ARMS = [
['small', SMALL, 0, 'unique'],
['large', LARGE, 0, 'unique'],
['deep', SMALL, DEEP_PAD, 'unique'],
['collide', SMALL, 0, 'collide'],
['collide_large', LARGE, 0, 'collide'],
];
/** Derived, never hand-written: the shape/fingerprint gate below iterates these
* names, so a new arm is asserted by construction rather than measured,
* printed and silently left out of the gate. */
const SCALES = ARMS.map(([name]) => name);
const report = {};
for (const lang of LANGS) {
const scales = {};
for (const [name, fileCount, pad, shape] of ARMS) {
const { files, imports } = buildRepo(lang, fileCount, pad, shape);
const { outcomes, resolved } = identityPass(lang, files, imports);
scales[name] = {
files: files.length,
imports: imports.length,
// Reported, not asserted on its own: a corpus edit that collapsed the
// resolved share would still produce a "valid" fingerprint over far less.
resolved,
distinct_outcomes: outcomes.size,
ms: Number(timeResolution(lang, files, imports).toFixed(3)),
fingerprint: fingerprint(outcomes),
};
}
report[lang] = {
...scales,
scaling_ratio: Number((scales.large.ms / scales.small.ms / (LARGE / SMALL)).toFixed(3)),
// `scaling_ratio` divides the file count out, so it is scale-invariant and
// structurally cannot see a cost that grows with path DEPTH instead — and
// `buildSuffixIndex` (C#, Ruby) and Kotlin's `suffixByStem` both emit one
// entry per '/' in a path. Same file count, ~6x the components.
depth_ratio: Number((scales.deep.ms / scales.small.ms).toFixed(3)),
// Same measurement on the shared-leaf layout. Legitimately above the 1.8
// budget for go/csharp/dart — see the scope-of-claim note in the header.
collide_scaling_ratio: Number(
(scales.collide_large.ms / scales.collide.ms / (LARGE / SMALL)).toFixed(3),
),
fingerprint: scales.large.fingerprint,
};
}
// AFTER every timing arm, never interleaved with them: the heap arm allocates a
// 32k-path corpus and a ~75 MiB index, and leaving that garbage behind for the
// next language's timed loop to collect would tax an arm it has nothing to do
// with.
for (const lang of HEAP_LANGS) report[lang].heap = measureHeap(lang);
if (!CHECK) {
console.log(JSON.stringify(report, null, 2));
process.exit(0);
}
const baseline = JSON.parse(fs.readFileSync(BASELINE_PATH, 'utf-8'));
const failures = [];
for (const lang of LANGS) {
const got = report[lang];
const want = baseline.languages[lang];
if (got.fingerprint !== want.fingerprint) {
failures.push(
`${lang}: fingerprint drift ${got.fingerprint} != ${want.fingerprint} — the resolver ` +
`returned a DIFFERENT target set. That is a behaviour change, not a perf one; see the ` +
`parity harness in test/unit/scope-resolution/import-target-index-parity.test.ts.`,
);
}
// One shape, five facts, so the five budgets read side by side and the shared
// trailing sentence exists once instead of drifting into five wordings. Each
// `why` stays the arm's OWN: it is what tells a triager which corpus shape
// regressed, and flattening it would cost the message its whole value.
const timingChecks = [
{
label: 'scaling',
got: got.scaling_ratio,
budget: baseline.scaling_budget,
why: 'per-import cost grows with corpus size again.',
},
{
label: 'depth',
got: got.depth_ratio,
budget: baseline.depth_budget[lang],
why:
'cost grows with path DEPTH at a fixed file count, which scaling_ratio divides out and ' +
'cannot see.',
},
{
label: 'collide scaling',
got: got.collide_scaling_ratio,
budget: baseline.collide_scaling_budget[lang],
why:
'on the SHARED-LEAF layout (svcN/internal, SrcN/Models, a repeated basename per package) ' +
'per-import cost grew beyond what this shape already costs by construction.',
},
{
label: 'small arm ms',
got: got.small.ms,
budget: baseline.small_ms_ceiling[lang],
why:
'an ABSOLUTE bound, because a constant-factor regression that grows both arms equally ' +
'passes the ratio.',
},
{
label: 'collide arm ms',
got: got.collide.ms,
budget: baseline.collide_ms_ceiling[lang],
why: 'the ABSOLUTE bound on the shared-leaf layout.',
},
];
for (const check of timingChecks) {
if (check.got > check.budget) {
failures.push(
`${lang}: ${check.label} ${check.got} > budget ${check.budget} — ${check.why} ` +
`Timing arm: re-run on an idle machine before investigating.`,
);
}
}
for (const arm of ['deep', 'collide']) {
if (got[arm].resolved !== got.small.resolved) {
failures.push(
`${lang}: ${arm} arm resolved ${got[arm].resolved} vs small ${got.small.resolved} — the ` +
`${arm} arm was supposed to change ${arm === 'deep' ? 'path depth' : 'directory and file NAMING'} ` +
`and nothing else, so that it times the same workload. An arm that stopped resolving ` +
`would be timing the null path and its ratio would mean nothing.`,
);
}
// Count-neutral by design, so neutering the arm (DEEP_PAD = 0, a collideDir
// that forwards to uniqueDir) moves NO asserted count. Comparing the two
// fingerprints is the only arm that notices.
if (got[arm].fingerprint === got.small.fingerprint) {
failures.push(
`${lang}: ${arm}.fingerprint equals small.fingerprint — the ${arm} arm is resolving the ` +
`IDENTICAL corpus, so it measures nothing. ` +
`${arm === 'deep' ? 'DEEP_PAD is 0 or the padding stopped reaching buildFiles' : 'collideDir is returning the uniqueDir layout'}. ` +
`This is a deterministic arm: a re-run will not change it.`,
);
}
}
for (const scale of SCALES) {
for (const field of ['files', 'imports', 'resolved', 'distinct_outcomes', 'fingerprint']) {
if (got[scale][field] !== want[scale][field]) {
failures.push(
`${lang}.${scale}.${field}: ${got[scale][field]} != ${want[scale][field]} — the corpus ` +
`changed shape or the resolver changed its answer for this arm. Every scale is ` +
`asserted separately: the arms differ only in padding and layout, so a defect that ` +
`touches one of them alone moves nothing in the others.`,
);
}
}
}
}
// Driven by the BASELINE's keys, not the report's, so deleting a heap
// measurement fails instead of silently dropping the gate.
for (const [lang, ceiling] of Object.entries(baseline.heap_ceiling_bytes)) {
const heap = report[lang]?.heap;
if (heap == null) {
failures.push(
`${lang}: heap arm missing though heap_ceiling_bytes has a budget for it — the retained-` +
`index measurement was removed or skipped. It is the only arm that can see memory.`,
);
continue;
}
if (heap.bytes_large > ceiling) {
failures.push(
`${lang}: retained WorkspaceFileIndex ${heap.mib_large} MiB at ${heap.files_large} files ` +
`(${heap.bytes_large} B) > ceiling ${ceiling} B — buildSuffixIndex is O(files × depth) ` +
`and this is the ABSOLUTE bound on it (#2649). Deterministic: a re-run will not change it.`,
);
}
if (heap.ratio > baseline.heap_ratio_budget) {
failures.push(
`${lang}: retained-heap ratio ${heap.ratio} > budget ${baseline.heap_ratio_budget} ` +
`(${heap.bytes_small} B at ${heap.files_small} files -> ${heap.bytes_large} B at ` +
`${heap.files_large}) — the index stopped growing linearly in the file count.`,
);
}
}
console.log(JSON.stringify(report, null, 2));
if (failures.length > 0) {
console.error(`[import-target --check] FAIL\n - ${failures.join('\n - ')}`);
process.exit(1);
}
console.log('[import-target --check] PASS');

193
gitnexus/src/core/ingestion/import-resolvers/package-dir-index.ts Normal file
View file

@ -0,0 +1,193 @@
/**
* "Which files live DIRECTLY inside a directory whose path ends with
* `<pkgPath>`?" — the query Go's package resolution and C#'s namespace-directory
* fallback both answered with a full `allFilePaths` scan per import.
*
* Both scans ran the same predicate: normalize to forward slashes, apply the
* language's extension filter, find the FIRST `'/' + pkgPath + '/'` occurrence,
* and keep the file only if nothing after that occurrence contains a slash.
*
* That predicate depends only on the file's DIRECTORY, so it can be answered
* from an index built once per file set:
*
* let D = '/' + <normalized dir of the file> + '/'
* let P = '/' + pkgPath + '/'
* match ⟺ D.length >= P.length && D.indexOf(P) === D.length - P.length
*
* The right-hand side says two things at once, and BOTH are load-bearing:
* 1. `D` ends with `P` — the file's directory ends with `pkgPath`;
* 2. that trailing occurrence is the FIRST one — so `a/pkg/b/pkg/x.go` does
* NOT answer `pkg`, because the original `indexOf` found the earlier `/pkg/`
* and `b/pkg/x.go` still contained a slash. Dropping condition 2 looks like
* a cleanup and moves edges in every repository that nests a directory name
* inside itself (`internal/…/internal`, `Models/…/Models`).
*
* Candidates are narrowed by the directory's LAST segment rather than by
* indexing every directory suffix: a suffix map costs O(files × depth) entries,
* which is exactly the memory this codebase runs out of at kernel scale
* (#2649), while the last-segment bucket is O(directories) and is a superset of
* the matches (`D` ends with `P` ⟹ the dir's last segment is `pkgPath`'s last
* segment).
*
* Results keep Set-iteration order via the recorded `ord`, because the callers'
* scans emitted in that order and Go returns the whole list as the import
* target (one `ImportEdge` per file).
*
* Each language owns its own `WeakMap` memo and `accept` predicate, so the
* STORED index holds only that language's files — the build pass itself still
* walks every path it is handed once per language. That is not a polyglot tax
* in practice: `scope-resolution/pipeline/run.ts:673` rebuilds `allFilePaths`
* from the provider's own `parsedFiles`, so the set already contains only that
* language's files.
*/
interface IndexedFile {
readonly raw: string;
/**
* Position in `allFilePaths` iteration order. Still load-bearing:
* `filesDirectlyInPkgDir` sorts on it to interleave several directories back
* into the order the original single-pass scan emitted.
*/
readonly ord: number;
}
/**
* Deeply read-only on purpose. The memo hoist turned what used to be per-call
* scratch into state shared by every import in a run, and `readonly` on the
* PROPERTY still lets a caller do `idx.rootFiles.sort()` in place. Typing the
* containers as read-only makes the copy-before-mutating rule compile-enforced
* instead of comment-enforced — but `readonly` is erased at runtime and is not
* hard to widen back (the sibling Kotlin index documents `Array.isArray`'s
* `arg is any[]` predicate doing exactly that), so the one container callers
* read directly is handed out through `sortedRootFiles` rather than raw.
*/
export interface PackageDirIndex {
/** Last path segment of a directory → every normalized directory ending in it. */
readonly dirsByLastSegment: ReadonlyMap<string, readonly string[]>;
/** Normalized directory → the accepted files directly inside it, in Set order. */
readonly filesByDir: ReadonlyMap<string, readonly IndexedFile[]>;
/** Accepted files with no directory at all, in Set order. */
readonly rootFiles: readonly string[];
}
/**
* @param accept Runs on the normalized (forward-slash) path; return `false` to
* leave the file out of the index entirely.
*/
export function buildPackageDirIndex(
allFilePaths: ReadonlySet<string>,
accept: (normalized: string) => boolean,
): PackageDirIndex {
const dirsByLastSegment = new Map<string, string[]>();
const filesByDir = new Map<string, IndexedFile[]>();
const rootFiles: string[] = [];
let ord = 0;
for (const raw of allFilePaths) {
const ownOrd = ord++;
const normalized = raw.replace(/\\/g, '/');
if (!accept(normalized)) continue;
const lastSlash = normalized.lastIndexOf('/');
if (lastSlash < 0) {
// No directory: `'/x.go'.indexOf('/pkg/')` can never hit, so a root file
// answers no `pkgPath` query. Kept separately for Go's root-package leg.
rootFiles.push(raw);
continue;
}
const dir = normalized.slice(0, lastSlash);
let files = filesByDir.get(dir);
if (files === undefined) {
files = [];
filesByDir.set(dir, files);
const lastSegment = dir.slice(dir.lastIndexOf('/') + 1);
let dirs = dirsByLastSegment.get(lastSegment);
if (dirs === undefined) {
dirs = [];
dirsByLastSegment.set(lastSegment, dirs);
}
dirs.push(dir);
}
files.push({ raw, ord: ownOrd });
}
return { dirsByLastSegment, filesByDir, rootFiles };
}
/** Every indexed directory matching `pkgPath`, in first-seen order. */
function* matchingDirs(index: PackageDirIndex, pkgPath: string): Generator<readonly IndexedFile[]> {
const lastSegment = pkgPath.slice(pkgPath.lastIndexOf('/') + 1);
const dirs = index.dirsByLastSegment.get(lastSegment);
if (dirs === undefined) return;
const needle = `/${pkgPath}/`;
for (const dir of dirs) {
const haystack = `/${dir}/`;
// The length guard is not redundant: for a shorter `haystack`,
// `indexOf` returns -1 and `haystack.length - needle.length` can also be
// -1, which would report a bogus match.
if (haystack.length < needle.length) continue;
if (haystack.indexOf(needle) !== haystack.length - needle.length) continue;
const files = index.filesByDir.get(dir);
if (files !== undefined) yield files;
}
}
/**
* Every accepted file directly inside a directory ending with `pkgPath`, in
* `allFilePaths` iteration order.
*/
export function filesDirectlyInPkgDir(index: PackageDirIndex, pkgPath: string): string[] {
// The first bucket is held by reference, not copied into an accumulator: one
// matching directory is the overwhelmingly common case (every unique-leaf
// call, and any query whose package path has more than one segment), and it
// then reaches the `map` with zero intermediate copies.
//
// A second directory promotes that reference to a real accumulator, which is
// appended to once per file from then on — never re-spread per directory,
// because that costs O(files × dirs²) copies, which a monorepo carrying the
// same package directory under many services (`svcN/internal/models`, queried
// by Go's two-segment GOPATH tail) would pay on every import.
let first: readonly IndexedFile[] | null = null;
let merged: IndexedFile[] | null = null;
for (const files of matchingDirs(index, pkgPath)) {
if (first === null) {
first = files;
continue;
}
if (merged === null) merged = [...first];
for (const f of files) merged.push(f);
}
if (first === null) return [];
// One directory is already in Set order; several interleave and need merging
// back onto the order the original single-pass scan emitted.
if (merged === null) return first.map((f) => f.raw);
merged.sort((a, b) => a.ord - b.ord);
return merged.map((f) => f.raw);
}
/** Root-package files in sorted order. Copies: the index array is shared by
* every import in the run and the result leaves as an edge target list. */
export function sortedRootFiles(index: PackageDirIndex): string[] {
return [...index.rootFiles].sort();
}
/**
* The FIRST accepted file (in `allFilePaths` iteration order) directly inside a
* directory ending with `pkgPath`, or `null`.
*/
export function firstFileDirectlyInPkgDir(index: PackageDirIndex, pkgPath: string): string | null {
// Returning the FIRST match is already the minimum-`ord` answer, and it is
// the build loop that makes it so: `buildPackageDirIndex` appends a directory
// to its last-segment bucket at the moment it accepts that directory's first
// file, so bucket order IS ascending first-file-`ord` order. Comparing `ord`
// across the remaining directories can never improve on the first hit
// (differentially verified: 0 divergences). Change that append point — buffer
// the directories, sort them, populate `filesByDir` before `dirsByLastSegment`
// — and this early return silently starts answering with the wrong file.
for (const files of matchingDirs(index, pkgPath)) {
const first = files[0];
if (first !== undefined) return first.raw;
}
return null;
}

72
gitnexus/src/core/ingestion/import-resolvers/workspace-file-index.ts Normal file
View file

@ -0,0 +1,72 @@
/**
* Per-file-set workspace index for the import-target resolvers that need the
* shared `SuffixIndex` (C#, Ruby).
*
* The scope-resolution orchestrator passes the SAME `allFilePaths` Set object to
* every `resolveImportTarget` call in a pass (`pipeline/run.ts` builds it once),
* so memoizing on the Set's identity in a `WeakMap` turns the per-import
* "materialize two arrays + build a suffix index" cost into a one-time build.
*
* IMPORTANT for callers: the Set must be passed THROUGH, never copied. A
* defensive `new Set(allFilePaths)` in an adapter hands a fresh `WeakMap` key
* per call and silently restores the O(imports × files) behaviour — the exact
* bug PR #1918 shipped and had to fix in review (P1).
*
* Two layers guard that, and they guard different things:
* - ADAPTER BOUNDARY, where the defensive-copy hazard actually lives:
* `test/integration/<lang>-import-index-reuse.test.ts` (csharp and ruby for
* this index; go, dart, kotlin and python for the sibling ones) resolves
* through `<lang>ScopeResolver.resolveImportTarget` — the orchestrator
* adapter — and asserts the file set is traversed once per run (twice for
* C#, which builds two indexes). Kotlin and Python instead count index
* BUILDS from production (`languages/<lang>/index-stats.ts`); either way, a
* copy inserted in an adapter fails these.
* - RESOLVER LEVEL: `test/unit/scope-resolution/import-target-index-parity.test.ts`
* calls the resolvers directly, so it never crosses the adapter boundary and
* a copy there leaves it green. What it catches is a rescan reintroduced
* INSIDE a resolver, by counting how many times the Set is iterated.
*/
import { buildSuffixIndex, type SuffixIndex } from './utils.js';
export interface WorkspaceFileIndex {
/** Every path, backslashes normalized to `/`. Parallel to `all`. */
readonly normalized: string[];
/** Every path, exactly as it appears in the Set. Parallel to `normalized`. */
readonly all: string[];
/** Segment-suffix → first file (in Set iteration order) carrying that suffix. */
readonly index: SuffixIndex;
/**
* Normalized path → first raw path that normalizes to it. Answers "is there a
* file whose WHOLE path is X", which `index.get(X)` cannot: the suffix map
* conflates a whole-path hit with a `…/X` suffix hit, and C#'s
* `resolveDirectMatch` lets a whole-path match win over an earlier suffix
* match.
*/
readonly normToRaw: Map<string, string>;
}
const WORKSPACE_FILE_INDEX_CACHE = new WeakMap<ReadonlySet<string>, WorkspaceFileIndex>();
export function getWorkspaceFileIndex(allFilePaths: ReadonlySet<string>): WorkspaceFileIndex {
const cached = WORKSPACE_FILE_INDEX_CACHE.get(allFilePaths);
if (cached !== undefined) return cached;
const all = [...allFilePaths];
const normalized = all.map((f) => f.replace(/\\/g, '/'));
const normToRaw = new Map<string, string>();
for (let i = 0; i < normalized.length; i++) {
// First wins, mirroring the `for (const raw of allFilePaths)` scans this
// replaces: they returned on the first match in iteration order.
if (!normToRaw.has(normalized[i])) normToRaw.set(normalized[i], all[i]);
}
const built: WorkspaceFileIndex = {
normalized,
all,
index: buildSuffixIndex(normalized, all),
normToRaw,
};
WORKSPACE_FILE_INDEX_CACHE.set(allFilePaths, built);
return built;
}

125
gitnexus/src/core/ingestion/languages/csharp/import-target.ts
View file

@ -22,7 +22,15 @@
import type { ParsedImport, WorkspaceIndex } from 'gitnexus-shared';
import type { CSharpProjectConfig, CSharpNamespaceEvidence } from '../../language-config.js';
import { resolveCSharpImportInternal } from '../../import-resolvers/csharp.js';
import { buildSuffixIndex, type SuffixIndex } from '../../import-resolvers/utils.js';
import {
getWorkspaceFileIndex,
type WorkspaceFileIndex,
} from '../../import-resolvers/workspace-file-index.js';
import {
buildPackageDirIndex,
firstFileDirectlyInPkgDir,
type PackageDirIndex,
} from '../../import-resolvers/package-dir-index.js';
import { csharpSuffixFallbackAllowed } from '../../csharp-namespace-gate.js';
export interface CsharpResolveContext {
@ -32,25 +40,19 @@ export interface CsharpResolveContext {
readonly namespaces?: CSharpNamespaceEvidence;
}
/** Normalized file list + suffix index, built once per workspace `allFilePaths`. */
interface WorkspaceFileIndex {
readonly normalized: string[];
readonly all: string[];
readonly index: SuffixIndex;
}
/**
* Namespace-directory index over the `.cs` files, memoized on the Set's
* identity. Feeds `firstFileDirectlyInPkgDir` (in
* `import-resolvers/package-dir-index.ts`), which the no-csproj path calls once
* for the direct match and then up to once per stripped namespace prefix.
*/
const csharpDirIndexCache = new WeakMap<ReadonlySet<string>, PackageDirIndex>();
// Memoize on Set identity: the orchestrator passes the SAME `allFilePaths`
// Set through every `resolveImportTarget` call in a pass, so this rebuilds
// the normalized list + suffix index once instead of once per import (#1881 #2).
const workspaceFileIndexCache = new WeakMap<ReadonlySet<string>, WorkspaceFileIndex>();
function getWorkspaceFileIndex(allFilePaths: ReadonlySet<string>): WorkspaceFileIndex {
const cached = workspaceFileIndexCache.get(allFilePaths);
function getCsharpDirIndex(allFilePaths: ReadonlySet<string>): PackageDirIndex {
const cached = csharpDirIndexCache.get(allFilePaths);
if (cached) return cached;
const all = [...allFilePaths];
const normalized = all.map((f) => f.replace(/\\/g, '/'));
const built: WorkspaceFileIndex = { normalized, all, index: buildSuffixIndex(normalized, all) };
workspaceFileIndexCache.set(allFilePaths, built);
const built = buildPackageDirIndex(allFilePaths, (normalized) => normalized.endsWith('.cs'));
csharpDirIndexCache.set(allFilePaths, built);
return built;
}
@ -101,12 +103,19 @@ export function resolveCsharpImportTarget(
}
// Exact file / nested-suffix / namespace-dir direct-child match.
const direct = resolveDirectMatch(ctx.allFilePaths, pathLike);
//
// The no-csproj path used to take the raw Set and re-scan it — up to eight
// full workspace passes for a four-segment `using` — past the memoized index
// sitting right there for the csproj branch (#2878). Both legs now read the
// same per-run indexes.
const ws = getWorkspaceFileIndex(ctx.allFilePaths);
const dirs = getCsharpDirIndex(ctx.allFilePaths);
const direct = resolveDirectMatch(ws, dirs, pathLike);
if (direct !== null) return direct;
// Progressive prefix stripping — mirrors csproj's root-namespace mapping
// without the csproj.
return resolveByProgressiveStripping(ctx.allFilePaths, pathLike);
return resolveByProgressiveStripping(ws, dirs, pathLike);
}
/**
@ -131,40 +140,28 @@ function narrowContext(workspaceIndex: WorkspaceIndex): CsharpResolveContext | n
* exact whole-path file > nested suffix file > first `.cs` directly inside
* the namespace directory.
*/
function resolveDirectMatch(allFilePaths: ReadonlySet<string>, pathLike: string): string | null {
function resolveDirectMatch(
ws: WorkspaceFileIndex,
dirs: PackageDirIndex,
pathLike: string,
): string | null {
const exactName = `${pathLike}.cs`;
const nestedSuffix = `/${exactName}`;
let suffixFile: string | null = null;
for (const raw of allFilePaths) {
const f = raw.replace(/\\/g, '/');
if (!f.endsWith('.cs')) continue;
if (f === exactName) return raw; // exact whole-path match wins
if (suffixFile === null && f.endsWith(nestedSuffix)) suffixFile = raw;
}
if (suffixFile !== null) return suffixFile;
return findDirectChild(allFilePaths, pathLike);
}
/**
* First `.cs` file that lives directly inside the namespace directory
* `dirSegment` (at repo root or nested under a project prefix), not deeper.
* The legacy resolver emits all of them; the scope-resolver contract is
* single-target so we take one.
*/
function findDirectChild(allFilePaths: ReadonlySet<string>, dirSegment: string): string | null {
const dirPrefix = `${dirSegment}/`;
const nestedDirPrefix = `/${dirPrefix}`;
for (const raw of allFilePaths) {
const f = raw.replace(/\\/g, '/');
if (!f.endsWith('.cs')) continue;
const atRoot = f.startsWith(dirPrefix);
const atNested = f.includes(nestedDirPrefix);
if (!atRoot && !atNested) continue;
const idx = atRoot ? 0 : f.indexOf(nestedDirPrefix) + 1;
const after = f.slice(idx + dirPrefix.length);
if (after.length > 0 && !after.includes('/')) return raw;
}
return null;
// An exact whole-path match wins even when a `…/<exactName>` suffix match
// appeared EARLIER in iteration order, so the two lookups stay separate:
// `index.get` conflates them and would return the earlier suffix hit.
const exact = ws.normToRaw.get(exactName);
if (exact !== undefined) return exact;
// No whole-path file exists, so every segment-suffix hit is a `/<exactName>`
// match and `index.get` yields the first one in iteration order — exactly the
// `suffixFile` the scan kept. Only a `.cs` file can carry a `.cs` suffix key,
// so the old `endsWith('.cs')` filter is implied.
const suffixFile = ws.index.get(exactName);
if (suffixFile !== undefined) return suffixFile;
// First `.cs` file living directly inside the namespace directory `pathLike`
// (at repo root or nested under a project prefix), not deeper. The legacy
// resolver emits all of them; the scope-resolver contract is single-target so
// we take one.
return firstFileDirectlyInPkgDir(dirs, pathLike);
}
/**
@ -174,26 +171,20 @@ function findDirectChild(allFilePaths: ReadonlySet<string>, dirSegment: string):
* prefix (the scope-resolver layer has no csproj to consult).
*/
function resolveByProgressiveStripping(
allFilePaths: ReadonlySet<string>,
ws: WorkspaceFileIndex,
dirs: PackageDirIndex,
pathLike: string,
): string | null {
const segments = pathLike.split('/').filter(Boolean);
for (let skip = 1; skip < segments.length; skip++) {
const tail = segments.slice(skip).join('/');
if (tail === '') continue;
const tailFile = `${tail}.cs`;
const tailSuffix = `/${tailFile}`;
let tailFileMatch: string | null = null;
for (const raw of allFilePaths) {
const f = raw.replace(/\\/g, '/');
if (!f.endsWith('.cs')) continue;
if (f === tailFile || f.endsWith(tailSuffix)) {
tailFileMatch = raw;
break;
}
}
if (tailFileMatch !== null) return tailFileMatch;
const child = findDirectChild(allFilePaths, tail);
// `f === tailFile || f.endsWith('/' + tailFile)`, first in iteration order —
// no exact-wins rule here, unlike `resolveDirectMatch`, so the conflated
// suffix lookup is the right one.
const tailFileMatch = ws.index.get(`${tail}.cs`);
if (tailFileMatch !== undefined) return tailFileMatch;
const child = firstFileDirectlyInPkgDir(dirs, tail);
if (child !== null) return child;
}
return null;

65
gitnexus/src/core/ingestion/languages/dart/import-target.ts
View file

@ -15,6 +15,60 @@
import { DART_HERITAGE_PREFIX } from './interpret.js';
/**
* Basename → files carrying it, in `allFilePaths` iteration order, memoized on
* the Set's identity (#2879).
*
* Both resolution legs answered `fp === candidate || fp.endsWith('/' + candidate)`
* with a full workspace scan, and the `package:` leg ran one scan PER candidate
* — for an external package both candidates miss, so both scans always ran to
* completion. The orchestrator passes the same Set to every import in a pass,
* so the index is built once per run.
*
* Bucketing by basename is exact rather than a heuristic: a path satisfying
* either arm of the match ends with `candidate`, so its last `/`-delimited
* segment is `candidate`'s. Paths are indexed RAW, without slash normalization,
* because the scans this replaces compared raw paths too — normalizing here
* would start resolving backslash paths that previously returned null.
*/
interface DartFileIndex {
readonly byBasename: Map<string, string[]>;
}
const DART_FILE_INDEX_CACHE = new WeakMap<ReadonlySet<string>, DartFileIndex>();
function getDartFileIndex(allFilePaths: ReadonlySet<string>): DartFileIndex {
const cached = DART_FILE_INDEX_CACHE.get(allFilePaths);
if (cached !== undefined) return cached;
const byBasename = new Map<string, string[]>();
for (const fp of allFilePaths) {
const base = fp.slice(fp.lastIndexOf('/') + 1);
let bucket = byBasename.get(base);
if (bucket === undefined) {
bucket = [];
byBasename.set(base, bucket);
}
bucket.push(fp);
}
const built: DartFileIndex = { byBasename };
DART_FILE_INDEX_CACHE.set(allFilePaths, built);
return built;
}
/** First file (in Set-iteration order) that IS `candidate` or ends with
* `/<candidate>` — the exact predicate of the scans this replaces. */
function findByPathSuffix(allFilePaths: ReadonlySet<string>, candidate: string): string | null {
const bucket = getDartFileIndex(allFilePaths).byBasename.get(
candidate.slice(candidate.lastIndexOf('/') + 1),
);
if (bucket === undefined) return null;
const suffix = '/' + candidate;
for (const fp of bucket) {
if (fp === candidate || fp.endsWith(suffix)) return fp;
}
return null;
}
/** Resolve a relative path against the importer's directory, normalizing
* `.`/`..` segments, then confirm it exists in the workspace file set. */
function resolveRelative(
@ -33,10 +87,7 @@ function resolveRelative(
const target = parts.join('/');
if (allFilePaths.has(target)) return target;
// Suffix fallback for absolute/rooted workspace paths.
for (const fp of allFilePaths) {
if (fp === target || fp.endsWith('/' + target)) return fp;
}
return null;
return findByPathSuffix(allFilePaths, target);
}
export function resolveDartImportTarget(
@ -56,10 +107,10 @@ export function resolveDartImportTarget(
const slash = targetRaw.indexOf('/');
if (slash === -1) return null;
const relPath = targetRaw.slice(slash + 1);
// Candidate priority is load-bearing: `lib/<rel>` before bare `<rel>`.
for (const candidate of [`lib/${relPath}`, relPath]) {
for (const fp of allFilePaths) {
if (fp === candidate || fp.endsWith('/' + candidate)) return fp;
}
const hit = findByPathSuffix(allFilePaths, candidate);
if (hit !== null) return hit;
}
return null; // external package
}

64
gitnexus/src/core/ingestion/languages/go/import-target.ts
View file

@ -1,4 +1,10 @@
import type { GoModuleConfig } from '../../language-config.js';
import {
buildPackageDirIndex,
filesDirectlyInPkgDir,
sortedRootFiles,
type PackageDirIndex,
} from '../../import-resolvers/package-dir-index.js';
/**
* Resolve a Go import path to ALL .go files in the matching package directory.
@ -50,34 +56,40 @@ export function resolveGoImportTarget(
return null;
}
/**
* Package index over the file set, memoized on the Set's identity (#2877).
*
* Every leg above used to walk all of `allFilePaths`, and the GOPATH fallback
* walks once per path segment — so a single unresolved import (which is most of
* them: stdlib and third-party module paths run the whole cascade to completion
* before returning null) cost several full workspace scans, making resolution
* O(imports × files).
*
* The orchestrator hands the same Set to every import in a pass, so the index
* is built once per run. `resolveGoImportTarget` must therefore never copy the
* Set before this point — see `import-resolvers/workspace-file-index.ts`.
*/
const GO_PACKAGE_INDEX_CACHE = new WeakMap<ReadonlySet<string>, PackageDirIndex>();
/** Go packages exclude `_test.go` files: they are a separate package. */
function isGoPackageFile(normalized: string): boolean {
return normalized.endsWith('.go') && !normalized.endsWith('_test.go');
}
function getGoPackageIndex(allFilePaths: ReadonlySet<string>): PackageDirIndex {
const cached = GO_PACKAGE_INDEX_CACHE.get(allFilePaths);
if (cached !== undefined) return cached;
const built = buildPackageDirIndex(allFilePaths, isGoPackageFile);
GO_PACKAGE_INDEX_CACHE.set(allFilePaths, built);
return built;
}
function findRootPackageFiles(allFilePaths: ReadonlySet<string>): string[] {
const result: string[] = [];
for (const raw of allFilePaths) {
const normalized = raw.replace(/\\/g, '/');
if (normalized.includes('/')) continue;
if (!normalized.endsWith('.go') || normalized.endsWith('_test.go')) continue;
result.push(raw);
}
return result.sort();
return sortedRootFiles(getGoPackageIndex(allFilePaths));
}
function findAllFilesInPkgDir(allFilePaths: ReadonlySet<string>, pkgPath: string): string[] {
const pkgDir = '/' + pkgPath + '/';
const result: string[] = [];
for (const raw of allFilePaths) {
const normalized = '/' + raw.replace(/\\/g, '/');
if (!normalized.includes(pkgDir)) continue;
if (!normalized.endsWith('.go') || normalized.endsWith('_test.go')) continue;
// Ensure file is directly in the package directory (not a subdirectory)
const afterPkg = normalized.substring(normalized.indexOf(pkgDir) + pkgDir.length);
if (!afterPkg.includes('/')) result.push(raw);
}
return result;
}
/** Preserved for backward compat. */
export interface GoResolveContext {
readonly fromFile: string;
readonly allFilePaths: ReadonlySet<string>;
readonly goModule?: GoModuleConfig;
// Deliberately UNSORTED, unlike the root leg: the previous single-pass scan
// emitted in Set-iteration order and `filesDirectlyInPkgDir` reproduces it.
return filesDirectlyInPkgDir(getGoPackageIndex(allFilePaths), pkgPath);
}

2
gitnexus/src/core/ingestion/languages/go/index.ts
View file

@ -10,7 +10,7 @@ export { synthesizeGoTypeBindings } from './type-binding.js';
export { goArityCompatibility } from './arity.js';
export { goMergeBindings } from './merge-bindings.js';
export { goBindingScopeFor, goImportOwningScope, goReceiverBinding } from './simple-hooks.js';
export { resolveGoImportTarget, type GoResolveContext } from './import-target.js';
export { resolveGoImportTarget } from './import-target.js';
export { populateGoPackageSiblings } from './package-siblings.js';
export { populateGoRangeBindings } from './range-binding.js';
export { detectGoInterfaceImplementations } from './interface-impls.js';

13
gitnexus/src/core/ingestion/languages/ruby/import-target.ts
View file

@ -8,7 +8,7 @@
*/
import { resolveRubyImportInternal } from '../../import-resolvers/ruby.js';
import { buildSuffixIndex } from '../../import-resolvers/utils.js';
import { getWorkspaceFileIndex } from '../../import-resolvers/workspace-file-index.js';
import { isHeritageMarker } from '../../utils/heritage-marker.js';
export interface RubyResolveContext {
@ -100,9 +100,10 @@ function resolveRelative(
* via suffix matching using the existing Ruby import resolver.
*/
function resolveBare(targetRaw: string, allFilePaths: ReadonlySet<string>): string | null {
const normalizedFileList = [...allFilePaths].map((f) => f.replace(/\\/g, '/'));
const allFileList = [...allFilePaths];
const index = buildSuffixIndex(normalizedFileList, allFileList);
return resolveRubyImportInternal(targetRaw, normalizedFileList, allFileList, index);
// Was: two array materializations plus a full `buildSuffixIndex` per require,
// thrown away on return — every require paid to index every file in the repo
// (#2880). `buildSuffixIndex` is a pure function of the file set, so this is a
// hoist, not a behaviour change.
const { normalized, all, index } = getWorkspaceFileIndex(allFilePaths);
return resolveRubyImportInternal(targetRaw, normalized, all, index);
}

161
gitnexus/test/helpers/counting-file-set.ts Normal file
View file

@ -0,0 +1,161 @@
/**
* A `Set<string>` that counts how many times it is TRAVERSED in full — the
* measuring instrument behind the import-target index-reuse guards
* (`test/unit/scope-resolution/import-target-index-parity.test.ts` and the
* per-language `test/integration/<lang>-import-index-reuse.test.ts` files).
*
* ## Why a counting Set rather than a production build counter
*
* Kotlin and Python count index BUILDS from production (`languages/<lang>/
* index-stats.ts`). That catches the per-import rebuild, but it is blind to a
* scan added BESIDE a reused index: the cache still hits, the build count still
* reads 1. Counting traversals of the file set instead needs no production
* surface at all and catches both failures with one number:
*
* - an adapter that copies the set (`new Set(allFilePaths)`) hands a fresh
* `WeakMap` key per import, so the count rises to the import count;
* - a scan reintroduced next to the index raises the count by one per scan.
*
* The second is the case the benchmark provably cannot see: a full workspace
* scan on 1-in-32 imports passes every timing arm in `bench/import-target/`
* (measured, see `baselines.json` `_blind_spot`) while this counter reads 14
* instead of 1.
*
* ## Why every traversal entry point is overridden
*
* `for…of`, spread and `new Set(x)` go through `[Symbol.iterator]`, but
* `forEach`, `values`, `keys` and `entries` walk the same elements without
* touching it. Overriding only `[Symbol.iterator]` would let a reintroduced
* scan spelled `allFilePaths.forEach(…)` or `[...allFilePaths.values()]` sit
* under the guard uncounted. `Set.prototype[Symbol.iterator]` and
* `Set.prototype.values` are the same function object in the spec, but the
* `super.*` lookups below resolve on `Set.prototype`, not on this subclass, so
* a single traversal is still counted exactly once.
*
* ## What it does NOT see
*
* Only traversals of the SET. Once an index has materialized the file list into
* an array (`WorkspaceFileIndex.normalized` / `.all`, Dart's basename buckets,
* `PackageDirIndex.filesByDir`), a scan over that array is invisible here.
* Guarding that would mean either instrumenting production or proxying an index
* internal; see the header of the parity test for why neither is in place.
*
* `instanceof Set` still holds, which matters: C#'s `narrowContext` rejects a
* workspace context whose `allFilePaths` is not a `Set`, so a plain object with
* a counter would silently resolve nothing and every assertion would pass on
* `null === null`.
*
* `expectDistinctFileSetsGetOwnIndex` below is the one arm of those guards that
* is identical in every language once the four values that differ are named, so
* it lives here beside the instrument it reads rather than in each guard.
*/
import { expect } from 'vitest';
import type { ScopeResolver } from '../../src/core/ingestion/scope-resolution/contract/scope-resolver.js';
export class CountingSet extends Set<string> {
/** Full traversals of this set, by any entry point. */
scans = 0;
override [Symbol.iterator](): SetIterator<string> {
this.scans++;
return super[Symbol.iterator]();
}
override values(): SetIterator<string> {
this.scans++;
return super.values();
}
override keys(): SetIterator<string> {
this.scans++;
return super.keys();
}
override entries(): SetIterator<[string, string]> {
this.scans++;
return super.entries();
}
override forEach(
callbackfn: (value: string, value2: string, set: Set<string>) => void,
thisArg?: unknown,
): void {
this.scans++;
super.forEach(callbackfn, thisArg);
}
}
/**
* Everything that differs between the per-language spellings of the
* distinct-file-set arm. Nothing else about that arm varies, which is why it is
* a parameter list rather than four copies.
*/
export interface DistinctFileSetArm {
/**
* The orchestrator adapter under test — `<lang>ScopeResolver
* .resolveImportTarget`, NOT the language's `resolve<Lang>ImportTarget`. The
* adapter is the surface the unit parity test cannot reach, and the surface a
* defensive `new Set(allFilePaths)` copy would break.
*/
readonly resolveImportTarget: ScopeResolver['resolveImportTarget'];
/**
* Builds one workspace. Called twice, and must return a FRESH `CountingSet`
* each time: the two sets being distinct objects is the whole subject of the
* arm, since the indexes are memoized on Set identity via a `WeakMap`.
*/
readonly buildWorkspace: () => CountingSet;
/** The import spelling to resolve, as it would appear in source. */
readonly targetRaw: string;
/** The importing file the target is resolved against. */
readonly fromFile: string;
/**
* The resolver's `resolutionConfig` argument (Go's `{ modulePath }`). Not
* optional: the languages that take none pass `undefined` in the open, so a
* call site never hides which adapters read this channel behind an omission.
*/
readonly resolutionConfig: unknown;
/**
* What `targetRaw` must resolve to — a path for the string-returning
* resolvers, a path list for Go. Never `null`: the pairing rule below exists
* precisely because an adapter that has stopped resolving anything returns
* `null` and still posts a perfect scan count, so a `null` expectation would
* reinstate the hole it closes.
*/
readonly expected: string | readonly string[];
/** Traversals of ONE file set that full reuse permits: one per index built. */
readonly expectedScans: number;
}
/** Resolutions driven against each file set before the counts are read. */
const DISTINCT_FILE_SET_REPEATS = 20;
/**
* Assert that two independently built file sets each get their own index, built
* once — no stale reuse of one set's index for the other, and no rebuild per
* import within either.
*
* The repeats are driven bare, following the equivalent arm in
* `test/unit/scope-resolution/import-target-index-parity.test.ts`: asserting
* inside the loop restates one bit of information forty times. The two asserted
* resolutions afterwards are the pairing rule the guards' headers state — a
* scan count must never be the count of an adapter that resolves nothing.
*/
export function expectDistinctFileSetsGetOwnIndex(arm: DistinctFileSetArm): void {
const a = arm.buildWorkspace();
const b = arm.buildWorkspace();
for (let i = 0; i < DISTINCT_FILE_SET_REPEATS; i++) {
arm.resolveImportTarget(arm.targetRaw, arm.fromFile, a, arm.resolutionConfig);
arm.resolveImportTarget(arm.targetRaw, arm.fromFile, b, arm.resolutionConfig);
}
expect(arm.resolveImportTarget(arm.targetRaw, arm.fromFile, a, arm.resolutionConfig)).toEqual(
arm.expected,
);
expect(arm.resolveImportTarget(arm.targetRaw, arm.fromFile, b, arm.resolutionConfig)).toEqual(
arm.expected,
);
expect(a.scans).toBe(arm.expectedScans);
expect(b.scans).toBe(arm.expectedScans);
}

116
gitnexus/test/integration/csharp-import-index-reuse.test.ts Normal file
View file

@ -0,0 +1,116 @@
/**
* Production-path regression guard for the C# import-resolution indexes (#2878).
*
* The no-csproj `using` path reads TWO per-file-set indexes, each memoized on
* the `allFilePaths` Set identity via its own WeakMap: the shared
* `getWorkspaceFileIndex` (`import-resolvers/workspace-file-index.ts`, used by
* both the csproj and no-csproj legs) and `getCsharpDirIndex`
* (`languages/csharp/import-target.ts`, the namespace-directory index behind
* `firstFileDirectlyInPkgDir`). A four-segment `using` used to cost up to eight
* full workspace passes.
*
* Resolution reaches both through `csharpScopeResolver.resolveImportTarget` —
* the orchestrator adapter — not by calling `resolveCsharpImportTarget`
* directly the way the unit parity test does. The adapter must therefore pass
* the Set THROUGH; a defensive copy (`new Set(allFilePaths)`) would hand a
* fresh WeakMap key per call and rebuild BOTH indexes on every `using`,
* restoring the O(usings × files) behaviour this replaced. Python hit exactly
* that (PR #1918 review P1), and the parity test cannot see it: it never
* crosses the adapter.
*
* C# is also the reason the counting instrument has to be a real `Set`
* subclass: `narrowContext` rejects a workspace context whose `allFilePaths`
* fails `instanceof Set`, and a rejected context resolves nothing — every
* assertion would then pass on `null === null`.
*
* The traversal-count assertions are the perf guard. They are paired with
* result assertions on purpose: a count of 2 is equally true of an adapter that
* has stopped resolving anything at all, so counting alone would stay green
* while every C# IMPORTS edge disappeared.
*/
import { describe, it, expect } from 'vitest';
import { csharpScopeResolver } from '../../src/core/ingestion/languages/csharp/scope-resolver.js';
import { CountingSet, expectDistinctFileSetsGetOwnIndex } from '../helpers/counting-file-set.js';
const { resolveImportTarget } = csharpScopeResolver;
const FROM_FILE = 'App/Program.cs';
/**
* A synthetic C# solution with no `.csproj` discovered, which is the leg #2878
* moved onto the indexes. `App/Models/User.cs` answers the whole-path lookup,
* `App/Services/` answers the namespace-directory lookup, and `Domain/Order.cs`
* is reachable only after progressive prefix stripping.
*/
function buildWorkspace(fileCount: number): CountingSet {
const files: string[] = [];
for (let i = 0; i < fileCount; i++) {
files.push(`App/Services/Service${String(i).padStart(5, '0')}.cs`);
}
files.push('App/Models/User.cs');
files.push('Domain/Order.cs');
files.push('App/Program.cs');
return new CountingSet(files);
}
describe('C# import resolution — index reuse across usings (#2878)', () => {
it('builds each index once for many usings over a stable file set', () => {
const files = buildWorkspace(300);
const resolved: (string | readonly string[] | null)[] = [];
for (let i = 0; i < 200; i++) {
// A whole-path hit, a namespace-directory hit, and a miss that runs the
// full progressive-stripping cascade — the case that used to re-scan the
// workspace once per stripped prefix.
resolved.push(resolveImportTarget('App.Models.User', FROM_FILE, files, undefined));
resolved.push(resolveImportTarget('App.Services', FROM_FILE, files, undefined));
resolved.push(
resolveImportTarget(`Vendor${i}.Ghost.Deep.Missing`, FROM_FILE, files, undefined),
);
}
// Two passes: the shared workspace/suffix index and the namespace-dir index.
expect(files.scans).toBe(2);
// Paired result assertion — a count of 2 must not be the count of an
// adapter that resolves nothing.
expect(resolved[0]).toBe('App/Models/User.cs');
expect(resolved[1]).toBe('App/Services/Service00000.cs');
expect(resolved[2]).toBeNull();
});
it('a distinct file set gets its own indexes (no stale cross-run reuse)', () => {
expectDistinctFileSetsGetOwnIndex({
resolveImportTarget,
buildWorkspace: () => buildWorkspace(20),
targetRaw: 'App.Models.User',
fromFile: FROM_FILE,
resolutionConfig: undefined,
expected: 'App/Models/User.cs',
// Two, not one: the shared workspace/suffix index and the namespace-dir
// index are separate WeakMaps over the same Set.
expectedScans: 2,
});
});
it('still resolves real usings correctly (the perf test is not vacuous)', () => {
const files = buildWorkspace(5);
// Whole-path match on the namespace path.
expect(resolveImportTarget('App.Models.User', FROM_FILE, files, undefined)).toBe(
'App/Models/User.cs',
);
// First `.cs` living directly inside the namespace directory.
expect(resolveImportTarget('App.Services', FROM_FILE, files, undefined)).toBe(
'App/Services/Service00000.cs',
);
// Progressive prefix stripping: the repo has no `CrossFile/` prefix.
expect(resolveImportTarget('CrossFile.Domain.Order', FROM_FILE, files, undefined)).toBe(
'Domain/Order.cs',
);
// BCL usings stay gated (#1881) and unknown namespaces resolve to nothing.
expect(resolveImportTarget('System.Threading.Tasks', FROM_FILE, files, undefined)).toBeNull();
expect(resolveImportTarget('Vendor.Ghost.Missing', FROM_FILE, files, undefined)).toBeNull();
});
});

108
gitnexus/test/integration/dart-import-index-reuse.test.ts Normal file
View file

@ -0,0 +1,108 @@
/**
* Production-path regression guard for the Dart import-resolution index (#2879).
*
* The basename index (`getDartFileIndex` in `languages/dart/import-target.ts`)
* is memoized on the `allFilePaths` Set identity via a WeakMap. Resolution
* reaches it through `dartScopeResolver.resolveImportTarget` — the orchestrator
* adapter — not by calling `resolveDartImportTarget` directly the way the unit
* parity test does. The adapter must therefore pass the Set THROUGH; a
* defensive copy (`new Set(allFilePaths)`) would hand a fresh WeakMap key per
* call and rebuild the index on every import, restoring the O(imports × files)
* behaviour this replaced. Python hit exactly that (PR #1918 review P1), and
* the parity test cannot see it: it never crosses the adapter.
*
* Dart is the language the benchmark's measured blind spot was quantified on
* (`bench/import-target/baselines.json` `_blind_spot`), so its adapter is the
* one with the least timing cover.
*
* The traversal-count assertions are the perf guard. They are paired with
* result assertions on purpose: a count of 1 is equally true of an adapter that
* has stopped resolving anything at all, so counting alone would stay green
* while every Dart IMPORTS edge disappeared.
*/
import { describe, it, expect } from 'vitest';
import { dartScopeResolver } from '../../src/core/ingestion/languages/dart/scope-resolver.js';
import { CountingSet, expectDistinctFileSetsGetOwnIndex } from '../helpers/counting-file-set.js';
const { resolveImportTarget } = dartScopeResolver;
const FROM_FILE = 'lib/main.dart';
/**
* A synthetic Dart package: many library files under `lib/src/`, plus the two
* targets the `package:` leg addresses — one reachable as `lib/<rel>` and one
* only as bare `<rel>`, which is the second candidate and therefore the leg
* that used to run a second full scan for every external import.
*/
function buildWorkspace(fileCount: number): CountingSet {
const files: string[] = [];
for (let i = 0; i < fileCount; i++) {
files.push(`lib/src/widget${String(i).padStart(5, '0')}.dart`);
}
files.push('lib/models.dart');
files.push('lib/src/util.dart');
files.push('tool/generate.dart');
files.push('lib/main.dart');
return new CountingSet(files);
}
describe('Dart import resolution — index reuse across imports (#2879)', () => {
it('builds the file index once for many imports over a stable file set', () => {
const files = buildWorkspace(300);
const resolved: (string | readonly string[] | null)[] = [];
for (let i = 0; i < 200; i++) {
// Three shapes: an in-package hit through `lib/<rel>`, a bare-`<rel>` hit
// that only the SECOND candidate answers, and an external package whose
// two candidates both miss — the case that used to cost two full
// workspace scans per import.
resolved.push(resolveImportTarget('package:app/models.dart', FROM_FILE, files));
resolved.push(resolveImportTarget('package:app/tool/generate.dart', FROM_FILE, files));
resolved.push(resolveImportTarget(`package:vendor${i}/ghost${i}.dart`, FROM_FILE, files));
}
expect(files.scans).toBe(1);
// Paired result assertion — a count of 1 must not be the count of an
// adapter that resolves nothing.
expect(resolved[0]).toBe('lib/models.dart');
expect(resolved[1]).toBe('tool/generate.dart');
expect(resolved[2]).toBeNull();
});
it('a distinct file set gets its own index (no stale cross-run reuse)', () => {
expectDistinctFileSetsGetOwnIndex({
resolveImportTarget,
buildWorkspace: () => buildWorkspace(20),
targetRaw: 'package:app/models.dart',
fromFile: FROM_FILE,
resolutionConfig: undefined,
expected: 'lib/models.dart',
expectedScans: 1,
});
});
it('still resolves real imports correctly (the perf test is not vacuous)', () => {
const files = buildWorkspace(5);
// `package:` leg, first candidate: `lib/<rel>`.
expect(resolveImportTarget('package:app/models.dart', FROM_FILE, files)).toBe(
'lib/models.dart',
);
// `package:` leg, second candidate: bare `<rel>`, reached only after
// `lib/<rel>` misses entirely.
expect(resolveImportTarget('package:app/tool/generate.dart', FROM_FILE, files)).toBe(
'tool/generate.dart',
);
// Relative import against the importer's directory.
expect(resolveImportTarget('src/util.dart', FROM_FILE, files)).toBe('lib/src/util.dart');
expect(resolveImportTarget('./src/util.dart', FROM_FILE, files)).toBe('lib/src/util.dart');
expect(resolveImportTarget('../models.dart', 'lib/src/main.dart', files)).toBe(
'lib/models.dart',
);
// SDK imports and external packages resolve to nothing in the workspace.
expect(resolveImportTarget('dart:core', FROM_FILE, files)).toBeNull();
expect(resolveImportTarget('package:collection/collection.dart', FROM_FILE, files)).toBeNull();
});
});

121
gitnexus/test/integration/go-import-index-reuse.test.ts Normal file
View file

@ -0,0 +1,121 @@
/**
* Production-path regression guard for the Go import-resolution index (#2877).
*
* The package-directory index (`getGoPackageIndex` in
* `languages/go/import-target.ts`) is memoized on the `allFilePaths` Set
* identity via a WeakMap. Resolution reaches it through
* `goScopeResolver.resolveImportTarget` — the orchestrator adapter — not by
* calling `resolveGoImportTarget` directly the way the unit parity test does.
* The adapter must therefore pass the Set THROUGH; a defensive copy
* (`new Set(allFilePaths)`) would hand a fresh WeakMap key per call and rebuild
* the index on every import, restoring the O(imports × files) behaviour this
* replaced. Python hit exactly that (PR #1918 review P1), and the parity test
* cannot see it: it never crosses the adapter.
*
* Kotlin and Python count index BUILDS from production (`index-stats.ts`).
* These four use `CountingSet` (`test/helpers/counting-file-set.ts`) instead,
* which counts full traversals of the file set and so catches BOTH the
* per-import rebuild and a scan reintroduced beside a reused index — with no
* production surface added for a test-only observation.
*
* The traversal-count assertions are the perf guard. They are paired with
* result assertions on purpose: a count of 1 is equally true of an adapter that
* has stopped resolving anything at all, so counting alone would stay green
* while every Go IMPORTS edge disappeared.
*/
import { describe, it, expect } from 'vitest';
import { goScopeResolver } from '../../src/core/ingestion/languages/go/scope-resolver.js';
import { CountingSet, expectDistinctFileSetsGetOwnIndex } from '../helpers/counting-file-set.js';
// `resolveImportTarget` is a required member of `ScopeResolver`, so this is a
// plain read — no optional call, and no `toBeDefined()` guarding a branch that
// cannot be taken.
const { resolveImportTarget } = goScopeResolver;
/** The value `loadGoModulePath` produces for a repo with a `go.mod`. */
const GO_MODULE = { modulePath: 'example.com/mod' };
const FROM_FILE = 'main.go';
/**
* A synthetic Go module: many sibling packages under `internal/`, one package
* with two real files plus a `_test.go` that the package leg must exclude, and
* a root-package file for the `targetRaw === modulePath` leg.
*/
function buildWorkspace(pkgCount: number): CountingSet {
const files: string[] = [];
for (let i = 0; i < pkgCount; i++) {
files.push(`internal/pkg${String(i).padStart(5, '0')}/service.go`);
}
files.push('internal/models/user.go');
files.push('internal/models/order.go');
files.push('internal/models/user_test.go');
files.push('main.go');
return new CountingSet(files);
}
describe('Go import resolution — index reuse across imports (#2877)', () => {
it('builds the package index once for many imports over a stable file set', () => {
const files = buildWorkspace(300);
const resolved: (string | readonly string[] | null)[] = [];
for (let i = 0; i < 200; i++) {
// Three shapes that between them reach every leg: the module-relative
// package leg, the root-package leg, and a third-party path that misses
// and so runs the whole GOPATH suffix cascade to completion — the case
// that used to cost one full workspace scan per path segment.
resolved.push(
resolveImportTarget('example.com/mod/internal/models', FROM_FILE, files, GO_MODULE),
);
resolved.push(resolveImportTarget('example.com/mod', FROM_FILE, files, GO_MODULE));
resolved.push(
resolveImportTarget(`github.com/vendor/dep${i}/sub`, FROM_FILE, files, GO_MODULE),
);
}
// One pass: the package-dir index (root files are collected in the same pass).
expect(files.scans).toBe(1);
// Paired result assertion — a count of 1 must not be the count of an
// adapter that resolves nothing.
expect(resolved[0]).toEqual(['internal/models/user.go', 'internal/models/order.go']);
expect(resolved[1]).toEqual(['main.go']);
expect(resolved[2]).toBeNull();
});
it('a distinct file set gets its own index (no stale cross-run reuse)', () => {
expectDistinctFileSetsGetOwnIndex({
resolveImportTarget,
buildWorkspace: () => buildWorkspace(20),
targetRaw: 'example.com/mod/internal/models',
fromFile: FROM_FILE,
resolutionConfig: GO_MODULE,
expected: ['internal/models/user.go', 'internal/models/order.go'],
expectedScans: 1,
});
});
it('still resolves real imports correctly (the perf test is not vacuous)', () => {
const files = buildWorkspace(5);
// Module-relative package: every non-test `.go` file in the directory, in
// file-set order, one ImportEdge target each.
expect(
resolveImportTarget('example.com/mod/internal/models', FROM_FILE, files, GO_MODULE),
).toEqual(['internal/models/user.go', 'internal/models/order.go']);
// Root package: the module path itself, and this leg IS sorted.
expect(resolveImportTarget('example.com/mod', FROM_FILE, files, GO_MODULE)).toEqual([
'main.go',
]);
// No go.mod config: the GOPATH cascade reaches the same package by suffix.
expect(
resolveImportTarget('example.com/mod/internal/models', FROM_FILE, files, undefined),
).toEqual(['internal/models/user.go', 'internal/models/order.go']);
// Stdlib and third-party imports resolve to nothing in the workspace.
expect(resolveImportTarget('fmt', FROM_FILE, files, GO_MODULE)).toBeNull();
expect(resolveImportTarget('github.com/spf13/cobra', FROM_FILE, files, GO_MODULE)).toBeNull();
});
});

101
gitnexus/test/integration/ruby-import-index-reuse.test.ts Normal file
View file

@ -0,0 +1,101 @@
/**
* Production-path regression guard for the Ruby import-resolution index (#2880).
*
* Ruby's bare `require` leg reads the shared `getWorkspaceFileIndex`
* (`import-resolvers/workspace-file-index.ts`), memoized on the `allFilePaths`
* Set identity via a WeakMap. Before #2880 every single `require` materialized
* two arrays AND built a whole `buildSuffixIndex` over the repo, then threw it
* away — the most expensive of the four resolvers hoisted.
*
* Resolution reaches that index through `rubyScopeResolver.resolveImportTarget`
* — the orchestrator adapter — not by calling `resolveRubyImportTarget`
* directly the way the unit parity test does. The adapter must therefore pass
* the Set THROUGH; a defensive copy (`new Set(allFilePaths)`) would hand a
* fresh WeakMap key per call and restore the per-require rebuild. Python hit
* exactly that (PR #1918 review P1), and the parity test cannot see it: it
* never crosses the adapter.
*
* The traversal-count assertions are the perf guard. They are paired with
* result assertions on purpose: a count of 1 is equally true of an adapter that
* has stopped resolving anything at all, so counting alone would stay green
* while every Ruby IMPORTS edge disappeared.
*/
import { describe, it, expect } from 'vitest';
import { rubyScopeResolver } from '../../src/core/ingestion/languages/ruby/scope-resolver.js';
import { CountingSet, expectDistinctFileSetsGetOwnIndex } from '../helpers/counting-file-set.js';
const { resolveImportTarget } = rubyScopeResolver;
const FROM_FILE = 'lib/main.rb';
/**
* A synthetic Ruby app: many service files plus the two requires below —
* `app/models/user` (a multi-segment suffix hit) and `util` (a single-segment
* one). `index.rb` covers the `require_relative` directory form.
*/
function buildWorkspace(fileCount: number): CountingSet {
const files: string[] = [];
for (let i = 0; i < fileCount; i++) {
files.push(`lib/app/services/service${String(i).padStart(5, '0')}.rb`);
}
files.push('lib/app/models/user.rb');
files.push('lib/util.rb');
files.push('lib/support/index.rb');
files.push('lib/main.rb');
return new CountingSet(files);
}
describe('Ruby import resolution — index reuse across requires (#2880)', () => {
it('builds the workspace index once for many requires over a stable file set', () => {
const files = buildWorkspace(300);
const resolved: (string | readonly string[] | null)[] = [];
for (let i = 0; i < 200; i++) {
// A resolvable bare require, and a gem-shaped one that misses. The miss
// is the expensive case: it walks every suffix × every extension before
// returning null, and used to rebuild the suffix index first.
resolved.push(resolveImportTarget('app/models/user', FROM_FILE, files));
resolved.push(resolveImportTarget(`gem${i}/missing`, FROM_FILE, files));
}
expect(files.scans).toBe(1);
// Paired result assertion — a count of 1 must not be the count of an
// adapter that resolves nothing.
expect(resolved[0]).toBe('lib/app/models/user.rb');
expect(resolved[1]).toBeNull();
});
it('a distinct file set gets its own index (no stale cross-run reuse)', () => {
expectDistinctFileSetsGetOwnIndex({
resolveImportTarget,
buildWorkspace: () => buildWorkspace(20),
targetRaw: 'app/models/user',
fromFile: FROM_FILE,
resolutionConfig: undefined,
expected: 'lib/app/models/user.rb',
expectedScans: 1,
});
});
it('still resolves real requires correctly (the perf test is not vacuous)', () => {
const files = buildWorkspace(5);
// Bare `require`: multi-segment and single-segment suffix matches.
expect(resolveImportTarget('app/models/user', FROM_FILE, files)).toBe('lib/app/models/user.rb');
expect(resolveImportTarget('util', FROM_FILE, files)).toBe('lib/util.rb');
// `require_relative`: resolved against the importer's directory, `.rb`
// first and then `<dir>/index.rb`. This leg answers from `Set.has` and
// never touches the index, which is why the counting arm drives bare
// requires instead.
expect(resolveImportTarget('./util', FROM_FILE, files)).toBe('lib/util.rb');
expect(resolveImportTarget('./support', FROM_FILE, files)).toBe('lib/support/index.rb');
expect(resolveImportTarget('../models/user', 'lib/app/services/service00000.rb', files)).toBe(
'lib/app/models/user.rb',
);
// Gems with no matching file in the repo resolve to nothing.
expect(resolveImportTarget('net/http', FROM_FILE, files)).toBeNull();
});
});

819
gitnexus/test/unit/scope-resolution/import-target-index-parity.test.ts Normal file
View file

@ -0,0 +1,819 @@
/**
* Differential harness for the import-target index hoist (#2877 go, #2878
* csharp, #2879 dart, #2880 ruby).
*
* Each of those resolvers answered its lookups with a full `allFilePaths` scan
* per import — Ruby went further and rebuilt a whole `buildSuffixIndex` per
* `require`. Replacing the scans with a per-run index is a pure performance
* change ONLY if every implicit tie-break survives, and those tie-breaks are
* expressed through Set-iteration order and `indexOf` positions rather than
* through anything the type system or the existing tests can see:
*
* - Go sorts the root-package leg and does NOT sort the package-dir leg;
* - Go and C# both take the FIRST occurrence of `/<segment>/` in the path, so
* a directory nested inside a same-named directory does not match;
* - C#'s `resolveDirectMatch` lets a whole-path match win over a suffix match
* found EARLIER in iteration order, while `resolveByProgressiveStripping`
* takes whichever comes first;
* - Dart tries `lib/<rel>` fully before bare `<rel>`, and compares raw paths
* (no backslash normalization) on both legs.
*
* So this file keeps verbatim copies of the pre-change implementations and
* asserts the new ones agree with them on a deterministic corpus built to force
* exactly those cases. The copies are the specification; if a future change
* makes one of these fail, the resolver's OUTPUT moved and the graph's edges
* move with it.
*
* The second half asserts the index is built once per file set rather than once
* per import, by counting how often the Set is iterated. It is the DETERMINISTIC
* guard against a scan reintroduced beside a reused index, which the benchmark
* provably cannot see: a full workspace scan on 1-in-32 imports scores 1.458
* against a 1.8 scaling budget and 1.736 ms against a 4 ms ceiling — it passes
* everything — while this counter reads 14 instead of 1. Timing gates catch the
* constant factor; this catches the scan. Kotlin (#2872) is covered there too,
* because its own guard counts index BUILDS and a scan beside a reused index
* moves no build count.
*
* It is NOT the guard for PR #1918 review finding P1. That failure — a
* defensive `new Set(allFilePaths)` in the orchestrator ADAPTER, handing a fresh
* `WeakMap` key per call — lives one layer above everything in this file, which
* calls the resolver functions directly. Inserting that copy into
* `<lang>/scope-resolver.ts` leaves every arm here green. The adapter is guarded
* by `test/integration/{go,csharp,dart,ruby,kotlin,python}-import-index-reuse.test.ts`.
*/
import { describe, expect, it } from 'vitest';
import type { ParsedImport, WorkspaceIndex } from 'gitnexus-shared';
import { resolveGoImportTarget } from '../../../src/core/ingestion/languages/go/import-target.js';
import { resolveDartImportTarget } from '../../../src/core/ingestion/languages/dart/import-target.js';
import { resolveRubyImportTarget } from '../../../src/core/ingestion/languages/ruby/import-target.js';
import {
resolveCsharpImportTarget,
type CsharpResolveContext,
} from '../../../src/core/ingestion/languages/csharp/import-target.js';
import { resolveKotlinImportTarget } from '../../../src/core/ingestion/languages/kotlin/import-target.js';
import { resolveRubyImportInternal } from '../../../src/core/ingestion/import-resolvers/ruby.js';
import { buildSuffixIndex } from '../../../src/core/ingestion/import-resolvers/utils.js';
import { isHeritageMarker } from '../../../src/core/ingestion/utils/heritage-marker.js';
import { csharpSuffixFallbackAllowed } from '../../../src/core/ingestion/csharp-namespace-gate.js';
import { DART_HERITAGE_PREFIX } from '../../../src/core/ingestion/languages/dart/interpret.js';
import { CountingSet } from '../../helpers/counting-file-set.js';
// ─── verbatim pre-change implementations ─────────────────────────────────────
function legacyFindRootPackageFiles(allFilePaths: ReadonlySet<string>): string[] {
const result: string[] = [];
for (const raw of allFilePaths) {
const normalized = raw.replace(/\\/g, '/');
if (normalized.includes('/')) continue;
if (!normalized.endsWith('.go') || normalized.endsWith('_test.go')) continue;
result.push(raw);
}
return result.sort();
}
function legacyFindAllFilesInPkgDir(allFilePaths: ReadonlySet<string>, pkgPath: string): string[] {
const pkgDir = '/' + pkgPath + '/';
const result: string[] = [];
for (const raw of allFilePaths) {
const normalized = '/' + raw.replace(/\\/g, '/');
if (!normalized.includes(pkgDir)) continue;
if (!normalized.endsWith('.go') || normalized.endsWith('_test.go')) continue;
const afterPkg = normalized.substring(normalized.indexOf(pkgDir) + pkgDir.length);
if (!afterPkg.includes('/')) result.push(raw);
}
return result;
}
function legacyResolveGoImportTarget(
targetRaw: string,
allFilePaths: ReadonlySet<string>,
modulePath: string | undefined,
): string | readonly string[] | null {
if (!targetRaw) return null;
if (
modulePath !== undefined &&
(targetRaw === modulePath || targetRaw.startsWith(`${modulePath}/`))
) {
const relativePkg = targetRaw === modulePath ? '' : targetRaw.slice(modulePath.length + 1);
const files =
relativePkg === ''
? legacyFindRootPackageFiles(allFilePaths)
: legacyFindAllFilesInPkgDir(allFilePaths, relativePkg);
if (files.length > 0) return files;
}
const parts = targetRaw.split('/').filter(Boolean);
for (let i = 0; i < parts.length - 1; i++) {
const files = legacyFindAllFilesInPkgDir(allFilePaths, parts.slice(i).join('/'));
if (files.length > 0) return files;
}
return null;
}
function legacyResolveDartRelative(
rel: string,
fromFile: string,
allFilePaths: ReadonlySet<string>,
): string | null {
const normFrom = fromFile.replace(/\\/g, '/');
const fromDir = normFrom.includes('/') ? normFrom.slice(0, normFrom.lastIndexOf('/')) : '';
const parts = fromDir.length > 0 ? fromDir.split('/') : [];
for (const seg of rel.replace(/\\/g, '/').split('/')) {
if (seg === '' || seg === '.') continue;
if (seg === '..') parts.pop();
else parts.push(seg);
}
const target = parts.join('/');
if (allFilePaths.has(target)) return target;
for (const fp of allFilePaths) {
if (fp === target || fp.endsWith('/' + target)) return fp;
}
return null;
}
function legacyResolveDartImportTarget(
targetRaw: string,
fromFile: string,
allFilePaths: ReadonlySet<string>,
): string | readonly string[] | null {
if (targetRaw.startsWith(DART_HERITAGE_PREFIX)) return null;
if (targetRaw === '') return null;
if (targetRaw.startsWith('dart:')) return null;
if (targetRaw.startsWith('package:')) {
const slash = targetRaw.indexOf('/');
if (slash === -1) return null;
const relPath = targetRaw.slice(slash + 1);
for (const candidate of [`lib/${relPath}`, relPath]) {
for (const fp of allFilePaths) {
if (fp === candidate || fp.endsWith('/' + candidate)) return fp;
}
}
return null;
}
return legacyResolveDartRelative(targetRaw, fromFile, allFilePaths);
}
function legacyResolveRubyBare(
targetRaw: string,
allFilePaths: ReadonlySet<string>,
): string | null {
const normalizedFileList = [...allFilePaths].map((f) => f.replace(/\\/g, '/'));
const allFileList = [...allFilePaths];
const index = buildSuffixIndex(normalizedFileList, allFileList);
return resolveRubyImportInternal(targetRaw, normalizedFileList, allFileList, index);
}
function legacyResolveRubyRelative(
targetRaw: string,
fromDir: string,
allFilePaths: ReadonlySet<string>,
): string | null {
const segments = (fromDir ? fromDir + '/' + targetRaw : targetRaw).split('/');
const resolved: string[] = [];
for (const seg of segments) {
if (seg === '' || seg === '.') continue;
if (seg === '..') resolved.pop();
else resolved.push(seg);
}
const resolvedPath = resolved.join('/');
const rbFile = `${resolvedPath}.rb`;
if (allFilePaths.has(rbFile)) return rbFile;
const indexFile = `${resolvedPath}/index.rb`;
if (allFilePaths.has(indexFile)) return indexFile;
if (resolvedPath.endsWith('.rb') && allFilePaths.has(resolvedPath)) return resolvedPath;
return null;
}
function legacyResolveRubyImportTarget(
targetRaw: string,
fromFile: string,
allFilePaths: ReadonlySet<string>,
): string | readonly string[] | null {
if (!targetRaw) return null;
if (isHeritageMarker(targetRaw)) return null;
const fromNormalized = fromFile.replace(/\\/g, '/');
const fromDir = fromNormalized.includes('/')
? fromNormalized.slice(0, fromNormalized.lastIndexOf('/'))
: '';
if (targetRaw.startsWith('./') || targetRaw.startsWith('../')) {
return legacyResolveRubyRelative(targetRaw, fromDir, allFilePaths);
}
return legacyResolveRubyBare(targetRaw, allFilePaths);
}
function legacyFindDirectChild(
allFilePaths: ReadonlySet<string>,
dirSegment: string,
): string | null {
const dirPrefix = `${dirSegment}/`;
const nestedDirPrefix = `/${dirPrefix}`;
for (const raw of allFilePaths) {
const f = raw.replace(/\\/g, '/');
if (!f.endsWith('.cs')) continue;
const atRoot = f.startsWith(dirPrefix);
const atNested = f.includes(nestedDirPrefix);
if (!atRoot && !atNested) continue;
const idx = atRoot ? 0 : f.indexOf(nestedDirPrefix) + 1;
const after = f.slice(idx + dirPrefix.length);
if (after.length > 0 && !after.includes('/')) return raw;
}
return null;
}
function legacyResolveDirectMatch(
allFilePaths: ReadonlySet<string>,
pathLike: string,
): string | null {
const exactName = `${pathLike}.cs`;
const nestedSuffix = `/${exactName}`;
let suffixFile: string | null = null;
for (const raw of allFilePaths) {
const f = raw.replace(/\\/g, '/');
if (!f.endsWith('.cs')) continue;
if (f === exactName) return raw;
if (suffixFile === null && f.endsWith(nestedSuffix)) suffixFile = raw;
}
if (suffixFile !== null) return suffixFile;
return legacyFindDirectChild(allFilePaths, pathLike);
}
function legacyResolveByProgressiveStripping(
allFilePaths: ReadonlySet<string>,
pathLike: string,
): string | null {
const segments = pathLike.split('/').filter(Boolean);
for (let skip = 1; skip < segments.length; skip++) {
const tail = segments.slice(skip).join('/');
if (tail === '') continue;
const tailFile = `${tail}.cs`;
const tailSuffix = `/${tailFile}`;
let tailFileMatch: string | null = null;
for (const raw of allFilePaths) {
const f = raw.replace(/\\/g, '/');
if (!f.endsWith('.cs')) continue;
if (f === tailFile || f.endsWith(tailSuffix)) {
tailFileMatch = raw;
break;
}
}
if (tailFileMatch !== null) return tailFileMatch;
const child = legacyFindDirectChild(allFilePaths, tail);
if (child !== null) return child;
}
return null;
}
/** The no-csproj leg, which is the half #2878 moved onto the index. */
function legacyResolveCsharpNoCsproj(
targetRaw: string,
allFilePaths: ReadonlySet<string>,
): string | null {
if (targetRaw === '') return null;
const pathLike = targetRaw.replace(/\./g, '/');
if (!csharpSuffixFallbackAllowed(targetRaw, undefined)) return null;
const direct = legacyResolveDirectMatch(allFilePaths, pathLike);
if (direct !== null) return direct;
return legacyResolveByProgressiveStripping(allFilePaths, pathLike);
}
// ─── current implementations, called the way the orchestrator calls them ─────
function csharp(targetRaw: string, allFilePaths: ReadonlySet<string>): string | null {
const ws: CsharpResolveContext = { fromFile: 'App/Program.cs', allFilePaths };
const parsedImport: ParsedImport = {
kind: 'namespace',
localName: '_',
importedName: '_',
targetRaw,
};
return resolveCsharpImportTarget(parsedImport, ws as unknown as WorkspaceIndex);
}
// ─── deterministic corpus ────────────────────────────────────────────────────
/** Murmur3 finalizer — a reproducible stand-in for `Math.random()`. */
function mix(n: number): number {
let x = n >>> 0;
x = Math.imul(x ^ (x >>> 16), 0x85ebca6b) >>> 0;
x = Math.imul(x ^ (x >>> 13), 0xc2b2ae35) >>> 0;
return (x ^ (x >>> 16)) >>> 0;
}
/**
* Directory shapes, chosen so the corpus contains every case where the naive
* "does the dir end with the segment" rewrite diverges from the original
* first-`indexOf` predicate: a directory name nested inside itself
* (`pkg/pkg`, `a/pkg/b/pkg`), the same leaf under several parents (collision
* tie-breaks), an absolute-rooted layout, and the repo root.
*/
const DIRS = [
'',
'pkg',
'a/pkg',
'b/pkg',
'pkg/pkg',
'a/pkg/b/pkg',
'internal/models',
'x/internal/models',
'lib',
'lib/src',
'vendor/lib',
'/repo/pkg',
'/repo/internal/models',
'Models',
'App/Models',
'Models/Models',
];
const STEMS = ['main', 'models', 'util', 'client', 'index', 'pkg', 'lib', 'server'];
function corpus(seed: number, extension: string, fileCount: number): Set<string> {
const files = new Set<string>();
for (let i = 0; i < fileCount; i++) {
const a = mix(seed * 7919 + i);
const b = mix(a ^ 0x9e3779b9);
const dir = DIRS[a % DIRS.length];
const stem = STEMS[b % STEMS.length];
// A slice of Go files are `_test.go`, which the package leg must exclude.
const suffix = extension === '.go' && b % 5 === 0 ? '_test.go' : extension;
const rel = `${stem}${suffix}`;
files.add(dir === '' ? rel : `${dir}/${rel}`);
}
// Windows-shaped paths: the resolvers differ on whether they normalize, and
// that difference is part of what must not move.
files.add(`win\\dir\\thing${extension}`);
return files;
}
const GO_TARGETS = [
'',
'fmt',
'os',
'pkg',
'a/pkg',
'pkg/pkg',
'b/pkg',
'internal/models',
'x/internal/models',
'example.com/mod',
'example.com/mod/pkg',
'example.com/mod/internal/models',
'example.com/mod/nope',
'github.com/org/repo/pkg',
'github.com/org/repo/internal/models',
'golang.org/x/sync/errgroup',
'lib/src',
'win/dir',
];
const DART_TARGETS = [
'',
'dart:core',
'package:app/models.dart',
'package:app/src/models.dart',
'package:app',
'package:other/lib/util.dart',
'models.dart',
'./models.dart',
'../models.dart',
'../../lib/util.dart',
'src/models.dart',
'win\\dir\\thing.dart',
`${DART_HERITAGE_PREFIX}Foo`,
];
const RUBY_TARGETS = [
'',
'json',
'models',
'util',
'pkg/models',
'internal/models',
'./models',
'../util',
'./index',
'lib/src/client',
'Models',
// Addresses the corpus's `win\dir\thing.rb`. Without a target that reaches
// it, deleting the `raw.replace(/\\/g, '/')` normalization in
// `workspace-file-index.ts` moves no assertion here: the backslash file is in
// every corpus but nothing asked for it. Spelled with forward slashes because
// that is how a `require` is written; the normalization is what bridges the
// two spellings.
'win/dir/thing',
];
const CSHARP_TARGETS = [
'',
'System',
'System.Threading.Tasks',
'Models',
'App.Models',
'Models.Models',
'App.Models.Client',
'Internal.Models',
'X.Internal.Models',
'Lib.Src',
'Pkg.Pkg',
// The C# twin of the Ruby entry above: addresses the corpus's
// `win\dir\thing.cs` so the deletion of `workspace-file-index.ts`'s
// normalization has something to break. Lowercase because the corpus spells
// the file that way and the C# direct/suffix lookups are case-SENSITIVE
// (`getInsensitive` is only reached from the csproj leg).
'win.dir.thing',
];
const DART_FROM_FILES = ['lib/main.dart', 'lib/src/main.dart', '/repo/pkg/main.dart', 'main.dart'];
const RUBY_FROM_FILES = ['lib/main.rb', 'lib/src/main.rb', 'a/pkg/main.rb', 'main.rb'];
// ─── parity ──────────────────────────────────────────────────────────────────
describe('import-target index hoist — output parity with the pre-change scans', () => {
it('go: resolveGoImportTarget (#2877)', () => {
let checked = 0;
for (let repo = 0; repo < 40; repo++) {
const files = corpus(repo, '.go', 6 + (repo % 25));
for (const modulePath of [undefined, 'example.com/mod']) {
const config = modulePath === undefined ? undefined : { modulePath };
for (const target of GO_TARGETS) {
const actual = resolveGoImportTarget(target, 'main.go', files, config);
const expected = legacyResolveGoImportTarget(target, files, modulePath);
// Array identity AND order: Go returns the whole list as the edge's
// targets, so a reordering is an observable change.
expect(actual, `go ${target} module=${modulePath} repo=${repo}`).toEqual(expected);
checked++;
}
}
}
expect(checked).toBe(40 * 2 * GO_TARGETS.length);
});
it('dart: resolveDartImportTarget (#2879)', () => {
let checked = 0;
for (let repo = 0; repo < 40; repo++) {
const files = corpus(repo, '.dart', 6 + (repo % 25));
for (const fromFile of DART_FROM_FILES) {
for (const target of DART_TARGETS) {
const actual = resolveDartImportTarget(target, fromFile, files);
const expected = legacyResolveDartImportTarget(target, fromFile, files);
expect(actual, `dart ${target} from=${fromFile} repo=${repo}`).toEqual(expected);
checked++;
}
}
}
expect(checked).toBe(40 * DART_FROM_FILES.length * DART_TARGETS.length);
});
it('ruby: resolveRubyImportTarget (#2880)', () => {
let checked = 0;
for (let repo = 0; repo < 40; repo++) {
const files = corpus(repo, '.rb', 6 + (repo % 25));
for (const fromFile of RUBY_FROM_FILES) {
for (const target of RUBY_TARGETS) {
const actual = resolveRubyImportTarget(target, fromFile, files);
const expected = legacyResolveRubyImportTarget(target, fromFile, files);
expect(actual, `ruby ${target} from=${fromFile} repo=${repo}`).toEqual(expected);
checked++;
}
}
}
expect(checked).toBe(40 * RUBY_FROM_FILES.length * RUBY_TARGETS.length);
});
it('csharp: the no-csproj path (#2878)', () => {
let checked = 0;
for (let repo = 0; repo < 40; repo++) {
const files = corpus(repo, '.cs', 6 + (repo % 25));
for (const target of CSHARP_TARGETS) {
const actual = csharp(target, files);
const expected = legacyResolveCsharpNoCsproj(target, files);
expect(actual, `csharp ${target} repo=${repo}`).toEqual(expected);
checked++;
}
}
expect(checked).toBe(40 * CSHARP_TARGETS.length);
});
/**
* Hand-built layouts for the tie-breaks the generated corpus cannot reach —
* every one of these was verified to FAIL against a plausible "simplified"
* rewrite of the resolver it covers. `new Set([...])` preserves insertion
* order, which IS the tie-break for most of them.
*/
const HANDBUILT: {
lang: 'go' | 'dart' | 'ruby' | 'csharp';
why: string;
files: string[];
target: string;
fromFile?: string;
modulePath?: string;
}[] = [
{
lang: 'csharp',
why: 'whole-path match wins over a suffix match found EARLIER in Set order',
files: ['a/Models.cs', 'Models.cs'],
target: 'Models',
},
{
lang: 'csharp',
why: 'no whole-path file: first suffix match in Set order wins',
files: ['b/Models.cs', 'a/Models.cs'],
target: 'Models',
},
{
lang: 'csharp',
why: 'direct child of a NESTED namespace dir beats the root-level one when it comes first',
files: ['App/Models/First.cs', 'Models/Second.cs'],
target: 'Models',
},
{
lang: 'csharp',
why: 'a namespace dir nested inside itself does not answer the query',
files: ['Models/Models/User.cs'],
target: 'Models',
},
{
lang: 'csharp',
why: 'progressive prefix stripping reaches the tail namespace dir',
files: ['Models/User.cs'],
target: 'CrossFile.Models',
},
{
// `normToRaw` keeps the FIRST raw path per normalized key
// (`workspace-file-index.ts`), mirroring the `for (const raw of
// allFilePaths)` scan it replaced. No GENERATED corpus file set contains
// a normalization twin, so flipping that to last-wins moved nothing —
// the rule lived in a comment. Two spellings of one path, and only the
// first-wins reading returns the backslash one.
lang: 'csharp',
why: 'normToRaw keeps the FIRST raw path that normalizes to a key, not the last',
files: ['App\\Models.cs', 'App/Models.cs'],
target: 'App.Models',
},
{
lang: 'go',
why: 'root package leg is SORTED',
files: ['b.go', 'a.go', 'c_test.go'],
target: 'example.com/mod',
modulePath: 'example.com/mod',
},
{
lang: 'go',
why: 'package-dir leg is NOT sorted — it keeps Set order',
files: ['x/pkg/b.go', 'x/pkg/a.go'],
target: 'example.com/mod/x/pkg',
modulePath: 'example.com/mod',
},
{
lang: 'go',
why: 'two directories share the suffix: results interleave back into Set order',
files: ['a/pkg/one.go', 'b/a/pkg/two.go', 'a/pkg/three.go'],
target: 'a/pkg',
},
{
lang: 'go',
why: 'a package dir nested inside itself does not answer the query',
files: ['a/pkg/b/pkg/x.go'],
// Addressed through the MODULE leg as the single segment `pkg`, not as
// `a/pkg`. `a/pkg` never reached the first-occurrence branch this case is
// named for: `'/a/pkg/b/pkg/'.endsWith('/a/pkg/')` is already false, so
// the naive `endsWith` rewrite agreed with the real predicate and the
// case passed either way. With `pkg`, `endsWith('/pkg/')` is TRUE and only
// the "…and that occurrence is the FIRST" half rejects it. The module leg
// is required because the GOPATH cascade skips single-segment targets.
target: 'example.com/mod/pkg',
modulePath: 'example.com/mod',
},
{
lang: 'go',
why: '_test.go files are a different package and never match',
files: ['x/pkg/a_test.go'],
target: 'x/pkg',
},
{
lang: 'dart',
why: '`lib/<rel>` beats bare `<rel>` even when the bare hit comes FIRST in Set order',
files: ['a/models.dart', 'z/lib/models.dart'],
target: 'package:app/models.dart',
},
{
lang: 'dart',
why: 'bare `<rel>` is reached only after `lib/<rel>` misses entirely',
files: ['a/models.dart'],
target: 'package:app/models.dart',
},
{
lang: 'dart',
why: 'paths are matched RAW — a backslash path is not normalized into a hit',
files: ['win\\dir\\thing.dart'],
target: 'package:app/dir/thing.dart',
},
{
// The negative case above pins the guard NEXT DOOR to the one it names:
// the basename bucket lookup misses before the raw comparison is ever
// consulted, so normalizing only the bucket key, or only the comparison,
// still yields null and still matches. This positive twin puts the
// backslashes in the TARGET so a hit depends on both halves staying raw.
lang: 'dart',
why: 'a backslash TARGET matches only because neither the bucket key nor the comparison normalizes',
files: ['dir\\thing.dart'],
target: 'package:app/dir\\thing.dart',
},
{
lang: 'dart',
why: 'relative import prefers the exact path over an earlier suffix hit',
files: ['z/lib/src/models.dart', 'lib/src/models.dart'],
target: './models.dart',
fromFile: 'lib/src/main.dart',
},
{
lang: 'ruby',
why: 'bare require suffix match keeps its first-in-order winner',
files: ['a/json.rb', 'json.rb'],
target: 'json',
},
{
lang: 'ruby',
why: 'require_relative resolves against the importer dir before any suffix match',
files: ['z/lib/util.rb', 'lib/util.rb'],
target: './util',
fromFile: 'lib/main.rb',
},
];
it.each(HANDBUILT)('$lang hand-built: $why', ({ lang, files, target, fromFile, modulePath }) => {
const set = new Set(files);
if (lang === 'go') {
const config = modulePath === undefined ? undefined : { modulePath };
expect(resolveGoImportTarget(target, 'main.go', set, config)).toEqual(
legacyResolveGoImportTarget(target, set, modulePath),
);
return;
}
if (lang === 'dart') {
const from = fromFile ?? 'lib/main.dart';
expect(resolveDartImportTarget(target, from, set)).toEqual(
legacyResolveDartImportTarget(target, from, set),
);
return;
}
if (lang === 'ruby') {
const from = fromFile ?? 'lib/main.rb';
expect(resolveRubyImportTarget(target, from, set)).toEqual(
legacyResolveRubyImportTarget(target, from, set),
);
return;
}
expect(csharp(target, set)).toEqual(legacyResolveCsharpNoCsproj(target, set));
});
it('every hand-built layout resolves to something (they pin a winner, not a null)', () => {
// `toEqual(null) === toEqual(null)` would make the arm above pass for the
// wrong reason. Only the three "must NOT match" layouts may be null.
const mustBeNull = new Set([
'a namespace dir nested inside itself does not answer the query',
'a package dir nested inside itself does not answer the query',
'_test.go files are a different package and never match',
'paths are matched RAW — a backslash path is not normalized into a hit',
]);
for (const c of HANDBUILT) {
const set = new Set(c.files);
const got =
c.lang === 'go'
? legacyResolveGoImportTarget(c.target, set, c.modulePath)
: c.lang === 'dart'
? legacyResolveDartImportTarget(c.target, c.fromFile ?? 'lib/main.dart', set)
: c.lang === 'ruby'
? legacyResolveRubyImportTarget(c.target, c.fromFile ?? 'lib/main.rb', set)
: legacyResolveCsharpNoCsproj(c.target, set);
if (mustBeNull.has(c.why)) expect(got, c.why).toBeNull();
else expect(got, c.why).not.toBeNull();
}
});
it('the corpus actually exercises the resolvers (the parity arms are not vacuous)', () => {
// A corpus that resolved nothing would make every arm above pass on
// `null === null`. Pin a floor on real hits per language.
const hits = { go: 0, dart: 0, ruby: 0, csharp: 0 };
for (let repo = 0; repo < 40; repo++) {
const go = corpus(repo, '.go', 6 + (repo % 25));
for (const t of GO_TARGETS) {
if (resolveGoImportTarget(t, 'main.go', go, { modulePath: 'example.com/mod' }) !== null) {
hits.go++;
}
}
const dart = corpus(repo, '.dart', 6 + (repo % 25));
for (const t of DART_TARGETS) {
if (resolveDartImportTarget(t, 'lib/src/main.dart', dart) !== null) hits.dart++;
}
const ruby = corpus(repo, '.rb', 6 + (repo % 25));
for (const t of RUBY_TARGETS) {
if (resolveRubyImportTarget(t, 'lib/src/main.rb', ruby) !== null) hits.ruby++;
}
const cs = corpus(repo, '.cs', 6 + (repo % 25));
for (const t of CSHARP_TARGETS) {
if (csharp(t, cs) !== null) hits.csharp++;
}
}
// Measured on this corpus: go 364, dart 75, ruby 259, csharp 196. Ruby and
// C# gained 40 each from the `win\dir\thing.<ext>` targets — one per repo,
// which is also the floor those two arms now defend.
expect(hits.go).toBeGreaterThan(300);
expect(hits.dart).toBeGreaterThan(60);
expect(hits.ruby).toBeGreaterThan(220);
expect(hits.csharp).toBeGreaterThan(160);
});
});
// ─── index reuse ─────────────────────────────────────────────────────────────
/**
* `CountingSet` (`test/helpers/counting-file-set.ts`) counts full traversals of
* the file set by every entry point — `for…of`, spread, `forEach`, `values`,
* `keys`, `entries`. Each index build in these resolvers walks the set exactly
* once, so on a stable set the scan count is the number of index builds PLUS
* any scan reintroduced beside a reused index, which is the mutation the
* benchmark cannot see.
*
* The number is not a complete scan census, and the docstring here used to
* claim it was. It watches the SET; the resolvers hold materialized arrays of
* the same file list (`WorkspaceFileIndex.normalized` / `.all`, Dart's basename
* buckets, `PackageDirIndex.filesByDir`), and a scan over one of those arrays
* moves nothing here. Closing that would mean instrumenting production or
* proxying an index internal for a test; neither is in place, so treat these
* arms as covering set-level scans only.
*
* These arms drive the resolver FUNCTIONS directly, which is one layer below
* the `new Set(allFilePaths)` hazard they are sometimes cited for: production
* reaches the resolvers through `<lang>ScopeResolver.resolveImportTarget`, and
* a defensive copy inserted in that adapter leaves every arm below green. The
* adapter-level guards are
* `test/integration/{go,csharp,dart,ruby,kotlin,python}-import-index-reuse.test.ts`.
*/
function countingCorpus(seed: number, extension: string): CountingSet {
return new CountingSet(corpus(seed, extension, 200));
}
describe('import-target index hoist — built once per file set, not once per import', () => {
it('go builds one index for many imports (#2877)', () => {
const files = countingCorpus(1, '.go');
for (let i = 0; i < 200; i++) {
resolveGoImportTarget(`github.com/org/repo${i}/pkg`, 'main.go', files, {
modulePath: 'example.com/mod',
});
}
// One pass: the package-dir index (root files are collected in the same pass).
expect(files.scans).toBe(1);
});
it('dart builds one index for many imports (#2879)', () => {
const files = countingCorpus(2, '.dart');
for (let i = 0; i < 200; i++) {
resolveDartImportTarget(`package:pkg${i}/ghost${i}.dart`, 'lib/main.dart', files);
}
expect(files.scans).toBe(1);
});
it('ruby builds one index for many requires (#2880)', () => {
const files = countingCorpus(3, '.rb');
for (let i = 0; i < 200; i++) {
resolveRubyImportTarget(`ghost${i}/missing`, 'lib/main.rb', files);
}
expect(files.scans).toBe(1);
});
it('csharp builds one index per structure for many usings (#2878)', () => {
const files = countingCorpus(4, '.cs');
for (let i = 0; i < 200; i++) {
csharp(`Ghost${i}.Missing.Deep`, files);
}
// Two passes: the shared workspace/suffix index and the namespace-dir index.
expect(files.scans).toBe(2);
});
it('kotlin builds one index for many imports (#2872)', () => {
// Kotlin's own guard (`test/integration/kotlin-import-index-reuse.test.ts`)
// counts index BUILDS. That catches the per-import rebuild, but a scan added
// beside a reused index moves no build count — this arm sees it, because it
// counts iterations of the Set rather than cache misses.
const files = countingCorpus(7, '.kt');
for (let i = 0; i < 200; i++) {
resolveKotlinImportTarget(
{ kind: 'named', localName: 'X', importedName: 'X', targetRaw: `ghost${i}.deep.Missing` },
{ fromFile: 'App.kt', allFilePaths: files } as unknown as WorkspaceIndex,
);
}
expect(files.scans).toBe(1);
});
it('a distinct file set gets its own index (no stale cross-run reuse)', () => {
const a = countingCorpus(5, '.go');
const b = countingCorpus(6, '.go');
for (let i = 0; i < 20; i++) {
resolveGoImportTarget('github.com/org/repo/pkg', 'main.go', a, undefined);
resolveGoImportTarget('github.com/org/repo/pkg', 'main.go', b, undefined);
}
expect(a.scans).toBe(1);
expect(b.scans).toBe(1);
});
});