mirror of
https://github.com/abhigyanpatwari/GitNexus.git
synced 2026-09-09 22:33:39 +00:00
c6b24162d9
Some checks are pending
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Scorecard / Scorecard analysis (push) Waiting to run
Skill copy sync / shipped skills drift guard (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
* perf(kotlin): index import resolution instead of scanning per import
`resolveKotlinImportTarget` walked the entire workspace on every import.
Its four tiers — exact/suffix, directory child, package fan-out and
progressive prefix strip — each ran `for (const raw of allFilePaths)` with a
`replace(/\\/g, '/')` and several string scans per entry, and they are tried
in cascade, so one unresolved import cost two to four full passes.
Across a repository with tens of thousands of Kotlin files that is
O(imports x files): on the order of 10^10 string operations on a single
thread. It does not look like a hot loop from the outside - analyze sits at
exactly 1.00 core with a completely flat heap and emits nothing for hours,
because every allocation is a short-lived string and nothing accumulates to
hint at progress. Small repositories hide it entirely: at a few hundred files
each pass is free.
Three maps, built once per `allFilePaths` Set and memoized on its identity,
make each tier O(1): stem -> path for the exact tier, every component-suffix
of the stem for the suffix tier, and directory -> direct children for both the
fan-out and the first-child fallback. Cost becomes O(files) once plus O(1) per
import. This mirrors the existing Python index (`getPythonFileIndex`), down to
the WeakMap keying and the build counter.
Semantics are unchanged, including the parts the scans expressed only through
iteration order:
- an exact match anywhere beats a suffix match found earlier, because the
scan returned on the first exact hit but merely remembered the first
suffix hit;
- "first match" stays first in set-iteration order, so both stem maps keep
the earliest path inserted for a key;
- a directory-name match still honours the scan's `startsWith`-then-`indexOf`
rule, which only ever considered the FIRST occurrence of `/dir/`. A path
like `data/src/main/kotlin/com/example/data/Repo.kt` is therefore still
NOT a child of `data`. That is arguably wrong, but fixing it here would
silently move edges in every Kotlin repository; it belongs in its own
change with its own fixtures.
That claim is gated, not asserted. `bench/kotlin-import-target` fingerprints
every `fromFile | targetRaw -> result` triple over an exhaustive branch matrix
plus a deterministic fuzz, each file set resolved in BOTH iteration orders
because that is the only place the tie-breaks above are expressed. The
committed baseline is the value the PRE-INDEX implementation produces: both
implementations print
5ad605c179081505705ff7698a09dbdbdc4831080af6d9fdec5499cc6bce28ee over the same
20074 cases, 11612 of them non-null, and anyone can re-run it by pointing the
harness's module specifier at the old file.
Its second arm is the scaling ratio, `(t_large/t_small)/(1600/400)` over a
synthetic Kotlin monorepo whose imports are ~40% unresolvable — only a miss
drives all four tiers, which is where the scan was worst. The index measures
0.99 (8.0 ms / 31.7 ms); the implementation it replaces measures 3.737
(2207.8 ms / 33003.5 ms) on that same corpus, so the budget of 1.6 separates
them by a wide margin. Take the absolute times as an order of magnitude only
(~276x, ~1041x): the floor arm was run once cold because best-of-seven against
a quadratic implementation costs minutes, while the index arm is the usual
best-of-seven. The ratios are the comparable pair. Both arms run in the
existing always-on `benchmarks (GITNEXUS_BENCH)` job, next to the C++ guard
from #2788 and the Python one from #1918.
Two unit-level guards sit alongside it: a parity test pinning the curated
cases, and an integration test asserting the index is built once across many
imports — the adapter must pass the Set through, since a defensive copy would
hand a fresh WeakMap key per call and restore the old behaviour (the same trap
Python hit in PR #1918).
Two other providers have the same defect and are left alone here, having no
repository at hand to verify a change against:
- `go/import-target.ts`: `findRootPackageFiles` and `findAllFilesInPkgDir`
scan unmemoized, and the GOPATH fallback calls the latter once per path
segment but the last, so a single import can trigger several full passes;
- `dart/import-target.ts`: the `package:` branch scans once per candidate
path — `lib/<rel>` and bare `<rel>` — and `resolveRelative` scans again in
its suffix fallback, also unmemoized.
`csharp/import-target.ts` is a partial case worth noting: it already builds a
memoized `getWorkspaceFileIndex`, but that is reached only when a `.csproj` is
found; the no-csproj path hands the raw Set to `resolveDirectMatch` and
`resolveByProgressiveStripping`, which scan past it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* test(kotlin): close the blind axes in the import-resolution gate
Review of #2872 found the weak part was the gate, not the resolver: four
plausible follow-up mutations passed `--check` with a byte-identical
fingerprint, `cases` AND `non_null`. Each is now caught, and each was
re-checked against the mutation it exists to stop.
- The hashed record carried `order | fromFile | targetRaw | result` but not
the FILE SET, so a corpus edit that swapped the workspace under a case
while leaving its result string alone was invisible. Leaving the resolver
untouched and editing only the corpus, two documented load-bearing cases
could be gutted — the "exact beats an earlier suffix" case losing its
competing file, the repeated-directory negative case losing its file
entirely — with the gate green. The file set is now part of the record, and
that same edit now moves the fingerprint.
- The corpus capped path depth at 8 components and packages at 16 files,
which are precisely the two axes the loops this change added run on. It now
carries 11- and 13-component paths, queries against suffix keys deeper than
seven segments, a 40-file package, and a fuzz that spans both. Verified:
capping suffix-key depth at 7, skipping the `dirChildren` suffix loop above
depth 8, and capping a bucket at 17 entries each now move the fingerprint,
where all three previously passed.
- `non_null` was reported but never asserted; it is asserted beside `cases`.
That closes only the "resolves nothing at all" hole — it stayed 11612 under
all three code mutations above and under the corpus edit — so it is a
companion to the two fixes above, not a substitute for either.
- A ratio cannot see a constant factor, and a file-count ratio cannot see a
depth cost. `--check` now also asserts a DEPTH ratio (file count fixed,
paths 24 components against 8) and an absolute ceiling on the small arm: a
full workspace scan reintroduced on 1-in-32 imports scores 1.490, inside
the scaling budget, while running 2.8x slower.
The baseline is re-derived, not adjusted: the pre-index implementation and the
index both print
ebf1790bf1d42dad483a51f2cbdeb2351e493b9e8236e4eedeef592dd81e2c5c over the new
20106-case corpus, 13256 of them non-null.
Both test suites were shown to be non-load-bearing and now are:
- the parity test's repeated-directory case put `data` at the LEADING
segment, so the `startsWith` guard fired and the `indexOf` rule its own
comment describes was never reached — a resolver with that check relaxed to
`>= 0` passed all 18 cases. A mid-path case now pins it, and a backslash
fan-out case pins `norm.lastIndexOf` against `raw.lastIndexOf`, which was
also bench-only. Both mutations now fail the unit suite.
- the index-reuse test discarded all 200 return values, so a build count of 1
was equally true of an adapter that had stopped resolving anything. It now
asserts results, and its docstring premise is corrected: every one of its
imports hit the tier-1 suffix lookup and none reached the fan-out it
claimed to exercise. Half now genuinely do. The `undefined as never` casts
and the `?.` are gone — both trailing parameters are optional and the
member is required.
Resolver changes, all output-identical against the differential above:
- `dirChildren` buckets are frozen once built. `findKotlinPackageFiles` hands
a bucket straight out of the index, and the `readonly string[]` return type
does not survive the caller: the finalize pass normalizes with
`Array.isArray(t) ? t : [t]`, and `isArray`'s `arg is any[]` predicate
widens the true branch, so `tsc --strict` accepts a `.sort()` there. A
downstream sort would permanently reorder the cached bucket and flip the
first-child tier for every later import in the run.
- `stripped` is computed only after tier 1 misses, with `lastIndexOf`/`slice`
instead of `split`/`slice`/`join`. Measured -20% small arm, -21% large arm.
- `KOTLIN_EXTENSIONS` now comes from the existing `import-resolvers/jvm.ts`
export instead of a fourth inlined copy.
- A note on why the shared `buildSuffixIndex` is not reused, with the four
probes that diverge, and the measured basename-bucket comparison — the one
place this was less documented than the Python precedent it follows, and
the question the Go/Dart/C# follow-ups will each face.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Gergő Magyar <gergomagyar@icloud.com>
|
||
|---|---|---|
| .. | ||
| c | ||
| cpp | ||
| csharp | ||
| go | ||
| java | ||
| javascript | ||
| kotlin | ||
| php | ||
| python | ||
| ruby | ||
| rust | ||
| swift | ||
| typescript | ||
| callable-flow-captures.test.ts | ||
| callable-value-flow-env.test.ts | ||
| callable-value-flow-worklist.test.ts | ||
| callable-value-target-index.test.ts | ||
| construction-syntax-wiring.test.ts | ||
| def-index.test.ts | ||
| definition-id.test.ts | ||
| emit-references.test.ts | ||
| finalize-algorithm.test.ts | ||
| finalize-orchestrator.test.ts | ||
| graph-bridge-label-split.test.ts | ||
| import-target-adapter.test.ts | ||
| imported-return-types.test.ts | ||
| method-dispatch-index.test.ts | ||
| module-scope-index.test.ts | ||
| namespace-channel-lookup.test.ts | ||
| namespace-targets-import-path.test.ts | ||
| node-lookup-determinism.test.ts | ||
| overload-narrowing.test.ts | ||
| parse-worker-scope-integration.test.ts | ||
| pick-implicit-this-overload.test.ts | ||
| pick-unique-global-callable.test.ts | ||
| pick-unique-global-class.test.ts | ||
| position-index.test.ts | ||
| property-dispatch-fanout-env.test.ts | ||
| property-name-index.test.ts | ||
| qualified-name-index.test.ts | ||
| receiver-chain-fold.test.ts | ||
| receiver-chain-wiring.test.ts | ||
| reconcile-ownership.test.ts | ||
| registries.test.ts | ||
| resolve-ambiguous-inheritance-base.test.ts | ||
| resolve-references.test.ts | ||
| resolve-type-ref.test.ts | ||
| run-progress.test.ts | ||
| scope-extractor.test.ts | ||
| scope-id.test.ts | ||
| scope-source-content-policy.test.ts | ||
| scope-tree.test.ts | ||
| strip-cast-wrappers.test.ts | ||
| type-parameters.test.ts | ||
| unresolved-receiver-files.test.ts | ||
| unresolved-receivers.test.ts | ||
| validate-bindings-immutability.test.ts | ||
| walkers-augmentations.test.ts | ||
| workspace-index.test.ts | ||