mirror of
https://github.com/abhigyanpatwari/GitNexus.git
synced 2026-08-28 05:25:25 +00:00
c6b24162d9
Some checks are pending
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Scorecard / Scorecard analysis (push) Waiting to run
Skill copy sync / shipped skills drift guard (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
* perf(kotlin): index import resolution instead of scanning per import
`resolveKotlinImportTarget` walked the entire workspace on every import.
Its four tiers — exact/suffix, directory child, package fan-out and
progressive prefix strip — each ran `for (const raw of allFilePaths)` with a
`replace(/\\/g, '/')` and several string scans per entry, and they are tried
in cascade, so one unresolved import cost two to four full passes.
Across a repository with tens of thousands of Kotlin files that is
O(imports x files): on the order of 10^10 string operations on a single
thread. It does not look like a hot loop from the outside - analyze sits at
exactly 1.00 core with a completely flat heap and emits nothing for hours,
because every allocation is a short-lived string and nothing accumulates to
hint at progress. Small repositories hide it entirely: at a few hundred files
each pass is free.
Three maps, built once per `allFilePaths` Set and memoized on its identity,
make each tier O(1): stem -> path for the exact tier, every component-suffix
of the stem for the suffix tier, and directory -> direct children for both the
fan-out and the first-child fallback. Cost becomes O(files) once plus O(1) per
import. This mirrors the existing Python index (`getPythonFileIndex`), down to
the WeakMap keying and the build counter.
Semantics are unchanged, including the parts the scans expressed only through
iteration order:
- an exact match anywhere beats a suffix match found earlier, because the
scan returned on the first exact hit but merely remembered the first
suffix hit;
- "first match" stays first in set-iteration order, so both stem maps keep
the earliest path inserted for a key;
- a directory-name match still honours the scan's `startsWith`-then-`indexOf`
rule, which only ever considered the FIRST occurrence of `/dir/`. A path
like `data/src/main/kotlin/com/example/data/Repo.kt` is therefore still
NOT a child of `data`. That is arguably wrong, but fixing it here would
silently move edges in every Kotlin repository; it belongs in its own
change with its own fixtures.
That claim is gated, not asserted. `bench/kotlin-import-target` fingerprints
every `fromFile | targetRaw -> result` triple over an exhaustive branch matrix
plus a deterministic fuzz, each file set resolved in BOTH iteration orders
because that is the only place the tie-breaks above are expressed. The
committed baseline is the value the PRE-INDEX implementation produces: both
implementations print
5ad605c179081505705ff7698a09dbdbdc4831080af6d9fdec5499cc6bce28ee over the same
20074 cases, 11612 of them non-null, and anyone can re-run it by pointing the
harness's module specifier at the old file.
Its second arm is the scaling ratio, `(t_large/t_small)/(1600/400)` over a
synthetic Kotlin monorepo whose imports are ~40% unresolvable — only a miss
drives all four tiers, which is where the scan was worst. The index measures
0.99 (8.0 ms / 31.7 ms); the implementation it replaces measures 3.737
(2207.8 ms / 33003.5 ms) on that same corpus, so the budget of 1.6 separates
them by a wide margin. Take the absolute times as an order of magnitude only
(~276x, ~1041x): the floor arm was run once cold because best-of-seven against
a quadratic implementation costs minutes, while the index arm is the usual
best-of-seven. The ratios are the comparable pair. Both arms run in the
existing always-on `benchmarks (GITNEXUS_BENCH)` job, next to the C++ guard
from #2788 and the Python one from #1918.
Two unit-level guards sit alongside it: a parity test pinning the curated
cases, and an integration test asserting the index is built once across many
imports — the adapter must pass the Set through, since a defensive copy would
hand a fresh WeakMap key per call and restore the old behaviour (the same trap
Python hit in PR #1918).
Two other providers have the same defect and are left alone here, having no
repository at hand to verify a change against:
- `go/import-target.ts`: `findRootPackageFiles` and `findAllFilesInPkgDir`
scan unmemoized, and the GOPATH fallback calls the latter once per path
segment but the last, so a single import can trigger several full passes;
- `dart/import-target.ts`: the `package:` branch scans once per candidate
path — `lib/<rel>` and bare `<rel>` — and `resolveRelative` scans again in
its suffix fallback, also unmemoized.
`csharp/import-target.ts` is a partial case worth noting: it already builds a
memoized `getWorkspaceFileIndex`, but that is reached only when a `.csproj` is
found; the no-csproj path hands the raw Set to `resolveDirectMatch` and
`resolveByProgressiveStripping`, which scan past it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* test(kotlin): close the blind axes in the import-resolution gate
Review of #2872 found the weak part was the gate, not the resolver: four
plausible follow-up mutations passed `--check` with a byte-identical
fingerprint, `cases` AND `non_null`. Each is now caught, and each was
re-checked against the mutation it exists to stop.
- The hashed record carried `order | fromFile | targetRaw | result` but not
the FILE SET, so a corpus edit that swapped the workspace under a case
while leaving its result string alone was invisible. Leaving the resolver
untouched and editing only the corpus, two documented load-bearing cases
could be gutted — the "exact beats an earlier suffix" case losing its
competing file, the repeated-directory negative case losing its file
entirely — with the gate green. The file set is now part of the record, and
that same edit now moves the fingerprint.
- The corpus capped path depth at 8 components and packages at 16 files,
which are precisely the two axes the loops this change added run on. It now
carries 11- and 13-component paths, queries against suffix keys deeper than
seven segments, a 40-file package, and a fuzz that spans both. Verified:
capping suffix-key depth at 7, skipping the `dirChildren` suffix loop above
depth 8, and capping a bucket at 17 entries each now move the fingerprint,
where all three previously passed.
- `non_null` was reported but never asserted; it is asserted beside `cases`.
That closes only the "resolves nothing at all" hole — it stayed 11612 under
all three code mutations above and under the corpus edit — so it is a
companion to the two fixes above, not a substitute for either.
- A ratio cannot see a constant factor, and a file-count ratio cannot see a
depth cost. `--check` now also asserts a DEPTH ratio (file count fixed,
paths 24 components against 8) and an absolute ceiling on the small arm: a
full workspace scan reintroduced on 1-in-32 imports scores 1.490, inside
the scaling budget, while running 2.8x slower.
The baseline is re-derived, not adjusted: the pre-index implementation and the
index both print
ebf1790bf1d42dad483a51f2cbdeb2351e493b9e8236e4eedeef592dd81e2c5c over the new
20106-case corpus, 13256 of them non-null.
Both test suites were shown to be non-load-bearing and now are:
- the parity test's repeated-directory case put `data` at the LEADING
segment, so the `startsWith` guard fired and the `indexOf` rule its own
comment describes was never reached — a resolver with that check relaxed to
`>= 0` passed all 18 cases. A mid-path case now pins it, and a backslash
fan-out case pins `norm.lastIndexOf` against `raw.lastIndexOf`, which was
also bench-only. Both mutations now fail the unit suite.
- the index-reuse test discarded all 200 return values, so a build count of 1
was equally true of an adapter that had stopped resolving anything. It now
asserts results, and its docstring premise is corrected: every one of its
imports hit the tier-1 suffix lookup and none reached the fan-out it
claimed to exercise. Half now genuinely do. The `undefined as never` casts
and the `?.` are gone — both trailing parameters are optional and the
member is required.
Resolver changes, all output-identical against the differential above:
- `dirChildren` buckets are frozen once built. `findKotlinPackageFiles` hands
a bucket straight out of the index, and the `readonly string[]` return type
does not survive the caller: the finalize pass normalizes with
`Array.isArray(t) ? t : [t]`, and `isArray`'s `arg is any[]` predicate
widens the true branch, so `tsc --strict` accepts a `.sort()` there. A
downstream sort would permanently reorder the cached bucket and flip the
first-child tier for every later import in the run.
- `stripped` is computed only after tier 1 misses, with `lastIndexOf`/`slice`
instead of `split`/`slice`/`join`. Measured -20% small arm, -21% large arm.
- `KOTLIN_EXTENSIONS` now comes from the existing `import-resolvers/jvm.ts`
export instead of a fourth inlined copy.
- A note on why the shared `buildSuffixIndex` is not reused, with the four
probes that diverge, and the measured basename-bucket comparison — the one
place this was less documented than the Python precedent it follows, and
the question the Go/Dart/C# follow-ups will each face.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Gergő Magyar <gergomagyar@icloud.com>
|
||
|---|---|---|
| .. | ||
| cfg | ||
| cli | ||
| group | ||
| mcp | ||
| optional-grammars | ||
| resolvers | ||
| analyze-atomic-swap.test.ts | ||
| analyze-embedding-flags-e2e.test.ts | ||
| analyze-heap-oom-e2e.test.ts | ||
| analyze-index-lock-concurrency.test.ts | ||
| analyze-wal-checkpoint-failure.test.ts | ||
| analyzer-identity-cli.test.ts | ||
| antigravity-hook-e2e.test.ts | ||
| api-impact-e2e.test.ts | ||
| api-impact-method-e2e.test.ts | ||
| api-query.test.ts | ||
| ast-helpers-object-literal-binding.test.ts | ||
| augmentation.test.ts | ||
| basicblock-roundtrip.test.ts | ||
| block-scope-shadowing.test.ts | ||
| c-cpp-typedef-legacy-parse.test.ts | ||
| callable-capture-options-literal-gate.test.ts | ||
| caller-identity-regression.test.ts | ||
| cjs-exports-assignment.test.ts | ||
| class-impact-all-languages.test.ts | ||
| cli-e2e.test.ts | ||
| cli-limit-e2e.test.ts | ||
| closure-binding-labels.test.ts | ||
| closure-review-findings.test.ts | ||
| cobol-pipeline-benchmark.test.ts | ||
| const-function-twin.test.ts | ||
| context-cross-language-anchor.test.ts | ||
| context-resource-staleness.test.ts | ||
| context-typed-property.test.ts | ||
| copy-parallel-invariant.test.ts | ||
| cpp-adl-benchmark.test.ts | ||
| cpp-captures-typeclass-benchmark.test.ts | ||
| cpp-pipeline-benchmark.test.ts | ||
| cross-file-binding.test.ts | ||
| csharp-pipeline-benchmark.test.ts | ||
| csharp-scope-capture-tripwire.test.ts | ||
| csv-pipeline.test.ts | ||
| dispatch-guard-route-pipeline.test.ts | ||
| django-route-extraction-e2e.test.ts | ||
| doc-comment-description-e2e.test.ts | ||
| enrichment.test.ts | ||
| expo-routes.test.ts | ||
| extension-binary-real.test.ts | ||
| fastapi-composed-route-constants.test.ts | ||
| fastapi-prefix-pipeline.test.ts | ||
| filesystem-walker.test.ts | ||
| fts-cjk-segmentation-search.test.ts | ||
| fts-description-search.test.ts | ||
| fts-extension-e2e.test.ts | ||
| fts-fullfile-search.test.ts | ||
| fts-repair-warm-session.test.ts | ||
| fts-stemmer-sweep.test.ts | ||
| function-local-identity.test.ts | ||
| go-multi-name-worker-metadata.test.ts | ||
| go-pipeline-benchmark.test.ts | ||
| grammar-introspection.test.ts | ||
| grammar-literal-validation.test.ts | ||
| graph-emit-streaming-roundtrip.test.ts | ||
| has-method.test.ts | ||
| hooks-e2e.test.ts | ||
| http-inline-handler-symbol-roundtrip.test.ts | ||
| ignore-and-skip-e2e.test.ts | ||
| impact-ambiguous-blast-radius.test.ts | ||
| impact-epistemic-lower-bound.test.ts | ||
| impact-pdg-callsummary-degradation.test.ts | ||
| impact-pdg-degradation.test.ts | ||
| impact-pdg-e2e.test.ts | ||
| impact-pdg-fixtures.test.ts | ||
| impact-pdg-fullchain-e2e.test.ts | ||
| impact-pdg-id-degradation.test.ts | ||
| impact-pdg-interproc.test.ts | ||
| impact-pdg-shape.test.ts | ||
| impact-pdg-statement-precise.test.ts | ||
| impact-pdg-traversal.test.ts | ||
| impact-zero-caller-risk.test.ts | ||
| instance-ownership-pipeline-benchmark.test.ts | ||
| java-class-impact.test.ts | ||
| js-array-method-callback-attribution.test.ts | ||
| kotlin-import-index-reuse.test.ts | ||
| lbug-close-handle-release.test.ts | ||
| lbug-conn-serialization.test.ts | ||
| lbug-core-adapter.test.ts | ||
| lbug-delete-nodes-for-files.test.ts | ||
| lbug-load-overlap-errors.test.ts | ||
| lbug-load-overlap.test.ts | ||
| lbug-load-prof.test.ts | ||
| lbug-lock-retry.test.ts | ||
| lbug-multiwriter-deadlock.test.ts | ||
| lbug-non-ascii-path.test.ts | ||
| lbug-open-retry.test.ts | ||
| lbug-orphan-sidecar-recovery.test.ts | ||
| lbug-pool-stability.test.ts | ||
| lbug-pool.test.ts | ||
| lbug-query-importers-batch.test.ts | ||
| lbug-readonly-init.test.ts | ||
| lbug-vector-extension.test.ts | ||
| literal-collectors.test.ts | ||
| local-backend-calltool.test.ts | ||
| local-backend.test.ts | ||
| local-symbol-pruner-pipeline.test.ts | ||
| markdown-processor-crlf.test.ts | ||
| mcp-line-display.test.ts | ||
| multi-branch-analyze.test.ts | ||
| multi-verb-route-identity.test.ts | ||
| object-literal-method-exports.test.ts | ||
| object-literal-owner-resolution.test.ts | ||
| orm-dataflow.test.ts | ||
| parse-impl-chunk-concurrency.test.ts | ||
| parse-impl-clone-skip.test.ts | ||
| parse-impl-env-reads.test.ts | ||
| parse-impl-large-fixture.test.ts | ||
| parse-impl-progress-monotonic.test.ts | ||
| parse-impl-quarantine-cache-skip.test.ts | ||
| parsing.test.ts | ||
| pdg-emit-streaming-roundtrip.test.ts | ||
| pdg-query.test.ts | ||
| php-pipeline-benchmark.test.ts | ||
| php-scope-capture-tripwire.test.ts | ||
| pipeline-graph-golden.test.ts | ||
| pipeline.test.ts | ||
| python-import-index-reuse.test.ts | ||
| python-scope-capture-tripwire.test.ts | ||
| qualified-class-lookups.test.ts | ||
| query-compilation.test.ts | ||
| route-handler-symbol-roundtrip.test.ts | ||
| route-method-roundtrip.test.ts | ||
| route-parse-skip.test.ts | ||
| ruby-pipeline-benchmark.test.ts | ||
| ruby-scope-capture-tripwire.test.ts | ||
| run-analyze-adopt-failure.test.ts | ||
| rust-pipeline-benchmark.test.ts | ||
| rust-scope-capture-tripwire.test.ts | ||
| search-core.test.ts | ||
| search-pool.test.ts | ||
| server-analyze-token-validation.test.ts | ||
| server-analyze.test.ts | ||
| server-http-startup.test.ts | ||
| setup-antigravity.test.ts | ||
| setup-skills.test.ts | ||
| setup-uninstall-roundtrip.test.ts | ||
| shape-check-regression.test.ts | ||
| skills-e2e.test.ts | ||
| spring-aop-benchmark.test.ts | ||
| spring-aop-mcp.test.ts | ||
| spring-aop-pipeline.test.ts | ||
| spring-bean-mcp.test.ts | ||
| spring-bean-metadata-roundtrip.test.ts | ||
| spring-bean-pipeline.test.ts | ||
| spring-bean-resource-benchmark.test.ts | ||
| spring-bean-resource-pipeline.test.ts | ||
| spring-conditionals-pipeline.test.ts | ||
| spring-config-mcp.test.ts | ||
| spring-config-pipeline.test.ts | ||
| spring-di-benchmark.test.ts | ||
| spring-di-pipeline.test.ts | ||
| spring-inheritance-benchmark.test.ts | ||
| spring-interface-inheritance-pipeline.test.ts | ||
| spring-route-pipeline.test.ts | ||
| staleness-and-stability.test.ts | ||
| structural-pair-coverage.test.ts | ||
| swift-conditional-directive.test.ts | ||
| swift-scope-capture-tripwire.test.ts | ||
| taint-explain.test.ts | ||
| this-boundary.test.ts | ||
| tree-sitter-languages.test.ts | ||
| typescript-async-generator-functions.test.ts | ||
| vue-pipeline-benchmark.test.ts | ||
| worker-pool.test.ts | ||