mirror of
https://github.com/abhigyanpatwari/GitNexus.git
synced 2026-10-03 02:21:44 +00:00
* fix(ingestion): reduce parse-phase memory for huge repos (#1983)
Stop retaining full parse-cache chunks in RAM alongside the merged graph,
slim on-disk shards, defer worker ParsedFile emission for scope-resolver
languages, and add GITNEXUS_DEBUG_HEAP probes for OOM diagnosis.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(ingestion): address #2038 tri-review findings (parse-phase memory)
Resolves the confirmed review findings on PR #2038:
- P1: thread exportedTypeMap through the sequential parse path
(processParsingSequential) so a no-worker run over a partially-warm
cache no longer silently drops the sequential-miss files' exported
types. Cache hits made exportedTypeMap.size > 0, suppressing the
end-of-loop buildExportedTypeMapFromGraph rebuild, but the sequential
path never populated the map. Regression test added (fails on the
pre-fix tree, passes after) plus a fully-sequential differential oracle.
- P2: saveParseCache builds its on-disk index from hashes actually
written/copied (writtenKeys), never a usedKeys hash whose shard write
or copy was skipped — no more phantom index entries.
- P2: add a unit test asserting SCOPE_RESOLUTION_LANGUAGES stays in sync
with SCOPE_RESOLVERS (asymmetric drift would lose a language's ParsedFile).
- Backfill cache coverage: loadParseCacheChunk missing/corrupt -> undefined,
pruneCache onDiskKeys branch, slim preserves nodes, saveParseCache
copy-evicted-shard round-trip.
- Cleanups: single-source heap-probe gating via isDebugHeapEnabled();
hoist the per-chunk mkdir in persistParseCacheChunk behind a
process-scoped Set; gate COBOL's unused worker-side ParsedFile
extraction (graph nodes still come from cobolPhase) while keeping
fileCount/progress unconditional.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(ingestion): remove dead worker-side ParsedFile extraction
After #2038 gated worker `ParsedFile` emission behind `!isScopeResolutionLanguage(language)`, and with all 16 SupportedLanguages registered in SCOPE_RESOLVERS, that gate was structurally always true — the worker already produced no ParsedFiles and scope-resolution re-extracts each file from source on the main thread (run.ts). Remove the now-dead machinery:
- Drop both worker `extractParsedFile` call-sites (tree-sitter processFileGroup + the standalone-provider branch) and the `result.parsedFiles.push`. The standalone branch keeps fileCount/onFileProcessed per file. `result.parsedFiles` stays declared but empty (field removal deferred).
- Remove the now-orphaned `scopeSourceKind` var + `ScopeCaptureSourceKind`/`extractParsedFile`/`isScopeResolutionLanguage` imports.
- Delete the consumerless `migrated-languages.ts` (isScopeResolutionLanguage + SCOPE_RESOLUTION_LANGUAGES) and its drift-guard test — parse-worker was their only importer. Also improves AGENTS.md "shared ingestion code must not name languages" compliance.
`extractParsedFile` and the scope-extractor-bridge stay (scope-resolution/run.ts + Vue resolver use them). Behavior-preserving: worker-sequential-parity passes before and after; tsc/eslint clean; no baseline/golden drift.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(ingestion): worker-pool-only parsing; remove sequential parser (#1983)
Completes the #1983 huge-repo parse-OOM effort by making the worker pool
GitNexus's sole parse path.
Parallel serialization (the perf core): workers serialize their ParsedFiles to
a disk store in parallel and stream them back to scope-resolution, so the main
thread no longer re-parses every file (the tree-sitter native-memory leak that
caused the OOM). Adds chunk merge-pipelining + work-proportional chunk sizing so
the pool stays saturated.
Remove the sequential parser: `--workers 0`, `GITNEXUS_WORKER_POOL_SIZE=0`, and
`skipWorkers` now hard-error (no silent degrade — #1741); the small-repo
threshold no longer selects an in-process path; pool creation stays lazy /
cache-miss-gated so warm all-hit runs never spawn workers.
Worker-path parity fixes — removing sequential surfaced two pre-existing gaps
that tiny-fixture tests had masked by running below the worker threshold, both
fixed by carrying per-file metadata as DATA across the worker boundary (never
re-parsing on the main thread, preserving the OOM fix):
- C++: templateConstraints wired into worker node identity (SFINAE overload
disambiguation) + ADL / inline-namespace capture side-channel serialized
onto the ParsedFile.
- Kotlin: companion-scope side-channel serialized the same way (companion /
static dispatch).
Validation: tsc + build clean; full suite green (10,190 pass — the only
deterministic failures were the now-fixed C++/Kotlin worker-path gaps; the 2
remaining full-run failures are pre-existing load flakiness, green in
isolation); cpp-pipeline benchmark stays linear on a 1-worker pool.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(ingestion): wire C static-linkage side-channel + ADL O(1) collect + tri-review cleanups (#1983)
Follow-up to the worker-pool-only refactor, from a tri-review of the parse path.
- C static-linkage side-channel (P1): cProvider had no collect/applyCaptureSideChannel,
so on the now-sole worker path C `static` file-local marks were lost across the worker
boundary -> false cross-file CALLS edges + over-broad #include wildcard visibility on
every C analysis (the Linux kernel is C). Mirror the C++/Kotlin wiring: serialize
`staticNames` per file onto ParsedFile.captureSideChannel and restore it on the main
thread (no re-parse). + a worker-path regression test (the existing c-static-isolation
fixture passed vacuously — its collision resolves via #include before the global
free-call fallback ever consults static-linkage).
- captureSideChannel `kind` discriminant: add `kind:'cpp'`/`kind:'c'` tags + guards
(Kotlin already had one) now that C/C++/Kotlin share the single generic field.
- Perf: collectCppAdlSideChannel scanned the whole argInfoBySite/noAdlSites maps per file
(O(F^2) per sub-batch, ~100M parseSiteKey calls at kernel scale). Add per-filePath
lockstep indexes -> O(1) collect; serialized snapshot byte-identical.
- Cleanups: inline the one-line processParsingWithWorkers wrapper into processParsing;
drop the always-empty WorkerExtractedData.calls/assignments/constructorBindings fields;
remove the voided astCache param from processParsing; refresh stale "sequential
fallback" JSDoc.
Validation: tsc + build clean; cpp 297/297, c 8/8 (incl. the new worker-path
static-linkage guard), typescript + parsedfile-store green; cpp ADL benchmark stays linear.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* perf(scope-resolution): index C/C++ #include resolution in finalize (O(n²)→O(n))
Kernel-scale C/C++ analysis ground in finalizeScopeModel because three
per-#include operations each did a full O(F) scan with no index — the
finalize O(n²) that surfaced once the #1983 parse-phase OOM was fixed:
- expand{C,Cpp}WildcardNames: parsedFiles.find() per wildcard edge → O(R·F)
- resolveImportTarget: new Set(allFilePaths) rebuilt per #include
- resolveCImportTarget: suffix-match scanned all workspace paths
Each is replaced with a WeakMap-per-pass index keyed on the stable
parsedFiles/allFilePaths references that scope-resolution run.ts passes
once per pass:
- Map<ScopeId,ParsedFile> for wildcard expansion (c/static-linkage.ts +
cpp/file-local-linkage.ts)
- memoized augmented header set (c/scope-resolver.ts + cpp/scope-resolver.ts)
- basename-bucketed suffix index in resolveCImportTarget (c/import-target.ts),
shared by C and C++ since resolveCppImportTarget delegates to it
Collapses the C/C++ finalize from O(R·F) to O(R+F). Pure-perf, byte-identical
edge output: 962 targeted tests green (490 C + 472 C/C++ scope-resolution);
the basename index preserves the exact endsWith('/'+target) match and the
fewest-path-components-then-lexicographic tie-break.
The kernel's ~25-30k .h headers are classified C++, so both providers must
be fixed. Proven on the Linux kernel: the C finalize completed
(sr-post-finalize lang=c → sr-end lang=c), which the pre-fix run never
reached in 16+ min of grinding.
Build-independent follow-ups (separate from this finalize fix), documented
for later: emitFreeCallFallback same-name buckets (emit phase),
buildGraphNodeLookup + precount global setup, the ParsedFile store-load,
the dart/go/ruby expand-wildcards .find siblings, and the ~26GB
scope-resolution memory floor (full kernel completion needs >~40GB RAM).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(bench): regenerate C scope-capture baseline for the #1983 c-static-linkage-worker fixture
bench/scope-capture/measure.mjs fingerprints emitCScopeCaptures over the
lang-resolution/c-* fixture corpus. The #1983 PR added the
c-static-linkage-worker fixture (caller.c/lib.c/lib.h/local.c — the
worker-path static-linkage side-channel test) but did not regenerate the C
baseline, so `--check` has been red on this branch (main, lacking the
fixture, still matches 0de009b).
Pure fixture-corpus drift — no c/captures.ts or query change branch-vs-main,
existing fixtures' captures byte-identical (c-captures.test.ts 45/45),
scaling stays linear (~0.97). Regenerated: 0de009b -> 39f3a83. Bench now
PASS (14 languages). Unrelated to the finalize O(n²) fix.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* perf(scope-resolution): lower kernel-scale resident memory floor + setup cost
Reduce the scope-resolution resident-memory floor and setup throughput on
huge repos (Linux kernel), the wall that remains after #1983 (parse OOM) and
the finalize O(n^2) fix (b71c77b8). Five units; all preserve byte-identical
edge output (C fixture 177n/255e + c/cpp/cross-file/php/static-linkage suites
green, 619 tests).
U1 (src/cli/analyze.ts): RAM-aware auto heap-cap. Replace the hardcoded
16384MB cap with computeHeapCapMb = max(16384, floor(0.75*effectiveRAM)),
where effectiveRAM = min(os.totalmem(), process.constrainedMemory()) with the
unconstrained-sentinel guard. Add --max-semi-space-size=128 on the respawn.
A user-supplied NODE_OPTIONS heap still wins (no re-exec). Verified: 23973MB
on a 31964MB box, 16384 floor on small machines, cgroup-aware, sentinel safe.
U2 (src/storage/parsedfile-store.ts, .../pipeline/phase.ts): export forceGc()
and call it at the per-language eviction boundary, so a finished language's
ParsedFiles are reclaimed before the next language's store-load instead of
collected lazily under the next pass's allocation pressure (which at cap>=RAM
degrades into swap-thrash). Measured on a real drivers/net/ethernet run:
C 2113->894MB and C++ 1754->1057MB reclaimed at the boundary (no fragmentation
defeat). Answers the plan's Open Question 1.
U3 (src/storage/parsedfile-store.ts): intern def objects by nodeId in the load
reviver so a SymbolDefinition's three serialized copies (localDefs /
scope.ownedDefs / scope.bindings[].def) collapse to one shared object on load.
Per-shard def pool (a def's copies are shard-local). Measured ~42% off the
def-object retained heap (3->1; 1.8M->600k distinct objects on 600k defs).
U4 (.../passes/free-call-fallback.ts): memoize pickUniqueGlobalCallable's
post-filter candidate list per (name, callerFilePath), only when no per-caller
visibility filter applies (the list is then a pure function of name+file), so
repeated free calls of one name from a file reuse the same-name-bucket scan
instead of re-walking a potentially huge bucket per site. The cached array is
read-only-consumed by the .filter()-based arity/overload narrowers. Exported
pickUniqueGlobalCallable + buildGlobalCallableIndex and added an equivalence
test (memoized == un-memoized reference for every (name, file, arity),
including warm-cache repeats and cross-file file-local exclusion).
U5 (.../pipeline/phase.ts): replace the O(L*F) per-language precount + repeated
scannedFiles.filter() with a single O(F) partition-by-language pass; bracket
buildGraphNodeLookup with scope-setup-nodeLookup heap probes so the long setup
is no longer silent.
Plan: docs/plans/2026-06-06-001-perf-kernel-scope-resolution-memory-plan.md
(U6 out-of-core global index deferred). Note: the kernel's full C++ pass floor
(~20k headers + the 8.8GB graph) likely still exceeds 24GB by itself, which is
why U6 remains the only unit that clears the wall.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(test): match OOM-guidance e2e assertions to the U1 reworded hint
The analyze-heap-oom-e2e real-child-OOM test still asserted the pre-U1
wording ('...out of memory.' + a hardcoded 24576 cap). U1 reworded the hint
to mention the auto heap-cap and use a <MB> placeholder, so the three
toContain substrings no longer matched (the assertion at line 62 failed on
all platforms). Update them to the current message. The unit twin
(analyze-heap-respawn) was already updated in 85bfc216; this integration
test was missed by the targeted local run.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* perf(lbug): U6a — deterministic id-sorted graph output behind GITNEXUS_SORT_GRAPH_OUTPUT
First increment of U6 (out-of-core scope-resolution). Adds an optional
deterministic ordering of node + relationship CSV rows by their unique graph
id, behind GITNEXUS_SORT_GRAPH_OUTPUT (default OFF = today's graph-insertion
order, byte-identical — the iterator is returned untouched). With the flag ON
the CSV becomes a pure function of the node/edge SET rather than of emit order.
This is the structural enabler for the windowed/out-of-core resolve (U6b-U6d):
csv-generator.ts:518 currently iterates graph.iterRelationships() in insertion
order with NO terminal sort, so any deviation from parsedFiles-order emit would
change bytes. With U6a on, a windowed emit need only reproduce the same edge
SET, not the global insertion order — removing the single largest byte-identical
hazard from every later windowing step.
Verified: default off keeps the existing csv-pipeline suite byte-identical; on,
node rows are id-sorted and output is independent of graph insertion order
(set-build) with the same node/edge set.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* perf(storage): U6d foundation — disk-backed scope store + lazy ScopeTree
Adds scope-index-store.ts: persistScopeShards (per-file scope shards via the
proven mapReplacer + def-interning reviver) + DiskBackedScopeTree, a lazy
ScopeTree that serves getScope from a bounded LRU of decoded shards plus a small
resident skeleton (scopeId -> {shard, childIds, parent}). Exports
makeInterningReviver from parsedfile-store for reuse.
This is the contained, highest-risk mechanism of U6d (out-of-core scope
resolution): the emit passes reach the heavy per-Scope binding payload
(~17-20GB on the kernel) ONLY through scopeTree.getScope (a point lookup) and
getChildren — they never read parsed.scopes directly — so moving that payload to
disk behind getScope is transparent. Every consumer reads a Scope BY VALUE, so a
value-faithful disk round-trip is byte-identical to resolution.
Proven in isolation: DiskBackedScopeTree is value-identical to buildScopeTree
for getScope/getChildren/getParent/getAncestors/has/size across multiple files
and after LRU eviction, and preserves the def-identity collapse (ownedDefs[i]
=== binding.def). Nothing wires it yet (the resolution-pipeline integration is
the next increment) — zero production impact; default off.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* perf(scope-resolution): U6d integration — seal scopeTree to disk before emit (GITNEXUS_DISK_SCOPE_INDEX)
Wires the U6d out-of-core scope index into the live pipeline behind
GITNEXUS_DISK_SCOPE_INDEX (default OFF = byte-identical). When on:
- finalize-orchestrator builds a TransitionalScopeTree (validated, fully
resident) instead of buildScopeTree, so finalize/propagate/resolve are
unchanged.
- After resolve, before emit, run.ts seals it: persists the scopes to a
file-sharded scope-index-store, swaps the model's scopeTree to disk-backed
serving from the inside (the frozen bundle can't be reassigned, but the
wrapper nulls its own resident backing), and drops the heavy Scope.bindings
payload from all THREE holders — the model's tree (seal), the caller's
preExtractedParsedFiles, and run.ts's own parsedFiles (scope-stripped copies
for emit). Emit reads scopes only via scopeTree.getScope (a point lookup,
now disk-backed + LRU) — verified it never reads parsed.scopes.
Purpose: lower the per-language resident PEAK (kernel C pass ~20→~12 GB by
moving the ~8-9 GB scope payload to disk) so the analysis fits on smaller-RAM
machines. At >=24 GB the full kernel already fits with U1-U5 (U2's 8.7 GB
inter-language forceGc reclaim keeps each pass under cap) — empirically
confirmed — so this is the sub-24 GB lever, not needed at 24 GB.
Byte-identical evidence: DiskBackedScopeTree/TransitionalScopeTree return
value-identical scopes vs buildScopeTree (getScope/getChildren/getParent/
getAncestors, across files + after LRU eviction + post-seal); emit reads only
getScope + referenceSites; flag-off (394 tests) and flag-on-resident (91 tests)
resolver suites stay green; an end-to-end A/B on a 212-file C+cpp+rust subset
produced identical 17,444 nodes / 31,343 edges with the seal firing per language
(c: 410→141 MB reclaimed). Kernel-scale peak-drop measurement pending the
in-flight verdict run freeing memory.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* perf(scope-resolution): U6d — id-back workspaceIndex so the disk seal can reclaim scopes
The kernel run revealed the contained scopeTree seal didn't lower the heap:
WorkspaceResolutionIndex held Scope OBJECTS (classScopeByDefId / moduleScopeByFile),
built from every ParsedFile and live through emit, so the ~28k module + class
scopes stayed pinned past the seal (sr-seal-pre 17,583 -> sr-seal-post 17,771 MB,
no drop). It was the sole residual Scope-object holder (SemanticModel holds none).
Fix: classScopeByDefId / moduleScopeByFile become id-backed ScopeByKeyView
instances — a ReadonlyMap<K, Scope> facade over a K->ScopeId map + the scopeTree,
whose .get fetches via scopeTree.getScope(id). The index now pins only ids, so
once the tree seals to disk the scopes become collectible. Byte-identical: the
view returns the same Scope the resident tree holds (or a value-identical revived
one in disk mode), and iteration keeps the old insertion order. buildWorkspace
ResolutionIndex takes an optional scopeTree (live pipeline passes it); without it
(unit tests) the legacy direct Scope-object maps are returned unchanged.
Verified byte-identical: 733 tests across workspace-index / imported-return-types
/ c / cpp / cross-file / go / java. Kernel peak-drop re-measurement to follow.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* perf(scope-resolution): U6d — precompute exportedCallableByName (fix disk-getScope thrash)
The workspaceIndex id-backing freed the kernel scopes but exposed a throughput
collapse: findExportedDefByName's workspace fallback (walkers.ts:1019) scanned
EVERY module scope's bindings per unresolved free call, and under the U6d
disk-backed scopeTree each module-scope access faulted a shard in from disk —
lib ON went ~1min -> ~7.5min.
Fix: precompute the fallback result once into
WorkspaceResolutionIndex.exportedCallableByName (simpleName -> first module-local
callable def, first-file-wins — the exact semantics the scan returned), built
from the resident module-scope bindings at index-build time. findExportedDefByName
now does an O(1) lookup with zero disk reads.
Result: lib ON ~7.5min -> 21s (cache-warm), byte-identical 17,444/31,343; 758
tests green across workspace-index + c/cpp/cross-file/go/python.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: rename cryptic U-unit codes to descriptive names in comments
The plan-unit shorthand (U3/U4/U6a/U6d/...) was meaningless in the code.
Renamed in comments + test descriptions (no behavior change, byte-identical):
out-of-core scope index (was U6)
deterministic output (was U6a)
disk-backed scope seal (was U6d)
def-object interning (was U3)
free-call candidate cache (was U4)
Also renamed throughout the PR title/summary. Pushed commit messages keep
their original U-codes as historical record.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(ingestion): durable ParsedFile shards for warm-cache coverage (#2038)
On a warm re-analyze where every chunk is a parse-cache HIT, no parse worker
runs, the run-scoped ParsedFile store is cleared at parse start, and the cached
ParseWorkerResult carries no ParsedFiles (the worker writes them to the store
and empties them from the message). Scope-resolution then found an empty store
and fell back to main-thread extractParsedFile — re-opening the #1983
tree-sitter native-leak OOM the disk store closes (abhigyanpatwari review on
parse-cache.ts).
Fix: workers ALSO write their ParsedFiles to a durable, content-addressed store
(parsedfile-cache/) keyed by chunk hash, mirroring the parse cache's lifecycle
(version-gated by PARSE_CACHE_VERSION, pruned in lockstep to the surviving
keys). On a warm hit the chunk's durable shards are byte-COPIED into the
run-scoped store (no re-parse, no re-serialize -> byte-identical), so
scope-resolution streams them exactly as on a cold run. A coherence gate
re-dispatches the worker whenever a cached chunk's durable shards are missing
(migration / pruned / version-stale) -- never the main-thread extract.
- worker-pool/parse-worker: thread chunkHash through dispatch->job->flush
(incl. split/requeue) so the worker tags its durable shard by content
- parsedfile-store: durable persist / restore / index / prune API (sibling
dir, never cleared per run); content-addressing makes stale reuse impossible
- parse-impl: load durable index, gate the cache hit on durable coverage,
restore on hit, dispatch chunkHash on miss
- run-analyze: prune+save the durable store to the parse cache's surviving keys
- saveParseCache returns its written keys (the durable keepKeys)
Verified on linux/lib: warm preExtractedHits = full coverage (520/207/1, zero
main-thread re-parse), byte-identical cold==warm (17,456n/31,353e), warm 8.5x
faster. New two-run + mixed-mode + coherence-gate regression test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(ingestion): clear stale scope-index-store shards on each seal (#2038)
The disk-backed scope index writes sequential s<n>.json shards into a shared
<storagePath>/scope-index-store/ dir, with the index resetting per
persistScopeShards call. A seal that writes fewer shards than a previous one
(a later language with fewer files, or a re-run of a shrunken repo) left stale
tail shards on disk indefinitely -- never read by the disk-backed tree, but
multi-GB on kernel-scale repos.
Add clearScopeIndexStore() and clear at the start of persistScopeShards: the
previously sealed language has finished emit and been released before the next
seal runs, so its DiskBackedScopeTree never reads those shards again. Unit
tests: a stale prior-run shard is removed, a fewer-files re-seal leaves no tail
shards, and the helper is idempotent.
Addresses abhigyanpatwari review on run.ts (disk hygiene for the
GITNEXUS_DISK_SCOPE_INDEX path).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
220 lines
8.8 KiB
TypeScript
220 lines
8.8 KiB
TypeScript
/**
|
||
* Unit tests for `pickUniqueGlobalCallable` + its per-pass `scopeDefsCache`
|
||
* memo (free-call candidate cache — kernel scope-resolution throughput, 2026-06-06).
|
||
*
|
||
* It memoizes the post-filter candidate list keyed (simpleName, callerFilePath)
|
||
* so repeated free calls of the same name from one file reuse the same-name
|
||
* bucket scan instead of re-walking a (potentially huge) bucket per site. The
|
||
* memo is behavior-PRESERVING: the candidate list is a pure function of
|
||
* (name, callerFilePath) whenever no per-caller visibility filter applies, and
|
||
* the cached array is only ever read (never mutated) by the arity / overload
|
||
* narrowers. The only observable change is performance.
|
||
*
|
||
* These tests exercise the helper via synthetic `SymbolDefinition` stubs — no
|
||
* fixtures, no pipeline — mirroring `pick-unique-global-class.test.ts`. The
|
||
* load-bearing test is the EQUIVALENCE battery: the memoized path must agree
|
||
* with an un-memoized reference scan (cache `undefined`) on the resolved nodeId
|
||
* for every (name, callerFile, arity), including warm-cache repeats and
|
||
* interleaved callers — which is what guards the byte-identical edge invariant
|
||
* (the C fixture 177n/255e + the c/cpp/cross-file resolver suites).
|
||
*/
|
||
|
||
import { describe, it, expect } from 'vitest';
|
||
import type { SymbolDefinition } from 'gitnexus-shared';
|
||
import {
|
||
buildGlobalCallableIndex,
|
||
pickUniqueGlobalCallable,
|
||
} from '../../../src/core/ingestion/scope-resolution/passes/free-call-fallback.js';
|
||
import type { ScopeResolutionIndexes } from '../../../src/core/ingestion/model/scope-resolution-indexes.js';
|
||
import type { SemanticModel } from '../../../src/core/ingestion/model/semantic-model.js';
|
||
|
||
const mkDef = (overrides: Partial<SymbolDefinition> & { nodeId: string }): SymbolDefinition => ({
|
||
filePath: 'x.c',
|
||
type: 'Function',
|
||
qualifiedName: overrides.nodeId,
|
||
...overrides,
|
||
});
|
||
|
||
/** Wrap a flat def list as the `scopes.defs.byId` map `buildGlobalCallableIndex`
|
||
* iterates. Insertion order is preserved by `Map`, so this reproduces the
|
||
* `defs.byId.values()` iteration order the per-site scan walked. */
|
||
const mkScopes = (defs: readonly SymbolDefinition[]): ScopeResolutionIndexes =>
|
||
({
|
||
defs: {
|
||
byId: new Map(defs.map((d) => [d.nodeId, d])),
|
||
},
|
||
}) as unknown as ScopeResolutionIndexes;
|
||
|
||
/** Empty semantic model — the free-call fallback's model-side pool returns
|
||
* nothing, so resolution is driven entirely by the scope-index path under
|
||
* test (the path the memo covers). */
|
||
const EMPTY_MODEL = {
|
||
symbols: { lookupCallableByName: () => [] as readonly SymbolDefinition[] },
|
||
methods: { lookupMethodByName: () => [] as readonly SymbolDefinition[] },
|
||
} as unknown as SemanticModel;
|
||
|
||
// Corpus: a mix of unique names, a same-name-across-files bucket with one
|
||
// file-local (C `static`) member, a distinct-qualifiedName ambiguous pair,
|
||
// and a same-file arity-overload pair.
|
||
const CORPUS: readonly SymbolDefinition[] = [
|
||
mkDef({ nodeId: 'def:probe', qualifiedName: 'drv.probe', filePath: 'drv.c', parameterCount: 1 }),
|
||
// `open` lives in two files; util.c's is file-local (static).
|
||
mkDef({
|
||
nodeId: 'def:open#core',
|
||
qualifiedName: 'core.open',
|
||
filePath: 'core.c',
|
||
parameterCount: 2,
|
||
requiredParameterCount: 2,
|
||
}),
|
||
mkDef({
|
||
nodeId: 'def:open#util',
|
||
qualifiedName: 'util.open',
|
||
filePath: 'util.c',
|
||
parameterCount: 3,
|
||
requiredParameterCount: 3,
|
||
}),
|
||
// distinct-qualifiedName collision (genuinely ambiguous).
|
||
mkDef({ nodeId: 'def:init#a', qualifiedName: 'a.init', filePath: 'a.c', parameterCount: 0 }),
|
||
mkDef({ nodeId: 'def:init#b', qualifiedName: 'b.init', filePath: 'b.c', parameterCount: 0 }),
|
||
// same-file arity overloads.
|
||
mkDef({
|
||
nodeId: 'def:read#1',
|
||
qualifiedName: 'io.read',
|
||
filePath: 'io.c',
|
||
parameterCount: 1,
|
||
requiredParameterCount: 1,
|
||
}),
|
||
mkDef({
|
||
nodeId: 'def:read#2',
|
||
qualifiedName: 'io.read',
|
||
filePath: 'io.c',
|
||
parameterCount: 2,
|
||
requiredParameterCount: 2,
|
||
}),
|
||
];
|
||
|
||
const INDEX = buildGlobalCallableIndex(mkScopes(CORPUS));
|
||
|
||
/** Only util.c's `open` is file-local (mirrors a C `static` helper). */
|
||
const isFileLocalDef = (d: SymbolDefinition): boolean => d.nodeId === 'def:open#util';
|
||
|
||
const callUnmemoized = (
|
||
name: string,
|
||
callerFile: string,
|
||
arity?: number,
|
||
): SymbolDefinition | undefined =>
|
||
pickUniqueGlobalCallable(name, EMPTY_MODEL, INDEX, callerFile, isFileLocalDef, arity);
|
||
|
||
const callMemoized = (
|
||
cache: Map<string, readonly SymbolDefinition[]>,
|
||
name: string,
|
||
callerFile: string,
|
||
arity?: number,
|
||
): SymbolDefinition | undefined =>
|
||
pickUniqueGlobalCallable(
|
||
name,
|
||
EMPTY_MODEL,
|
||
INDEX,
|
||
callerFile,
|
||
isFileLocalDef,
|
||
arity,
|
||
undefined,
|
||
undefined,
|
||
undefined,
|
||
undefined,
|
||
cache,
|
||
);
|
||
|
||
// Every (name, callerFile, arity) the equivalence test exercises. Ordering
|
||
// matters: `open` is hit from three different files (cross-file keying) and
|
||
// some calls repeat (warm-cache path).
|
||
const CALLS: ReadonlyArray<readonly [name: string, file: string, arity: number | undefined]> = [
|
||
['probe', 'x.c', 1],
|
||
['open', 'core.c', 2], // util.c's static excluded → core.open
|
||
['open', 'util.c', 3], // both visible; arity narrows to util.open
|
||
['open', 'other.c', 2], // util.c's static excluded → core.open
|
||
['open', 'core.c', 2], // repeat → warm cache
|
||
['init', 'a.c', 0], // distinct-qn ambiguous → undefined
|
||
['read', 'io.c', 1], // arity overload → read#1
|
||
['read', 'io.c', 2], // arity overload → read#2
|
||
['open', 'util.c', 3], // repeat from a different file → warm cache
|
||
['missing', 'x.c', 0], // miss → undefined
|
||
];
|
||
|
||
describe('pickUniqueGlobalCallable — scopeDefsCache memo', () => {
|
||
it('resolves a unique simple-name match', () => {
|
||
expect(callUnmemoized('probe', 'x.c', 1)?.nodeId).toBe('def:probe');
|
||
});
|
||
|
||
it('excludes a cross-file file-local (static) candidate', () => {
|
||
// From other.c, util.c's static `open` is invisible → only core.open remains.
|
||
expect(callUnmemoized('open', 'other.c', 2)?.nodeId).toBe('def:open#core');
|
||
});
|
||
|
||
it('keeps a unique name even when its arity does not match the call', () => {
|
||
// Single surviving candidate wins regardless of arity — the memo caches the
|
||
// full bucket (not an arity-filtered slice), so this contract is preserved.
|
||
expect(callUnmemoized('open', 'core.c', 99)?.nodeId).toBe('def:open#core');
|
||
});
|
||
|
||
it('narrows by arity when multiple candidates survive', () => {
|
||
expect(callUnmemoized('read', 'io.c', 1)?.nodeId).toBe('def:read#1');
|
||
expect(callUnmemoized('read', 'io.c', 2)?.nodeId).toBe('def:read#2');
|
||
});
|
||
|
||
it('returns undefined for a distinct-qualifiedName collision', () => {
|
||
expect(callUnmemoized('init', 'a.c', 0)).toBeUndefined();
|
||
});
|
||
|
||
it('equivalence: memoized result == un-memoized reference for every call', () => {
|
||
const cache = new Map<string, readonly SymbolDefinition[]>();
|
||
for (const [name, file, arity] of CALLS) {
|
||
const ref = callUnmemoized(name, file, arity);
|
||
const memo = callMemoized(cache, name, file, arity);
|
||
expect(memo?.nodeId).toBe(ref?.nodeId);
|
||
}
|
||
});
|
||
|
||
it('a warm cache from one file does not poison another file’s result', () => {
|
||
const cache = new Map<string, readonly SymbolDefinition[]>();
|
||
// Prime (open, util.c) — its bucket includes the static util.open.
|
||
expect(callMemoized(cache, 'open', 'util.c', 3)?.nodeId).toBe('def:open#util');
|
||
// A different caller file must NOT reuse util.c's bucket: core.c can't see
|
||
// util.c's static, so it resolves to core.open.
|
||
expect(callMemoized(cache, 'open', 'core.c', 2)?.nodeId).toBe('def:open#core');
|
||
expect(callMemoized(cache, 'open', 'other.c', 2)?.nodeId).toBe('def:open#core');
|
||
// Distinct cache entries, one per (name, file).
|
||
expect(cache.size).toBe(3);
|
||
});
|
||
|
||
it('a warm-cache repeat returns the identical def for the same (name, file)', () => {
|
||
const cache = new Map<string, readonly SymbolDefinition[]>();
|
||
const first = callMemoized(cache, 'open', 'core.c', 2);
|
||
const second = callMemoized(cache, 'open', 'core.c', 2);
|
||
expect(second).toBe(first); // same object reference
|
||
expect(cache.size).toBe(1); // one scan, reused
|
||
});
|
||
|
||
it('does not cache when a per-caller visibility filter is present', () => {
|
||
// When isCallerVisible is provided, the candidate list depends on the
|
||
// caller's scope, not just its file — the memo must be bypassed even if a
|
||
// cache map is passed, so it stays empty.
|
||
const cache = new Map<string, readonly SymbolDefinition[]>();
|
||
const visibleAll = (): boolean => true;
|
||
const result = pickUniqueGlobalCallable(
|
||
'open',
|
||
EMPTY_MODEL,
|
||
INDEX,
|
||
'other.c',
|
||
isFileLocalDef,
|
||
2,
|
||
visibleAll,
|
||
undefined,
|
||
undefined,
|
||
undefined,
|
||
cache,
|
||
);
|
||
expect(result?.nodeId).toBe('def:open#core');
|
||
expect(cache.size).toBe(0); // gate: visibility filter ⇒ no caching
|
||
});
|
||
});
|