From 347cc99b4a237f9a71c0084167bcbcbd9d06e610 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Gerg=C5=91=20Magyar?= Date: Sun, 1 Nov 2026 16:48:35 +0000 Subject: [PATCH] perf(scope): reuse proven empty namespace maps (#3798) Co-authored-by: Gergo Magyar --- gitnexus/bench/analyze-phase-breakdown.md | 81 ++++++++++++++++++- .../scope/namespace-target-cache.ts | 11 ++- .../namespace-target-cache.test.ts | 65 +++++++++++++++ 3 files changed, 154 insertions(+), 3 deletions(-) diff --git a/gitnexus/bench/analyze-phase-breakdown.md b/gitnexus/bench/analyze-phase-breakdown.md index cfd4dd93e..a3bbc5cd3 100644 --- a/gitnexus/bench/analyze-phase-breakdown.md +++ b/gitnexus/bench/analyze-phase-breakdown.md @@ -1,7 +1,7 @@ # Analyze phase breakdown — where the time actually goes -Updated 2026-11-01 with production hardening and measurements from main on -LadybugDB **0.22.0** and an experimental fast path for unchanged dirty checkouts. +Updated 2026-11-01 with scope-emission measurements after #3796 merged, plus +production-hardening measurements on LadybugDB **0.22.0** for unchanged dirty checkouts. Records what the `analyze` pipeline costs per phase and the results of optimization experiments. The September measurements remain below as historical @@ -269,6 +269,83 @@ was 73.939s on main and 91.838s on the candidate, accounting for most of the establish its cause. Host load changed and this pair has one sample per variant: neither a real-edit speedup nor a stable regression is established. +### Scope-emission experiment after #3796 — 2026-11-01 + +Baseline runtime and corpus: merged main at +`568ebdc1d9e6fce80a4da306bcb1ecebb20b60e4`. This includes the dirty-snapshot +shortcut and the intervening Java overload changes. The corpus now has **3,375 +indexed files**, **3,111 parseable files**, **118,517 nodes / 356,369 relationships / +3,495 communities / 767 processes**. These measurements form a new paired +comparison; the earlier corpus timings are not its baseline. + +A separate Node CPU profile of a one-file edit identified repeated namespace +lookups inside receiver-bound call emission. The main-thread profile recorded +**29.6s in garbage collection**, **21.7s inclusive in `emitReceiverBoundCalls`**, +and **6.2s inclusive in `collectNamespaceTargets`**. Those intervals overlap and +must not be added. The profile includes startup and post-run graph hashing; +its **140.158s** analysis run is excluded from the timing comparison below. + +The candidate adds a proof to `createNamespaceTargetCache`: while traversing a +file's scopes, record whether any resolved namespace import could produce a +target. If that traversal is complete and has none, return one empty map for +covered scopes instead of repeatedly resolving named imports and rebuilding +empty maps. The lexical receiver-position policy remains unchanged. Incomplete +scope views and lookups outside the traversed scopes retain the existing path. +This changes no native bindings, dependencies, public flags or graph schema. + +Each runtime and corpus has its own index and stable runner identity. Both are +on local overlay storage. The first use of the separate baseline runtime caused +an identity rebuild; that setup run is excluded. Candidate seeding is also +excluded. The timer, options and instrumentation match the method above: +Node **22.23.3**, LadybugDB/FTS **0.22.0**, Tree-sitter **0.25.1**, 16 workers, +8 GiB parent heap limit, full retention and FTS, no embeddings or PDG. CPU +profiling is disabled for every sample in the table. + +Each edit replaces `gitnexus/src/cli/update-notice.ts` with its pristine contents +plus `\n// scope-emission profile trial N\n`, for `N = 3, 4, 5`. These measure the +smallest one-file edit path. Run order is baseline 1, candidate 1, candidate 2, +baseline 2, baseline 3, candidate 3. No local test suite runs alongside them. +The host remains shared; one-minute loads recorded before and after the samples +range from **1.67 to 4.51**. + +| One-file edit | Main baseline (s) | Candidate (s) | +| ------------- | ----------------: | ------------: | +| Sample 1 | 122.120 | 111.554 | +| Sample 2 | 126.536 | 114.429 | +| Sample 3 | 124.307 | 116.258 | +| **Median** | **124.307** | **114.429** | + +The ratio of medians is **1.086×**, or **7.95% less elapsed time**. Every paired +candidate sample is faster. This is an edit-path improvement; the earlier 65× +result measures unchanged dirty repeats. + +Independent medians of the existing phase timers locate the reduction: + +| Interval | Main baseline (s) | Candidate (s) | +| --------------------------------------------------------- | ----------------: | ------------: | +| Scope resolution | 77.028 | 66.340 | +| `emit`, summed across languages (inside scope resolution) | 50.616 | 38.961 | +| Parsing | 6.995 | 6.966 | +| Graph load | 5.044 | 5.179 | +| FTS | 13.041 | 12.510 | + +All six runs reported **one changed file**, **22 reparsed / 3,089 cached files**, +six importer dependents, an incremental write, no rebuild reasons, and available +native FTS. Their reported graph counts and filtered in-memory node/relationship +hashes matched exactly under the exclusions defined above. The same default +bounded-analysis configuration applies; this is not exhaustive persisted-graph +equivalence or a claim about every repository, language mix or change-set size. + +A functional control then added two exported functions, one calling the other, +and a unique search token to the same file in both corpora. Both produced +**118,519 nodes / 356,372 relationships** and matching filtered graph hashes. +Read-only native queries returned identical function IDs and source spans, the +same File FTS hit, and all **35** Function-to-Function call edges touching the +file, including confidence and reason. The new caller-to-callee edge was present +with confidence **0.85** and reason `local-call`. This control took 138.832s on +main and 117.450s on the candidate; it is one pair and is excluded from the +comment-edit medians above. + ### Reproducing the timing boundaries Use separate runtime and corpus checkouts of the pinned commit on local storage. diff --git a/gitnexus/src/core/ingestion/scope-resolution/scope/namespace-target-cache.ts b/gitnexus/src/core/ingestion/scope-resolution/scope/namespace-target-cache.ts index 97807bfd0..5667fe903 100644 --- a/gitnexus/src/core/ingestion/scope-resolution/scope/namespace-target-cache.ts +++ b/gitnexus/src/core/ingestion/scope-resolution/scope/namespace-target-cache.ts @@ -19,6 +19,7 @@ export function createNamespaceTargetCache( const scopeIds = new Set(); const importNames = new Set(); const accessibleNamespaces = new Set(); + let hasNamespaceImports = false; const tree = scopes.scopeTree; let complete = typeof tree?.getChildren === 'function'; let requiresLexicalLookup = importsBindAtLexicalScope; @@ -49,7 +50,11 @@ export function createNamespaceTargetCache( collectBindings(scope.bindings); collectBindings(scopes.bindings?.get(id)); collectBindings(scopes.bindingAugmentations?.get(id)); - for (const edge of scopes.imports?.get(id) ?? scope.imports) importNames.add(edge.localName); + for (const edge of scopes.imports?.get(id) ?? scope.imports) { + importNames.add(edge.localName); + if (edge.kind === 'namespace' && edge.targetFile !== null && edge.linkStatus !== 'unresolved') + hasNamespaceImports = true; + } for (const namespace of scopes.accessibleNamespacesByScope?.get(id) ?? []) { accessibleNamespaces.add(namespace); } @@ -74,8 +79,12 @@ export function createNamespaceTargetCache( (point, i) => i === 0 || compare(point, boundaries[i - 1]!) !== 0, ); const byScope = new Map>>(); + const emptyTargets = new Map(); const at = (inScope = parsed.moduleScope, position?: SourcePosition): Map => { + // A complete walk can prove every lexical snapshot is empty. Keep the + // position-sensitive receiver lookup policy, but avoid rebuilding maps. + if (complete && !hasNamespaceImports && scopeIds.has(inScope)) return emptyTargets; let bucket: number | string = -1; // An unspecified position sees every activation. if (position !== undefined) { if (!complete || !scopeIds.has(inScope)) { diff --git a/gitnexus/test/unit/scope-resolution/namespace-target-cache.test.ts b/gitnexus/test/unit/scope-resolution/namespace-target-cache.test.ts index e412dc1a4..076387a6f 100644 --- a/gitnexus/test/unit/scope-resolution/namespace-target-cache.test.ts +++ b/gitnexus/test/unit/scope-resolution/namespace-target-cache.test.ts @@ -44,6 +44,71 @@ const indexes = (fileScopes: Scope[]): ScopeResolutionIndexes => const parsed = (fileScopes: Scope[]) => ({ moduleScope: 'module', scopes: fileScopes }); describe('namespace target activation cache', () => { + it('shares a proven empty namespace map without discarding lexical receiver lookup', () => { + const module = scope('module', null, { + imports: [{ ...edge('values.ts', 0, 'value'), kind: 'named' }], + nameClaims: [{ name: 'value', kind: 'import', range: range(0) }], + }); + const fn = scope('function', module.id, { + nameClaims: [{ name: 'local', kind: 'blocked', range: range(10), availableFrom: at(10) }], + }); + const fileScopes = [module, fn]; + const base = indexes(fileScopes); + const cache = createNamespaceTargetCache(parsed(fileScopes), base, {}, false); + const getScope = vi.spyOn(base.scopeTree, 'getScope'); + expect(cache.requiresLexicalLookup).toBe(true); + const empty = cache.at(); + expect(empty.size).toBe(0); + for (let line = 1; line <= 1000; line++) { + expect(cache.at(module.id, at(line))).toBe(empty); + expect(cache.at(fn.id, at(line))).toBe(empty); + } + // Once absence is proven, new receiver positions require no scope walks. + expect(getScope).not.toHaveBeenCalled(); + }); + + it.each(['memory', 'sealed'] as const)( + 'finds nested namespace imports when the module has none (%s)', + (storage) => { + const module = scope('module', null); + const fn = scope('function', module.id, { imports: [edge('inner.ts', 10)] }); + const fileScopes = [module, fn]; + const base = indexes(fileScopes); + const scopeTree = + storage === 'sealed' + ? new DiskBackedScopeTree(tmp, persistScopeShards(tmp, fileScopes)) + : base.scopeTree; + const cache = createNamespaceTargetCache(parsed([]), { ...base, scopeTree }, {}, true); + expect(cache.at().size).toBe(0); + expect(cache.at(fn.id, at(11)).get('pkg')).toEqual(['inner.ts']); + }, + ); + + it('looks up scopes outside the file even when its own namespace maps are empty', () => { + const module = scope('module', null); + const foreign = scope('foreign', null, { imports: [edge('foreign.ts', 0)] }); + const cache = createNamespaceTargetCache( + parsed([module]), + indexes([module, foreign]), + {}, + false, + ); + expect(cache.at().size).toBe(0); + expect(cache.at(foreign.id, at(1)).get('pkg')).toEqual(['foreign.ts']); + }); + + it('retains exact-position lookup when an import-free index view is incomplete', () => { + const module = scope('module', null); + const base = indexes([module]); + const scopeTree = { getScope: base.scopeTree.getScope } as ScopeResolutionIndexes['scopeTree']; + const cache = createNamespaceTargetCache(parsed([]), { ...base, scopeTree }, {}, false); + expect(cache.requiresLexicalLookup).toBe(true); + const first = cache.at(module.id, at(1)); + expect(first.size).toBe(0); + expect(cache.at(module.id, at(2))).not.toBe(first); + expect(cache.at(module.id, at(1))).toBe(first); + }); + it('collects stable imports once for a thousand distinct receiver positions', () => { const imports = Array.from({ length: 32 }, (_, i) => edge(`pkg${i}.ts`, 0, `pkg${i}`)); const module = scope('module', null, {