perf(scope): reuse proven empty namespace maps (#3798)

Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
This commit is contained in:
Gergő Magyar 2026-11-01 16:48:35 +00:00 • committed by GitHub
parent 626b607878
commit 347cc99b4a
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
3 changed files with 154 additions and 3 deletions

View file

@ -1,7 +1,7 @@
# Analyze phase breakdown — where the time actually goes
Updated 2026-11-01 with production hardening and measurements from main on
LadybugDB **0.22.0** and an experimental fast path for unchanged dirty checkouts.
Updated 2026-11-01 with scope-emission measurements after #3796 merged, plus
production-hardening measurements on LadybugDB **0.22.0** for unchanged dirty checkouts.
Records what the `analyze`
pipeline costs per phase and the results of optimization experiments.
The September measurements remain below as historical
@ -269,6 +269,83 @@ was 73.939s on main and 91.838s on the candidate, accounting for most of the
establish its cause. Host load changed and this pair has one sample per
variant: neither a real-edit speedup nor a stable regression is established.
### Scope-emission experiment after #3796 — 2026-11-01
Baseline runtime and corpus: merged main at
`568ebdc1d9e6fce80a4da306bcb1ecebb20b60e4`. This includes the dirty-snapshot
shortcut and the intervening Java overload changes. The corpus now has **3,375
indexed files**, **3,111 parseable files**, **118,517 nodes / 356,369 relationships /
3,495 communities / 767 processes**. These measurements form a new paired
comparison; the earlier corpus timings are not its baseline.
A separate Node CPU profile of a one-file edit identified repeated namespace
lookups inside receiver-bound call emission. The main-thread profile recorded
**29.6s in garbage collection**, **21.7s inclusive in `emitReceiverBoundCalls`**,
and **6.2s inclusive in `collectNamespaceTargets`**. Those intervals overlap and
must not be added. The profile includes startup and post-run graph hashing;
its **140.158s** analysis run is excluded from the timing comparison below.
The candidate adds a proof to `createNamespaceTargetCache`: while traversing a
file's scopes, record whether any resolved namespace import could produce a
target. If that traversal is complete and has none, return one empty map for
covered scopes instead of repeatedly resolving named imports and rebuilding
empty maps. The lexical receiver-position policy remains unchanged. Incomplete
scope views and lookups outside the traversed scopes retain the existing path.
This changes no native bindings, dependencies, public flags or graph schema.
Each runtime and corpus has its own index and stable runner identity. Both are
on local overlay storage. The first use of the separate baseline runtime caused
an identity rebuild; that setup run is excluded. Candidate seeding is also
excluded. The timer, options and instrumentation match the method above:
Node **22.23.3**, LadybugDB/FTS **0.22.0**, Tree-sitter **0.25.1**, 16 workers,
8 GiB parent heap limit, full retention and FTS, no embeddings or PDG. CPU
profiling is disabled for every sample in the table.
Each edit replaces `gitnexus/src/cli/update-notice.ts` with its pristine contents
plus `\n// scope-emission profile trial N\n`, for `N = 3, 4, 5`. These measure the
smallest one-file edit path. Run order is baseline 1, candidate 1, candidate 2,
baseline 2, baseline 3, candidate 3. No local test suite runs alongside them.
The host remains shared; one-minute loads recorded before and after the samples
range from **1.67 to 4.51**.
| One-file edit | Main baseline (s) | Candidate (s) |
| ------------- | ----------------: | ------------: |
| Sample 1 | 122.120 | 111.554 |
| Sample 2 | 126.536 | 114.429 |
| Sample 3 | 124.307 | 116.258 |
| **Median** | **124.307** | **114.429** |
The ratio of medians is **1.086×**, or **7.95% less elapsed time**. Every paired
candidate sample is faster. This is an edit-path improvement; the earlier 65×
result measures unchanged dirty repeats.
Independent medians of the existing phase timers locate the reduction:
| Interval | Main baseline (s) | Candidate (s) |
| --------------------------------------------------------- | ----------------: | ------------: |
| Scope resolution | 77.028 | 66.340 |
| `emit`, summed across languages (inside scope resolution) | 50.616 | 38.961 |
| Parsing | 6.995 | 6.966 |
| Graph load | 5.044 | 5.179 |
| FTS | 13.041 | 12.510 |
All six runs reported **one changed file**, **22 reparsed / 3,089 cached files**,
six importer dependents, an incremental write, no rebuild reasons, and available
native FTS. Their reported graph counts and filtered in-memory node/relationship
hashes matched exactly under the exclusions defined above. The same default
bounded-analysis configuration applies; this is not exhaustive persisted-graph
equivalence or a claim about every repository, language mix or change-set size.
A functional control then added two exported functions, one calling the other,
and a unique search token to the same file in both corpora. Both produced
**118,519 nodes / 356,372 relationships** and matching filtered graph hashes.
Read-only native queries returned identical function IDs and source spans, the
same File FTS hit, and all **35** Function-to-Function call edges touching the
file, including confidence and reason. The new caller-to-callee edge was present
with confidence **0.85** and reason `local-call`. This control took 138.832s on
main and 117.450s on the candidate; it is one pair and is excluded from the
comment-edit medians above.
### Reproducing the timing boundaries
Use separate runtime and corpus checkouts of the pinned commit on local storage.

View file

@ -19,6 +19,7 @@ export function createNamespaceTargetCache(
const scopeIds = new Set<ScopeId>();
const importNames = new Set<string>();
const accessibleNamespaces = new Set<string>();
let hasNamespaceImports = false;
const tree = scopes.scopeTree;
let complete = typeof tree?.getChildren === 'function';
let requiresLexicalLookup = importsBindAtLexicalScope;
@ -49,7 +50,11 @@ export function createNamespaceTargetCache(
collectBindings(scope.bindings);
collectBindings(scopes.bindings?.get(id));
collectBindings(scopes.bindingAugmentations?.get(id));
for (const edge of scopes.imports?.get(id) ?? scope.imports) importNames.add(edge.localName);
for (const edge of scopes.imports?.get(id) ?? scope.imports) {
importNames.add(edge.localName);
if (edge.kind === 'namespace' && edge.targetFile !== null && edge.linkStatus !== 'unresolved')
hasNamespaceImports = true;
}
for (const namespace of scopes.accessibleNamespacesByScope?.get(id) ?? []) {
accessibleNamespaces.add(namespace);
}
@ -74,8 +79,12 @@ export function createNamespaceTargetCache(
(point, i) => i === 0 || compare(point, boundaries[i - 1]!) !== 0,
);
const byScope = new Map<ScopeId, Map<number | string, Map<string, string[]>>>();
const emptyTargets = new Map<string, string[]>();
const at = (inScope = parsed.moduleScope, position?: SourcePosition): Map<string, string[]> => {
// A complete walk can prove every lexical snapshot is empty. Keep the
// position-sensitive receiver lookup policy, but avoid rebuilding maps.
if (complete && !hasNamespaceImports && scopeIds.has(inScope)) return emptyTargets;
let bucket: number | string = -1; // An unspecified position sees every activation.
if (position !== undefined) {
if (!complete || !scopeIds.has(inScope)) {

View file

@ -44,6 +44,71 @@ const indexes = (fileScopes: Scope[]): ScopeResolutionIndexes =>
const parsed = (fileScopes: Scope[]) => ({ moduleScope: 'module', scopes: fileScopes });
describe('namespace target activation cache', () => {
it('shares a proven empty namespace map without discarding lexical receiver lookup', () => {
const module = scope('module', null, {
imports: [{ ...edge('values.ts', 0, 'value'), kind: 'named' }],
nameClaims: [{ name: 'value', kind: 'import', range: range(0) }],
});
const fn = scope('function', module.id, {
nameClaims: [{ name: 'local', kind: 'blocked', range: range(10), availableFrom: at(10) }],
});
const fileScopes = [module, fn];
const base = indexes(fileScopes);
const cache = createNamespaceTargetCache(parsed(fileScopes), base, {}, false);
const getScope = vi.spyOn(base.scopeTree, 'getScope');
expect(cache.requiresLexicalLookup).toBe(true);
const empty = cache.at();
expect(empty.size).toBe(0);
for (let line = 1; line <= 1000; line++) {
expect(cache.at(module.id, at(line))).toBe(empty);
expect(cache.at(fn.id, at(line))).toBe(empty);
}
// Once absence is proven, new receiver positions require no scope walks.
expect(getScope).not.toHaveBeenCalled();
});
it.each(['memory', 'sealed'] as const)(
'finds nested namespace imports when the module has none (%s)',
(storage) => {
const module = scope('module', null);
const fn = scope('function', module.id, { imports: [edge('inner.ts', 10)] });
const fileScopes = [module, fn];
const base = indexes(fileScopes);
const scopeTree =
storage === 'sealed'
? new DiskBackedScopeTree(tmp, persistScopeShards(tmp, fileScopes))
: base.scopeTree;
const cache = createNamespaceTargetCache(parsed([]), { ...base, scopeTree }, {}, true);
expect(cache.at().size).toBe(0);
expect(cache.at(fn.id, at(11)).get('pkg')).toEqual(['inner.ts']);
},
);
it('looks up scopes outside the file even when its own namespace maps are empty', () => {
const module = scope('module', null);
const foreign = scope('foreign', null, { imports: [edge('foreign.ts', 0)] });
const cache = createNamespaceTargetCache(
parsed([module]),
indexes([module, foreign]),
{},
false,
);
expect(cache.at().size).toBe(0);
expect(cache.at(foreign.id, at(1)).get('pkg')).toEqual(['foreign.ts']);
});
it('retains exact-position lookup when an import-free index view is incomplete', () => {
const module = scope('module', null);
const base = indexes([module]);
const scopeTree = { getScope: base.scopeTree.getScope } as ScopeResolutionIndexes['scopeTree'];
const cache = createNamespaceTargetCache(parsed([]), { ...base, scopeTree }, {}, false);
expect(cache.requiresLexicalLookup).toBe(true);
const first = cache.at(module.id, at(1));
expect(first.size).toBe(0);
expect(cache.at(module.id, at(2))).not.toBe(first);
expect(cache.at(module.id, at(1))).toBe(first);
});
it('collects stable imports once for a thousand distinct receiver positions', () => {
const imports = Array.from({ length: 32 }, (_, i) => edge(`pkg${i}.ts`, 0, `pkg${i}`));
const module = scope('module', null, {