mirror of
https://github.com/abhigyanpatwari/GitNexus.git
synced 2026-11-11 08:37:35 +00:00
perf(scope): reuse proven empty namespace maps (#3798)
Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
This commit is contained in:
parent
626b607878
commit
347cc99b4a
3 changed files with 154 additions and 3 deletions
|
|
@ -1,7 +1,7 @@
|
|||
# Analyze phase breakdown — where the time actually goes
|
||||
|
||||
Updated 2026-11-01 with production hardening and measurements from main on
|
||||
LadybugDB **0.22.0** and an experimental fast path for unchanged dirty checkouts.
|
||||
Updated 2026-11-01 with scope-emission measurements after #3796 merged, plus
|
||||
production-hardening measurements on LadybugDB **0.22.0** for unchanged dirty checkouts.
|
||||
Records what the `analyze`
|
||||
pipeline costs per phase and the results of optimization experiments.
|
||||
The September measurements remain below as historical
|
||||
|
|
@ -269,6 +269,83 @@ was 73.939s on main and 91.838s on the candidate, accounting for most of the
|
|||
establish its cause. Host load changed and this pair has one sample per
|
||||
variant: neither a real-edit speedup nor a stable regression is established.
|
||||
|
||||
### Scope-emission experiment after #3796 — 2026-11-01
|
||||
|
||||
Baseline runtime and corpus: merged main at
|
||||
`568ebdc1d9e6fce80a4da306bcb1ecebb20b60e4`. This includes the dirty-snapshot
|
||||
shortcut and the intervening Java overload changes. The corpus now has **3,375
|
||||
indexed files**, **3,111 parseable files**, **118,517 nodes / 356,369 relationships /
|
||||
3,495 communities / 767 processes**. These measurements form a new paired
|
||||
comparison; the earlier corpus timings are not its baseline.
|
||||
|
||||
A separate Node CPU profile of a one-file edit identified repeated namespace
|
||||
lookups inside receiver-bound call emission. The main-thread profile recorded
|
||||
**29.6s in garbage collection**, **21.7s inclusive in `emitReceiverBoundCalls`**,
|
||||
and **6.2s inclusive in `collectNamespaceTargets`**. Those intervals overlap and
|
||||
must not be added. The profile includes startup and post-run graph hashing;
|
||||
its **140.158s** analysis run is excluded from the timing comparison below.
|
||||
|
||||
The candidate adds a proof to `createNamespaceTargetCache`: while traversing a
|
||||
file's scopes, record whether any resolved namespace import could produce a
|
||||
target. If that traversal is complete and has none, return one empty map for
|
||||
covered scopes instead of repeatedly resolving named imports and rebuilding
|
||||
empty maps. The lexical receiver-position policy remains unchanged. Incomplete
|
||||
scope views and lookups outside the traversed scopes retain the existing path.
|
||||
This changes no native bindings, dependencies, public flags or graph schema.
|
||||
|
||||
Each runtime and corpus has its own index and stable runner identity. Both are
|
||||
on local overlay storage. The first use of the separate baseline runtime caused
|
||||
an identity rebuild; that setup run is excluded. Candidate seeding is also
|
||||
excluded. The timer, options and instrumentation match the method above:
|
||||
Node **22.23.3**, LadybugDB/FTS **0.22.0**, Tree-sitter **0.25.1**, 16 workers,
|
||||
8 GiB parent heap limit, full retention and FTS, no embeddings or PDG. CPU
|
||||
profiling is disabled for every sample in the table.
|
||||
|
||||
Each edit replaces `gitnexus/src/cli/update-notice.ts` with its pristine contents
|
||||
plus `\n// scope-emission profile trial N\n`, for `N = 3, 4, 5`. These measure the
|
||||
smallest one-file edit path. Run order is baseline 1, candidate 1, candidate 2,
|
||||
baseline 2, baseline 3, candidate 3. No local test suite runs alongside them.
|
||||
The host remains shared; one-minute loads recorded before and after the samples
|
||||
range from **1.67 to 4.51**.
|
||||
|
||||
| One-file edit | Main baseline (s) | Candidate (s) |
|
||||
| ------------- | ----------------: | ------------: |
|
||||
| Sample 1 | 122.120 | 111.554 |
|
||||
| Sample 2 | 126.536 | 114.429 |
|
||||
| Sample 3 | 124.307 | 116.258 |
|
||||
| **Median** | **124.307** | **114.429** |
|
||||
|
||||
The ratio of medians is **1.086×**, or **7.95% less elapsed time**. Every paired
|
||||
candidate sample is faster. This is an edit-path improvement; the earlier 65×
|
||||
result measures unchanged dirty repeats.
|
||||
|
||||
Independent medians of the existing phase timers locate the reduction:
|
||||
|
||||
| Interval | Main baseline (s) | Candidate (s) |
|
||||
| --------------------------------------------------------- | ----------------: | ------------: |
|
||||
| Scope resolution | 77.028 | 66.340 |
|
||||
| `emit`, summed across languages (inside scope resolution) | 50.616 | 38.961 |
|
||||
| Parsing | 6.995 | 6.966 |
|
||||
| Graph load | 5.044 | 5.179 |
|
||||
| FTS | 13.041 | 12.510 |
|
||||
|
||||
All six runs reported **one changed file**, **22 reparsed / 3,089 cached files**,
|
||||
six importer dependents, an incremental write, no rebuild reasons, and available
|
||||
native FTS. Their reported graph counts and filtered in-memory node/relationship
|
||||
hashes matched exactly under the exclusions defined above. The same default
|
||||
bounded-analysis configuration applies; this is not exhaustive persisted-graph
|
||||
equivalence or a claim about every repository, language mix or change-set size.
|
||||
|
||||
A functional control then added two exported functions, one calling the other,
|
||||
and a unique search token to the same file in both corpora. Both produced
|
||||
**118,519 nodes / 356,372 relationships** and matching filtered graph hashes.
|
||||
Read-only native queries returned identical function IDs and source spans, the
|
||||
same File FTS hit, and all **35** Function-to-Function call edges touching the
|
||||
file, including confidence and reason. The new caller-to-callee edge was present
|
||||
with confidence **0.85** and reason `local-call`. This control took 138.832s on
|
||||
main and 117.450s on the candidate; it is one pair and is excluded from the
|
||||
comment-edit medians above.
|
||||
|
||||
### Reproducing the timing boundaries
|
||||
|
||||
Use separate runtime and corpus checkouts of the pinned commit on local storage.
|
||||
|
|
|
|||
|
|
@ -19,6 +19,7 @@ export function createNamespaceTargetCache(
|
|||
const scopeIds = new Set<ScopeId>();
|
||||
const importNames = new Set<string>();
|
||||
const accessibleNamespaces = new Set<string>();
|
||||
let hasNamespaceImports = false;
|
||||
const tree = scopes.scopeTree;
|
||||
let complete = typeof tree?.getChildren === 'function';
|
||||
let requiresLexicalLookup = importsBindAtLexicalScope;
|
||||
|
|
@ -49,7 +50,11 @@ export function createNamespaceTargetCache(
|
|||
collectBindings(scope.bindings);
|
||||
collectBindings(scopes.bindings?.get(id));
|
||||
collectBindings(scopes.bindingAugmentations?.get(id));
|
||||
for (const edge of scopes.imports?.get(id) ?? scope.imports) importNames.add(edge.localName);
|
||||
for (const edge of scopes.imports?.get(id) ?? scope.imports) {
|
||||
importNames.add(edge.localName);
|
||||
if (edge.kind === 'namespace' && edge.targetFile !== null && edge.linkStatus !== 'unresolved')
|
||||
hasNamespaceImports = true;
|
||||
}
|
||||
for (const namespace of scopes.accessibleNamespacesByScope?.get(id) ?? []) {
|
||||
accessibleNamespaces.add(namespace);
|
||||
}
|
||||
|
|
@ -74,8 +79,12 @@ export function createNamespaceTargetCache(
|
|||
(point, i) => i === 0 || compare(point, boundaries[i - 1]!) !== 0,
|
||||
);
|
||||
const byScope = new Map<ScopeId, Map<number | string, Map<string, string[]>>>();
|
||||
const emptyTargets = new Map<string, string[]>();
|
||||
|
||||
const at = (inScope = parsed.moduleScope, position?: SourcePosition): Map<string, string[]> => {
|
||||
// A complete walk can prove every lexical snapshot is empty. Keep the
|
||||
// position-sensitive receiver lookup policy, but avoid rebuilding maps.
|
||||
if (complete && !hasNamespaceImports && scopeIds.has(inScope)) return emptyTargets;
|
||||
let bucket: number | string = -1; // An unspecified position sees every activation.
|
||||
if (position !== undefined) {
|
||||
if (!complete || !scopeIds.has(inScope)) {
|
||||
|
|
|
|||
|
|
@ -44,6 +44,71 @@ const indexes = (fileScopes: Scope[]): ScopeResolutionIndexes =>
|
|||
const parsed = (fileScopes: Scope[]) => ({ moduleScope: 'module', scopes: fileScopes });
|
||||
|
||||
describe('namespace target activation cache', () => {
|
||||
it('shares a proven empty namespace map without discarding lexical receiver lookup', () => {
|
||||
const module = scope('module', null, {
|
||||
imports: [{ ...edge('values.ts', 0, 'value'), kind: 'named' }],
|
||||
nameClaims: [{ name: 'value', kind: 'import', range: range(0) }],
|
||||
});
|
||||
const fn = scope('function', module.id, {
|
||||
nameClaims: [{ name: 'local', kind: 'blocked', range: range(10), availableFrom: at(10) }],
|
||||
});
|
||||
const fileScopes = [module, fn];
|
||||
const base = indexes(fileScopes);
|
||||
const cache = createNamespaceTargetCache(parsed(fileScopes), base, {}, false);
|
||||
const getScope = vi.spyOn(base.scopeTree, 'getScope');
|
||||
expect(cache.requiresLexicalLookup).toBe(true);
|
||||
const empty = cache.at();
|
||||
expect(empty.size).toBe(0);
|
||||
for (let line = 1; line <= 1000; line++) {
|
||||
expect(cache.at(module.id, at(line))).toBe(empty);
|
||||
expect(cache.at(fn.id, at(line))).toBe(empty);
|
||||
}
|
||||
// Once absence is proven, new receiver positions require no scope walks.
|
||||
expect(getScope).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it.each(['memory', 'sealed'] as const)(
|
||||
'finds nested namespace imports when the module has none (%s)',
|
||||
(storage) => {
|
||||
const module = scope('module', null);
|
||||
const fn = scope('function', module.id, { imports: [edge('inner.ts', 10)] });
|
||||
const fileScopes = [module, fn];
|
||||
const base = indexes(fileScopes);
|
||||
const scopeTree =
|
||||
storage === 'sealed'
|
||||
? new DiskBackedScopeTree(tmp, persistScopeShards(tmp, fileScopes))
|
||||
: base.scopeTree;
|
||||
const cache = createNamespaceTargetCache(parsed([]), { ...base, scopeTree }, {}, true);
|
||||
expect(cache.at().size).toBe(0);
|
||||
expect(cache.at(fn.id, at(11)).get('pkg')).toEqual(['inner.ts']);
|
||||
},
|
||||
);
|
||||
|
||||
it('looks up scopes outside the file even when its own namespace maps are empty', () => {
|
||||
const module = scope('module', null);
|
||||
const foreign = scope('foreign', null, { imports: [edge('foreign.ts', 0)] });
|
||||
const cache = createNamespaceTargetCache(
|
||||
parsed([module]),
|
||||
indexes([module, foreign]),
|
||||
{},
|
||||
false,
|
||||
);
|
||||
expect(cache.at().size).toBe(0);
|
||||
expect(cache.at(foreign.id, at(1)).get('pkg')).toEqual(['foreign.ts']);
|
||||
});
|
||||
|
||||
it('retains exact-position lookup when an import-free index view is incomplete', () => {
|
||||
const module = scope('module', null);
|
||||
const base = indexes([module]);
|
||||
const scopeTree = { getScope: base.scopeTree.getScope } as ScopeResolutionIndexes['scopeTree'];
|
||||
const cache = createNamespaceTargetCache(parsed([]), { ...base, scopeTree }, {}, false);
|
||||
expect(cache.requiresLexicalLookup).toBe(true);
|
||||
const first = cache.at(module.id, at(1));
|
||||
expect(first.size).toBe(0);
|
||||
expect(cache.at(module.id, at(2))).not.toBe(first);
|
||||
expect(cache.at(module.id, at(1))).toBe(first);
|
||||
});
|
||||
|
||||
it('collects stable imports once for a thousand distinct receiver positions', () => {
|
||||
const imports = Array.from({ length: 32 }, (_, i) => edge(`pkg${i}.ts`, 0, `pkg${i}`));
|
||||
const module = scope('module', null, {
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue