GitNexus/gitnexus/test/unit/incremental-derived-writeback.test.ts
Gergő Magyar 94f67d79d5
fix(analyze): make incremental analyze skip the derived layers it can reuse (#3016) (#3102)
* fix(analyze): make incremental analyze skip the derived layers it can reuse (#3016)

A warm incremental run only ever wrote a handful of files, but it still
paid for the whole graph on the way out: Leiden ran over every node,
flow extraction re-derived every process, and all FTS indexes were
dropped and rebuilt from scratch. On a small edit that tail dominated
the run, which is why "incremental" did not feel incremental.

Reuse what the previous run already derived when the write plan allows
it. The pipeline holds back community detection and flow extraction
whenever the persisted metadata says this run is a candidate for a
surgical write; the DB keeps its Community/Process rows instead of a
wipe-and-rewrite; and the FTS sweep is narrowed to the indexes the run
actually has to touch.

The bet is placed before the pipeline and settled after it. Any plan
that turns out to need a freshly derived layer — full rebuild, escalated
write, or an incremental diff with deleted files — runs the held-back
phases through `runDeferredDerivedPhases`, against the same graph and
phase outputs, so its output is identical to never having skipped them.

Correctness details worth naming, since each one silently loses data if
got wrong:

- The MEMBER_OF / STEP_IN_PROCESS edges of the changed files are snapshotted
  before the DETACH DELETE and reattached after the subgraph load. Both
  endpoints are matched by explicit label: `labels(n)[0]` over an unlabelled
  match returns an empty string on this engine, which produced a snapshot
  that restored nothing.
- The FTS narrowing unions three sets — what the writeback deletes (a DB
  probe, because a symbol the edit removed is in no fresh graph but is
  still a row), what it inserts (the fresh graph), and what is missing
  right now (else a prior escalation's dropped indexes would never come
  back). An unreadable index catalog withdraws the narrowing entirely.
- Deletions disqualify reuse outright: persisted derived rows can reference
  nodes this run removes, and nothing short of re-deriving can tell which.

Covered by the existing incremental suites, including the
incremental-equals-force byte-equivalence test and the #2589
drop-before-delete ordering test, plus unit tests for the new helpers.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(analyze): address #3102 review on derived reuse and FTS narrowing

Re-run Leiden/flows unless the file-hash diff is empty, restore ENTRY_POINT_OF
on the preserve path, always drop class_fts before Spring synthetic Class DML,
and reject seeded duplicate phase names. Prettier and exact FTS drop-ordering
assertions unblock CI and pin the #2589/#3016 contract.

Co-authored-by: Cursor <cursoragent@cursor.com>

* refactor(analyze): reuse FileHashDiff for derived-layer preserve

Drop the count DTO, share phase-name uniqueness, and remove the File
FTS sentinel that Class already makes unreachable.

Refs #3102

Co-authored-by: Cursor <cursoragent@cursor.com>

* style(analyze): prettier-wrap shouldPreservePersistedDerivedGraph

quality / format failed on the Pick<FileHashDiff> signature wrapping.

Refs #3102

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-30 17:07:57 +00:00

98 lines
3.8 KiB
TypeScript

import { describe, expect, it } from 'vitest';
import type { GraphNode } from 'gitnexus-shared';
import { NODE_TABLES } from 'gitnexus-shared';
import { createKnowledgeGraph } from '../../src/core/graph/graph.js';
import {
ftsTablesAmong,
incrementalFtsTablesFromGraph,
nodeTablesForIncrementalDelete,
shouldPreservePersistedDerivedGraph,
} from '../../src/core/incremental/derived-writeback.js';
const node = (id: string, label: string, filePath: string): GraphNode =>
({
id,
label,
properties: { filePath, name: id },
}) as unknown as GraphNode;
describe('shouldPreservePersistedDerivedGraph (#3016)', () => {
const empty = { deleted: [] as string[], added: [] as string[], changed: [] as string[] };
it('is true only when the file-hash diff is empty', () => {
expect(shouldPreservePersistedDerivedGraph(empty)).toBe(true);
});
it('is false when any file was deleted (old labels are not in the fresh graph)', () => {
expect(shouldPreservePersistedDerivedGraph({ ...empty, deleted: ['gone.ts'] })).toBe(false);
});
it('is false when a file was added (new symbols have no persisted membership)', () => {
expect(shouldPreservePersistedDerivedGraph({ ...empty, added: ['new.ts'] })).toBe(false);
});
it('is false when a file changed (in-file add/rename/CALLS can change Leiden/flows)', () => {
expect(shouldPreservePersistedDerivedGraph({ ...empty, changed: ['a.ts'] })).toBe(false);
});
});
describe('incrementalFtsTablesFromGraph', () => {
it('returns only FTS tables that have write-set nodes', () => {
const g = createKnowledgeGraph();
g.addNode(node('f', 'File', 'a.ts'));
g.addNode(node('fn', 'Function', 'a.ts'));
g.addNode(node('tr', 'Trait', 'b.rs'));
const touched = incrementalFtsTablesFromGraph(g, new Set(['a.ts']));
expect([...touched].sort()).toEqual(['File', 'Function']);
});
it('ignores labels that are not FTS-indexed', () => {
const g = createKnowledgeGraph();
g.addNode(node('folder', 'Folder', 'src'));
const touched = incrementalFtsTablesFromGraph(g, new Set(['src']));
expect(touched.size).toBe(0);
});
it('cannot see a table whose last row the edit removed — hence the DB probe', () => {
// The graph is what the run WILL write. A trait deleted by this edit is
// absent here but still a row in the index, so on its own this answer
// would leave that row behind with a live index over it. run-analyze
// unions this with nodeTablesWithRowsForFiles for exactly that reason.
const g = createKnowledgeGraph();
g.addNode(node('f', 'File', 'a.rs'));
const touched = incrementalFtsTablesFromGraph(g, new Set(['a.rs']));
expect(touched.has('Trait')).toBe(false);
});
});
describe('ftsTablesAmong', () => {
it('keeps the FTS-backed tables and drops the rest', () => {
expect([...ftsTablesAmong(['File', 'Folder', 'Function'])].sort()).toEqual([
'File',
'Function',
]);
});
it('is empty for a probe that found only non-indexed tables', () => {
expect(ftsTablesAmong(['Folder']).size).toBe(0);
});
});
describe('nodeTablesForIncrementalDelete', () => {
it('keeps the FTS tables being rebuilt and drops the rest from the delete', () => {
const tables = nodeTablesForIncrementalDelete(NODE_TABLES, new Set(['File', 'Function']));
expect(tables).toContain('File');
expect(tables).toContain('Function');
expect(tables).not.toContain('Trait');
});
it('never withholds a non-FTS table, whatever is being rebuilt', () => {
const tables = nodeTablesForIncrementalDelete(NODE_TABLES, new Set(['File']));
expect(tables).toContain('Folder');
});
it('targets every FTS table when every FTS index is being rebuilt', () => {
const tables = nodeTablesForIncrementalDelete(NODE_TABLES, new Set(NODE_TABLES));
expect(tables).toEqual([...NODE_TABLES]);
});
});