feat(analyze): record a collapsed graph write instead of reporting fresh

The dangerous half of a broken refresh: metadata IS written, so the index
reads as fresh, hooks re-arm, and every tool answers from a graph missing
most of its edges — indistinguishable from a codebase that genuinely has no
such relationships. Reported in the field as edges collapsing 23009 -> 2170
and as a CodeRelation table that never materialized.

`analyze` now compares the relationship count the pipeline PRODUCED against
what the DB hands back after the write. Both numbers are already in scope at
the same point, so the shortfall is provable rather than inferred — no
comparison against the previous index, which cannot distinguish a failed
write from a repo that legitimately shrank. A missing relation table needs
no special case: it reads back as a persisted count of zero.

On a collapse the run records `graphWriteCollapsed` in metadata, which
`getIndexIncompleteReasons` turns into `graph-write-collapsed` so status and
the MCP resources report the index INCOMPLETE rather than fresh.

A ratio, not equality: some relationship types do not round-trip one-for-one
and `--pdg` writes MORE rows into the same table, so demanding equality
would fire on healthy runs. Only a collapse is a defect. Fail-safe when the
expected count is unavailable — an implementation that offloads
relationships out of memory may not be able to report a total, and a false
"your index is broken" is worse than a missed one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
ReidenXerx 2026-08-06 01:25:20 +03:00
parent 1e451bb8a4
commit e11bb186bc
4 changed files with 130 additions and 1 deletions

View file

@ -5,16 +5,26 @@ export const INDEX_INCOMPLETE_REASONS = [
'incremental-in-progress',
'embedding-checkpoint-pending',
'embedding-count-unverified',
'graph-write-collapsed',
] as const;
export type IndexIncompleteReason = (typeof INDEX_INCOMPLETE_REASONS)[number];
/** Stable machine-readable reasons an index cannot be certified complete. */
export function getIndexIncompleteReasons(
meta: Pick<RepoMeta, 'incrementalInProgress' | 'embeddingCheckpoint'> | null | undefined,
meta:
| Pick<RepoMeta, 'incrementalInProgress' | 'embeddingCheckpoint' | 'graphWriteCollapsed'>
| null
| undefined,
): IndexIncompleteReason[] {
const reasons: IndexIncompleteReason[] = [];
if (meta?.incrementalInProgress) reasons.push('incremental-in-progress');
// The run finished and wrote metadata, but far fewer edges reached the DB
// than the pipeline produced — the "refresh reported success, the index is
// unusable" failure. Without this the index reads as fresh and every tool
// answers from a graph missing most of its edges, which is indistinguishable
// from a codebase that genuinely has no such relationships.
if (meta?.graphWriteCollapsed) reasons.push('graph-write-collapsed');
if (meta?.embeddingCheckpoint) {
// The three checkpoint kinds are not one operator-facing state. GUARDRAILS
// and the runbook document `embedding-checkpoint-pending` as "N node(s)

View file

@ -502,6 +502,20 @@ export interface AnalyzeResult {
// Class-neutral lead, reused for the missing-dependency degrade path (#2383 F2):
// its remedy already explains that reinstalling will NOT help, so appending the
// generic "install with network access" tail below would contradict it.
/**
* Fraction of the pipeline's relationship count that must survive into the DB
* before the write is treated as a collapse. Deliberately generous: this is a
* catastrophe detector for "most of the graph did not persist" (the reported
* case lost ~91%), not a reconciliation of every edge.
*/
const GRAPH_WRITE_COLLAPSE_RATIO = 0.5;
/**
* Below this many relationships the ratio is meaningless — a handful of edges
* lost to legitimate filtering would trip it — so small repos are exempt.
*/
const GRAPH_WRITE_COLLAPSE_MIN_EDGES = 100;
const FTS_UNAVAILABLE_LEAD = 'FTS extension unavailable; skipping search-index creation.';
const FTS_UNAVAILABLE_MESSAGE =
`${FTS_UNAVAILABLE_LEAD} ` +
@ -2520,6 +2534,37 @@ async function runFullAnalysisInner(
// ── Phase 4: Embeddings (90–98%) ──────────────────────────────────
const stats = await getLbugStats();
// Post-write integrity: the pipeline knows exactly how many relationships
// it produced, and `stats` is what the DB hands back after the write, so a
// large shortfall is provable rather than inferred — no comparison against
// the previous index needed. This is the guard for a refresh that reports
// SUCCESS while leaving the index unusable: edges collapsing to a fraction
// of what was built, or a `CodeRelation` table that never materialized
// (which surfaces here as a persisted count of zero).
//
// A RATIO, not equality: some relationship types legitimately do not round
// -trip one-for-one, and `--pdg` writes MORE rows into the same table than
// the call-graph produced, so demanding equality would fire on healthy
// runs. Only a collapse is a defect.
//
// Fail-safe when `expected` reads 0: an implementation that offloads
// relationships out of memory may no longer be able to report a total, and
// a false "your index is broken" is worse than a missed one.
const expectedRelationships = pipelineResult.graph.relationshipCount;
const graphWriteCollapsed =
expectedRelationships >= GRAPH_WRITE_COLLAPSE_MIN_EDGES &&
stats.edges < expectedRelationships * GRAPH_WRITE_COLLAPSE_RATIO
? { expected: expectedRelationships, persisted: stats.edges }
: undefined;
if (graphWriteCollapsed) {
log(
`Warning: graph write incomplete — the pipeline produced ${expectedRelationships} ` +
`relationships but only ${stats.edges} are readable from the index. Recording the ` +
`index as INCOMPLETE (graph-write-collapsed) rather than fresh; re-run ` +
`\`gitnexus analyze --force\`.`,
);
}
let embeddingSkipped = true;
let semanticMode: 'vector-index' | 'exact-scan' | undefined;
// Hoisted out of the Phase 4 block so the Phase 5 gate can tell "the
@ -2929,6 +2974,9 @@ async function runFullAnalysisInner(
// origin remote, which is fine: paths-only repos behave as
// before.
remoteUrl: hasGitDir(repoPath) ? getRemoteUrl(repoPath) : undefined,
// Absent on a healthy run; present it and the index reports as
// incomplete rather than fresh (`graph-write-collapsed`).
...(graphWriteCollapsed ? { graphWriteCollapsed } : {}),
stats: {
files: pipelineResult.totalFileCount,
nodes: stats.nodes,

View file

@ -312,6 +312,25 @@ export interface RepoMeta {
* run sees the flag and forces a full rebuild — the cheapest path back
* to a known-good index.
*/
/**
* Set when a run finished but the persisted edge count came back far short
* of what the pipeline produced — the B2 "refresh reports SUCCESS while the
* index is unusable" failure (observed as edges collapsing 23009 -> 2170,
* and as a missing `CodeRelation` table, which reads here as a persisted
* count of zero).
*
* Recorded rather than thrown because the metadata IS written and the DB
* does hold rows; what is false is the claim that the index is complete.
* `getIndexIncompleteReasons` turns this into `graph-write-collapsed` so
* `status` and the MCP resources report the index as incomplete instead of
* fresh. Absent on a healthy run.
*/
graphWriteCollapsed?: {
/** Relationships the pipeline produced in memory. */
expected: number;
/** Relationships readable from the DB after the write. */
persisted: number;
};
incrementalInProgress?: {
/** When the run started (epoch ms). */
startedAt: number;

View file

@ -0,0 +1,52 @@
/**
* B2 — a refresh that reports SUCCESS while leaving the index unusable.
*
* The dangerous variant of a broken refresh: metadata IS written, so the index
* reads as fresh, hooks re-arm, and every tool answers from a graph missing
* most of its edges — which is indistinguishable from a codebase that
* genuinely has no such relationships. Observed as edges collapsing
* 23009 -> 2170, and as a `CodeRelation` table that never materialized (which
* reads back as a persisted count of zero).
*
* `analyze` compares the relationship count the pipeline produced against what
* the DB hands back after the write and records `graphWriteCollapsed`. This
* covers the translation of that record into the operator-facing reason, which
* is what `status` and the MCP resources report.
*/
import { describe, it, expect } from 'vitest';
import {
getIndexIncompleteReasons,
INDEX_INCOMPLETE_REASONS,
} from '../../src/core/index-freshness.js';
describe('graph-write-collapsed incomplete reason (B2)', () => {
it('is part of the stable reason vocabulary', () => {
expect(INDEX_INCOMPLETE_REASONS).toContain('graph-write-collapsed');
});
it('reports a collapsed write as incomplete rather than fresh', () => {
expect(
getIndexIncompleteReasons({ graphWriteCollapsed: { expected: 23009, persisted: 2170 } }),
).toEqual(['graph-write-collapsed']);
});
it('treats a missing relation table (zero persisted) the same way', () => {
expect(
getIndexIncompleteReasons({ graphWriteCollapsed: { expected: 23009, persisted: 0 } }),
).toEqual(['graph-write-collapsed']);
});
it('says nothing on a healthy run', () => {
expect(getIndexIncompleteReasons({})).toEqual([]);
expect(getIndexIncompleteReasons(null)).toEqual([]);
});
it('reports alongside other reasons rather than masking them', () => {
const reasons = getIndexIncompleteReasons({
incrementalInProgress: { startedAt: 1, toWriteCount: 0 },
graphWriteCollapsed: { expected: 500, persisted: 10 },
});
expect(reasons).toContain('incremental-in-progress');
expect(reasons).toContain('graph-write-collapsed');
});
});