scriptoriumadmin/GitNexus
1
0
Fork 0
mirror of https://github.com/abhigyanpatwari/GitNexus.git synced 2026-08-28 05:25:25 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 9

fix(parse-impl): suppress chunk-cache write when any chunk file was quarantined (U20.U2)

Browse source 
The chunk hash at parse-impl.ts:424-428 is computed from every file
in the chunk. The worker pool's Layer 3 quarantine
(worker-pool.ts createQuarantine) filters quarantined files out of
dispatch, so `rawResults` reflects only the surviving files. Before
this commit, the write at line 500-507 stored that partial result
under the full-coverage chunk hash — and on the next analyze with
unchanged content, the cache HIT branch (line 439-464) silently
replayed the incomplete result. Symbols from the quarantined file
were missing from the graph for as long as the cache survived.

Codex's adversarial review of PR #1693 flagged this as a silent-
corruption class because there's no failure signal: no warn log
during the replay, no graph-equivalence check, no exit code change.
The corruption only surfaces if an operator notices a missing symbol
in `gitnexus_query` output.

Guard the write with `chunkFiles.some(f => quarantineSet.has(f.path))`.
When any chunk file is in the worker pool's cumulative quarantine
snapshot, skip the `parseCache.entries.set` call. Emits a verbose-
only info log so operators investigating "why aren't my chunks
caching" have a diagnostic trail.

Skipping the write means the next analyze gets a cache miss for this
chunk and re-dispatches it. Quarantine is session-scoped (a fresh
createWorkerPool starts with an empty quarantine), so the new pool
gives the quarantined file another chance. If quarantine fires again,
U20.U1's sequential gap-fill still produces a complete graph for that
run; the cache stays empty for the chunk until a fully-clean
dispatch lands.

The cache-hit replay branch at parse-impl.ts:439-464 is unchanged.
Its contract strengthens: "cache entries are complete" becomes true
post-fix, but the replay code doesn't need to know that.

Closes the cross-run side of the Codex finding. U20.U3 adds the
regression test.

References plan: docs/plans/2026-05-20-002-fix-chunk-cache-corruption-on-worker-quarantine-plan.md
This commit is contained in:
Gergo Magyar 2026-05-20 13:14:38 +01:00
parent 7dd489e997
commit 7c9c955645
1 changed files with 37 additions and 5 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

42
gitnexus/src/core/ingestion/pipeline-phases/parse-impl.ts
View file

@ -497,12 +497,44 @@ export async function runChunkedParseAndResolve(
// Persist the raw results for this chunk hash. Sequential path
// doesn't populate rawResults (it writes directly to graph), so
// small repos without worker pool simply don't cache. That's fine.
//
// U20.U2: refuse the write when any chunk file is in the
// worker pool's cumulative quarantine snapshot. The chunkHash
// is computed from EVERY file in the chunk, but the pool's
// Layer 3 quarantine filters quarantined files out of dispatch
// — so `rawResults` is narrower than the chunkHash key implies.
// Caching it would silently replay incomplete results on the
// next run with unchanged content (the corruption class Codex's
// adversarial review of PR #1693 flagged).
//
// Skipping the write means the next analyze gets a cache miss
// for this chunk and re-dispatches against a fresh worker pool
// (quarantine is session-scoped — `createQuarantine` is called
// per-pool at worker-pool.ts), giving the quarantined file
// another chance. If quarantine fires again, U20.U1's
// sequential gap-fill still produces a complete graph for this
// run; the cache just stays empty for this chunk until a fully-
// clean dispatch lands.
if (parseCache && chunkHash && rawResults.length > 0) {
parseCache.entries.set(chunkHash, rawResults);
if (isDev) {
logger.info(
`📦 parse-cache MISS+store: chunk ${chunkIdx + 1}/${numChunks} (${chunkFiles.length} files, ${chunkHash.slice(0, 8)})`,
);
const quarantineSnapshot = workerPool?.getQuarantinedPaths?.() ?? [];
const quarantineSet = new Set(quarantineSnapshot);
const chunkHadQuarantine = chunkFiles.some((f) => quarantineSet.has(f.path));
if (chunkHadQuarantine) {
if (isDev) {
const quarantinedInChunk = chunkFiles.filter((f) => quarantineSet.has(f.path)).length;
logger.info(
`📦 parse-cache SKIP: chunk ${chunkIdx + 1}/${numChunks} ` +
`had ${quarantinedInChunk} worker-quarantined file(s); ` +
`next run will rediscover (${chunkHash.slice(0, 8)})`,
);
}
} else {
parseCache.entries.set(chunkHash, rawResults);
if (isDev) {
logger.info(
`📦 parse-cache MISS+store: chunk ${chunkIdx + 1}/${numChunks} (${chunkFiles.length} files, ${chunkHash.slice(0, 8)})`,
);
}
}
}
}