mirror of
https://github.com/abhigyanpatwari/GitNexus.git
synced 2026-09-15 23:32:49 +00:00
ceaff27c1e
* fix(parse-cache): retire a chunk whose durable generation could not be reset #3200 skipped the parse-cache write when `prepareDurableParsedFileChunk` failed, but the chunk hash was already in `usedKeys` from the lookup. When a previous generation existed on disk — reachable because the coherence gate re-dispatches a chunk whose `.v8` shard is live but whose durable shards are unreadable — `saveParseCache` copied that old shard forward, and the durable prune, which keeps exactly the saved keys, retained the mixed directory. The next run then served a warm hit out of a directory the previous run had already decided it could not account for. Retire the hash instead of only skipping the write: - `ParseCache.staleKeys` is a transient set that `saveParseCache` filters out of its key list. Filtering at save is what makes it survive the post-parse key merges in run-analyze (#2106 sibling fold, unreadable-meta retention), and it reaches both stores at once because the durable prune keeps exactly the keys `saveParseCache` returns. - The hash is retired at the reset-failure site, which runs unconditionally. The parse-cache write branch sits behind `rawResults.length > 0`, so a chunk whose worker round returns nothing would never have been retired there. - Worker-quarantined chunks get the same treatment for the same reason: they also reach the save with no in-memory entry, which is what triggers the copy-forward. That branch was previously unreachable when the worker died on the chunk and returned no results. - Guard the durable prune's non-survivor `fs.rm`. The causes that break the reset break that delete too, and it sat outside the validation try — one undeletable directory aborted the loop and cost every remaining chunk its index entry. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(review): apply review findings Retire a chunk only when a generation nobody cleared is still on disk. `prepareDurableParsedFileChunk` is rm-then-mkdir, and the catch could not tell the two apart: an rm that succeeded before a failing mkdir leaves NO directory, so the workers recreate it and write a clean generation. Retiring there discarded a good `.v8` for no safety gain — and under a correlated failure (an empty durable index turns every chunk into a re-dispatched miss, then a descriptor burst rejects the resets en masse) it would have wiped both shared stores for every branch, where the pre-#3204 posture cost only the writes. `durableChunkHasStaleShards` is the discriminator. Finish the delete guard on the path that runs before it. The staged→live overlay in `mergeStagedDurableParsedFileStore` awaited `replaceDurableChunkDir` unguarded, so on the cold-rebuild path one undeletable directory threw out of the merge before the prune ever ran — the durable index was never rewritten and a retired chunk kept its directory. Same log-and-continue treatment, plus best-effort handling of the two `.replacing` backup removals. Aggregate the prune's delete-failure warning: a store-wide cause hits every non-survivor, and one line per directory buries the message that matters. Tests: - Guard the chmod-based prune test with the repo's `skipIf` for root/Windows and assert the directory survived, so it cannot pass vacuously where the delete succeeds. - Add a two-chunk control: one chunk's reset fails, and the sibling must stay warm through the next run. One chunk plus a global spawn marker could not tell "retires the failing chunk" from "retires everything". - Add the rm-succeeded/mkdir-failed case, which must NOT retire. - Model both post-parse merges in the R4 test (the sibling fold re-adds the key, the unreadable-meta fallback unions `entries`), and move the in-memory `entries` assertion to a direct helper test — the sharded path never populates `entries`, so the old assertion proved nothing. - Type the cache factory as `ParseCache`; the `staleKeys` assertions were TS2339 and `?? false` read as a pass regardless. - Register the store test in the cross-platform filesystem list. Correct two comments that still described a quarantined chunk by the premise this fix disproves. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(review): stop swallowing the backup removal in replaceDurableChunkDir Swallowing that `fs.rm` manufactured the very hazard this PR removes. When a non-empty `${to}.replacing` survives, the following `fs.rename(to, backup)` cannot overwrite it and is suppressed as "dest was missing", so `backedUp` stays false and the `fs.cp(from, to)` fallback merges the staged generation INTO the live directory — old shards alongside new, which the prune then indexes as one valid survivor. Let it throw; the per-entry guard added to `mergeStagedDurableParsedFileStore` already stops one such chunk from costing the others their prune. The post-publish backup cleanup stays best-effort, where an undeletable leftover really is litter. Also: - Make the sibling-isolation test perform the run its title claims. It asserted index membership and stopped; an index entry does not exercise the warm-hit path, so it would have passed even if the sibling re-dispatched. Each chunk now runs alone so the single spawn marker names which one re-parsed. - Correct two comments that outran the implementation: retirement is gated on shards actually surviving, and an undeletable directory is dropped from the index rather than removed from disk. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|---|---|---|
| .. | ||
| bench | ||
| spikes | ||
| assert-publish-grammar-coverage.cjs | ||
| assert-web-assets.mjs | ||
| bench-scope-resolution.ts | ||
| build-tree-sitter-grammars.cjs | ||
| build-web.js | ||
| build.js | ||
| cross-platform-shard.ts | ||
| cross-platform-tests.ts | ||
| ensure-fts.ts | ||
| install-duckdb-extension.mjs | ||
| run-cross-platform.ts | ||
| shard-arg.ts | ||
| sync-plugin-manifests.mjs | ||