2 commits

Author	SHA1	Message	Date
Gergő Magyar	72edf40087	perf(store): V8 sidecars plus hardlinked ParsedFile restore (#3099) ... * perf(store): add best-effort V8 sidecars beside canonical JSON caches Warm ParsedFile and parse-cache loads skip JSON.parse when a sidecar is present. JSON remains authoritative: envelope validation plus v8.deserialize decide the hit, and any failure falls back without reparsing. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(store): require generation bind or sidecar drop before cache overwrite A same-length JSON rewrite could accept a leftover V8 sidecar if both generation rotation and unlink failed. Refuse the new generation unless at least one of those invalidations succeeds; skip publishing a sidecar when only the drop succeeded. detect_changes --scope all: 7 files, risk low, no affected processes. tsc --noEmit clean; 115/115 relevant unit tests; cache-related integration tests pass. parse-impl-env-reads worker-ready timeout is pre-existing (same 5 failures with this change set stashed). ESLint 0 errors; remaining warnings are pre-existing and not on changed lines. Co-authored-by: Cursor <cursoragent@cursor.com> * chore(autofix): apply prettier + eslint fixes via /autofix command * refactor(store): share V8 overwrite invalidation across persist paths The bind-or-drop gate lived in five writers. One helper keeps the protocol in a single place and lets bind/drop run together on the async path. detect_changes --scope all: 3 files, risk low, no affected processes. Co-authored-by: Cursor <cursoragent@cursor.com> * perf(store): hardlink durable ParsedFile shards into the run store Warm restore of parsedfile-cache into parsedfile-store now publishes all four shard files via fs.link, falling back to copy-into-tmp + rename so a leftover dest hardlink can never be written through. JSON remains the canonical cache; V8 sidecars ride the same path. Co-authored-by: Cursor <cursoragent@cursor.com> * perf(store): load immutable V8 shards in place, drop JSON fallback Warm analyze was still paying JSON.parse plus a restore copy. One .v8 envelope per shard and SCHEMA_BUMP 81 make a miss re-extract instead of serving a stale JSON twin. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(store): validate durable V8 warm-cache restores Reject incomplete or corrupt durable generations and snapshot valid shards before skipping parse workers, preserving ParsedFiles when persistence fails. Co-authored-by: Cursor <cursoragent@cursor.com> * refactor(store): drop unused durable load path Load ParsedFiles only from the run-store snapshot and share one checksummed payload reader so inspect and deserialize stay consistent. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>	2026-08-30 20:50:20 +00:00
Gergő Magyar	414c1a5693	fix(storage): give every registry write its own tmp path (#2888) (#2920) ... * fix(storage): give every registry write its own tmp path (#2888) `writeRegistry` staged the global registry through a FIXED `~/.gitnexus/registry.json.tmp`. The rename is atomic with respect to readers, but the tmp path is not private to the writer, and that file is the one file every gitnexus process on the machine writes. Two of them starting together stage through the same inode: the second `writeFile` overwrites the first's bytes, the second `rename` moves that inode onto `registry.json`, and the first's own rename then finds nothing at the source and rejects with ENOENT: no such file or directory, rename '<home>/registry.json.tmp' -> '<home>/registry.json' which kills the MCP server, because it lands on the startup path (`mcpCommand` -> `LocalBackend.init` -> `refreshRepos` -> `listRegisteredRepos({validate:true})`) where nothing catches — the client just reports "Server disconnected". #2716's `withRegistryLock` serializes the callers and hides this in the normal path, but it deliberately degrades to UNLOCKED after a 5s `IndexLockTimeoutError` (availability over serialization), so the window is still live. Measured on this branch's parent with 12 concurrent processes pruning a stale registry while another process held the registry lock: 4/12 crashed with the trace above. Same harness with 24 processes and no lock contention: 0/24. So the write itself has to be collision-proof rather than relying on the lock. `writeMetaFile` (repo-manager), `writeBridgeMeta` (group/bridge-db) and `writeContractRegistry` (group/storage) already carried the correct shape — random tmp suffix, `'wx'` + `0o600`, `retryRename` — as three byte-identical copies, none of which cleaned up its tmp file on failure. Rather than adding a fourth copy, that sequence moves to `writeFileAtomic` in storage/fs-atomic.ts (beside `retryRename`, which it uses) and all four writers call it. The helper also unlinks the tmp before rethrowing: with a fixed name a leaked tmp was self-limiting because the next writer overwrote it, but a random suffix would drop a fresh orphan beside the target on every failed publish. Second half of the same crash: the prune write inside `listRegisteredRepos({validate:true})` is housekeeping, not the caller's request. Every caller consumes the returned `valid` array and the prune set is recomputed from scratch on the next validating read, so a failed write costs a retry, never correctness — while rethrowing it took down the whole MCP server. It is now caught and warned about, which also covers the read-only-home and full-disk variants of the same startup death. Note: `registry.json` is now created `0o600` (it inherited the umask before, typically `0o644`), matching what `gitnexus.json` has always used. A rewrite tightens the mode on existing installs. Verified: the five new tests in test/unit/repo-manager-registry-atomic-write.test.ts all fail on the parent commit — four with the exact ENOENT above — and pass here; the process-level repro goes 4/12 -> 0/12 crashes with the lock held. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VXSu2fTmm7EZGDVquWeBrL * refactor(storage): trim the atomic-write helper and its guards Follow-up polish on the #2888 fix, no behaviour change except where noted. - `writeFileAtomic` drops the `mode` parameter (no caller ever varied it) and inlines `0o600`, and gains an `attempts` pass-through to `retryRename`. The prune write in `listRegisteredRepos` now passes `attempts: 1`: it discards a failure anyway, so the 300ms of rename backoff bought nothing and was spent holding the registry lock, on a path with a sub-500ms cold-start budget (`gitnexus augment`) and on MCP startup. - `saveMeta` serialises `meta` once instead of once per written file. `meta` carries a `fileHashes` entry per file — 263KB and ~420us on this repo, linear in file count — and it was being stringified twice per save, several times per analyze. `writeMetaFile` was a one-line forwarder after the previous commit, so it folds into `saveMeta`. - Comments: the four writers were each restating the primitive's contract, and the #2888 narrative appeared in four files. Kept one authoritative copy in the helper, one registry-specific note at `writeRegistry` (why the lock is not enough), and deleted the rest. - Tests: new test/unit/storage/fs-atomic.test.ts covers the primitive behaviourally — published bytes, `0o600` on the result, three concurrent publishers to one target all resolving, no leftover tmp and intact previous content when the publish fails. That is what the source-text regexes in insecure-tempfile.test.ts were approximating, so those shrink to the one thing regex is good for: this module does not hand-roll a tmp path. The registry test drops the assertions the primitive now owns, an unused `fs.writeFile` capture, a type alias with two `as unknown as` casts the sibling harnesses do without, and moves its two path-only temp repos to `beforeAll`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VXSu2fTmm7EZGDVquWeBrL --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-10 21:16:42 +01:00