mirror of
https://github.com/abhigyanpatwari/GitNexus.git
synced 2026-11-16 09:22:00 +00:00
* docs(plans): add resumable analyze plan * perf(bench): measure resumable storage primitives * feat(embeddings): bound and persist work selection * feat(embeddings): persist resumable vector jobs Keep graph readers independent of vector writes. Acknowledge units only after complete chunk verification and a native checkpoint. * feat(embeddings): stream resumable job units * feat(storage): add generation publication Publish durable graph generations independently of vector jobs and gate readers on matching identities. Retain the legacy graph inode under a private name so older binaries cannot serve a stale canonical graph. Coordinator integration follows separately. Refs #3598 * feat(embeddings): coordinate durable jobs Join published graph selection, durable work, bounded native handles and vector publication behind one inactive controller. Limit optional HTTP response decoding within the job reservation and admit supported large inputs under larger budgets. Refs #3598 * feat(analyze): pass phase resource allocations Carry per-run worker heaps and native pools without changing legacy defaults. Hold parser reservations through safe native exit; refuse database reallocation after an unverified close. * feat(embeddings): reserve sidecar resources Pin each durable job to an admitted model heap and CPU config. Retain native allocations after caller cancellation until response or process exit, and reject phase handoff when release is unverified. * feat(status): separate graph and vector readiness Report embedding debt independently from graph content freshness. Read bounded durable progress without loading native runtimes, gate stale vector capability stamps, and allowlist additive phase counters on public SSE and ops payloads. * fix(embeddings): recover interrupted job setup Recover an owned empty native catalog after interrupted bootstrap, and rebuild damaged failure accounting without repeating completed provider work. Preserve foreign artifacts and propagate filesystem errors. * fix(resources): verify native pool release Keep failed and concurrent native closes tracked until release succeeds. Refuse another reserved allocation while cleanup is unverified, including failed connection construction. Let graph readers tear down after result-close errors. * fix(resources): drain graph before phase admission Wait for session work and verify the prior main graph pool has closed before parser or embedding admission. Recheck cancellation after the drain so cancelled jobs do not read or mutate job metadata. * docs(plans): revise resumable analyze seams * perf(bench): measure paired analyze runs * feat(index): publish logical graph generations Retain private database files across incremental generations and fence prepared publication against superseding writers. Exit-only producers must exit before publication; interrupted in-place writes remain explicitly dirty. * feat(index): guard graph process handoff Wait for actual process exit and reacquire the ordinary slot lock before publishing the prepared attempt. Recheck placement and the registered name, and retain graph discoverability after cancellation. Add a bounded nonterminal worker receipt without serializing the live graph. Production phase activation remains separate. * feat(cli): defer success across graph handoff Finish live-graph skill generation before the child hands off. Transfer only bounded counters, identity and the pre-run self-commit safety flags; wait for IPC delivery before returning for native process exit. Reject missing supervisors and mismatched prepared results before skill writes. Finalization, self-commit and success remain deferred until the production supervisor completes embedding; core activation follows the pending-artifact and resource contracts. Refs #3598 * feat(embeddings): rebase pending paid artifacts Separate graph job identity from mutable artifact ownership. Keep paid compatible groups across source edits using bounded revalidation and selection pruning; retain one sealed fallback for the active space. * feat(embeddings): preserve historical coverage * feat(embeddings): reuse verified legacy vectors Retain a directly referenced legacy source across pending-artifact rebases. Stream checkpoint exclusions and build a native metadata index without copying vectors or retaining corpus-sized JavaScript maps. Require recorded provider identity and provable complete input groups. Keep unknown provenance explicit, source artifacts immutable, compatible relocations free of provider calls, and new inputs in normal batches. The controller remains inactive until coordinated writer activation. Refs #3598 * feat(analyze): account for supervised phases Charge parent memory once per phase. Carry allowlisted progress through worker IPC and preserve counters across logs without completing the job. Refs #3598 * fix(storage): retain owned analysis generations Keep private generation files in place through shared-store preparation and publication. Retain prepared publications and pending embedding work across the unlocked process handoff, including work in branch slots. Refs #3598 * fix(embeddings): share pinned vector readers Share immutable vector file identities so sibling hard links retain one native reservation. Keep the handle alive until its final reader closes, including failed initialization and replacement of another reader's path. Refs #3598 * feat(storage): share completed vector artifacts Pin immutable vectors independently in checkout slots and shared graph snapshots. Keep mutable jobs with their owning checkout; resume interrupted adoption without overwriting unverified files. Refs #3598 * feat(analyze): retain caches across phase handoff Keep prepared staging during lock reacquisition and share cache publication between the graph writer and exit continuation. Bound the filesystem handoff to chunk keys; never copy live cache contents. Refs #3598 * fix(analyze): retain owned generation routing Keep prepared and pending work under its original slot lock during shared-store adoption. Preserve private generation filenames during incremental writes without copying the graph. Drain model and native resources only when graph work actually starts, including FTS repair. Leave the warm no-op path unchanged. Validated 158 tests, source build, test typecheck, formatting and lint. Complete graph gates cover 29 symbols in two files and the expected coordinator flow. End-to-end embedding activation and performance acceptance remain pending. * feat(analyze): wire resumable phase execution Publish verified graphs before embedding and hand off resource ownership across CLI, watch, auto-sync, and HTTP phases. Keep completed vectors readable while durable embedding units resume in a separate generation. Preserve legacy vector reuse, avoid graph rebuilds for dimension changes, and cover interruption boundaries with fresh-process restart tests. Refs #3598 * fix(analyze): close recovery and ownership gaps Retain interrupted private work, clear stale query widths, and keep native reservations until confirmed worker exit. Keep shared-member admission gates outside garbage-collected directories for flat and branch writers. Reject invalid routes and overlaps, and distinguish validation from uncertain writes. Make test imports work before build, isolate host-capacity assertions, and give legacy vector fixtures the production native pool allocation. Focused regressions, clean-build types, formatting and lint pass. Graph analysis maps 51 symbols across 21 paths. Two import-only test paths remain unmapped; direct review and integration tests cover them. The parent full suite passed 32,084 tests with five failures: two fixed legacy-vector fixture cases and three recovery cases awaiting isolation. Final full-suite and performance acceptance remain pending on the draft. Refs #3598 * perf(embeddings): batch reused vector writes Avoid a native transaction per reused node while retaining the byte budget and durable unit boundary. Cover mixed inputs, cancellation, and replay after read or committed-write failures. Native regression fails before the fix (65 commits for 64 nodes). 79 focused tests and all three former recovery timeouts pass. Fresh graph gate: 18 symbols, 2 files, 36 affected flows; complete mapping. Final whole-command performance acceptance remains pending. Refs #3598 * fix(analyze): honor explicit worker overrides Preserve the existing CPU oversubscription override while reserving memory for every requested worker. Automatic concurrency stays CPU-bounded. Validate on one- and four-CPU hosts: 81 tests pass; the override regressions fail before the fix. * fix(storage): validate opened artifact files Validate descriptors before reading bounded receipts and manifests. Use nonblocking opens to reject FIFO substitutions and verify directory entries when O_NOFOLLOW is unavailable. Preserve the metadata cache fast path and guard generation paths explicitly. Remove the redundant pending-node fallback. Add race, symlink, cleanup, and cache regressions to platform coverage. * fix(storage): guard the graph path directly Apply the containment check to the same path passed to lstat. The nested normalization guarded a different expression in CodeQL. path.relative already resolves its inputs, so remove the redundant normalization and retain relative-path behavior. * fix(embeddings): preserve AST inputs on resume Carry indexed source policy into bounded resumable jobs and preserve exact-input reuse across sibling edits. Include strategy and retention in job identity; admit source memory before reading. * chore: run checks for the rebased stack * refactor(embeddings): simplify reuse plumbing Name reuse-validation options and share the owned-directory loop. Preserve callback order, directory sync, and ownership checks. * test: fix recovery and Windows CI fixtures Distinguish legacy checkpoints without input provenance from verified source-aware chunks. Check the embed API at its shared-phase boundary and model pathname races without rename-over-open on Windows. * fix: close resource admission and CI gaps Keep live sidecars charged after a reap timeout. Let sync and HTTP jobs use automatic host budgets instead of overcommitting to default V8 limits. Explain the incompatible native zero-pool override, clean up failed HTTP fixtures, and run the native vector suites across platforms. * fix: preserve analysis resource ownership Pin admitted model settings and retain worker and supervisor reservations. Recheck canonical directory identity before receipt reads or writes. Canonicalize the native publication fixture for macOS and Windows. * test: distinguish parameterized CI cases * docs: remove implementation plan from PR * ci: gate resumable analyze performance * fix(analyze): close recovery and CI gaps Retain ownership receipts and paid vectors through interrupted work and cleanup. Restore automatic heap and parser-worker sizing, and cover publication, cancellation and capacity failures. Refs #3829, #3598 Note: pre-existing setup.test.ts and setup-skills.test.ts failures with inherited CODEX_HOME; all 62 tests pass with it unset for the test process. * fix(embeddings): normalize graph parent paths Bind the canonical job root to the same graph path on macOS temp aliases and Windows short paths. Keep graph identity and owned-directory checks. Canonicalize the retention fixture and show subprocess stderr on failure. Validation: 65 focused tests; 26 tests through an aliased TMPDIR; build, test typecheck, formatting and lint; graph change analysis (4 files). * fix(embeddings): batch writes and guard recovery Batch bounded vector and receipt writes to remove per-node native calls without changing validation, transactions, or durable checkpoints. Pin publication descriptors before entry validation. Preserve excluded vector coverage, reject malformed status paths, retry orphan sidecars, and complete shutdown after child spawn failures. Validation: 159 focused tests and 89 native/restart tests; build, test typecheck, lint and formatting. Refreshed graph analysis maps production changes; three test files remain unmapped. * ci: profile fresh analyze overhead Capture process and parser-worker CPU profiles in a separate CI-only run. Reject profiling with --check so diagnostic overhead never affects the acceptance gate. * perf(analyze): trim continuation startup work * fix(analyze): preserve recovery and storage bounds Keep automatic writer pools page-scaled and restore frontend heap caps. Adopt a generation recovered during handoff, reclaim sealed artifacts, and retire compatible legacy sources after migration completes. Report semantic degradation and omit provider causes from API logs. Enforce no paid replay in the CI resume benchmark. * perf(analyze): defer unused parser imports * fix(workers): retain failed shutdown reservations * perf: skip unused budget supervisor receipt * fix: bound rejection history and watch budgets * perf: batch provider vector writes within units * fix: harden cleanup and semantic availability * perf: avoid repeated analysis startup probes Skip unused automatic-heap parent provenance and reuse one npm version probe for runner selection and its warning. Keep child/watch identity capture and manual/test heap overrides. Refs #3829. * fix: normalize legacy export paths on Windows Cover Unicode export directories and restore the heap parser comment. Refs #3829. * test: restore watch resource mock contract Provide the supervisor reservation used by watch refresh admission. Refs #3829. * fix: retain watch budgets and bound lookahead Keep explicit heap reservations across refreshes and limit brace scans to the existing three-line window. Refs #3829. * fix: stop retries on shutdown and allow empty parse * refactor: simplify embedding setup and test lists * perf: batch vector completion hash updates Keep hash finalization inside the existing transaction and row bounds. Run embedding input-isolation tests in the serial native project. * fix: address PR #3829 recovery feedback Preserve terminal worker results, reclaim orphan attempt metadata, retain artifacts with unrecognized space ownership, and correct the cancellation comment. * perf: retain simple vector completion query * perf: skip redundant alias identity probes * perf: avoid Git probes for ordinary checkout roots * perf: avoid redundant vector transactions Standalone writes already execute in automatic transactions. Keep multi-query transactions and durable unit checkpoints unchanged. * docs: clarify cancellation grace period * fix: finish vector recovery and explicit drops Preserve paid peers through source-admission aborts and complete legacy retirement after interrupted publication. Explicit drops remove validated vector payloads before their ownership receipts so later runs regenerate them. Refs #3829 * fix: preserve storage errors during aborts Only unwrap aborts raised by source admission. A simultaneous native write failure must propagate instead of entering the provider-abort completion path. Refs #3829 * fix: validate retained analysis artifacts Reject malformed cache keys before shard writes and retain legacy work before shared-store adoption. Explicit drops validate legacy ownership and keep receipts until payload cleanup succeeds. Damaged registered graphs block stale local hook fallback. * fix: preserve legacy and preflight contracts Keep the native-default pool override, retain incomplete legacy seeds with their owner, and include resource preflight in the worker timeout. * fix: honor resource and preflight boundaries Release optional cache space before rejecting an admissible source read. Start server deadlines before preflight and reject null progress counts. Pass the native-default pool sentinel through the database opener. * test: align native and drop contracts Accept the documented native-default zero sentinel. Verify the rebuilt graph and empty vectors instead of an obsolete escalation log. Refs #3829 * perf: avoid redundant vector checkpoints Acknowledge verified WAL commits without rewriting the database per unit. Keep the final publication checkpoint and prove crash recovery from WAL. Give publication test cases distinct titles for the CI execution audit. Refs #3829 --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> |
||
|---|---|---|
| .. | ||
| languages | ||
| optimize | ||
| plans | ||
| cli-reference.md | ||
| deployment.md | ||
| editor-setup.md | ||
| mcp-reference.md | ||
| wiki-generation.md | ||