4 commits

Author	SHA1	Message	Date
Gergő Magyar	ac9a4e9abd	fix(embeddings): keep ONNX off the install and analyze critical path (#3287) ... * fix(embeddings): isolate local ONNX inference in a child_process sidecar The analyze parent must not load onnxruntime-node. Fork a sidecar for vectors only and reap it on worker exit; keep Ladybug writes in-process. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(embeddings): share the sidecar client across MCP, serve, and sync Query hosts now use the core façade instead of a second in-process ONNX embedder. Search skips an empty table, sync reaps beside closeLbug, and ready means the stack is resolvable rather than a warm singleton. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(embeddings): refuse Intel Mac and unloadable prefix before npm heal Analyze, sync, install, and the sidecar client now consult the platform blocker before forking or downloading the optional stack. HTTP stays the escape hatch; wasm is not treated as a rescue. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(embeddings): take the ONNX stack off default npm install Pins live in gitnexusEmbeddingStack. embeddings install writes prefix overrides before npm spawn. Leftover 1.6.12 package-first trees are residual. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(publish): drop grammar source from the published tarball Every vendored grammar has 6/6 prebuilds, so files ships those plus Leiden and FTS instead of parser.c. First ship stays above 80 MiB. Co-authored-by: Cursor <cursoragent@cursor.com> * test(embeddings): match MCP missing-stack warn to the R20 copy Default install no longer calls the stack optional, so the once-per-backend stderr assertion must look for the new lead line. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(review): bound sidecar death, cancel writes, and publish-file guards Init-time native crashes no longer respawn a child on every query. Local embedBatch honors AbortSignal after sidecar return, MCP query() surfaces vector-lane degradation, disconnect always reaps, and the grammar prepack guard checks files globs instead of on-disk prebuilds. Co-authored-by: Cursor <cursoragent@cursor.com> * refactor(embeddings): share runtime preflight and sidecar reap helpers Analyze and embeddings-sync used the same blocker/prefix/install gate with different error routing. One assessment keeps those paths aligned without changing CLI vs thrown-error behavior. Co-authored-by: Cursor <cursoragent@cursor.com> * Address PR review feedback (#3287) Keep a reaped sidecar from resetting its replacement, wait for dispose, tighten the publish-files guard, and stop assuming a leftover ONNX tree in CI. Co-authored-by: Cursor <cursoragent@cursor.com> * Address remaining PR review feedback (#3287) Clear the sidecar reap timeout, add init IPC slack, and isolate embeddings-sync tests from HTTP-mode env. Co-authored-by: Cursor <cursoragent@cursor.com> * test(embeddings): unstub globals after sidecar HTTP-mode tests Keep a leaked fetch stub from failing assertions out of later tests in the same file. Co-authored-by: Cursor <cursoragent@cursor.com> * test(embeddings): pin sidecar success cases off darwin/x64 The runtime blocker reads the real process platform before the fork mock, so local-success tests must not inherit an Intel Mac host. Co-authored-by: Cursor <cursoragent@cursor.com> * Address remaining PR review feedback (#3287) Keep vector degradation per query, treat leftover Intel-Mac stacks as not ready, and document that the CLI image no longer ships ONNX. Co-authored-by: Cursor <cursoragent@cursor.com> * Simplify embedding sidecar shutdown and search hot paths Drop redundant sidecar reaps and unused child helpers, and run FTS alongside semantic search. Co-authored-by: Cursor <cursoragent@cursor.com> * Address remaining PR review feedback (#3287) Share HF attempt parsing with the sidecar init deadline, abort embed waits without killing the child, and restore last init options on recreate. Co-authored-by: Cursor <cursoragent@cursor.com> * Address remaining PR review feedback (#3287) Treat sub-1 HF attempt env values as invalid, and drop leaked sidecar waiters when IPC send throws. Co-authored-by: Cursor <cursoragent@cursor.com> * Address remaining PR review feedback (#3287) Keep sidecar init on a shared chain; each waiter can abort only its own wait. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(ci): declare embedding-table existence probe as unordered LIMIT The empty-table skip in semanticSearch is existence-only; declare it so the #2787 determinism guard stops failing coverage shard 3/3. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com>	2026-09-15 12:11:47 +01:00
azizur100389	3f5ca8cdb7	fix(storage): guard stale file-lock reclamation (#3234) ... * fix(storage): guard stale file-lock reclamation * fix(storage): close lock recovery failure paths * Address PR review feedback (#3234) - Flush the lock-child stderr diagnostic before process.exit - Treat explicit NaN timeouts as the default ceiling - Document non-retryable guard timeouts on the worker IPC contract Co-authored-by: Cursor <cursoragent@cursor.com> * fix(storage): stop mislabeling live lock waits as orphan recovery A brief peer inspect must not attach guardPath or send operators to RUNBOOK delete steps. Refuse lock-free embeddings sync, stop --watch only on a true guard timeout, and drop an unreadable self-created guard before failing closed. Co-authored-by: Cursor <cursoragent@cursor.com> * chore(autofix): apply prettier + eslint fixes via /autofix command * Address PR review feedback (#3234) - Verify lock/guard absence before degrading a denied main-lock create - Launch the third contender from unlinkSync, not the dead rename path - Document group-lock timeouts for unrecoverable guard leftovers Co-authored-by: Cursor <cursoragent@cursor.com> * style: prettier index-lock reclaim guard tests Co-authored-by: Cursor <cursoragent@cursor.com> * fix(storage): treat O_EXCL as the lock-file presence check CodeQL flagged existsSync-then-wx on analyze.lock. Create with wx first and only reclaim unreadable leftovers after grace, so a successor is never unlinked from a lost race. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Gergő Magyar <gergomagyar@icloud.com> Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>	2026-09-12 11:47:12 +01:00
Gergő Magyar	7bbaf6b73b	fix(review): fail closed on foreign embedding identity and vector-width drift (#3260) ... Applied from a ce-code-review pass over this branch (run 20260911-053832-b49f88d6). - `embeddings sync` now gates EVERY checkpoint kind on embedding identity. `unverified-count` was exempted, but `decideEmbeddingResume` abandons that kind before it compares identity, so the exemption was the only thing keeping a foreign model from filling holes beside the old model's vectors — two vector spaces in one CodeEmbedding table, reported healthy. The test that asserted this run succeeds now asserts the refusal. - `embeddings sync` refuses when the index's recorded vector width differs from this run's. The column is FLOAT[N] fixed at build time and the pipeline deletes each batch's stale rows immediately before inserting, so a width change deleted rows it could not re-insert. `analyze` already forces a rebuild on the same mismatch; only a rebuild can retype the column. - `GITNEXUS_EMBEDDING_RETRY_TIMEOUTS` parses with the repo's truthy convention (`1`/`true`/`yes`). The integer parser threw on `true`, so the conventional spelling hard-failed every embedding call rather than enabling the flag or leaving it off. README names the accepted spellings. - One `persistMeta` write path replaces three inlined metadata read-modify-writes; #2790 traced two production drifts to hand-copied writers of these exact fields. - Tests: the pipeline mock now invokes `onCheckpointWindowStart`/`onCheckpoint`, so the resume contract actually executes under test. Added coverage for the incomplete-index refusal, the partial-checkpoint branch, and the body-read timeout retry site that the existing opt-in test never reached. Verified: tsc --noEmit clean; 99 tests pass across the three affected suites; prettier clean. Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-11 07:37:08 +00:00
panbergco	32a5fc3c4f	feat: Make long HTTP embedding jobs resumable (#3065) ... * Make long HTTP embedding jobs resumable Retry endpoint timeouts when explicitly configured, creating a fresh abort signal for every attempt. Add gitnexus embeddings to fill an existing index in place: every successful batch is durable, and rerunning skips vectors whose content hash still matches instead of rebuilding the structural graph. * Address PR review feedback (#3065) Hold the per-index lock around embeddings sync, fail closed on a foreign-identity partial checkpoint, and refuse to create an empty DB from leftover metadata. Use the canonical embedding count, keep closeLbug from masking the real error, load hashes instead of full vectors, and document GITNEXUS_EMBEDDING_RETRY_TIMEOUTS. Co-authored-by: Cursor <cursoragent@cursor.com> * Split embeddings sync off the install module so install no longer loads Ladybug. Share the opt-in TimeoutError wrap between fetch-throw and .json(), and cover a non-file LadybugDB path. Co-authored-by: Cursor <cursoragent@cursor.com> * Describe embeddings sync checkpoints as periodic and report cached node count. Pipeline checkpoints every 5,000 nodes, and fetchExistingEmbeddingHashes is keyed by nodeId, not vector rows. Co-authored-by: Cursor <cursoragent@cursor.com> * Leave an unverified-count checkpoint when embeddings sync cannot count rows. A finished run must not keep an interrupted window marker; the next sync should re-derive the count instead of hitting the identity gate. Co-authored-by: Cursor <cursoragent@cursor.com> * chore(autofix): apply prettier + eslint fixes via /autofix command --------- Co-authored-by: panbergco <panbergco@users.noreply.github.com> Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Gergő Magyar <gergomagyar@icloud.com>	2026-09-10 19:48:50 +01:00