Commit graph

13 commits

Author SHA1 Message Date
Felipe
c2ca132620
fix: web citation/code panel bugs and serve analyze/route hardening (#3348)
* chore: ignore local Vercel link artifacts

Keep .vercel and env files out of the repo after a local preview link.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(web): repair citation chips, code panel line math, stale agent state

Audit findings in the web client, each verified against the source:

- RightPanel: citation chips ([[path:10-20]], [[Class:Foo]]) were wired to a
  stub resolver that always returned null, so clicking any citation did
  nothing. Expose resolveFilePath from useAppState and use it.
- CodeReferencesPanel: graph startLine/endLine are 1-based but were treated
  as 0-based, so the highlighted range and scroll target were off by one
  line; AI citation cards always rendered "code not available" because the
  snippet loader was a stub. Fetch per-citation snippets via /api/file.
- useAppState: sendChatMessage read llmSettings.activeProvider outside its
  deps (stale provider capabilities after switching provider);
  initializeAgent trapped projectName at '' for callers without an override
  (system prompt labelled the codebase "project"); the embeddings 409 dedup
  matched a message the server never sends for same-repo jobs.
- tools.ts impact: for path targets every symbol defined in the file shares
  the filePath, so the disambiguation always picked the first row and could
  analyze an arbitrary symbol while reporting a file impact. Prefer the File
  node.
- useSigma: the layout timeout called stop() but never kill(), leaking one
  ForceAtlas2 Web Worker plus four graph listeners per completed layout.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(server): release repo lock on cancel, IPC-first worker cancel, route hardening

Audit findings in gitnexus/src, each verified against the source:

- analyze-launch: cancelJob marks the job failed before the worker exits,
  so the exit handler's terminal early-return skipped releaseLockOnce and
  the repo stayed locked ("Another job is already active") until restart.
  Release on terminal exit and when forkWorker bails on a terminal job.
- analyze-job: cancellation now sends { type: 'cancel' } over IPC first and
  signals only after a 15s grace. On Windows child.kill('SIGTERM') is a
  forceful termination, so leading with it could kill the worker inside a
  LadybugDB write. Mirrors core/auto-sync/analysis-worker-launch.
- api resolveRepo: a job that FAILED during the hold-queue wait fell through
  to the { __timedOut } sentinel ("taking longer than expected") instead of
  404; only /api/repo checked the sentinel, so graph/query/search/file/grep/
  embed/delete crashed on entry.storagePath with a 500 after a 5 minute hang.
  Return null on failed jobs and check the sentinel in every consumer.
- api processes/process/clusters/cluster: resolve ?repo= through the HTTP
  resolver (documented policy on resolveRegisteredRepoEntry) and pass the
  registered absolute path to the backend; add the standard rate limiter.
  The raw param previously reached the MCP resolver, which runs a
  cwd-relative realpathSync probe + registry refresh on a bare-name miss
  and accepts unambiguous partial names.
- api body handling: Express 5 leaves req.body undefined without a JSON
  content type, turning "Missing X" 400s into TypeError 500s; body-parser
  4xx errors (malformed JSON, over-limit) were also reported as 500.
- /api/file: the lexical path.relative check cannot see symlinks; re-check
  containment on realpath so a cloned repo containing evil -> /etc/passwd
  cannot read outside the root.
- repo-manager unregisterRepo: used the lenient reader, so a transient read
  error (EBUSY/EPERM racing another process's atomic rename) turned into
  writing [] and deregistering every repo. Use the strict-if-present reader.
- clean --branch: compared registry paths with raw path.resolve instead of
  the canonical registryPathEquals used everywhere else (macOS /private/var,
  Windows short names / drive-letter case) and reported indexed branches as
  not indexed.

Tests: cancelJob IPC-before-signal contract; /api/file symlink escape (403)
and in-repo symlink (200), skipped where the host cannot create symlinks.

Co-authored-by: Cursor <cursoragent@cursor.com>

* chore: keep example env files visible and ignore local Cursor config

.env* also hid gitnexus/.env.example and eval/.env.example. .vercel was already ignored. The web app's .cursor/ stays local, including its MCP file.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Add realtime execution ops dashboard for Vercel monitoring.

Expose /api/ops snapshots over the local serve process and a ?view=ops SPA panel so analyze/embed jobs can be watched live from the hosted web UI.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix: address gitnexus-check review on cite/embed/resolve paths

Pass req into resolveRepo for process/cluster routes, tighten same-repo embed 409 handling, guard empty citation paths, fix snippet retry races, and drop the lone-File ambiguity fallback.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(ops): harden CORS, redact paths, and bound ops streams

Restrict Vercel CORS to exact production hosts, omit raw repo paths/URLs from the unauthenticated ops feed, rate-limit and cap SSE connections, and fix dashboard SSE/poll edge cases.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(web): stabilize citation fetches and file impact matching

Retry cancelled snippet loads without duplicate in-flight reads, cap range-less citation downloads, and make impact file matching unique-suffix-aware with a synthetic File target when LIMIT drops the File node.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(web/ops): close gitnexus-check review threads on SSE and redaction

Cap citation reads, reconnect ops on applied server URL, skip SSE onError after abort, and strip URL query/fragment from public repoName.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(ops): abort SSE on poll fallback and harden repoName parsing

Prevent dual SSE+poll after a failed safety snapshot, skip overlapping poll ticks, ignore aborted streamSSE onError, and basename Windows drive-letter URLs so ops never leaks path segments.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(ops/web): close gitnexus-check threads on SSE budget and credential leak

Keep finite SSE retries across short 200s, strip backend URL userinfo before ?server=, and redact progress messages on the public ops feed.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Address PR review feedback (#3348)

Restore 0-based GraphNode line math, redact public job poll/error fields, fix omit-?repo= 400, hold the analyze lock across cancel-during-settle, and restore the Vercel shared compile.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(cli): contain leftover-slot reclaim to the slot and keep --stale status honest

Preview and force now share one branches/ containment rule, nested junctions cannot walk a sibling index, and a mid-loop git failure no longer claims leftovers were not deleted after a successful rm.

Co-authored-by: Cursor <cursoragent@cursor.com>

* refactor(cli): share leftover-slot helpers without changing reclaim behavior

Pull the rolling I/O pool and owned-cwd storage lookup into one place so clean --stale/--branch and leftover listing stop restating the same ownership and concurrency paths.

Co-authored-by: Cursor <cursoragent@cursor.com>

* chore(autofix): apply prettier + eslint fixes via /autofix command

* Address PR review feedback (#3348)

Keep the omitted-repo snapshot instead of re-listing, stop citation and ops races, redact full public repo URLs, and restore fake timers in teardown.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Address remaining PR review feedback (#3348)

Store graph node citation lines as 0-based offsets and correct the default-port origin comment.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Address remaining PR review feedback (#3348)

Redact public SSE progress text, stop citation append retries from
cancelling in-flight reads, and keep the ops dashboard from showing a
stale snapshot or clearing a failed Connect.

Note: pre-existing failure in incremental-index-extension-dml-gate and other lbug/env unit tests not addressed by this PR.
Co-authored-by: Cursor <cursoragent@cursor.com>

* Address remaining PR review feedback (#3348)

Prune citation snippets when AI refs are cleared, and poll /api/ops at 2s so the fallback stays under the 60/min limit.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(ci): apply prettier class order for format check (#3348)

CI quality/format runs root-only npm ci, so prettier-plugin-tailwindcss
sorts scrollbar-thin without the web Tailwind catalog.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Address PR review feedback (#3348)

- Require a unique suffix match for graph-backed citation paths so
  ambiguous names like index.ts no longer open the first graph file.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Address PR review feedback (#3348)

- Require a path-component boundary so unique citation suffixes cannot match filename substrings like myindex.ts

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(server): redact filesystem paths in ops text

Unauthenticated /api/ops and poll replay worker errors. URLs were
scrubbed but home-directory and Windows paths still leaked.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(server): omit repoPath from public SSE frames

/api/ops lists job ids, so the unauthenticated progress stream
must not replay the analyzed filesystem path.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(web): treat embed lock 409 as a busy error

Analyze and embed share the same lock string. Mapping that 409 to
embedding hid an in-flight analyze as a successful embed start.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(web): bound impact File path suffix matches

Unbounded endsWith let lib/foo.ts select src/mylib/foo.ts. Require
an exact path or a unique /suffix, matching resolveUniqueIndexedPath.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(server): keep canceled analyze slot until exit

Marking failed before the worker exited let a second POST start
cloneOrPull against a LadybugDB file still being written.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(server): skip Windows SIGTERM on job dispose

child.kill('SIGTERM') is TerminateProcess there. Ask over IPC first
and leave the 15s grace timer to SIGKILL.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(server): let process routes skip the analyze hold

GET /api/processes and /api/clusters always waited up to 300s.
?awaitAnalysis=false fails fast; default still waits like /api/repo.

Co-authored-by: Cursor <cursoragent@cursor.com>

* test(web): cover public ops and analyze SSE user flows

Lock the unauthenticated dashboard and analyze complete/fail/cancel paths so a leaked repoPath, token, or home path cannot ship unnoticed.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Address PR review feedback (#3348)

Keep caller cancel reasons over the worker's generic IPC, skip publish while cancel is pending, and redact scp-style remotes on the public ops feed.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Address remaining PR review feedback (#3348)

Release the analyze slot when a worker fails to spawn, and omit branch refs from the unauthenticated ops feed.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(server): do not reuse an analyze job that is pending cancel

A dying same-repo job still occupies the single slot; 202-reuse would
attach a new client to a cancel in flight.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(server): hold the analyze lock until the worker exits after cancel

Cancel error IPC used to drop the repo lock while the child was still
checkpointing. Abort settle immediately on pending cancel so the slot
is not held for a 60s disk poll.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(server): redact known repo paths with spaces in public ops text

Known repoPath/repoUrl literals are replaced first so a clone dir with
spaces cannot leak past the whitespace-bounded path regex.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(server): return the live job on analyze and embed DELETE

Hard-coding failed made clients retry immediately and 409 while the
child still occupied the slot. resolveRepo now returns not-found as
soon as that job fails instead of waiting out the hold timeout.

Co-authored-by: Cursor <cursoragent@cursor.com>

* test(web): drive public analyze and ops through a live gitnexus serve

Spawn the real backend and observe requests instead of intercepting
them, so slot occupancy, redaction, and reconnect stay honest.

Co-authored-by: Cursor <cursoragent@cursor.com>

* refactor(web): reuse code-panel helpers and drop dead UI aliases

Citation fetches already had selectedNodeFileRange and snippetRepoKey;
the impact File suffix filter already handled exact paths.

Co-authored-by: Cursor <cursoragent@cursor.com>

* chore(autofix): apply prettier + eslint fixes via /autofix command

* Address PR review feedback (#3348)

- Skip createJob reuse after cancel IPC is consumed while the child remains
- Hold the repo lock until exit when complete IPC races a pending cancel
- Scrub full remote URLs before known repoUrl prefixes in public ops text
- Reject unique impact File suffix matches from a truncated LIMIT 10 page
- Make live e2e helpers bound probes, clean up failed startups, and wait out the cancel slot

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(web): open live e2e pages on the Vite host CI actually bound

Absolute 127.0.0.1:5173 navigation refused on Actions because wait-on
and Vite use localhost (often ::1). Honor FRONTEND_URL when set, else
pick the first of localhost / 127.0.0.1 that answers.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Address PR review feedback (#3348)

Hold the analyze lock until worker exit when cancel aborts settle, and assert GitLab failure chrome does not leak host or path.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(web): keep live analyze e2e under the analyze rate limit

POST /api/analyze allows 10 requests per minute per IP. The slot-free
helper re-POSTed every 400ms while a cancelled worker was exiting, spent
that budget, and the lock test's hold request got 429 instead of 202.

- postAnalyze waits out a 429 using the RateLimit reset and retries
- slot polling backs off to 2s and leaves a small POST budget for callers
- the slot probe is a clone that fails before any worker fork, so the
  probe itself no longer holds the slot after reporting failed
- the lock test holds the slot with a real local analyze
- token and GitLab tests wait for a free slot before posting from the UI
- request fetches carry a timeout; teardown signals the serve process group
- an empty FRONTEND_URL falls back to the default base URL

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* Address PR review feedback (#3348)

- ops view: a `?server=` link no longer auto-connects to another origin
  while a deploy token is held; it prefills and waits for Connect
- analyze completion: a local-path run reconnects by the path this client
  submitted, so duplicate basenames stay collision-safe without repoPath
  on the public SSE frame
- stale slot cleanup: revalidate each nested directory (lstat + realpath)
  right before readdir, so a mid-cleanup junction swap aborts instead of
  walking an outside tree; list phases run sequentially so the slot-I/O
  cap is global
- e2e: 429 backoff honours the caller deadline; the unreachable-backend
  ops test navigates through the resolved frontend URL
- drop the unused `repo` member from the clean integration helper

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* feat(server): reconnect after analyze by an opaque repo id

Public job views and the SSE terminal frame no longer carry repoPath, and
repoName is not unique, so a post-analyze reconnect by name could load a
same-named sibling. The server now issues `repoId`: an HMAC of the
canonical registry path under a per-process random key. It is set on
complete jobs (ops view, analyze poll, SSE terminal frame) and matches
the new `id` on `GET /api/repos` entries. The web client resolves it to
the exact entry path on completion; unknown ids fall back to the name.
This covers URL clones and folder uploads, and replaces the local-path
only fallback.

With reconnect off the label, public `repoName` for a branch-pinned URL
clone is the repository name, not the `<repo>__<branch slug>` registry
name, so the requested branch stays off the unauthenticated feed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* Address PR review feedback (#3348)

- stale slot cleanup: a descendant that vanishes before its unlink is
  treated as removed instead of aborting the reclaim
- e2e teardown: escalate to SIGKILL on the process group when the live
  backend ignores SIGTERM for 5s
- ops view: drop the dead initial value CodeQL flagged

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* Address PR review feedback (#3348)

- public redaction: a known repoPath now also consumes its descendant
  tail, so `<repoPath>/src/secret.ts` becomes `[path]` instead of
  `[path]/src/secret.ts`; a same-prefix sibling is left to the path scrub
- e2e: the cancel test waits for the analyze slot the previous failed
  local-path job still holds; `fetchOps` carries the request timeout
- docs: SSE terminal payload and RepoAnalyzer `onComplete` describe
  `repoId` resolution

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* Address PR review feedback (#3348)

- RepoAnalyzer: drop a completion that resolves after unmount, so a slow
  /api/repos lookup cannot switch repos after the sheet was dismissed
- e2e: select the local-path input by test id (the placeholder differs on
  Windows); the slot probe's job wait honours the caller's deadline

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(server): assert the public SSE terminal frame in analyze-api

The #2790 terminality tests still expected `repoPath` on the terminal
frame. This PR replaced it with the opaque `repoId`, so assert that shape
and that the analyzed path never appears in the stream.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(web): wait for the analyze slot between duplicate-repo setup runs

The server now keeps the single analyze slot until the worker exits, even
after its job reports complete. repo-path-identity posted the second
duplicate's analyze immediately and got 409 in CI. Use the shared
slot-aware POST, which also waits out a 429.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Gergő Magyar <gergomagyar@icloud.com>
Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-23 13:27:22 +01:00
Gergő Magyar
620fd18a5c
feat(cli): reclaim leftover per-branch indexes after branch delete (#3338)
Some checks are pending
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Scorecard / Scorecard analysis (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
* feat(storage): classify leftover per-branch index slots

Operators need a shared enumerator for deleted-branch leftovers before clean --stale or doctor can reclaim or report them.

Co-authored-by: Cursor <cursoragent@cursor.com>

* feat(storage): reclaim a per-branch slot and empty branches/

Named clean --branch now shares one rm-then-registry helper so the last leftover slot can drop the empty branches directory, and a failed rm still keeps the retryable summary.

Co-authored-by: Cursor <cursoragent@cursor.com>

* feat(cli): add clean --stale to reclaim leftover branch indexes

Operators can drop per-branch slots whose recorded branch is gone without remembering each name, while a git-list failure stays a no-op.

Co-authored-by: Cursor <cursoragent@cursor.com>

* feat(cli): report leftover branch indexes in doctor

Operators can see cwd orphaned per-branch slots and their size, then reclaim them with clean --stale, without doctor deleting anything.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(review): keep unreadable branch slots out of stale reclaim

A stat error other than ENOENT/ENOTDIR must not look like a missing
directory, or clean --stale --force drops the registry row and leaves
the slot on disk.

Co-authored-by: Cursor <cursoragent@cursor.com>

* refactor(cli): keep leftover-slot display helpers in the CLI layer

Preview and doctor share one size formatter and an i18n path for
registry-only rows, so storage no longer owns display copy.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(review): keep leftover reclaim moving after a registry drop failure

Catch removeBranchIndex rejections so --stale continues, match doctor
registry rows through canonicalizePath, and size leftover slots sequentially.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(review): match leftover-slot registry rows with canonicalizePath

Use the repo-manager path contract so clean --stale and --branch still
see registry-only leftover rows when cwd and the stored path differ.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(review): contain leftover-slot deletes and re-check live heads

Refuse symlink and junction escapes under branches/, unlink slot links
instead of removing through them, and skip --stale --force when a name
is a local head again.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Address PR review feedback (#3338)

- Bound listLocalHeads spawnSync with GIT_PATH_LIST_MAX_BUFFER.
- Clarify that doctor leftover reporting is cwd-only, not registry-wide.
- Drop the MCP/serve assumption from clean --stale delete failures.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Address PR review feedback (#3338)

- Describe disk-only leftover slots in --stale help, not only recorded branches.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(cli): stop doctor reclaim copy when heads cannot be listed

Doctor was naming clean --stale for leftover rows even when git cannot
list local heads, which is a no-op. Print the retry-git message instead (#3337).

Co-authored-by: Cursor <cursoragent@cursor.com>

* revert: drop Unreleased changelog notes from this branch

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(cli): keep live branch pins when a tag shares the name

%(refname:short) disambiguates against tags, so clean --stale treated
still-local heads as leftover. Fail closed on obstructed slots and
unlistable branches/ directories.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Address PR review feedback (#3338)

Bound leftover-slot listing, revalidate paths immediately before delete, and keep registry rows when a stray disk-only directory claims a recorded branch.

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-20 16:03:18 +01:00
Gergő Magyar
ac9a4e9abd
fix(embeddings): keep ONNX off the install and analyze critical path (#3287)
Some checks are pending
Gitleaks / gitleaks (push) Waiting to run
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Scorecard / Scorecard analysis (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
* fix(embeddings): isolate local ONNX inference in a child_process sidecar

The analyze parent must not load onnxruntime-node. Fork a sidecar for
vectors only and reap it on worker exit; keep Ladybug writes in-process.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(embeddings): share the sidecar client across MCP, serve, and sync

Query hosts now use the core façade instead of a second in-process ONNX
embedder. Search skips an empty table, sync reaps beside closeLbug, and
ready means the stack is resolvable rather than a warm singleton.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(embeddings): refuse Intel Mac and unloadable prefix before npm heal

Analyze, sync, install, and the sidecar client now consult the platform
blocker before forking or downloading the optional stack. HTTP stays the
escape hatch; wasm is not treated as a rescue.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(embeddings): take the ONNX stack off default npm install

Pins live in gitnexusEmbeddingStack. embeddings install writes prefix
overrides before npm spawn. Leftover 1.6.12 package-first trees are residual.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(publish): drop grammar source from the published tarball

Every vendored grammar has 6/6 prebuilds, so files ships those plus
Leiden and FTS instead of parser.c. First ship stays above 80 MiB.

Co-authored-by: Cursor <cursoragent@cursor.com>

* test(embeddings): match MCP missing-stack warn to the R20 copy

Default install no longer calls the stack optional, so the once-per-backend
stderr assertion must look for the new lead line.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(review): bound sidecar death, cancel writes, and publish-file guards

Init-time native crashes no longer respawn a child on every query. Local
embedBatch honors AbortSignal after sidecar return, MCP query() surfaces
vector-lane degradation, disconnect always reaps, and the grammar prepack
guard checks files globs instead of on-disk prebuilds.

Co-authored-by: Cursor <cursoragent@cursor.com>

* refactor(embeddings): share runtime preflight and sidecar reap helpers

Analyze and embeddings-sync used the same blocker/prefix/install gate
with different error routing. One assessment keeps those paths aligned
without changing CLI vs thrown-error behavior.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Address PR review feedback (#3287)

Keep a reaped sidecar from resetting its replacement, wait for dispose,
tighten the publish-files guard, and stop assuming a leftover ONNX tree in CI.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Address remaining PR review feedback (#3287)

Clear the sidecar reap timeout, add init IPC slack, and isolate embeddings-sync tests from HTTP-mode env.

Co-authored-by: Cursor <cursoragent@cursor.com>

* test(embeddings): unstub globals after sidecar HTTP-mode tests

Keep a leaked fetch stub from failing assertions out of later tests in the same file.

Co-authored-by: Cursor <cursoragent@cursor.com>

* test(embeddings): pin sidecar success cases off darwin/x64

The runtime blocker reads the real process platform before the fork mock, so local-success tests must not inherit an Intel Mac host.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Address remaining PR review feedback (#3287)

Keep vector degradation per query, treat leftover Intel-Mac stacks as not ready, and document that the CLI image no longer ships ONNX.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Simplify embedding sidecar shutdown and search hot paths

Drop redundant sidecar reaps and unused child helpers, and run FTS alongside semantic search.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Address remaining PR review feedback (#3287)

Share HF attempt parsing with the sidecar init deadline, abort embed waits without killing the child, and restore last init options on recreate.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Address remaining PR review feedback (#3287)

Treat sub-1 HF attempt env values as invalid, and drop leaked sidecar waiters when IPC send throws.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Address remaining PR review feedback (#3287)

Keep sidecar init on a shared chain; each waiter can abort only its own wait.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(ci): declare embedding-table existence probe as unordered LIMIT

The empty-table skip in semanticSearch is existence-only; declare it so the #2787 determinism guard stops failing coverage shard 3/3.

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-15 12:11:47 +01:00
Gergő Magyar
21a52af1d4
fix(lbug): ship FTS per-platform and recover in-place native aborts (#3274)
* fix(lbug): pin Ladybug core so Dependabot cannot ship a skewed FTS artifact

The extension version is a separate upstream constant. Ignore daily core bumps and fail the pairing gate when the committed manifest does not name the installed core.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(lbug): make doctor and CI FTS gates resolve the packaged artifact

Doctor and the REQUIRE_FTS file gates still treated an empty ~/.lbdb as
unavailable, which would turn three CI jobs red once analyze stops
installing into that tree.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(lbug): name native-abort and tuple-missing so analyze cannot mis-advise

The CLI summary's trailing else treated every unknown skip reason as a
missing extension. New crash and platform causes must get their own
remedies, not a network-install hint.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(lbug): delete the dead read-path FTS index create

ensureFTSIndex had no production callers and swallowed read-only
CREATE_FTS_INDEX failures, which hid the only signal that a reader
tried to write.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(lbug): vendor per-platform FTS artifacts so analyze needs no host install

Keyword search depended on a CDN fetch into ~/.lbdb. Shipping the five
published tuples inside the package makes air-gapped and ignore-scripts
installs load the same artifact the publish gate checksums.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(lbug): load the packaged FTS artifact before any network install

Analyze still required a CDN fetch into ~/.lbdb even when the package
already shipped the file. FTS now path-loads the vendored tuple first
and records source labels so a later truncated home copy cannot steal
the diagnosis.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(lbug): diagnose a core/extension version skew instead of a missing runtime

A structurally valid FTS artifact whose path version disagrees with the
packaged pin must name both versions, not prescribe VC++ or OpenSSL.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(lbug): stamp an FTS phase so repair stays usable after an in-place abort

A native CREATE_FTS_INDEX abort leaves no skip reason; the next run infers
it from the dirty flag, and --repair-fts must not treat that phase as a
half-written graph.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(lbug): park an in-place FTS crash WAL without wiping the graph

An FTS abort after a successful checkpoint must reopen the live index on
macOS, Windows, and Linux. Staging never parks the live WAL; readers keep
today's large-WAL refusal.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(lbug): refuse read-only opens of an FTS-poisoned WAL

MCP and serve cannot repair a leftover in-place abort. Fail before the
native open and name --repair-fts, on macOS, Windows, and Linux.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(lbug): name a vendor-neutral Windows OpenSSL prerequisite

OQ1 is unanswered here so GitNexus does not ship OpenSSL DLLs. Windows
FTS now asks for a system OpenSSL 3 runtime instead of Git Bash PATH.

Co-authored-by: Cursor <cursoragent@cursor.com>

* test(lbug): inject the FTS vendor root and redact it on HTTP and MCP

Path-loaded artifacts no longer vary with HOME. Tests pass an injected
vendor tree and assert search warnings never leak a filesystem path.

Co-authored-by: Cursor <cursoragent@cursor.com>

* docs(lbug): document load-only as the global FTS install default

Analyze still overrides to auto. Packaged per-platform artifacts load
before any network install on macOS, Windows, and Linux.

Co-authored-by: Cursor <cursoragent@cursor.com>

* docs(lbug): format the FTS install-policy README table

Prettier does not run on Markdown in pre-commit, so the U10 table wrap
needs its own formatting commit.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(lbug): skip FTS CREATE after a persisted native abort

A recovered analyze run was retrying CREATE_FTS_INDEX from skipReason
alone. Keep that skip until --repair-fts, fail closed on unsupported
tuples, and honor the checkpoint warrant for park/repair.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(lbug): honor checkpoint flushed warrant and align FTS tests with packaged vendor

A no-op CHECKPOINT must not satisfy the FTS park warrant, and CI still asserted HOME-only FTS isolation after analyze started path-LOADing the packaged artifact.

Co-authored-by: Cursor <cursoragent@cursor.com>

* test(lbug): accept a nonempty incremental write set in the #2790 recovery check

FTS-phase recovery can incremental-add files (changed=0, added=1). That is not the #2790 empty-diff wipe skip.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(lbug): compare FTS home versions to the core pin and tighten the publish filename gate

Ladybug's ~/.lbdb/extension directory is the runtime/core version; treating it as the artifact version false-diagnosed skew. The publish guard now rejects a path-escaping filename the same way the fetch script does.

Co-authored-by: Cursor <cursoragent@cursor.com>

* test(lbug): seed FTS e2e fixtures from the packaged vendor artifact

A machine with no ~/.lbdb copy should still run the vendor-survivorship cases; the seed no longer depends on HOME or a network install.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Address PR review feedback (#3274)

Keep in-place FTS abort evidence after persist so a second CREATE abort
cannot fail-open readers, and close the CLI, loader, embed, and e2e gaps
the review called out.

Note: full npm test hit Ladybug worker-pool startup failures under memory
pressure; tsc and 180 targeted unit tests passed.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Address PR review feedback (#3274)

Run the vendored-path symlink guard on the OS matrix, put e2e HOME
fixtures on Ladybug's real extension layout, pin the embed crash-WAL
gate before the writable open, and let analyze writers park through
missing-shadow recovery.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(lbug): keep --repair-fts CI green after vendored-first FTS

Never-installed warning fixtures must not inspect a packaged vendor binary, and a failed dirty restamp must not abort an otherwise successful --repair-fts run.

Co-authored-by: Cursor <cursoragent@cursor.com>

* test(cli): give the #1169 analyze e2e the same 90s Windows budget as its sibling

The first #1169 persist-meta case was still on a 60s spawn/it budget and was killed banner-only on windows-latest after the FTS warning fixture no longer failed the shard first.

Co-authored-by: Cursor <cursoragent@cursor.com>

* test(ci): reweight Windows shards after the FTS e2e grew

Vendored-first HOME fixtures pushed fts-extension-e2e to ~6 minutes on windows-latest, so the old 146s weight packed it with skills-e2e and blew the 20-minute watchdog.

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-14 08:52:24 +01:00
EVA
25974caf05
fix(doctor): distinguish vector capability from repository index state (#3228)
Co-authored-by: Eva <eva@100yen.org>
Co-authored-by: Gergő Magyar <gergomagyar@icloud.com>
2026-09-09 13:06:41 +01:00
drdave
021ac30376
feat(cli): add a bunx lane so bun-only machines can run gitnexus (#2765)
Some checks failed
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Scorecard / Scorecard analysis (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
Skill copy sync / shipped skills drift guard (push) Has been cancelled
* feat(cli): add a bunx lane to the runner ladder

The ladder assumed a Node toolchain: global gitnexus, then pnpm dlx or
npx in some order, with npx as the last resort. On a bun-only machine
npm, npx and pnpm are all absent, so every rung fell through to npx and
both the emitted hint and the generated .gitnexus/run.cjs produced a
command the machine could not run at all.

Add bun as a fourth mode, invoked as an install-free bunx one-shot, on
two rungs:

  - npm 11+ with no pnpm to fall back on — bunx dodges the same arborist
    install crash the pnpm rung exists for (#1939);
  - npm and pnpm both absent — previously the dead end described above.

Every pre-existing outcome is preserved: pnpm still wins on npm 11+, npx
still wins on npm < 11, and pnpm still wins over bunx when npm is absent.
Regression tests pin each of those. The bun PATH probe is lazy, so a
machine with a Node toolchain pays no extra scan and the stale-index hook
budget is unchanged.

bunx takes no allow-build equivalent: bun's --trust is a bun add/install
flag that writes trustedDependencies into a project package.json, which a
one-shot has none of, so the argv stays flag-free.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016R9psS9gJ73MRyquoBoPKg

* fix(lbug): restore the prebuilt native binary when install scripts were skipped

Without this the new bunx lane resolves to a command that still fails:
bun skips lifecycle scripts for a bunx fetch, so @ladybugdb/core's
install script never copies lbugjs.node up from its per-platform
sub-package and every native command dead-ends on 'LadybugDB native
binary (lbugjs.node) is missing'.

The existing guidance cannot rescue that case. It offers pnpm
--allow-build, a global install, or adding trustedDependencies to a
project package.json — bunx has no project package.json to add to, no
per-invocation opt-in, and re-extracts the package on every run, so an
out-of-band repair is wiped before the next invocation. In-process
recovery is the only thing that can work.

Recovery is cheap because nothing is actually absent: the binary is
already on disk in @ladybugdb/core-<platform>-<arch>, and the skipped
script only copied it up. Redo that copy (prebuilt only — never a source
build, never a network fetch) before reporting failure. Best-effort by
construction: read-only node_modules, an absent sub-package or an
unsupported platform all fall through to the existing diagnostics
unchanged, which a test pins.

Also covers pnpm dlx without --allow-build and npm --ignore-scripts.

Declare trustedDependencies so a plain `bun install` in this repo
produces a working native binary too — the remedy the error message
already prescribes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016R9psS9gJ73MRyquoBoPKg

* fix(ai-context): name every install-free runner in the generated bootstrap note

The emitted gitnexus:start block told a reader with no runner yet to run
`npx gitnexus analyze`, falling back to a global npm install. Both name
binaries a bun-only machine does not have, so the generated AGENTS.md and
CLAUDE.md offered it no reachable bootstrap path.

List npx, bunx and pnpm dlx instead of resolving one. The block is
committed, so emitting the command this machine happens to resolve would
make two contributors on different package managers rewrite it at each
other on every analyze — the per-machine churn #1706 removed. Naming all
three keeps the note machine-independent and correct everywhere.

Regenerates this repo's own committed block to match.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016R9psS9gJ73MRyquoBoPKg

* fix(cli): address PR #2765 review — bunx liveness, restore diagnostics, docs

Addresses all five review comments on #2765.

P1 — `hasBun()` was a PATH-existence check only, so a present-but-broken
`bunx` shim (partial uninstall, failed `bun upgrade`) was selected with no
functional validation. Because selecting `bun` also suppresses the npm-11
npx-crash warning, the result was a silent dead end: no diagnostic, and a
`bunx gitnexus@latest analyze` command that only fails at execution time.
Add `probeRuns()` — a real `bunx --version` liveness probe, gated behind the
cheap spawn-free PATH scan so machines with npm/pnpm still pay nothing. It
ignores the output on purpose (a banner or unparseable version still counts
as alive); only a spawn failure, non-zero exit, or timeout rejects. Injectable
via a new `bunRuns` dep so the mode tests stay host-independent.

P2 — the `gitnexus-cli` skill (and both shipped mirrors) still described the
pre-bunx ladder, stranding exactly this PR's audience: a bun-only machine
whose agent bootstraps from that file was told to use npx/npm/pnpm, none of
which exist there. All three copies now name `bunx` in the ladder and the
bootstrap fallback, with a `shipped-skills-sync` fragment assertion so the
gap is CI-caught (these copies are not byte-compared, only the engineering
family is).

P2 — `restorePrebuiltNativeBinary` collapsed every failure into `false`, so an
EACCES/EROFS from `copyFileSync` was indistinguishable from "no prebuilt
sub-package exists". Users on a read-only `node_modules` layer (a baked
container image mounted read-only — a common CI pattern) got the generic
lifecycle-script advice, which cannot fix a non-writable filesystem. Return a
`RestoreOutcome` instead and route `copy-failed` to its own message.

P2 — document that `trustedDependencies` only takes effect for `bun install` /
`pnpm install` run inside this repo: it does nothing for a `bunx` one-shot or
for a consumer's `bun add gitnexus`. The note sits on
`restorePrebuiltNativeBinary` so a future maintainer cannot mistake that
function for redundant and delete the thing the bunx path actually relies on.

P3 — the `binary_missing` bun advice told `bunx` one-shot users to edit a
package.json they do not have, and listed 1 of the 3 packages this package
now trusts. Both repair messages now share one `BUN_REPAIR_LINES` const with
the full package list and a `bun install -g gitnexus` alternative.

Also: shortened the bootstrap note and raised the CLAUDE.md block budget
2900 -> 2950. The note has to name every install-free runner (that is the
point of the bun lane), and main's own growth since this PR's last green CI
had already pushed the generated block over the old ceiling.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015epfxkEMsmHFNVSFQqAkB4

* refactor(cli): simplify the #2765 review fixes

Cleanup pass over the previous commit — no intended behavior change except
the doctor status line noted below.

Reuse: `probeRuns()` duplicated `probeVersion()`'s entire spawn setup — same
argv, timeout, `windowsHide`, and the CVE-2024-27980 Windows-shim workaround —
in a file with two byte-identical committed copies, so the shim rule lived at
four sites. Its docstring's own objection was to the RETURN SHAPE, not to
reuse, so `probeVersion` now returns `{ ran, major, minor }` and `hasBun` reads
`.ran`. Existing callers only read `major`/`minor`, so nothing else changes.

Also dropped a pointless `const runs = () => …` thunk (`&&` already
short-circuits), and deleted a new test that was a character-for-character
duplicate of `falls back to npx when npm is null-absent and pnpm is also
absent` — its cheapest-first-gate rationale moved into that test's comment.

Correctness in the budget comment: the claim that the bun rung is free because
"pnpm is absent there, so its probe never ran" was wrong. `formatAnalyzeCommand`
spawns `pnpm --version` unconditionally when no global `gitnexus` is on PATH —
that spawn IS how pnpm presence is discovered. Real worst case is 5 subprocesses
/ ~8s, and the 8s needs Windows (`shell: true` spawns cmd.exe for an absent
pnpm); on POSIX an absent pnpm ENOENTs in ~1ms. Comment now says that. Likewise
"a machine with npm or pnpm never pays" was wrong for npm 11+ without pnpm —
that IS the rung that pays.

Altitude: `copy-failed` changed only the message text while still returning
`kind: 'binary_missing'`, so `doctor` would have printed "✗ lbugjs.node missing"
directly above a message saying the binary IS present — exactly the
contradiction #2672 removed. Added a `binary_unwritable` kind, a doctor case,
and a `nativeStatusCases` row. The binary-missing message construction moved
out of `checkLbugNative` into `unrestorableBinaryFailure`, typed
`Exclude<RestoreOutcome, 'restored'>` so a new outcome forces a decision
instead of silently inheriting the lifecycle-script advice.

Drift: the trusted-package list was hand-spelled in five places in
native-check.ts, with "matches gitnexus/package.json" asserted only in a
comment. All five now render from one `NATIVE_BUILD_PACKAGES` const (rendered
output is byte-identical), and the test reads the list out of package.json
instead of restating it, so a fourth native package fails the test rather than
silently shipping stale advice.

Finally, replaced the absolute CLAUDE.md block cap with the ratio the two prior
justifications actually appealed to (`< 5465 * 0.55`). Raising 2700 -> 2900 ->
2950 was a ratchet with no ratchet: an absolute cap can only fail on the PR
that adds the character, and the fix is always to nudge the number. Also fixed
a stale runner ladder in skills-steering.test.ts that still omitted bunx.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015epfxkEMsmHFNVSFQqAkB4

---------

Co-authored-by: drdave-flexnteos <revenaugh.david@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Gergő Magyar <gergomagyar@icloud.com>
Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
2026-08-06 08:40:36 +00:00
Gergő Magyar
3f1e23ba83
fix: stop misdiagnosing glibc-too-old native loads (#2672) and name the Windows FTS zero-install fix (#2669) (#2689)
* docs(plans): add glibc-windows-fts-diagnostics plan

Implementation plan for #2672 (glibc-too-old native-load misdiagnosis)
and #2669 (Windows FTS prerequisites + Git Bash zero-install workaround).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(cli): stop prescribing a reinstall when the host glibc is too old (#2672)

The LadybugDB prebuilt binary requires GLIBC_2.34 (dlopen/pthread_* at 2.34,
fstat64/lstat at 2.33). On an older host the loader reports

  version `GLIBC_2.34' not found (required by .../lbugjs.node)

and checkLbugNative answered with "truncated file, ABI mismatch, or
wrong-platform binary" plus instructions to re-run install.js. That advice is
actively wrong for this class: every download ships the same prebuilt binary,
so the reinstall fails identically and the user loops.

Add glibcTooOldMessage: match a GLIBC_<version> token on a "not found" line,
report the highest required version (compared numerically, so 2.9 < 2.34)
alongside this host's glibc from process.report, state that reinstalling will
NOT help, and point at the real options. The branch sits on the arm where the
probe actually ran and failed, so an unrunnable probe still fails open (#2441).

The glibc read is local rather than analyzer-identity's detectLibcVariant:
native-check is the dependency-light startup gate and must not statically pull
in a module the CLI reaches through a dynamic import.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(lbug): name the Git Bash zero-install fix for Windows FTS load failures (#2669)

The Windows error-126 remedy already refuses to prescribe a reinstall and names
the VC++ redistributable and the OpenSSL 3 DLLs, but not where those DLLs
already exist on the machine. #2669's reporter had the redistributable
installed and still failed: the same command failed in PowerShell and succeeded
in Git Bash, because Git for Windows puts libssl-3-x64.dll and
libcrypto-3-x64.dll on PATH via C:\Program Files\Git\mingw64\bin.

Add that hint to the Windows-126 and structural missing-dependency remedies
through one shared const, following the VC_REDIST_INSTALL_HINT anti-drift
pattern (#2383 F5). Placing it in the builders rather than at a call site is
load-bearing: markUnavailable caches the whole diagnosis (#2383 F3) and
ftsDegradedWarning replays that cached remedy, so a call-site fix would miss
the MCP query and /api/search surfaces.

The hint is a fixed system path, never a user-profile one — remedy text is not
path-redacted, and fts-degraded-warning.test.ts asserts no C:\Users\ path ever
reaches a user. Both touched tests now assert that property directly.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs(readme): document the Linux glibc floor and Windows FTS prerequisites (#2672, #2669)

Requirements listed only Node and git, so neither runtime prerequisite that
these two issues turn on was discoverable before hitting the failure.

- Linux: the LadybugDB prebuilt binary needs glibc 2.34+; name the distro
  versions that clear it and state plainly that reinstalling does not help.
- Windows: full-text search needs the VC++ 2015-2022 x64 redistributable AND
  OpenSSL 3 on PATH. The redistributable alone is not sufficient (#2669's
  reporter had it), and Git for Windows already ships the OpenSSL DLLs, so
  running from Git Bash or prepending mingw64\bin is a zero-install fix.
  Without them analyze still succeeds but the index carries no search tables.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* chore: drop the plan document from version control

docs/* is gitignored; the plan was force-added so it would travel with the
work. It is working material, not a repository artifact — the code, tests and
README carry the reasoning that matters.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(cli): stop doctor reporting a present-but-unloadable binary as missing (#2672)

doctor printed "✗ lbugjs.node missing" for every failed native check — including
the case this PR is about, where the binary is right there and merely fails to
load because the host glibc is too old. It then wrote the real detail to stderr
directly beneath, so the two lines contradicted each other and the headline sent
users to reinstall a file they already had. It said the same for a truncated
download and for an entirely absent @ladybugdb/core package.

checkLbugNative already knows which of the three it found, so record it: a
`kind` discriminator ('package_missing' | 'binary_missing' | 'load_failed') set
at each failure return. doctor renders it through a new exported
`nativeStatusLine`, following the existing pageSizeDoctorLines/poolSizeDoctorLine
pure-helper pattern — which also makes the line testable, where before it had no
coverage at all. An unrecognized or absent kind keeps the conservative "missing".

Deriving this in doctor with a second existsSync would have re-stat'd a file the
check had already inspected, and could disagree with what it actually observed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 10:05:57 +01:00
Gergő Magyar
9538be957d
fix(lbug): scale the buffer-pool budget by the OS page-size granule ratio (#2631) (#2636)
* fix(lbug): scale the buffer-pool budget by the OS-page discard-granule ratio (#2631)

LadybugDB bills buffer-pool budget per discard granule, not per 4 KiB frame:
the engine's vm_region.cpp sets discardGranuleSize = max(frameSize, osPageSize),
claimFrame charges the whole granule when its first frame becomes resident, and
releaseFrame refunds only when the granule's last frame leaves — while
BufferManager::reserve measures eviction progress in refunded bytes and throws
'The buffer pool is full and no memory could be freed!' after three zero-refund
passes. On a 64 KiB-page kernel (Ascend/aarch64 openEuler — the #2631
reporter's host) that is 16 frames per granule: the same COPY bills up to 16×
the budget it needs on x86, and whole eviction passes can evict frames yet
refund nothing. Apple Silicon macOS (16 KiB pages) is the same mechanism at 4×.

Measured with the reporter's exact command and version: vllm-ascend needs a
(128, 256] MiB pool on 4 KiB pages — 64/128 MiB reproduce the reporter's
byte-identical error, 256 MiB and the 576 MiB adaptive pool succeed — so their
64 KiB host cannot survive on a page-size-blind budget.

Scale every derived pool size by granuleRatio = max(1, osPageSize/4096):
the per-element estimate, the COPY-safety floor, and the default cap (still
bounded by 80% of RAM). 4 KiB hosts are byte-identical to before — proven by
pinning the existing sizing tests to an explicit 4096 page size, which also
stops them drifting on 16 KiB Apple Silicon runners. GITNEXUS_LBUG_BUFFER_POOL_SIZE
keeps absolute precedence and 0 still restores the native default.

Also: bufferPoolExhaustionRemedy() gives the exhaustion error an actionable
cause→consequence→remedy message; the isLbugPageSizeFrameError comment that
called pool exhaustion 'a sizing problem, not a page-size one' is corrected —
that framing inverted when #2582 made pool size a function of a page-size-blind
estimate. Cannot execute on a 64 KiB kernel here: the scaled path is proven by
unit stubs plus the engine-source math above; the env override remains the
field escape hatch.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(cli): actionable pool-exhaustion remedies at the COPY sites and a doctor pool line (#2631)

The node-COPY throw and the relationship-COPY warning now append
bufferPoolExhaustionRemedy() when the failure is the engine's pool-exhaustion
class: the raw binder text gave the operator nothing to act on, and on
non-4K-page hosts the pool bills up to pageSize/4KiB × faster than the sizing
was calibrated for. The relationship path appends the remedy once per bulk
load, not once per failed pair. doctor prints the effective pool size next to
the page-size line ('pool size 2048 MiB', with an '(×N page-size scaling)'
suffix on non-4K hosts) so support triage sees the sizing inputs at a glance.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(lbug): re-anchor getEffectiveBufferPoolSize's placement and reuse granuleRatio in doctor

Self-review fixes: the getter's insertion had orphaned resolveBufferManagerSize's
doc comment (it read as documenting the wrong function), and doctor's scale note
duplicated the granule math with a hardcoded 4096. granuleRatio is now exported
(it already carried the test-seam default param) and doctor consumes it.
No behavioral change — the sizing suite pins byte-identical outputs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(lbug): keep the hintless pool default unscaled and make both remedies visible (#2631)

Review fixes:
- Scale only the analyze-path cap (scaledAnalyzePoolCap), not
  defaultBufferPoolSize: the pool is an eager native allocation at DB open
  (measured, see POOL_BYTES_PER_ELEMENT), so a page-size-scaled hintless
  default would hand a long-lived MCP process up to 80% of RAM — the #2557
  OOM exposure the 2 GiB cap removed. Fix the MAP_NORESERVE claim that
  contradicted that measurement.
- Log the rel-pair pool remedy (loadGraphToLbug returns warnings that no
  call site reads) and dedup it with a local boolean instead of matching
  the remedy's own wording.
- Label the GITNEXUS_LBUG_BUFFER_POOL_SIZE=0 sentinel as the native
  80%-of-RAM default in both the remedy and doctor instead of '0 MiB'.
- Extract poolSizeDoctorLine (pageSizeDoctorLines convention): mark env
  overrides, drop the scaling suffix that misdescribed absolute values.
- Fold _resetOsPageSizeCacheForTest into _setOsPageSizeForTests(undefined).
- Document the analyze-path scaling in both README env tables.

---------

Co-authored-by: Gergo Magyar <abhigyan1.patwari@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 20:09:48 +01:00
VL
117587d543
fix(cli): actionable diagnostics for non-4K page-size buffer manager failures (#2424)
Some checks are pending
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Scorecard / Scorecard analysis (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
2026-07-10 19:12:53 +01:00
Gergő Magyar
a7a5ea65a6
fix: report custom HTTP embedding endpoint failures instead of huggingface download errors (#2385) (#2386) 2026-07-06 22:06:08 +01:00
Gergő Magyar
cdad478c96
fix: proxy-blocked installs survive onnxruntime-node postinstall and self-heal embeddings (#2370) (#2372)
Some checks are pending
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Scorecard / Scorecard analysis (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
2026-07-05 16:15:10 +01:00
Gergő Magyar
fca3494807
fix(embeddings): guard local ONNX runtime on macOS Intel before transformers.js import (#1987)
Some checks are pending
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Scorecard / Scorecard analysis (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
* fix(embeddings): guard local ONNX runtime on macOS Intel before transformers.js import

macOS Intel (darwin/x64) crashed on `gitnexus analyze --embeddings` with a raw
`Cannot find module .../bin/napi-v6/darwin/x64/onnxruntime_binding.node`: both
embedders imported @huggingface/transformers at module scope, which loads
onnxruntime-node and resolves the (unshipped) native binding before any backend
could be selected. ONNX_WEB_BACKEND=wasm could not help (#1516).

- Add a native-free runtime-support guard (getLocalEmbeddingRuntimeBlocker) that
  returns a clear, actionable message on darwin/x64 and null elsewhere.
- Convert both the core and MCP embedders to type-only transformers imports plus
  a guarded lazy `await import()`; throw the blocker in initEmbedder before any
  transformers.js / onnxruntime-node resolution. HTTP mode is unaffected.
- Surface the blocker cleanly in the analyze CLI instead of the misleading
  "installation may be corrupt" module-not-found hint.
- Add unit tests: guard DI, lazy-import timing, core+MCP darwin/x64 rejection,
  and HTTP mode not blocked.

Refs #1515, #1516

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(doctor): surface macOS Intel local-embedding limitation

`gitnexus doctor` now reports whether the local embedding runtime can load on
the current platform. macOS Intel (darwin/x64) users see up front that local
embeddings are unavailable — plus the recommended alternatives — instead of
only discovering it when `analyze --embeddings` fails (#1515).

The Embeddings section gains a "Support" line; on a blocked platform the full
guidance (reused from getLocalEmbeddingRuntimeBlocker, single source of truth)
is written to stderr. doctor stays import-safe — it never loads transformers.js
or onnxruntime-node, so it runs cleanly on macOS Intel.

Refs #1515

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(embeddings): close #1515 guard coverage gaps + PR #1987 review polish

Resolves the maintainer tri-review feedback on PR #1987:

- Add the analyze error-branch test (new analyze-local-embedding-error.test.ts):
  a darwin/x64 blocker routes to the clean local-embedding-unsupported message
  (exit 1), not the module-not-found "installation may be corrupt" branch, and
  wins over isHfDownloadFailure even when both match (guards the reorder below).
- Cover the MCP embedQuery darwin/x64 paths — HTTP bypass via httpEmbedQuery
  without importing transformers, and local-mode rejection before the import.
- Make the "defaults platform/arch" guard test falsifiable by stubbing the
  platform, instead of asserting null === null on the CI host.
- analyze.ts: evaluate the blocker-message branch before the network-heuristic
  isHfDownloadFailure branch so the explicit platform message takes priority.
- runtime-support.ts: the blocker message now also notes GITNEXUS_EMBEDDING_DEVICE
  =wasm/cpu cannot help, not only ONNX_WEB_BACKEND=wasm.
- doctor.ts: resolve platform/arch once instead of re-resolving after the guard.

Refs #1515, #1516

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-03 09:46:16 +01:00
ChamHerry
fc6007e70b
feat(i18n): make web and CLI language-aware (#1748) 2026-05-23 06:14:24 +01:00