mirror of
https://github.com/abhigyanpatwari/GitNexus.git
synced 2026-09-15 23:32:49 +00:00
Some checks are pending
Gitleaks / gitleaks (push) Waiting to run
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Scorecard / Scorecard analysis (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
* fix(embeddings): isolate local ONNX inference in a child_process sidecar The analyze parent must not load onnxruntime-node. Fork a sidecar for vectors only and reap it on worker exit; keep Ladybug writes in-process. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(embeddings): share the sidecar client across MCP, serve, and sync Query hosts now use the core façade instead of a second in-process ONNX embedder. Search skips an empty table, sync reaps beside closeLbug, and ready means the stack is resolvable rather than a warm singleton. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(embeddings): refuse Intel Mac and unloadable prefix before npm heal Analyze, sync, install, and the sidecar client now consult the platform blocker before forking or downloading the optional stack. HTTP stays the escape hatch; wasm is not treated as a rescue. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(embeddings): take the ONNX stack off default npm install Pins live in gitnexusEmbeddingStack. embeddings install writes prefix overrides before npm spawn. Leftover 1.6.12 package-first trees are residual. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(publish): drop grammar source from the published tarball Every vendored grammar has 6/6 prebuilds, so files ships those plus Leiden and FTS instead of parser.c. First ship stays above 80 MiB. Co-authored-by: Cursor <cursoragent@cursor.com> * test(embeddings): match MCP missing-stack warn to the R20 copy Default install no longer calls the stack optional, so the once-per-backend stderr assertion must look for the new lead line. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(review): bound sidecar death, cancel writes, and publish-file guards Init-time native crashes no longer respawn a child on every query. Local embedBatch honors AbortSignal after sidecar return, MCP query() surfaces vector-lane degradation, disconnect always reaps, and the grammar prepack guard checks files globs instead of on-disk prebuilds. Co-authored-by: Cursor <cursoragent@cursor.com> * refactor(embeddings): share runtime preflight and sidecar reap helpers Analyze and embeddings-sync used the same blocker/prefix/install gate with different error routing. One assessment keeps those paths aligned without changing CLI vs thrown-error behavior. Co-authored-by: Cursor <cursoragent@cursor.com> * Address PR review feedback (#3287) Keep a reaped sidecar from resetting its replacement, wait for dispose, tighten the publish-files guard, and stop assuming a leftover ONNX tree in CI. Co-authored-by: Cursor <cursoragent@cursor.com> * Address remaining PR review feedback (#3287) Clear the sidecar reap timeout, add init IPC slack, and isolate embeddings-sync tests from HTTP-mode env. Co-authored-by: Cursor <cursoragent@cursor.com> * test(embeddings): unstub globals after sidecar HTTP-mode tests Keep a leaked fetch stub from failing assertions out of later tests in the same file. Co-authored-by: Cursor <cursoragent@cursor.com> * test(embeddings): pin sidecar success cases off darwin/x64 The runtime blocker reads the real process platform before the fork mock, so local-success tests must not inherit an Intel Mac host. Co-authored-by: Cursor <cursoragent@cursor.com> * Address remaining PR review feedback (#3287) Keep vector degradation per query, treat leftover Intel-Mac stacks as not ready, and document that the CLI image no longer ships ONNX. Co-authored-by: Cursor <cursoragent@cursor.com> * Simplify embedding sidecar shutdown and search hot paths Drop redundant sidecar reaps and unused child helpers, and run FTS alongside semantic search. Co-authored-by: Cursor <cursoragent@cursor.com> * Address remaining PR review feedback (#3287) Share HF attempt parsing with the sidecar init deadline, abort embed waits without killing the child, and restore last init options on recreate. Co-authored-by: Cursor <cursoragent@cursor.com> * Address remaining PR review feedback (#3287) Treat sub-1 HF attempt env values as invalid, and drop leaked sidecar waiters when IPC send throws. Co-authored-by: Cursor <cursoragent@cursor.com> * Address remaining PR review feedback (#3287) Keep sidecar init on a shared chain; each waiter can abort only its own wait. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(ci): declare embedding-table existence probe as unordered LIMIT The empty-table skip in semanticSearch is existence-only; declare it so the #2787 determinism guard stops failing coverage shard 3/3. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com>
136 lines
7.5 KiB
Text
136 lines
7.5 KiB
Text
ARG BUILDPLATFORM
|
|
ARG TARGETPLATFORM
|
|
# Pinned npm version used to replace the bundled npm in the upstream Node
|
|
# image. Bumping requires a coordinated update in Dockerfile.web and
|
|
# gitnexus/Dockerfile.test so all images bootstrap the same npm.
|
|
ARG NPM_VERSION=11.14.1
|
|
|
|
# -- Builder -----------------------------------------------------------
|
|
# Native modules (tree-sitter-*, node-gyp builds for
|
|
# tree-sitter-proto / tree-sitter-swift) require python3 + a C/C++ toolchain.
|
|
# onnxruntime-node is not installed by `npm ci`; local embeddings need
|
|
# `gitnexus embeddings install` (or HTTP env / a bind-mounted prefix). The
|
|
# runtime stage strips npm, so this image cannot auto-heal the stack.
|
|
# node:22-bookworm-slim
|
|
FROM node:22-bookworm-slim@sha256:9f6d5975c7dca860947d3915877f85607946403fc55349f39b4bc3688448bb6e AS builder
|
|
ARG NPM_VERSION
|
|
|
|
WORKDIR /app
|
|
|
|
RUN npx --yes npm@${NPM_VERSION} install -g npm@${NPM_VERSION}
|
|
|
|
# Toolchain for node-gyp / native builds.
|
|
RUN apt-get update && apt-get install -y --no-install-recommends python3 make g++ git && rm -rf /var/lib/apt/lists/*
|
|
|
|
# Build gitnexus-shared first - gitnexus depends on it as a workspace.
|
|
COPY gitnexus-shared/package.json gitnexus-shared/package-lock.json ./gitnexus-shared/
|
|
RUN npm ci --prefix gitnexus-shared
|
|
COPY gitnexus-shared ./gitnexus-shared
|
|
RUN rm -f gitnexus-shared/tsconfig.tsbuildinfo
|
|
RUN npm run build --prefix gitnexus-shared
|
|
|
|
# Copy the full gitnexus package before installing - `npm ci` triggers
|
|
# `postinstall` (patches tree-sitter-swift, builds the vendored
|
|
# tree-sitter-proto) and `prepare` (compiles TypeScript via scripts/build.js),
|
|
# both of which need the source tree.
|
|
COPY gitnexus ./gitnexus
|
|
RUN npm ci --prefix gitnexus
|
|
|
|
# Drop dev dependencies for a smaller runtime layer.
|
|
RUN npm prune --omit=dev --prefix gitnexus
|
|
|
|
# `npm prune` removes anything not in package.json's dependency tree — which
|
|
# includes the VENDORED tree-sitter grammars (materialized into node_modules/ by
|
|
# postinstall, but not declared as deps) and their freshly-built native bindings.
|
|
# The `serve` image analyzes/parses uploaded repos at runtime, so those grammars
|
|
# must survive into the runtime layer. Re-run the grammar postinstall here in the
|
|
# builder (which still has python3/make/g++ and the hoisted node-addon-api /
|
|
# node-gyp-build) to re-materialize + rebuild them after the prune. This is
|
|
# load-bearing for tree-sitter-c (a core, REQUIRED grammar now vendored, #2116):
|
|
# as a former `dependency` it used to survive prune; vendored, it would not.
|
|
RUN npm run postinstall --prefix gitnexus
|
|
|
|
# -- Runtime -----------------------------------------------------------
|
|
# node:22-bookworm-slim
|
|
FROM node:22-bookworm-slim@sha256:9f6d5975c7dca860947d3915877f85607946403fc55349f39b4bc3688448bb6e AS runtime
|
|
|
|
# curl for the healthcheck; git for cloning; procps for watch process identity;
|
|
# ca-certificates for TLS verification.
|
|
RUN apt-get update && apt-get install -y --no-install-recommends curl git procps ca-certificates && rm -rf /var/lib/apt/lists/* \
|
|
&& rm -rf /usr/local/lib/node_modules/npm \
|
|
&& rm -rf /usr/local/lib/node_modules/corepack \
|
|
&& rm -f /usr/local/bin/npm /usr/local/bin/npx /usr/local/bin/corepack
|
|
|
|
WORKDIR /app
|
|
|
|
# Pre-create the data directory and hand it to the unprivileged `node` user
|
|
# so the bind-mounted volume is writable without root.
|
|
RUN mkdir -p /data/gitnexus && chown -R node:node /data
|
|
|
|
COPY --from=builder --chown=node:node /app/gitnexus/dist ./gitnexus/dist
|
|
COPY --from=builder --chown=node:node /app/gitnexus/node_modules ./gitnexus/node_modules
|
|
COPY --from=builder --chown=node:node /app/gitnexus/package.json ./gitnexus/package.json
|
|
COPY --from=builder --chown=node:node /app/gitnexus/scripts/install-duckdb-extension.mjs ./gitnexus/scripts/install-duckdb-extension.mjs
|
|
COPY --from=builder --chown=node:node /app/gitnexus/vendor ./gitnexus/vendor
|
|
|
|
# Expose the `gitnexus` binary on PATH so the documented Docker workflow
|
|
# (`docker compose exec gitnexus-server gitnexus index /workspace/<repo>`)
|
|
# works without users having to invoke `node /app/gitnexus/dist/cli/index.js`.
|
|
# `npm prune --omit=dev` in the builder stage strips `node_modules/.bin/`
|
|
# entries, so the `gitnexus` bin declared in package.json (`dist/cli/index.js`,
|
|
# which already carries `#!/usr/bin/env node` and 755 perms) is otherwise
|
|
# unreachable from $PATH.
|
|
RUN ln -s /app/gitnexus/dist/cli/index.js /usr/local/bin/gitnexus
|
|
|
|
# Bake the LadybugDB FTS extension into the image so BM25 keyword search works
|
|
# at runtime. The server runs the default `load-only` extension policy (the read
|
|
# pool pins `{ policy: 'load-only' }`), so a runtime `LOAD EXTENSION fts` never
|
|
# INSTALLs — the extension must already exist in the runtime user's HOME
|
|
# extension dir, or every keyword search silently degrades (no FTS indexes are
|
|
# written and ranking falls back to vector-only with only a `warning` field).
|
|
# Run the installer as the `node` user with the SAME HOME the server runs under,
|
|
# so `INSTALL fts` materializes the extension under `$HOME/.lbdb/extension` where
|
|
# the runtime `LOAD` resolves it offline. `ENV HOME` is pinned because Docker
|
|
# does not derive HOME from `USER`, so without it build-install and runtime-load
|
|
# would resolve different paths. Requires network egress for the one-time
|
|
# INSTALL; the build fails loudly if it cannot fetch the extension. The DB-size
|
|
# default comes from GITNEXUS_LBUG_MAX_DB_SIZE (single source of truth, matches
|
|
# the runtime) — it only sizes the throwaway scratch DB used to run INSTALL.
|
|
# The second `--verify-only` step re-LOADs the extension in a FRESH process
|
|
# under the same HOME, so a HOME/extension-dir mismatch fails the build here
|
|
# rather than silently degrading keyword search to vector-only at runtime.
|
|
ENV HOME=/home/node \
|
|
GITNEXUS_LBUG_MAX_DB_SIZE=17179869184
|
|
RUN su node -s /bin/sh -c "HOME=/home/node node /app/gitnexus/scripts/install-duckdb-extension.mjs fts" \
|
|
&& su node -s /bin/sh -c "HOME=/home/node node /app/gitnexus/scripts/install-duckdb-extension.mjs fts --verify-only"
|
|
|
|
# Published runtime assets (in package.json `files`). Placed AFTER the DuckDB
|
|
# FTS-extension RUN above so editing hook/skill content does not invalidate that
|
|
# network-fetching cache layer; they have no input dependency on it.
|
|
# `hooks/`: dist/cli/resolve-invocation.js does
|
|
# `require('../../hooks/claude/resolve-analyze-cmd.cjs')` at module load — the
|
|
# single source of truth for the npm-11 npx-crash invocation decision (#1939).
|
|
# Without it, `gitnexus analyze` inside the image crashes with MODULE_NOT_FOUND
|
|
# before it does any work (#2130). `skills/`: the CLI reads the bundled SKILL.md
|
|
# templates from `<pkg>/skills/` for `gitnexus analyze --skills` and `gitnexus
|
|
# setup`/`uninstall`; absent, those degrade silently (placeholder content / zero
|
|
# skills installed). (The web UI bundle `web/`, also in `files`, is deliberately
|
|
# NOT shipped: this builder never builds gitnexus-web, so the image is API-only;
|
|
# the UI is the separate Dockerfile.web image / hosted app.)
|
|
COPY --from=builder --chown=node:node /app/gitnexus/hooks ./gitnexus/hooks
|
|
COPY --from=builder --chown=node:node /app/gitnexus/skills ./gitnexus/skills
|
|
|
|
USER node
|
|
|
|
# The web UI defaults to http://localhost:4747 - keep that contract.
|
|
ENV GITNEXUS_HOME=/data/gitnexus \
|
|
GITNEXUS_NO_UPDATE_NOTIFIER=1 \
|
|
NODE_ENV=production \
|
|
PORT=4747
|
|
|
|
EXPOSE 4747
|
|
|
|
# Bind 0.0.0.0 for the host's mapped port, honoring an injected $PORT (Render
|
|
# sets one). `sh -c` expands it; `exec` keeps the server PID 1 so SIGTERM still
|
|
# reaches it. Platforms can rely on this instead of a dockerCommand override.
|
|
CMD ["sh", "-c", "exec gitnexus serve --host 0.0.0.0 --port \"${PORT:-4747}\""]
|