scriptoriumadmin/GitNexus
1
0
Fork 0
mirror of https://github.com/abhigyanpatwari/GitNexus.git synced 2026-08-28 05:25:25 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 9
1209 commits 100 branches 842 tags 270 MiB
Commit graph

8 commits

Author SHA1 Message Date
Gergő Magyar
cef63dd044
feat(install): toolchain-free tree-sitter via vendored prebuilds (#2113) 
* feat(install): toolchain-free tree-sitter via vendored GitNexus-built prebuilds

Eliminate the C/C++-toolchain requirement at install for the at-risk grammars
(dart, proto, kotlin) by generating + vendoring native prebuilds, mirroring the
existing vendored tree-sitter-swift. The 10 grammars that already ship 6 upstream
prebuilds stay npm dependencies (toolchain-free AND dependency-review-tracked).

- .github/workflows/build-tree-sitter-prebuilds.yml: a registry-parameterized
  workflow that builds {dart,proto,kotlin} x {linux,darwin,win32}-{x64,arm64}
  prebuilds natively, validates each loads + parses on its arch, and opens a PR
  vendoring them. A `guard` job gates the heavy matrix to run ONLY on dispatch
  or a real grammar-version change — ordinary code PRs cost zero matrix minutes.
- dart/proto: prefer a committed prebuild; fall back to today's source build
  when none matches (no behavior change until prebuilds are vendored).
- kotlin: vendor it (Swift parity) instead of compiling the third-party
  optionalDependency from source at the user's install — supersedes #2110's
  optionalDependency mechanism. The ~23 MB parser.c is NOT vendored (the
  workflow builds from the published package); only node-types + bindings +
  prebuilds are. Removed from optionalDependencies; lock regenerated; probe,
  parser-loader note, README/.devcontainer docs, and the #2110 tests updated.

DO NOT MERGE until vendor/tree-sitter-kotlin/prebuilds/ is populated by the
build-tree-sitter-prebuilds workflow: until then Kotlin is unavailable (vendored
with no source-build fallback). dart/proto remain fully functional throughout.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(install): guard 6/6 N-API prebuild coverage for every grammar

Regression guard so a toolchain-less install can never silently lose a tree-sitter
language on a supported platform-arch:

- Vendored grammars (vendor/tree-sitter-*): every one MUST ship a loadable N-API
  prebuild for all 6 tuples {linux,darwin,win32}-{x64,arm64}. Asserts the
  napi_register_module_v1 entry symbol in each .node (cross-platform, no need to
  run the binary). Currently RED for dart/proto/kotlin until the
  build-tree-sitter-prebuilds workflow populates their prebuilds/ — this is the
  must-fill-before-merge gate (swift already passes 6/6).
- npm-dependency grammars: asserts upstream ships 6/6 N-API too, catching a
  future platform drop. tree-sitter-c is allow-listed at 4/6 (missing
  linux-arm64/win32-arm64) pending #2116; the guard also fails if that gap is
  silently closed (prompting allow-list removal).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(install): vendor tree-sitter-c at 0.21.4 with GitNexus-built prebuilds (#2116)

tree-sitter-c is the one grammar dependency upstream ships incomplete prebuilds
for (4/6 — no linux-arm64/win32-arm64), AND it is a REQUIRED grammar: its own
`install` (node-gyp-build) compiles from source when no prebuild matches and
exits non-zero, so on a toolchain-less ARM host `npm install gitnexus` HARD-FAILS
at the c step — during npm's dependency phase, before any GitNexus postinstall
runs (so a postinstall "supplement" can't help).

Fix: vendor c prebuild-only at the pinned 0.21.4 (Kotlin pattern), with all six
prebuilds GitNexus-cross-built, and drop it from `dependencies`:
- vendor/tree-sitter-c/ (bindings + node-types + manifest + prebuilds); build
  probe scripts/build-tree-sitter-c.cjs; added to the build workflow registry
  (kind 'npm' — built from c@0.21.4 source).
- materialize-vendor-grammars.cjs: c is REQUIRED, so it is always materialized,
  even under GITNEXUS_SKIP_OPTIONAL_GRAMMARS (it needs no toolchain).
- Removed from package.json dependencies + lockfile (nothing else needs npm c —
  tree-sitter-cpp's dep on c is dev-only and not installed). Preserves the #1242
  ABI pin: vendoring 0.21.4 keeps the good ABI while closing the ARM gap.
- parser-loader note + the prebuild-coverage guard + a cli-commands assertion
  updated; c moves from the npm-gap allow-list into the vendored 6/6 cohort.

Verified: tsc clean, 31 unit tests pass, c loads/parses; the guard is RED for
c/dart/proto/kotlin until the workflow populates prebuilds (the must-fill gate).
Closes the operational risk in #2116.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(ci): source-build fallback for vendored c/kotlin so CI is healthy pre-prebuilds

The vendored prebuild-only grammars (c, kotlin) had empty prebuilds/ until the
build-tree-sitter-prebuilds workflow runs, so they could not load in CI — and
C is hard-required by cross-platform tests (tree-sitter-languages/parsing on
ubuntu+macos+windows), which I cannot pre-build for macos/windows locally. The
robust fix is a source-build fallback that works on every CI runner (all have a
toolchain), mirroring dart/proto:

- Vendor the grammar source (binding.gyp + src/) for c and kotlin; their build
  scripts now PREFER a committed prebuild (toolchain-free) and fall back to
  `node-gyp rebuild` from the vendored source when no prebuild matches. Verified
  both compile against the hoisted node-addon-api@^8 and the runtime loads.
- prebuild-coverage guard is now bootstrap-tolerant: a grammar that vendors its
  source (binding.gyp) may have an incomplete prebuild set (the workflow fills
  it); a prebuild-only grammar (swift) still must ship all six. Any present
  prebuild must still be N-API. Guard goes green; it re-tightens per-grammar as
  the workflow populates prebuilds.
- actionlint: silence a false-positive SC2016 (JS template literals inside the
  single-quoted `node -e` validate block).

Note: kotlin's generated parser.c is large (~23 MB on disk; compresses heavily
in git). Once the workflow populates all six kotlin prebuilds, the source serves
only as the fallback and could be slimmed if desired.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(docker): re-materialize+rebuild vendored grammars after npm prune

`npm prune --omit=dev` in the gitnexus CLI image drops anything not in
package.json's dependency tree — including the VENDORED tree-sitter grammars
(materialized by postinstall, not declared deps) and their built bindings. The
`serve` image analyzes/parses repos at runtime, so re-run the grammar postinstall
after the prune (in the toolchain-equipped builder) to restore them. Load-bearing
for tree-sitter-c, a core REQUIRED grammar now vendored (#2116): as a former
dependency it survived prune; vendored, it would not. Also restores
swift/dart/proto/kotlin, which were silently pruned from the image before.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(grammars): unify tree-sitter-swift with the vendored-source build pipeline

Swift was the last grammar handled differently — it shipped only upstream
prebuilds, while c/dart/proto/kotlin vendor their grammar source and use a
prefer-prebuild -> source-build-fallback activation script. Vendor swift's
source so all five are handled identically (one uniform build path).

- vendor/tree-sitter-swift: add binding.gyp (win-hardened), bindings/node/
  binding.cc, src/parser.c (ABI-14 default, ~18 MB), src/scanner.c, and
  src/tree_sitter/ headers. The 6/6 prebuilds are retained. The legacy
  parser_abi13.c alternate is intentionally not vendored.
- build-tree-sitter-swift.cjs: rewrite the prebuild probe into the dart-style
  prefer-prebuild then source-build fallback (keeps the GITNEXUS_SKIP gate and
  the never-exit-non-zero postinstall invariant).
- build-tree-sitter-prebuilds.yml: register swift (kind 'vendored'); add its
  package.json to the version-gated pull_request paths and a validate snippet.
- prebuild-coverage guard auto-moves swift into the source-fallback cohort
  (binding.gyp now present); refresh the stale "swift is prebuild-only" comments.
- tests: add build-tree-sitter-swift-probe.test.ts; fix the pre-existing
  build-tree-sitter-kotlin-probe.test.ts breakage (it still asserted the old
  probe strings after kotlin's dart-style conversion); assert swift's vendored
  source in cli-commands.test.ts.
- docs: README / .devcontainer / kotlin vendor README — swift's prebuilds are
  now GitNexus-cross-built from vendored source like the rest, not upstream-only.

Verified: swift source-builds against node-addon-api@8 -> N-API binary -> loads
against the pinned tree-sitter@0.21.1 (ABI 14) -> parses cleanly.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(publish): gate a lean prebuilds-only npm tarball behind a coverage guard

Vendoring grammar source (parser.c) alongside the prebuilds means the npm
tarball now carries ~50 MB of generated source it almost never compiles (every
supported platform-arch has a prebuild). Prepare to drop it from the published
package once all prebuilds exist — safely.

- .npmignore: add a GATED, commented-out "lean publish" block that excludes the
  source-build inputs (parser.c/scanner.c/tree_sitter/binding.gyp/binding.cc) but
  keeps prebuilds/ + the runtime files. Uncommenting ships prebuilds-only.
- scripts/assert-publish-grammar-coverage.cjs: a prepack guard that refuses to
  pack/publish if the source exclusion is active while any vendored grammar still
  lacks 6/6 prebuilds (which would ship a grammar with no loadable binding). Wired
  into `prepack` (runs on npm pack + publish, incl. the publish.yml dry-run) and
  exposed as `npm run assert-publish-coverage`.
- test: pure-core decision cases + a real-repo publish-safety check that fails CI
  if .npmignore is activated prematurely.

Net: the prebuilds already publish today (files: ["vendor"]); this makes the
future switch to a prebuilds-only tarball a one-line uncomment that can't ship a
dead grammar. The guard currently reports "source + prebuilds" (only swift has
6/6 prebuilds so far) and passes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(grammars): consolidate the 5 build-tree-sitter-*.cjs into one

The per-grammar activation scripts (c/dart/proto/swift/kotlin) were ~95%
identical — same prefer-prebuild → source-build → never-fail flow, differing only
in name, target_name, required-vs-optional, and the display label in warnings.

- scripts/build-tree-sitter-grammars.cjs: one registry-driven script. Bare call
  builds all (postinstall); `... <name>` builds only the named grammars (so the
  probe test can isolate one). c is `required: true` (ignores the opt-out gate);
  the rest honor GITNEXUS_SKIP_OPTIONAL_GRAMMARS. Per-grammar try/catch + a final
  process.exit(0) preserve the postinstall never-exit-non-zero invariant.
- package.json: postinstall is now `materialize && build-tree-sitter-grammars.cjs`
  (was five chained `build-tree-sitter-<name>.cjs` calls).
- tests: replace the two near-identical *-probe.test.ts files with one
  parameterized build-tree-sitter-grammars-probe.test.ts that also covers the
  required-vs-optional opt-out split and an unknown-grammar arg.
- update cli-commands.test.ts postinstall assertions + the vendor c/kotlin/swift
  README + swift provenance to reference the consolidated script.

Behavior is preserved (warnings normalized to one consistent format). Removes 5
scripts + 1 test file; adds 1 script + 1 test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(ingestion): lazy-load tree-sitter-c to prevent module-load crash

tree-sitter-c is now vendored prebuild-only (#2116) with 0/6 committed
prebuilds, so on a toolchain-less or `--ignore-scripts` install C has no native
binding. Three modules loaded it via a hard top-level `import C from
'tree-sitter-c'`, which throws ERR_MODULE_NOT_FOUND at module-load — crashing
`analyze` before parser-loader's optional/severity:error degradation can run.
This is the #2091/#2093 bug class (previously fixed for swift/dart/kotlin); C was
left static because it used to be an always-present npm dependency.

- languages/c/query.ts: load via the lazy guarded getLanguageGrammar(C), mirroring
  swift/query.ts; the main-thread isLanguageAvailable filter ensures the getters
  are reached only when C is present.
- workers/parse-worker.ts: guarded `_require('tree-sitter-c')` + conditional
  languageMap spread, like swift/dart/kotlin.
- group/extractors/include-extractor.ts: guarded `_require`; getLanguageForFile
  returns null for .c/.h when absent, so C include-extraction degrades to a no-op
  (C++ unaffected).
- extend the registry-import-closure regression test (#2091/#2093) to assert C
  also loads lazily at registry static-import time.

* fix(ci): repin attest-build-provenance to the real v2.4.0 SHA

The workflow pinned actions/attest-build-provenance@bd77c077… commented
`# v2.4.0`, but v2.4.0 is e8998f94… (verified via the GitHub API); bd77c077…
is an untagged mid-stream commit, so the SLSA-attestation step ran unvetted
action code and the comment misrepresented what runs. Repin to the real
v2.4.0 commit and drop the `# PLACEHOLDER-PIN` markers on both this line and
the setup-python pin (a26af69b… is already the correct v5.6.0 — only its
comment was stale). Update the header NOTE accordingly.

* fix(ci): skip the prebuild-PR aggregate when release App secrets are absent

The aggregate job mints a GitHub App token as its first step; with
RELEASE_APP_ID/RELEASE_APP_PRIVATE_KEY unset it hard-failed AFTER a full
(up-to-6-runner) native build. Since the `secrets` context isn't available in
a job-level `if:`, the guard job now computes a `release_app` boolean output
(a step can read secrets) and emits an actionable `::notice::`; aggregate
gates on it and skips cleanly, while the build job's artifacts still upload
(run with open_pr=false for artifacts-only).

* chore(ci): drop package-lock.json from the prebuild paths filter; widen build timeout

`gitnexus/package-lock.json` changes on nearly every dependency PR, so it
fired the prebuild workflow's guard job on unrelated churn (the matrix stayed
correctly skipped — `gitnexus/package.json` already covers the transition-window
pin, so removing the lock only drops guard noise). Also bump the native build
job timeout 30 -> 45 min for headroom compiling the 23 MB kotlin / 18 MB swift
parser.c, especially under arm emulation.

* fix(ci): event-gate the aggregate open-PR condition explicitly

`inputs.open_pr` is null on pull_request events, and the prior
`inputs.open_pr != false` leg relied on GHA's direction-ambiguous null
coercion (Codex F4) to decide whether to open the prebuild PR. Gate
explicitly on the event: a non-fork pull_request that bumped a grammar
version opens the prebuild PR (the documented flow), and `open_pr` is only
consulted on workflow_dispatch — so a manual run with open_pr=false stays
artifacts-only and no event's behavior rests on coercion.

* fix(publish): validate the effective npm-pack contents in the coverage guard

The publish guard inferred "is source shipped?" from a single .npmignore toggle
line, which a partial/out-of-order edit could defeat (exclude binding.gyp but
leave parser.c → unbuildable yet "source-shipping"). It now inspects the
EFFECTIVE tarball via `npm pack --dry-run --ignore-scripts --json` (the
--ignore-scripts avoids re-entering this guard through prepack): a grammar
"ships source" only when EVERY on-disk source-build input (binding.gyp +
binding.cc + parser.c + scanner.c when present + a tree_sitter header) is
actually in the packed file list.

This also surfaced that the gated lean-publish .npmignore block was inert:
package.json's `files: ["vendor"]` allow-list overrides .npmignore for the
vendored subtree, so those exclusion lines never dropped anything. Replace the
dead toggle with documentation of the real mechanism (narrow the `files` field)
and note the guard enforces safety on the effective pack regardless of how the
slim is done.

* test(prebuild): hard-gate declared-fully-prebuilt grammars on 6/6 coverage

The strict 6/6 prebuild assertion was dormant whenever a grammar vendors source
(binding.gyp) — which is every grammar — so a dropped prebuild passed CI
silently. Add a FULLY_PREBUILT allowlist of grammars GitNexus has committed 6/6
for (today: swift); those must keep all six even with a source fallback, so
losing one now fails CI. Grammars graduate into the set as the
build-tree-sitter-prebuilds workflow lands their binaries. (The static-import
degradation smoke is covered by the registry-import-closure regression test
extended in the C lazy-load commit.)

* chore(deps): promote node-gyp-build/node-addon-api to regular dependencies

Every vendored grammar's index.js does `require("node-gyp-build")` at runtime
to load even a prebuilt .node, so node-gyp-build is runtime-load-critical (and
node-addon-api is needed for the source-build fallback). They were
optionalDependencies, surviving `--omit=optional` only via the required
tree-sitter's transitive edge — correct today but fragile. Promote both to
regular dependencies so the contract is explicit (optionalDependencies is now
empty and removed). Lock the contract with a cli-commands assertion.

* chore(vendor): add Windows cflags parity block to tree-sitter-c/binding.gyp

c's binding.gyp used an unconditional `cflags_c: ["-std=c11"]`, while
kotlin/swift gate MSVC flags behind an `OS=='win'` condition (/std:c11 /utf-8).
Inert today (no non-ASCII bytes in c's parser.c, and node-gyp ignores cflags_c
on MSVC anyway), but align the three so a future source-build fallback on
Windows behaves consistently.

* docs(agents): correct stale optional-grammar / postinstall notes

AGENTS.md still said postinstall "patches tree-sitter-swift, builds
tree-sitter-proto" and that only kotlin/swift are "optional". Update to the
vendored-uniform model: postinstall materializes the vendored grammars and
prefers a committed prebuild (source-build only when none matches); c is
required while dart/proto/swift/kotlin are optional + skippable via
GITNEXUS_SKIP_OPTIONAL_GRAMMARS=1, with non-fatal warnings only on a
toolchain-less host with no matching prebuild.

* fix(install): preserve the backup and warn loudly on a failed materialize rollback

If renameSync(partial, dest) failed AND the rollback renameSync(backup, dest)
also failed, the grammar was left unmaterialized (node_modules/<name> missing)
with only a generic "could not materialize" warning — the recoverable backup at
<dest>.materialize-bak was unmentioned. Emit a CRITICAL warning naming the
backup path and the recovery command on that double-failure, and document that
the fail-soft catch removes only the scratch `partial`, never the `backup`
(which may be the sole recoverable copy). Never-throw / exit-0 contract intact.

* fix(publish): make the coverage guard's npm-pack inspection script-safe

The prepack guard shelled out to `npm pack --dry-run --ignore-scripts --json`,
but the `--ignore-scripts` flag is not reliably honored by npm pack's
prepare/prepack lifecycle on the CI npm — so build.js ran, polluted the --json
stdout with `[build] …`, and the guard's JSON.parse threw. That broke every
`npm pack` (packaged-install-smoke on ubuntu+windows) and failed the guard's own
real-repo unit test (the only coverage-job failure). Force script-skipping via
the reliable `npm_config_ignore_scripts` env config (also removes the prepack
re-entry/recursion risk) and parse defensively from the JSON-array start.

* fix(publish): make the coverage guard deterministic — read `files`, not `npm pack`

The npm-pack-based guard timed out in CI: `npm pack`'s prepare/prepack lifecycle
is not skipped by `--ignore-scripts` (flag or env config) on the CI npm, so the
inner pack ran the full build (~20s+) — fine for the slow smoke job, but it blew
past vitest's 30s test timeout in the coverage job (and risked re-entering this
prepack guard).

Replace it with a deterministic, fast (~0.1s) check that needs no subprocess:
since `files: ["vendor"]` OVERRIDES `.npmignore` for the vendored subtree (so
`.npmignore` can never drop vendored source — verified), the ONLY lever that can
exclude source is narrowing the package.json `files` field. The guard now reads
`files` directly: a grammar "ships source" iff `files` includes the vendor
subtree AND the grammar carries a buildable source set on disk. A lean publish
that narrows `files` while a grammar lacks 6/6 prebuilds still fails the gate.

* feat(ci): vendored tree-sitter grammar update monitor

Adds a weekly (+ dispatchable) workflow that checks each vendored grammar against
its source-of-origin (npm for swift/kotlin, the GitHub default branch for
dart/proto; c is excluded — held at 0.21.4 for ABI safety) and opens a PR
re-vendoring any update that is ABI-COMPATIBLE with the pinned tree-sitter@0.21.1
(LANGUAGE_VERSION 13-14).

ABI awareness is the point: most upstreams have moved to ABI 15 (newer
tree-sitter), so a blind "bump to latest" would open PRs that can't build. The
monitor fetches the candidate source, reads its parser.c LANGUAGE_VERSION, and
only re-vendors 13/14 — incompatible updates are reported (notice + job summary),
never applied. (Confirmed live: dart/proto upstreams are ABI 15 today and are
correctly held; swift/kotlin are current.)

The re-vendor refreshes only the source-build inputs + runtime entrypoints,
preserving the GitNexus-hardened binding.gyp / README / prebuilds; the version
bump then triggers build-tree-sitter-prebuilds.yml, whose ABI-validation is the
final safety net so a subtly-wrong re-vendor can't silently ship. PR creation is
gated on the RELEASE_APP secret (skips with a notice if absent), mirroring the
build aggregate. Unit test locks the ABI gate; the script is import-safe.

* feat(ci): monitor tree-sitter-c too (report-only, ABI-pinned)

c was excluded from the update monitor, so an upstream c update went unnoticed.
Include it, but as report-only via a `hold`: c is ABI-pinned at 0.21.4
(#1242/#858) and must not auto-bump without a tree-sitter runtime upgrade, so an
available c update is detected + surfaced (notice + job summary) but never
auto-PR'd — even if it were ABI-13/14. `--apply c` refuses defensively. (Live:
upstream c is 0.24.1 / ABI 15 today, so c is doubly held — reported, not applied.)

* fix(ci): drop the shell in the grammar monitor's github fetch (CodeQL)

CodeQL flagged the GitHub-tarball fetch — it used `bash -c "gh api …/tarball/$ref
> src.tgz && tar xzf src.tgz"`, interpolating the API-derived ref into a shell
command (the shell-command-injection family: "this shell command depends on an
uncontrolled file name"). Replace it with a shell-free path: capture `gh api`'s
binary tarball as a Buffer via execFileSync, write it to a fixed file, and
extract with execFileSync('tar', …). No shell, no injection surface. Verified the
dart/proto fetch + ABI read still work.

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-09 18:16:24 +01:00
Gergő Magyar
288b96f3e5
fix: batch query enrichment, bake FTS extension into CLI image, add FTS memory repro (#2108) 
* perf(query): batch per-symbol process/cohesion/content lookups (N+1 -> 2-3)

Port of the local-backend query-batching from gitnexus-enterprise PR #222
into the OSS local MCP backend. The query tool traced each matched symbol
to its processes + cohesion (+ content) with up to 3N sequential pool
round-trips; batch them into 2-3 'WHERE n.id IN $nodeIds' queries keyed
back to each symbol by a prepended 'n.id AS nodeId' column. Output is
identical: the aggregation loop is unchanged, iterates merged in the same
order, and reads pre-fetched maps instead of issuing a query per symbol.

Adaptations over a blind cherry-pick (would otherwise change output):
- per-nodeId first-row community pick replaces the per-symbol LIMIT 1, so
  each symbol keeps its own community (not one for the whole batch);
- batched rows regrouped to the originating merged item by nodeId so the
  JS-side RRF item.score still drives process ranking;
- positional fallbacks shift +1 (process row[1..6], cohesion [1]/[2],
  content [1]); CodeRelation{type:...} relation form kept; IN-list chunked
  at 100 like the impact path.

Adds a regression test asserting per-node community/content association
(func:login keeps comm:auth; func:validate inherits no community).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(docker): bake LadybugDB FTS extension into the CLI/serve image

The container runs `serve` under the default `load-only` extension policy
(the read pool pins {policy:'load-only'}), so a runtime LOAD EXTENSION fts
never INSTALLs. Dockerfile.cli copied the extension installer but never ran
it, so the runtime user's HOME had no FTS extension: keyword search
silently degraded (no FTS indexes written, ranking falls back to
vector-only with only a warning field). Same class of footgun fixed for
the Hub image in gitnexus-enterprise PR #222.

Run install-duckdb-extension.mjs as the `node` user with the runtime HOME
so INSTALL fts materializes the extension under $HOME/.lbdb/extension where
the runtime LOAD resolves it offline. Pin ENV HOME=/home/node because
Docker does not derive HOME from USER — without it the build-install and
runtime-load would resolve different paths. Verified locally: INSTALL lands
in $HOME/.lbdb/extension/0.17.0 and a fresh offline load-only
`LOAD EXTENSION fts` resolves it. Dockerfile.web is unaffected (static
frontend, no @ladybugdb backend).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(lbug): FTS evict->reload RSS repro + inert pool RSS tracing

Settles the gitnexus-enterprise PR #222 root-cause hypothesis for OSS:
does re-running LOAD EXTENSION fts on every pool evict->reload strand the
native FTS arena (unbounded RSS growth in long-lived MCP serve), or does
db.close() reclaim it (bounded by MAX_POOL_SIZE)? Static read could not
decide — the native lbugjs.node binary documents no close->extension-unload
contract.

Adds gitnexus/scripts/bench/fts-evict-reload-rss.mjs: a NATIVE mode that
reproduces the exact native sequence doInitLbug()+closeOne() perform
(open Database -> Connection -> LOAD EXTENSION fts -> QUERY_FTS_INDEX ->
close) across K self-built FTS fixtures, and a --via-pool mode that drives
the real compiled pool (initLbug/executeParameterized/closeLbug) against an
existing analyzed repo. Plus a behavior-neutral GITNEXUS_POOL_RSS_TRACE=1
stderr trace on pool init/close (stdout reserved for MCP JSON-RPC; single
env read when disabled).

RESULT (native, 24 and 40 cycles x 6 fixtures, --expose-gc): PLATEAU. RSS
warms up to ~400 MB then flattens (40-cycle: +36 MB over cycles 1-10, +3 MB
over 30-40; decelerating), not the linear climb a per-reload arena leak
would produce (240 reloads x stranded arena = multi-GB). db.close()
reclaims the FTS arena. The unbounded-leak hypothesis is NOT reproduced for
the OSS path: the pool's LRU eviction + close-on-evict BOUNDS the footprint,
which is exactly the protection the enterprise Hub supervisor lacked (it
opened bridge DBs in-process without eviction -> 15 GB). => plan U4
(worker/process isolation) is NOT justified by this evidence; U1 + U2 are
the only OSS-shared changes. Caveat: small fixtures + awaited close; a
--via-pool run against a large analyzed repo over a long session is the
production-faithful follow-up (instrumentation is in place for it).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(review): apply ce-code-review autofix feedback (#222 migration)

Adversarial review found the U3 bench PLATEAU->no-leak conclusion was
over-claimed from a 600-row fixture: a size-proportional FTS-arena leak
would be sub-threshold at that scale. Strengthen the bench and make its
verdict honest:
- scale the fixture (--rows, UNWIND batch insert), probe ALL 5 FTS indexes
  in --via-pool (not 2 of 5), add a --no-await-close variant (the pool
  fire-and-forget close shape), and replace the absolute-delta gate with a
  SLOPE-DECELERATION 3-way verdict (PLATEAU / CLIMB / INCONCLUSIVE) plus
  step-discontinuity detection. At production-representative scale the
  synthetic runs are noisy/INCONCLUSIVE (deceleration argues against an
  UNBOUNDED leak but does not prove bounded), so plan U4 stays GATED on a
  --via-pool run against a real large analyzed repo -- not closed.
- Dockerfile.cli: source the scratch-DB size from ENV GITNEXUS_LBUG_MAX_DB_SIZE
  (single source of truth) and add a build-time verify-only LOAD gate
  that fails the build on a HOME/extension-dir mismatch instead of silently
  degrading runtime keyword search.
- install-duckdb-extension.mjs: additive verify-only mode (LOAD-only in a
  fresh process) + robust size parse; back-compatible with the runtime
  positional-size caller (validated).
- tests: wire func:validate into a second process (proc:beta-flow) so the
  batched STEP_IN_PROCESS row[1..6] positional shift is exercised by a
  genuine multi-process symbol, and assert process ranking. No blast radius
  (75 seed-consuming tests pass).
- pool-adapter.ts: trim the traceRss narrated-code comment (DoD 2.3).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(bench): classify a sustained sub-floor RSS slope as INCONCLUSIVE, not PLATEAU

Tri-review P2: the FTS evict->reload verdict short-circuited to PLATEAU
whenever secondHalfSlope < SUSTAIN_FLOOR, BEFORE the deceleration check —
so a sustained (non-decelerating) linear leak below 0.5 MB/cycle was
labeled PLATEAU ("no leak"), the label that would wrongly close plan U4.

Extract median/slopeMbPerCycle/classifyVerdict into a pure, side-effect-free
fts-rss-verdict.mjs (zero imports) so it is unit-testable without loading the
native addon or running the bench, and fix the classifier:
- epsilon-first gate: a truly flat tail (< 0.1 MB/cycle) is PLATEAU regardless
  of decelRatio (guards against over-correcting a real negative into
  INCONCLUSIVE);
- a sustained sub-floor positive slope (>= epsilon, < floor, decelRatio >= 0.6)
  is INCONCLUSIVE — a slow creep RSS cannot distinguish from noise at this
  scale, so the honest label is "not resolved", never a clean PLATEAU;
- the noise floor now scales with the WORKING-SET growth (peak-baseline), not
  the pre-DB baseline RSS (which is interpreter/addon overhead, larger in
  --via-pool mode, and would inflate the floor and HIDE leaks).

Reconcile the stale "per-row-relative delta floor" docstring; add floor +
decelRatio to the MACHINE line. New fts-rss-verdict.test.ts pins all label
boundaries (flat->PLATEAU, sustained-sub-floor->INCONCLUSIVE,
decelerated->PLATEAU, sustained-linear->CLIMB, step->INCONCLUSIVE,
working-set floor, no import side effects). U1 does NOT add detection power
for sub-floor leaks (RSS cannot attribute that magnitude) — it stops the
false PLATEAU and routes that regime to the --via-pool run.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(query): signal partial/warning on a real enrichment failure (not benign missing-table)

Tri-review P2: when a batched enrichment query (process/cohesion/content)
threw, it was caught + logged and the chunk's symbols silently fell back to
`definitions` with no signal — the caller could not tell "genuinely
standalone" from "enrichment failed".

Track an `enrichmentDegraded` flag in the three enrichment catch blocks and,
at response build, compose a single `warning` (FTS-missing and/or the
enrichment message, so neither overwrites the other) plus `partial: true`.
Both fields are omitted on the clean path, so the success-path response shape
is byte-identical.

Crucially, the flag fires ONLY for a REAL failure (timeout / lock / native
fault), NOT the benign "no Process/Community table" prepare error — a repo
analyzed without processes/communities is a normal config, and firing
`partial` on every such query would desensitize callers
(isBenignMissingTableError gates it).

New unit test test/unit/query-degraded-signal.test.ts (vi.mock pool-adapter,
override hybrid search to feed one matched symbol, route STEP_IN_PROCESS ->
throw): real failure -> warning+partial+symbol still returned; benign
missing-table -> no signal; FTS-missing + enrichment failure -> both messages
in one warning. Plus a success-path no-warning/no-partial assertion in the
calltool integration test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-09 08:46:46 +01:00
Léon Simmons
507f84b69a
fix(docker): symlink gitnexus binary onto $PATH in runtime image (#1551) 
The README documents the Docker workflow as:

    WORKSPACE_DIR=$HOME/code docker compose up -d
    docker compose exec gitnexus-server gitnexus index /workspace/my-repo

…but `gitnexus` is not on $PATH inside the published image:

    $ docker compose exec gitnexus-server which gitnexus
    (empty)
    $ docker compose exec gitnexus-server gitnexus --version
    exec: "gitnexus": executable file not found in $PATH

The package.json `bin` entry (`"gitnexus": "dist/cli/index.js"`) would
normally surface via `node_modules/.bin/gitnexus`, but `npm prune
--omit=dev` in the builder stage strips that directory before the runtime
stage copies it in. The `dist/cli/index.js` itself already has the
`#!/usr/bin/env node` shebang and 755 permissions, so a single symlink
into /usr/local/bin makes the README's literal command work.

Verified locally:

    $ docker build -f Dockerfile.cli -t gitnexus:local-pr-test .
    $ docker run --rm gitnexus:local-pr-test gitnexus --version
    1.6.4
    $ docker run --rm gitnexus:local-pr-test gitnexus --help
    Usage: gitnexus [options] [command]
    …
    $ docker run --rm -d --name t gitnexus:local-pr-test \
      && sleep 4 && docker exec t curl -s localhost:4747/api/health
    {"status":"ok"}

CMD continues to invoke `node gitnexus/dist/cli/index.js serve …`
unchanged, so the change is additive and the server boot path is
untouched.

Refs #1549.
 2026-05-13 17:14:52 +01:00
Hugo Gu
38ff7365e8
fix(docker): install ca-certificates in runtime image for TLS verification (#1545) (#1547) 
Close: #1545

Co-authored-by: Gergő Magyar <gergomagyar@icloud.com>
 2026-05-13 14:45:37 +01:00
uwence
a3af0dcce2
fix(docker): include duckdb installer script in runtime image (#1502) 2026-05-11 11:38:59 +01:00
Copilot
e02c56f653
fix(security): Pin Docker Node base images, remove runtime package-manager CVE surface, verify Trivy on PRs, and harden Dependabot policy (#1455) 
* fix: pin Docker node base images and remediate bundled npm CVEs

Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/e0605c79-296e-4b3a-b6c3-4ad375950935

Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com>

* fix: run trivy on docker PR changes and remove corepack

Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/4d714047-4fc1-4af1-9734-91400a15568f

Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com>

* chore: add docker digest updates and normalize dockerfile comments

Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/7d980908-a823-4c28-b074-9134ec672e84

Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com>

* fix: add dependabot cooldown policies

Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/a8531b8d-384b-4c54-84dd-a98b31993c44

Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com>

* fix: remove unsupported dependabot cooldown keys

Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/166df50e-c2fe-4d7f-ab41-e94c703338f6

Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com>

* chore(node): bump CI + engines to Node 22; centralize NPM_VERSION via build ARG

Closes the LOW findings from Claude Final Re-Review on PR #1455:

- Bump engines.node to >=22.0.0 and align all CI workflows (ci-quality,
  pr-autofix, publish, release-candidate) and the composite setup
  actions on Node 22. Node 20 reached EOL on 2026-04-30; the test
  Docker image was already on 22.
- Centralize the bootstrapped npm version in a single ARG NPM_VERSION
  per Dockerfile (cli, web, gitnexus/Dockerfile.test) so a security
  bump only requires updating one default per file with a clear
  cross-reference comment.

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com>
Co-authored-by: Gergo Magyar <gergomagyar@icloud.com>
 2026-05-09 16:55:31 +01:00
evolution
6ad53ff5c7
fix(ci): Change docker base image from alpine to debian (#1014) 
* fix(docker): switch Dockerfile.cli from Alpine to Debian slim

Alpine uses musl libc which is incompatible with @ladybugdb/core's
glibc-compiled native binary, causing ERR_DLOPEN_FAILED on startup.

Closes #1008

* fix(docker): resolve build and runtime failures in Dockerfile.cli

- Add **/*.tsbuildinfo to .dockerignore and rm -f tsbuildinfo in
  builder to prevent stale incremental cache from skipping
  gitnexus-shared compilation
- Install libstdc++6 from Debian Trixie for @ladybugdb/core native
  module compatibility (requires GLIBCXX_3.4.31)

* fix(docker): use node:22-trixie-slim for GLIBCXX_3.4.31 support

Replaces the manual Trixie libstdc++6 backport with the official
node:22-trixie-slim base image, which ships GCC 14 runtime natively.
 2026-04-21 21:31:58 +01:00
Copilot
3adb97e993
feat(docker): ship signed UI + CLI/server images via docker-compose (#967) 
* Initial plan

* docker: ship signed UI + CLI/server images via docker-compose

Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/883bcee1-4a1d-4b3d-bbb9-accd8846da96

Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com>

* docker: lock image version to npm package + harden cosign verify guidance

Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/6afd4fcd-5656-4e02-b796-a22b59000bde

Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com>

* docker: add Sigstore ClusterImagePolicy + k8s admission docs

Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/aea3dd70-e2a9-443a-b578-cb3eca4093e1

Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com>

* docker(k8s): collapse redundant image globs in ClusterImagePolicy

Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/aea3dd70-e2a9-443a-b578-cb3eca4093e1

Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com>

* docker(ci): drop deprecated COSIGN_EXPERIMENTAL, dead build-args, and loose verify regex in comment

Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/bdf0d2cf-607c-4558-982a-be9b216b2d36

Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com>

* docker(ci): use ${{ github.repository }} in verify-comment regex for fork portability

Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/bdf0d2cf-607c-4558-982a-be9b216b2d36

Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com>

* style(deploy): prettier-format cluster-image-policy.yaml (single quotes)

Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/b09a016e-56a1-4b73-bacc-e69084a48782

* ci(docker): drop workflow_dispatch, harden signing loop, fix verify-comment placeholder

Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/6755064c-7871-4b2e-9b46-b4779eb215ac

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com>
 2026-04-18 20:55:19 +01:00