mirror of
https://github.com/abhigyanpatwari/GitNexus.git
synced 2026-08-28 05:25:25 +00:00
10 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
f2717c6a7c
|
feat(render): add one-click deploy to render support (#2804)
Some checks are pending
Scorecard / Scorecard analysis (push) Waiting to run
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Skill copy sync / shipped skills drift guard (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
|
||
|
|
4f9d595c73
|
fix(docker): ship runtime-needed published assets (hooks/, skills/) into the image (#2130) (#2132)
* fix(docker): copy hooks/ into Dockerfile.cli runtime stage (#2130) `gitnexus analyze` inside the official image (akonlabs/gitnexus, ghcr.io/abhigyanpatwari/gitnexus) crashed at startup with: Error: Cannot find module '../../hooks/claude/resolve-analyze-cmd.cjs' Require stack: - /app/gitnexus/dist/cli/resolve-invocation.js `dist/cli/resolve-invocation.js` does `createRequire(import.meta.url)('../../hooks/claude/resolve-analyze-cmd.cjs')` at module load (it is the single source of truth for the npm-11 npx-crash invocation decision, #1939), and `analyze.ts` statically imports it. The Dockerfile.cli runtime stage copied dist/node_modules/package.json/the duckdb script/vendor but never `hooks/`, so the require throws before the command does any work. `hooks/` is in package.json `files`, so npm already ships it — Docker was the only distribution dropping it. Fix: copy `hooks/` into the runtime stage, mirroring what npm publishes. Also add `test/unit/dockerfile-runtime-asset-parity.test.ts`: a regression guard that derives every out-of-dist `require()`/`createRequire()` target from source and asserts each is a runtime-stage `COPY`. Scoped to the require family (not `fs.access`/`new URL`), so it locks the #2130 class without false-flagging the intentionally-omitted, gracefully-degrading `web/` and `skills/` assets. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(docker): also ship skills/ into the runtime image Follow-up to the hooks/ fix: `skills/` is another published runtime asset (in package.json `files`) the Docker image dropped. The CLI reads the bundled SKILL.md templates from `<pkg>/skills/` for `gitnexus analyze --skills` (ai-context skill generation) and `gitnexus setup`/`uninstall` (installing skills into editor configs). Unlike the hooks/ require(), these reads degrade SILENTLY when the dir is absent — `--skills` writes minimal placeholder content (ai-context.ts), `setup` installs zero skills (setup.ts readdir → []) — so the image looked fine but produced wrong output. Copy `skills/` so the image is fully usable for all CLI tooling. `web/` (also in `files`) is intentionally NOT shipped: this image never builds gitnexus-web (the builder doesn't copy it, build.js logs "skipping web UI"), so it is API-only by design — the UI is the separate Dockerfile.web image / hosted app. The duckdb script is the only runtime asset needed from scripts/, so that stays a single-file copy. Extends the runtime-asset-parity guard with an explicit skills/ assertion. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(test): correct stale docstring that listed skills/ as not copied The 2nd commit on this branch added a skills/ COPY + an it('copies skills/…') assertion, but the top-of-file docstring still grouped skills/ with web/ as 'intentionally not copied / out of scope'. Drop skills/ from that sentence and note it is shipped (and covered by its own test). web/ remains the sole fs-accessed-but-uncopied example. Documentation-only; assertions unchanged. * fix(test): make runtime-stage detection case-insensitive on AS Docker accepts a lowercase `as runtime`; the parity guard's stage-detection regex was case-sensitive on `AS`, so a future Dockerfile reformat would empty the parsed COPY set and trip the named assertions. Add the /i flag. * fix(test): stop runtime-stage COPY parsing at the next FROM runtimeStageCopiedSources scanned from the runtime FROM to EOF. Bound the scan to the runtime stage (start after its FROM, break on the next FROM) so a build stage added after runtime can't have its COPY lines misattributed. No-op today (runtime is the last stage); the copied set is unchanged. * fix(test): assert at least one runtime COPY is parsed (no vacuous pass) If the runtime FROM or the /app/gitnexus/ source prefix ever stops matching, the copied set goes empty and the parity assertion passes vacuously. Add an explicit copied.length>0 guard so that failure mode is loud and named. * fix(test): strip line comments before require-scanning requiredExternalAssets() regex-scanned raw source, so a future doc-comment such as a commented-out require('../../web/x') in a shallow src file would resolve outside dist/ and spuriously fail the parity guard. Strip // line comments first. Block comments are deliberately not stripped (a naive block strip mangles slash-star inside string/glob literals). Verified the real-tree scanner output is byte-identical with and without the strip, and resolve-invocation.ts's multi-line createRequire is still detected. (Also swaps a stray non-ASCII glyph in the prior commit's comment for ASCII.) * fix(test): account for aliased + computed module-load requires (fail-closed) The parity scanner only matched string-literal require/createRequire, so it missed module-load requires via aliased createRequire bindings and computed paths — and already failed to see community-processor.ts's `_require(leidenPath)` -> vendor/leiden, making the "every out-of-dist asset" claim untrue. Broaden the scan: - Discover per-file createRequire bindings (requireCJS, _require, …) and match their literal-arg calls; keep the createRequire(...)('…') IIFE form. - Detect COMPUTED (non-literal) requires and gate them on MODULE-LOAD position (brace-depth 0), so the four in-function computed requires that target node_modules/package.json (optional-grammars, native-check, capabilities, parse-cache) are correctly out of charter and ignored. A module-load computed require must be vetted in KNOWN_COMPUTED_REQUIRES (seed: community-processor -> vendor/leiden) or the test FAILS CLOSED for manual review. - Allowlist entries are coverage-checked via isCovered, never trusted: a new test removes the `vendor` COPY from a fixture and asserts leiden surfaces as uncovered (so deleting a COPY can't silently pass — the #2130 class). - Exclude `<id>.resolve(...)` (a path lookup, not a load). - Upgrade the comment stripper to a string-aware pass that removes line AND block comments without mangling slash-star inside string/glob literals — the computed branch needs JSDoc requires (e.g. javascript/index.ts) gone, and the literal scan output stays byte-identical. Honest claim wording: the 4th test now says coverage = resolvable + vetted module-load requires, unrecognized computed requires fail for review. Adds unit tests for fail-closed, aliased-literal, and in-function-ignored paths. * fix(test): also scan shipped .cjs/.mjs assets for sibling requires The guard only scanned src/**/*.ts, so hand-written shipped runtime files were invisible — and they DO require siblings: hooks/claude/gitnexus-hook.cjs and hooks/antigravity/gitnexus-antigravity-hook.cjs each require('./hook-lock.cjs'), './hook-db-lock-probe.cjs', './resolve-analyze-cmd.cjs'. Add a second pass over shipped .cjs/.mjs assets (the runtime COPY set minus dep/data roots), resolving each relative require against the asset's OWN package-relative dir and checking COPY coverage — by prefix, NOT on-disk existence: the antigravity hook's './hook-lock.cjs' resolves to hooks/antigravity/hook-lock.cjs (which doesn't physically exist; hook-lock.cjs lives under hooks/claude) yet is covered by the whole-hooks COPY. All 6 shipped sibling requires resolve under the hooks COPY. * fix(docker): move hooks/skills COPYs past the DuckDB FTS RUN The hooks/ and skills/ COPYs sat between the vendor COPY and the DuckDB FTS-extension install RUN, so any edit to hook/skill content invalidated that RUN's cache layer — which performs a one-time network INSTALL of the extension (~tens of seconds per affected build). The COPYs have no input dependency on the DuckDB step; relocate them to after it (before USER node) so stable infrastructure layers are not rebuilt on hook/skill churn. Image contents are unchanged. The runtime-asset-parity guard still detects both (its scan covers the whole runtime stage), and the two are consolidated under one comment. --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
cef63dd044
|
feat(install): toolchain-free tree-sitter via vendored prebuilds (#2113)
* feat(install): toolchain-free tree-sitter via vendored GitNexus-built prebuilds
Eliminate the C/C++-toolchain requirement at install for the at-risk grammars
(dart, proto, kotlin) by generating + vendoring native prebuilds, mirroring the
existing vendored tree-sitter-swift. The 10 grammars that already ship 6 upstream
prebuilds stay npm dependencies (toolchain-free AND dependency-review-tracked).
- .github/workflows/build-tree-sitter-prebuilds.yml: a registry-parameterized
workflow that builds {dart,proto,kotlin} x {linux,darwin,win32}-{x64,arm64}
prebuilds natively, validates each loads + parses on its arch, and opens a PR
vendoring them. A `guard` job gates the heavy matrix to run ONLY on dispatch
or a real grammar-version change — ordinary code PRs cost zero matrix minutes.
- dart/proto: prefer a committed prebuild; fall back to today's source build
when none matches (no behavior change until prebuilds are vendored).
- kotlin: vendor it (Swift parity) instead of compiling the third-party
optionalDependency from source at the user's install — supersedes #2110's
optionalDependency mechanism. The ~23 MB parser.c is NOT vendored (the
workflow builds from the published package); only node-types + bindings +
prebuilds are. Removed from optionalDependencies; lock regenerated; probe,
parser-loader note, README/.devcontainer docs, and the #2110 tests updated.
DO NOT MERGE until vendor/tree-sitter-kotlin/prebuilds/ is populated by the
build-tree-sitter-prebuilds workflow: until then Kotlin is unavailable (vendored
with no source-build fallback). dart/proto remain fully functional throughout.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(install): guard 6/6 N-API prebuild coverage for every grammar
Regression guard so a toolchain-less install can never silently lose a tree-sitter
language on a supported platform-arch:
- Vendored grammars (vendor/tree-sitter-*): every one MUST ship a loadable N-API
prebuild for all 6 tuples {linux,darwin,win32}-{x64,arm64}. Asserts the
napi_register_module_v1 entry symbol in each .node (cross-platform, no need to
run the binary). Currently RED for dart/proto/kotlin until the
build-tree-sitter-prebuilds workflow populates their prebuilds/ — this is the
must-fill-before-merge gate (swift already passes 6/6).
- npm-dependency grammars: asserts upstream ships 6/6 N-API too, catching a
future platform drop. tree-sitter-c is allow-listed at 4/6 (missing
linux-arm64/win32-arm64) pending #2116; the guard also fails if that gap is
silently closed (prompting allow-list removal).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(install): vendor tree-sitter-c at 0.21.4 with GitNexus-built prebuilds (#2116)
tree-sitter-c is the one grammar dependency upstream ships incomplete prebuilds
for (4/6 — no linux-arm64/win32-arm64), AND it is a REQUIRED grammar: its own
`install` (node-gyp-build) compiles from source when no prebuild matches and
exits non-zero, so on a toolchain-less ARM host `npm install gitnexus` HARD-FAILS
at the c step — during npm's dependency phase, before any GitNexus postinstall
runs (so a postinstall "supplement" can't help).
Fix: vendor c prebuild-only at the pinned 0.21.4 (Kotlin pattern), with all six
prebuilds GitNexus-cross-built, and drop it from `dependencies`:
- vendor/tree-sitter-c/ (bindings + node-types + manifest + prebuilds); build
probe scripts/build-tree-sitter-c.cjs; added to the build workflow registry
(kind 'npm' — built from c@0.21.4 source).
- materialize-vendor-grammars.cjs: c is REQUIRED, so it is always materialized,
even under GITNEXUS_SKIP_OPTIONAL_GRAMMARS (it needs no toolchain).
- Removed from package.json dependencies + lockfile (nothing else needs npm c —
tree-sitter-cpp's dep on c is dev-only and not installed). Preserves the #1242
ABI pin: vendoring 0.21.4 keeps the good ABI while closing the ARM gap.
- parser-loader note + the prebuild-coverage guard + a cli-commands assertion
updated; c moves from the npm-gap allow-list into the vendored 6/6 cohort.
Verified: tsc clean, 31 unit tests pass, c loads/parses; the guard is RED for
c/dart/proto/kotlin until the workflow populates prebuilds (the must-fill gate).
Closes the operational risk in #2116.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(ci): source-build fallback for vendored c/kotlin so CI is healthy pre-prebuilds
The vendored prebuild-only grammars (c, kotlin) had empty prebuilds/ until the
build-tree-sitter-prebuilds workflow runs, so they could not load in CI — and
C is hard-required by cross-platform tests (tree-sitter-languages/parsing on
ubuntu+macos+windows), which I cannot pre-build for macos/windows locally. The
robust fix is a source-build fallback that works on every CI runner (all have a
toolchain), mirroring dart/proto:
- Vendor the grammar source (binding.gyp + src/) for c and kotlin; their build
scripts now PREFER a committed prebuild (toolchain-free) and fall back to
`node-gyp rebuild` from the vendored source when no prebuild matches. Verified
both compile against the hoisted node-addon-api@^8 and the runtime loads.
- prebuild-coverage guard is now bootstrap-tolerant: a grammar that vendors its
source (binding.gyp) may have an incomplete prebuild set (the workflow fills
it); a prebuild-only grammar (swift) still must ship all six. Any present
prebuild must still be N-API. Guard goes green; it re-tightens per-grammar as
the workflow populates prebuilds.
- actionlint: silence a false-positive SC2016 (JS template literals inside the
single-quoted `node -e` validate block).
Note: kotlin's generated parser.c is large (~23 MB on disk; compresses heavily
in git). Once the workflow populates all six kotlin prebuilds, the source serves
only as the fallback and could be slimmed if desired.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(docker): re-materialize+rebuild vendored grammars after npm prune
`npm prune --omit=dev` in the gitnexus CLI image drops anything not in
package.json's dependency tree — including the VENDORED tree-sitter grammars
(materialized by postinstall, not declared deps) and their built bindings. The
`serve` image analyzes/parses repos at runtime, so re-run the grammar postinstall
after the prune (in the toolchain-equipped builder) to restore them. Load-bearing
for tree-sitter-c, a core REQUIRED grammar now vendored (#2116): as a former
dependency it survived prune; vendored, it would not. Also restores
swift/dart/proto/kotlin, which were silently pruned from the image before.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(grammars): unify tree-sitter-swift with the vendored-source build pipeline
Swift was the last grammar handled differently — it shipped only upstream
prebuilds, while c/dart/proto/kotlin vendor their grammar source and use a
prefer-prebuild -> source-build-fallback activation script. Vendor swift's
source so all five are handled identically (one uniform build path).
- vendor/tree-sitter-swift: add binding.gyp (win-hardened), bindings/node/
binding.cc, src/parser.c (ABI-14 default, ~18 MB), src/scanner.c, and
src/tree_sitter/ headers. The 6/6 prebuilds are retained. The legacy
parser_abi13.c alternate is intentionally not vendored.
- build-tree-sitter-swift.cjs: rewrite the prebuild probe into the dart-style
prefer-prebuild then source-build fallback (keeps the GITNEXUS_SKIP gate and
the never-exit-non-zero postinstall invariant).
- build-tree-sitter-prebuilds.yml: register swift (kind 'vendored'); add its
package.json to the version-gated pull_request paths and a validate snippet.
- prebuild-coverage guard auto-moves swift into the source-fallback cohort
(binding.gyp now present); refresh the stale "swift is prebuild-only" comments.
- tests: add build-tree-sitter-swift-probe.test.ts; fix the pre-existing
build-tree-sitter-kotlin-probe.test.ts breakage (it still asserted the old
probe strings after kotlin's dart-style conversion); assert swift's vendored
source in cli-commands.test.ts.
- docs: README / .devcontainer / kotlin vendor README — swift's prebuilds are
now GitNexus-cross-built from vendored source like the rest, not upstream-only.
Verified: swift source-builds against node-addon-api@8 -> N-API binary -> loads
against the pinned tree-sitter@0.21.1 (ABI 14) -> parses cleanly.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(publish): gate a lean prebuilds-only npm tarball behind a coverage guard
Vendoring grammar source (parser.c) alongside the prebuilds means the npm
tarball now carries ~50 MB of generated source it almost never compiles (every
supported platform-arch has a prebuild). Prepare to drop it from the published
package once all prebuilds exist — safely.
- .npmignore: add a GATED, commented-out "lean publish" block that excludes the
source-build inputs (parser.c/scanner.c/tree_sitter/binding.gyp/binding.cc) but
keeps prebuilds/ + the runtime files. Uncommenting ships prebuilds-only.
- scripts/assert-publish-grammar-coverage.cjs: a prepack guard that refuses to
pack/publish if the source exclusion is active while any vendored grammar still
lacks 6/6 prebuilds (which would ship a grammar with no loadable binding). Wired
into `prepack` (runs on npm pack + publish, incl. the publish.yml dry-run) and
exposed as `npm run assert-publish-coverage`.
- test: pure-core decision cases + a real-repo publish-safety check that fails CI
if .npmignore is activated prematurely.
Net: the prebuilds already publish today (files: ["vendor"]); this makes the
future switch to a prebuilds-only tarball a one-line uncomment that can't ship a
dead grammar. The guard currently reports "source + prebuilds" (only swift has
6/6 prebuilds so far) and passes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(grammars): consolidate the 5 build-tree-sitter-*.cjs into one
The per-grammar activation scripts (c/dart/proto/swift/kotlin) were ~95%
identical — same prefer-prebuild → source-build → never-fail flow, differing only
in name, target_name, required-vs-optional, and the display label in warnings.
- scripts/build-tree-sitter-grammars.cjs: one registry-driven script. Bare call
builds all (postinstall); `... <name>` builds only the named grammars (so the
probe test can isolate one). c is `required: true` (ignores the opt-out gate);
the rest honor GITNEXUS_SKIP_OPTIONAL_GRAMMARS. Per-grammar try/catch + a final
process.exit(0) preserve the postinstall never-exit-non-zero invariant.
- package.json: postinstall is now `materialize && build-tree-sitter-grammars.cjs`
(was five chained `build-tree-sitter-<name>.cjs` calls).
- tests: replace the two near-identical *-probe.test.ts files with one
parameterized build-tree-sitter-grammars-probe.test.ts that also covers the
required-vs-optional opt-out split and an unknown-grammar arg.
- update cli-commands.test.ts postinstall assertions + the vendor c/kotlin/swift
README + swift provenance to reference the consolidated script.
Behavior is preserved (warnings normalized to one consistent format). Removes 5
scripts + 1 test file; adds 1 script + 1 test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(ingestion): lazy-load tree-sitter-c to prevent module-load crash
tree-sitter-c is now vendored prebuild-only (#2116) with 0/6 committed
prebuilds, so on a toolchain-less or `--ignore-scripts` install C has no native
binding. Three modules loaded it via a hard top-level `import C from
'tree-sitter-c'`, which throws ERR_MODULE_NOT_FOUND at module-load — crashing
`analyze` before parser-loader's optional/severity:error degradation can run.
This is the #2091/#2093 bug class (previously fixed for swift/dart/kotlin); C was
left static because it used to be an always-present npm dependency.
- languages/c/query.ts: load via the lazy guarded getLanguageGrammar(C), mirroring
swift/query.ts; the main-thread isLanguageAvailable filter ensures the getters
are reached only when C is present.
- workers/parse-worker.ts: guarded `_require('tree-sitter-c')` + conditional
languageMap spread, like swift/dart/kotlin.
- group/extractors/include-extractor.ts: guarded `_require`; getLanguageForFile
returns null for .c/.h when absent, so C include-extraction degrades to a no-op
(C++ unaffected).
- extend the registry-import-closure regression test (#2091/#2093) to assert C
also loads lazily at registry static-import time.
* fix(ci): repin attest-build-provenance to the real v2.4.0 SHA
The workflow pinned actions/attest-build-provenance@bd77c077… commented
`# v2.4.0`, but v2.4.0 is e8998f94… (verified via the GitHub API); bd77c077…
is an untagged mid-stream commit, so the SLSA-attestation step ran unvetted
action code and the comment misrepresented what runs. Repin to the real
v2.4.0 commit and drop the `# PLACEHOLDER-PIN` markers on both this line and
the setup-python pin (a26af69b… is already the correct v5.6.0 — only its
comment was stale). Update the header NOTE accordingly.
* fix(ci): skip the prebuild-PR aggregate when release App secrets are absent
The aggregate job mints a GitHub App token as its first step; with
RELEASE_APP_ID/RELEASE_APP_PRIVATE_KEY unset it hard-failed AFTER a full
(up-to-6-runner) native build. Since the `secrets` context isn't available in
a job-level `if:`, the guard job now computes a `release_app` boolean output
(a step can read secrets) and emits an actionable `::notice::`; aggregate
gates on it and skips cleanly, while the build job's artifacts still upload
(run with open_pr=false for artifacts-only).
* chore(ci): drop package-lock.json from the prebuild paths filter; widen build timeout
`gitnexus/package-lock.json` changes on nearly every dependency PR, so it
fired the prebuild workflow's guard job on unrelated churn (the matrix stayed
correctly skipped — `gitnexus/package.json` already covers the transition-window
pin, so removing the lock only drops guard noise). Also bump the native build
job timeout 30 -> 45 min for headroom compiling the 23 MB kotlin / 18 MB swift
parser.c, especially under arm emulation.
* fix(ci): event-gate the aggregate open-PR condition explicitly
`inputs.open_pr` is null on pull_request events, and the prior
`inputs.open_pr != false` leg relied on GHA's direction-ambiguous null
coercion (Codex F4) to decide whether to open the prebuild PR. Gate
explicitly on the event: a non-fork pull_request that bumped a grammar
version opens the prebuild PR (the documented flow), and `open_pr` is only
consulted on workflow_dispatch — so a manual run with open_pr=false stays
artifacts-only and no event's behavior rests on coercion.
* fix(publish): validate the effective npm-pack contents in the coverage guard
The publish guard inferred "is source shipped?" from a single .npmignore toggle
line, which a partial/out-of-order edit could defeat (exclude binding.gyp but
leave parser.c → unbuildable yet "source-shipping"). It now inspects the
EFFECTIVE tarball via `npm pack --dry-run --ignore-scripts --json` (the
--ignore-scripts avoids re-entering this guard through prepack): a grammar
"ships source" only when EVERY on-disk source-build input (binding.gyp +
binding.cc + parser.c + scanner.c when present + a tree_sitter header) is
actually in the packed file list.
This also surfaced that the gated lean-publish .npmignore block was inert:
package.json's `files: ["vendor"]` allow-list overrides .npmignore for the
vendored subtree, so those exclusion lines never dropped anything. Replace the
dead toggle with documentation of the real mechanism (narrow the `files` field)
and note the guard enforces safety on the effective pack regardless of how the
slim is done.
* test(prebuild): hard-gate declared-fully-prebuilt grammars on 6/6 coverage
The strict 6/6 prebuild assertion was dormant whenever a grammar vendors source
(binding.gyp) — which is every grammar — so a dropped prebuild passed CI
silently. Add a FULLY_PREBUILT allowlist of grammars GitNexus has committed 6/6
for (today: swift); those must keep all six even with a source fallback, so
losing one now fails CI. Grammars graduate into the set as the
build-tree-sitter-prebuilds workflow lands their binaries. (The static-import
degradation smoke is covered by the registry-import-closure regression test
extended in the C lazy-load commit.)
* chore(deps): promote node-gyp-build/node-addon-api to regular dependencies
Every vendored grammar's index.js does `require("node-gyp-build")` at runtime
to load even a prebuilt .node, so node-gyp-build is runtime-load-critical (and
node-addon-api is needed for the source-build fallback). They were
optionalDependencies, surviving `--omit=optional` only via the required
tree-sitter's transitive edge — correct today but fragile. Promote both to
regular dependencies so the contract is explicit (optionalDependencies is now
empty and removed). Lock the contract with a cli-commands assertion.
* chore(vendor): add Windows cflags parity block to tree-sitter-c/binding.gyp
c's binding.gyp used an unconditional `cflags_c: ["-std=c11"]`, while
kotlin/swift gate MSVC flags behind an `OS=='win'` condition (/std:c11 /utf-8).
Inert today (no non-ASCII bytes in c's parser.c, and node-gyp ignores cflags_c
on MSVC anyway), but align the three so a future source-build fallback on
Windows behaves consistently.
* docs(agents): correct stale optional-grammar / postinstall notes
AGENTS.md still said postinstall "patches tree-sitter-swift, builds
tree-sitter-proto" and that only kotlin/swift are "optional". Update to the
vendored-uniform model: postinstall materializes the vendored grammars and
prefers a committed prebuild (source-build only when none matches); c is
required while dart/proto/swift/kotlin are optional + skippable via
GITNEXUS_SKIP_OPTIONAL_GRAMMARS=1, with non-fatal warnings only on a
toolchain-less host with no matching prebuild.
* fix(install): preserve the backup and warn loudly on a failed materialize rollback
If renameSync(partial, dest) failed AND the rollback renameSync(backup, dest)
also failed, the grammar was left unmaterialized (node_modules/<name> missing)
with only a generic "could not materialize" warning — the recoverable backup at
<dest>.materialize-bak was unmentioned. Emit a CRITICAL warning naming the
backup path and the recovery command on that double-failure, and document that
the fail-soft catch removes only the scratch `partial`, never the `backup`
(which may be the sole recoverable copy). Never-throw / exit-0 contract intact.
* fix(publish): make the coverage guard's npm-pack inspection script-safe
The prepack guard shelled out to `npm pack --dry-run --ignore-scripts --json`,
but the `--ignore-scripts` flag is not reliably honored by npm pack's
prepare/prepack lifecycle on the CI npm — so build.js ran, polluted the --json
stdout with `[build] …`, and the guard's JSON.parse threw. That broke every
`npm pack` (packaged-install-smoke on ubuntu+windows) and failed the guard's own
real-repo unit test (the only coverage-job failure). Force script-skipping via
the reliable `npm_config_ignore_scripts` env config (also removes the prepack
re-entry/recursion risk) and parse defensively from the JSON-array start.
* fix(publish): make the coverage guard deterministic — read `files`, not `npm pack`
The npm-pack-based guard timed out in CI: `npm pack`'s prepare/prepack lifecycle
is not skipped by `--ignore-scripts` (flag or env config) on the CI npm, so the
inner pack ran the full build (~20s+) — fine for the slow smoke job, but it blew
past vitest's 30s test timeout in the coverage job (and risked re-entering this
prepack guard).
Replace it with a deterministic, fast (~0.1s) check that needs no subprocess:
since `files: ["vendor"]` OVERRIDES `.npmignore` for the vendored subtree (so
`.npmignore` can never drop vendored source — verified), the ONLY lever that can
exclude source is narrowing the package.json `files` field. The guard now reads
`files` directly: a grammar "ships source" iff `files` includes the vendor
subtree AND the grammar carries a buildable source set on disk. A lean publish
that narrows `files` while a grammar lacks 6/6 prebuilds still fails the gate.
* feat(ci): vendored tree-sitter grammar update monitor
Adds a weekly (+ dispatchable) workflow that checks each vendored grammar against
its source-of-origin (npm for swift/kotlin, the GitHub default branch for
dart/proto; c is excluded — held at 0.21.4 for ABI safety) and opens a PR
re-vendoring any update that is ABI-COMPATIBLE with the pinned tree-sitter@0.21.1
(LANGUAGE_VERSION 13-14).
ABI awareness is the point: most upstreams have moved to ABI 15 (newer
tree-sitter), so a blind "bump to latest" would open PRs that can't build. The
monitor fetches the candidate source, reads its parser.c LANGUAGE_VERSION, and
only re-vendors 13/14 — incompatible updates are reported (notice + job summary),
never applied. (Confirmed live: dart/proto upstreams are ABI 15 today and are
correctly held; swift/kotlin are current.)
The re-vendor refreshes only the source-build inputs + runtime entrypoints,
preserving the GitNexus-hardened binding.gyp / README / prebuilds; the version
bump then triggers build-tree-sitter-prebuilds.yml, whose ABI-validation is the
final safety net so a subtly-wrong re-vendor can't silently ship. PR creation is
gated on the RELEASE_APP secret (skips with a notice if absent), mirroring the
build aggregate. Unit test locks the ABI gate; the script is import-safe.
* feat(ci): monitor tree-sitter-c too (report-only, ABI-pinned)
c was excluded from the update monitor, so an upstream c update went unnoticed.
Include it, but as report-only via a `hold`: c is ABI-pinned at 0.21.4
(#1242/#858) and must not auto-bump without a tree-sitter runtime upgrade, so an
available c update is detected + surfaced (notice + job summary) but never
auto-PR'd — even if it were ABI-13/14. `--apply c` refuses defensively. (Live:
upstream c is 0.24.1 / ABI 15 today, so c is doubly held — reported, not applied.)
* fix(ci): drop the shell in the grammar monitor's github fetch (CodeQL)
CodeQL flagged the GitHub-tarball fetch — it used `bash -c "gh api …/tarball/$ref
> src.tgz && tar xzf src.tgz"`, interpolating the API-derived ref into a shell
command (the shell-command-injection family: "this shell command depends on an
uncontrolled file name"). Replace it with a shell-free path: capture `gh api`'s
binary tarball as a Buffer via execFileSync, write it to a fixed file, and
extract with execFileSync('tar', …). No shell, no injection surface. Verified the
dart/proto fetch + ABI read still work.
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
|
288b96f3e5
|
fix: batch query enrichment, bake FTS extension into CLI image, add FTS memory repro (#2108)
* perf(query): batch per-symbol process/cohesion/content lookups (N+1 -> 2-3) Port of the local-backend query-batching from gitnexus-enterprise PR #222 into the OSS local MCP backend. The query tool traced each matched symbol to its processes + cohesion (+ content) with up to 3N sequential pool round-trips; batch them into 2-3 'WHERE n.id IN $nodeIds' queries keyed back to each symbol by a prepended 'n.id AS nodeId' column. Output is identical: the aggregation loop is unchanged, iterates merged in the same order, and reads pre-fetched maps instead of issuing a query per symbol. Adaptations over a blind cherry-pick (would otherwise change output): - per-nodeId first-row community pick replaces the per-symbol LIMIT 1, so each symbol keeps its own community (not one for the whole batch); - batched rows regrouped to the originating merged item by nodeId so the JS-side RRF item.score still drives process ranking; - positional fallbacks shift +1 (process row[1..6], cohesion [1]/[2], content [1]); CodeRelation{type:...} relation form kept; IN-list chunked at 100 like the impact path. Adds a regression test asserting per-node community/content association (func:login keeps comm:auth; func:validate inherits no community). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(docker): bake LadybugDB FTS extension into the CLI/serve image The container runs `serve` under the default `load-only` extension policy (the read pool pins {policy:'load-only'}), so a runtime LOAD EXTENSION fts never INSTALLs. Dockerfile.cli copied the extension installer but never ran it, so the runtime user's HOME had no FTS extension: keyword search silently degraded (no FTS indexes written, ranking falls back to vector-only with only a warning field). Same class of footgun fixed for the Hub image in gitnexus-enterprise PR #222. Run install-duckdb-extension.mjs as the `node` user with the runtime HOME so INSTALL fts materializes the extension under $HOME/.lbdb/extension where the runtime LOAD resolves it offline. Pin ENV HOME=/home/node because Docker does not derive HOME from USER — without it the build-install and runtime-load would resolve different paths. Verified locally: INSTALL lands in $HOME/.lbdb/extension/0.17.0 and a fresh offline load-only `LOAD EXTENSION fts` resolves it. Dockerfile.web is unaffected (static frontend, no @ladybugdb backend). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(lbug): FTS evict->reload RSS repro + inert pool RSS tracing Settles the gitnexus-enterprise PR #222 root-cause hypothesis for OSS: does re-running LOAD EXTENSION fts on every pool evict->reload strand the native FTS arena (unbounded RSS growth in long-lived MCP serve), or does db.close() reclaim it (bounded by MAX_POOL_SIZE)? Static read could not decide — the native lbugjs.node binary documents no close->extension-unload contract. Adds gitnexus/scripts/bench/fts-evict-reload-rss.mjs: a NATIVE mode that reproduces the exact native sequence doInitLbug()+closeOne() perform (open Database -> Connection -> LOAD EXTENSION fts -> QUERY_FTS_INDEX -> close) across K self-built FTS fixtures, and a --via-pool mode that drives the real compiled pool (initLbug/executeParameterized/closeLbug) against an existing analyzed repo. Plus a behavior-neutral GITNEXUS_POOL_RSS_TRACE=1 stderr trace on pool init/close (stdout reserved for MCP JSON-RPC; single env read when disabled). RESULT (native, 24 and 40 cycles x 6 fixtures, --expose-gc): PLATEAU. RSS warms up to ~400 MB then flattens (40-cycle: +36 MB over cycles 1-10, +3 MB over 30-40; decelerating), not the linear climb a per-reload arena leak would produce (240 reloads x stranded arena = multi-GB). db.close() reclaims the FTS arena. The unbounded-leak hypothesis is NOT reproduced for the OSS path: the pool's LRU eviction + close-on-evict BOUNDS the footprint, which is exactly the protection the enterprise Hub supervisor lacked (it opened bridge DBs in-process without eviction -> 15 GB). => plan U4 (worker/process isolation) is NOT justified by this evidence; U1 + U2 are the only OSS-shared changes. Caveat: small fixtures + awaited close; a --via-pool run against a large analyzed repo over a long session is the production-faithful follow-up (instrumentation is in place for it). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(review): apply ce-code-review autofix feedback (#222 migration) Adversarial review found the U3 bench PLATEAU->no-leak conclusion was over-claimed from a 600-row fixture: a size-proportional FTS-arena leak would be sub-threshold at that scale. Strengthen the bench and make its verdict honest: - scale the fixture (--rows, UNWIND batch insert), probe ALL 5 FTS indexes in --via-pool (not 2 of 5), add a --no-await-close variant (the pool fire-and-forget close shape), and replace the absolute-delta gate with a SLOPE-DECELERATION 3-way verdict (PLATEAU / CLIMB / INCONCLUSIVE) plus step-discontinuity detection. At production-representative scale the synthetic runs are noisy/INCONCLUSIVE (deceleration argues against an UNBOUNDED leak but does not prove bounded), so plan U4 stays GATED on a --via-pool run against a real large analyzed repo -- not closed. - Dockerfile.cli: source the scratch-DB size from ENV GITNEXUS_LBUG_MAX_DB_SIZE (single source of truth) and add a build-time verify-only LOAD gate that fails the build on a HOME/extension-dir mismatch instead of silently degrading runtime keyword search. - install-duckdb-extension.mjs: additive verify-only mode (LOAD-only in a fresh process) + robust size parse; back-compatible with the runtime positional-size caller (validated). - tests: wire func:validate into a second process (proc:beta-flow) so the batched STEP_IN_PROCESS row[1..6] positional shift is exercised by a genuine multi-process symbol, and assert process ranking. No blast radius (75 seed-consuming tests pass). - pool-adapter.ts: trim the traceRss narrated-code comment (DoD 2.3). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(bench): classify a sustained sub-floor RSS slope as INCONCLUSIVE, not PLATEAU Tri-review P2: the FTS evict->reload verdict short-circuited to PLATEAU whenever secondHalfSlope < SUSTAIN_FLOOR, BEFORE the deceleration check — so a sustained (non-decelerating) linear leak below 0.5 MB/cycle was labeled PLATEAU ("no leak"), the label that would wrongly close plan U4. Extract median/slopeMbPerCycle/classifyVerdict into a pure, side-effect-free fts-rss-verdict.mjs (zero imports) so it is unit-testable without loading the native addon or running the bench, and fix the classifier: - epsilon-first gate: a truly flat tail (< 0.1 MB/cycle) is PLATEAU regardless of decelRatio (guards against over-correcting a real negative into INCONCLUSIVE); - a sustained sub-floor positive slope (>= epsilon, < floor, decelRatio >= 0.6) is INCONCLUSIVE — a slow creep RSS cannot distinguish from noise at this scale, so the honest label is "not resolved", never a clean PLATEAU; - the noise floor now scales with the WORKING-SET growth (peak-baseline), not the pre-DB baseline RSS (which is interpreter/addon overhead, larger in --via-pool mode, and would inflate the floor and HIDE leaks). Reconcile the stale "per-row-relative delta floor" docstring; add floor + decelRatio to the MACHINE line. New fts-rss-verdict.test.ts pins all label boundaries (flat->PLATEAU, sustained-sub-floor->INCONCLUSIVE, decelerated->PLATEAU, sustained-linear->CLIMB, step->INCONCLUSIVE, working-set floor, no import side effects). U1 does NOT add detection power for sub-floor leaks (RSS cannot attribute that magnitude) — it stops the false PLATEAU and routes that regime to the --via-pool run. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(query): signal partial/warning on a real enrichment failure (not benign missing-table) Tri-review P2: when a batched enrichment query (process/cohesion/content) threw, it was caught + logged and the chunk's symbols silently fell back to `definitions` with no signal — the caller could not tell "genuinely standalone" from "enrichment failed". Track an `enrichmentDegraded` flag in the three enrichment catch blocks and, at response build, compose a single `warning` (FTS-missing and/or the enrichment message, so neither overwrites the other) plus `partial: true`. Both fields are omitted on the clean path, so the success-path response shape is byte-identical. Crucially, the flag fires ONLY for a REAL failure (timeout / lock / native fault), NOT the benign "no Process/Community table" prepare error — a repo analyzed without processes/communities is a normal config, and firing `partial` on every such query would desensitize callers (isBenignMissingTableError gates it). New unit test test/unit/query-degraded-signal.test.ts (vi.mock pool-adapter, override hybrid search to feed one matched symbol, route STEP_IN_PROCESS -> throw): real failure -> warning+partial+symbol still returned; benign missing-table -> no signal; FTS-missing + enrichment failure -> both messages in one warning. Plus a success-path no-warning/no-partial assertion in the calltool integration test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
507f84b69a
|
fix(docker): symlink gitnexus binary onto $PATH in runtime image (#1551)
The README documents the Docker workflow as:
WORKSPACE_DIR=$HOME/code docker compose up -d
docker compose exec gitnexus-server gitnexus index /workspace/my-repo
…but `gitnexus` is not on $PATH inside the published image:
$ docker compose exec gitnexus-server which gitnexus
(empty)
$ docker compose exec gitnexus-server gitnexus --version
exec: "gitnexus": executable file not found in $PATH
The package.json `bin` entry (`"gitnexus": "dist/cli/index.js"`) would
normally surface via `node_modules/.bin/gitnexus`, but `npm prune
--omit=dev` in the builder stage strips that directory before the runtime
stage copies it in. The `dist/cli/index.js` itself already has the
`#!/usr/bin/env node` shebang and 755 permissions, so a single symlink
into /usr/local/bin makes the README's literal command work.
Verified locally:
$ docker build -f Dockerfile.cli -t gitnexus:local-pr-test .
$ docker run --rm gitnexus:local-pr-test gitnexus --version
1.6.4
$ docker run --rm gitnexus:local-pr-test gitnexus --help
Usage: gitnexus [options] [command]
…
$ docker run --rm -d --name t gitnexus:local-pr-test \
&& sleep 4 && docker exec t curl -s localhost:4747/api/health
{"status":"ok"}
CMD continues to invoke `node gitnexus/dist/cli/index.js serve …`
unchanged, so the change is additive and the server boot path is
untouched.
Refs #1549.
|
||
|
|
38ff7365e8
|
fix(docker): install ca-certificates in runtime image for TLS verification (#1545) (#1547)
Close: #1545 Co-authored-by: Gergő Magyar <gergomagyar@icloud.com> |
||
|
|
a3af0dcce2
|
fix(docker): include duckdb installer script in runtime image (#1502) | ||
|
|
e02c56f653
|
fix(security): Pin Docker Node base images, remove runtime package-manager CVE surface, verify Trivy on PRs, and harden Dependabot policy (#1455)
* fix: pin Docker node base images and remediate bundled npm CVEs Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/e0605c79-296e-4b3a-b6c3-4ad375950935 Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * fix: run trivy on docker PR changes and remove corepack Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/4d714047-4fc1-4af1-9734-91400a15568f Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * chore: add docker digest updates and normalize dockerfile comments Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/7d980908-a823-4c28-b074-9134ec672e84 Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * fix: add dependabot cooldown policies Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/a8531b8d-384b-4c54-84dd-a98b31993c44 Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * fix: remove unsupported dependabot cooldown keys Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/166df50e-c2fe-4d7f-ab41-e94c703338f6 Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * chore(node): bump CI + engines to Node 22; centralize NPM_VERSION via build ARG Closes the LOW findings from Claude Final Re-Review on PR #1455: - Bump engines.node to >=22.0.0 and align all CI workflows (ci-quality, pr-autofix, publish, release-candidate) and the composite setup actions on Node 22. Node 20 reached EOL on 2026-04-30; the test Docker image was already on 22. - Centralize the bootstrapped npm version in a single ARG NPM_VERSION per Dockerfile (cli, web, gitnexus/Dockerfile.test) so a security bump only requires updating one default per file with a clear cross-reference comment. --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> Co-authored-by: Gergo Magyar <gergomagyar@icloud.com> |
||
|
|
6ad53ff5c7
|
fix(ci): Change docker base image from alpine to debian (#1014)
* fix(docker): switch Dockerfile.cli from Alpine to Debian slim Alpine uses musl libc which is incompatible with @ladybugdb/core's glibc-compiled native binary, causing ERR_DLOPEN_FAILED on startup. Closes #1008 * fix(docker): resolve build and runtime failures in Dockerfile.cli - Add **/*.tsbuildinfo to .dockerignore and rm -f tsbuildinfo in builder to prevent stale incremental cache from skipping gitnexus-shared compilation - Install libstdc++6 from Debian Trixie for @ladybugdb/core native module compatibility (requires GLIBCXX_3.4.31) * fix(docker): use node:22-trixie-slim for GLIBCXX_3.4.31 support Replaces the manual Trixie libstdc++6 backport with the official node:22-trixie-slim base image, which ships GCC 14 runtime natively. |
||
|
|
3adb97e993
|
feat(docker): ship signed UI + CLI/server images via docker-compose (#967)
* Initial plan * docker: ship signed UI + CLI/server images via docker-compose Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/883bcee1-4a1d-4b3d-bbb9-accd8846da96 Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * docker: lock image version to npm package + harden cosign verify guidance Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/6afd4fcd-5656-4e02-b796-a22b59000bde Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * docker: add Sigstore ClusterImagePolicy + k8s admission docs Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/aea3dd70-e2a9-443a-b578-cb3eca4093e1 Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * docker(k8s): collapse redundant image globs in ClusterImagePolicy Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/aea3dd70-e2a9-443a-b578-cb3eca4093e1 Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * docker(ci): drop deprecated COSIGN_EXPERIMENTAL, dead build-args, and loose verify regex in comment Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/bdf0d2cf-607c-4558-982a-be9b216b2d36 Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * docker(ci): use ${{ github.repository }} in verify-comment regex for fork portability Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/bdf0d2cf-607c-4558-982a-be9b216b2d36 Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * style(deploy): prettier-format cluster-image-policy.yaml (single quotes) Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/b09a016e-56a1-4b73-bacc-e69084a48782 * ci(docker): drop workflow_dispatch, harden signing loop, fix verify-comment placeholder Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/6755064c-7871-4b2e-9b46-b4779eb215ac --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> |