scriptoriumadmin/GitNexus
1
0
Fork 0
mirror of https://github.com/abhigyanpatwari/GitNexus.git synced 2026-09-20 00:11:37 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 7
GitNexus/gitnexus
History
Gergő Magyar d3a7ce95a5
feat(core): adopt pino structured logger (#1336) 
* feat(core): adopt pino structured logger + add no-console eslint forcing function

Adds `pino` as the project-wide structured logger via a thin wrapper at
`gitnexus/src/core/logger.ts` exposing `createLogger(name, opts?)` and a
default `logger` singleton. Migrates the only security-relevant `console.warn`
site (`bridge-db.ts` `openBridgeDbReadOnly` retry-exhaustion path) to
`bridgeLogger.debug({groupDir, err, attempts}, 'msg')`.

Pino's NDJSON output is structurally log-injection-resistant (one record per
newline, all string fields JSON-escaped) — replaces the hand-rolled
`sanitizeLogValue` pattern that PR #1329 added on the `fix/insecure-tempfile-core`
branch. PR #1329's sanitizer remains as fallback until CodeQL confirms #466
closes via pino on this branch.

Also adds an ESLint `no-console: warn` rule scoped to
`gitnexus/src/**/*.ts` (excluding `cli/`, `server/`, `test/`, `bin/`, and the
logger module itself) as the forcing function — new code can't regress.
Existing 134 sites in `core/`, `mcp/`, `config/`, `storage/` get a
`// eslint-disable-next-line no-console -- TODO(pino-migration)` marker in a
follow-up commit so lint stays clean and the remaining work is grep-able.

Operator behaviour preserved:
  - `GITNEXUS_DEBUG_BRIDGE` truthy → bridgeLogger logs at debug level
  - `GITNEXUS_DEBUG_BRIDGE` unset → bridgeLogger filters debug messages
  - Output is NDJSON in production / CI / vitest
  - pino-pretty engages only when stdout is a TTY AND CI/VITEST env unset

Tests: 11 new logger.test.ts cases (level methods, debugEnvVar gating,
destination capture, undefined Error.message safety, CR/LF/U+2028/ANSI
single-record invariant). Group test suite (388 tests) passes unchanged.

`--no-verify`: pre-commit hook fails on PR #1302's pre-existing TS regression
at `scope-resolution/pipeline/run.ts:160` on main; documented in commit
`348d0c91` and recurring across the security-fix series.

Refs: #466 (codeql js/log-injection), PR #1329 follow-up.

* chore(lint): baseline-suppress 134 existing console.* sites with TODO(pino-migration)

Mechanical pass: prepends `// eslint-disable-next-line no-console -- TODO(pino-migration)`
above each existing `console.*` call in `gitnexus/src/{config,core,mcp,storage}/`
that the new ESLint rule would otherwise flag. CLI/server are exempt at the
config level (legitimate stdout output).

Zero functional changes. Generated by an in-repo node script that consumes
`eslint --format json` output and prepends the marker line at each reported
location. Verification:
  npx eslint gitnexus/src/      → 0 no-console warnings
  grep -rn "TODO(pino-migration)" gitnexus/src/ | wc -l  → 134

The marker tags inventory the remaining migration surface so future sweep
PRs can grep their target list. When a follow-up PR migrates a site, the
marker comment is removed alongside the `console.*` → `logger.*` swap.

`--no-verify`: same as parent commit (PR #1302 pre-existing TS regression on main).

* refactor(core): complete pino migration — replace all 134 console.* sites + flip ESLint to error

Codebase-wide sweep of every `TODO(pino-migration)` site flagged in commit
3e8e7c2a. 49 source files migrated, 134 `console.*` calls converted to
`logger.*` using pino's structured-arg convention (object first, message
second). All `TODO(pino-migration)` markers removed. ESLint `no-console`
flipped from `warn` to `error` so future regressions fail CI.

Source-side changes (49 files):
- Mechanical pattern: `console.X(msg)` → `logger.X(msg)`,
  `console.X(msg, val)` → `logger.X({val}, msg)` (bare-id shorthand) or
  `logger.X({err: val}, msg)` for Error-shaped names.
- Hand-fixed special cases:
  * `import-processor.ts`: `console.group/groupEnd` block → single
    `logger.error({...}, 'tree-sitter query error')` with merged fields.
  * `extension-loader.ts`: `console.warn` as default callback →
    `(msg) => logger.warn(msg)` lambda binding.
  * `cursor-client.ts`: variadic `console.log(...args)` → `logger.info({args}, '[cursor-cli]')`.
- `console.log` → `logger.info` (preserves operator visibility at default level)

Logger module (`gitnexus/src/core/logger.ts`) updates:
- Default level `info` (matches pino default; preserves `console.log` visibility)
- Default destination is **stderr (fd 2)** — keeps stdout (fd 1) clean for
  CLI tool data output (#324). Pino's default is stdout, which would
  contaminate `gitnexus query`/`cypher`/`impact` JSON output.
- Pretty-print TTY check now reads `process.stderr.isTTY` (matches new sink).
- `_captureLogger()` test helper: Proxy-backed singleton lets tests redirect
  the shared logger to a `MemoryWritable` and assert on captured NDJSON
  records via `cap.records()` / `cap.text()`. Restored on teardown.

Test-side changes (10 files):
- `max-file-size.test.ts`, `filesystem-walker.test.ts`, `worker-pool.test.ts`,
  `calltool-dispatch.test.ts`, `grpc-extractor.test.ts`,
  `ignore-service.test.ts`, `index-repo-command.test.ts`,
  `sequential-language-availability.test.ts`, `sync.test.ts`,
  `rust-workspace-extractor.test.ts`: replace `vi.spyOn(console, 'X')`
  patterns and ad-hoc `console.warn = ...` reassignments with
  `_captureLogger()` + `cap.records()` assertions.
- `analyze-worker-timeout.test.ts`: kept original `vi.spyOn(console, 'error')`
  — exercises CLI code (cli/analyze.ts) which is exempt from the migration
  (legitimate stderr output is the contract).

ESLint config: removed the `warn` baseline; new rule block is `error`
scoped to `gitnexus/src/**/*.ts` with the existing cli/server exemption
preserved. Logger module + test/ + bin/ remain off.

Verification:
- `npm test` — 7762/7762 pass (excluding 29 pre-existing PR #1302 Go
  resolver failures unrelated to this change)
- `npx eslint gitnexus/src/` — 0 errors, 426 pre-existing warnings unchanged
- `npx tsc --noEmit` — only the pre-existing PR #1302 TS error
- `git grep -n "TODO(pino-migration)"` — 0 matches
- `git grep -n "console\." gitnexus/src/ | grep -v cli/ | grep -v server/ | grep -v logger.ts` — 2 comment references only

`--no-verify`: pre-commit hook fails on PR #1302's TS regression at
`scope-resolution/pipeline/run.ts:161` on main; same justification as the
parent commits in this PR series.

Refs: #466 (codeql js/log-injection), PR #1336.

* chore(tests): remove unused 'vi' import from worker pool and grpc extractor tests

* test: replace console.warn with logger capture in loadIgnoreRules error handling

* refactor(cli/server): tighten no-console — migrate diagnostic warn/error to pino

Tighten the cli/server ESLint exemption from `'no-console': 'off'` to
`'no-console': ['error', { allow: ['log'] }]`. `console.log` IS the contract
on stdout (CLI tool output for `gitnexus query | jq` consumers, server
pretty-printed banners) and remains permitted. Diagnostic logging
(`warn`/`error`/`debug`/`info`) goes through pino like the rest of the
codebase — same NDJSON-on-stderr routing, same structured-fields convention,
same log-injection-resistance.

Migrated 88 sites across 13 files (cli + server). Three sites in
`cli/analyze.ts` are intentional UI patterns (the progress-bar swaps
`console.warn`/`console.error` to `barLog` to prevent terminal corruption
during long-running indexing); these carry inline `// eslint-disable-next-line
no-console -- intentional console-routing for progress bar UX` comments
explaining why they bypass the rule.

Test wiring updated:
- `analyze-worker-timeout.test.ts`: switched back to `_captureLogger` (was
  reverted to console-spy in an earlier commit when cli/ was exempt).
  Imports `_captureLogger` dynamically inside each test so it sees the
  same module instance as analyze.js after `vi.resetModules()` rebuilds
  the singleton.
- `web-ui-serving.test.ts`: console-warn assertion swapped to
  `cap.records()` lookup of the new structured log shape (`r.err`).

Verification: full test suite passes (7791/7791 excluding 29 pre-existing
PR #1302 Go failures); 0 lint errors; 0 tsc errors (after the earlier
gitnexus-shared rebuild fix).

Refs: PR #1336.

* fix(logger): address PR review findings — pretty-stderr, log levels, structured fields

Three findings from the multi-agent review on PR #1336:

**[CRITICAL] pino-pretty was writing to stdout, breaking piped CLI output.**
`tryBuildPrettyTransport()` did not set the pino-pretty `destination`
option. pino-pretty defaults to fd 1 (stdout) even when pino's own
destination is fd 2 (stderr). With `shouldUsePretty()` true (interactive
shell, stderr-TTY) the formatted log lines landed on stdout — so
`gitnexus query "auth" | jq` saw query-timing log noise interleaved with
the JSON result and `jq` failed. Fix: pass `destination: 2` to the
pino-pretty transport options. The non-pretty path already used
`pino.destination({dest: 2})`; this aligns the two paths.

**[HIGH] `logQueryTiming()` and MCP startup banner used `logger.error()`
for non-error conditions.** Migration artifacts. Operator alerting rules
fire on every level≥40 record, so per-query timing telemetry at error
level would generate false positives on every successful query, and a
healthy MCP startup would page on-call.

  - `local-backend.ts:logQueryTiming` → `logger.debug` with structured
    `{ query, totalMs, phases }` fields. Operators wanting per-query
    timing set the appropriate log level.
  - `local-backend.ts:logQueryError` → kept at `error` (it IS an error)
    but restructured to `{ context, err: msg }` instead of template-literal
    interpolation.
  - `mcp.ts` "starting with N repos" banner → `logger.info` with
    `{ repoCount, repos }` structured fields.
  - `mcp.ts` "no repos yet" notice → `logger.warn` (operator-actionable
    but non-fatal; server still starts and serves).

**[MEDIUM] Hot-path worker-pool warns used template-literal
interpolation.** Two `logger.warn` sites in `core/ingestion/workers/
worker-pool.ts` (job-split timeout, single-item retry) embedded all
diagnostic context in the message string instead of pino's
mergingObject. Restructured to canonical
`logger.warn({ workerIndex, items, estimatedBytes, ... }, 'msg')` so log
aggregators can query fields independently. Existing tests pin on
`r.msg.includes('Splitting into ...')` / `'Retrying with ...'` — preserved
in the message string so test assertions still pass.

Verification:
- Logger tests 11/11 pass
- Worker-pool integration tests 21/21 pass
- Full suite 7791/7791 pass (excl. pre-existing PR #1302 Go failures)
- Lint 0 errors; tsc clean
- pino-pretty `destination: 2` confirmed via the pretty-build path

Refs: PR #1336 review.

* fix(logger): address ce-code-review findings — best-judgment auto-fix batch

Multi-agent review of PR #1336 (post-merge with main) found 17 actionable
findings. This commit applies the concrete fixes; remaining items are
documented as residual work below.

APPLIED (12 fixes across 13 files)

P1 — bugs introduced by the migration

- parse-worker.ts:1451 — restore the dropped `else`. The migration replaced
  `if (parentPort) ...; else console.warn(message)` with an unconditional
  `logger.warn(message)`, double-logging every warning when running in a
  worker thread.
- grpc-extractor.test.ts:585 — remove the spurious
  `import { _captureLogger } from '...';` line that was injected INSIDE
  the TypeScript template-literal string used as the `auth.client.ts`
  test fixture. It was being parsed as part of the fake source and
  could mask deduplication regressions.
- eval-server.ts (8 sites), mcp/core/embedder.ts (2 sites), local-backend.ts
  (1 site) — `logger.error` → `logger.info`/`logger.warn` for informational
  lifecycle banners (listening on, route listings, idle-timeout, model-load,
  vector-fallback). These were emitting at pino level 50 and tripping
  log-aggregator error alerts on every successful start.
- core/logger.ts — wire `GITNEXUS_LOG_LEVEL` env var into `buildBaseOptions`.
  The `logQueryTiming` comment told operators to set this var; previously
  it had zero effect because `buildBaseOptions` hardcoded `level: 'info'`.
- core/logger.ts — add a guard to `_captureLogger()` that throws when a
  prior capture is still active. Forgetting `restore()` between captures
  silently abandoned the previous MemoryWritable and corrupted logger
  state for the rest of the vitest worker.
- core/logger.ts — Proxy `get` trap now uses `Reflect.get(inner, prop, inner)`
  instead of `(inner as ...)[prop as string]`. The `prop as string` cast
  silently coerced symbol-keyed lookups (e.g. Symbol.toPrimitive) to the
  wrong key.
- embedding-pipeline.ts:259 — restore the `if (!vectorAvailable && isDev)`
  guard around `vectorUnavailableMessage`. The migration dropped both
  guards, emitting a warn on every production analyze run on non-VECTOR
  platforms.

P2 — error-shape fixes for pino's err serializer

- serve.ts (uncaughtException + unhandledRejection) — pass the Error
  itself in `{ err }` so pino's serializer captures type/message/stack.
  Was passing `err.message` (string) which lost the stack and shape.
- api.ts:1823 — same fix; was passing `err?.stack || err`.
- wiki.ts:587 — was passing the bare Error as the first arg to
  `logger.error(err)`, which pino coerces via `.toString()` and loses the
  shape; changed to `logger.error({ err }, 'wiki command failed')`.

P2 — design hygiene

- core/logger.ts — hoist `MemoryWritable` out of `_captureLogger` and
  export it; also export `PinoLogRecord` and `LoggerCapture`. Removes
  the duplicate definition in `logger.test.ts`.
- core/logger.ts — `_getInner()` now delegates to `createLogger()` for
  both branches instead of constructing pino directly when an active
  destination is set. Future `createLogger` defaults (serializers,
  redaction) now apply uniformly to test-capture mode.
- eslint.config.mjs — extract the three MCP stdout-write selectors into
  a shared `mcpStdoutWriteSelectors` const so the lbug-adapter
  file-specific override spreads them in instead of re-listing them
  verbatim. Stops a future selector addition from silently dropping
  protection in lbug-adapter.

P2 — test coverage

- worker-pool.test.ts ("rejects dispatch when replacement worker crashes")
  — added an assertion on `cap.records()` so the test actually verifies
  the warn-level emission, not just the rejection. Was capturing pino
  output and discarding it.
- logger.test.ts — added 4 new tests for `_captureLogger` lifecycle:
  basic capture, restore-stops-writes, double-capture-throws, and
  recapture-after-restore. The mechanism every converted test depends on
  was previously untested in its own module.

NOT APPLIED — residual actionable work (5 findings)

- #7 CLI human-readable error messages emit as JSON in non-TTY contexts
  (analyze.ts validators, EADDRINUSE banners, OOM/ERESOLVE recovery
  blocks). Design issue: needs a dedicated `cliMessage()` helper that
  bypasses pino. Scope is too large for this batch.
- #10 `tryBuildPrettyTransport()` unreachable catch / pino-pretty
  resolves lazily — the catch can never fire. Fix is to probe with
  `require.resolve('pino-pretty')` inside the try block. Mechanical but
  changes the safety contract; deferred for review.
- #11 inconsistent logger call shapes across the migration (bare strings
  vs `{ field }, 'msg'` vs multi-line banners). Advisory — no concrete
  mechanical fix; needs a stylistic convention pass.
- #12 `pino.destination({ dest: 2, sync: true })` blocks the event loop
  on every logger call from the main process. Fix needs `sync: false` +
  `flushSync()` hooks on `beforeExit`/`SIGTERM`. Non-trivial; deferred.
- #17 `pino.final()` not registered in serve.ts crash handlers — async
  pretty-print path may not flush before `process.exit(1)` on dev TTY.
  Defer; bounded to dev TTY scenarios.

Validation
- `tsc --noEmit` clean
- ESLint MCP-reachable scope: 0 errors, 219 pre-existing any/non-null warnings
- `vitest run test/unit`: 5204 passed, 10 skipped (4 new lifecycle tests)
- focused: logger.test.ts 26/26, worker-pool.test.ts 22/22, grpc-extractor 39/39

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(logger): harden runtime — pino-pretty packaging, sync writes, CLI UX

Implements the 5 logger-runtime findings from the multi-agent code review
and Codex's adversarial review (plan: docs/plans/2026-05-07-001-fix-pino-logger-runtime-hardening-plan.md).

U1 — pino-pretty to runtime dependencies (Codex P1, no-ship)
- Move pino-pretty from devDependencies to dependencies in
  gitnexus/package.json so production installs (npm i -g, npx) don't
  crash inside createLogger() the first time stderr is a TTY.
- Lockfile regenerated; npm ls --omit=dev confirms placement.

U2 — Real pino-pretty availability probe
- Replace tryBuildPrettyTransport()'s dead try/catch (wrapped a plain
  object literal that cannot throw) with a require.resolve('pino-pretty')
  probe via createRequire. Memoize via _prettyAvailable cache.
- On miss, emit a single stderr warning and fall back to defaultDestination
  (NDJSON on stderr). Belt-and-suspenders for --omit=optional and any
  other install variant where pino-pretty turns out to be missing.
- Export _tryBuildPrettyTransport + _resetPrettyAvailableCache for tests.
- Add 3 unit tests covering happy path, memoization, and warning bound.

U3 — Async destination + graceful-exit flush
- Switch defaultDestination() to pino.destination({ dest: 2, sync: false })
  so logger calls don't issue a blocking write(2) syscall on every record.
- Cache the destination in module-level _dest. Register process.on(
  'beforeExit', flushSync) once at module load (gated on !VITEST so
  vitest's between-test cleanup doesn't fight _captureLogger).
- Export flushLoggerSync() helper. Wire into existing shutdown handlers
  in cli/analyze.ts (SIGINT) and mcp/server.ts (SIGINT/SIGTERM/shutdown
  helper) so async-buffered records reach stderr before process.exit.
- Add smoke test for flushLoggerSync's no-op-on-empty-state contract.

U4 — Crash flush in serve.ts and api.ts
- Add flushLoggerSync() between logger.error and process.exit(1) in
  serve.ts uncaughtException/unhandledRejection handlers and api.ts
  uncaughtException handler.
- Pino v10 removed pino.final (the v10 transport architecture handles
  worker-thread flush on process exit automatically), so the simpler
  log + flush + exit pattern replaces the original plan's pino.final
  integration. Captured in the commented logger.ts JSDoc.
- api.ts shutdown() also flushes before process.exit(0).

U5 — CLI message helper + migrate top offenders
- New gitnexus/src/cli/cli-message.ts exporting cliInfo/cliWarn/cliError.
  Each writes plain text to process.stderr AND tees a structured pino
  record so users see human-readable banners while log aggregators get
  NDJSON. Auto-newlines, preserves embedded newlines, accepts structured
  fields.
- Add 6 unit tests covering tee shape, level mapping, newline handling,
  multi-line preservation, empty-message edge case.
- Migrate top user-facing offenders identified in review:
  - cli/analyze.ts: validators (--worker-timeout, --embeddings, --embedding-*,
    --embedding-device) + recovery blocks (RegistryNameCollisionError,
    OOM/heap, ERESOLVE, MODULE_NOT_FOUND). Multi-line recovery hints
    consolidated into single cliError calls instead of N consecutive
    logger.error('') lines that emitted N empty NDJSON records.
  - cli/serve.ts: EADDRINUSE banner + Failed-to-start error.
  - cli/eval-server.ts: listening banner with full endpoint list (split
    plain-text human banner from structured aggregator record so users
    don't see {"level":30,"endpoints":[...]} in their terminal).
- Update analyze-embeddings-limit.test.ts to spy on process.stderr.write
  instead of console.error (the validator now bypasses console).

Validation
- tsc --noEmit clean
- ESLint touched-file scope: 0 errors, pre-existing any/non-null warnings only
- vitest run test/unit: 5213 passed / 10 skipped (modulo a pre-existing
  parallel-worker flake in test/unit/group/insecure-tempfile.test.ts that
  doesn't reproduce when group/ is run in isolation — 456/456 there)
- focused: logger.test.ts 19/19, cli-message.test.ts 6/6,
  analyze-embeddings-limit.test.ts 9/9

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(cli): route hard-exit diagnostics through cliError to defeat buffer drain race

Codex's adversarial review on PR #1336 flagged that nine `logger.error/warn`
+ `process.exit(N)` sites in CLI subcommands could lose the diagnostic
because the pino destination is `sync: false` (plan 001 U3) and
`process.exit` skips the `beforeExit` flush hook. Symptom: a non-zero
exit with no visible message.

U1: migrate the nine sites to `cliError`/`cliWarn`
- gitnexus/src/cli/tool.ts (5 sites — query/context/impact/cypher usage
  errors + the no-index init failure)
- gitnexus/src/cli/remove.ts (3 sites — ambiguous-target, unsafe-storage-
  path, and rm-failed catches)
- gitnexus/src/cli/eval-server.ts (1 site — the no-index startup warn,
  using cliWarn to preserve the warn-level semantics)

`cliError`/`cliWarn` (gitnexus/src/cli/cli-message.ts, plan 001 U5) write
plain text directly to process.stderr AND tee a structured pino record.
The direct-stderr path bypasses the buffered destination entirely, so the
diagnostic survives any subsequent `process.exit` regardless of buffer
state. Removed the now-unused `import { logger }` from tool.ts (lint
caught it).

U2: regression test at gitnexus/test/integration/cli/tool-no-index-stderr.test.ts
- Spawns `node dist/cli/index.js query whatever` with empty
  GITNEXUS_HOME, asserts exit code 1 + stderr contains the no-index
  diagnostic. Pattern mirrors test/integration/mcp/server-startup.test.ts.

Honesty caveat: the regression signal is not deterministic. The
SonicBoom buffer happens to drain in time for short messages on a piped
stderr, so the test passes both pre- and post-fix in this environment.
The architectural fix is still correct — `cliError` removes the timing
dependency entirely, so future pino changes or platform-specific buffer
behavior can't reintroduce the race. The test locks the user-visible
contract (stderr must carry the diagnostic) even if it doesn't reproduce
the exact failure mode under controlled timing.

Validation:
- `tsc --noEmit` clean
- ESLint touched-file scope: 0 errors, 19 pre-existing any warnings
- `vitest run test/unit/cli-message.test.ts test/unit/logger.test.ts`:
  25/25 pass
- New regression test passes against built dist/

Closes Codex P1 from the post-runtime-hardening review.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(ci): replace console.error with cliWarn in optional-grammars

CI lint failure on the merged tree: the repo-wide pino-migration rule
(no-console: ['error', { allow: ['log'] }] for cli/) forbids
console.error in CLI code. optional-grammars.ts was added by PR #1383
and used console.error for missing/broken-grammar warnings; that worked
under the MCP-narrow ESLint rule alone but breaks once the merged
broader rule applies.

Two sites migrated to cliWarn (operator-actionable warnings, not
errors): the broken-binding diagnostic (line 69) and the missing-grammar
diagnostic (line 99). Each now writes plain text to stderr AND tees a
structured logger.warn record with grammar/extensions/error fields.

Also: hoisted opts?.relevantExtensions into a local const so the closure
inside .some() narrows correctly without the no-non-null-assertion lint
warning at line 96.

Validation
- ESLint optional-grammars.ts: 0 errors, 0 warnings (was 2 errors + 1 warning)
- tsc --noEmit clean
- vitest run cli-message + logger: 25/25 pass

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 20:56:25 +01:00
..
.claude feat: configure prettier with pre-commit hook (#563) 2026-03-28 14:58:04 +00:00
hooks/claude fix(hook): resolve canonical repo root + guard read-only FTS ensure (#1226) 2026-04-30 18:12:03 +01:00
scripts fix(mcp): close MCP server timeout — stdout discipline + cold-start friction (#1383) 2026-05-07 09:14:33 +01:00
shadow-parity-dashboard feat(ingestion): shadow-mode parity harness + static dashboard (#923, RFC #909 Ring 2 PKG) (#972) 2026-04-18 21:30:43 +01:00
skills docs: clarify PostToolUse hook is notification-only, not auto-reindex (#1070) 2026-04-26 06:06:15 +01:00
src feat(core): adopt pino structured logger (#1336) 2026-05-07 20:56:25 +01:00
test feat(core): adopt pino structured logger (#1336) 2026-05-07 20:56:25 +01:00
vendor fix(swift): use official prebuilt parser runtime (#1130) 2026-04-28 09:57:42 +01:00
.env.example feat: add .env.example, empty batch guard, dimension mismatch warning 2026-03-20 20:15:59 -04:00
.npmignore fix: add preinstall cleanup to prevent ENOTEMPTY on global upgrade (#843) 2026-04-15 14:10:49 +01:00
CHANGELOG.md chore: release v1.6.3 (#1064) 2026-04-24 16:33:59 +01:00
Dockerfile.test fix(swift): use official prebuilt parser runtime (#1130) 2026-04-28 09:57:42 +01:00
package-lock.json feat(core): adopt pino structured logger (#1336) 2026-05-07 20:56:25 +01:00
package.json feat(core): adopt pino structured logger (#1336) 2026-05-07 20:56:25 +01:00
README.md fix(lbug): prevent DuckDB extension install hangs (#1129) 2026-04-27 23:09:17 +01:00
tsconfig.json feat: configure prettier with pre-commit hook (#563) 2026-03-28 14:58:04 +00:00
tsconfig.test.json test: add test suite with vitest (unit + integration + fixtures) 2026-03-01 20:07:02 +05:30
vitest.config.ts test(ci): isolate native LadybugDB and CLI e2e flakes 2026-04-30 19:45:26 +01:00

README.md

GitNexus

Graph-powered code intelligence for AI agents. Index any codebase into a knowledge graph, then query it via MCP or CLI.

Works with Cursor, Claude Code, Codex, Windsurf, Cline, OpenCode, and any MCP-compatible tool.

npm version License: PolyForm Noncommercial

Why?

AI coding tools don't understand your codebase structure. They edit a function without knowing 47 other functions depend on it. GitNexus fixes this by precomputing every dependency, call chain, and relationship into a queryable graph.

Three commands to give your AI agent full codebase awareness.

Quick Start

# Index your repo (run from repo root)
npx gitnexus analyze

That's it. This indexes the codebase, installs agent skills, registers Claude Code hooks, and creates AGENTS.md / CLAUDE.md context files — all in one command.

To configure MCP for your editor, run npx gitnexus setup once — or set it up manually below.

gitnexus setup auto-detects your editors and writes the correct global MCP config. You only need to run it once.

Editor Support

Editor MCP Skills Hooks (auto-augment) Support
Claude Code Yes Yes Yes (PreToolUse) Full
Cursor Yes Yes — MCP + Skills
Codex Yes Yes — MCP + Skills
Windsurf Yes — — MCP
OpenCode Yes Yes — MCP + Skills

Claude Code gets the deepest integration: MCP tools + agent skills + PreToolUse hooks that automatically enrich grep/glob/bash calls with knowledge graph context.

Community Integrations

Agent Install Source
pi pi install npm:pi-gitnexus pi-gitnexus

MCP Setup (manual)

If you prefer to configure manually instead of using gitnexus setup:

Claude Code (full support — MCP + skills + hooks)

# macOS / Linux
claude mcp add gitnexus -- npx -y gitnexus@latest mcp

# Windows
claude mcp add gitnexus -- cmd /c npx -y gitnexus@latest mcp

Codex (full support — MCP + skills)

codex mcp add gitnexus -- npx -y gitnexus@latest mcp

Cursor / Windsurf

Add to ~/.cursor/mcp.json (global — works for all projects):

{
  "mcpServers": {
    "gitnexus": {
      "command": "npx",
      "args": ["-y", "gitnexus@latest", "mcp"]
    }
  }
}

OpenCode

Add to ~/.config/opencode/config.json:

{
  "mcp": {
    "gitnexus": {
      "command": "npx",
      "args": ["-y", "gitnexus@latest", "mcp"]
    }
  }
}

How It Works

GitNexus builds a complete knowledge graph of your codebase through a multi-phase indexing pipeline:

  1. Structure — Walks the file tree and maps folder/file relationships
  2. Parsing — Extracts functions, classes, methods, and interfaces using Tree-sitter ASTs
  3. Resolution — Resolves imports and function calls across files with language-aware logic
    • Field & Property Type Resolution — Tracks field types across classes and interfaces for deep chain resolution (e.g., user.address.city.getName())
    • Return-Type-Aware Variable Binding — Infers variable types from function return types, enabling accurate call-result binding
  4. Clustering — Groups related symbols into functional communities
  5. Processes — Traces execution flows from entry points through call chains
  6. Search — Builds hybrid search indexes for fast retrieval

The result is a LadybugDB graph database stored locally in .gitnexus/ with full-text search and semantic embeddings.

MCP Tools

Your AI agent gets these tools automatically:

Tool What It Does repo Param
list_repos Discover all indexed repositories —
query Process-grouped hybrid search (BM25 + semantic + RRF) Optional
context 360-degree symbol view — categorized refs, process participation Optional
impact Blast radius analysis with depth grouping and confidence Optional
detect_changes Git-diff impact — maps changed lines to affected processes Optional
rename Multi-file coordinated rename with graph + text search Optional
cypher Raw Cypher graph queries Optional

With one indexed repo, the repo param is optional. With multiple, specify which: query({query: "auth", repo: "my-app"}).

MCP Resources

Resource Purpose
gitnexus://repos List all indexed repositories (read first)
gitnexus://repo/{name}/context Codebase stats, staleness check, and available tools
gitnexus://repo/{name}/clusters All functional clusters with cohesion scores
gitnexus://repo/{name}/cluster/{name} Cluster members and details
gitnexus://repo/{name}/processes All execution flows
gitnexus://repo/{name}/process/{name} Full process trace with steps
gitnexus://repo/{name}/schema Graph schema for Cypher queries

MCP Prompts

Prompt What It Does
detect_impact Pre-commit change analysis — scope, affected processes, risk level
generate_map Architecture documentation from the knowledge graph with mermaid diagrams

CLI Commands

gitnexus setup                   # Configure MCP for your editors (one-time)
gitnexus analyze [path]          # Index a repository (or update stale index)
gitnexus analyze --force         # Force full re-index
gitnexus analyze --embeddings    # Enable embedding generation (slower, better search)
gitnexus analyze --skip-agents-md  # Preserve custom AGENTS.md/CLAUDE.md gitnexus section edits
gitnexus analyze --verbose       # Log skipped files when parsers are unavailable
gitnexus analyze --max-file-size 1024  # Skip files larger than N KB (default: 512, cap: 32768)
gitnexus analyze --worker-timeout 60  # Increase worker idle timeout for slow parses
gitnexus mcp                     # Start MCP server (stdio) — serves all indexed repos
gitnexus serve                   # Start local HTTP server (multi-repo) for web UI
gitnexus index                   # Register an existing .gitnexus/ folder into the global registry
gitnexus list                    # List all indexed repositories
gitnexus status                  # Show index status for current repo
gitnexus clean                   # Delete index for current repo
gitnexus clean --all --force     # Delete all indexes
gitnexus wiki [path]             # Generate LLM-powered docs from knowledge graph
gitnexus wiki --model <model>    # Wiki with custom LLM model (default: gpt-4o-mini)

# Repository groups (multi-repo / monorepo service tracking)
gitnexus group create <name>                                   # Create a repository group
gitnexus group add <group> <groupPath> <registryName>          # Add a repo to a group. <groupPath> is a hierarchy path (e.g. hr/hiring/backend); <registryName> is the repo's name from the registry (see `gitnexus list`)
gitnexus group remove <group> <groupPath>                      # Remove a repo from a group by its hierarchy path
gitnexus group list [name]                                     # List groups, or show one group's config
gitnexus group sync <name>                                     # Extract contracts and match across repos/services
gitnexus group contracts <name>  # Inspect extracted contracts and cross-links
gitnexus group query <name> <q>  # Search execution flows across all repos in a group
gitnexus group status <name>     # Check staleness of repos in a group

Remote Embeddings

Set these env vars to use a remote OpenAI-compatible /v1/embeddings endpoint instead of the local model:

export GITNEXUS_EMBEDDING_URL=http://your-server:8080/v1
export GITNEXUS_EMBEDDING_MODEL=BAAI/bge-large-en-v1.5
export GITNEXUS_EMBEDDING_DIMS=1024          # optional, default 384
export GITNEXUS_EMBEDDING_API_KEY=your-key   # optional, default: "unused"
gitnexus analyze . --embeddings

Works with Infinity, vLLM, TEI, llama.cpp, Ollama, LM Studio, or OpenAI. When unset, local embeddings are used unchanged.

Multi-Repo Support

GitNexus supports indexing multiple repositories. Each gitnexus analyze registers the repo in a global registry (~/.gitnexus/registry.json). The MCP server serves all indexed repos automatically.

Supported Languages

TypeScript, JavaScript, Python, Java, C, C++, C#, Go, Rust, PHP, Kotlin, Swift, Ruby

Language Feature Matrix

Language Imports Named Bindings Exports Heritage Type Annotations Constructor Inference Config Frameworks Entry Points
TypeScript ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓
JavaScript ✓ ✓ ✓ ✓ — ✓ ✓ ✓ ✓
Python ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓
Java ✓ ✓ ✓ ✓ ✓ ✓ — ✓ ✓
Kotlin ✓ ✓ ✓ ✓ ✓ ✓ — ✓ ✓
C# ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓
Go ✓ — ✓ ✓ ✓ ✓ ✓ ✓ ✓
Rust ✓ ✓ ✓ ✓ ✓ ✓ — ✓ ✓
PHP ✓ ✓ ✓ — ✓ ✓ ✓ ✓ ✓
Ruby ✓ — ✓ ✓ — ✓ — ✓ ✓
Swift — — ✓ ✓ ✓ ✓ ✓ ✓ ✓
C — — ✓ — ✓ ✓ — ✓ ✓
C++ — — ✓ ✓ ✓ ✓ — ✓ ✓

Imports — cross-file import resolution · Named Bindings — import { X as Y } / re-export tracking · Exports — public/exported symbol detection · Heritage — class inheritance, interfaces, mixins · Type Annotations — explicit type extraction for receiver resolution · Constructor Inference — infer receiver type from constructor calls (self/this resolution included for all languages) · Config — language toolchain config parsing (tsconfig, go.mod, etc.) · Frameworks — AST-based framework pattern detection · Entry Points — entry point scoring heuristics

Agent Skills

GitNexus ships with skill files that teach AI agents how to use the tools effectively:

  • Exploring — Navigate unfamiliar code using the knowledge graph
  • Debugging — Trace bugs through call chains
  • Impact Analysis — Analyze blast radius before changes
  • Refactoring — Plan safe refactors using dependency mapping

Installed automatically by both gitnexus analyze (per-repo) and gitnexus setup (global).

Requirements

  • Node.js >= 18
  • Git repository (uses git for commit tracking)

Release candidates

Stable releases publish to the default latest dist-tag. When a pull request with non-documentation changes merges into main, an automated workflow also publishes a prerelease build under the rc dist-tag, so early adopters can try in-flight fixes without waiting for the next stable cut. (Docs-only merges are skipped.)

# Try the latest release candidate (pre-stable — may change at any time)
npm install -g gitnexus@rc
# — or —
npx gitnexus@rc analyze

Release-candidate versions follow the standard semver prerelease format X.Y.Z-rc.N, where X.Y.Z is the next stable target (bumped from the current latest by patch by default; minor or major when kicking off a bigger cycle) and N increments per published rc. Example sequence: 1.6.2-rc.1, 1.6.2-rc.2, …, then once 1.6.2 ships stable, 1.6.3-rc.1. See the Releases page for the full list; stable latest is unaffected.

Troubleshooting

Cannot destructure property 'package' of 'node.target' as it is null

This crash was caused by a dependency URL format that is incompatible with certain npm/arborist versions (npm/cli#8126). It is fixed in gitnexus v1.6.2+. Upgrade to the latest version:

npx gitnexus@latest analyze          # always uses the newest release
# — or —
npm install -g gitnexus@latest       # upgrade a global install

If you still hit npm install issues after upgrading, these generic workarounds may help:

npm install -g npm@latest            # update npm itself
npm cache clean --force              # clear a possibly corrupt cache

Installation fails with native module errors

Some optional language grammars (Dart, Kotlin, Swift) require native compilation. If they fail, GitNexus still works — those languages will be skipped.

If npm install -g gitnexus fails on native modules:

# Ensure build tools are available (Linux/macOS)
# Ubuntu/Debian: sudo apt install python3 make g++
# macOS: xcode-select --install

# Retry installation
npm install -g gitnexus

Analyze warns about unavailable FTS or VECTOR extensions

GitNexus uses optional DuckDB extensions for BM25 and vector search. The gitnexus serve and MCP read paths only ever try to LOAD the extensions — they never block on a network install. The analyze command, by default, attempts one bounded out-of-process INSTALL if LOAD fails and proceeds even when that install times out, so the index is always written to disk; BM25/vector search degrade gracefully until the extensions become available.

Configure the behavior with two environment variables:

Variable Values Default Effect
GITNEXUS_LBUG_EXTENSION_INSTALL auto, load-only, never auto auto runs one bounded INSTALL if LOAD fails. load-only only uses already-installed extensions (recommended for offline / firewalled environments). never skips optional extensions entirely.
GITNEXUS_LBUG_EXTENSION_INSTALL_TIMEOUT_MS positive integer 15000 Wall-clock budget for the out-of-process INSTALL child before it is killed. 
# Offline/airgapped: never reach the network for extensions
GITNEXUS_LBUG_EXTENSION_INSTALL=load-only npx gitnexus analyze

# Slow network: give extension downloads more time
GITNEXUS_LBUG_EXTENSION_INSTALL_TIMEOUT_MS=30000 npx gitnexus analyze

Analysis runs out of memory

For very large repositories:

# Increase Node.js heap size
NODE_OPTIONS="--max-old-space-size=16384" npx gitnexus analyze

# Exclude large directories
echo "vendor/" >> .gitnexusignore
echo "dist/" >> .gitnexusignore

Large files are being skipped

By default the walker skips files larger than 512 KB (see log line Skipped N large files (>512KB)). Raise the threshold via either the CLI flag or the environment variable — both accept a value in KB:

# CLI flag (takes precedence over the env var)
npx gitnexus analyze --max-file-size 2048     # skip only files > 2 MB

# Environment variable (persists across commands)
export GITNEXUS_MAX_FILE_SIZE=2048
npx gitnexus analyze

Values above 32768 KB (32 MB) are clamped to the tree-sitter parser ceiling; invalid values fall back to the 512 KB default with a one-time warning. When an override is active, analyze prints the effective threshold in its startup banner (e.g. GITNEXUS_MAX_FILE_SIZE: effective threshold 2048KB (default 512KB)).

Analyze reports a worker timeout

Worker parse timeouts are recoverable. GitNexus retries stalled worker jobs with backoff, splits large jobs to isolate slow files, and falls back to the sequential parser when needed. If a large repository needs more time per worker job, use either:

# CLI flag, in seconds
npx gitnexus analyze --worker-timeout 60

# Environment variable, in milliseconds
export GITNEXUS_WORKER_SUB_BATCH_TIMEOUT_MS=60000
npx gitnexus analyze

For repositories with very large source files, GITNEXUS_WORKER_SUB_BATCH_MAX_BYTES controls the worker job byte budget. The default is 8388608 bytes (8 MB).

Privacy

  • All processing happens locally on your machine
  • No code is sent to any server
  • Index stored in .gitnexus/ inside your repo (gitignored)
  • Global registry at ~/.gitnexus/ stores only paths and metadata

Web UI

GitNexus also has a browser-based UI at gitnexus.vercel.app — 100% client-side, your code never leaves the browser.

Local Backend Mode: Run gitnexus serve and open the web UI locally — it auto-detects the server and shows all your indexed repos, with full AI chat support. No need to re-upload or re-index. The agent's tools (Cypher queries, search, code navigation) route through the backend HTTP API automatically.

License

PolyForm Noncommercial 1.0.0

Free for non-commercial use. Contact for commercial licensing.