mirror of https://github.com/abhigyanpatwari/GitNexus.git synced 2026-08-28 05:25:25 +00:00

perf(hooks): cmdline-first Linux db-lock scan, drop the lsof fallback (#2180 ) (#2183 )

* perf(hooks): cmdline-first Linux db-lock scan, drop the lsof fallback (#2180)

The probe's Linux scan was O(processes × fds) — stat every fd of every
process — so on a busy host it blew its budget and fell through to lsof,
which then timed out (~2 s) and fail-closed. Every Grep/Glob/Bash hook
spent ~2 s of CPU to conclude 'couldn't tell'.

Rewrite linuxProcScanFindGitNexusServer (name kept; return type now
tri-state 'owned' | 'not-owned' | 'timeout') as three phases:
  0. /proc/<pid>/comm prefilter — kernel task->comm, never touches the
     target's memory maps; truncation-safe whitelist match (comm is
     capped at 15 visible chars). Calibrated to what a real server
     reports: @ladybugdb/core's worker_threads rename the main thread to
     'MainThread', so that is whitelisted alongside the launcher
     basenames — omitting it would blind the probe to every server.
  1. bounded /proc/<pid>/cmdline read (openSync+readSync, default 16 KiB
     with a floor of 4 KiB and a bounded escalation up to a hard ceiling)
     so a D-state holder cannot stall the hook and the mcp/serve mode
     token is never clipped off a long interpreter path.
  2. dev+ino fd match for the 0–2 survivors only.

Dispatch: 'owned' and 'timeout' both map to true. Timeout is now
fail-closed (overload self-throttle) instead of falling through to lsof;
the Linux lsof fallback is removed entirely. End-to-end semantics on
busy hosts are unchanged (the old lsof arm also fail-closed there) — the
~2 s of wasted work and the orphan-spawning lsof are what's gone.
macOS lsof+ps and Windows Restart Manager paths are untouched.

Also: fix the budget parse bug (Number(raw && trim()) treated '0' as
1200; now parseInt-then-validate, with <= 0 an explicit immediate
timeout) and add GITNEXUS_HOOK_PROC_ROOT so the Linux scan can be unit
tested against a fixture procfs instead of the host's real /proc.

Measured on a 583-process host with 6 background gitnexus mcp servers:
owner detection 6–12 ms (was ~1216 ms + lsof timeout), ~100x.

Tests: new hook-db-lock-probe.test.ts drives all three phases against a
fake procfs (comm-truncation safety, Phase 0 trap, 4 KiB-boundary
owner-miss guard, budget=0 immediate timeout, EACCES fail-closed) plus a
live-/proc e2e that pins the fd-visible lbug-handle property against a
real subprocess holder. The lsof/ps owner-detection suites are relaned
to macOS (Linux no longer takes that path); the lsof orphan-reaping
suite is removed (no lsof is spawned on Linux now) with a rationale note.

Note: pre-commit typecheck skipped; remaining tsc errors are pre-existing
on main (none in files touched here).

* fix(hooks): honest EACCES verdict + real escalation coverage (#2183 review)

Addresses the tri-review (maintainer + Codex):

- [P2] Phase-2 fd-dir EACCES no longer claims 'owned'. /proc/<pid>/fd is
  owner-only (0500), so a cross-user/root gitnexus server serving ANY
  repo cleared Phase 0+1 and hit EACCES here, and the old catch returned
  'owned' — falsely claiming it locks THIS repo's lbug (dev+ino never
  compared) and permanently suppressing augment. Split the failure
  shapes: ENOENT -> continue (raced away); EACCES/EPERM and transient
  EIO/ESTALE -> 'timeout' (unverifiable -> fail-closed, but honest, not a
  false ownership claim); ENOTDIR/other structural errors -> continue
  (not a real fd dir). Same fail-closed dispatcher outcome, no false
  'owned', plus a GITNEXUS_DEBUG diagnostic so an operator can tell this
  skip path from a real owner.
- [P2] The escalation test now actually iterates the escalation loop:
  the gitnexus token sits under 4 KB while the mode token is padded past
  GITNEXUS_HOOK_PROC_CMDLINE_MAX=4096, and a readSync spy asserts >1 read
  (the old 9 KB-under-16 KB-cap shape read once and never escalated).
- escalation loop now re-checks the budget each iteration and returns a
  distinct timeout sentinel (never '' — an empty string would read as
  'not a candidate' and could drop a real owner -> fail-open); the caller
  maps it to 'timeout'.
- GITNEXUS_HOOK_PROC_ROOT is gated to test context so a stray production
  env export can't disable Linux owner detection (fail-open).
- New uid-agnostic spy tests pin every fd-readdir errno branch
  (EACCES/EPERM/EIO/ESTALE -> timeout, ENOTDIR -> not-owned) regardless
  of the runner's uid (the disk chmod-000 tests no-op under root).

Note: pre-commit typecheck skipped; remaining tsc errors are pre-existing
on main (none in files touched here).

* fix(hooks): drop the always-true outOfBudget presence guard (CodeQL #2183)

CodeQL flagged `typeof outOfBudget === 'function' && outOfBudget()` as
unneeded defensive code: readLinuxCmdline has a single caller
(linuxProcScanFindGitNexusServer) that always passes the callback, so
the typeof guard is dead. Drop it, leaving `if (outOfBudget())`, and note
the invariant in the comment. Mirrored in the byte-identical plugin copy.

* fix(hooks): parse numeric hook env with Number() so scientific notation works (#2183 review)

getCmdlineMaxBytes and resolveLinuxProcBudgetMs parsed their env via
Number.parseInt(raw, 10), so a value like "16e3" silently became 16 (parseInt
stops at 'e') instead of 16000. Switch both to Number(String(raw).trim()),
which honors scientific notation and is stricter on trailing garbage
("123abc" -> NaN -> default) — matching the repo-majority Number()+isFinite
env idiom (src/cli/analyze.ts, src/core/embeddings/hf-env.ts).

The two functions had DIFFERENT guard skeletons, so a verbatim swap would
regress the budget: resolveLinuxProcBudgetMs used `raw != null ?` with no
empty-string short-circuit, and Number("")===0 (vs parseInt("")===NaN) would
make a set-but-empty GITNEXUS_HOOK_LINUX_PROC_BUDGET_MS="" resolve to budget 0
=> immediate fail-CLOSED timeout => augment permanently skipped. Added the
`&& String(raw).trim()` guard so ''/whitespace fall to the 1200 default while
"0" still parses to the deliberate #2180 immediate-timeout vector.

Exported both helpers for white-box tests (the values are otherwise only
observable indirectly through scan timing) and added platform-independent
coverage: "16e3"->16000, ""/whitespace->1200 (the regression guard), "0"->0,
"123abc"/unset->1200, cmdline "8e3"->8000, "2e3"/""/unset->16384.

Both byte-identical hook-db-lock-probe.cjs copies updated together.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(hooks): allocUnsafe the per-chunk cmdline read buffer (#2183 review)

readLinuxCmdline allocated each per-chunk read buffer with Buffer.alloc(chunkCap),
zero-filling memory that readSync immediately and fully overwrites. Switch the
hot read buffer to Buffer.allocUnsafe — safe because readSync initializes
exactly [0, bytes), only buf.subarray(0, bytes) is consumed, and Buffer.concat
deep-copies that slice into `collected`, so the uninitialized tail can never
reach the decoded cmdline. The zero-length `collected = Buffer.alloc(0)` is left
unchanged (allocUnsafe gains nothing on a 0-length buffer). The existing D3
multi-chunk decode tests cover the read path and stay green.

Both byte-identical hook-db-lock-probe.cjs copies updated together.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(hooks): harden the live /proc owner-detection e2e against CI flake (#2183 review)

Two flake mechanisms, fixed without weakening what the e2e proves:

- Holder readiness (the genuine false-FAIL): the pid-file poll was 200x25ms=5s;
  a loaded runner can be slow to spawn the child, tripping
  expect(holderPid).toBeGreaterThan(0). Widened to ~10s and raised the per-test
  timeout 20s -> 40s.
- Scan budget (kept the assertion honest): the live scan ran at the default
  1200ms. Because the dispatcher maps a budget 'timeout' to owned=TRUE, a busy
  host exhausting 1200ms before reaching the holder would make the assertion
  pass for the WRONG reason (a hollow timeout, not real fd-visible detection).
  Set a generous explicit 10000ms budget via the existing setEnv() helper so the
  module afterEach restores it (replacing the raw `delete process.env...` that
  bypassed env tracking). Raised the coarse timing regression guard to sit ABOVE
  the budget (5000 -> 15000) so a legitimately-slow-but-correct scan can't trip
  it.

The load-bearing asserts (dev+ino fd-visibility precheck, owned===true for our
own lbug) are unchanged. Verified the e2e executes (not skipped) on Linux.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore(changelog): empty the root CHANGELOG [Unreleased] section

Per maintainer request, nothing should sit under [Unreleased] in the root
CHANGELOG.md (the release-owned changelog is gitnexus/CHANGELOG.md, whose
[Unreleased] is already empty). Removes all three accumulated blocks — Fixed
(#2163), Performance (#2180), Changed (KuzuDB->LadybugDB) — leaving only the
[Unreleased] header above [1.5.3]. Pure removal; no release sections touched.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Gergő Magyar <gergomagyar@icloud.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

2026-06-13 11:52:14 +01:00

7.6 KiB

Raw Permalink Blame History

Changelog

All notable changes to GitNexus will be documented in this file.

[Unreleased]

[1.5.3] - 2026-04-01

Added

TypeScript/JavaScript MethodExtractor config — shared extraction config covering abstract methods, visibility modifiers, async/override keywords, decorators, rest/optional/destructured parameters, and return types (#588) — @compound-ai

Fixed

Azure OpenAI compatibility — use max_completion_tokens instead of deprecated max_tokens (newer models reject max_tokens); skip temperature for Azure provider (some models reject non-default values) (#618)
Simplified Azure interactive setup — 3 prompts (endpoint, deployment, key) instead of 7 (#618)
Wiki HTML viewer script injection — escape </script> in embedded JSON so LLM-generated markdown no longer breaks the viewer (#618)
Ensure import rewrites survive npm publish lifecycle

[1.4.0] - 2026-03-13

Added

Language-aware symbol resolution engine with 3-tier resolver: exact FQN → scope-walk → guarded fuzzy fallback that refuses ambiguous matches (#238) — @magyargergo
Method Resolution Order (MRO) with 5 language-specific strategies: C++ leftmost-base, C#/Java class-over-interface, Python C3 linearization, Rust qualified syntax, default BFS (#238) — @magyargergo
Constructor & struct literal resolution across all languages — new Foo(), User{...}, C# primary constructors, target-typed new (#238) — @magyargergo
Receiver-constrained resolution using per-file TypeEnv — disambiguates user.save() vs repo.save() via ownerId matching (#238) — @magyargergo
Heritage & ownership edges — HAS_METHOD, OVERRIDES, Go struct embedding, Swift extension heritage, method signatures (parameterCount, returnType) (#238) — @magyargergo
Language-specific resolver directory (resolvers/) — extracted JVM, Go, C#, PHP, Rust resolvers from monolithic import-processor (#238) — @magyargergo
Type extractor directory (type-extractors/) — per-language type binding extraction with Record<SupportedLanguages, Handler> + satisfies dispatch (#238) — @magyargergo
Export detection dispatch table — compile-time exhaustive Record + satisfies pattern replacing switch/if chains (#238) — @magyargergo
Language config module (language-config.ts) — centralized tsconfig, go.mod, composer.json, .csproj, Swift package config loaders (#238) — @magyargergo
Optional skill generation via npx gitnexus analyze --skills — generates AI agent skills from KuzuDB knowledge graph (#171) — @zander-raycraft
First-class C# support — sibling-based modifier scanning, record/delegate/property/field/event declaration types (#163, #170, #178 via #237) — @Alice523, @benny-yamagata, @jnMetaCode
C/C++ support fixes — .h → C++ mapping, static-linkage export detection, qualified/parenthesized declarators, 48 entry point patterns (#163, #227 via #237) — @Alice523, @bitgineer
Rust support fixes — sibling-based visibility_modifier scanning for pub detection (#227 via #237) — @bitgineer
Adaptive tree-sitter buffer sizing — Math.min(Math.max(contentLength * 2, 512KB), 32MB) (#216 via #237) — @JasonOA888
Call expression matching in tree-sitter queries (#234 via #237) — @ex-nihilo-jg
DeepSeek model configurations (#217) — @JasonOA888
282+ new unit tests, 178 integration resolver tests across 9 languages, 53 test files, 1146 total tests passing

Fixed

Skip unavailable native Swift parsers in sequential ingestion (#188) — @Gujiassh
Heritage heuristic language-gated — no longer applies class/interface rules to wrong languages (#238) — @magyargergo
C# base_list distinguishes EXTENDS vs IMPLEMENTS via symbol table + I[A-Z] heuristic (#238) — @magyargergo
Go qualified_type (models.User) correctly unwrapped in TypeEnv (#238) — @magyargergo
Global tier no longer blocks resolution when kind/arity filtering can narrow to 1 candidate (#238) — @magyargergo

Changed

import-processor.ts reduced from 1412 → 711 lines (50% reduction) via resolver and config extraction (#238) — @magyargergo
type-env.ts reduced from 635 → ~125 lines via type-extractor extraction (#238) — @magyargergo
CI/CD workflows hardened with security fixes and fork PR support (#222, #225) — @magyargergo

[1.3.11] - 2026-03-08

Security

Fix FTS Cypher injection by escaping backslashes in search queries (#209) — @magyargergo

Added

Auto-reindex hook that runs gitnexus analyze after commits and merges, with automatic embeddings preservation (#205) — @L1nusB
968 integration tests (up from ~840) covering unhappy paths across search, enrichment, CLI, pipeline, worker pool, and KuzuDB (#209) — @magyargergo
Coverage auto-ratcheting so thresholds bump automatically on CI (#209) — @magyargergo
Rich CI PR report with coverage bars, test counts, and threshold tracking (#209) — @magyargergo
Modular CI workflow architecture with separate unit-test, integration-test, and orchestrator jobs (#209) — @magyargergo

Fixed

KuzuDB native addon crashes on Linux/macOS by running integration tests in isolated vitest processes with --pool=forks (#209) — @magyargergo
Worker pool MODULE_NOT_FOUND crash when script path is invalid (#209) — @magyargergo

Changed

Added macOS to the cross-platform CI test matrix (#208) — @magyargergo

[1.3.10] - 2026-03-07

Security

MCP transport buffer cap: Added 10 MB MAX_BUFFER_SIZE limit to prevent out-of-memory attacks via oversized Content-Length headers or unbounded newline-delimited input
Content-Length validation: Reject Content-Length values exceeding the buffer cap before allocating memory
Stack overflow prevention: Replaced recursive readNewlineMessage with iterative loop to prevent stack overflow from consecutive empty lines
Ambiguous prefix hardening: Tightened looksLikeContentLength to require 14+ bytes before matching, preventing false framing detection on short input
Closed transport guard: send() now rejects with a clear error when called after close(), with proper write-error propagation

Added

Dual-framing MCP transport (CompatibleStdioServerTransport): Auto-detects Content-Length (Codex/OpenCode) and newline-delimited JSON (Cursor/Claude Code) framing on the first message, responds in the same format (#207)
Lazy CLI module loading: All CLI subcommands now use createLazyAction() to defer heavy imports (tree-sitter, ONNX, KuzuDB) until invocation, significantly improving gitnexus mcp startup time (#207)
Type-safe lazy actions: createLazyAction uses constrained generics to validate export names against module types at compile time
Regression test suite: 13 unit tests covering transport framing, security hardening, buffer limits, and lazy action loading

Fixed

CALLS edge sourceId alignment: findEnclosingFunctionId now generates IDs with :startLine suffix matching node creation format, fixing process detector finding 0 entry points (#194)
LRU cache zero maxSize crash: Guard createASTCache against maxSize=0 when repos have no parseable files (#144)

Changed

Transport constructor accepts NodeJS.ReadableStream / NodeJS.WritableStream (widened from concrete ReadStream/WriteStream)
processReadBuffer simplified to break on first error instead of stale-buffer retry loop

[1.3.9] - 2026-03-06

Fixed

Aligned CALLS edge sourceId with node ID format in parse worker (#194)

[1.3.8] - 2026-03-05

Fixed

Force-exit after analyze to prevent KuzuDB native cleanup hang (#192)

7.6 KiB Raw Permalink Blame History

Changelog

[Unreleased]

[1.5.3] - 2026-04-01

Added

Fixed

[1.4.0] - 2026-03-13

Added

Fixed

Changed

[1.3.11] - 2026-03-08

Security

Added

Fixed

Changed

[1.3.10] - 2026-03-07

Security

Added

Fixed

Changed

[1.3.9] - 2026-03-06

Fixed

[1.3.8] - 2026-03-05

Fixed

7.6 KiB

Raw Permalink Blame History