mirror of
https://github.com/abhigyanpatwari/GitNexus.git
synced 2026-09-21 00:21:30 +00:00
36 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Gergő Magyar
|
d3a7ce95a5
|
feat(core): adopt pino structured logger (#1336)
* feat(core): adopt pino structured logger + add no-console eslint forcing function
Adds `pino` as the project-wide structured logger via a thin wrapper at
`gitnexus/src/core/logger.ts` exposing `createLogger(name, opts?)` and a
default `logger` singleton. Migrates the only security-relevant `console.warn`
site (`bridge-db.ts` `openBridgeDbReadOnly` retry-exhaustion path) to
`bridgeLogger.debug({groupDir, err, attempts}, 'msg')`.
Pino's NDJSON output is structurally log-injection-resistant (one record per
newline, all string fields JSON-escaped) — replaces the hand-rolled
`sanitizeLogValue` pattern that PR #1329 added on the `fix/insecure-tempfile-core`
branch. PR #1329's sanitizer remains as fallback until CodeQL confirms #466
closes via pino on this branch.
Also adds an ESLint `no-console: warn` rule scoped to
`gitnexus/src/**/*.ts` (excluding `cli/`, `server/`, `test/`, `bin/`, and the
logger module itself) as the forcing function — new code can't regress.
Existing 134 sites in `core/`, `mcp/`, `config/`, `storage/` get a
`// eslint-disable-next-line no-console -- TODO(pino-migration)` marker in a
follow-up commit so lint stays clean and the remaining work is grep-able.
Operator behaviour preserved:
- `GITNEXUS_DEBUG_BRIDGE` truthy → bridgeLogger logs at debug level
- `GITNEXUS_DEBUG_BRIDGE` unset → bridgeLogger filters debug messages
- Output is NDJSON in production / CI / vitest
- pino-pretty engages only when stdout is a TTY AND CI/VITEST env unset
Tests: 11 new logger.test.ts cases (level methods, debugEnvVar gating,
destination capture, undefined Error.message safety, CR/LF/U+2028/ANSI
single-record invariant). Group test suite (388 tests) passes unchanged.
`--no-verify`: pre-commit hook fails on PR #1302's pre-existing TS regression
at `scope-resolution/pipeline/run.ts:160` on main; documented in commit
`348d0c91` and recurring across the security-fix series.
Refs: #466 (codeql js/log-injection), PR #1329 follow-up.
* chore(lint): baseline-suppress 134 existing console.* sites with TODO(pino-migration)
Mechanical pass: prepends `// eslint-disable-next-line no-console -- TODO(pino-migration)`
above each existing `console.*` call in `gitnexus/src/{config,core,mcp,storage}/`
that the new ESLint rule would otherwise flag. CLI/server are exempt at the
config level (legitimate stdout output).
Zero functional changes. Generated by an in-repo node script that consumes
`eslint --format json` output and prepends the marker line at each reported
location. Verification:
npx eslint gitnexus/src/ → 0 no-console warnings
grep -rn "TODO(pino-migration)" gitnexus/src/ | wc -l → 134
The marker tags inventory the remaining migration surface so future sweep
PRs can grep their target list. When a follow-up PR migrates a site, the
marker comment is removed alongside the `console.*` → `logger.*` swap.
`--no-verify`: same as parent commit (PR #1302 pre-existing TS regression on main).
* refactor(core): complete pino migration — replace all 134 console.* sites + flip ESLint to error
Codebase-wide sweep of every `TODO(pino-migration)` site flagged in commit
|
||
|
azizur100389
|
b486d04d75
|
refactor(lbug): extract safeClose helper to consolidate WAL flush (#1377) | ||
|
azizur100389
|
a418c47e29
|
fix(server): use ipKeyGenerator for IPv6 subnet normalisation (#1360) (#1374)
The custom keyGenerator in createRouteLimiter referenced req.ip without passing it through express-rate-limit's ipKeyGenerator helper. This caused ERR_ERL_KEY_GEN_IPV6 on startup when binding to 0.0.0.0, and meant each full IPv6 address got its own rate-limit counter — trivially bypassing the per-IP limit. Wrap the IP through ipKeyGenerator so IPv6 addresses are collapsed to their /56 subnet before keying the counter. The existing fallback chain (req.ip → socket.remoteAddress → 'unknown') is preserved to keep ERR_ERL_UNDEFINED_IP_ADDRESS from firing on abruptly closed connections. Tests: 3 new assertions (construction-time regression guard, source-grep for import and call site). |
||
|
azizur100389
|
46e4c979c4
|
fix(server): flush WAL after /api/embed so search sees new embeddings (#1149) (#1359) | ||
|
azizur100389
|
8fc32e6af9
|
fix(docker): add dedicated health endpoint for container healthcheck (#1147) (#1355)
Some checks are pending
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Release Candidate / Check if release candidate should run (push) Waiting to run
Release Candidate / ci (push) Blocked by required conditions
Release Candidate / Publish release candidate to npm (push) Blocked by required conditions
Release Candidate / Build & Push RC Docker images (push) Blocked by required conditions
Scorecard / Scorecard analysis (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
|
||
|
azizur100389
|
f10135649e
|
fix(server): rate-limit /api/analyze and /api/embed endpoints (#1328) (#1339)
Some checks are pending
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Release Candidate / Check if release candidate should run (push) Waiting to run
Release Candidate / ci (push) Blocked by required conditions
Release Candidate / Publish release candidate to npm (push) Blocked by required conditions
Release Candidate / Build & Push RC Docker images (push) Blocked by required conditions
Scorecard / Scorecard analysis (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
|
||
|
Gergő Magyar
|
0add072f25
|
fix(server): add per-route rate limiting on FS-touching endpoints (U4) (#1327)
* fix(server): add per-route rate limiting on FS-touching endpoints (U4) U4 of the security remediation plan. Closes the four CodeQL js/missing-rate-limiting high alerts on FS-touching routes: #180 app.get(SPA_FALLBACK_REGEX, ...) (api.ts:225) #181 app.delete('/api/repo', ...) (api.ts:845) #444 app.get('/api/file', ...) (api.ts:1158) #183 app.get('/api/grep', ...) (api.ts:1169) The threat model: file-handle / disk-I/O exhaustion from a single attacker repeating requests. The local-bound HTTP server has a small surface (localhost by default; CORS allowlist for private-network reverse-proxy deployments), so a per-IP limiter sized for interactive web-UI use is the right shape — not global throttling, not hand-rolled, not Redis-backed. Architectural choices (cite DoD as I go): - Library: express-rate-limit ^8.4.1 — canonical, ~30KB, no native deps, memory store. (DoD §2.5: third-party dep justified, reputable, no supply-chain regression — found 0 vulnerabilities on install.) - Per-route limiters (independent counters): /api/file traffic does not push /api/grep into 429. Each route gets its own createRouteLimiter() instance. - Uniform default (60 rpm/IP): single tier across all 4 routes. Tiered per-route limits are over-engineering until traffic patterns demand it. (DoD §2.3: smallest correct solution.) - trust proxy = 'loopback, linklocal, uniquelocal': honors X-Forwarded-For only from local/private origins, exactly aligned with the CORS allowlist. Without this, every request through a Docker bridge or reverse proxy would count as a single req.ip and one user would trip the per-IP limiter for everyone (residual review F5 on the U2 plan, now fixed at the source rather than deferred). - No env-var override (e.g. GITNEXUS_RATE_LIMIT_RPM) in this PR. Per scope-guardian residual review F7: env vars are feature scope, not security remediation. Add tunability if and when operators ask. (DoD §2.3 + §6 not-done: avoid scope creep.) - New helper createRouteLimiter(opts?) in validation.ts wraps rateLimit with project-uniform defaults (status, headers, message). Justified by DRY across 4 callers and one place to tune later — not speculative abstraction. (DoD §2.3.) - 429 response body matches the project's { error: '...' } JSON shape so the web UI's error display stays uniform; draft-7 RateLimit-* headers (no legacy X-RateLimit-*) so callers can read the limit and back off. Tests (6 new in test/unit/rate-limit.test.ts; 136 total server-area): - createRouteLimiter exports DEFAULT_RATE_LIMIT_RPM = 60 - Returns a different middleware instance per call (independent counters) - Produces a callable express RequestHandler (3-arg signature) - Integration: 3 requests through, 4th returns 429 with { error } body (the exact regression guard CodeQL would re-fire if the limiter were dropped from any production route) - draft-7 RateLimit response header emitted, no legacy X-RateLimit-* - 429 body matches { error: '...' } shape The integration test mounts a route that does fs.readFile (the same FS sink CodeQL flags) behind createRouteLimiter on a tiny isolated express app. Tests use { windowMs: 1000, max: 3 } to keep them fast and deterministic. Pre-commit bypassed (--no-verify) — same pre-existing TS regression on main from PR #1302; this PR does not touch the affected file. * fix(server): address U4 code-review findings — best-judgment fix pass Code review on PR #1327 surfaced a cluster of P1/P2 findings the multi- agent pipeline corroborated across reviewers (correctness, security, adversarial, testing, maintainability, project-standards, api-contract, reliability, performance, kieran-typescript). This commit applies the high-confidence fixes that improve quality without expanding scope. Scope-decision items (cloud-LB trust-proxy override, /api/analyze and /api/embed rate limiting, --no-verify Go-provider TS regression) are deferred and surfaced in the PR body's residual section. validation.ts (createRouteLimiter): - Renamed `max` to canonical `limit` (express-rate-limit v8+; `max` is the deprecated alias that now logs a deprecation notice). - Replaced `Partial<RateLimitOptions>` with a narrow RouteLimiterOverrides type exposing only { windowMs?, limit? }. Closes the security regression vector where a caller could pass `{ skip: () => true }` and silently disable limiting on a route. - Added passOnStoreError: true so a memory-store failure lets the request through rather than producing an HTML 500 from Express's default error handler (the limiter middleware fires before the route's try/catch). - Added a custom keyGenerator with req.socket?.remoteAddress fallback so abruptly closed connections do not trigger ERR_ERL_UNDEFINED_IP_ADDRESS (which would 500 the request via Express's default error handler). - Widened return type from RequestHandler to RateLimitRequestHandler so callers can access .resetKey() if needed. - Unexported DEFAULT_RATE_LIMIT_RPM (consumed only internally; the test now asserts the observable behavior — 60 requests pass under default policy — instead of pinning the constant value). api.ts: - Expanded the trust-proxy comment with a SCOPE note (process-wide effect on every middleware/route) and a CLOUD-DEPLOY CAVEAT explicitly naming AWS ALB / Cloudflare / Fly.io edge / CGNAT as topologies that need an env-var override before production deployment. Tracked as follow-up. - Raised SPA fallback limit from 60 rpm/IP to 300 rpm/IP (5 req/s sustained). The original 60 was tight enough that multi-tab browser navigation, prefetch, and service-worker revalidation could legitimately trip it; the SPA fallback only does sendFile of a constant-path index.html, so the heavier limit is fine. JSON-on-429 to HTML clients is now a much rarer code path in practice; full content-negotiation on the 429 itself is tracked as follow-up. - Dropped CodeQL alert-ID numbers (#180/#181/#183/#444) from per-route comments — those IDs rotate per scan and would rot. The rule name (js/missing-rate-limiting) is the stable anchor. gitnexus-web backend-client.ts (web-client 429 handling): - Added 'rate_limited' to BackendError.code union; populated for 429 responses. - Added retryAfterMs?: number to BackendError, parsed from the Retry-After header on 429 responses (accepts both integer-seconds and HTTP-date forms; unparseable yields undefined). - assertOk now classifies 429 as 'rate_limited' (not generic 'client') so callers can pattern-match on it. test/unit/rate-limit.test.ts — major restructure: - Each integration test now uses a fresh server + fresh limiter instance via beforeEach/afterEach. Counter state never carries between tests, eliminating the inter-test ordering dependency. - Tightened windowMs from 1000 to 100 in tests; window-rollover test now waits 200ms (2x margin) for the window to expire — eliminates the 1100ms-margin flake under slow CI. - Added "window resets after windowMs" test (proves counter rollover works, replacing the timing-fragile prior shape). - Added "Retry-After header" test (proves the 429 surfaces the spec header so clients can back off — was a coverage gap flagged by api-contract reviewer). - Strengthened the draft-7 header assertion from toBeTruthy to toMatch on the `limit=N, remaining=N, reset=N` format so a future switch to draft-8 won't pass silently. - Replaced the constant-pin assertion (DEFAULT_RATE_LIMIT_RPM = 60) with a behavioral pin: 60 requests pass under the default policy. This pins the contract, not the magic number. - New "production routes — rate-limit middleware wiring" describe block: structural assertions that grep the api.ts source for createRouteLimiter adjacent to each of the 4 protected routes plus the trust-proxy setting. Closes the gap reviewers flagged where a maintainer could drop the limiter from a route and no test would fail. Tests: 143/143 pass server-area (was 136 before this commit; +7 in rate-limit.test.ts, including the production-wiring assertions). Pre-commit bypassed (--no-verify) — same pre-existing TS regression on main from PR #1302; this PR does not touch the affected file. * docs(server): fix misleading SPA-fallback comment + Retry-After test claim PR #1327 production-readiness review surfaced two comment-correctness findings (medium + low). Both are doc-only, no behavioral change. api.ts SPA fallback comment (medium): The previous comment claimed "On 429 we content-negotiate: if the client accepts HTML (browser navigation), serve the SPA shell" — but no content-negotiation is implemented; createRouteLimiter sends a fixed JSON body via the `message` option. The follow-up note below correctly stated content-negotiation was deferred, creating a direct internal contradiction and risking a future maintainer believing the behavior was implemented. Rewrote as a single coherent block: notes that 300 rpm/IP is high enough that browser navigation rarely trips it (the cosmetic JSON-on- 429 path is low-likelihood), and that proper content negotiation is deferred and would require swapping `message` for a `handler` function. No claim of unimplemented behavior remains. rate-limit.test.ts Retry-After comment (low): The previous comment said "Either an integer-seconds form or an HTTP-date — both are spec-valid", but the assertion (`Number.isFinite (Number(retryAfter))`) only accepts integer-seconds: an HTTP-date string would parse as NaN and fail. express-rate-limit v8 emits integer-seconds, so the test passes correctly today, but the comment overstates what's actually validated. Updated comment to say ERL v8 emits integer-seconds and to flag that a future ERL switch to HTTP-date would require an additional branch. Assertion unchanged. 13/13 rate-limit tests still pass; 143/143 server-area unchanged. |
||
|
Gergő Magyar
|
0844973f52
|
fix(server): harden git-clone — close 6 path-injection / CLI-injection / ReDoS alerts (U3) (#1325)
* fix(server): close 6 git-clone path-injection / CLI-injection / ReDoS alerts (U3) U3 of the security remediation plan. Closes the six high-severity CodeQL alerts in gitnexus/src/server/git-clone.ts: #185 js/polynomial-redos (line 16) #176 js/path-injection (line 209) #177 js/path-injection (line 219) #178 js/path-injection (line 230) #166 js/second-order-command-line-injection (line 221) #167 js/second-order-command-line-injection (line 221) Approach (DoD-aligned: smallest correct fix; barriers inline at sinks): extractRepoName — js/polynomial-redos (#185) The previous `url.replace(/\/+$/, '')` regex was flagged for polynomial backtracking on inputs with many trailing slashes. Replaced with an O(n) charCode loop. Also tightened the function's contract: it now throws when the last segment isn't a filesystem-safe name (^[a-zA-Z0-9._-]+$, with `.` and `..` explicitly rejected). This prevents a malicious URL like `https://github.com/owner/repo:..` from yielding a `repoName` that `getCloneDir(repoName)` would resolve outside ~/.gitnexus/repos/. getCloneDir — defense in depth Re-validates repoName against the same safe pattern at the boundary, so callers that don't go through extractRepoName (test helpers, future scripts) still can't construct an escape. cloneOrPull — js/path-injection (#176/#177/#178) Added a containment barrier at function entry using the canonical path.relative idiom CodeQL recognizes: const safeTarget = path.resolve(targetDir); const rel = path.relative(CLONE_ROOT, safeTarget); if (rel === '' || rel.startsWith('..') || path.isAbsolute(rel)) throw Every downstream filesystem operation uses safeTarget, with no reassignment between barrier and sink. Same idiom as PR #1322's U2. cloneOrPull — js/second-order-command-line-injection (#166/#167) Added the `--` separator to the git clone arg list: runGit(['clone', '--depth', '1', '--', url, safeTarget]) Without it, a URL beginning with `--` (e.g. `--upload-pack=evil ...`) would be parsed by git as an option flag rather than the clone source, enabling arbitrary subprocess execution. Per residual review F2 (ce-doc-review): intentionally did NOT add a host allowlist (`GITNEXUS_ALLOWED_HOSTS=github.com,...`). The existing SSRF protection in validateGitUrl (BLOCKED_HOSTNAMES + private-IP checks) plus the new safe-name and `--` separator address all 6 CodeQL alerts without breaking the CLI's `gitnexus analyze <url>` flow for gitlab/bitbucket/self-hosted users. A host allowlist would be feature work, not security remediation. Tests: - 5 new tests in git-clone.test.ts covering: `..` traversal rejection, `.` rejection, shell-metachar rejection, empty-input rejection, `getCloneDir('..')` / `getCloneDir('foo/bar')` rejection, and a sanity check that 10k trailing slashes resolve in <100ms (the polynomial-ReDoS regression guard). - 82/82 server-area tests pass (was 77). - Existing extractRepoName cases for github/gitlab URLs and SSH form continue to pass — the safe-name pattern accepts them all. Pre-commit bypassed (--no-verify) — same pre-existing TS regression on main from PR #1302; this PR does not touch the affected file. * fix(server): address PR #1325 review — close test gaps + fix delete regression PR #1325 review identified one HIGH and one MEDIUM blocker on the U3 git-clone hardening work. Both addressed below, plus two LOW hygiene items fixed while in the file. [HIGH] cloneOrPull had zero test coverage on the security-critical paths (DoD §2.7 violation: a regression in the path.relative containment barrier or the `--` separator in clone args would not have caused any test to fail). - Extracted buildCloneArgs(url, targetDir) so the `--` separator placement can be unit-tested without mocking child_process.spawn. cloneOrPull now calls runGit(buildCloneArgs(url, safeTarget)). - Added 7 new tests in git-clone.test.ts covering: * buildCloneArgs places `--` before the URL * buildCloneArgs treats `--upload-pack=evil` as a positional argument, not a flag (the exact second-order-CLI-injection mitigation) * buildCloneArgs preserves --depth 1 before the `--` separator * cloneOrPull rejects an absolute target outside CLONE_ROOT * cloneOrPull rejects CLONE_ROOT itself (the rel === '' branch) * cloneOrPull rejects parent-directory traversal * cloneOrPull rejects a sibling directory with a common prefix (CLONE_ROOT-evil) — documents that the path.relative idiom catches what startsWith(root + sep) would have missed. - These tests do not mock spawn — the barrier throws synchronously before git is invoked, so rejections are observable directly. [MEDIUM] Functional regression in api.ts:864 DELETE /api/repo flow. The new strict getCloneDir validation throws for any name outside [a-zA-Z0-9._-], which broke deletion of locally-registered repos with names like 'my project' or 'org/repo' — they returned 500 instead of completing the delete. - Wrapped the getCloneDir(entry.name) call in try/catch since clone-dir cleanup is advisory: local repos legitimately have no clone dir, and the existing inner try/catch already handled the missing-dir case. The throw is caught and treated as 'nothing to clean up'. [LOW] Hygiene fixes flagged by the same review: - git-clone.test.ts:75 — replaced em dash (U+2014) in error message with standard ASCII; switched the manual if/throw to expect().toBeLessThan() so the timing check uses vitest's normal assertion path. - Added a comment at the cloneOrPull barrier documenting that lexical containment is the CodeQL-recognized form and that symlink escape requires pre-existing local write access (out of scope for U3 threat model; tracked for follow-up). Test results: 115/115 server-area tests pass (was 82 before this commit, +33 from earlier in this PR + 7 new in this commit). buildCloneArgs and cloneOrPull boundary failures all surface in vitest now. Pre-commit bypassed (--no-verify) — same pre-existing TS regression on main from PR #1302; this PR does not touch the affected file. * fix(server): close SSRF-bypass + wrong-repo-pull on cloneOrPull (Codex review) Codex's adversarial review on PR #1325 surfaced one HIGH: cloneOrPull's existing-clone branch ran git pull --ff-only with neither validateGitUrl nor a remote-origin match check. Combined with the API's basename-derived target dir (api.ts:1359), this opened two real-world failure modes: 1. SSRF / scheme bypass: cloneOrPull('http://127.0.0.1/myproject.git', existingDir) → pulls the existing remote without ever validating the URL. validateGitUrl only fired on the new-clone branch. 2. Wrong-repo silent analysis: Existing clone → ~/.gitnexus/repos/myproject (origin = github.com/legitorg/myproject) Request URL → gitlab.example/attacker/myproject (same basename) cloneOrPull saw the existing .git/, ran git pull --ff-only against legitorg's remote, and returned an analysis labelled with the attacker's URL. DoD §2.1 (correctness) and §2.5 (security) violations. Fixed by: 1. validateGitUrl(url) is now called unconditionally at the top of cloneOrPull, after the path-containment barrier and before the existence probe. The pull branch can no longer be reached with a URL that hasn't passed SSRF/scheme/private-IP checks. 2. Added assertRemoteMatchesRequestedUrl(targetDir, url): reads the existing clone's remote.origin.url via `git config --get` and compares it (normalized) to the requested URL. Throws on mismatch or missing remote. Called in the existing-clone branch before `git pull`. 3. Added normalizeGitUrlForCompare(url): strips trailing .git and slashes, lowercases hostname, strips default ports and userinfo, so equivalent URL forms compare equal (with/without .git, with/ without trailing slash, https://github.com:443/x vs https://github.com/x). Path comparison stays case-sensitive — Git hosts treat path as case-sensitive on the wire. 4. Added getRemoteOriginUrl(cwd): one-shot spawn that captures the remote URL or returns null (missing remote / not a git repo / spawn error). Caller decides what null means; for cloneOrPull, null on an existing .git/ is a refuse-to-pull condition. Architectural choice: did NOT take Codex's broader "rekey clone dirs by URL hash" recommendation. That changes the persisted naming scheme and affects every existing user's clones (DoD §2.4 contract change, §2.9 reversibility risk). The verify-before-pull approach closes the same vulnerability surface with strictly smaller blast radius (DoD §2.3 smallest correct solution). Tests (15 new, 59 total in git-clone.test.ts; 130/130 across server-area): - cloneOrPull rejects URLs that fail validateGitUrl even when the target shape is valid (the SSRF-bypass closure) - normalizeGitUrlForCompare: 7 tests covering .git stripping, trailing slashes, hostname case, default ports, userinfo, host/path distinction - assertRemoteMatchesRequestedUrl: 5 tests using a tmpdir + git init fixture (anywhere on disk — independent of CLONE_ROOT, no user-state pollution): accepts matching URL, accepts equivalent forms, rejects different host with same basename (the exact wrong-repo vector), rejects different owner, rejects when no remote.origin - getRemoteOriginUrl returns null for non-git directories Pre-commit bypassed (--no-verify) — same pre-existing TS regression on main from PR #1302; this PR does not touch the affected file. |
||
|
Gergő Magyar
|
95aa10630e
|
fix(server): close js/path-injection cluster — /api/file + docker-server.mjs (U2) (#1322)
* fix(server): close path-injection cluster — sanitizer inline at sink (U2) U2 of the security remediation plan. Closes the four path-injection high alerts in /api/file (#179) and docker-server.mjs (#173/#174/#175 plus their post-refactor renumbers). Architectural approach: every filesystem sink is now immediately preceded by the canonical CodeQL-recognized sanitizer barrier: const rel = path.relative(root, candidate); if (rel.startsWith('..') || path.isAbsolute(rel)) reject; The barrier is inline at each sink — not behind a helper — because CodeQL's js/path-injection sanitizer recognition does not follow user-defined helpers across the request handler in vanilla JS. Earlier iterations of this work used assertSafePath / resolveWithinRoot helpers and a `startsWith(root + sep)` check; both were semantically correct but neither was recognized as a barrier by the analyzer. api.ts /api/file: - assertString on req.query.path (closes the type-confusion side-channel that lets `?path=a&path=b` slip past length-based guards). - Inline path.resolve + path.relative + isAbsolute + startsWith('..') check immediately before fs.readFile. docker-server.mjs: - Removed the resolvePath helper. The handler is now a single inline pipeline: decode → null-byte guard → resolve → barrier #1 → stat → pick finalPath → barrier #2 → stat + readStream. - Each barrier guards every following sink up to the next reassignment, so the analyzer can prove containment without crossing helper boundaries. - Switched all path construction from `join` to `path.resolve` for normalization (CodeQL does not treat `join` as normalizing). assertSafePath remains exported from validation.ts for non-CodeQL-sink callers; it just isn't used at this PR's sinks. Tests: 61/61 server-adjacent pass. Pre-commit bypassed (--no-verify) — pre-existing TS regression on main from PR #1302 (Go scope-resolution at scope-resolution/pipeline/run.ts:160) blocks every PR's pre-commit. Tracked separately; this PR does not touch that file. * fix(server): address PR #1322 review — wire /api/file catch + add route tests PR #1322 review (github-actions / Claude security review) identified two HIGH-severity blocking findings on the U2 path-injection cluster fix: 1. /api/file catch returned 500 for BadRequestError. assertString throws BadRequestError on array-form `?path=a&path=b`, but the catch block at api.ts:1108 only special-cased `err.code === 'ENOENT'` and otherwise returned hardcoded 500. The PR body claimed this was already fixed — it wasn't. Now uses statusFromError, which honors `err instanceof BadRequestError` per the U1 helper. 2. Zero route-level tests for /api/file. The U1 helper tests prove assertString and assertSafePath in isolation but cannot prove the route's error → status mapping, which is exactly where finding #1 lived. Changes: - api.ts /api/file catch: replaced hardcoded 500 with statusFromError(err). BadRequestError → 400 (array form), ForbiddenError → 403 (traversal), unrecognized → 500. ENOENT → 404 path is unchanged. - New gitnexus/test/unit/api-file-route.test.ts: 10 route-level tests that spin up a tiny isolated express app with the /api/file handler and exercise via real HTTP. Covers: - 200 for valid relative path + nested path - 400 for missing/empty path - 400 for ?path=a&path=b (the reproducer for finding #1) - 403 for parent-directory traversal - 403 for percent-encoded traversal (Express decodes before handler) - 403 for absolute escape - 404 for in-root non-existent path - 403 for common-prefix sibling escape (the path.relative idiom catches what startsWith(root + sep) would have missed) - docker-server.test.mjs: added two tests addressing the MEDIUM finding — encoded traversal (%2e%2e%2f) and malformed encoding (%GG). Both confirm the docker-server's inline barrier and the decodeURIComponent try/catch return 400 as expected. Test results: 71/71 pass in vitest (was 61, +10 new). Two pre-existing Windows-only failures in docker-server.test.mjs (asset cache check uses '/', tmpdir EBUSY cleanup race) are unchanged by this PR — confirmed by running the test suite against the merged base before applying this commit. Pre-commit bypassed (--no-verify) — same pre-existing TS regression on main from PR #1302; this PR does not touch the affected file. * refactor(server): extract handleFileRequest, test it directly without app.get CodeQL flagged gitnexus/test/unit/api-file-route.test.ts:81 with js/missing-rate-limiting High because the test mounted the /api/file handler on a real Express app via app.get(...) and bound a port. The query is correct for production route handlers; mounting in a test produces a false positive the analyzer cannot distinguish. The principled fix is structural, not a suppression: 1. Extracted the /api/file handler body into an exported handleFileRequest function in api.ts. The function takes (req, res, repoPath) and is a pure async function — no Express server, no route registration, no port. 2. The production /api/file route in createServer is now a thin caller that resolves the repo entry then delegates to handleFileRequest. 3. The test imports handleFileRequest and invokes it directly with a mock res object that captures status() and json() calls. No app.get, no listen, no port. Same coverage of the security wiring (10 tests covering valid path, missing path, array-form 400, traversal 403, encoded traversal 403, absolute escape 403, missing file 404, common-prefix sibling 403). Faster too — no port allocation per test. Production route behavior is unchanged. The diff is a true refactor: handler logic moved verbatim, just parameterized on repoPath rather than closure-captured from createServer's scope. 71/71 tests pass. This also cleanly separates the "is the route mounted with rate limiting" concern (production createServer wiring, addressed in plan unit U4) from the "does the handler do the right thing" concern (this test file). * style: prettier format api-file-route.test.ts |
||
|
Gergő Magyar
|
fa36254ed5
|
fix(server): close critical type-confusion + add validation helper module (#1317)
* fix(server): close js/type-confusion-through-parameter-tampering at /api/grep The /api/grep handler cast `req.query.pattern` to `string` and then guarded against `pattern.length > 200`. Express returns `string | string[] | ParsedQs` for query parameters; when a caller passes the same key twice (`?pattern=a&pattern=b`), the value arrives as an array and `.length` counts array elements, bypassing the length guard. The array is then coerced to a comma-joined string by `new RegExp(pattern, 'gim')`. Adds gitnexus/src/server/validation.ts with three helpers — assertString, assertSafePath, escapeRegExp — plus a typed BadRequestError/ForbiddenError pair. The helpers throw typed errors that the existing route try/catch blocks translate via statusFromError, which is extended to honor `err.status` for any BadRequestError instance before falling back to message-string matching. Wires assertString into /api/grep (api.ts:1118) and updates the route's catch to use statusFromError so validation rejections return 400 rather than 500. This is U1 of docs/plans/2026-05-04-001-fix-medium-to-critical-security-findings-plan.md — the foundational PR. Closes the single CodeQL critical alert and establishes the validation-helper pattern that U2-U7 reuse. Tests: 18 new unit tests in test/unit/server-validation.test.ts; 35/35 passing across the server-adjacent test files. Pre-commit hook bypassed via --no-verify due to a pre-existing TS regression on main introduced today by PR #1302 (Go scope-resolution) at gitnexus/src/core/ingestion/scope-resolution/pipeline/run.ts:160. That error is unrelated to this PR's changes (verified by re-running tsc against the unmodified base) and blocks every PR's pre-commit until fixed separately. * fix(server): close js/regex-injection at /api/grep — literal substring search by default Pivot /api/grep from "user-controlled regex" to "literal substring search by default, opt-in regex via ?regex=true". Closes the CodeQL js/regex-injection high-severity alert that PR-time CodeQL surfaced on this branch (and that the remediation plan tracks as U5). Audited callers before flipping the default: - gitnexus-web backend-client.grep() passes pattern raw, no flag → gets literal - gitnexus-web LLM tool description: "Search for exact text patterns... error messages, TODOs, variable names" — every documented use case is literal - No other callers in tree Pattern is now escaped via the validation.ts escapeRegExp helper before constructing the RegExp. The 200-char cap and try/catch on RegExp construction remain as defense-in-depth. Callers that genuinely need regex syntax (none exist today) opt in with ?regex=true or ?regex=1. This bundles plan unit U5 into the same PR as U1 because the helper landed here, the alert was surfaced by this PR's own CodeQL run, and the integration is one line at the route. The pre-existing escapeRegExp tests in test/unit/server-validation.test.ts already cover the literal-matching behavior; no new test file needed. 61/61 server-adjacent tests pass. * Potential fix for pull request finding 'CodeQL / Regular expression injection' Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> --------- Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> |
||
|
@aaronjmars
|
95814847bd
|
fix(security): block IPv4-compatible IPv6 and NAT64 SSRF bypasses in validateGitUrl (#1148)
* fix(security): block IPv4-compatible IPv6 and NAT64 SSRF bypasses Vulnerability: SSRF via IPv6 forms that embed IPv4 addresses Severity: high Location: gitnexus/src/server/git-clone.ts:assertNotPrivateIPv6 validateGitUrl() blocks ::ffff:x.x.x.x (IPv4-mapped) but two related forms still slipped through — both routable to the embedded IPv4 on common stacks: 1. IPv4-compatible IPv6 (RFC 4291 § 2.5.5.1, deprecated): http://[::127.0.0.1]/ — Node's URL parser collapses this to "::7f00:1" with no ::ffff: marker, so the existing check missed it. 2. NAT64 well-known prefix (RFC 6052: 64:ff9b::/96, plus RFC 8215's 64:ff9b:1::/48 local prefix): a host with NAT64 enabled translates 64:ff9b::7f00:1 to 127.0.0.1, reaching loopback. Impact: an attacker who can submit a clone URL to /api/analyze (any caller in the CORS-allowlisted origin set — localhost, RFC 1918 LAN, or gitnexus.vercel.app) could direct git clone at loopback or cloud metadata addresses (169.254.169.254 → ::a9fe:a9fe, 64:ff9b::a9fe:a9fe). Fix: extend assertNotPrivateIPv6 to reject any address compressed to ::xxxx[:yyyy] and any address starting with the NAT64 prefix 64:ff9b:. Tests added for both forms plus the cloud-metadata variants. * fix(security): block 6to4 SSRF bypass and add expanded-form regression tests Address review findings on PR #1148: - Block 6to4 (2002::/16, RFC 3056). The prefix encodes an IPv4 address in bits 17-48, so 2002:7f00:0001::* routes to 127.0.0.1 on 6to4-capable stacks. RFC 7526 deprecated the protocol and the public relay anycast has been retired, so broad-blocking has near-zero false-positive cost. - Expand the NAT64 comment to justify the broader-than-CIDR check: the whole 64:ff9b::/32 block is IANA-reserved for IPv4-IPv6 translation, so a future narrower CIDR refactor would silently re-open the bypass for 64:ff9b:1::/48 or any new translation range. - Add tests for expanded / zero-padded IPv4-compatible IPv6 forms ([0:0:0:0:0:0:7f00:1], fully zero-padded, mixed [0:...:127.0.0.1]). These pin the assumption that the WHATWG URL parser collapses these inputs to ::xxxx[:yyyy]; without them, a future Node anomaly would silently regress the bypass. - Add public IPv6 positive tests (Cloudflare 2606:4700::, Google 2001:4860::). Regression guard against over-blocking. - Add NAT64 + RFC1918 embedded-IP tests (10/8, 172.16/12, 192.168/16) to document SSRF coverage explicitly rather than relying on the prefix check. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * style: apply prettier formatting to git-clone.test.ts --------- Co-authored-by: aeonframework <aeon@aaronjmars.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: Gergo Magyar <gergomagyar@icloud.com> |
||
|
Tom Hale
|
94a4365e6d
|
fix(serve): serve web UI at root path instead of 404 (#1048)
* fix(serve): serve web UI at root path instead of 404 gitnexus serve returned Cannot GET / because no route handler existed for the root path. Now serves the built gitnexus-web dist at / with SPA fallback for client-side routing. Falls back to a helpful landing page with API links when the web UI hasn't been built yet. Also updates the build script to build and copy gitnexus-web into gitnexus/web/ for the published npm package. * fix(serve): address Copilot review feedback - Use regex SPA fallback that excludes /api paths (avoids serving index.html for unknown API routes) - Add rel="noopener noreferrer" to external link (reverse-tabnabbing) - Move build "done" log after web UI step * fix(build): use npm run build for web UI, add npm install guard The build script ran `npx tsc -b && npx vite build` in gitnexus-web/, but CI only installs node_modules for gitnexus/ — not gitnexus-web/. npx then resolved the wrong `tsc` package (a trojan on npm), causing all CI jobs to fail. Fix: add an npm install guard when node_modules is missing, and use `npm run build` (which runs the local typescript) instead of npx. * feat(serve): styled fallback page, asset 404s, build script safety - Add landingPageHtml() with gitnexus-web design tokens (void bg, surface cards, accent color, terminal-style build command block). - Add resolveWebDistDir() helper with non-ENOENT error logging. - Register express.static with Cache-Control headers (no-cache HTML, immutable assets) and SPA fallback route. - Replace wildcard SPA fallback with regex that excludes /api/* AND asset-like file extensions (.js, .css, .ico, .woff2, .map, etc.). - Add ordering comment warning about SPA fallback route placement. scripts/build.js: - Change npm install to npm ci. - Add timeout: 120_000 to all execSync calls. Test coverage: - 26 new unit tests for design tokens, terminal block, external links, SPA regex acceptance/exclusion, cache headers, and fs.access edge cases. Closes #1048 (review feedback) * fix: format, lint, and add GITNEXUS_WEB_DIST env var - Remove unused fsType import from web-ui-serving.test.ts (lint error) - Run prettier on fallback-page-screenshot.html and test file - Add GITNEXUS_WEB_DIST env var as primary override in resolveWebDistDir - Add tests for env var: prefer when set, fallback when dir missing * fix: use cross-platform path matching in env var tests Path.includes('/env/dist') fails on Windows where path.join produces backslashed paths. Normalize via path.sep replacement before matching. * fix(serve): address PR #1048 review findings - Add uncaughtException/unhandledRejection crash guards to HTTP serve path - Export SPA_FALLBACK_REGEX so tests use the production constant (no drift) - Export staticCacheControlSetHeaders so tests verify the real production function - Add real Express dispatch tests for API 404 and asset 404 isolation - Delete committed debug artifact fallback-page-screenshot.html |
||
|
Copilot
|
2b0392cd83
|
feat(analyze): preserve existing embeddings by default; --force regenerates them; add --drop-embeddings opt-out (CLI + HTTP API) (#1055)
* Initial plan * fix(analyze): preserve existing embeddings by default; add --drop-embeddings opt-out Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/da1da041-afcd-4d38-8a2f-39ca52a462ff Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * analyze: --force on embedded repo now regenerates embeddings (preserve+top-up) Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/e2759765-b8f6-453a-8c28-595439d23cb4 Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * analyze: wire dropEmbeddings into HTTP API; log cache-load failures; extract pure deriveEmbeddingMode + behavioral tests; sync GUARDRAILS.md Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/7d88e595-cbd8-47b2-ba4f-fb5b9a60cda4 Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> |
||
|
evolution
|
02739085d2
|
feat(embeddings): AST-aware chunking with offset-based splitting (#889)
Some checks are pending
CI / quality (push) Waiting to run
CI / tests (push) Waiting to run
CI / e2e (push) Waiting to run
CI / Save PR Metadata (push) Blocked by required conditions
CI / CI Gate (push) Blocked by required conditions
Release Candidate / Check if release candidate should run (push) Waiting to run
Release Candidate / ci (push) Blocked by required conditions
Release Candidate / Publish release candidate to npm (push) Blocked by required conditions
|
||
|
Copilot
|
1df79c2eab
|
fix: content-hash staleness detection for embeddings and vector index creation on zero-node path (#831)
* Initial plan * fix: stale vectors preserved on content edits and vector index missing after zero-node run Issue 1: Add contentHash to EMBEDDING_SCHEMA and embedding pipeline. - contentHash column persisted per CodeEmbedding row - POST /api/embed queries nodeId+contentHash, compares per-node hash - Stale rows (hash mismatch) are DELETE'd before re-embedding - Legacy DBs without contentHash treated as stale (full re-embed) - loadCachedEmbeddings and run-analyze cache restore include contentHash Issue 2: createVectorIndex called unconditionally before zero-node early return. Regression tests: - contentHashForNode determinism and content-change detection - EMBEDDING_SCHEMA includes contentHash STRING column - Pipeline exports verified Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/1581c0c0-f359-4376-b47e-62d24a28fd2d Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * fix: use parameterized query for stale embedding DELETE, revert package-lock.json Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/1581c0c0-f359-4376-b47e-62d24a28fd2d Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * fix: address review feedback — config consistency, narrow catches, extract DB logic Bug #1: Use finalConfig consistently in contentHashForNode (line 224 was using raw `config` while line 307 used `finalConfig`). Cache precomputed hashes in filter phase to avoid double computation (Perf #5). Bug #2: Narrow catch in loadCachedEmbeddings to only fall back on column/table-missing errors. Rethrow transient/connection errors. Bug #3: Log non-trivial DELETE failures instead of silently swallowing. Arch Violation #3: Extract fetchExistingEmbeddingHashes from api.ts into lbug-adapter.ts. Server layer now calls a single adapter function instead of re-implementing the DB query logic with nested try-catch. Tests: Add config consistency test, note that fetchExistingEmbeddingHashes tests require native module (run in CI). Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/b8c4f6b0-4095-4507-a15d-d8469793efac Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * fix: narrow Column error match to 'contentHash' in lbug-adapter fallback checks Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/b8c4f6b0-4095-4507-a15d-d8469793efac Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * fix: address production-readiness review — eliminate competing state, use schema constants, hard-fail on stale DELETE, add incremental filter tests Gap A / Arch Violation 1: Remove duplicate vectorExtensionLoaded flag from embedding-pipeline.ts — delegate to lbug-adapter's loadVectorExtension() which owns the VECTOR extension lifecycle and resets on DB reconnect. Arch Violation 2: Replace all hardcoded 'CodeEmbedding' and 'code_embedding_idx' strings in embedding-pipeline.ts and run-analyze.ts with EMBEDDING_TABLE_NAME, EMBEDDING_INDEX_NAME, and CREATE_VECTOR_INDEX_QUERY imported from schema.ts. Add EMBEDDING_INDEX_NAME export to schema.ts. Gap B: Make DELETE failure for stale vectors a hard throw (not just a warning). Continuing after failed DELETE risks Kuzu vector-index corruption since the constraint requires DELETE-before-INSERT for vector-indexed properties. "not found" / "does not exist" errors are still safe to ignore. STALE_HASH_SENTINEL: Define a named constant in embedding types.ts for the empty-string sentinel convention. Used consistently in lbug-adapter.ts and run-analyze.ts so the invariant is self-documenting. Tests: Add comprehensive unit tests for the incremental filter logic with mocked embedder: - New node → embedded - Unchanged node (hash matches) → skipped - Stale node (hash mismatch) → DELETE + re-embed - STALE_HASH_SENTINEL → treated as stale - Zero nodes after filter → createVectorIndex still called - DELETE failure with non-trivial error → throws Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/b21edee7-c9c5-4742-947b-d0def4fb26aa Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * fix: tighten error classification — extract isMissingColumnOrTableError helper, remove broad pattern matching - Extract isMissingColumnOrTableError() helper in lbug-adapter for consistent schema-error detection (replaces duplicate inline checks) - Tighten 'contentHash' match: now requires 'property' AND 'contentHash' (Kuzu-specific pattern) instead of broad 'contentHash' substring - Tighten DELETE error check: only ignore 'does not exist' (Kuzu's actual message), not broad 'not found' which could mask connection errors - Fix test node ID/name/filePath consistency Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/b21edee7-c9c5-4742-947b-d0def4fb26aa Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * fix: CI failures and final review — move STALE_HASH_SENTINEL to schema, tighten error matching, fix test mocking, format - Move STALE_HASH_SENTINEL from embeddings/types.ts to lbug/schema.ts (fixes inverted layer dependency: lbug should not import from embeddings) - Tighten isMissingColumnOrTableError: replace broad msg.includes('not found') with /(table|column|property).*not found/i regex to avoid matching transient errors - Add vi.resetModules() in test beforeEach for explicit module isolation (fixes vi.doMock not intercepting loadVectorExtension in CI) - Skip precomputedHashes.set() on unchanged (return false) path - Run prettier on all 5 files flagged by CI format check Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/e20311fd-4361-47b4-a137-9adc3e533b35 * fix: address remaining review nits — rename precomputedHashes, generalize error matcher, revert package-lock - Rename precomputedHashes → computedStaleHashes (hashes are computed on-demand during filter, only cached for stale nodes being re-embedded) - Remove contentHash-specific clause from isMissingColumnOrTableError — the regex /(table|column|property).*not found/i already covers it - Revert package-lock.json ssh→https protocol change Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/e20311fd-4361-47b4-a137-9adc3e533b35 --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> |
||
|
Jonas Vanderhaegen
|
c100577e5e
|
fix(embeddings): prevent batch errors from CodeEmbedding PK violations and vector-index SET restriction (#823)
* fix(csv-generator): deduplicate all node types, not just File nodes The pipeline can produce duplicate node IDs across all symbol types (Class, Method, Function, etc.). Only File nodes were guarded by a seenFileIds Set, leaving every other type unprotected. When the CSV was COPY'd into LadybugDB, duplicate PKs caused mass "Batch execution error: Found duplicated primary key value" warnings on gitnexus serve. Replace the per-type seenFileIds with a single seenNodeIds Set checked at the top of the iteration loop, before the switch, so every label is covered by the same O(1) deduplication guard. Fixes: #822 * fix(embeddings): use MERGE instead of CREATE for CodeEmbedding inserts CREATE fails with duplicate PK when a CodeEmbedding node already exists, which happens when: - A PostToolUse hook triggers a concurrent gitnexus analyze during an active analyze run (git commits fire the hook) - A partial prior run left some embeddings in the DB before a crash Switching to MERGE makes the insert idempotent: existing embeddings are updated in place, new ones are created, no PK violations. Fixes: #822 * fix(server): skip already-embedded nodes in POST /api/embed to avoid vector-index SET error Kuzu/LadybugDB forbids SET on a property that is part of a vector index. The /api/embed endpoint was calling runEmbeddingPipeline without skipNodeIds, causing it to attempt MERGE+SET on every node including those already embedded. Fix: query existing CodeEmbedding nodeIds before running the pipeline and pass them as skipNodeIds so only new (unembedded) nodes are processed. * fix(server): narrow catch to table-not-exist errors only in POST /api/embed Bare catch{} would silently swallow connection errors and proceed to re-embed all nodes, hiding infrastructure issues. Now only swallows errors where the CodeEmbedding table does not yet exist. * style: prettier format gitnexus/src/server/api.ts * fix(server): log skip-embedding count and table-not-found swallow path Addresses review feedback on PR #823: - Log count of already-embedded nodes when skipNodeIds is populated (aids debugging if Kuzu driver row shape changes). - Log when the 'table does not exist' swallow path fires so ops can catch it if Kuzu ever changes error wording. - Document the {} config positional argument with an inline comment referencing the runEmbeddingPipeline signature. --------- Co-authored-by: jonasvanderhaegen-xve <> Co-authored-by: Gergo Magyar <gergomagyar@icloud.com> |
||
|
Yogesh Singh
|
d87744fffc
|
fix: resolve false 404 errors and stale repo context during multi-repo switching on Windows (#633)
* fix: resolve false 404s and stale repo context during multi-repo switching on Windows * test(e2e): add repo-switching tests — hold-queue 503, ?project= URL, Windows path normalization * test(e2e): fix repo-switching specs — use live backend with ?server= param |
||
|
Mr. WorldwideBrown
|
6147579e54
|
Fix security issues and critical bugs found in code review (#709) | ||
|
JaysonAlbert
|
338cb01ee0
|
[codex] fix large repository graph loading (#732)
* fix(web): stream large graph responses * fix(server): harden graph streaming * fix(ci): stabilize graph loading coverage --------- Co-authored-by: gfwangjie <gfwangjie@gf.com.cn> |
||
|
Abhigyan Patwari
|
14791ded4a
|
fix(server): return clean CORS rejection instead of 500 error (#646) | ||
|
ivkond
|
52277247fe |
feat(group): add group infrastructure and contract matching
Core foundation for repository group analysis: - Type system: ContractType, ExtractedContract, StoredContract, CrossLink with optional `service` field for intra-repo matching - Config parser for group.yaml (repos, detection flags, matching thresholds) - Contract registry storage with atomic writes - Exact matching engine with per-type normalization (HTTP, gRPC, topic) and intra-repo support (different services within same repo can match) - Extract LadybugDB pool-adapter from MCP backend for reuse by sync pipeline - Git staleness checker for group status reporting Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Gergő Magyar
|
bf09eab95b
|
feat: configure prettier with pre-commit hook (#563)
* feat: configure prettier with pre-commit hook integration Add prettier, lint-staged, and prettier-plugin-tailwindcss at the repo root with husky pre-commit hook integration. Moves husky from gitnexus/ to root package.json for reliable hook installation. - Root package.json with prepare/format/format:check scripts - .prettierrc with endOfLine:lf and tailwindStylesheet for TW v4 - .prettierignore excluding fixtures, vendor, generated, *.d.ts, *.md - .gitattributes enforcing LF line endings for Windows consistency - Pre-commit hook uses direct node_modules/.bin/ paths (no npx) * style: apply prettier formatting to entire codebase One-time bulk format. No logic changes. Use .git-blame-ignore-revs to skip this commit in git blame. * chore: add .git-blame-ignore-revs for prettier format commit * perf: pre-commit hook runs only tests related to staged files Use vitest --related to scope test execution to tests that import the changed files, instead of running the full suite on every commit. * perf: remove vitest from pre-commit hook, keep in CI only Pre-commit now runs lint-staged + tsc only. Tests run in CI (ci-tests.yml) where they belong — keeps commits fast. * ci: add prettier format check to quality workflow PRs will now fail if code isn't formatted with prettier. |
||
|
Gergő Magyar
|
fd7fb5bf1f
|
feat: unify web and cli ingestion pipeline (#536)
* feat: add server-side ingestion API (POST /api/analyze, SSE progress)
Extract core analysis orchestration from CLI into shared run-analyze.ts
module. Add server-side analyze endpoints so the web app can trigger
ingestion via HTTP instead of running the full pipeline in-browser.
New files:
- src/core/run-analyze.ts — shared runFullAnalysis() orchestrator
- src/server/analyze-job.ts — job manager (single-slot, dedup, SSE events)
- src/server/analyze-worker.ts — forked child process (8GB heap, IPC)
- src/server/git-clone.ts — shallow clone/pull with SSRF protection
API endpoints:
- POST /api/analyze — start analysis (returns 202 + jobId)
- GET /api/analyze/:jobId — poll job status
- GET /api/analyze/:jobId/progress — SSE progress stream
Security: URL validation blocks private IPs and non-HTTP schemes.
Path validation requires absolute paths. Git stderr not leaked to API.
* feat(web): add server-side analyze UI (Phase 2)
Add "Analyze on Server" flow to the web app's Server tab so users
can trigger server-side ingestion from the browser. On completion,
the graph is automatically loaded via the existing connectToServer flow.
New files:
- AnalyzeProgress.tsx — progress bar with phase label, elapsed time, cancel
Modified files:
- backend.ts — startAnalyze(), streamAnalyzeProgress() SSE client
- DropZone.tsx — analyze URL input + button below Connect section
- App.tsx — onServerAnalyze handler wires analyze -> connect flow
* feat: add job cancellation, timeout, and child process tracking (Phase 3)
- DELETE /api/analyze/:jobId — cancel running analysis (SIGTERM to worker)
- 30-minute timeout kills long-running workers automatically
- Child process refs tracked in JobManager for cleanup on shutdown
- dispose() kills all active children on SIGINT/SIGTERM
- Web cancel button now calls server DELETE endpoint
- cancelAnalyze() added to web backend client
* refactor(web): remove browser ingestion pipeline (Phase 4)
Delete 16 duplicated ingestion files, 2 unused service files
(git-clone, zip), and tree-sitter parser-loader from gitnexus-web.
All ingestion now runs server-side via POST /api/analyze.
Deleted (18 files, ~5,000 lines):
- core/ingestion/*.ts (16 pipeline processors)
- core/tree-sitter/parser-loader.ts (WASM tree-sitter loader)
- services/git-clone.ts (isomorphic-git client-side clone)
- services/zip.ts (JSZip extraction)
Simplified:
- DropZone.tsx — server-only (removed ZIP/GitHub tabs)
- ingestion.worker.ts — removed runPipeline/runPipelineFromFiles
- useAppState.tsx — removed pipeline callbacks
- App.tsx — removed handleFileSelect/handleGitClone
- main.tsx — removed Buffer polyfill for isomorphic-git
- types/pipeline.ts — removed PipelineResult/serialize helpers
Kept: cluster-enricher.ts (LLM enrichment, still used by worker)
Dependencies now removable: web-tree-sitter, isomorphic-git,
@isomorphic-git/lightning-fs, jszip (estimated 3-4MB bundle savings)
* refactor(web): sync graph schema from CLI + delete WASM grammars
Sync graph/types.ts and lbug/schema.ts from the CLI (source of truth)
to the web module so the browser LadybugDB can handle all node and
relationship types the server pipeline produces.
Synced types: Route, Tool, Section node labels; HANDLES_ROUTE, FETCHES,
HANDLES_TOOL, ENTRY_POINT_OF, WRAPS, QUERIES relationship types;
description fields on Function/Class/Interface/Method/CodeElement.
Deleted: public/wasm/ directory (14 tree-sitter WASM grammars + core).
Removed deps: web-tree-sitter, isomorphic-git, @isomorphic-git/lightning-fs,
jszip, buffer, @types/jszip (~3-4MB bundle savings).
* feat: create gitnexus-shared package for unified type definitions
Create a new gitnexus-shared package that is the single source of truth
for types shared between the CLI and web modules:
- SupportedLanguages enum (15 languages)
- Graph types: NodeLabel, NodeProperties, RelationshipType, GraphNode, GraphRelationship
- Schema constants: NODE_TABLES, REL_TYPES, REL_TABLE_NAME, EMBEDDING_TABLE_NAME
- Pipeline types: PipelinePhase, PipelineProgress
Both gitnexus (CLI) and gitnexus-web import from gitnexus-shared via
file: dependency. Each package re-exports and extends with platform-specific
additions (CLI: KnowledgeGraph with mutation methods; Web: simpler KnowledgeGraph).
This ensures types can never drift between packages — adding a new
language, node type, or relationship type in gitnexus-shared automatically
propagates to both consumers.
* refactor: import shared types directly from gitnexus-shared at call sites
Replace all re-export patterns with direct imports from gitnexus-shared.
72 files updated across CLI and web:
- SupportedLanguages: 49 CLI files now import from 'gitnexus-shared'
instead of '../config/supported-languages.js'
- GraphNode, GraphRelationship, NodeLabel: 22 CLI + 10 web files now
import from 'gitnexus-shared' instead of local re-export wrappers
- NODE_TABLES: api.ts imports from 'gitnexus-shared'
- PipelineProgress: useAppState.tsx imports from 'gitnexus-shared'
Local types.ts files now only define platform-specific KnowledgeGraph
(CLI has mutation methods, web has add-only). No more re-exports.
* fix: update lock files for gitnexus-shared, remove stale vite polyfills
Add gitnexus-shared@1.0.0 to lock files so npm ci succeeds in CI.
Remove buffer polyfill and global define from vite.config.ts (isomorphic-git was removed).
* fix(security): add write guard to HTTP /api/query, fix CORS proxy bypass
- Add isWriteQuery() check to POST /api/query handler — blocks CREATE,
DELETE, SET, MERGE, DROP, etc. via HTTP API (guard was only in MCP
pool adapter and browser-side, not the HTTP server path)
- Extend CYPHER_WRITE_RE with CALL, INSTALL, LOAD keywords
- Fix CORS proxy subdomain bypass: endsWith('github.com') allowed
'evil-github.com'. Now requires exact match or '.github.com' suffix
* feat(server): enhance /api/search with enrichment, add /api/grep, strip graph content
- POST /api/search: add mode param (hybrid|semantic|bm25), server-side
enrichment returns connections/cluster/processes per result in one call
(collapses 31 sequential HTTP calls to 1 for the agent search tool)
- GET /api/grep: regex search across indexed file contents, eliminates
need to transfer all file contents to browser
- GET /api/graph: strip content field by default (80-95% payload
reduction). Use ?includeContent=true for backward compat
- Add LRU cache invalidation hook point for future caching
* feat(server): add /api/embed endpoint for server-side embedding generation
- POST /api/embed: triggers embedding pipeline via onnxruntime-node
with JobManager for single-slot concurrency, timeout, and dedup
- GET /api/embed/:jobId: poll job status
- GET /api/embed/:jobId/progress: SSE stream with heartbeat, event IDs,
and X-Accel-Buffering:no header for proxy compatibility
- DELETE /api/embed/:jobId: cancel running embedding job
- Maps embedding pipeline phases (ready→complete, error→failed) to
JobManager status conventions
* feat(web): create consolidated BackendClient module
Single HTTP client replacing backend.ts, server-connection.ts, and
worker HTTP helpers. Includes:
- Typed methods: runQuery, search (enriched), grep, readFile, connect
- Generic streamSSE<T> utility extracted from analyze progress pattern
- BackendError with discriminated code field (network/server/client/timeout)
- Embed API: startEmbeddings, streamEmbeddingProgress, cancelEmbeddings
- Search with mode param (hybrid|semantic|bm25) and enrichment
* refactor(web): rewrite Graph RAG tools for backend-only HTTP queries
- Search tool: uses enriched /api/search (1 call replaces 31 sequential queries)
- Cypher tool: removes browser-side embedding; {{QUERY_VECTOR}} routes to
/api/search with mode:'semantic' instead of local transformers.js
- Grep tool: uses /api/grep instead of in-memory fileContents map
- Read tool: uses /api/file instead of fileContents map lookup
- Impact tool: getCallSiteSnippet now async via /api/file
- createGraphRAGTools now accepts GraphRAGBackend interface instead of
7 separate function params + fileContents map
- createGraphRAGAgent simplified to (config, backend, context?)
- Removed imports: embedder, lbug/schema (replaced with gitnexus-shared)
- Net: -205 lines
* refactor(web): delete WASM infrastructure, remove 7 packages (-5242 lines)
Delete browser-side LadybugDB, embeddings, search, and worker:
- gitnexus-web/src/core/lbug/ (adapter, csv-generator, schema, query-result)
- gitnexus-web/src/core/embeddings/ (embedder, pipeline, text-gen, types)
- gitnexus-web/src/core/search/ (bm25-index, hybrid-search)
- gitnexus-web/src/workers/ingestion.worker.ts (828 lines)
- gitnexus-web/src/services/server-connection.ts (merged into backend-client)
- gitnexus-web/src/types/lbug-wasm.d.ts
Remove packages: @ladybugdb/wasm-core, @huggingface/transformers,
comlink, minisearch, vite-plugin-wasm, vite-plugin-top-level-await,
vite-plugin-static-copy
Update vite.config.ts: remove WASM plugins, COOP/COEP headers,
worker config, optimizeDeps exclude
Update imports: App.tsx, DropZone, Header, AnalyzeProgress,
BackendRepoSelector, useBackend → backend-client
* refactor(web): replace Worker/Comlink with direct BackendClient calls
- useAppState: remove Worker instantiation, Comlink.wrap, apiRef.
All queries now go through BackendClient HTTP functions directly.
- Agent runs on main thread (I/O-bound LLM streaming, not CPU-bound)
- initializeAgent: creates GraphRAGAgent with GraphRAGBackend interface
bound to BackendClient methods (runQuery, search, grep, readFile)
- startEmbeddings: calls POST /api/embed + SSE progress instead of
running browser-side transformers.js pipeline
- switchRepo: no longer loads graph into WASM DB or extracts fileContents
- App.tsx: handleServerConnect simplified (no fileContents, no loadServerGraph)
- Delete old backend.ts (replaced by backend-client.ts)
- Net: -396 lines
* fix(web): fix await-in-map build error in agent streaming
Move dynamic import of AIMessage outside .map() callback to avoid
"await can only be used inside an async function" build error.
* fix(web): remove stale apiRef references that broke chat functionality
sendChatMessage referenced apiRef.current (deleted Worker ref) which
would throw TypeError. Replaced with agentRef.current guard since agent
now runs on main thread.
* fix(server): dispose embedJobManager on shutdown, fix job mutation
- Add embedJobManager.dispose() to shutdown handler (was missing,
causing cleanup timer to keep Node process alive)
- Replace direct job.repoName/status mutation with updateJob() to
ensure SSE event emission for initial status change
* fix(server): parameterize Cypher, harden grep, unify SSE endpoints
- Search enrichment: replace string interpolation with executePrepared()
using $nid parameter binding to prevent Cypher injection
- Add executePrepared() to core lbug-adapter (prepare/execute pattern)
- /api/grep: add 200-char pattern length limit (ReDoS protection),
search files on disk instead of loading entire corpus into memory
(constant memory usage regardless of repo size)
- Extract mountSSEProgress() shared helper for SSE streaming — both
analyze and embed endpoints now have consistent heartbeat (30s),
event IDs (reconnection support), and X-Accel-Buffering header
* refactor(web): remove dead code from Worker-era architecture
- Remove loadServerGraph no-op function, interface member, and all consumers
- Remove testArrayParams stub and interface member
- Remove fileContents state from GraphStateProvider (never populated in
server-side architecture)
- Remove forceDevice parameter from startEmbeddings (server-side, no device choice)
- Replace phantom EmbeddingProgress type with inline { phase, percent }
- Replace resolvePathFromContents (needed fileContents Map) with graph-based
file path resolution using filePathIndex built from graph nodes
- Fix: AI citation grounding ([[file.ts:10]]) now works via graph node lookup
instead of broken fileContents-based resolution
* fix(web): use streamAgentResponse for full tool_call/reasoning streaming
Replace naive agent.stream() loop that only handled content chunks with
streamAgentResponse() generator from agent.ts. This properly routes:
- reasoning tokens (before/between tool calls)
- tool_call events (name, args, status)
- tool_result events (completed tool output)
- content tokens (final answer after all tools done)
Previously the onChunk handler for tool_call/tool_result/reasoning was
dead code since the streaming loop only emitted content events.
* fix(web): resolve CI type errors from dead code removal
- Import GraphNode/GraphRelationship from gitnexus-shared in graph.ts
(not re-exported from local types.ts)
- Add Route, Tool entries to NODE_COLORS and NODE_SIZES constants
- Add PipelineResult type to web types/pipeline.ts
- Remove fileContents from CodeReferencesPanel and RightPanel
- Remove testArrayParams and forceDevice from EmbeddingStatus
- Remove forceDevice args from startEmbeddings() calls in App.tsx
- Fix embeddingProgress property accesses for simplified type
* fix(ci): add setup-gitnexus-web action, build shared once per job
- Remove prepare script from gitnexus-shared (tsc not available during
npm ci of consuming packages)
- Create .github/actions/setup-gitnexus-web composite action: builds
gitnexus-shared then runs npm ci for gitnexus-web
- setup-gitnexus action: already builds gitnexus-shared for CLI jobs
- ci-quality typecheck-web: uses setup-gitnexus-web (DRY)
- ci-e2e: uses setup-gitnexus-web (DRY)
- ci-tests: gitnexus-shared already built by setup-gitnexus, just
install web deps without rebuilding
* fix(ci): use prepare script so gitnexus-shared builds during npm ci
Move typescript from devDependencies to dependencies in gitnexus-shared
so the prepare script (tsc) works when npm resolves file: deps during
npm ci. No GHA modifications needed — npm handles the build lifecycle
automatically.
Remove manual gitnexus-shared build steps from setup-gitnexus and
setup-gitnexus-web actions.
* fix(ci): build gitnexus-shared explicitly in setup actions
The file: dependency protocol doesn't reliably run prepare scripts
because devDependencies aren't installed first. Instead of fragile
lifecycle hacks, build gitnexus-shared explicitly in both setup actions:
- setup-gitnexus: npm install && npm run build in gitnexus-shared/
- setup-gitnexus-web: same, before npm ci in gitnexus-web/
- ci-tests: shared already built by setup-gitnexus, web just npm ci
No prepare script, no dist in git, no typescript as a prod dependency.
* fix: remove CALL from CYPHER_WRITE_RE — breaks FTS and vector search
CALL is used by read-only procedures: CALL QUERY_FTS_INDEX(...) and
CALL QUERY_VECTOR_INDEX(...). Adding it to the write guard blocked all
FTS search, causing 3 test failures. The database is opened in read-only
mode as defense-in-depth against write procedures via CALL.
Keep INSTALL and LOAD in the blocklist (genuinely dangerous).
* fix(web): update vercel.json for gitnexus-shared, remove COOP/COEP
- Add installCommand that builds gitnexus-shared before installing
web deps (Vercel doesn't know about the monorepo file: dependency)
- Remove Cross-Origin-Opener-Policy and Cross-Origin-Embedder-Policy
headers (no longer needed — WASM LadybugDB removed)
* fix(web): update tests for deleted modules
- Delete csv-generator.test.ts (tests deleted WASM-only csv-generator)
- Update security-guards.test.ts: import NODE_TABLES/REL_TYPES from
gitnexus-shared instead of deleted src/core/lbug/schema
- Update server-connection.test.ts: import normalizeServerUrl from
backend-client, remove extractFileContents tests (function deleted)
* fix(e2e): remove Server tab click — UI is now server-only
The DropZone no longer has ZIP/GitHub/Server tabs (browser ingestion
was removed). The server URL input is directly visible on the landing
page. Update e2e test to skip the tab click and go straight to input.
All 5 e2e tests pass locally.
* refactor: use gitnexus-shared for PipelinePhase/PipelineProgress types
CLI was duplicating PipelinePhase and PipelineProgress locally instead
of importing from gitnexus-shared. Updated all consumers to import
directly. Also removed dead code: SerializablePipelineResult,
serializePipelineResult(), deserializePipelineResult().
* fix(server): address PR #536 review — security, race conditions, dead code
- Fix path traversal in POST /api/analyze: split into isAbsolute + normalize check
- Add shared repo lock (activeRepoPaths) preventing concurrent analyze+embed on same repo
- Fix 202 response returning actual job.status instead of hardcoded 'queued'
- Add 30-minute timeout for embedding jobs (was missing unlike analyze jobs)
- Fix DropZone calling startAnalyze without setting backend URL first
- Add SSE reconnect with exponential backoff (3 retries) and Last-Event-ID
- Fix normalizeServerUrl to return base URL (no /api suffix) — clear contract
- Delete dead code: proxy.ts, server-graph-hydration.ts, pipeline.ts re-export barrel
- Update LoadingOverlay to import PipelineProgress directly from gitnexus-shared
* fix(server): fix repo lock key mismatch and embed cancel race
- Use getStoragePath(targetPath) as lock key in analyze handler to match
embed handler's entry.storagePath — keys now always align
- Guard embed completion: don't overwrite 'failed' with 'complete' when
job was cancelled while pipeline was still running
- Remove unused jobType parameter from acquireRepoLock
- Log backend.init() errors instead of silently swallowing
* fix: add gitnexus-shared as a local dependency in package-lock.json
* refactor: move language detection to gitnexus-shared, add syntax highlighting for all 15 languages
Move getLanguageFromFilename() from CLI to gitnexus-shared with COBOL
support added. Add getSyntaxLanguageFromFilename() for Prism-compatible
syntax highlighting covering all 15 code languages plus auxiliary
formats (json, yaml, markdown, html, css, bash, sql, xml).
Refactor CodeReferencesPanel to use shared function instead of a local
30-line switch. Delete dead gitnexus-web/src/config/supported-languages.ts
(web already imports SupportedLanguages from gitnexus-shared).
* feat(web): add first-time user onboarding with auto server detection
Replace the manual "Connect to Server" panel with an automatic onboarding
flow that guides first-time users through starting the GitNexus server.
Server detection:
- useBackend hook polls via setTimeout chain (3s, no overlap)
- Page Visibility API pauses polling when tab is hidden
- SSE heartbeat (/api/heartbeat) for instant disconnect detection
Onboarding UI (OnboardingGuide.tsx):
- Step-by-step flow: copy command → run → auto-connect
- Smart command: shows `gitnexus serve` in dev, `npx gitnexus@latest serve` in prod
- Node.js version auto-detected from package.json via Vite define
- Faux terminal windows with copy-to-clipboard, platform tabs, polling indicator
Transitions (DropZone.tsx):
- Crossfade wrapper with snapshot pattern for smooth phase transitions
- Three phases: onboarding → success (1.2s hold) → loading → graph
- Auto-recovery: falls back to onboarding if server dies or connect fails
Server changes:
- GET /api/heartbeat: SSE endpoint for liveness detection
- GET /api/info: version, launch context, Node.js version
- npm run serve script for local development
- app.disable('x-powered-by') hardening
* feat(web): add repo analysis UI, SSE heartbeat, and review fixes
Repo analysis:
- AnalyzeOnboarding: empty-state card when server has zero repos
- RepoAnalyzer: GitHub URL + Local Folder tabs with browse button
- Header repo dropdown: click project badge to switch repos or analyze new
- DropZone 'analyze' phase integrated into Crossfade transitions
Reliability fixes from 5-agent review:
- Polling: stop scheduling timers when tab hidden, restart on visibility return
- Heartbeat: exponential backoff (1s/2s/4s, 3 retries) prevents graph loss on blip
- RepoAnalyzer: completion timer tracked in ref, cleaned up on unmount
- DropZone: standardized card padding (p-7), heading sizes (text-lg)
Accessibility:
- prefers-reduced-motion global CSS rule (WCAG 2.3.3)
- focus-visible rings on CopyButton
- cursor-pointer on all Header buttons
- Consistent rounded-xl on all dropdowns
Cleanup:
- Deleted dead AnalyzeSheet.tsx (219 LOC) and BackendRepoSelector.tsx (89 LOC)
- Fixed AnalyzeProgress lucide import (lucide-react → @/lib/lucide-icons)
* fix(server): resolve analyze worker fork crash in dev mode
The forked analyze worker was crashing immediately with exit code 1
when running via `npm run serve` (tsx). Two issues:
1. Worker path resolved to `analyze-worker.js` but only `.ts` exists
in the source directory — the `.js` file is only in `dist/`.
2. On Windows, bare `--import tsx` in execArgv fails because Node's
ESM resolver for --import uses the child's CWD, not the parent's
node_modules. Windows also rejects raw paths as `d:` is not a
valid URL scheme.
Fix: detect dev vs prod via `import.meta.url` extension. In dev mode,
resolve `tsx/esm` to an absolute `file://` URL via `pathToFileURL()`
anchored to the parent's `createRequire` context. This works on all
platforms and doesn't depend on the child's CWD or PATH.
Also captures child stderr for better crash diagnostics.
Verified: `POST /api/analyze` with GitHub URL completes successfully
in dev mode (tsx) — status goes from cloning → analyzing → complete.
* fix(server): add worker auto-retry, error handling, and crash diagnostics
Worker resilience:
- Auto-retry up to 2 times with exponential backoff (1s, 2s) on crash
- SSE progress shows "Retrying after crash (1/2)..." during retry
- Captures child stderr for crash diagnostics in failure message
- AnalyzeJob tracks retryCount per job
Server error handling:
- app.listen wrapped in Promise so EADDRINUSE/EACCES propagate cleanly
- serve.ts catches startup errors with friendly messages and exit code 1
- EADDRINUSE gets actionable guidance (stop other process or --port flag)
- Global uncaughtException/unhandledRejection handlers prevent silent exits
- DEBUG=1 env var shows full stack traces
* feat: add e2e tests for onboarding flows, worker retry, and error handling
E2E tests (onboarding.spec.ts — 11 tests):
- Flow 1: OnboardingGuide shown when server unreachable (6 tests)
- Flow 2: Auto-connect with success card, analyze phase for zero repos
- Flow 3: Analyze form — GitHub URL validation, Local Folder tab, tab switching
- Flow 4: Repo dropdown in exploring view (skipped without live server)
Updated server-connect.spec.ts:
- Replaced manual Connect button flow with auto-connect waitForGraphLoaded
Server resilience:
- Worker auto-retry (2 attempts with exponential backoff) on crash
- Friendly error messages for serve startup failures (EADDRINUSE etc.)
- Global uncaughtException/unhandledRejection handlers prevent silent exits
- app.listen wrapped in Promise for proper error propagation
* refactor(shared): enforce exhaustive language coverage via Record types
Replace the if/else chain in getLanguageFromFilename with two exhaustive
Record<SupportedLanguages, ...> maps:
- EXTENSION_MAP: every language → its file extensions
- SYNTAX_MAP: every language → its Prism syntax identifier
Adding a new member to the SupportedLanguages enum without adding it to
both maps now produces a TypeScript compile error:
Property '[SupportedLanguages.NewLang]' is missing in type...
This matches the existing pattern in languages/index.ts (providers table)
which already uses `satisfies Record<SupportedLanguages, LanguageProvider>`.
Three compile-time enforcement points now exist:
1. EXTENSION_MAP in language-detection.ts (file extensions)
2. SYNTAX_MAP in language-detection.ts (Prism syntax identifiers)
3. providers in languages/index.ts (LanguageProvider instances)
* feat(web): load source code from server and scroll to selected line
CodeReferencesPanel now fetches file content via GET /api/file when a
node is selected, instead of showing "Code not available in memory".
- Fetches via readFile() from backend-client when selectedFilePath changes
- Shows loading spinner while fetching
- After content loads, auto-scrolls to the selected node's startLine
- Highlights the selected line range with a cyan left border
- Cancels in-flight fetch if selection changes before it completes
Also: refactored language-detection.ts to use exhaustive Record types
(EXTENSION_MAP and SYNTAX_MAP) so adding a new SupportedLanguages enum
member without implementing extensions/syntax is a compile error.
* feat: buffered file reading for Code Inspector
Server: GET /api/file now supports ?startLine=N&endLine=M for reading
a line range instead of the entire file. Returns { content, startLine,
endLine, totalLines }.
Client: readFile() returns ReadFileResult with metadata. When selecting
a symbol (function, class, method), fetches only ±50 lines around the
symbol's startLine/endLine instead of the full file. File nodes still
fetch the entire file.
SyntaxHighlighter startingLineNumber set from the buffer offset so line
numbers are correct even for partial reads.
* fix: adapt readFile callers to new ReadFileResult return type
tools.ts: readFile comes from GraphRAGBackend interface which returns
Promise<string> (the adapter in useAppState extracts .content), so
revert the { content } destructuring back to plain string assignment.
useAppState.tsx: wrap backendReadFile with { repo } options object
and extract .content to satisfy the GraphRAGBackend interface.
* fix(web): ensure new repos appear in list immediately after analysis
Two fixes:
1. DropZone: handleAnalyzeComplete now passes the repoName through to
connectToServer so the specific newly-analyzed repo loads — not the
server's default first repo.
2. App.tsx: fetchRepos() is now awaited BEFORE handleServerConnect in
both the DropZone and Header flows. This ensures the repo list is
populated before the exploring view renders, so the new repo appears
in the header dropdown immediately without a page reload.
* feat: delete repos, re-analyze with force, select after analysis
Server — DELETE /api/repo:
- Acquires repo lock first (409 if analyze/embed in flight)
- Closes LadybugDB, deletes index + clone dir, unregisters, re-inits
- Lock released in finally block
Server — analyze complete:
- backend.init() must succeed before SSE complete fires
- If backend.init() fails, job is marked failed (not complete)
Web — Header repo dropdown:
- Re-analyze: calls POST /api/analyze with force=true, shows spinning
icon + inline progress bar via SSE
- Delete: acquires lock, aborts any running re-analysis SSE for same
repo, refreshes list, switches to next repo
- After analysis completes: refreshes repo list, connects to the
specific repo by name, loads graph, shows in explorer
- Retry with 1.5s backoff on 404 (server may still be reinitializing)
Type safety:
- err: any → err: unknown + instanceof BackendError in retry loop
- Added missing BackendRepo + BackendError imports in App.tsx
|
||
|
Shunsuke Hayashi
|
d07a69c3b1 |
fix(server): allow private/LAN network origins in CORS (#390)
Allow RFC 1918 private network ranges in the CORS origin allowlist so users running GitNexus on their home or office LAN can access the web UI from another device on the same network. Permitted private ranges: 10.0.0.0/8 (10.x.x.x) 172.16.0.0/12 (172.16.x.x – 172.31.x.x) 192.168.0.0/16 (192.168.x.x) The origin check is extracted into an exported isAllowedOrigin() helper so it can be unit-tested in isolation. A new test file covers: - No origin (curl / server-to-server) - localhost and 127.0.0.1 variants - All three RFC 1918 ranges including boundary values - The deployed gitnexus.vercel.app site - Public / untrusted origins that must be rejected The server bind address (127.0.0.1 by default) is unchanged; this PR only affects which cross-origin browser requests are accepted. Closes #390 |
||
|
Candido Sales Gomes
|
5a5850832c
|
refactor: migrate from KuzuDB to LadybugDB v0.15 (#275)
* refactor: migrate from KuzuDB to LadybugDB v0.15 KuzuDB was archived (Apple acquisition, Oct 2025). LadybugDB is the community fork with full API compatibility. - Package swap: kuzu → @ladybugdb/core, kuzu-wasm → @ladybugdb/wasm-core - Rename all internal paths: kuzu → lbug (adapters, schema, storage) - Storage path: .gitnexus/kuzu → .gitnexus/lbug (with auto-cleanup) - Add explicit VECTOR extension loading (required in v0.15) - Update CI workflow, documentation, and all tests - 1151 unit + 27 integration tests passing Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address code review findings (P1-P3) P1: Fix WASM adapter to use getAll() API, wire cleanupOldKuzuFiles into analyze command, add symlink path traversal protection. P2: Cache VECTOR extension load state, batch augmentation engine queries (20→4), fix web getCopyQuery for multi-language tables, fix stale KuzuDB references, correct brainstorm package names. P3: Complete lbug-wasm.d.ts type declarations, batch semantic search per-label, update stale BM25 comment. * chore: remove outdated KuzuDB migration brainstorming document * fix: load FTS extension in MCP pool adapter on init The read-only pool adapter never loaded the FTS extension, so all QUERY_FTS_INDEX calls failed silently. This broke search-pool and augmentation integration tests, and caused empty results in the web UI server mode. * feat: implement shared Database caching and connection reference counting * feat: enhance KuzuDB migration handling and status reporting * fix: mock cleanupOldKuzuFiles in local backend callTool tests * fix: update mock for cleanupOldKuzuFiles and adjust imports in callTool tests --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
abhigyanpatwari
|
989673a624 |
fix: lazy-import embeddings to avoid onnxruntime crash on unsupported Node versions
Convert static imports of @huggingface/transformers (which triggers onnxruntime-node native binary loading) to dynamic import() calls. This prevents crashes on Node versions whose ABI isn't supported by the prebuilt onnxruntime binaries (e.g. Node v24). Affected entry points: - cli/analyze.ts: embedding pipeline only loaded when --embeddings is passed - mcp/local/local-backend.ts: embedder only loaded on first semantic search - server/api.ts: embedder only loaded when search endpoint needs embeddings Fixes #89 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Nico Prieto
|
04be81f655 | fix(server): harden multi-repo API and MCP safety | ||
|
Nico Prieto
|
f047a84d82 |
fix(server): restore security guards and error handling
Address code review feedback on the server-mode PR: Critical fixes: - Restore CORS whitelist (localhost + gitnexus.vercel.app only) - Bind to 127.0.0.1 by default; add --host CLI flag for opt-in remote access - Restore path traversal guard on /api/file (resolve + startsWith check) - Restore try/catch on all route handlers + global error middleware - Restore SIGINT/SIGTERM graceful shutdown handlers Bug fixes: - Add mutex to core initKuzu to prevent race conditions on concurrent DB switches (two requests for different repos no longer corrupt state) - Track ftsLoaded flag and reset on DB switch / close so FTS extension is reloaded for each new database connection - Restore input validation on /api/query (cypher required) and /api/search (query required) Improvements: - Add TTL-based cleanup for orphaned MCP sessions (30min idle eviction) to prevent memory leaks from network drops that skip onclose Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Nico Prieto
|
7ee2dd1087 |
feat: add remote server connection mode and multi-repo switching
Replace the old local-only backend mode with a new server connection flow that lets the web UI connect to any running GitNexus server, download the pre-built knowledge graph, and explore it without WASM. - Add server-connection service with streaming download and progress - Replace DropZone backend tab with server connect UI (URL input, progress bar, cancel) - Add repo switcher dropdown in Header when multiple repos are indexed - Mount MCP server over StreamableHTTP at /api/mcp for remote AI tool access - Refactor api.ts to query KuzuDB directly instead of routing through LocalBackend - Fix KuzuDB adapter to support switching between databases (close old before opening new) - Extract createMCPServer() from startMCPServer() for transport-agnostic reuse - Support ?server= query param for bookmarkable auto-connect Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Paul Robello
|
acd918f44e |
refactor(api): use executeCypher directly in buildGraph
Add public executeCypher(repoName, query) method to LocalBackend,
bypassing the callTool string dispatch and resolving the repo handle
directly. Update buildGraph in api.ts to call it instead of going
through callTool('cypher', ...) on every query.
|
||
|
Paul Robello
|
a71924f774 |
fix: address PR review — CORS, search quality, and embedding flags
- Add https://gitnexus.vercel.app to CORS allowed origins so the deployed site can connect to the local backend server. - Replace naive createHttpTextSearch (substring matching via Cypher) with createHttpHybridSearch that calls /api/search for full BM25 + semantic + RRF hybrid search on the server. - Set isEmbeddingReady to false in backend mode (no local embedder), which also fixes {{QUERY_VECTOR}} handling — the existing error path in tools.ts now correctly returns a helpful message. - Set isBM25Ready to true (available via server's hybrid search). - Removes the Cypher injection bug (wrong escape char) since the function that contained it was replaced entirely. |
||
|
Paul Robello
|
36c7b3ed12 |
fix: address final code review issues
- Fix field name mismatch in /api/repo (repoPath -> path) - Add setProgress(null) on successful backend repo load - Prevent DropZone tab auto-switch from re-triggering - Make isDatabaseReady() return true in backend mode - Increase fetchGraph timeout to 60s for large repos - Replace dynamic import with static import for runCypherQuery |
||
|
Paul Robello
|
102850455d | fix(serve): address code review - security, error handling, CORS | ||
|
Paul Robello
|
8e8bb90fe4 |
refactor(serve): rewrite API server to use LocalBackend for multi-repo support
Replace direct single-repo KuzuDB access with LocalBackend, the same backend the MCP server uses. The serve command now works with ALL indexed repos via the global registry instead of only the CWD repo. New/updated endpoints: - GET /api/repos (list all indexed repos) - GET /api/repo?name=X (repo metadata) - GET /api/graph?repo=X (full knowledge graph) - POST /api/query (raw Cypher via backend) - POST /api/search (process-grouped search via backend) - GET /api/file?repo=X&path=Y (file read with path traversal protection) - GET /api/processes?repo=X, /api/process?repo=X&name=Y - GET /api/clusters?repo=X, /api/cluster?repo=X&name=Y |
||
|
abhigyanpatwari
|
1735c0ffcd | gitnexus wal cleanup preventing reindexing issue fixed | ||
|
abhigyanpatwari
|
c90576442e | feat: merge gitnexus-mcp into gitnexus package - unified CLI+MCP |