mirror of https://github.com/abhigyanpatwari/GitNexus.git synced 2026-09-07 08:26:11 +00:00

fix(server,web): honor the grep tool contract — real regex, fileFilter, caseSensitive (#3109 )

* fix(server,web): honor grep tool contract — real regex, fileFilter, caseSensitive (Patch 12)

Background
==========

The web chat's grep tool schema has always promised regex search with an
optional path-substring fileFilter and caseSensitive control, but the
GET /api/grep handler escapeRegExp()'d every pattern into a literal
substring (a ReDoS hardening from fa36254e / #1317 that never re-synced
the tool contract). Consequences, verified in production use against the
sr-next backend repo (23k-file Java monorepo):

- An agent sending the documented alternation form ("sign|Sign") got
  zero hits and concluded the sign/签署 interface did not exist.
- The schema's own example pattern ("console\\.log") could never match:
  the escaped literal searched for a backslash in the source.
- fileFilter / caseSensitive were read by nobody — pure schema fiction.
- The web handler worked around the server with a (?=.*filter).*pattern
  lookahead splice that the same escaping also defeated.
- Collateral: the impact tool's grep fallback (\b${escapeRegex(name)}\b)
  was silently dead code under literal semantics; it comes back to life
  with this fix (expected improvement, noted for reviewers).

Fix
===

Server (gitnexus):
- New src/server/grep-params.ts — pure query-param parser (no Express /
  native imports, per the #2790 helper-extraction convention):
  regex construction (default real regex; literal=1 restores the old
  escaped-substring semantics as an opt-out), lowercase path-substring
  fileFilter, caseSensitive flag, limit clamp [1,200] default 50,
  BadRequestError error paths (mapped to 400 by statusFromError).
- /api/grep handler in api.ts becomes thin wiring: fileFilter path
  filtering before the (unchanged) traversal guard, a 5s wall-clock
  budget checked between files (partial results plus timedOut: true),
  read-only DB open unchanged. The regex is deliberately built WITHOUT
  the 'g' flag: the handler tests line-by-line and a stale lastIndex
  would skip matches (the old code had to reset it manually); 'm' is
  likewise omitted — each test sees one line, so ^/$ already anchor at
  string boundaries.

Web (gitnexus-web):
- tools.ts: drop the lookahead splice; description now tells the model
  the truth (real regex, alternation works, path-substring filter,
  case-insensitive default, result cap and time budget).
- backend-client.ts: grep() takes GrepOptions {fileFilter,caseSensitive}
  and forwards them as query params.
- useAppState.tsx: assembly site threads the options through.

Security — residual ReDoS exposure (read this before deploying)
===============================================================

The literal-only era was accidentally ReDoS-immune; this patch knowingly
trades that immunity back for the promised contract. The bounds (200-char
pattern cap, line-by-line matching, result cap, 5s budget) do NOT cover a
single catastrophically backtracking regex.test(): it blocks the Node
event loop synchronously, the budget (checked between files) cannot
interrupt it, and the whole server is unresponsive for the duration
(measured: (a+)+$ against a 35-char line exceeds 120 seconds). Accepted
because local serve binds loopback by default and hosted deploys gate
/api/grep behind the edge token; documented in SECURITY.md (new section)
with the worker_threads+terminate / optional-re2 follow-up called out.
literal=1 restores full immunity for untrusted callers.

Compatibility audit
===================

Repo-wide: /api/grep's only HTTP caller is backend-client.grep(); the MCP
tool surface has no grep tool; eval/ uses shell grep, not this endpoint;
the endpoint is undocumented (docs/llms.txt) with no known third-party
consumers. Breaking surface ≈ zero. Pattern metacharacter semantics
change for direct curl users ("array[0]" now needs escaping or literal=1).

Tests
=====

+20 cases in test/unit/grep-params.test.ts: alternation (the regression
that burned the agent), the schema's own example, case flags, literal
compat, CJK patterns, ^/$ line anchors, fileFilter normalization +
array-form rejection, limit clamping, type-confusion guards, invalid
regex, and a source-level handler wiring assertion (api-readonly-wiring
style). Full unit suite: no new failures (22 pre-existing failures
reproduced identically with this patch stashed — analyzer-identity dist
fingerprint + lbug native-env classes).

Upstream plan
=============

Issue + PR to abhigyanpatwari/GitNexus; the PR description must front the
ReDoS trade-off with the worker-isolation follow-up. Repro for the issue:
curl ".../api/grep?pattern=TODO%7CFIXME" — 0 hits under literal
semantics, both marker classes under regex semantics.

Custom-patch ledger: CUSTOM_PATCHES.md Patch 12.

* style: prettier

* fix(web): surface grep timedOut so partial scans are not silent misses

Propagate the server timeout flag through the backend client and chat tool, check the 5s budget between lines, and document the accepted regex-injection CodeQL finding next to new RegExp.

* Address PR review feedback (#3109)

- Cover empty and null fileFilter in the grep client test
- Keep timedOut as a required boolean and reuse GrepOptions
- Sample the grep deadline every 256 lines instead of every line

Note: pre-existing failure in impact-tool.test.ts not addressed by this PR.

* Address PR review feedback (#3109)

Run /api/grep matching in a worker_threads worker so terminate() can
cut a catastrophic regex.test without blocking the parent event loop.

* Address PR review feedback (#3109)

Reset lastIndex per line, restore the missing-pattern 400 message, and
make the traversal test create a real outside file.

* fix(web): align agent grep opts with GrepOptions

Use GrepOptions so fileFilter null is accepted by the local GraphRAGBackend stub.

* fix(server): silence CodeQL js/regex-injection on intentional grep regex

Split literal vs regex construction and suppress with the correct rule id
(js/regex-injection). Real regex remains the default contract; literal=1
still escapes.

* Address PR review feedback (#3109)

Clean up grep-scan temp dirs after each test, and exclude the intentional
grep-params RegExp site from CodeQL so js/regex-injection does not re-file.

---------

Co-authored-by: l.cx <l.cx@winning.com.cn>
Co-authored-by: ChunxueLi <mecoloud@users.noreply.gitee.com>
Co-authored-by: Gergő Magyar <gergomagyar@icloud.com>
Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>

2026-08-31 20:43:28 +01:00

7.1 KiB

Raw Permalink Blame History

Security Policy

Supported Versions

GitNexus is developed on main. Security fixes are applied to the latest released minor on npm (gitnexus) and to the published Docker images (Dockerfile.cli, Dockerfile.web). Older minors are not back-patched.

Reporting a Vulnerability

Please do not open a public GitHub issue for security reports.

Use GitHub Private Vulnerability Reporting for this repository:

→ https://github.com/abhigyanpatwari/GitNexus/security/advisories/new

Please include:

A description of the issue and its potential impact
Steps to reproduce (a minimal repro repo or commit hash if possible)
The affected version(s) — npm view gitnexus version, image digest, or commit SHA
Any suggested mitigation

What to expect

Acknowledgement: best-effort within 5 business days, subject to maintainer capacity.
Triage: we will confirm whether the report is in scope, request clarifications if needed, and propose a fix timeline.
Disclosure: coordinated. We will agree on a disclosure date with you before publishing an advisory.

Scope

In scope:

The gitnexus CLI and MCP server (gitnexus/)
The gitnexus-web thin client (gitnexus-web/)
The gitnexus-shared types package (gitnexus-shared/)
The published Docker images (Dockerfile.cli, Dockerfile.web)
GitHub Actions workflows in .github/workflows/

Out of scope:

Vulnerabilities in third-party dependencies that we have no influence over (please report upstream; if a viable mitigation exists at the GitNexus layer, that's in scope).
Issues requiring physical access to a developer machine or a compromised local environment.
Theoretical attacks without a practical exploit against a default GitNexus deployment.

Recommended Hardening for Forks and Self-Hosted Deployments

If you fork GitNexus or self-host it, we recommend enabling the following in your repository's Settings → Code security and analysis:

Private vulnerability reporting — the channel described above.
Dependabot alerts — alerts on advisories affecting your dependencies.
Dependabot security updates — automated PRs for security patches (this repo's .github/dependabot.yml already covers version updates).
Secret scanning and Push protection — blocks pushes that introduce known secret patterns. Defense-in-depth on top of the in-CI Gitleaks scan documented below.
Code scanning — surfaces SARIF results from CodeQL, Trivy, Scorecard, and zizmor in one place.

Hosted Deploys on Render

The render.yaml Blueprint (see the README's Deploy to Render) puts gitnexus serve on a private service with no public URL, and a public web service in front of it that reverse-proxies /api/*. What that does and does not protect:

The web service is public and its URL is discoverable. onrender.com hostnames appear in certificate transparency logs. Treat the URL as known rather than secret.
The generated GITNEXUS_SERVE_AUTH_TOKEN is the only access control. The proxy rejects any /api/* request without it with a 401 before forwarding. Rotate it by editing the environment variable on the gitnexus-web service and redeploying.
The CSRF guard is inert on this path. The proxy strips Origin before forwarding, so the server's write-origin guard does nothing for proxied traffic — it passes Origin-less requests through by design. The token is not a second layer behind the guard.
Anyone holding the token can read every indexed repo's source. These routes carry no origin guard, and the first three carry no rate limiter either: GET /api/repos, GET /api/graph, POST /api/query, GET /api/file, GET /api/grep. Whoever has the token can also index and delete repositories.
POST /api/mcp rides the same path. When GITNEXUS_MCP_AUTH_TOKEN is set on the backend, serve protects /api/mcp with the same constant-time Bearer check as the dedicated HTTP MCP server, before parsing the request body. The Render Blueprint does not set a backend MCP token by default. To enable it behind the proxy, set the same GITNEXUS_MCP_AUTH_TOKEN on both the gitnexus-web proxy and the gitnexus-server backend: the proxy consumes the edge GITNEXUS_SERVE_AUTH_TOKEN, then replaces Authorization with the MCP token on /api/mcp (and its subpaths) only — the edge credential is never forwarded, and other /api/* routes stay stripped. Configuring it on the backend alone makes every proxied MCP request 401.
A directly reachable serve still needs an explicit control. If neither GITNEXUS_MCP_AUTH_TOKEN nor an authenticated edge/private-network boundary is present, /api/mcp is unauthenticated. Do not bind that topology to a LAN or public interface: MCP readers can access indexed source and graph context.
Rate limits bound cost, not access. They cap what a token holder can spend; they do not decide who gets in.

Do not hand the URL out as a public demo. A token holder has read access to everything the deploy has indexed.

`/api/grep` regex semantics and residual ReDoS exposure

GET /api/grep executes caller-supplied patterns as real regular expressions (with an optional path-substring fileFilter and caseSensitive flag) to honor the web chat's grep tool contract; literal=1 restores the older escaped-substring mode. Mitigations: a 200-character pattern cap, line-by-line matching, a max-200 result cap, and a 5-second wall-clock budget. Matching runs in a worker_threads worker so a catastrophic pattern (e.g. (a+)+$) can be killed with terminate() when the budget expires — the parent event loop (other routes + SSE) stays responsive. A timed-out scan returns partial results with timedOut: true; the web grep tool surfaces that flag so an agent does not treat a cut-off scan as exhaustive. CodeQL still flags constructing a RegExp from the query string; that is the advertised contract, not accidental injection. Hosted deploys continue to gate the route behind the edge token.

Automated Scans Running in CI

This repository runs the following scans automatically. Findings appear under the repository's Security → Code scanning tab.

Scan	Tool	Trigger	Action on finding
Static analysis (JS/TS, Python)	CodeQL	PR, `main` push, weekly	Advisory (Security tab)
Dependency vulnerabilities (PR diff)	`dependency-review-action`	PR	Blocks PR at `high+` severity
Secret scanning	Gitleaks	PR, `main` push	Blocks PR on default rules
Supply-chain posture	OpenSSF Scorecard	Weekly, `main` push	Advisory (Security tab + public badge)
Workflow lint	zizmor	PR (touching `.github/**`)	Blocks PR at `high+` severity
Container image scan	Trivy	Weekly, `main` push	Advisory (Security tab)

Dependency version updates are managed separately by Dependabot — see .github/dependabot.yml.

7.1 KiB Raw Permalink Blame History