mirror of
https://github.com/abhigyanpatwari/GitNexus.git
synced 2026-08-28 05:25:25 +00:00
* fix(lbug): keep serve stable when sidecars are missing Shared missing-shadow WAL recovery prevents repeated read-only open warnings when LadybugDB sidecars are absent, while the Express preflight fix keeps `gitnexus serve` compatible with Express 5 route parsing. Constraint: LadybugDB read-only replay can require a `.shadow` sidecar that may be absent after interrupted writes or checkpoint edge cases. Rejected: keep reactive WARN-only quarantine in each adapter | it leaves repeated user-visible warnings and duplicate recovery behavior. Confidence: high Scope-risk: broad Directive: Do not silently delete large orphan WALs; only quarantine tiny orphan WALs before open and keep large WALs for explicit recovery. Tested: cd gitnexus && npx vitest run test/unit/sidecar-recovery.test.ts test/unit/lbug-adapter-wal-schema.test.ts test/unit/pool-wal-recovery.test.ts test/unit/web-ui-serving.test.ts && npx tsc --noEmit Not-tested: full npm test in this split branch; full unit suite passed on the source branch before PR split. Co-authored-by: OmX <omx@oh-my-codex.dev> * fix(lbug): pool-caller ENOENT guard, symmetric size gate, permission-aware errors (PR #1747 review) Addresses the production-readiness review of PR #1747 (Findings 1, 2, 3 of 6). Findings 4, 5, 6 are deferred to follow-ups per the plan. 1. ENOENT-tolerance scoped to pool-adapter callers only - `quarantineWalForMissingShadow` stays strict in `sidecar-recovery.ts`. The direct adapter calls it inside `acquireInitLock` (cross-process file lock) — ENOENT there means the file vanished under lock and remains a real bug to surface. - New `tryQuarantineForMissingShadow` local helper in `pool-adapter.ts` returns a discriminated union { kind: 'quarantined', path } | { kind: 'peer-handled' }. Catches ENOENT, re-verifies via statIfExists, and converts to 'peer-handled' only when WAL really is gone. Defensive: if ENOENT but WAL still present, throws as classified error rather than silently returning success. 2. Symmetric WAL-size gate on both recovery paths - `refuseLargeWalQuarantine` applied in both `reopenReadOnlyAfterMissingShadow` and `reopenWritableAfterMissingShadow`. Closes the read-only data-loss vector (large orphan WAL silently discarded would never be replayed by a later writable open). 3. Permission-aware error classifier - New `renameFailureMessage` and `isPermissionRenameError` in `sidecar-recovery.ts`. EACCES / EPERM / EBUSY now surface a permission-specific message pointing at ACLs, AV exclusions, and file-locks. Other codes (ENOSPC, EROFS, EIO, ENOENT) fall through to `shadowSidecarRecoveryMessage`. - Used at both pool-adapter and direct-adapter caller catches around `quarantineWalForMissingShadow`. - `doInitLbug`'s pass-through classifier extended to include the new permission message. The lock-retry substring match tightened so "file-lock error" in the permission message is not mistaken for a LadybugDB lock-retry trigger. Tests - sidecar-recovery.test.ts: 7 new tests for `renameFailureMessage` and `isPermissionRenameError`. - pool-wal-recovery.test.ts: 6 new tests covering ENOENT race, EACCES/EPERM/EBUSY classification, ENOSPC fallthrough, and the defensive "WAL still present after ENOENT" branch. - lbug-adapter-wal-schema.test.ts: 5 new tests covering the symmetric size gate on both recovery paths, including the boundary at exactly TINY_ORPHAN_WAL_BYTES (4096) and the off-by-one at 4097. Deferred (tracked as follow-up work) - Brittle LadybugDB error-string matching (Finding 4). - PNA header end-to-end coverage gap (Finding 5). - warnedKeys module-global persistence (Finding 6). - Cross-process init lock for pool-adapter. * fix(lbug): dedup shadow-replay predicate + counter-based warn anti-spam (PR #1747 review, Findings 4 & 6) Smallest viable response to the two remaining non-blocking findings from the production-readiness review of PR #1747. An earlier-revision plan proposed regex widening + a near-miss detector + per-dbPath warn scoping; an adversarial doc-review found those defended against hypothetical strings LadybugDB does not produce, added observability theater with no recovery behavior change, and did not actually fix the long-running gitnexus serve case for hot dbPaths (where finalizeLbugSidecarsAfterClose rarely fires). Scope shrunk to dedup + counter-based — strictly behavior-changing and fully testable. Finding 4 — dedup + version-coupling markers - `isReadOnlyShadowReplayError` was inlined in both `lbug-adapter.ts:451` and `pool-adapter.ts:317`. Centralized as an export from `sidecar-recovery.ts`. The two local copies are removed; both adapters now import from the shared module. - Both LadybugDB-coupled predicates (`isMissingShadowSidecarError` and `isReadOnlyShadowReplayError`) gain a `// LADYBUGDB-CONTRACT:` marker comment citing `@ladybugdb/core ^0.16.1`. When bumping LadybugDB, `git grep "LADYBUGDB-CONTRACT"` enumerates every version-coupled spot. - Strict matcher unchanged — when LadybugDB actually changes the error format, the failure mode stays loud (raw native error propagates) and the markers make every affected predicate trivially greppable. Finding 6 — counter-based warn anti-spam - `warnedKeys: Set<string>` → `warnedKeyCounts: Map<string, number>`. `warnOnce` keeps its signature `(logger, key, message)` and keying convention unchanged — the swap is internal. - `WARN_MILESTONES = [1, 10, 100, 1000, 10000]`. Logarithmic spacing gives O(log N) warns for a condition that fires N times. Past the first occurrence the warn message is suffixed with "(Nth occurrence of this condition)" so persistence is visible in the log line itself. - Solves the long-running serve case: a hot dbPath hitting the same condition 100 times now fires 3 warns (occurrences 1, 10, 100) instead of 1 warn + 99 silent debug lines. Tests (10 new in sidecar-recovery.test.ts, all green) - Centralized isReadOnlyShadowReplayError: positive match, false-positive guard, structural assertion that the duplicate regex is gone from both adapter files, LADYBUGDB-CONTRACT marker count. - Counter-based warnOnce: milestone-at-10 with suffix, milestone-at-100, key isolation across dbPaths, reset zeroes the counter, first-occurrence message does NOT carry the suffix. Deferred (tracked separately) - Finding 5 — PNA header end-to-end coverage gap (CORS boundary is sound). - LadybugDB structured error codes (if/when the library exposes them). - Per-call milestone configurability — re-open if tuning is needed. * chore(autofix): apply prettier + eslint fixes via /autofix command * ci: trigger CI rebuild --------- Co-authored-by: wangxc <wangxc_a_bj@si-tech.com.cn> Co-authored-by: OmX <omx@oh-my-codex.dev> Co-authored-by: Gergő Magyar <gergomagyar@icloud.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
341 lines
12 KiB
TypeScript
341 lines
12 KiB
TypeScript
import path from 'node:path';
|
|
import http from 'node:http';
|
|
import { readFileSync } from 'node:fs';
|
|
import express from 'express';
|
|
import { describe, expect, it, vi, beforeEach } from 'vitest';
|
|
import { _captureLogger } from '../../src/core/logger.js';
|
|
|
|
const { accessMock } = vi.hoisted(() => ({
|
|
accessMock: vi.fn(),
|
|
}));
|
|
|
|
vi.mock('fs/promises', () => ({
|
|
default: { access: accessMock },
|
|
access: accessMock,
|
|
}));
|
|
|
|
import {
|
|
registerWebUI,
|
|
resolveWebDistDir,
|
|
landingPageHtml,
|
|
SPA_FALLBACK_REGEX,
|
|
staticCacheControlSetHeaders,
|
|
} from '../../src/server/api.js';
|
|
|
|
type MockRoute = { method: string; path: string | RegExp; handler: Function[] };
|
|
type MockApp = {
|
|
use: ReturnType<typeof vi.fn>;
|
|
get: ReturnType<typeof vi.fn>;
|
|
_routes: MockRoute[];
|
|
};
|
|
|
|
const createMockApp = (): MockApp => {
|
|
const _routes: MockRoute[] = [];
|
|
return {
|
|
use: vi.fn(),
|
|
get: vi.fn((p: string | RegExp, ...h: Function[]) =>
|
|
_routes.push({ method: 'get', path: p, handler: h }),
|
|
),
|
|
_routes,
|
|
};
|
|
};
|
|
|
|
const invokeHandler = async (app: MockApp, method: string, reqPath: string) => {
|
|
for (const route of app._routes) {
|
|
if (route.method !== method) continue;
|
|
if (route.path instanceof RegExp) {
|
|
if (!route.path.test(reqPath)) continue;
|
|
} else {
|
|
if (route.path !== reqPath) continue;
|
|
}
|
|
const res: any = {
|
|
sendFile: vi.fn(),
|
|
type: vi.fn().mockReturnThis(),
|
|
send: vi.fn().mockReturnThis(),
|
|
setHeader: vi.fn(),
|
|
};
|
|
await route.handler[0]({ path: reqPath } as any, res, vi.fn());
|
|
return res;
|
|
}
|
|
return null;
|
|
};
|
|
|
|
describe('landingPageHtml', () => {
|
|
const html = landingPageHtml();
|
|
|
|
it('contains void background colour from gitnexus-web design tokens', () => {
|
|
expect(html).toContain('#06060a');
|
|
});
|
|
|
|
it('contains surface card colour from gitnexus-web design tokens', () => {
|
|
expect(html).toContain('#101018');
|
|
});
|
|
|
|
it('contains accent colour from gitnexus-web design tokens', () => {
|
|
expect(html).toContain('#7c3aed');
|
|
});
|
|
|
|
it('uses Outfit font with system-ui fallback', () => {
|
|
expect(html).toContain('Outfit');
|
|
expect(html).toContain('system-ui');
|
|
});
|
|
|
|
it('contains the build command in a terminal-style block', () => {
|
|
expect(html).toContain('cd gitnexus-web');
|
|
expect(html).toContain('npm run build');
|
|
});
|
|
|
|
it('contains the Vercel link with safe external attributes', () => {
|
|
expect(html).toContain('https://gitnexus.vercel.app');
|
|
expect(html).toContain('target="_blank"');
|
|
expect(html).toContain('rel="noopener noreferrer"');
|
|
});
|
|
|
|
it('contains the Web UI not found message', () => {
|
|
expect(html).toContain('Web UI not found');
|
|
});
|
|
});
|
|
|
|
describe('SPA fallback regex', () => {
|
|
it('allows root path', () => {
|
|
expect(SPA_FALLBACK_REGEX.test('/')).toBe(true);
|
|
});
|
|
|
|
it('allows SPA routes', () => {
|
|
expect(SPA_FALLBACK_REGEX.test('/processes')).toBe(true);
|
|
expect(SPA_FALLBACK_REGEX.test('/settings')).toBe(true);
|
|
expect(SPA_FALLBACK_REGEX.test('/clusters')).toBe(true);
|
|
});
|
|
|
|
it('excludes /api paths', () => {
|
|
expect(SPA_FALLBACK_REGEX.test('/api')).toBe(false);
|
|
expect(SPA_FALLBACK_REGEX.test('/api/')).toBe(false);
|
|
expect(SPA_FALLBACK_REGEX.test('/api/info')).toBe(false);
|
|
expect(SPA_FALLBACK_REGEX.test('/api/does-not-exist')).toBe(false);
|
|
});
|
|
|
|
it('excludes asset-like paths with file extensions', () => {
|
|
expect(SPA_FALLBACK_REGEX.test('/assets/missing.js')).toBe(false);
|
|
expect(SPA_FALLBACK_REGEX.test('/assets/missing.css')).toBe(false);
|
|
expect(SPA_FALLBACK_REGEX.test('/favicon.ico')).toBe(false);
|
|
expect(SPA_FALLBACK_REGEX.test('/assets/font.woff2')).toBe(false);
|
|
expect(SPA_FALLBACK_REGEX.test('/static/app.map')).toBe(false);
|
|
expect(SPA_FALLBACK_REGEX.test('/images/logo.png')).toBe(false);
|
|
});
|
|
|
|
it('allows SPA routes with dots not at the end', () => {
|
|
expect(SPA_FALLBACK_REGEX.test('/v1.0/api')).toBe(true);
|
|
expect(SPA_FALLBACK_REGEX.test('/docs/v2.0/guide')).toBe(true);
|
|
});
|
|
|
|
it('excludes paths ending in dot-plus-extension regardless of content before it', () => {
|
|
expect(SPA_FALLBACK_REGEX.test('/user@example.com')).toBe(false);
|
|
});
|
|
});
|
|
|
|
describe('registerWebUI', () => {
|
|
it('registers express.static and SPA fallback when staticDir provided', () => {
|
|
const app = createMockApp();
|
|
registerWebUI(app as any, '/some/dir');
|
|
expect(app.use).toHaveBeenCalledTimes(1);
|
|
expect(app.get).toHaveBeenCalledTimes(1);
|
|
const [regex] = app.get.mock.calls[0] as [RegExp, ...Function[]];
|
|
expect(regex.source).toBe(SPA_FALLBACK_REGEX.source);
|
|
});
|
|
|
|
it('registers landing page route when staticDir is null', () => {
|
|
const app = createMockApp();
|
|
registerWebUI(app as any, null);
|
|
expect(app.use).not.toHaveBeenCalled();
|
|
expect(app.get).toHaveBeenCalledTimes(1);
|
|
const [path] = app.get.mock.calls[0] as [string, ...Function[]];
|
|
expect(path).toBe('/');
|
|
});
|
|
|
|
it('landing page handler returns styled HTML', async () => {
|
|
const app = createMockApp();
|
|
registerWebUI(app as any, null);
|
|
const res = await invokeHandler(app, 'get', '/');
|
|
expect(res.type).toHaveBeenCalledWith('html');
|
|
expect(res.send).toHaveBeenCalledWith(expect.stringContaining('Web UI not found'));
|
|
expect(res.send).toHaveBeenCalledWith(expect.stringContaining('#06060a'));
|
|
expect(res.send).toHaveBeenCalledWith(expect.stringContaining('#7c3aed'));
|
|
});
|
|
|
|
it('Cache-Control setHeaders sets no-cache for HTML, immutable for assets', () => {
|
|
const captureHeaders = (filePath: string) => {
|
|
const headers: Record<string, string> = {};
|
|
const res = {
|
|
setHeader: (k: string, v: string) => {
|
|
headers[k] = v;
|
|
},
|
|
};
|
|
staticCacheControlSetHeaders(res as express.Response, filePath);
|
|
return headers;
|
|
};
|
|
expect(captureHeaders('index.html')).toEqual({ 'Cache-Control': 'no-cache' });
|
|
expect(captureHeaders('app.js')).toEqual({
|
|
'Cache-Control': 'public, max-age=31536000, immutable',
|
|
});
|
|
expect(captureHeaders('style.css')).toEqual({
|
|
'Cache-Control': 'public, max-age=31536000, immutable',
|
|
});
|
|
});
|
|
});
|
|
|
|
describe('resolveWebDistDir', () => {
|
|
beforeEach(() => {
|
|
vi.clearAllMocks();
|
|
});
|
|
|
|
it('returns primary dir when index.html exists', async () => {
|
|
accessMock.mockImplementation(async (p: string) => {
|
|
if (p.includes('primary')) return undefined;
|
|
throw Object.assign(new Error('not found'), { code: 'ENOENT' });
|
|
});
|
|
const result = await resolveWebDistDir('/primary', '/fallback');
|
|
expect(result).toBe('/primary');
|
|
});
|
|
|
|
it('returns fallback dir when primary missing', async () => {
|
|
accessMock.mockImplementation(async (p: string) => {
|
|
if (p.includes('fallback')) return undefined;
|
|
throw Object.assign(new Error('not found'), { code: 'ENOENT' });
|
|
});
|
|
const result = await resolveWebDistDir('/primary', '/fallback');
|
|
expect(result).toBe('/fallback');
|
|
});
|
|
|
|
it('returns null when both dirs missing', async () => {
|
|
accessMock.mockImplementation(async () => {
|
|
throw Object.assign(new Error('not found'), { code: 'ENOENT' });
|
|
});
|
|
const result = await resolveWebDistDir('/primary', '/fallback');
|
|
expect(result).toBeNull();
|
|
});
|
|
|
|
it('warns on non-ENOENT errors but continues', async () => {
|
|
const cap = _captureLogger();
|
|
accessMock.mockImplementation(async (p: string) => {
|
|
if (p.includes('primary'))
|
|
throw Object.assign(new Error('permission denied'), { code: 'EACCES' });
|
|
if (p.includes('fallback')) return undefined;
|
|
throw Object.assign(new Error('not found'), { code: 'ENOENT' });
|
|
});
|
|
const result = await resolveWebDistDir('/primary', '/fallback');
|
|
expect(result).toBe('/fallback');
|
|
expect(
|
|
cap
|
|
.records()
|
|
.some(
|
|
(r) =>
|
|
String(r.msg ?? '').includes('could not access web UI dir /primary') &&
|
|
r.err === 'permission denied',
|
|
),
|
|
).toBe(true);
|
|
cap.restore();
|
|
});
|
|
|
|
it('prefers GITNEXUS_WEB_DIST env var when set', async () => {
|
|
const original = process.env.GITNEXUS_WEB_DIST;
|
|
process.env.GITNEXUS_WEB_DIST = '/env/dist';
|
|
try {
|
|
accessMock.mockImplementation(async (p: string) => {
|
|
const normalized = p.split(path.sep).join('/');
|
|
if (normalized.includes('/env/dist')) return undefined;
|
|
throw Object.assign(new Error('not found'), { code: 'ENOENT' });
|
|
});
|
|
const result = await resolveWebDistDir('/primary', '/fallback');
|
|
expect(result).toBe('/env/dist');
|
|
} finally {
|
|
if (original === undefined) {
|
|
delete process.env.GITNEXUS_WEB_DIST;
|
|
} else {
|
|
process.env.GITNEXUS_WEB_DIST = original;
|
|
}
|
|
}
|
|
});
|
|
|
|
it('falls back to primary when GITNEXUS_WEB_DIST dir missing', async () => {
|
|
const original = process.env.GITNEXUS_WEB_DIST;
|
|
process.env.GITNEXUS_WEB_DIST = '/env/dist';
|
|
try {
|
|
accessMock.mockImplementation(async (p: string) => {
|
|
const normalized = p.split(path.sep).join('/');
|
|
if (normalized.includes('/env/dist'))
|
|
throw Object.assign(new Error('not found'), { code: 'ENOENT' });
|
|
if (normalized.includes('/primary')) return undefined;
|
|
throw Object.assign(new Error('not found'), { code: 'ENOENT' });
|
|
});
|
|
const result = await resolveWebDistDir('/primary', '/fallback');
|
|
expect(result).toBe('/primary');
|
|
} finally {
|
|
if (original === undefined) {
|
|
delete process.env.GITNEXUS_WEB_DIST;
|
|
} else {
|
|
process.env.GITNEXUS_WEB_DIST = original;
|
|
}
|
|
}
|
|
});
|
|
});
|
|
|
|
describe('Real Express dispatch — API and asset isolation', () => {
|
|
const makeRequest = (app: express.Express, method: string, url: string): Promise<number> => {
|
|
return new Promise((resolve) => {
|
|
const server = app.listen(0, () => {
|
|
const opts = {
|
|
hostname: 'localhost',
|
|
port: (server.address() as any).port,
|
|
method,
|
|
path: url,
|
|
};
|
|
const req = http.request(opts, (res) => {
|
|
server.close();
|
|
resolve(res.statusCode ?? 0);
|
|
});
|
|
req.on('error', () => {
|
|
server.close();
|
|
resolve(0);
|
|
});
|
|
req.end();
|
|
});
|
|
});
|
|
};
|
|
|
|
it('GET /api/does-not-exist returns 404, not SPA HTML', async () => {
|
|
const app = express();
|
|
app.get('/api/info', (_req, res) => res.json({ ok: true }));
|
|
registerWebUI(app, '/nonexistent');
|
|
const status = await makeRequest(app, 'GET', '/api/does-not-exist');
|
|
expect(status).toBe(404);
|
|
});
|
|
|
|
it('GET /assets/missing.js returns 404, not SPA HTML', async () => {
|
|
const app = express();
|
|
app.get('/api/info', (_req, res) => res.json({ ok: true }));
|
|
registerWebUI(app, '/nonexistent');
|
|
const status = await makeRequest(app, 'GET', '/assets/missing.js');
|
|
expect(status).toBe(404);
|
|
});
|
|
|
|
it('GET / returns the landing page when no web build exists', async () => {
|
|
const app = express();
|
|
registerWebUI(app, null);
|
|
const status = await makeRequest(app, 'GET', '/');
|
|
expect(status).toBe(200);
|
|
});
|
|
|
|
it('does not register a legacy "*" OPTIONS route (Express 5 startup crash regression guard)', async () => {
|
|
// The original PR #1747 startup crash was `app.options('*', ...)` throwing
|
|
// under Express 5's stricter path parser. The fix on main is to NOT register
|
|
// any explicit OPTIONS route — cors() handles preflights automatically and
|
|
// the Access-Control-Allow-Private-Network header is set by global middleware
|
|
// before cors. This regression guard fails loudly if someone re-adds a legacy
|
|
// wildcard route to api.ts.
|
|
const apiSource = readFileSync(
|
|
path.join(__dirname, '..', '..', 'src', 'server', 'api.ts'),
|
|
'utf-8',
|
|
);
|
|
expect(apiSource).not.toMatch(/app\.options\(\s*['"`]\*['"`]/);
|
|
});
|
|
});
|