GitNexus/gitnexus/test/unit/analyze-wal-error.test.ts
Copilot 87b91c821e
fix(lbug): add WAL checkpoint-threshold control (#1772)
* Initial plan

* fix(analyze): add WAL auto-checkpoint CLI control and default-off behavior

* test(analyze): share lbug auto-checkpoint parsing and align validation

* fix(analyze): always enable lbug auto-checkpoint and expose threshold control

* refactor(lbug): inline always-on auto-checkpoint constructor arg

* fix(analyze): guide checkpoint-threshold on Ladybug WAL checkpoint IO failures

* test(analyze): cover checkpoint IO guidance and add integration guard

* fix(analyze): tighten checkpoint IO detection and remove test hook

* fix(analyze): remove checkpoint test hook and tighten error matching

* fix(analyze): rename to wal-checkpoint-threshold, raise default, add manual checkpoint driver with retry

Address review feedback on PR #1772:

- Rename CLI flag, env var, AnalyzeOptions field, recovery-hint tag, and
  parser/constants from lbug-* to engine-neutral wal-* (matches the existing
  WAL_RECOVERY_SUGGESTION / isWalCorruptionError convention).
- Raise default threshold from -1 (Ladybug stock ~16 MiB) to 64 MiB so users
  on the default config no longer hit the original rename/remove race.
- Align both READMEs to publish 67108864 (64 MiB) instead of 65536 (which
  would have made the crash more frequent).
- Add wal-checkpoint-driver.ts: a periodic manual CHECKPOINT driver wrapped
  in a 3-attempt jittered retry (50/200/500 ms), driven from runFullAnalysis.
  Opt-out via GITNEXUS_WAL_MANUAL_CHECKPOINT=0. Moves the race window into a
  JS-controllable retry surface while keeping native auto-checkpoint on.
- Move LBUG_CHECKPOINT_RENAME_RE / REMOVE_RE plus the predicate (renamed to
  isLbugCheckpointIoError) into lbug-config.ts alongside isWalCorruptionError.
  Predicate is now exported. Add a permissive fallback matcher and pin the
  matched Ladybug version in comments.
- Warn instead of silently defaulting when GITNEXUS_WAL_CHECKPOINT_THRESHOLD
  is set to a non-empty unparseable value (closes the CLI-vs-env asymmetry).
- Add a typed RecoveryHint string-literal union in cli-message.ts so future
  hint tags can't drift.
- Add a real integration test under test/integration/ that triggers a
  Ladybug checkpoint IO failure via a pre-existing directory at the rename
  target (portable across platforms; no test-only injection hook).
- Add small-disk / CI caveat (32 MiB secondary suggestion) to the recovery
  hint and README env-var rows.
- Document CLI/env precedence in the analyze --help block.
- Help placeholder: <value> -> <bytes>.
- Rename analyze-lbug-auto-checkpoint.test.ts to use the new wal-* token.

* chore(lbug): remove dead jitteredDelay helper and apply prettier

- Drop unused `jitteredDelay` function flagged by CodeQL in PR #1772; the
  retry loop already inlines the same calculation with the injectable
  `randomImpl` so the helper was dead. Move the non-cryptographic-by-design
  comment next to the actual jitter site.
- Apply `prettier --write` to wal-checkpoint-driver.ts and the new
  integration test to absorb the PR autofix bot's formatting findings.

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Gergő Magyar <gergomagyar@icloud.com>
Co-authored-by: Test <test@example.com>
2026-05-22 14:46:49 +01:00

215 lines
7.6 KiB
TypeScript

/**
* Tests for WAL corruption error handling in the `analyzeCommand` CLI.
*
* Before this fix, a WAL corruption error surfaced as a raw stack-trace dump.
* After the fix, it is caught before the generic error path and rendered as
* a clean, actionable message telling the user to run `gitnexus analyze --force`.
*
* Mirrors the test shape of analyze-worker-timeout.test.ts:
* - vi.mock the heavy dependencies so no real DB / git is touched
* - drive `analyzeCommand` with a mocked `runFullAnalysis` that throws
* - assert on process.exitCode and the logged output
*/
import { beforeEach, describe, expect, it, vi } from 'vitest';
const runFullAnalysisMock = vi.fn();
vi.mock('../../src/core/run-analyze.js', () => ({
runFullAnalysis: runFullAnalysisMock,
}));
vi.mock('../../src/core/lbug/lbug-adapter.js', () => ({
closeLbug: vi.fn(async () => undefined),
}));
vi.mock('../../src/storage/repo-manager.js', () => ({
getStoragePaths: vi.fn(() => ({ storagePath: '.gitnexus', lbugPath: '.gitnexus/lbug' })),
getGlobalRegistryPath: vi.fn(() => 'registry.json'),
RegistryNameCollisionError: class RegistryNameCollisionError extends Error {},
AnalysisNotFinalizedError: class AnalysisNotFinalizedError extends Error {},
assertAnalysisFinalized: vi.fn(async () => undefined),
}));
vi.mock('../../src/storage/git.js', () => ({
getGitRoot: vi.fn(() => '/repo'),
hasGitDir: vi.fn(() => true),
}));
vi.mock('../../src/core/ingestion/utils/max-file-size.js', () => ({
getMaxFileSizeBannerMessage: vi.fn(() => null),
}));
// analyze.ts imports isHfDownloadFailure from hf-env.js, which in turn imports
// from gitnexus-shared (not linked in dev). Mock the module to break the chain.
vi.mock('../../src/core/embeddings/hf-env.js', () => ({
isHfDownloadFailure: vi.fn(() => false),
}));
// ─── Tests ───────────────────────────────────────────────────────────────────
describe('analyzeCommand WAL corruption error handling', () => {
beforeEach(() => {
vi.resetModules();
runFullAnalysisMock.mockReset();
process.exitCode = undefined;
// Ensure ensureHeap() short-circuits (heap already at target size)
process.env.NODE_OPTIONS = `${process.env.NODE_OPTIONS ?? ''} --max-old-space-size=8192`.trim();
});
it('surfaces a clean recovery message on a re-wrapped WAL corruption error', async () => {
// This error shape is what lbug-adapter throws after detecting WAL corruption
// in doInitLbug and re-wrapping it with the recovery suggestion.
const walError = new Error(
'LadybugDB WAL corruption detected at /repo/.gitnexus/lbug. ' +
'Run `gitnexus analyze` to rebuild the index.\n' +
' Original error: Runtime exception: Corrupted wal file.',
);
runFullAnalysisMock.mockRejectedValue(walError);
const { _captureLogger } = await import('../../src/core/logger.js');
const cap = _captureLogger();
const { analyzeCommand } = await import('../../src/cli/analyze.js');
await analyzeCommand(undefined, {});
expect(process.exitCode).toBe(1);
const records = cap.records();
const walRecord = records.find(
(r) => typeof r.msg === 'string' && r.msg.includes('gitnexus analyze --force'),
);
expect(walRecord).toBeDefined();
// Raw stack trace must NOT appear via cliError
const stackRecord = records.find(
(r) => typeof r.msg === 'string' && r.msg.includes('at analyzeCommand'),
);
expect(stackRecord).toBeUndefined();
cap.restore();
});
it('surfaces a clean recovery message when the native WAL error fires directly', async () => {
// isWalCorruptionError fires on the native engine message before re-wrapping.
const nativeWalError = new Error(
'Runtime exception: Corrupted wal file. Read out invalid WAL record type.',
);
runFullAnalysisMock.mockRejectedValue(nativeWalError);
const { _captureLogger } = await import('../../src/core/logger.js');
const cap = _captureLogger();
const { analyzeCommand } = await import('../../src/cli/analyze.js');
await analyzeCommand(undefined, {});
expect(process.exitCode).toBe(1);
const records = cap.records();
const walRecord = records.find(
(r) => typeof r.msg === 'string' && r.msg.includes('gitnexus analyze --force'),
);
expect(walRecord).toBeDefined();
cap.restore();
});
it('does NOT route non-WAL errors through the WAL handler', async () => {
const genericError = new Error('Some unexpected failure unrelated to WAL');
runFullAnalysisMock.mockRejectedValue(genericError);
const { _captureLogger } = await import('../../src/core/logger.js');
const cap = _captureLogger();
const { analyzeCommand } = await import('../../src/cli/analyze.js');
await analyzeCommand(undefined, {});
expect(process.exitCode).toBe(1);
// The WAL recovery message must NOT appear for unrelated errors
const records = cap.records();
const walRecord = records.find(
(r) => typeof r.msg === 'string' && r.msg.includes('gitnexus analyze --force'),
);
expect(walRecord).toBeUndefined();
cap.restore();
});
it('recommends --wal-checkpoint-threshold on Ladybug checkpoint I/O failures', async () => {
runFullAnalysisMock.mockRejectedValue(
new Error(
'Runtime exception: IO exception: Error renaming file /repo/.gitnexus/lbug.wal to /repo/.gitnexus/lbug.wal.checkpoint. ErrorMessage: Permission denied',
),
);
const { _captureLogger } = await import('../../src/core/logger.js');
const cap = _captureLogger();
const { analyzeCommand } = await import('../../src/cli/analyze.js');
await analyzeCommand(undefined, {});
expect(process.exitCode).toBe(1);
const records = cap.records();
expect(
records.some(
(r) =>
typeof r.msg === 'string' &&
r.msg.includes('gitnexus analyze --wal-checkpoint-threshold'),
),
).toBe(true);
cap.restore();
});
it('also recommends threshold on .wal.checkpoint remove failures', async () => {
runFullAnalysisMock.mockRejectedValue(
new Error(
'Runtime exception: IO exception: Error removing directory or file /repo/.gitnexus/lbug.wal.checkpoint. Error Message: Permission denied',
),
);
const { _captureLogger } = await import('../../src/core/logger.js');
const cap = _captureLogger();
const { analyzeCommand } = await import('../../src/cli/analyze.js');
await analyzeCommand(undefined, {});
expect(process.exitCode).toBe(1);
const records = cap.records();
expect(
records.some(
(r) =>
typeof r.msg === 'string' &&
r.msg.includes('gitnexus analyze --wal-checkpoint-threshold'),
),
).toBe(true);
cap.restore();
});
it('does not recommend threshold for non-checkpoint IO exceptions', async () => {
runFullAnalysisMock.mockRejectedValue(
new Error(
'Runtime exception: IO exception: Error renaming file /repo/.gitnexus/data.tmp to /repo/.gitnexus/data.tmp.bak. ErrorMessage: Permission denied',
),
);
const { _captureLogger } = await import('../../src/core/logger.js');
const cap = _captureLogger();
const { analyzeCommand } = await import('../../src/cli/analyze.js');
await analyzeCommand(undefined, {});
expect(process.exitCode).toBe(1);
const records = cap.records();
expect(
records.some(
(r) =>
typeof r.msg === 'string' &&
r.msg.includes('gitnexus analyze --wal-checkpoint-threshold'),
),
).toBe(false);
cap.restore();
});
});