GitNexus/gitnexus/test/unit/lbug-pool-storage-version-lock-retry.test.ts
Gergő Magyar 21a52af1d4
fix(lbug): ship FTS per-platform and recover in-place native aborts (#3274)
* fix(lbug): pin Ladybug core so Dependabot cannot ship a skewed FTS artifact

The extension version is a separate upstream constant. Ignore daily core bumps and fail the pairing gate when the committed manifest does not name the installed core.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(lbug): make doctor and CI FTS gates resolve the packaged artifact

Doctor and the REQUIRE_FTS file gates still treated an empty ~/.lbdb as
unavailable, which would turn three CI jobs red once analyze stops
installing into that tree.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(lbug): name native-abort and tuple-missing so analyze cannot mis-advise

The CLI summary's trailing else treated every unknown skip reason as a
missing extension. New crash and platform causes must get their own
remedies, not a network-install hint.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(lbug): delete the dead read-path FTS index create

ensureFTSIndex had no production callers and swallowed read-only
CREATE_FTS_INDEX failures, which hid the only signal that a reader
tried to write.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(lbug): vendor per-platform FTS artifacts so analyze needs no host install

Keyword search depended on a CDN fetch into ~/.lbdb. Shipping the five
published tuples inside the package makes air-gapped and ignore-scripts
installs load the same artifact the publish gate checksums.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(lbug): load the packaged FTS artifact before any network install

Analyze still required a CDN fetch into ~/.lbdb even when the package
already shipped the file. FTS now path-loads the vendored tuple first
and records source labels so a later truncated home copy cannot steal
the diagnosis.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(lbug): diagnose a core/extension version skew instead of a missing runtime

A structurally valid FTS artifact whose path version disagrees with the
packaged pin must name both versions, not prescribe VC++ or OpenSSL.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(lbug): stamp an FTS phase so repair stays usable after an in-place abort

A native CREATE_FTS_INDEX abort leaves no skip reason; the next run infers
it from the dirty flag, and --repair-fts must not treat that phase as a
half-written graph.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(lbug): park an in-place FTS crash WAL without wiping the graph

An FTS abort after a successful checkpoint must reopen the live index on
macOS, Windows, and Linux. Staging never parks the live WAL; readers keep
today's large-WAL refusal.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(lbug): refuse read-only opens of an FTS-poisoned WAL

MCP and serve cannot repair a leftover in-place abort. Fail before the
native open and name --repair-fts, on macOS, Windows, and Linux.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(lbug): name a vendor-neutral Windows OpenSSL prerequisite

OQ1 is unanswered here so GitNexus does not ship OpenSSL DLLs. Windows
FTS now asks for a system OpenSSL 3 runtime instead of Git Bash PATH.

Co-authored-by: Cursor <cursoragent@cursor.com>

* test(lbug): inject the FTS vendor root and redact it on HTTP and MCP

Path-loaded artifacts no longer vary with HOME. Tests pass an injected
vendor tree and assert search warnings never leak a filesystem path.

Co-authored-by: Cursor <cursoragent@cursor.com>

* docs(lbug): document load-only as the global FTS install default

Analyze still overrides to auto. Packaged per-platform artifacts load
before any network install on macOS, Windows, and Linux.

Co-authored-by: Cursor <cursoragent@cursor.com>

* docs(lbug): format the FTS install-policy README table

Prettier does not run on Markdown in pre-commit, so the U10 table wrap
needs its own formatting commit.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(lbug): skip FTS CREATE after a persisted native abort

A recovered analyze run was retrying CREATE_FTS_INDEX from skipReason
alone. Keep that skip until --repair-fts, fail closed on unsupported
tuples, and honor the checkpoint warrant for park/repair.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(lbug): honor checkpoint flushed warrant and align FTS tests with packaged vendor

A no-op CHECKPOINT must not satisfy the FTS park warrant, and CI still asserted HOME-only FTS isolation after analyze started path-LOADing the packaged artifact.

Co-authored-by: Cursor <cursoragent@cursor.com>

* test(lbug): accept a nonempty incremental write set in the #2790 recovery check

FTS-phase recovery can incremental-add files (changed=0, added=1). That is not the #2790 empty-diff wipe skip.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(lbug): compare FTS home versions to the core pin and tighten the publish filename gate

Ladybug's ~/.lbdb/extension directory is the runtime/core version; treating it as the artifact version false-diagnosed skew. The publish guard now rejects a path-escaping filename the same way the fetch script does.

Co-authored-by: Cursor <cursoragent@cursor.com>

* test(lbug): seed FTS e2e fixtures from the packaged vendor artifact

A machine with no ~/.lbdb copy should still run the vendor-survivorship cases; the seed no longer depends on HOME or a network install.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Address PR review feedback (#3274)

Keep in-place FTS abort evidence after persist so a second CREATE abort
cannot fail-open readers, and close the CLI, loader, embed, and e2e gaps
the review called out.

Note: full npm test hit Ladybug worker-pool startup failures under memory
pressure; tsc and 180 targeted unit tests passed.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Address PR review feedback (#3274)

Run the vendored-path symlink guard on the OS matrix, put e2e HOME
fixtures on Ladybug's real extension layout, pin the embed crash-WAL
gate before the writable open, and let analyze writers park through
missing-shadow recovery.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(lbug): keep --repair-fts CI green after vendored-first FTS

Never-installed warning fixtures must not inspect a packaged vendor binary, and a failed dirty restamp must not abort an otherwise successful --repair-fts run.

Co-authored-by: Cursor <cursoragent@cursor.com>

* test(cli): give the #1169 analyze e2e the same 90s Windows budget as its sibling

The first #1169 persist-meta case was still on a 60s spawn/it budget and was killed banner-only on windows-latest after the FTS warning fixture no longer failed the shard first.

Co-authored-by: Cursor <cursoragent@cursor.com>

* test(ci): reweight Windows shards after the FTS e2e grew

Vendored-first HOME fixtures pushed fts-extension-e2e to ~6 minutes on windows-latest, so the old 146s weight packed it with skills-e2e and blew the 20-minute watchdog.

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-14 08:52:24 +01:00

157 lines
5.4 KiB
TypeScript

import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import * as os from 'node:os';
import * as path from 'node:path';
import { mkdtempSync, writeFileSync, rmSync } from 'node:fs';
const { loadFTSExtensionMock, loadVectorExtensionMock, createLbugDatabaseMock } = vi.hoisted(
() => ({
loadFTSExtensionMock: vi.fn(),
loadVectorExtensionMock: vi.fn().mockResolvedValue(false),
createLbugDatabaseMock: vi.fn(),
}),
);
vi.mock('@ladybugdb/core', () => ({
default: {
Database: vi.fn(),
Connection: vi.fn(function (this: any) {
this.query = vi.fn().mockResolvedValue({
getAll: vi.fn().mockResolvedValue([]),
close: vi.fn(),
});
this.close = vi.fn().mockResolvedValue(undefined);
}),
},
}));
vi.mock('../../src/core/lbug/lbug-adapter.js', () => ({
isReadOnlyDbError: vi.fn(() => false),
loadFTSExtension: loadFTSExtensionMock,
loadVectorExtension: loadVectorExtensionMock,
}));
vi.mock('../../src/core/lbug/lbug-config.js', async (importOriginal) => {
const actual = await importOriginal<typeof import('../../src/core/lbug/lbug-config.js')>();
return {
...actual,
createLbugDatabase: createLbugDatabaseMock,
toNativeSafePath: vi.fn((p: string) => p),
isWalCorruptionError: vi.fn(() => false),
WAL_RECOVERY_SUGGESTION: '',
};
});
vi.mock('../../src/core/lbug/sidecar-recovery.js', () => ({
preflightLbugSidecars: vi.fn().mockResolvedValue(undefined),
guardWalQuarantine: vi.fn().mockResolvedValue(undefined),
isMissingFsError: vi.fn(() => false),
isMissingShadowSidecarError: vi.fn(() => false),
isReadOnlyShadowReplayError: vi.fn(() => false),
quarantineWalForMissingShadow: vi.fn().mockResolvedValue(''),
quarantineSidecarsForDirtyRecovery: vi
.fn()
.mockResolvedValue({ moved: [], removed: [], failed: [] }),
renameFailureMessage: vi.fn((p: string) => `rename failed for ${p}`),
statIfExists: vi.fn().mockResolvedValue(null),
assertReadOnlyFtsCrashSafe: vi.fn().mockResolvedValue(undefined),
FtsReaderUnrepairableError: class FtsReaderUnrepairableError extends Error {
readonly code = 'FTS_READER_UNREPAIRABLE' as const;
constructor(dbPath = '') {
super(dbPath);
this.name = 'FtsReaderUnrepairableError';
}
},
}));
const { initLbug, closeLbug, isLbugReady, unpinRepo } =
await import('../../src/core/lbug/pool-adapter.js');
const { STORAGE_VERSION_MISMATCH_SUGGESTION } = await import('../../src/core/lbug/lbug-config.js');
const NATIVE_STORAGE_VERSION_MISMATCH =
'Runtime exception: Trying to read a database file with a different version. Database file version: 43, Current build storage version: 42';
function goodDb() {
return {
init: vi.fn().mockResolvedValue(undefined),
close: vi.fn().mockResolvedValue(undefined),
};
}
describe('pool-adapter storage-version fail-fast and lock-retry backoff', () => {
let tmpDir: string;
const touched = new Set<string>();
const dbPathFor = (repoId: string): string => {
const p = path.join(tmpDir, `${repoId}.lbug`);
writeFileSync(p, '');
return p;
};
beforeEach(() => {
tmpDir = mkdtempSync(path.join(os.tmpdir(), 'gn-pool-sv-lock-'));
loadFTSExtensionMock.mockResolvedValue(true);
createLbugDatabaseMock.mockReset();
createLbugDatabaseMock.mockImplementation(() => goodDb());
});
afterEach(async () => {
vi.useRealTimers();
await closeLbug().catch(() => {});
for (const id of touched) unpinRepo(id);
touched.clear();
loadFTSExtensionMock.mockReset();
rmSync(tmpDir, { recursive: true, force: true });
});
it('fail-fasts a storage-version mismatch with the rebuild hint and does not lock-retry', async () => {
const repoId = 'version-mismatch';
touched.add(repoId);
createLbugDatabaseMock.mockImplementation(() => {
throw new Error(NATIVE_STORAGE_VERSION_MISMATCH);
});
await expect(initLbug(repoId, dbPathFor(repoId))).rejects.toThrow(
STORAGE_VERSION_MISMATCH_SUGGESTION,
);
expect(createLbugDatabaseMock).toHaveBeenCalledTimes(1);
});
it('does not hold withPoolLock across lock-retry sleep — another repo can init during backoff', async () => {
vi.useFakeTimers({ toFake: ['setTimeout', 'clearTimeout'] });
const lockedId = 'locked';
const otherId = 'other';
touched.add(lockedId);
touched.add(otherId);
const lockedPath = dbPathFor(lockedId);
const otherPath = dbPathFor(otherId);
createLbugDatabaseMock.mockImplementation((_mod: unknown, dbPath: string) => {
if (
String(dbPath).includes(`${path.sep}locked.lbug`) ||
String(dbPath).endsWith('locked.lbug')
) {
throw new Error('Could not set lock on file : /tmp/locked.lbug');
}
return goodDb();
});
const lockedInit = initLbug(lockedId, lockedPath);
await vi.waitFor(() => {
expect(createLbugDatabaseMock).toHaveBeenCalledTimes(1);
});
const otherInit = initLbug(otherId, otherPath);
await expect(otherInit).resolves.toBe(true);
expect(isLbugReady(otherId)).toBe(true);
expect(createLbugDatabaseMock).toHaveBeenCalledTimes(2);
await vi.advanceTimersByTimeAsync(2000);
await vi.waitFor(() => {
expect(createLbugDatabaseMock).toHaveBeenCalledTimes(3);
});
await vi.advanceTimersByTimeAsync(4000);
await expect(lockedInit).rejects.toThrow(/LadybugDB unavailable for locked/);
expect(createLbugDatabaseMock).toHaveBeenCalledTimes(4);
});
});