mirror of
https://github.com/abhigyanpatwari/GitNexus.git
synced 2026-10-01 02:01:24 +00:00
* fix(lbug): pin Ladybug core so Dependabot cannot ship a skewed FTS artifact The extension version is a separate upstream constant. Ignore daily core bumps and fail the pairing gate when the committed manifest does not name the installed core. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): make doctor and CI FTS gates resolve the packaged artifact Doctor and the REQUIRE_FTS file gates still treated an empty ~/.lbdb as unavailable, which would turn three CI jobs red once analyze stops installing into that tree. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): name native-abort and tuple-missing so analyze cannot mis-advise The CLI summary's trailing else treated every unknown skip reason as a missing extension. New crash and platform causes must get their own remedies, not a network-install hint. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): delete the dead read-path FTS index create ensureFTSIndex had no production callers and swallowed read-only CREATE_FTS_INDEX failures, which hid the only signal that a reader tried to write. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): vendor per-platform FTS artifacts so analyze needs no host install Keyword search depended on a CDN fetch into ~/.lbdb. Shipping the five published tuples inside the package makes air-gapped and ignore-scripts installs load the same artifact the publish gate checksums. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): load the packaged FTS artifact before any network install Analyze still required a CDN fetch into ~/.lbdb even when the package already shipped the file. FTS now path-loads the vendored tuple first and records source labels so a later truncated home copy cannot steal the diagnosis. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): diagnose a core/extension version skew instead of a missing runtime A structurally valid FTS artifact whose path version disagrees with the packaged pin must name both versions, not prescribe VC++ or OpenSSL. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): stamp an FTS phase so repair stays usable after an in-place abort A native CREATE_FTS_INDEX abort leaves no skip reason; the next run infers it from the dirty flag, and --repair-fts must not treat that phase as a half-written graph. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): park an in-place FTS crash WAL without wiping the graph An FTS abort after a successful checkpoint must reopen the live index on macOS, Windows, and Linux. Staging never parks the live WAL; readers keep today's large-WAL refusal. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): refuse read-only opens of an FTS-poisoned WAL MCP and serve cannot repair a leftover in-place abort. Fail before the native open and name --repair-fts, on macOS, Windows, and Linux. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): name a vendor-neutral Windows OpenSSL prerequisite OQ1 is unanswered here so GitNexus does not ship OpenSSL DLLs. Windows FTS now asks for a system OpenSSL 3 runtime instead of Git Bash PATH. Co-authored-by: Cursor <cursoragent@cursor.com> * test(lbug): inject the FTS vendor root and redact it on HTTP and MCP Path-loaded artifacts no longer vary with HOME. Tests pass an injected vendor tree and assert search warnings never leak a filesystem path. Co-authored-by: Cursor <cursoragent@cursor.com> * docs(lbug): document load-only as the global FTS install default Analyze still overrides to auto. Packaged per-platform artifacts load before any network install on macOS, Windows, and Linux. Co-authored-by: Cursor <cursoragent@cursor.com> * docs(lbug): format the FTS install-policy README table Prettier does not run on Markdown in pre-commit, so the U10 table wrap needs its own formatting commit. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): skip FTS CREATE after a persisted native abort A recovered analyze run was retrying CREATE_FTS_INDEX from skipReason alone. Keep that skip until --repair-fts, fail closed on unsupported tuples, and honor the checkpoint warrant for park/repair. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): honor checkpoint flushed warrant and align FTS tests with packaged vendor A no-op CHECKPOINT must not satisfy the FTS park warrant, and CI still asserted HOME-only FTS isolation after analyze started path-LOADing the packaged artifact. Co-authored-by: Cursor <cursoragent@cursor.com> * test(lbug): accept a nonempty incremental write set in the #2790 recovery check FTS-phase recovery can incremental-add files (changed=0, added=1). That is not the #2790 empty-diff wipe skip. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): compare FTS home versions to the core pin and tighten the publish filename gate Ladybug's ~/.lbdb/extension directory is the runtime/core version; treating it as the artifact version false-diagnosed skew. The publish guard now rejects a path-escaping filename the same way the fetch script does. Co-authored-by: Cursor <cursoragent@cursor.com> * test(lbug): seed FTS e2e fixtures from the packaged vendor artifact A machine with no ~/.lbdb copy should still run the vendor-survivorship cases; the seed no longer depends on HOME or a network install. Co-authored-by: Cursor <cursoragent@cursor.com> * Address PR review feedback (#3274) Keep in-place FTS abort evidence after persist so a second CREATE abort cannot fail-open readers, and close the CLI, loader, embed, and e2e gaps the review called out. Note: full npm test hit Ladybug worker-pool startup failures under memory pressure; tsc and 180 targeted unit tests passed. Co-authored-by: Cursor <cursoragent@cursor.com> * Address PR review feedback (#3274) Run the vendored-path symlink guard on the OS matrix, put e2e HOME fixtures on Ladybug's real extension layout, pin the embed crash-WAL gate before the writable open, and let analyze writers park through missing-shadow recovery. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): keep --repair-fts CI green after vendored-first FTS Never-installed warning fixtures must not inspect a packaged vendor binary, and a failed dirty restamp must not abort an otherwise successful --repair-fts run. Co-authored-by: Cursor <cursoragent@cursor.com> * test(cli): give the #1169 analyze e2e the same 90s Windows budget as its sibling The first #1169 persist-meta case was still on a 60s spawn/it budget and was killed banner-only on windows-latest after the FTS warning fixture no longer failed the shard first. Co-authored-by: Cursor <cursoragent@cursor.com> * test(ci): reweight Windows shards after the FTS e2e grew Vendored-first HOME fixtures pushed fts-extension-e2e to ~6 minutes on windows-latest, so the old 146s weight packed it with skills-e2e and blew the 20-minute watchdog. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com>
122 lines
5.9 KiB
TypeScript
122 lines
5.9 KiB
TypeScript
import { existsSync, readdirSync, statSync } from 'node:fs';
|
|
import { homedir } from 'node:os';
|
|
import { join } from 'node:path';
|
|
import { resolveVendoredFtsPath } from '../../src/core/lbug/vendored-extension-path.js';
|
|
|
|
/** A valid `libfts.lbug_extension` is ~2.2MB; anything smaller is truncated/corrupt. */
|
|
const MIN_VALID_FTS_EXTENSION_BYTES = 1024 * 1024;
|
|
|
|
/**
|
|
* Find the installed FTS extension file under a `.lbdb/extension` root,
|
|
* discovering the version directory instead of assuming it equals the npm
|
|
* `@ladybugdb/core` package version. LadybugDB's native INSTALL/LOAD resolves
|
|
* its own extension-ABI version directory, which does not always track the
|
|
* npm package version — e.g. #2587: bumping the package from 0.18.1 to 0.18.2
|
|
* still installs into a `0.18.1` directory, because the underlying
|
|
* extension-ABI build did not change with that patch release.
|
|
*
|
|
* Scans every version subdirectory for a `<platform>/fts/libfts.lbug_extension`
|
|
* file and returns the most recently modified one (the one an install/load
|
|
* actually just resolved), or null when nothing is installed.
|
|
*/
|
|
export const findInstalledFtsExtension = (extensionRoot: string): string | null => {
|
|
// Fail closed on any FS error (permission quirks, AV file locks on Windows,
|
|
// a directory vanishing mid-scan) — same contract as the callers this
|
|
// replaces: "not found" is a valid outcome, a thrown exception is not.
|
|
try {
|
|
if (!existsSync(extensionRoot)) return null;
|
|
let best: { path: string; mtimeMs: number } | null = null;
|
|
for (const versionEntry of readdirSync(extensionRoot)) {
|
|
const versionDir = join(extensionRoot, versionEntry);
|
|
if (!statSync(versionDir).isDirectory()) continue;
|
|
for (const platformEntry of readdirSync(versionDir)) {
|
|
const candidate = join(versionDir, platformEntry, 'fts', 'libfts.lbug_extension');
|
|
if (!existsSync(candidate)) continue;
|
|
const stat = statSync(candidate);
|
|
if (stat.size < MIN_VALID_FTS_EXTENSION_BYTES) continue;
|
|
if (!best || stat.mtimeMs > best.mtimeMs) best = { path: candidate, mtimeMs: stat.mtimeMs };
|
|
}
|
|
}
|
|
return best?.path ?? null;
|
|
} catch {
|
|
return null;
|
|
}
|
|
};
|
|
|
|
/**
|
|
* Resolve the FTS extension the same way doctor/analyze will after vendoring:
|
|
* packaged artifact first, then a `~/.lbdb` install. File-path CI gates must
|
|
* accept the vendored file so they stay green when ensure-fts no longer
|
|
* populates the home cache.
|
|
*/
|
|
export const resolveFtsExtension = (opts?: {
|
|
vendorRoot?: string;
|
|
homeExtensionRoot?: string;
|
|
}): string | null =>
|
|
resolveVendoredFtsPath({ vendorRoot: opts?.vendorRoot }) ??
|
|
findInstalledFtsExtension(opts?.homeExtensionRoot ?? join(homedir(), '.lbdb', 'extension'));
|
|
|
|
export const FTS_UNAVAILABLE_NOTE =
|
|
'FTS extension unavailable (load-only policy; LOAD failed on this machine)';
|
|
|
|
/**
|
|
* Dynamically skip an FTS-primitive test when the extension cannot load.
|
|
* `ctx.skip()` aborts the test, so callers should `await` this first thing.
|
|
*
|
|
* Honors GITNEXUS_REQUIRE_FTS=1 the same way `withTestLbugDB` does (see
|
|
* test/helpers/test-indexed-db.ts): when CI sets it, an unavailable extension is
|
|
* a HARD FAILURE, never a silent skip — otherwise these FTS-primitive tests
|
|
* (registered in LBUG_NATIVE, so they run on the ubuntu/macOS/windows jobs that
|
|
* all set GITNEXUS_REQUIRE_FTS=1) could vanish from a green run. Offline/local
|
|
* runs (no env var) still skip gracefully (#2299).
|
|
*
|
|
* Self-sufficient under sharding: the default load path is `load-only`, so these
|
|
* primitives only pass when *some other* test already installed FTS into the
|
|
* shared home. That co-location is not guaranteed once the cross-platform suite
|
|
* is sharded (a load-only file can land in a shard with no installer sibling —
|
|
* exactly what broke `lbug-core-adapter` on shard 2/3). So under REQUIRE_FTS we
|
|
* install-on-miss with `auto` (LOAD-first, then one bounded network INSTALL),
|
|
* matching `withTestIndexedDB`, before treating it as a hard failure.
|
|
*/
|
|
export const skipUnlessFtsAvailable = async (ctx: {
|
|
skip: (note?: string) => void;
|
|
}): Promise<void> => {
|
|
const { loadFTSExtension } = await import('../../src/core/lbug/lbug-adapter.js');
|
|
if (await loadFTSExtension()) return;
|
|
if (process.env.GITNEXUS_REQUIRE_FTS === '1') {
|
|
// Not pre-installed in this (possibly-sharded) CI VM — install it once, then
|
|
// it stays available for the rest of this file's tests. `auto` is LOAD-first
|
|
// so a pre-installed extension still costs no network.
|
|
if (await loadFTSExtension(undefined, { policy: 'auto' })) return;
|
|
throw new Error(
|
|
'FTS extension is required (GITNEXUS_REQUIRE_FTS=1) but could not be loaded or installed. ' +
|
|
'FTS-dependent tests must not be silently skipped in CI — install/repair the LadybugDB ' +
|
|
'FTS extension (see `gitnexus doctor`) or unset GITNEXUS_REQUIRE_FTS for offline/local runs.',
|
|
);
|
|
}
|
|
ctx.skip(FTS_UNAVAILABLE_NOTE);
|
|
};
|
|
|
|
/**
|
|
* Skip a structural FTS test when a required on-disk artifact (the vendored
|
|
* extension, a home install, or the native addon) is not resolvable — but
|
|
* HARD-FAIL under GITNEXUS_REQUIRE_FTS=1 (#2299, #2383 F6d) so it never
|
|
* silently vanishes from a green CI run. Used by tests that inspect the
|
|
* extension *file* directly and so need its path rather than a loaded
|
|
* connection (skipUnlessFtsAvailable needs an initialized LadybugDB).
|
|
*/
|
|
export const requireFtsResourceOrSkip = (
|
|
ctx: { skip: (note?: string) => void },
|
|
resource: string | null,
|
|
note: string,
|
|
): void => {
|
|
if (resource) return;
|
|
if (process.env.GITNEXUS_REQUIRE_FTS === '1') {
|
|
throw new Error(
|
|
`${note} is required (GITNEXUS_REQUIRE_FTS=1) but was not found on this machine. ` +
|
|
'FTS-dependent tests must not be silently skipped in CI — install/repair the LadybugDB ' +
|
|
'FTS extension (see `gitnexus doctor`) or unset GITNEXUS_REQUIRE_FTS for offline/local runs.',
|
|
);
|
|
}
|
|
ctx.skip(`${note} unavailable`);
|
|
};
|