mirror of
https://github.com/abhigyanpatwari/GitNexus.git
synced 2026-09-19 00:03:33 +00:00
* fix(lbug): pin Ladybug core so Dependabot cannot ship a skewed FTS artifact The extension version is a separate upstream constant. Ignore daily core bumps and fail the pairing gate when the committed manifest does not name the installed core. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): make doctor and CI FTS gates resolve the packaged artifact Doctor and the REQUIRE_FTS file gates still treated an empty ~/.lbdb as unavailable, which would turn three CI jobs red once analyze stops installing into that tree. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): name native-abort and tuple-missing so analyze cannot mis-advise The CLI summary's trailing else treated every unknown skip reason as a missing extension. New crash and platform causes must get their own remedies, not a network-install hint. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): delete the dead read-path FTS index create ensureFTSIndex had no production callers and swallowed read-only CREATE_FTS_INDEX failures, which hid the only signal that a reader tried to write. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): vendor per-platform FTS artifacts so analyze needs no host install Keyword search depended on a CDN fetch into ~/.lbdb. Shipping the five published tuples inside the package makes air-gapped and ignore-scripts installs load the same artifact the publish gate checksums. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): load the packaged FTS artifact before any network install Analyze still required a CDN fetch into ~/.lbdb even when the package already shipped the file. FTS now path-loads the vendored tuple first and records source labels so a later truncated home copy cannot steal the diagnosis. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): diagnose a core/extension version skew instead of a missing runtime A structurally valid FTS artifact whose path version disagrees with the packaged pin must name both versions, not prescribe VC++ or OpenSSL. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): stamp an FTS phase so repair stays usable after an in-place abort A native CREATE_FTS_INDEX abort leaves no skip reason; the next run infers it from the dirty flag, and --repair-fts must not treat that phase as a half-written graph. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): park an in-place FTS crash WAL without wiping the graph An FTS abort after a successful checkpoint must reopen the live index on macOS, Windows, and Linux. Staging never parks the live WAL; readers keep today's large-WAL refusal. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): refuse read-only opens of an FTS-poisoned WAL MCP and serve cannot repair a leftover in-place abort. Fail before the native open and name --repair-fts, on macOS, Windows, and Linux. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): name a vendor-neutral Windows OpenSSL prerequisite OQ1 is unanswered here so GitNexus does not ship OpenSSL DLLs. Windows FTS now asks for a system OpenSSL 3 runtime instead of Git Bash PATH. Co-authored-by: Cursor <cursoragent@cursor.com> * test(lbug): inject the FTS vendor root and redact it on HTTP and MCP Path-loaded artifacts no longer vary with HOME. Tests pass an injected vendor tree and assert search warnings never leak a filesystem path. Co-authored-by: Cursor <cursoragent@cursor.com> * docs(lbug): document load-only as the global FTS install default Analyze still overrides to auto. Packaged per-platform artifacts load before any network install on macOS, Windows, and Linux. Co-authored-by: Cursor <cursoragent@cursor.com> * docs(lbug): format the FTS install-policy README table Prettier does not run on Markdown in pre-commit, so the U10 table wrap needs its own formatting commit. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): skip FTS CREATE after a persisted native abort A recovered analyze run was retrying CREATE_FTS_INDEX from skipReason alone. Keep that skip until --repair-fts, fail closed on unsupported tuples, and honor the checkpoint warrant for park/repair. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): honor checkpoint flushed warrant and align FTS tests with packaged vendor A no-op CHECKPOINT must not satisfy the FTS park warrant, and CI still asserted HOME-only FTS isolation after analyze started path-LOADing the packaged artifact. Co-authored-by: Cursor <cursoragent@cursor.com> * test(lbug): accept a nonempty incremental write set in the #2790 recovery check FTS-phase recovery can incremental-add files (changed=0, added=1). That is not the #2790 empty-diff wipe skip. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): compare FTS home versions to the core pin and tighten the publish filename gate Ladybug's ~/.lbdb/extension directory is the runtime/core version; treating it as the artifact version false-diagnosed skew. The publish guard now rejects a path-escaping filename the same way the fetch script does. Co-authored-by: Cursor <cursoragent@cursor.com> * test(lbug): seed FTS e2e fixtures from the packaged vendor artifact A machine with no ~/.lbdb copy should still run the vendor-survivorship cases; the seed no longer depends on HOME or a network install. Co-authored-by: Cursor <cursoragent@cursor.com> * Address PR review feedback (#3274) Keep in-place FTS abort evidence after persist so a second CREATE abort cannot fail-open readers, and close the CLI, loader, embed, and e2e gaps the review called out. Note: full npm test hit Ladybug worker-pool startup failures under memory pressure; tsc and 180 targeted unit tests passed. Co-authored-by: Cursor <cursoragent@cursor.com> * Address PR review feedback (#3274) Run the vendored-path symlink guard on the OS matrix, put e2e HOME fixtures on Ladybug's real extension layout, pin the embed crash-WAL gate before the writable open, and let analyze writers park through missing-shadow recovery. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): keep --repair-fts CI green after vendored-first FTS Never-installed warning fixtures must not inspect a packaged vendor binary, and a failed dirty restamp must not abort an otherwise successful --repair-fts run. Co-authored-by: Cursor <cursoragent@cursor.com> * test(cli): give the #1169 analyze e2e the same 90s Windows budget as its sibling The first #1169 persist-meta case was still on a 60s spawn/it budget and was killed banner-only on windows-latest after the FTS warning fixture no longer failed the shard first. Co-authored-by: Cursor <cursoragent@cursor.com> * test(ci): reweight Windows shards after the FTS e2e grew Vendored-first HOME fixtures pushed fts-extension-e2e to ~6 minutes on windows-latest, so the old 146s weight packed it with skills-e2e and blew the 20-minute watchdog. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com>
246 lines
7.4 KiB
JavaScript
246 lines
7.4 KiB
JavaScript
#!/usr/bin/env node
|
|
/**
|
|
* Publish guard: core↔extension pairing plus vendored FTS artifact integrity.
|
|
*
|
|
* Does not shell out to `npm pack` (prepack re-entrancy; see the grammar gate).
|
|
* Checksums and the `files` allow-list are asserted as pure predicates so a
|
|
* future lean-publish narrowing cannot drop the artifacts silently.
|
|
*/
|
|
const fs = require('fs');
|
|
const path = require('path');
|
|
const crypto = require('crypto');
|
|
|
|
const WIN32_ARM64 = 'win32-arm64';
|
|
const SAFE_FILENAME = /^[\w.-]+\.lbug_extension$/;
|
|
const REQUIRED_SUPPORTED_TUPLES = [
|
|
'linux-x64',
|
|
'linux-arm64',
|
|
'darwin-x64',
|
|
'darwin-arm64',
|
|
'win32-x64',
|
|
];
|
|
|
|
/**
|
|
* Pure pairing core (exported for tests). Returns human-readable problem
|
|
* strings; an empty array means the core↔extension pin is consistent.
|
|
*/
|
|
const EXACT_CORE_PIN = /^\d+\.\d+\.\d+$/;
|
|
|
|
function findPairingProblems({
|
|
installedCoreVersion,
|
|
manifestCoreVersion,
|
|
manifestExtensionVersion,
|
|
}) {
|
|
const problems = [];
|
|
const installed = String(installedCoreVersion ?? '');
|
|
const pinned = String(manifestCoreVersion ?? '');
|
|
if (!EXACT_CORE_PIN.test(installed) || !EXACT_CORE_PIN.test(pinned)) {
|
|
problems.push(
|
|
`core pin must be exact x.y.z: installed '${installedCoreVersion ?? ''}' vs manifest '${manifestCoreVersion ?? ''}'`,
|
|
);
|
|
return problems;
|
|
}
|
|
if (installed !== pinned) {
|
|
problems.push(
|
|
`core pin mismatch: installed ${installed} vs manifest ${pinned}` +
|
|
(manifestExtensionVersion ? ` (extension ${manifestExtensionVersion})` : ''),
|
|
);
|
|
}
|
|
return problems;
|
|
}
|
|
|
|
function readInstalledCoreVersion(pkg) {
|
|
const raw = pkg?.dependencies?.['@ladybugdb/core'];
|
|
return raw == null ? '' : String(raw);
|
|
}
|
|
|
|
function normalizeFilesEntry(value) {
|
|
return String(value ?? '')
|
|
.replace(/\\/g, '/')
|
|
.replace(/\/+$/, '')
|
|
.replace(/\/\*\*?$/, '');
|
|
}
|
|
|
|
/** True when package.json `files` still ships the FTS prebuild tree. */
|
|
function filesCoverFtsArtifacts(filesField) {
|
|
return (filesField || []).some((entry) => {
|
|
const n = normalizeFilesEntry(entry);
|
|
return (
|
|
n === 'vendor' ||
|
|
n === 'vendor/lbug-fts' ||
|
|
n === 'vendor/lbug-fts/prebuilds' ||
|
|
n === 'vendor/**/prebuilds'
|
|
);
|
|
});
|
|
}
|
|
|
|
function supportedTuplesFromManifest(manifest) {
|
|
return (manifest?.tuples ?? []).map((entry) => entry.tuple);
|
|
}
|
|
|
|
function unsupportedTuplesFromManifest(manifest) {
|
|
return (manifest?.unsupportedTuples ?? []).map((entry) => entry.tuple);
|
|
}
|
|
|
|
function parseSha256Sums(text) {
|
|
const out = {};
|
|
for (const line of String(text ?? '').split(/\r?\n/)) {
|
|
const m = /^([a-fA-F0-9]{64})\s+\.\/(\S+)$/.exec(line.trim());
|
|
if (!m) continue;
|
|
out[m[2]] = m[1].toLowerCase();
|
|
}
|
|
return out;
|
|
}
|
|
|
|
function sha256File(filePath) {
|
|
return crypto.createHash('sha256').update(fs.readFileSync(filePath)).digest('hex');
|
|
}
|
|
|
|
/**
|
|
* Integrity + coverage predicates. `artifactByTuple` is injected so tests
|
|
* never invoke pack and never need the real binaries.
|
|
*/
|
|
function findArtifactProblems({
|
|
tuples,
|
|
unsupportedTuples,
|
|
filesField,
|
|
checksumByRelPath,
|
|
artifactByTuple,
|
|
filename,
|
|
}) {
|
|
const problems = [];
|
|
const filenameSafe = filename || 'libfts.lbug_extension';
|
|
if (!SAFE_FILENAME.test(filenameSafe)) {
|
|
problems.push(`invalid FTS artifact filename: ${filenameSafe}`);
|
|
}
|
|
const listed = new Set(tuples || []);
|
|
for (const required of REQUIRED_SUPPORTED_TUPLES) {
|
|
if (!listed.has(required)) {
|
|
problems.push(`manifest.tuples is missing required ${required}`);
|
|
}
|
|
}
|
|
if (!tuples || tuples.length === 0) {
|
|
problems.push('manifest.tuples is empty — refusing to publish with 0 artifacts');
|
|
}
|
|
if (!filesCoverFtsArtifacts(filesField)) {
|
|
problems.push('package.json files no longer covers vendor/lbug-fts/prebuilds');
|
|
}
|
|
if (!(unsupportedTuples || []).includes(WIN32_ARM64)) {
|
|
problems.push('win32-arm64 must be declared unsupported (no upstream artifact)');
|
|
}
|
|
if ((tuples || []).includes(WIN32_ARM64)) {
|
|
problems.push('win32-arm64 is listed as supported but has no upstream artifact');
|
|
}
|
|
for (const tuple of tuples || []) {
|
|
const rel = `${tuple}/${filenameSafe}`;
|
|
const artifact = artifactByTuple?.[tuple];
|
|
if (!artifact?.exists) {
|
|
problems.push(`missing artifact for ${tuple} (${rel})`);
|
|
continue;
|
|
}
|
|
const expected = checksumByRelPath?.[rel];
|
|
if (!expected) {
|
|
problems.push(`missing SHA-256 for ${rel}`);
|
|
continue;
|
|
}
|
|
if (artifact.hash !== expected) {
|
|
problems.push(
|
|
`checksum mismatch for ${tuple}: expected ${expected} got ${artifact.hash}` +
|
|
(artifact.sizeBytes != null ? ` (${artifact.sizeBytes} bytes)` : ''),
|
|
);
|
|
}
|
|
}
|
|
return problems;
|
|
}
|
|
|
|
function readDiskArtifacts(prebuildsDir, tuples, filename) {
|
|
const artifactByTuple = {};
|
|
for (const tuple of tuples) {
|
|
const filePath = path.join(prebuildsDir, tuple, filename);
|
|
if (!fs.existsSync(filePath)) {
|
|
artifactByTuple[tuple] = { exists: false };
|
|
continue;
|
|
}
|
|
const buf = fs.readFileSync(filePath);
|
|
artifactByTuple[tuple] = {
|
|
exists: true,
|
|
hash: crypto.createHash('sha256').update(buf).digest('hex'),
|
|
sizeBytes: buf.byteLength,
|
|
};
|
|
}
|
|
return artifactByTuple;
|
|
}
|
|
|
|
function main() {
|
|
const gitnexusRoot = path.join(__dirname, '..');
|
|
const pkg = JSON.parse(fs.readFileSync(path.join(gitnexusRoot, 'package.json'), 'utf8'));
|
|
const vendorDir = path.join(gitnexusRoot, 'vendor', 'lbug-fts');
|
|
const manifestPath = path.join(vendorDir, 'manifest.json');
|
|
let manifest;
|
|
try {
|
|
manifest = JSON.parse(fs.readFileSync(manifestPath, 'utf8'));
|
|
} catch (err) {
|
|
console.error(
|
|
`[fts-pairing] Refusing to publish — cannot read ${manifestPath}: ${err.message}`,
|
|
);
|
|
process.exit(1);
|
|
}
|
|
|
|
const installedCoreVersion = readInstalledCoreVersion(pkg);
|
|
const pairing = findPairingProblems({
|
|
installedCoreVersion,
|
|
manifestCoreVersion: manifest.coreVersion,
|
|
manifestExtensionVersion: manifest.extensionVersion,
|
|
});
|
|
|
|
const tuples = supportedTuplesFromManifest(manifest);
|
|
const unsupportedTuples = unsupportedTuplesFromManifest(manifest);
|
|
const filename = manifest.filename || 'libfts.lbug_extension';
|
|
const prebuildsDir = path.join(vendorDir, 'prebuilds');
|
|
let checksumByRelPath = {};
|
|
try {
|
|
checksumByRelPath = parseSha256Sums(
|
|
fs.readFileSync(path.join(prebuildsDir, 'SHA256SUMS'), 'utf8'),
|
|
);
|
|
} catch (err) {
|
|
pairing.push(`cannot read SHA256SUMS: ${err.message}`);
|
|
}
|
|
|
|
const artifacts = findArtifactProblems({
|
|
tuples,
|
|
unsupportedTuples,
|
|
filesField: pkg.files,
|
|
checksumByRelPath,
|
|
artifactByTuple: readDiskArtifacts(prebuildsDir, tuples, filename),
|
|
filename,
|
|
});
|
|
|
|
const problems = [...pairing, ...artifacts];
|
|
if (problems.length > 0) {
|
|
console.error('[fts-pairing] Refusing to publish — FTS artifact coverage failed:');
|
|
for (const p of problems) console.error(` - ${p}`);
|
|
console.error(
|
|
'\nFix: refresh vendor/lbug-fts via .github/scripts/fetch-lbug-fts-artifacts.mjs, ' +
|
|
'or restore the core pin / files allow-list.',
|
|
);
|
|
process.exit(1);
|
|
}
|
|
|
|
console.log(
|
|
`[fts-pairing] OK — core ${installedCoreVersion} ↔ extension ${manifest.extensionVersion}; ` +
|
|
`${tuples.length} artifacts.`,
|
|
);
|
|
}
|
|
|
|
if (require.main === module) main();
|
|
|
|
module.exports = {
|
|
findPairingProblems,
|
|
findArtifactProblems,
|
|
filesCoverFtsArtifacts,
|
|
parseSha256Sums,
|
|
readInstalledCoreVersion,
|
|
supportedTuplesFromManifest,
|
|
unsupportedTuplesFromManifest,
|
|
sha256File,
|
|
};
|