GitNexus/gitnexus/test/integration/extension-binary-real.test.ts
Gergő Magyar 76a1c90b02
Some checks are pending
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Scorecard / Scorecard analysis (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
fix(fts): diagnose Windows FTS missing-dependency load failures (#2374, Phase 1) (#2383)
* feat(lbug): classify FTS extension load errors with Windows missing-dependency guard (#2374)

Add classifyExtensionLoadError() — a pure-string, lbug-free four-way
classifier (missing_file / corrupt_file / missing_dependency / unknown).
The Windows catch-all guard keys missing_dependency strictly on the
error-126 signal, never LadybugDB's generic 'Failed to load library …
needed by extension' wrapper, so 127/5/1114 and truncated (193) files
route correctly.

* feat(fts): surface classified missing-dependency remedy in doctor, repair-fts, and degrade warnings (#2374)

Route the FTS load reason through classifyExtensionLoadError at all four
surfaces (doctor, --repair-fts error, analyze degrade log, ftsDegradedWarning).
For the Windows missing-dependency class, emit the runtime-install remedy
(VC++ redist, then OpenSSL) instead of the wrong reinstall-over-network
guidance; other classes keep their existing routing. Path redaction preserved
on the client-facing warning.

* test(fts): assert doctor surfaces the classified remedy end-to-end (#2374)

Extend the broken-file e2e: doctor now prints the corrupt-file re-download
remedy through the real CLI, and the Windows missing-dependency remedy
(VC++/OpenSSL) must not misfire on a corrupt file — the catch-all guard,
verified end-to-end. Also assert the repair path does not misfire.

* style(fts): apply prettier formatting to #2374 diagnosis files

* feat(fts): language-independent hedged fallback for Windows load failures (#2374)

The Windows OS-error tail is localized, so matching only en/zh 126 text left
other locales on the generic 'run doctor' remedy. lbug's 'Failed to load
library' wrapper is English on every platform and present for all load
failures, so use it as a fallback: when the localized tail matches no specific
class, emit a hedged remedy that points the user at their own OS error and
offers both branches (install runtime / --repair-fts) without prescribing the
wrong single fix. Precise en/zh 126 keeps its definite remedy.

* feat(fts): language-independent structural classifier via binary inspection (#2374)

Add diagnoseExtensionLoad: pull the extension's file path out of lbug's own
English wrapper and inspect the binary header (PE/ELF/Mach-O magic + arch)
directly, so corrupt-vs-valid is decided by the file itself, not the localized
OS-error tail. A valid binary that still failed to load ⇒ missing_dependency
(runtime dep), decided in any OS display language and on all three platforms.
Falls back to the string classifier (with its hedged fallback) when the file
can't be read. Wire all four surfaces to it. Event Viewer / GetLastError-via-FFI
were dead ends (lbug catches the failure — no crash event; no native FFI dep).

* test(fts): exercise the structural classifier on real binaries (#2374)

Add an integration suite that runs inspectExtensionBinary/diagnoseExtensionLoad
against genuine binaries — the running node executable, the real lbugjs.node
addon, and the installed FTS extension (valid); a truncated real binary and a
real text file (corrupt). Registered in cross-platform-tests PLATFORM_LOGIC so
it runs on the Windows + macOS matrix, proving the PE and Mach-O header parsing
on real PE/Mach-O files (ubuntu covers ELF).

* fix(fts): honor a corrupt_file verdict over a structurally-valid header (#2374)

The structural probe in diagnoseExtensionLoad inspects only the first 4 KB, so a
download truncated after its header reads 'valid' and was routed to the "install
VC++, reinstalling will NOT help" remedy — the exact loop #2374 exists to kill,
for the truncated-download case the module docstring claims it handles. Honor the
loader's own corruption report ("file too short" / Windows error 193
"not a valid Win32 application") before defaulting to the dependency remedy;
localized corrupt tails stay hedged missing_dependency, preserving
language-independence.

Addresses PR #2383 review finding F1.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(fts): return indeterminate for a PE header beyond the read window (#2374)

The structural probe reads only BINARY_HEADER_BYTES (4 KB). A valid PE with a
large DOS stub whose e_lfanew points past that window was wrongly called
'corrupt', routing a fine DLL to "re-download". A garbage e_lfanew from a truly
corrupt file is indistinguishable from here, so widen the header verdict with
'indeterminate' and return it in that case; the caller then defers to the
loader's own report instead of asserting a false verdict. Fat Mach-O stays valid
(LadybugDB ships thin per-arch binaries). Also covers the unmapped-arch and
garbage-PE-signature branches.

Addresses PR #2383 review finding F1-secondary.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(fts): drop contradictory reinstall guidance from the analyze degrade log (#2374)

For a missing runtime dependency the extension file is present, so appending
FTS_UNAVAILABLE_MESSAGE (which tells the user to install it "with network
access") to the remedy ("reinstalling will NOT help") produced self-contradictory
guidance on the main analyze surface. Lead the missing_dependency degrade log
with the class-neutral sentence (FTS_UNAVAILABLE_LEAD) and append only the
classified remedy; other classes keep FTS_UNAVAILABLE_MESSAGE unchanged.

Addresses PR #2383 review finding F2.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* perf(fts): cache the load diagnosis so the degraded warning does no per-request I/O (#2374)

ftsDegradedWarning() runs on every degraded /api/search response and MCP query,
and it was calling diagnoseExtensionLoad — a synchronous openSync/readSync of the
extension file — on every call. Compute the diagnosis once at mark-unavailable
time (the single load-failure sink, run per Database not per request), cache it on
ExtensionCapability, and have the warning read the cached result (falling back to
the pure, no-I/O string classifier if it is absent). Loader capability-shape
assertions relax from toEqual to toMatchObject for the new optional field.

Addresses PR #2383 review finding F3.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(fts): cover the missing_dependency remedy on the --repair-fts path (#2374)

The repair-fts error interpolates the classified remedy, but no test reached the
missing_dependency branch — only the corrupt/invalid-ELF path. Add a Windows
error-126 case asserting the thrown error carries the VC++ redistributable remedy
and omits the old "retry the network install" tail, and that no index is dropped.

Addresses PR #2383 review finding F6a (--repair-fts surface).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(fts): share the VC++ redistributable install hint (#2374)

The Microsoft Visual C++ redistributable name and aka.ms URL were duplicated
verbatim in WINDOWS_MISSING_DEPENDENCY_REMEDY and
STRUCTURAL_MISSING_DEPENDENCY_REMEDY. Factor a single VC_REDIST_INSTALL_HINT
constant so the pointer cannot drift between them; the composed remedy strings are
byte-identical (existing exact-text assertions unchanged). Also adds a test
covering the previously-unexercised structural remedy branch.

Addresses PR #2383 review finding F5a.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(fts): guard FILE_CORRUPTION_SIGNATURES parity with the installer script (#2374)

The corruption-signature list is deliberately duplicated between
extension-load-error.ts and scripts/install-duckdb-extension.mjs (the .mjs cannot
import the .ts), with nothing guarding against drift — a one-sided edit would
desync the FORCE-INSTALL verb from remedy classification. Export the array from
both and add a parity test that compares regex source + flags element-wise.

Addresses PR #2383 review finding F5b.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore(test): run extension-binary-real in the sequential lbug-db vitest project (#2374)

extension-binary-real.test.ts imports @ladybugdb/core but ran in the parallel
`default` project, contrary to TESTING.md's rule that native-LadybugDB tests live
in the sequential `lbug-db` project. Add it to the lbug-db include list and the
default exclude list; it now runs under lbug-db and no longer under default.

Addresses PR #2383 review finding F6c.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(fts): fail loud, not silent-skip, on missing FTS artifacts under REQUIRE_FTS=1 (#2374)

The real-binary structural tests gated on raw .skipIf(!lbugNative) /
.skipIf(!installedFts), so under GITNEXUS_REQUIRE_FTS=1 a missing artifact would
silently vanish from a green CI run (the #2299 trap). These tests inspect the
extension file directly and need its path, not a loaded connection — so
skipUnlessFtsAvailable (which needs an initialized LadybugDB) does not fit. Add
requireFtsResourceOrSkip: skip gracefully offline, throw under REQUIRE_FTS=1. The
always-on process.execPath assertion still runs everywhere.

Addresses PR #2383 review finding F6d.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* style(fts): apply prettier formatting to the #2383 fix files (#2374)

Line-wrapping only; the quality/format CI check flagged three files. No behavior
change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 21:27:37 +01:00

111 lines
4.6 KiB
TypeScript

import {
copyFileSync,
existsSync,
mkdtempSync,
readdirSync,
readFileSync,
rmSync,
statSync,
writeFileSync,
} from 'node:fs';
import { homedir, tmpdir } from 'node:os';
import { join } from 'node:path';
import { afterAll, describe, expect, it } from 'vitest';
import lbug from '@ladybugdb/core';
import {
diagnoseExtensionLoad,
inspectExtensionBinary,
} from '../../src/core/lbug/extension-load-error.js';
import { requireFtsResourceOrSkip } from '../helpers/fts-availability.js';
/**
* #2374: exercise the language-independent structural classifier against REAL
* binaries, not synthetic headers. Registered in cross-platform-tests.ts
* PLATFORM_LOGIC so it runs on the Windows + macOS matrix too — where
* `process.execPath` / `lbugjs.node` are real PE / Mach-O files, proving the
* PE and Mach-O header parsing on genuine binaries (the ubuntu suite covers ELF).
*/
const tmpDirs: string[] = [];
function makeTmpFile(prefix: string, name: string): string {
const dir = mkdtempSync(join(tmpdir(), prefix));
tmpDirs.push(dir);
return join(dir, name);
}
afterAll(() => {
for (const dir of tmpDirs) rmSync(dir, { recursive: true, force: true });
});
/** The real LadybugDB native addon for this platform, if resolvable. */
function resolveLbugNative(): string | null {
const roots = [`core-${process.platform}-${process.arch}`, 'core'];
for (const root of roots) {
const candidate = join(process.cwd(), 'node_modules', '@ladybugdb', root, 'lbugjs.node');
if (existsSync(candidate)) return candidate;
}
return null;
}
/** The actual installed FTS extension binary for the running lbug version. */
function resolveInstalledFtsExtension(): string | null {
const home = process.env.USERPROFILE ?? process.env.HOME ?? homedir();
const base = join(home, '.lbdb', 'extension', lbug.VERSION);
try {
const platformDir = readdirSync(base).find((entry) =>
statSync(join(base, entry)).isDirectory(),
);
if (!platformDir) return null;
const ext = join(base, platformDir, 'fts', 'libfts.lbug_extension');
return existsSync(ext) ? ext : null;
} catch {
return null;
}
}
const lbugNative = resolveLbugNative();
const installedFts = resolveInstalledFtsExtension();
describe('structural classifier on real binaries (#2374)', () => {
it('the running Node executable is a valid host binary', () => {
// Real ELF (Linux), PE (Windows), or Mach-O (macOS) for the host arch.
expect(inspectExtensionBinary(process.execPath)).toBe('valid');
});
// These inspect the extension FILE directly, so they gate on the artifact's
// presence — but under GITNEXUS_REQUIRE_FTS=1 a missing artifact is a HARD FAILURE,
// never a silent skip that could vanish from a green CI run (#2299, #2383 F6d).
it('the real lbugjs.node native addon is a valid host binary', (ctx) => {
requireFtsResourceOrSkip(ctx, lbugNative, 'lbugjs.node native addon');
expect(inspectExtensionBinary(lbugNative)).toBe('valid');
});
it('the installed FTS extension is valid → a load failure is missing_dependency, in any language', (ctx) => {
requireFtsResourceOrSkip(ctx, installedFts, 'installed FTS extension');
expect(inspectExtensionBinary(installedFts)).toBe('valid');
// A localized OS tail we do not enumerate — the structural check decides it.
const reason = `Failed to load library: ${installedFts} which is needed by extension: fts. Error: <localized>`;
expect(diagnoseExtensionLoad(reason)).toMatchObject({ kind: 'missing_dependency' });
});
it('a real valid binary at a *.lbug_extension path diagnoses as missing_dependency', () => {
const ext = makeTmpFile('real-valid-', 'libfts.lbug_extension');
copyFileSync(process.execPath, ext);
const reason = `Failed to load library: ${ext} which is needed by extension: fts. Error: <localized>`;
expect(diagnoseExtensionLoad(reason)).toMatchObject({ kind: 'missing_dependency' });
});
it('a truncated real binary is corrupt', () => {
const ext = makeTmpFile('real-trunc-', 'libfts.lbug_extension');
// First 3 bytes of a real binary: a partial magic, too short for any header.
writeFileSync(ext, readFileSync(process.execPath).subarray(0, 3));
expect(inspectExtensionBinary(ext)).toBe('corrupt');
});
it('a real non-binary file placed as the extension is corrupt', () => {
const ext = makeTmpFile('real-text-', 'libfts.lbug_extension');
// A genuine text file (this repo's package.json) — the exact "user dropped the
// wrong file" mistake, caught structurally with no valid magic.
copyFileSync(join(process.cwd(), 'package.json'), ext);
expect(inspectExtensionBinary(ext)).toBe('corrupt');
});
});