mirror of
https://github.com/abhigyanpatwari/GitNexus.git
synced 2026-10-04 02:31:36 +00:00
* fix(lbug): pin Ladybug core so Dependabot cannot ship a skewed FTS artifact The extension version is a separate upstream constant. Ignore daily core bumps and fail the pairing gate when the committed manifest does not name the installed core. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): make doctor and CI FTS gates resolve the packaged artifact Doctor and the REQUIRE_FTS file gates still treated an empty ~/.lbdb as unavailable, which would turn three CI jobs red once analyze stops installing into that tree. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): name native-abort and tuple-missing so analyze cannot mis-advise The CLI summary's trailing else treated every unknown skip reason as a missing extension. New crash and platform causes must get their own remedies, not a network-install hint. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): delete the dead read-path FTS index create ensureFTSIndex had no production callers and swallowed read-only CREATE_FTS_INDEX failures, which hid the only signal that a reader tried to write. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): vendor per-platform FTS artifacts so analyze needs no host install Keyword search depended on a CDN fetch into ~/.lbdb. Shipping the five published tuples inside the package makes air-gapped and ignore-scripts installs load the same artifact the publish gate checksums. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): load the packaged FTS artifact before any network install Analyze still required a CDN fetch into ~/.lbdb even when the package already shipped the file. FTS now path-loads the vendored tuple first and records source labels so a later truncated home copy cannot steal the diagnosis. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): diagnose a core/extension version skew instead of a missing runtime A structurally valid FTS artifact whose path version disagrees with the packaged pin must name both versions, not prescribe VC++ or OpenSSL. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): stamp an FTS phase so repair stays usable after an in-place abort A native CREATE_FTS_INDEX abort leaves no skip reason; the next run infers it from the dirty flag, and --repair-fts must not treat that phase as a half-written graph. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): park an in-place FTS crash WAL without wiping the graph An FTS abort after a successful checkpoint must reopen the live index on macOS, Windows, and Linux. Staging never parks the live WAL; readers keep today's large-WAL refusal. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): refuse read-only opens of an FTS-poisoned WAL MCP and serve cannot repair a leftover in-place abort. Fail before the native open and name --repair-fts, on macOS, Windows, and Linux. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): name a vendor-neutral Windows OpenSSL prerequisite OQ1 is unanswered here so GitNexus does not ship OpenSSL DLLs. Windows FTS now asks for a system OpenSSL 3 runtime instead of Git Bash PATH. Co-authored-by: Cursor <cursoragent@cursor.com> * test(lbug): inject the FTS vendor root and redact it on HTTP and MCP Path-loaded artifacts no longer vary with HOME. Tests pass an injected vendor tree and assert search warnings never leak a filesystem path. Co-authored-by: Cursor <cursoragent@cursor.com> * docs(lbug): document load-only as the global FTS install default Analyze still overrides to auto. Packaged per-platform artifacts load before any network install on macOS, Windows, and Linux. Co-authored-by: Cursor <cursoragent@cursor.com> * docs(lbug): format the FTS install-policy README table Prettier does not run on Markdown in pre-commit, so the U10 table wrap needs its own formatting commit. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): skip FTS CREATE after a persisted native abort A recovered analyze run was retrying CREATE_FTS_INDEX from skipReason alone. Keep that skip until --repair-fts, fail closed on unsupported tuples, and honor the checkpoint warrant for park/repair. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): honor checkpoint flushed warrant and align FTS tests with packaged vendor A no-op CHECKPOINT must not satisfy the FTS park warrant, and CI still asserted HOME-only FTS isolation after analyze started path-LOADing the packaged artifact. Co-authored-by: Cursor <cursoragent@cursor.com> * test(lbug): accept a nonempty incremental write set in the #2790 recovery check FTS-phase recovery can incremental-add files (changed=0, added=1). That is not the #2790 empty-diff wipe skip. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): compare FTS home versions to the core pin and tighten the publish filename gate Ladybug's ~/.lbdb/extension directory is the runtime/core version; treating it as the artifact version false-diagnosed skew. The publish guard now rejects a path-escaping filename the same way the fetch script does. Co-authored-by: Cursor <cursoragent@cursor.com> * test(lbug): seed FTS e2e fixtures from the packaged vendor artifact A machine with no ~/.lbdb copy should still run the vendor-survivorship cases; the seed no longer depends on HOME or a network install. Co-authored-by: Cursor <cursoragent@cursor.com> * Address PR review feedback (#3274) Keep in-place FTS abort evidence after persist so a second CREATE abort cannot fail-open readers, and close the CLI, loader, embed, and e2e gaps the review called out. Note: full npm test hit Ladybug worker-pool startup failures under memory pressure; tsc and 180 targeted unit tests passed. Co-authored-by: Cursor <cursoragent@cursor.com> * Address PR review feedback (#3274) Run the vendored-path symlink guard on the OS matrix, put e2e HOME fixtures on Ladybug's real extension layout, pin the embed crash-WAL gate before the writable open, and let analyze writers park through missing-shadow recovery. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): keep --repair-fts CI green after vendored-first FTS Never-installed warning fixtures must not inspect a packaged vendor binary, and a failed dirty restamp must not abort an otherwise successful --repair-fts run. Co-authored-by: Cursor <cursoragent@cursor.com> * test(cli): give the #1169 analyze e2e the same 90s Windows budget as its sibling The first #1169 persist-meta case was still on a 60s spawn/it budget and was killed banner-only on windows-latest after the FTS warning fixture no longer failed the shard first. Co-authored-by: Cursor <cursoragent@cursor.com> * test(ci): reweight Windows shards after the FTS e2e grew Vendored-first HOME fixtures pushed fts-extension-e2e to ~6 minutes on windows-latest, so the old 146s weight packed it with skills-e2e and blew the 20-minute watchdog. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com>
568 lines
27 KiB
TypeScript
568 lines
27 KiB
TypeScript
/**
|
||
* Integration coverage for `deleteNodesForFiles` — the batched incremental
|
||
* delete introduced for #2409.
|
||
*
|
||
* The per-file predecessor issued a count + DETACH DELETE per node table per
|
||
* FILE (~13k single-row write transactions on a ~700-file write set); the
|
||
* batched variant chunks paths into `IN [...]` lists. These tests pin the
|
||
* contract the incremental writeback depends on:
|
||
*
|
||
* - exactly the requested files' rows are deleted, across a >1-chunk set
|
||
* - DETACH semantics: relationships touching deleted nodes go away,
|
||
* relationships between survivors stay
|
||
* - single quotes in paths are escaped, not injected
|
||
* - unknown paths are a no-op success (zero-match ≠ error)
|
||
* - onChunk progress reports cumulative file counts
|
||
* - CodeEmbedding rows ride along with their file's nodes (tri-review
|
||
* 4669518496 P2-1): node ids are label-first (`Function:<fp>:fn:1`), so
|
||
* the delete joins `e.nodeId = n.id` through the still-present nodes —
|
||
* deleted/quoted files' rows go, survivors' rows stay.
|
||
*/
|
||
import { describe, it, expect, afterEach, vi } from 'vitest';
|
||
import path from 'path';
|
||
import { withTestLbugDB } from '../helpers/test-indexed-db.js';
|
||
import { buildTestGraph, type TestNodeInput, type TestRelInput } from '../helpers/test-graph.js';
|
||
import { DELETE_FILES_CHUNK_SIZE } from '../../src/core/lbug/lbug-adapter.js';
|
||
import { EMBEDDING_TABLE_NAME, EMBEDDING_DIMS } from '../../src/core/lbug/schema.js';
|
||
|
||
const FILE_COUNT = DELETE_FILES_CHUNK_SIZE + 30; // crosses the chunk boundary
|
||
const KEEP_COUNT = 10;
|
||
const QUOTED_PATH = "src/we'ird.ts";
|
||
|
||
const filePath = (i: number): string => `src/f-${String(i).padStart(4, '0')}.ts`;
|
||
|
||
function buildFixtureGraph() {
|
||
const nodes: TestNodeInput[] = [];
|
||
const rels: TestRelInput[] = [];
|
||
for (let i = 0; i < FILE_COUNT; i++) {
|
||
const fp = i === 0 ? QUOTED_PATH : filePath(i);
|
||
nodes.push({ id: `File:${fp}`, label: 'File', name: path.basename(fp), filePath: fp });
|
||
nodes.push({
|
||
id: `Function:${fp}:fn${i}:1`,
|
||
label: 'Function',
|
||
name: `fn${i}`,
|
||
filePath: fp,
|
||
startLine: 1,
|
||
endLine: 3,
|
||
isExported: true,
|
||
});
|
||
rels.push({ sourceId: `File:${fp}`, targetId: `Function:${fp}:fn${i}:1`, type: 'CONTAINS' });
|
||
if (i > 0) {
|
||
// Every function calls the previous file's function — so deleting a
|
||
// file must DETACH-drop edges on both sides of the kept/deleted
|
||
// boundary while the survivor-to-survivor edges remain.
|
||
const prev = i === 1 ? QUOTED_PATH : filePath(i - 1);
|
||
rels.push({
|
||
sourceId: `Function:${fp}:fn${i}:1`,
|
||
targetId: `Function:${prev}:fn${i - 1}:1`,
|
||
type: 'CALLS',
|
||
});
|
||
}
|
||
}
|
||
return buildTestGraph(nodes, rels);
|
||
}
|
||
|
||
withTestLbugDB('delete-nodes-for-files', (handle) => {
|
||
describe('deleteNodesForFiles (batched incremental delete, #2409)', () => {
|
||
it('deletes exactly the requested files across chunks with DETACH semantics, quote escaping, embedding-row joins, and zero-match no-ops', async () => {
|
||
const { loadGraphToLbug, deleteNodesForFiles, executeQuery, executeWithReusedStatement } =
|
||
await import('../../src/core/lbug/lbug-adapter.js');
|
||
const { batchInsertEmbeddings } =
|
||
await import('../../src/core/embeddings/embedding-pipeline.js');
|
||
|
||
await loadGraphToLbug(buildFixtureGraph(), '/tmp/repo', path.dirname(handle.dbPath));
|
||
|
||
const count = async (cypher: string): Promise<number> => {
|
||
const rows = (await executeQuery(cypher)) as Array<{ c: number | bigint }>;
|
||
return Number(rows[0]?.c ?? 0);
|
||
};
|
||
|
||
expect(await count('MATCH (n:File) RETURN count(n) AS c')).toBe(FILE_COUNT);
|
||
expect(await count('MATCH (n:Function) RETURN count(n) AS c')).toBe(FILE_COUNT);
|
||
const callsBefore = await count(
|
||
`MATCH ()-[r:CodeRelation]->() WHERE r.type = 'CALLS' RETURN count(r) AS c`,
|
||
);
|
||
expect(callsBefore).toBe(FILE_COUNT - 1);
|
||
|
||
// Seed embedding rows through the real batchInsertEmbeddings for a
|
||
// to-be-deleted plain-path file, the quoted-path file, and a survivor.
|
||
// nodeIds are the fixture's REAL label-first node ids — the exact
|
||
// format the old bare-path `STARTS WITH` shape could never match
|
||
// (tri-review 4669518496 P2-1). Zero vectors: the CodeEmbedding table
|
||
// is plain schema (no VECTOR extension involved).
|
||
const SURVIVOR_PATH = filePath(FILE_COUNT - 1);
|
||
const survivorEmbeddingNodeId = `Function:${SURVIVOR_PATH}:fn${FILE_COUNT - 1}:1`;
|
||
const survivorFileEmbeddingNodeId = `File:${SURVIVOR_PATH}`;
|
||
const seededEmbeddingNodeIds = [
|
||
`Function:${filePath(1)}:fn1:1`, // deleted, plain path
|
||
`File:${filePath(1)}`, // deleted fallback File embedding, plain path
|
||
`Function:${QUOTED_PATH}:fn0:1`, // deleted, quoted path
|
||
`File:${QUOTED_PATH}`, // deleted fallback File embedding, quoted path
|
||
survivorEmbeddingNodeId, // survives the delete
|
||
survivorFileEmbeddingNodeId, // fallback File embedding also survives
|
||
];
|
||
await batchInsertEmbeddings(
|
||
executeWithReusedStatement,
|
||
seededEmbeddingNodeIds.map((nodeId) => ({
|
||
nodeId,
|
||
chunkIndex: 0,
|
||
startLine: 1,
|
||
endLine: 3,
|
||
embedding: new Array(EMBEDDING_DIMS).fill(0),
|
||
})),
|
||
);
|
||
expect(await count(`MATCH (e:${EMBEDDING_TABLE_NAME}) RETURN count(e) AS c`)).toBe(
|
||
seededEmbeddingNodeIds.length,
|
||
);
|
||
|
||
// Delete everything except the last KEEP_COUNT files. Includes the
|
||
// quoted path (chunk 1), crosses into chunk 2, and appends a path with
|
||
// no rows at all — which must not fail the batch.
|
||
const toDelete: string[] = [QUOTED_PATH];
|
||
for (let i = 1; i < FILE_COUNT - KEEP_COUNT; i++) toDelete.push(filePath(i));
|
||
toDelete.push('src/never-existed.ts');
|
||
|
||
const chunkCalls: Array<[number, number]> = [];
|
||
await deleteNodesForFiles(toDelete, {
|
||
onChunk: (done, total) => chunkCalls.push([done, total]),
|
||
});
|
||
|
||
// Cumulative chunk progress: [200, 221] then [221, 221].
|
||
expect(chunkCalls).toEqual([
|
||
[DELETE_FILES_CHUNK_SIZE, toDelete.length],
|
||
[toDelete.length, toDelete.length],
|
||
]);
|
||
|
||
expect(await count('MATCH (n:File) RETURN count(n) AS c')).toBe(KEEP_COUNT);
|
||
expect(await count('MATCH (n:Function) RETURN count(n) AS c')).toBe(KEEP_COUNT);
|
||
// Quoted path really gone (escaping worked; nothing else was swept up).
|
||
expect(
|
||
await count(`MATCH (n:File) WHERE n.filePath = "${QUOTED_PATH}" RETURN count(n) AS c`),
|
||
).toBe(0);
|
||
// DETACH: the only CALLS edges left are between surviving functions —
|
||
// KEEP_COUNT survivors form a chain of KEEP_COUNT-1 edges; the edge from
|
||
// the first survivor into the deleted region is gone.
|
||
expect(
|
||
await count(`MATCH ()-[r:CodeRelation]->() WHERE r.type = 'CALLS' RETURN count(r) AS c`),
|
||
).toBe(KEEP_COUNT - 1);
|
||
// Survivors untouched.
|
||
expect(
|
||
await count(
|
||
`MATCH (n:File) WHERE n.filePath = '${filePath(FILE_COUNT - 1)}' RETURN count(n) AS c`,
|
||
),
|
||
).toBe(1);
|
||
// Embedding rows followed their files: ONLY the survivor's row remains
|
||
// — exact nodeId, not count-only, so a delete that swept the wrong rows
|
||
// (or none) cannot pass. The quoted-path row proves the join statement
|
||
// escapes list literals, not just the per-table deletes.
|
||
const embRows = (await executeQuery(
|
||
`MATCH (e:${EMBEDDING_TABLE_NAME}) RETURN e.nodeId AS nodeId`,
|
||
)) as Array<{ nodeId: string }>;
|
||
expect(embRows.map((r) => String(r.nodeId)).sort()).toEqual(
|
||
[survivorEmbeddingNodeId, survivorFileEmbeddingNodeId].sort(),
|
||
);
|
||
|
||
// Zero-match batch (all paths already gone) is a clean no-op.
|
||
await expect(deleteNodesForFiles([QUOTED_PATH, filePath(1)])).resolves.toBeUndefined();
|
||
// …and it left the surviving embedding row alone.
|
||
expect(await count(`MATCH (e:${EMBEDDING_TABLE_NAME}) RETURN count(e) AS c`)).toBe(2);
|
||
}, 120_000);
|
||
|
||
it('a File node without an embedding deletes cleanly and leaves other files’ embedding rows intact (FIX 4)', async () => {
|
||
const { deleteNodesForFiles, executeQuery } =
|
||
await import('../../src/core/lbug/lbug-adapter.js');
|
||
const count = async (cypher: string): Promise<number> => {
|
||
const rows = (await executeQuery(cypher)) as Array<{ c: number | bigint }>;
|
||
return Number(rows[0]?.c ?? 0);
|
||
};
|
||
|
||
// File can own fallback embeddings, but this fixture deliberately has
|
||
// none. The delete must still remove the node row without erroring, and
|
||
// embedding rows owned by OTHER files stay put.
|
||
const ASSET_PATH = 'src/assets-only.txt';
|
||
await executeQuery(
|
||
`CREATE (:File {id: 'File:${ASSET_PATH}', name: 'assets-only.txt', filePath: '${ASSET_PATH}'})`,
|
||
);
|
||
const embeddingsBefore = await count(
|
||
`MATCH (e:${EMBEDDING_TABLE_NAME}) RETURN count(e) AS c`,
|
||
);
|
||
|
||
await expect(deleteNodesForFiles([ASSET_PATH])).resolves.toBeUndefined();
|
||
|
||
expect(
|
||
await count(`MATCH (n:File) WHERE n.filePath = '${ASSET_PATH}' RETURN count(n) AS c`),
|
||
).toBe(0);
|
||
expect(await count(`MATCH (e:${EMBEDDING_TABLE_NAME}) RETURN count(e) AS c`)).toBe(
|
||
embeddingsBefore,
|
||
);
|
||
}, 120_000);
|
||
|
||
it('deleteNodesForFile removes a fallback embedding owned by the File node', async () => {
|
||
const { deleteNodesForFile, executeQuery, executeWithReusedStatement } =
|
||
await import('../../src/core/lbug/lbug-adapter.js');
|
||
const { batchInsertEmbeddings } =
|
||
await import('../../src/core/embeddings/embedding-pipeline.js');
|
||
const count = async (cypher: string): Promise<number> => {
|
||
const rows = (await executeQuery(cypher)) as Array<{ c: number | bigint }>;
|
||
return Number(rows[0]?.c ?? 0);
|
||
};
|
||
|
||
const filePath = 'docs/singular.md';
|
||
const nodeId = `File:${filePath}`;
|
||
await executeQuery(
|
||
`CREATE (:File {id: '${nodeId}', name: 'singular.md', filePath: '${filePath}'})`,
|
||
);
|
||
await batchInsertEmbeddings(executeWithReusedStatement, [
|
||
{
|
||
nodeId,
|
||
chunkIndex: 0,
|
||
startLine: 1,
|
||
endLine: 1,
|
||
embedding: new Array(EMBEDDING_DIMS).fill(0),
|
||
},
|
||
]);
|
||
|
||
await expect(deleteNodesForFile(filePath)).resolves.toEqual({ deletedNodes: 1 });
|
||
expect(
|
||
await count(
|
||
`MATCH (e:${EMBEDDING_TABLE_NAME}) WHERE e.nodeId = '${nodeId}' RETURN count(e) AS c`,
|
||
),
|
||
).toBe(0);
|
||
}, 120_000);
|
||
});
|
||
});
|
||
|
||
/**
|
||
* Missing-embedding-table tolerance (FIX 4): a DB created without
|
||
* EMBEDDING_SCHEMA raises `Binder exception: Table CodeEmbedding does not
|
||
* exist.` (probe-recorded on @ladybugdb/core 0.18.0) on the join-delete.
|
||
* deleteNodesForFiles must tolerate exactly that one case — warn and keep
|
||
* going — instead of bricking every incremental run until `--force`, while
|
||
* the node-table deletes still complete. Own withTestLbugDB block: the
|
||
* DROP TABLE would poison the sibling suite's shared DB.
|
||
*/
|
||
withTestLbugDB('delete-nodes-missing-embedding-table', () => {
|
||
describe('deleteNodesForFiles without a CodeEmbedding table (FIX 4)', () => {
|
||
it('resolves, still deletes the node rows, and later statements keep working', async () => {
|
||
const { deleteNodesForFiles, executeQuery } =
|
||
await import('../../src/core/lbug/lbug-adapter.js');
|
||
const count = async (cypher: string): Promise<number> => {
|
||
const rows = (await executeQuery(cypher)) as Array<{ c: number | bigint }>;
|
||
return Number(rows[0]?.c ?? 0);
|
||
};
|
||
|
||
await executeQuery(
|
||
`CREATE (:Function {id: 'Function:src/a.ts:fnA:1', name: 'fnA', filePath: 'src/a.ts', startLine: 1, endLine: 3, isExported: true, content: '', description: ''})`,
|
||
);
|
||
await executeQuery(
|
||
`CREATE (:Function {id: 'Function:src/b.ts:fnB:1', name: 'fnB', filePath: 'src/b.ts', startLine: 1, endLine: 3, isExported: true, content: '', description: ''})`,
|
||
);
|
||
// Build-variant DB without the embedding schema.
|
||
await executeQuery(`DROP TABLE ${EMBEDDING_TABLE_NAME}`);
|
||
|
||
await expect(deleteNodesForFiles(['src/a.ts'])).resolves.toBeUndefined();
|
||
|
||
// The node delete completed despite the tolerated missing-table warn…
|
||
expect(
|
||
await count(`MATCH (n:Function) WHERE n.filePath = 'src/a.ts' RETURN count(n) AS c`),
|
||
).toBe(0);
|
||
// …the untouched file survives…
|
||
expect(
|
||
await count(`MATCH (n:Function) WHERE n.filePath = 'src/b.ts' RETURN count(n) AS c`),
|
||
).toBe(1);
|
||
// …and the connection stays healthy for subsequent batches.
|
||
await expect(deleteNodesForFiles(['src/b.ts'])).resolves.toBeUndefined();
|
||
expect(await count(`MATCH (n:Function) RETURN count(n) AS c`)).toBe(0);
|
||
}, 120_000);
|
||
});
|
||
});
|
||
|
||
/**
|
||
* VECTOR-extension gate for embedding-row DML (#2623).
|
||
*
|
||
* LadybugDB refuses EVERY mutation of a table carrying an HNSW index while
|
||
* the VECTOR extension is not loaded on the connection. The surgical
|
||
* incremental writeback's FIRST statement is `deleteNodesForFiles`' embedding
|
||
* join-delete, and nothing on that path loaded VECTOR until Phase 4 — so an
|
||
* incremental analyze over a DB that already had `code_embedding_idx` died
|
||
* with "Trying to delete from an index on table CodeEmbedding but its
|
||
* extension is not loaded".
|
||
*
|
||
* `ensureEmbeddingRowDmlSafe` is the seam that answers "is embedding-row DML
|
||
* legal right now?" before a single row is touched. Own withTestLbugDB block:
|
||
* these cases close and reopen the DB under a different extension-install
|
||
* policy, which would wreck the sibling suites' shared connection.
|
||
*
|
||
* The block also carries the FTS twins of the same seam (#2841) —
|
||
* `ensureFtsRowDmlSafe` and, through the public `dropFTSIndex`, the catalog
|
||
* read behind it. They live here rather than in a new block because they need
|
||
* the identical machinery: the policy-reopen helper above, and the
|
||
* SHOW_INDEXES interception below that is the only way to stage an unreadable
|
||
* catalog. Where the VECTOR gate may fall back to a load it does not strictly
|
||
* need, both FTS readers must fall CLOSED — see the cases themselves.
|
||
*/
|
||
withTestLbugDB('embedding-row-dml-vector-gate', (handle) => {
|
||
describe('ensureEmbeddingRowDmlSafe (#2623)', () => {
|
||
const FILE_A = 'src/gate-a.ts';
|
||
const FILE_B = 'src/gate-b.ts';
|
||
const nodeIdFor = (fp: string): string => `Function:${fp}:fn:1`;
|
||
|
||
/** Reopen the singleton connection under an explicit install policy. */
|
||
const reopenWithPolicy = async (policy: string | undefined): Promise<void> => {
|
||
const { initLbug, closeLbug } = await import('../../src/core/lbug/lbug-adapter.js');
|
||
await closeLbug();
|
||
if (policy === undefined) delete process.env.GITNEXUS_LBUG_EXTENSION_INSTALL;
|
||
else process.env.GITNEXUS_LBUG_EXTENSION_INSTALL = policy;
|
||
await initLbug(handle.dbPath);
|
||
};
|
||
|
||
const seedTwoFilesWithEmbeddings = async (): Promise<void> => {
|
||
const { executeQuery, executeWithReusedStatement } =
|
||
await import('../../src/core/lbug/lbug-adapter.js');
|
||
const { batchInsertEmbeddings } =
|
||
await import('../../src/core/embeddings/embedding-pipeline.js');
|
||
for (const fp of [FILE_A, FILE_B]) {
|
||
await executeQuery(
|
||
`CREATE (:Function {id: '${nodeIdFor(fp)}', name: 'fn', filePath: '${fp}', startLine: 1, endLine: 3, isExported: true, content: '', description: ''})`,
|
||
);
|
||
}
|
||
await batchInsertEmbeddings(
|
||
executeWithReusedStatement,
|
||
[FILE_A, FILE_B].map((fp) => ({
|
||
nodeId: nodeIdFor(fp),
|
||
chunkIndex: 0,
|
||
startLine: 1,
|
||
endLine: 3,
|
||
embedding: new Array(EMBEDDING_DIMS).fill(0.1),
|
||
contentHash: `hash-${fp}`,
|
||
})),
|
||
);
|
||
};
|
||
|
||
const embeddingCountFor = async (fp: string): Promise<number> => {
|
||
const { executeQuery } = await import('../../src/core/lbug/lbug-adapter.js');
|
||
const rows = (await executeQuery(
|
||
`MATCH (e:${EMBEDDING_TABLE_NAME}) WHERE e.nodeId = '${nodeIdFor(fp)}' RETURN count(e) AS c`,
|
||
)) as Array<{ c: number | bigint }>;
|
||
return Number(rows[0]?.c ?? 0);
|
||
};
|
||
|
||
const clearSeed = async (): Promise<void> => {
|
||
const { executeQuery } = await import('../../src/core/lbug/lbug-adapter.js');
|
||
await executeQuery(`MATCH (e:${EMBEDDING_TABLE_NAME}) DELETE e`);
|
||
await executeQuery(`MATCH (n:Function) DETACH DELETE n`);
|
||
};
|
||
|
||
afterEach(async () => {
|
||
await reopenWithPolicy(undefined);
|
||
// Teardown deletes embedding rows, so it is itself subject to #2623 once
|
||
// a case has built the index — load VECTOR before clearing.
|
||
const { ensureEmbeddingRowDmlSafe } = await import('../../src/core/lbug/lbug-adapter.js');
|
||
await ensureEmbeddingRowDmlSafe();
|
||
await clearSeed();
|
||
});
|
||
|
||
it('no vector index + VECTOR unavailable → safe, and the delete still works', async () => {
|
||
await seedTwoFilesWithEmbeddings();
|
||
await reopenWithPolicy('never');
|
||
const { ensureEmbeddingRowDmlSafe, deleteNodesForFiles } =
|
||
await import('../../src/core/lbug/lbug-adapter.js');
|
||
|
||
// No HNSW index was ever built, so there is nothing to gate on — the
|
||
// degraded path must NOT escalate needlessly.
|
||
await expect(ensureEmbeddingRowDmlSafe()).resolves.toBe(true);
|
||
await expect(deleteNodesForFiles([FILE_A])).resolves.toBeUndefined();
|
||
expect(await embeddingCountFor(FILE_A)).toBe(0);
|
||
expect(await embeddingCountFor(FILE_B)).toBe(1);
|
||
}, 120_000);
|
||
|
||
it('vector index present + VECTOR unavailable → blocked, and the raw delete throws', async () => {
|
||
await seedTwoFilesWithEmbeddings();
|
||
const { createVectorIndex } = await import('../../src/core/lbug/lbug-adapter.js');
|
||
const built = await createVectorIndex();
|
||
if (!built) return; // VECTOR not installable here — nothing to assert.
|
||
|
||
await reopenWithPolicy('never');
|
||
const { ensureEmbeddingRowDmlSafe, deleteNodesForFiles } =
|
||
await import('../../src/core/lbug/lbug-adapter.js');
|
||
|
||
// The gate must SEE the hazard…
|
||
await expect(ensureEmbeddingRowDmlSafe()).resolves.toBe(false);
|
||
// …and the hazard must be real: this is the exact #2623 failure.
|
||
await expect(deleteNodesForFiles([FILE_A])).rejects.toThrow(/extension is not loaded/);
|
||
// Nothing was destroyed by the refused statement.
|
||
expect(await embeddingCountFor(FILE_A)).toBe(1);
|
||
}, 120_000);
|
||
|
||
it('vector index present + VECTOR loadable → safe, delete works, index survives', async () => {
|
||
await seedTwoFilesWithEmbeddings();
|
||
const { createVectorIndex } = await import('../../src/core/lbug/lbug-adapter.js');
|
||
const built = await createVectorIndex();
|
||
if (!built) return; // VECTOR not installable here — nothing to assert.
|
||
|
||
// Reopen so the in-process "already loaded" latch cannot mask a missing
|
||
// load — this is the state a second `analyze` run actually starts from.
|
||
await reopenWithPolicy(undefined);
|
||
const { ensureEmbeddingRowDmlSafe, deleteNodesForFiles, executeQuery } =
|
||
await import('../../src/core/lbug/lbug-adapter.js');
|
||
|
||
await expect(ensureEmbeddingRowDmlSafe()).resolves.toBe(true);
|
||
await expect(deleteNodesForFiles([FILE_A])).resolves.toBeUndefined();
|
||
expect(await embeddingCountFor(FILE_A)).toBe(0);
|
||
expect(await embeddingCountFor(FILE_B)).toBe(1);
|
||
|
||
// The surgical path KEEPS its index (run-analyze relies on HNSW
|
||
// self-maintaining across insert/delete) — it must still be there.
|
||
const indexes = (await executeQuery('CALL SHOW_INDEXES() RETURN *')) as Array<{
|
||
table_name?: string;
|
||
index_type?: string;
|
||
}>;
|
||
expect(
|
||
indexes.some((r) => r.table_name === EMBEDDING_TABLE_NAME && r.index_type === 'HNSW'),
|
||
).toBe(true);
|
||
}, 120_000);
|
||
|
||
/**
|
||
* Run `run` with every `CALL SHOW_INDEXES()` on the writable connection
|
||
* forced to fail, passing every other statement through to the real engine
|
||
* and recording the SQL that was attempted.
|
||
*
|
||
* Forcing the read is the ONLY way to reach the gates' "could not prove
|
||
* anything" branch: `SHOW_INDEXES` is readable with no extension loaded, so
|
||
* a genuine unreadable catalog cannot be staged by configuration alone.
|
||
*/
|
||
const withUnreadableIndexCatalog = async (
|
||
run: (seen: readonly string[]) => Promise<void>,
|
||
): Promise<void> => {
|
||
const { default: lbug } = await import('@ladybugdb/core');
|
||
const originalQuery = lbug.Connection.prototype.query;
|
||
const seen: string[] = [];
|
||
const spy = vi.spyOn(lbug.Connection.prototype, 'query').mockImplementation(function (
|
||
this: unknown,
|
||
sql: string,
|
||
...rest: unknown[]
|
||
) {
|
||
seen.push(sql);
|
||
if (sql.includes('SHOW_INDEXES')) {
|
||
return Promise.reject(new Error('Catalog exception: forced by test'));
|
||
}
|
||
return originalQuery.call(this, sql, ...rest);
|
||
});
|
||
|
||
try {
|
||
await run(seen);
|
||
} finally {
|
||
spy.mockRestore();
|
||
}
|
||
};
|
||
|
||
it('catalog read fails → falls back to attempting the extension load (fail-safe)', async () => {
|
||
// The one branch where the gate cannot cheaply prove safety: SHOW_INDEXES
|
||
// itself errors. It must fall through to loadVectorExtension — in this
|
||
// environment the extension IS loadable, so the verdict is still `true`
|
||
// and DML proceeds safely despite the unreadable catalog.
|
||
await seedTwoFilesWithEmbeddings();
|
||
// Reopen so the module-level "already loaded" latch cannot let
|
||
// loadVectorExtension return true without issuing a LOAD statement.
|
||
await reopenWithPolicy('load-only');
|
||
const { ensureEmbeddingRowDmlSafe } = await import('../../src/core/lbug/lbug-adapter.js');
|
||
|
||
await withUnreadableIndexCatalog(async (seen) => {
|
||
await expect(ensureEmbeddingRowDmlSafe()).resolves.toBe(true);
|
||
// The catalog read was attempted and failed…
|
||
expect(seen.some((s) => s.includes('SHOW_INDEXES'))).toBe(true);
|
||
// …and the fallback really attempted the LOAD instead of guessing.
|
||
expect(seen.some((s) => s.toUpperCase().includes('LOAD'))).toBe(true);
|
||
});
|
||
}, 120_000);
|
||
|
||
/**
|
||
* The FTS twin of the case above (#2841). Same seam, opposite polarity:
|
||
* `ensureEmbeddingRowDmlSafe` may fall back to a load it does not strictly
|
||
* need, but `ensureFtsRowDmlSafe`'s only job is refusing an unsafe write, so
|
||
* an unprovable catalog must never be answered "no FTS index seen ⇒ safe".
|
||
*
|
||
* Both halves are asserted because either one alone is satisfiable by a
|
||
* broken gate: the verdict alone could come from a gate that never looked at
|
||
* the extension, and the LOAD alone could come from a gate that issued it
|
||
* and then returned `true` regardless.
|
||
*/
|
||
it('FTS twin: an unreadable catalog fails CLOSED — verdict blocked, and the FTS load is attempted (#2841)', async () => {
|
||
const { ensureFtsRowDmlSafe } = await import('../../src/core/lbug/lbug-adapter.js');
|
||
|
||
// Half 1 — `never` makes the extension provably unloadable on every host,
|
||
// so the verdict is deterministic: a fail-OPEN gate answers `true` here.
|
||
await reopenWithPolicy('never');
|
||
await withUnreadableIndexCatalog(async (seen) => {
|
||
await expect(ensureFtsRowDmlSafe()).resolves.toBe(false);
|
||
expect(seen.some((s) => s.includes('SHOW_INDEXES'))).toBe(true);
|
||
});
|
||
|
||
// Half 2 — refusing is not enough: the gate must TRY to make the write
|
||
// legal, or an unreadable catalog would escalate every run to a full
|
||
// rebuild on a machine where FTS loads perfectly.
|
||
//
|
||
// Staging that needs care. `initLbug` pre-loads FTS itself whenever the
|
||
// policy permits (lbug-adapter.ts), and the adapter latches the result,
|
||
// so a gate running after a successful init-time load issues nothing and
|
||
// the assertion would be unfalsifiable. So: reopen under `never` (init's
|
||
// pre-load is refused, latch stays clear), then relax the policy WITHOUT
|
||
// reopening — the gate resolves it from the environment at call time, so
|
||
// the LOAD that appears is unambiguously its own. `afterEach`'s
|
||
// reopenWithPolicy(undefined) restores the variable.
|
||
//
|
||
// Deliberately not asserting the verdict here: whether FTS actually
|
||
// loads is a property of the host; ATTEMPTING it is the contract.
|
||
await reopenWithPolicy('never');
|
||
process.env.GITNEXUS_LBUG_EXTENSION_INSTALL = 'load-only';
|
||
await withUnreadableIndexCatalog(async (seen) => {
|
||
await ensureFtsRowDmlSafe();
|
||
const isFtsLoad = (sql: string): boolean =>
|
||
/^\s*LOAD\s+EXTENSION\b/i.test(sql) &&
|
||
(/\bfts\b/i.test(sql) || /libfts\.lbug_extension/i.test(sql));
|
||
expect(seen.some(isFtsLoad)).toBe(true);
|
||
// …and it did not charge the caller a VECTOR load it never needed.
|
||
expect(seen.some((s) => /^\s*LOAD EXTENSION vector\b/i.test(s))).toBe(false);
|
||
});
|
||
}, 120_000);
|
||
|
||
/**
|
||
* `ftsIndexExistsInCatalog` is private; `dropFTSIndex` is the public surface
|
||
* that consumes it, and the H3 fix lives entirely in its unreadable-catalog
|
||
* branch. With the extension unloaded the DROP always fails the same way
|
||
* (`Catalog exception: function DROP_FTS_INDEX is not defined`), so the
|
||
* catalog read is the ONLY thing deciding whether the caller is told — which
|
||
* makes the two calls below a controlled pair on one connection.
|
||
*/
|
||
it('FTS twin: dropFTSIndex rejects (not resolves) when the extension is unloaded and the catalog is unreadable (#2841 H3)', async () => {
|
||
await reopenWithPolicy('never');
|
||
const { dropFTSIndex } = await import('../../src/core/lbug/lbug-adapter.js');
|
||
// A name no index carries: with a READABLE catalog this is provably
|
||
// "nothing to drop", which is exactly what the unreadable read must NOT
|
||
// be allowed to imply.
|
||
const ABSENT_INDEX = 'delete_nodes_2841_absent_fts';
|
||
|
||
// Control — catalog readable, index provably absent ⇒ benign no-op.
|
||
await expect(dropFTSIndex('File', ABSENT_INDEX)).resolves.toBeUndefined();
|
||
|
||
// Same call, same connection, same engine error; only the catalog read
|
||
// changes. Before the fix this ALSO resolved silently — reporting an
|
||
// unprovable catalog as "index absent" and handing the caller a drop that
|
||
// never happened, one DML statement before the #2841 crash.
|
||
await withUnreadableIndexCatalog(async (seen) => {
|
||
// The message must claim only what the run can prove. This branch is
|
||
// reached when the catalog is UNREADABLE, and the only path into it
|
||
// (`ensureFtsRowDmlSafe` waved the surgical plan through because the
|
||
// catalog showed no FTS index, then a later read failed) is one where
|
||
// the same run already established the opposite of "exists" — so
|
||
// asserting existence here would state a fabricated fact while
|
||
// failing the run.
|
||
await expect(dropFTSIndex('File', ABSENT_INDEX)).rejects.toThrow(
|
||
/FTS index '.*' on table File could not be verified as absent/,
|
||
);
|
||
expect(seen.some((s) => s.includes('DROP_FTS_INDEX'))).toBe(true);
|
||
expect(seen.some((s) => s.includes('SHOW_INDEXES'))).toBe(true);
|
||
});
|
||
}, 120_000);
|
||
});
|
||
});
|