GitNexus/gitnexus/test/integration/lbug-pool-stability.test.ts
Gergő Magyar cae60c2bd4
Some checks are pending
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Scorecard / Scorecard analysis (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
fix(mcp): reject corrupt query identities and refresh index metadata (#3571)
* fix(mcp): reject corrupt query identities and refresh index metadata

* fix(mcp): preserve reader and search integrity

Keep replaced readers unknown until matching metadata is published.
Defer shared-handle refresh until other consumers release the old graph
to avoid rebuilding connections against the same stale generation.

Propagate identity corruption after both hybrid search lanes settle,
so the sibling cannot query after its database session is released.
Select the regressions for the OS matrix and retain Python coverage
without optional Swift grammars.

Validation: 393 focused tests, nine native generation tests and ten
race executions across five fresh processes pass. Build and source/test
typechecks pass. The early full run passed 26,096 tests and failed eight:
seven reproduce on baseline; the mixed-source publication case passes
in clean reruns. Windows/macOS native execution remains for CI.

Graph analysis maps all edited production symbols; coverage remains
partial for the anonymous FTS test callback, manually reviewed.

Refs #3571

* fix(lbug): bump @ladybugdb/core to 0.21.2 to stop doubled rows after delete and re-copy

LadybugDB 0.21.0 and 0.21.1 return every row of a table that spans more
than one native vector twice once the incremental writeback has DETACH
DELETEd it and a COPY into an earlier table has run. NodeGroup::scan let a
morsel that ended before the group's first live row wrap an unsigned
subtraction and rescan to the end of the group (LadybugDB #1085, fixed in
0.21.2). No primary key error fires. The identity guard from #3442 stops
the run with "unexpected or duplicate ID", so a Swift edit that pulls every
importer into the write set failed on repos with more than 2048 structs.
Only unreleased main was affected: 1.6.12 ships 0.18.3.

Pin 0.21.2 in package.json, update the six @ladybugdb/core entries in the
lockfile, and move coreVersion in the vendored FTS manifest. The FTS
extension stays at 0.21.0 because no v0.21.1 or v0.21.2 artifact is
published; the pairing assertion and its unit tests pass.

Add a regression to the delete-nodes-for-files suite: 2049 structs with
CONTAINS edges are loaded, every file is deleted, the graph is loaded
again, and each id and edge must appear exactly once.

Verification (Linux, Node 22.18):
- lbug-delete-nodes-for-files: 11/11 on 0.21.2. With the 0.21.1 native
  packages swapped in, the new test fails with 4098 rows instead of 2049.
- Reporter recipe end to end on the built CLI: incremental analyze exits 0
  with no duplicate ids on a Swift-heavy fixture and on a 70-file Python
  fixture, with and without FTS. On 0.21.1 the Swift fixture exits 1.
- A 0.21.1-written database opens and reads under 0.21.2.
- prettier, eslint and typecheck:tests are clean on the changed files.
- The full lbug-db project had not finished when this was committed.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* test(fts): say what the Windows warm-session branch covers

On Windows the repair step calls poolClose, which removes the pool entry
before the in-place writer opens. The step therefore checks the backend's
cached capability state, not a live pool connection. The comment now says so
instead of implying the pool stays warm. No behavior change.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* revert: take the @ladybugdb/core 0.21.2 bump out of this PR

Reverts 0cdc3dfe3. The bump fixes LadybugDB 0.21.0 and 0.21.1 returning every
row of a large table twice after a delete and re-copy, but on Windows it
crashes the vendored FTS extension, which is still the v0.21.0 build: four of
eight Windows shards failed with exit code 3221225477 (0xC0000005) in tests
that load libfts and build an FTS index. All eight passed on 9d79092ab with
core 0.21.1. A Windows extension is only published for v0.21.0, not for
v0.21.1 or v0.21.2.

The bump moves to its own draft PR (#3578) until a matching Windows
extension exists. This PR returns to its original query-integrity and
index-freshness scope.

Restores gitnexus/package.json, the six @ladybugdb/core entries in
package-lock.json, the coreVersion in vendor/lbug-fts/manifest.json, and
removes the regression test that belongs with the bump. All four files are
identical to origin/main (1b4e3f0a8) after this commit.

Verification:
- git diff origin/main for the four files is empty.
- fts-artifact-coverage and vendored-extension-path unit tests pass (31),
  and assert-publish-fts-coverage reports core 0.21.1 paired with extension
  0.21.0.
- The graph check on the staged revert was partial, because it only deletes
  test code. The forward change was gated clean: 4 files, 6 symbols, all in
  that test, no affected processes.
- Windows CI has not run on this commit.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-14 15:06:26 +01:00

250 lines
9.6 KiB
TypeScript

/**
* Integration Tests: LadybugDB Connection Pool — Parallel Stability
*
* Tests concurrency fixes from PR #349:
* - Pre-warmed pool handles max parallel queries
* - Waiter queue drains under overload
* - Concurrent initLbug deduplication
* - stdout.write restoration after parallel operations
* - No connection leaks over sequential workloads
* - Atomic pool entry visibility (pool.set last)
* - Mixed query types at full concurrency
*
* Connection budget: LadybugDB's native mmap budget caps at ~56
* simultaneous Connection objects per process. Tests that only need
* a ready pool share a single 'shared-repo' init (8 connections) to
* stay well under the limit.
*/
import { it, expect, afterAll } from 'vitest';
import fs from 'node:fs/promises';
import os from 'node:os';
import path from 'node:path';
import lbug from '@ladybugdb/core';
import {
initLbug,
executeQuery,
executeParameterized,
closeLbug,
isLbugReady,
} from '../../src/mcp/core/lbug-adapter.js';
import { withTestLbugDB } from '../helpers/test-indexed-db.js';
import { createLbugDatabase } from '../../src/core/lbug/lbug-config.js';
import { closeQueryResults } from '../../src/core/lbug/query-result-utils.js';
const SEED_DATA = [
`CREATE (f:File {id: 'file:index.ts', name: 'index.ts', filePath: 'src/index.ts', content: ''})`,
`CREATE (fn:Function {id: 'func:main', name: 'main', filePath: 'src/index.ts', startLine: 1, endLine: 10, isExported: true, content: '', description: ''})`,
`CREATE (fn2:Function {id: 'func:helper', name: 'helper', filePath: 'src/utils.ts', startLine: 1, endLine: 5, isExported: true, content: '', description: ''})`,
`MATCH (a:Function), (b:Function)
WHERE a.id = 'func:main' AND b.id = 'func:helper'
CREATE (a)-[:CodeRelation {type: 'CALLS', confidence: 1.0, reason: 'direct', step: 0}]->(b)`,
];
// ─── Shared-pool tests: reuse one init across 5 tests (8 connections total) ──
withTestLbugDB(
'pool-stability',
(handle) => {
const REPO = 'shared-par';
let inited = false;
const ensurePool = async () => {
if (!inited) {
await initLbug(REPO, handle.dbPath);
inited = true;
}
};
afterAll(async () => {
try {
await closeLbug(REPO);
} catch {
/* best-effort */
}
});
it('8 simultaneous queries complete without crashes', async () => {
await ensurePool();
const queries = Array.from({ length: 8 }, () =>
executeQuery(REPO, 'MATCH (n:Function) RETURN n.name AS name'),
);
const results = await Promise.all(queries);
expect(results).toHaveLength(8);
for (const rows of results) {
expect(rows.length).toBeGreaterThanOrEqual(2);
}
});
it('12 parallel queries overflow into waiter queue and all complete', async () => {
await ensurePool();
const queries = Array.from({ length: 12 }, () =>
executeQuery(REPO, 'MATCH (n:Function) RETURN n.name AS name'),
);
const results = await Promise.all(queries);
expect(results).toHaveLength(12);
for (const rows of results) {
expect(rows.length).toBeGreaterThanOrEqual(2);
}
});
it('process.stdout.write is functional after init and parallel queries', async () => {
await ensurePool();
const queries = Array.from({ length: 8 }, () =>
executeQuery(REPO, 'MATCH (n:Function) RETURN n.name AS name'),
);
await Promise.all(queries);
// stdout.write should be the real function, not the silenced no-op
expect(typeof process.stdout.write).toBe('function');
});
it('50 sequential queries do not leak connections', async () => {
await ensurePool();
for (let i = 0; i < 50; i++) {
await executeQuery(REPO, 'MATCH (n:Function) RETURN n.name AS name');
}
// If connections leaked, these 8 parallel queries would exhaust the pool
const queries = Array.from({ length: 8 }, () =>
executeQuery(REPO, 'MATCH (n:Function) RETURN n.name AS name'),
);
const results = await Promise.all(queries);
expect(results).toHaveLength(8);
});
it('mixed executeQuery + executeParameterized at full concurrency', async () => {
await ensurePool();
const queries = [
executeQuery(REPO, 'MATCH (n:Function) RETURN n.name AS name'),
executeParameterized(
REPO,
'MATCH (n:Function) WHERE n.name = $name RETURN n.name AS name',
{ name: 'main' },
),
executeQuery(REPO, 'MATCH (n:Function) RETURN n.name AS name'),
executeParameterized(
REPO,
'MATCH (n:Function) WHERE n.name = $name RETURN n.name AS name',
{ name: 'helper' },
),
executeQuery(REPO, 'MATCH (n:Function) RETURN n.name AS name'),
executeParameterized(
REPO,
'MATCH (n:Function) WHERE n.name = $name RETURN n.name AS name',
{ name: 'main' },
),
executeQuery(REPO, 'MATCH (n:Function) RETURN n.name AS name'),
executeQuery(REPO, 'MATCH (n:Function) RETURN n.name AS name'),
];
const results = await Promise.all(queries);
expect(results).toHaveLength(8);
});
// ─── Regression tests for #308 / #314 / #347 ──────────────────────────
// The impact command's enrichment phase runs 3 concurrent queries via
// Promise.all (local-backend.ts:1415). Before PR #349's pool pre-warming,
// this triggered lazy createConnection → silenceStdout → SIGSEGV.
it('3 concurrent queries via Promise.all (impact enrichment pattern, #308/#314/#347)', async () => {
await ensurePool();
// Mirrors the exact Promise.all pattern from local-backend.ts:1415
// that caused SIGSEGV in issues #308, #314, #347 before pool pre-warming.
const [r1, r2, r3] = await Promise.all([
executeQuery(REPO, 'MATCH (n:Function) RETURN n.name AS name LIMIT 5').catch(() => []),
executeQuery(REPO, 'MATCH (n:Function) RETURN n.id AS id LIMIT 5').catch(() => []),
executeQuery(REPO, 'MATCH ()-[r:CodeRelation]->() RETURN r.type AS type LIMIT 5').catch(
() => [],
),
]);
expect(r1.length).toBeGreaterThanOrEqual(1);
expect(r2.length).toBeGreaterThanOrEqual(1);
expect(r3.length).toBeGreaterThanOrEqual(1);
});
// ─── Fresh-state tests: need their own init ──────────────────────────
it('concurrent initLbug calls for the same repoId deduplicate', async () => {
const promises = Array.from({ length: 6 }, () => initLbug('dedup-repo', handle.dbPath));
await Promise.all(promises);
expect(isLbugReady('dedup-repo')).toBe(true);
const rows = await executeQuery('dedup-repo', 'MATCH (n:Function) RETURN n.name AS name');
expect(rows.length).toBeGreaterThanOrEqual(2);
await closeLbug('dedup-repo');
});
it('pool entry is not visible until initLbug fully resolves', async () => {
const initPromise = initLbug('atomic-repo', handle.dbPath);
// Synchronously, pool should NOT be ready (pool.set is the last operation)
expect(isLbugReady('atomic-repo')).toBe(false);
await initPromise;
expect(isLbugReady('atomic-repo')).toBe(true);
await closeLbug('atomic-repo');
});
},
{
seed: SEED_DATA,
poolAdapter: true,
},
);
// Windows cannot atomically replace a file held by a native reader. The same
// shared-consumer refresh logic has portable coverage in lbug-pool-pinning.
it.skipIf(process.platform === 'win32')(
'keeps shared readers on one native generation until the other consumer releases it',
async () => {
const root = await fs.mkdtemp(path.join(os.tmpdir(), 'gn-shared-refresh-'));
const dbPath = path.join(root, 'lbug');
const nextPath = path.join(root, 'next-lbug');
const reader = 'generation-reader';
const holder = 'generation-holder';
async function seed(dbFile: string, name: string): Promise<void> {
const db = createLbugDatabase(lbug, dbFile);
let conn: lbug.Connection | undefined;
try {
await db.init();
conn = new lbug.Connection(db);
for (const query of [
'CREATE NODE TABLE Entry(name STRING, PRIMARY KEY(name))',
`CREATE (:Entry {name: '${name}'})`,
'CHECKPOINT',
]) {
const result = await conn.query(query);
try {
for (const item of Array.isArray(result) ? result : [result]) {
await item.getAll();
}
} finally {
await closeQueryResults(result);
}
}
} finally {
await conn?.close();
await db.close();
}
}
const read = (repoId: string) => executeQuery(repoId, 'MATCH (n:Entry) RETURN n.name AS name');
try {
await seed(dbPath, 'before');
await seed(nextPath, 'after');
await initLbug(reader, dbPath);
await initLbug(holder, dbPath);
expect(await read(reader)).toEqual([{ name: 'before' }]);
await fs.rename(nextPath, dbPath);
for (let i = 0; i < 3; i++) {
expect(await initLbug(reader, dbPath)).toBe(false);
expect(await read(reader)).toEqual([{ name: 'before' }]);
expect(await read(holder)).toEqual([{ name: 'before' }]);
}
await closeLbug(holder);
expect(await initLbug(reader, dbPath)).toBe(true);
expect(await read(reader)).toEqual([{ name: 'after' }]);
expect(await initLbug(reader, dbPath)).toBe(false);
} finally {
await closeLbug(holder);
await closeLbug(reader);
await fs.rm(root, { recursive: true, force: true });
}
},
120_000,
);