mirror of
https://github.com/abhigyanpatwari/GitNexus.git
synced 2026-10-28 06:22:22 +00:00
Some checks are pending
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Scorecard / Scorecard analysis (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
* fix(mcp): reject corrupt query identities and refresh index metadata * fix(mcp): preserve reader and search integrity Keep replaced readers unknown until matching metadata is published. Defer shared-handle refresh until other consumers release the old graph to avoid rebuilding connections against the same stale generation. Propagate identity corruption after both hybrid search lanes settle, so the sibling cannot query after its database session is released. Select the regressions for the OS matrix and retain Python coverage without optional Swift grammars. Validation: 393 focused tests, nine native generation tests and ten race executions across five fresh processes pass. Build and source/test typechecks pass. The early full run passed 26,096 tests and failed eight: seven reproduce on baseline; the mixed-source publication case passes in clean reruns. Windows/macOS native execution remains for CI. Graph analysis maps all edited production symbols; coverage remains partial for the anonymous FTS test callback, manually reviewed. Refs #3571 * fix(lbug): bump @ladybugdb/core to 0.21.2 to stop doubled rows after delete and re-copy LadybugDB 0.21.0 and 0.21.1 return every row of a table that spans more than one native vector twice once the incremental writeback has DETACH DELETEd it and a COPY into an earlier table has run. NodeGroup::scan let a morsel that ended before the group's first live row wrap an unsigned subtraction and rescan to the end of the group (LadybugDB #1085, fixed in 0.21.2). No primary key error fires. The identity guard from #3442 stops the run with "unexpected or duplicate ID", so a Swift edit that pulls every importer into the write set failed on repos with more than 2048 structs. Only unreleased main was affected: 1.6.12 ships 0.18.3. Pin 0.21.2 in package.json, update the six @ladybugdb/core entries in the lockfile, and move coreVersion in the vendored FTS manifest. The FTS extension stays at 0.21.0 because no v0.21.1 or v0.21.2 artifact is published; the pairing assertion and its unit tests pass. Add a regression to the delete-nodes-for-files suite: 2049 structs with CONTAINS edges are loaded, every file is deleted, the graph is loaded again, and each id and edge must appear exactly once. Verification (Linux, Node 22.18): - lbug-delete-nodes-for-files: 11/11 on 0.21.2. With the 0.21.1 native packages swapped in, the new test fails with 4098 rows instead of 2049. - Reporter recipe end to end on the built CLI: incremental analyze exits 0 with no duplicate ids on a Swift-heavy fixture and on a 70-file Python fixture, with and without FTS. On 0.21.1 the Swift fixture exits 1. - A 0.21.1-written database opens and reads under 0.21.2. - prettier, eslint and typecheck:tests are clean on the changed files. - The full lbug-db project had not finished when this was committed. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * test(fts): say what the Windows warm-session branch covers On Windows the repair step calls poolClose, which removes the pool entry before the in-place writer opens. The step therefore checks the backend's cached capability state, not a live pool connection. The comment now says so instead of implying the pool stays warm. No behavior change. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * revert: take the @ladybugdb/core 0.21.2 bump out of this PR Reverts0cdc3dfe3. The bump fixes LadybugDB 0.21.0 and 0.21.1 returning every row of a large table twice after a delete and re-copy, but on Windows it crashes the vendored FTS extension, which is still the v0.21.0 build: four of eight Windows shards failed with exit code 3221225477 (0xC0000005) in tests that load libfts and build an FTS index. All eight passed on9d79092abwith core 0.21.1. A Windows extension is only published for v0.21.0, not for v0.21.1 or v0.21.2. The bump moves to its own draft PR (#3578) until a matching Windows extension exists. This PR returns to its original query-integrity and index-freshness scope. Restores gitnexus/package.json, the six @ladybugdb/core entries in package-lock.json, the coreVersion in vendor/lbug-fts/manifest.json, and removes the regression test that belongs with the bump. All four files are identical to origin/main (1b4e3f0a8) after this commit. Verification: - git diff origin/main for the four files is empty. - fts-artifact-coverage and vendored-extension-path unit tests pass (31), and assert-publish-fts-coverage reports core 0.21.1 paired with extension 0.21.0. - The graph check on the staged revert was partial, because it only deletes test code. The forward change was gated clean: 4 files, 6 symbols, all in that test, no affected processes. - Windows CI has not run on this commit. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
250 lines
9.6 KiB
TypeScript
250 lines
9.6 KiB
TypeScript
/**
|
|
* Integration Tests: LadybugDB Connection Pool — Parallel Stability
|
|
*
|
|
* Tests concurrency fixes from PR #349:
|
|
* - Pre-warmed pool handles max parallel queries
|
|
* - Waiter queue drains under overload
|
|
* - Concurrent initLbug deduplication
|
|
* - stdout.write restoration after parallel operations
|
|
* - No connection leaks over sequential workloads
|
|
* - Atomic pool entry visibility (pool.set last)
|
|
* - Mixed query types at full concurrency
|
|
*
|
|
* Connection budget: LadybugDB's native mmap budget caps at ~56
|
|
* simultaneous Connection objects per process. Tests that only need
|
|
* a ready pool share a single 'shared-repo' init (8 connections) to
|
|
* stay well under the limit.
|
|
*/
|
|
import { it, expect, afterAll } from 'vitest';
|
|
import fs from 'node:fs/promises';
|
|
import os from 'node:os';
|
|
import path from 'node:path';
|
|
import lbug from '@ladybugdb/core';
|
|
import {
|
|
initLbug,
|
|
executeQuery,
|
|
executeParameterized,
|
|
closeLbug,
|
|
isLbugReady,
|
|
} from '../../src/mcp/core/lbug-adapter.js';
|
|
import { withTestLbugDB } from '../helpers/test-indexed-db.js';
|
|
import { createLbugDatabase } from '../../src/core/lbug/lbug-config.js';
|
|
import { closeQueryResults } from '../../src/core/lbug/query-result-utils.js';
|
|
|
|
const SEED_DATA = [
|
|
`CREATE (f:File {id: 'file:index.ts', name: 'index.ts', filePath: 'src/index.ts', content: ''})`,
|
|
`CREATE (fn:Function {id: 'func:main', name: 'main', filePath: 'src/index.ts', startLine: 1, endLine: 10, isExported: true, content: '', description: ''})`,
|
|
`CREATE (fn2:Function {id: 'func:helper', name: 'helper', filePath: 'src/utils.ts', startLine: 1, endLine: 5, isExported: true, content: '', description: ''})`,
|
|
`MATCH (a:Function), (b:Function)
|
|
WHERE a.id = 'func:main' AND b.id = 'func:helper'
|
|
CREATE (a)-[:CodeRelation {type: 'CALLS', confidence: 1.0, reason: 'direct', step: 0}]->(b)`,
|
|
];
|
|
|
|
// ─── Shared-pool tests: reuse one init across 5 tests (8 connections total) ──
|
|
|
|
withTestLbugDB(
|
|
'pool-stability',
|
|
(handle) => {
|
|
const REPO = 'shared-par';
|
|
let inited = false;
|
|
|
|
const ensurePool = async () => {
|
|
if (!inited) {
|
|
await initLbug(REPO, handle.dbPath);
|
|
inited = true;
|
|
}
|
|
};
|
|
|
|
afterAll(async () => {
|
|
try {
|
|
await closeLbug(REPO);
|
|
} catch {
|
|
/* best-effort */
|
|
}
|
|
});
|
|
|
|
it('8 simultaneous queries complete without crashes', async () => {
|
|
await ensurePool();
|
|
const queries = Array.from({ length: 8 }, () =>
|
|
executeQuery(REPO, 'MATCH (n:Function) RETURN n.name AS name'),
|
|
);
|
|
const results = await Promise.all(queries);
|
|
expect(results).toHaveLength(8);
|
|
for (const rows of results) {
|
|
expect(rows.length).toBeGreaterThanOrEqual(2);
|
|
}
|
|
});
|
|
|
|
it('12 parallel queries overflow into waiter queue and all complete', async () => {
|
|
await ensurePool();
|
|
const queries = Array.from({ length: 12 }, () =>
|
|
executeQuery(REPO, 'MATCH (n:Function) RETURN n.name AS name'),
|
|
);
|
|
const results = await Promise.all(queries);
|
|
expect(results).toHaveLength(12);
|
|
for (const rows of results) {
|
|
expect(rows.length).toBeGreaterThanOrEqual(2);
|
|
}
|
|
});
|
|
|
|
it('process.stdout.write is functional after init and parallel queries', async () => {
|
|
await ensurePool();
|
|
const queries = Array.from({ length: 8 }, () =>
|
|
executeQuery(REPO, 'MATCH (n:Function) RETURN n.name AS name'),
|
|
);
|
|
await Promise.all(queries);
|
|
// stdout.write should be the real function, not the silenced no-op
|
|
expect(typeof process.stdout.write).toBe('function');
|
|
});
|
|
|
|
it('50 sequential queries do not leak connections', async () => {
|
|
await ensurePool();
|
|
for (let i = 0; i < 50; i++) {
|
|
await executeQuery(REPO, 'MATCH (n:Function) RETURN n.name AS name');
|
|
}
|
|
// If connections leaked, these 8 parallel queries would exhaust the pool
|
|
const queries = Array.from({ length: 8 }, () =>
|
|
executeQuery(REPO, 'MATCH (n:Function) RETURN n.name AS name'),
|
|
);
|
|
const results = await Promise.all(queries);
|
|
expect(results).toHaveLength(8);
|
|
});
|
|
|
|
it('mixed executeQuery + executeParameterized at full concurrency', async () => {
|
|
await ensurePool();
|
|
const queries = [
|
|
executeQuery(REPO, 'MATCH (n:Function) RETURN n.name AS name'),
|
|
executeParameterized(
|
|
REPO,
|
|
'MATCH (n:Function) WHERE n.name = $name RETURN n.name AS name',
|
|
{ name: 'main' },
|
|
),
|
|
executeQuery(REPO, 'MATCH (n:Function) RETURN n.name AS name'),
|
|
executeParameterized(
|
|
REPO,
|
|
'MATCH (n:Function) WHERE n.name = $name RETURN n.name AS name',
|
|
{ name: 'helper' },
|
|
),
|
|
executeQuery(REPO, 'MATCH (n:Function) RETURN n.name AS name'),
|
|
executeParameterized(
|
|
REPO,
|
|
'MATCH (n:Function) WHERE n.name = $name RETURN n.name AS name',
|
|
{ name: 'main' },
|
|
),
|
|
executeQuery(REPO, 'MATCH (n:Function) RETURN n.name AS name'),
|
|
executeQuery(REPO, 'MATCH (n:Function) RETURN n.name AS name'),
|
|
];
|
|
const results = await Promise.all(queries);
|
|
expect(results).toHaveLength(8);
|
|
});
|
|
|
|
// ─── Regression tests for #308 / #314 / #347 ──────────────────────────
|
|
// The impact command's enrichment phase runs 3 concurrent queries via
|
|
// Promise.all (local-backend.ts:1415). Before PR #349's pool pre-warming,
|
|
// this triggered lazy createConnection → silenceStdout → SIGSEGV.
|
|
|
|
it('3 concurrent queries via Promise.all (impact enrichment pattern, #308/#314/#347)', async () => {
|
|
await ensurePool();
|
|
// Mirrors the exact Promise.all pattern from local-backend.ts:1415
|
|
// that caused SIGSEGV in issues #308, #314, #347 before pool pre-warming.
|
|
const [r1, r2, r3] = await Promise.all([
|
|
executeQuery(REPO, 'MATCH (n:Function) RETURN n.name AS name LIMIT 5').catch(() => []),
|
|
executeQuery(REPO, 'MATCH (n:Function) RETURN n.id AS id LIMIT 5').catch(() => []),
|
|
executeQuery(REPO, 'MATCH ()-[r:CodeRelation]->() RETURN r.type AS type LIMIT 5').catch(
|
|
() => [],
|
|
),
|
|
]);
|
|
expect(r1.length).toBeGreaterThanOrEqual(1);
|
|
expect(r2.length).toBeGreaterThanOrEqual(1);
|
|
expect(r3.length).toBeGreaterThanOrEqual(1);
|
|
});
|
|
|
|
// ─── Fresh-state tests: need their own init ──────────────────────────
|
|
|
|
it('concurrent initLbug calls for the same repoId deduplicate', async () => {
|
|
const promises = Array.from({ length: 6 }, () => initLbug('dedup-repo', handle.dbPath));
|
|
await Promise.all(promises);
|
|
expect(isLbugReady('dedup-repo')).toBe(true);
|
|
const rows = await executeQuery('dedup-repo', 'MATCH (n:Function) RETURN n.name AS name');
|
|
expect(rows.length).toBeGreaterThanOrEqual(2);
|
|
await closeLbug('dedup-repo');
|
|
});
|
|
|
|
it('pool entry is not visible until initLbug fully resolves', async () => {
|
|
const initPromise = initLbug('atomic-repo', handle.dbPath);
|
|
// Synchronously, pool should NOT be ready (pool.set is the last operation)
|
|
expect(isLbugReady('atomic-repo')).toBe(false);
|
|
await initPromise;
|
|
expect(isLbugReady('atomic-repo')).toBe(true);
|
|
await closeLbug('atomic-repo');
|
|
});
|
|
},
|
|
{
|
|
seed: SEED_DATA,
|
|
poolAdapter: true,
|
|
},
|
|
);
|
|
|
|
// Windows cannot atomically replace a file held by a native reader. The same
|
|
// shared-consumer refresh logic has portable coverage in lbug-pool-pinning.
|
|
it.skipIf(process.platform === 'win32')(
|
|
'keeps shared readers on one native generation until the other consumer releases it',
|
|
async () => {
|
|
const root = await fs.mkdtemp(path.join(os.tmpdir(), 'gn-shared-refresh-'));
|
|
const dbPath = path.join(root, 'lbug');
|
|
const nextPath = path.join(root, 'next-lbug');
|
|
const reader = 'generation-reader';
|
|
const holder = 'generation-holder';
|
|
|
|
async function seed(dbFile: string, name: string): Promise<void> {
|
|
const db = createLbugDatabase(lbug, dbFile);
|
|
let conn: lbug.Connection | undefined;
|
|
try {
|
|
await db.init();
|
|
conn = new lbug.Connection(db);
|
|
for (const query of [
|
|
'CREATE NODE TABLE Entry(name STRING, PRIMARY KEY(name))',
|
|
`CREATE (:Entry {name: '${name}'})`,
|
|
'CHECKPOINT',
|
|
]) {
|
|
const result = await conn.query(query);
|
|
try {
|
|
for (const item of Array.isArray(result) ? result : [result]) {
|
|
await item.getAll();
|
|
}
|
|
} finally {
|
|
await closeQueryResults(result);
|
|
}
|
|
}
|
|
} finally {
|
|
await conn?.close();
|
|
await db.close();
|
|
}
|
|
}
|
|
|
|
const read = (repoId: string) => executeQuery(repoId, 'MATCH (n:Entry) RETURN n.name AS name');
|
|
try {
|
|
await seed(dbPath, 'before');
|
|
await seed(nextPath, 'after');
|
|
await initLbug(reader, dbPath);
|
|
await initLbug(holder, dbPath);
|
|
expect(await read(reader)).toEqual([{ name: 'before' }]);
|
|
await fs.rename(nextPath, dbPath);
|
|
|
|
for (let i = 0; i < 3; i++) {
|
|
expect(await initLbug(reader, dbPath)).toBe(false);
|
|
expect(await read(reader)).toEqual([{ name: 'before' }]);
|
|
expect(await read(holder)).toEqual([{ name: 'before' }]);
|
|
}
|
|
await closeLbug(holder);
|
|
expect(await initLbug(reader, dbPath)).toBe(true);
|
|
expect(await read(reader)).toEqual([{ name: 'after' }]);
|
|
expect(await initLbug(reader, dbPath)).toBe(false);
|
|
} finally {
|
|
await closeLbug(holder);
|
|
await closeLbug(reader);
|
|
await fs.rm(root, { recursive: true, force: true });
|
|
}
|
|
},
|
|
120_000,
|
|
);
|