mirror of
https://github.com/abhigyanpatwari/GitNexus.git
synced 2026-08-28 05:25:25 +00:00
fix: batch query enrichment, bake FTS extension into CLI image, add FTS memory repro (#2108)
* perf(query): batch per-symbol process/cohesion/content lookups (N+1 -> 2-3) Port of the local-backend query-batching from gitnexus-enterprise PR #222 into the OSS local MCP backend. The query tool traced each matched symbol to its processes + cohesion (+ content) with up to 3N sequential pool round-trips; batch them into 2-3 'WHERE n.id IN $nodeIds' queries keyed back to each symbol by a prepended 'n.id AS nodeId' column. Output is identical: the aggregation loop is unchanged, iterates merged in the same order, and reads pre-fetched maps instead of issuing a query per symbol. Adaptations over a blind cherry-pick (would otherwise change output): - per-nodeId first-row community pick replaces the per-symbol LIMIT 1, so each symbol keeps its own community (not one for the whole batch); - batched rows regrouped to the originating merged item by nodeId so the JS-side RRF item.score still drives process ranking; - positional fallbacks shift +1 (process row[1..6], cohesion [1]/[2], content [1]); CodeRelation{type:...} relation form kept; IN-list chunked at 100 like the impact path. Adds a regression test asserting per-node community/content association (func:login keeps comm:auth; func:validate inherits no community). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(docker): bake LadybugDB FTS extension into the CLI/serve image The container runs `serve` under the default `load-only` extension policy (the read pool pins {policy:'load-only'}), so a runtime LOAD EXTENSION fts never INSTALLs. Dockerfile.cli copied the extension installer but never ran it, so the runtime user's HOME had no FTS extension: keyword search silently degraded (no FTS indexes written, ranking falls back to vector-only with only a warning field). Same class of footgun fixed for the Hub image in gitnexus-enterprise PR #222. Run install-duckdb-extension.mjs as the `node` user with the runtime HOME so INSTALL fts materializes the extension under $HOME/.lbdb/extension where the runtime LOAD resolves it offline. Pin ENV HOME=/home/node because Docker does not derive HOME from USER — without it the build-install and runtime-load would resolve different paths. Verified locally: INSTALL lands in $HOME/.lbdb/extension/0.17.0 and a fresh offline load-only `LOAD EXTENSION fts` resolves it. Dockerfile.web is unaffected (static frontend, no @ladybugdb backend). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(lbug): FTS evict->reload RSS repro + inert pool RSS tracing Settles the gitnexus-enterprise PR #222 root-cause hypothesis for OSS: does re-running LOAD EXTENSION fts on every pool evict->reload strand the native FTS arena (unbounded RSS growth in long-lived MCP serve), or does db.close() reclaim it (bounded by MAX_POOL_SIZE)? Static read could not decide — the native lbugjs.node binary documents no close->extension-unload contract. Adds gitnexus/scripts/bench/fts-evict-reload-rss.mjs: a NATIVE mode that reproduces the exact native sequence doInitLbug()+closeOne() perform (open Database -> Connection -> LOAD EXTENSION fts -> QUERY_FTS_INDEX -> close) across K self-built FTS fixtures, and a --via-pool mode that drives the real compiled pool (initLbug/executeParameterized/closeLbug) against an existing analyzed repo. Plus a behavior-neutral GITNEXUS_POOL_RSS_TRACE=1 stderr trace on pool init/close (stdout reserved for MCP JSON-RPC; single env read when disabled). RESULT (native, 24 and 40 cycles x 6 fixtures, --expose-gc): PLATEAU. RSS warms up to ~400 MB then flattens (40-cycle: +36 MB over cycles 1-10, +3 MB over 30-40; decelerating), not the linear climb a per-reload arena leak would produce (240 reloads x stranded arena = multi-GB). db.close() reclaims the FTS arena. The unbounded-leak hypothesis is NOT reproduced for the OSS path: the pool's LRU eviction + close-on-evict BOUNDS the footprint, which is exactly the protection the enterprise Hub supervisor lacked (it opened bridge DBs in-process without eviction -> 15 GB). => plan U4 (worker/process isolation) is NOT justified by this evidence; U1 + U2 are the only OSS-shared changes. Caveat: small fixtures + awaited close; a --via-pool run against a large analyzed repo over a long session is the production-faithful follow-up (instrumentation is in place for it). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(review): apply ce-code-review autofix feedback (#222 migration) Adversarial review found the U3 bench PLATEAU->no-leak conclusion was over-claimed from a 600-row fixture: a size-proportional FTS-arena leak would be sub-threshold at that scale. Strengthen the bench and make its verdict honest: - scale the fixture (--rows, UNWIND batch insert), probe ALL 5 FTS indexes in --via-pool (not 2 of 5), add a --no-await-close variant (the pool fire-and-forget close shape), and replace the absolute-delta gate with a SLOPE-DECELERATION 3-way verdict (PLATEAU / CLIMB / INCONCLUSIVE) plus step-discontinuity detection. At production-representative scale the synthetic runs are noisy/INCONCLUSIVE (deceleration argues against an UNBOUNDED leak but does not prove bounded), so plan U4 stays GATED on a --via-pool run against a real large analyzed repo -- not closed. - Dockerfile.cli: source the scratch-DB size from ENV GITNEXUS_LBUG_MAX_DB_SIZE (single source of truth) and add a build-time verify-only LOAD gate that fails the build on a HOME/extension-dir mismatch instead of silently degrading runtime keyword search. - install-duckdb-extension.mjs: additive verify-only mode (LOAD-only in a fresh process) + robust size parse; back-compatible with the runtime positional-size caller (validated). - tests: wire func:validate into a second process (proc:beta-flow) so the batched STEP_IN_PROCESS row[1..6] positional shift is exercised by a genuine multi-process symbol, and assert process ranking. No blast radius (75 seed-consuming tests pass). - pool-adapter.ts: trim the traceRss narrated-code comment (DoD 2.3). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(bench): classify a sustained sub-floor RSS slope as INCONCLUSIVE, not PLATEAU Tri-review P2: the FTS evict->reload verdict short-circuited to PLATEAU whenever secondHalfSlope < SUSTAIN_FLOOR, BEFORE the deceleration check — so a sustained (non-decelerating) linear leak below 0.5 MB/cycle was labeled PLATEAU ("no leak"), the label that would wrongly close plan U4. Extract median/slopeMbPerCycle/classifyVerdict into a pure, side-effect-free fts-rss-verdict.mjs (zero imports) so it is unit-testable without loading the native addon or running the bench, and fix the classifier: - epsilon-first gate: a truly flat tail (< 0.1 MB/cycle) is PLATEAU regardless of decelRatio (guards against over-correcting a real negative into INCONCLUSIVE); - a sustained sub-floor positive slope (>= epsilon, < floor, decelRatio >= 0.6) is INCONCLUSIVE — a slow creep RSS cannot distinguish from noise at this scale, so the honest label is "not resolved", never a clean PLATEAU; - the noise floor now scales with the WORKING-SET growth (peak-baseline), not the pre-DB baseline RSS (which is interpreter/addon overhead, larger in --via-pool mode, and would inflate the floor and HIDE leaks). Reconcile the stale "per-row-relative delta floor" docstring; add floor + decelRatio to the MACHINE line. New fts-rss-verdict.test.ts pins all label boundaries (flat->PLATEAU, sustained-sub-floor->INCONCLUSIVE, decelerated->PLATEAU, sustained-linear->CLIMB, step->INCONCLUSIVE, working-set floor, no import side effects). U1 does NOT add detection power for sub-floor leaks (RSS cannot attribute that magnitude) — it stops the false PLATEAU and routes that regime to the --via-pool run. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(query): signal partial/warning on a real enrichment failure (not benign missing-table) Tri-review P2: when a batched enrichment query (process/cohesion/content) threw, it was caught + logged and the chunk's symbols silently fell back to `definitions` with no signal — the caller could not tell "genuinely standalone" from "enrichment failed". Track an `enrichmentDegraded` flag in the three enrichment catch blocks and, at response build, compose a single `warning` (FTS-missing and/or the enrichment message, so neither overwrites the other) plus `partial: true`. Both fields are omitted on the clean path, so the success-path response shape is byte-identical. Crucially, the flag fires ONLY for a REAL failure (timeout / lock / native fault), NOT the benign "no Process/Community table" prepare error — a repo analyzed without processes/communities is a normal config, and firing `partial` on every such query would desensitize callers (isBenignMissingTableError gates it). New unit test test/unit/query-degraded-signal.test.ts (vi.mock pool-adapter, override hybrid search to feed one matched symbol, route STEP_IN_PROCESS -> throw): real failure -> warning+partial+symbol still returned; benign missing-table -> no signal; FTS-missing + enrichment failure -> both messages in one warning. Plus a success-path no-warning/no-partial assertion in the calltool integration test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
3a4247ec36
commit
288b96f3e5
10 changed files with 965 additions and 71 deletions
|
|
@ -67,6 +67,28 @@ COPY --from=builder --chown=node:node /app/gitnexus/vendor ./gitnexus/vendor
|
|||
# unreachable from $PATH.
|
||||
RUN ln -s /app/gitnexus/dist/cli/index.js /usr/local/bin/gitnexus
|
||||
|
||||
# Bake the LadybugDB FTS extension into the image so BM25 keyword search works
|
||||
# at runtime. The server runs the default `load-only` extension policy (the read
|
||||
# pool pins `{ policy: 'load-only' }`), so a runtime `LOAD EXTENSION fts` never
|
||||
# INSTALLs — the extension must already exist in the runtime user's HOME
|
||||
# extension dir, or every keyword search silently degrades (no FTS indexes are
|
||||
# written and ranking falls back to vector-only with only a `warning` field).
|
||||
# Run the installer as the `node` user with the SAME HOME the server runs under,
|
||||
# so `INSTALL fts` materializes the extension under `$HOME/.lbdb/extension` where
|
||||
# the runtime `LOAD` resolves it offline. `ENV HOME` is pinned because Docker
|
||||
# does not derive HOME from `USER`, so without it build-install and runtime-load
|
||||
# would resolve different paths. Requires network egress for the one-time
|
||||
# INSTALL; the build fails loudly if it cannot fetch the extension. The DB-size
|
||||
# default comes from GITNEXUS_LBUG_MAX_DB_SIZE (single source of truth, matches
|
||||
# the runtime) — it only sizes the throwaway scratch DB used to run INSTALL.
|
||||
# The second `--verify-only` step re-LOADs the extension in a FRESH process
|
||||
# under the same HOME, so a HOME/extension-dir mismatch fails the build here
|
||||
# rather than silently degrading keyword search to vector-only at runtime.
|
||||
ENV HOME=/home/node \
|
||||
GITNEXUS_LBUG_MAX_DB_SIZE=17179869184
|
||||
RUN su node -s /bin/sh -c "HOME=/home/node node /app/gitnexus/scripts/install-duckdb-extension.mjs fts" \
|
||||
&& su node -s /bin/sh -c "HOME=/home/node node /app/gitnexus/scripts/install-duckdb-extension.mjs fts --verify-only"
|
||||
|
||||
USER node
|
||||
|
||||
# The web UI defaults to http://localhost:4747 - keep that contract.
|
||||
|
|
|
|||
374
gitnexus/scripts/bench/fts-evict-reload-rss.mjs
Normal file
374
gitnexus/scripts/bench/fts-evict-reload-rss.mjs
Normal file
|
|
@ -0,0 +1,374 @@
|
|||
#!/usr/bin/env node
|
||||
// FTS evict→reload RSS repro (gitnexus-enterprise PR #222 / local U3).
|
||||
//
|
||||
// Settles ONE empirical question that no static read can answer: when a
|
||||
// LadybugDB database that has `LOAD EXTENSION fts` applied is closed and a
|
||||
// fresh one is opened + re-LOADed (the pool's evict→reload cycle), does the
|
||||
// native FTS arena get reclaimed by `db.close()` — or is it stranded, so RSS
|
||||
// climbs without bound over a long-lived MCP `serve` session?
|
||||
//
|
||||
// • PLATEAU across cycles → db.close() reclaims the FTS arena; the OSS pool's
|
||||
// footprint is bounded by MAX_POOL_SIZE (~5 live arenas). No unbounded leak;
|
||||
// the #222 worker-isolation rewrite (plan U4) is NOT justified for OSS.
|
||||
// • MONOTONIC CLIMB → the FTS arena is stranded per reopen; the user's
|
||||
// hypothesis holds and U4 (route FTS reads through a reclaimable worker) is
|
||||
// justified.
|
||||
//
|
||||
// SCOPE OF THE VERDICT (read before citing it). A per-reload FTS-arena leak
|
||||
// would be PROPORTIONAL to the index size. A small fixture therefore produces a
|
||||
// small per-cycle increment that an absolute threshold can read as PLATEAU even
|
||||
// when a production-scale graph would leak visibly. So:
|
||||
// - `--rows` controls fixture size; run it LARGE (tens of thousands) before
|
||||
// concluding "no leak". The default is deliberately not tiny.
|
||||
// - The verdict (in fts-rss-verdict.mjs) keys on slope DECELERATION, not total
|
||||
// delta, with a noise floor that scales with the working-set growth
|
||||
// (peak−baseline) so sensitivity tracks fixture/arena size — NOT the pre-DB
|
||||
// baseline RSS. A sustained sub-floor positive slope is INCONCLUSIVE (a slow
|
||||
// creep RSS can't distinguish from noise), never a clean PLATEAU.
|
||||
// - The PLATEAU verdict is only valid for the corpus size it was run at; the
|
||||
// output states that size. The production-faithful confirmation is a
|
||||
// `--via-pool` run against a real large analyzed repo over a long session.
|
||||
//
|
||||
// Two modes:
|
||||
// (default) NATIVE — reproduces the native sequence doInitLbug()+closeOne()
|
||||
// perform (open Database → new Connection → LOAD EXTENSION fts →
|
||||
// QUERY_FTS_INDEX → close), against K self-built FTS fixtures, with no
|
||||
// gitnexus build required. `--no-await-close` mirrors the pool's
|
||||
// fire-and-forget close instead of awaiting (the production close shape).
|
||||
// --via-pool <lbugPath> — drives the REAL gitnexus pool from compiled dist
|
||||
// (initLbug → executeParameterized → closeLbug) against an existing analyzed
|
||||
// repo, exercising the production path + the GITNEXUS_POOL_RSS_TRACE
|
||||
// instrumentation. Probes ALL FTS indexes the repo has. Forces an explicit
|
||||
// close+reinit each cycle. Run `node scripts/build.js` first so the dist
|
||||
// reflects the current pool-adapter (incl. the RSS trace).
|
||||
//
|
||||
// Run with --expose-gc so RSS excludes V8-heap noise:
|
||||
// node --expose-gc gitnexus/scripts/bench/fts-evict-reload-rss.mjs
|
||||
// node --expose-gc gitnexus/scripts/bench/fts-evict-reload-rss.mjs --rows 40000 --cycles 30
|
||||
// GITNEXUS_POOL_RSS_TRACE=1 node --expose-gc \
|
||||
// gitnexus/scripts/bench/fts-evict-reload-rss.mjs --via-pool /path/to/repo/.gitnexus/lbug
|
||||
//
|
||||
// Flags by mode: --rows/--repos/--read-write/--no-await-close apply to NATIVE
|
||||
// only; --cycles applies to both. VIA-POOL warns when a NATIVE-only flag is set.
|
||||
//
|
||||
// Memory benches are noisy. Default is 24 cycles; trust the TREND (slope /
|
||||
// first-third vs last-third), never a single delta. A flat trend at a LARGE
|
||||
// fixture is a real NEGATIVE result (no unbounded leak), not a failed run.
|
||||
|
||||
import { createRequire } from 'node:module';
|
||||
import os from 'node:os';
|
||||
import path from 'node:path';
|
||||
import fs from 'node:fs';
|
||||
// Pure verdict classifier (median, slopeMbPerCycle, classifyVerdict) lives in a
|
||||
// side-effect-free sibling module so it is unit-testable without loading the
|
||||
// native addon or running this bench. See fts-rss-verdict.mjs.
|
||||
import { classifyVerdict, median, slopeMbPerCycle } from './fts-rss-verdict.mjs';
|
||||
|
||||
const require = createRequire(import.meta.url);
|
||||
const lbugModule = require('@ladybugdb/core');
|
||||
const lbug = lbugModule.default ?? lbugModule;
|
||||
|
||||
const LBUG_MAX_DB_SIZE = 16 * 1024 * 1024 * 1024;
|
||||
|
||||
// ── args ──────────────────────────────────────────────────────────────────
|
||||
function argVal(flag, dflt) {
|
||||
const i = process.argv.indexOf(flag);
|
||||
return i >= 0 && process.argv[i + 1] ? process.argv[i + 1] : dflt;
|
||||
}
|
||||
const CYCLES = Math.max(6, parseInt(argVal('--cycles', '24'), 10) || 24);
|
||||
const REPOS = Math.max(1, parseInt(argVal('--repos', '6'), 10) || 6); // >5 mirrors LRU thrash
|
||||
// Fixture size. Default is large enough that a size-proportional leak would be
|
||||
// visible across cycles; raise it further before trusting a PLATEAU verdict.
|
||||
const ROWS = Math.max(100, parseInt(argVal('--rows', '8000'), 10) || 8000);
|
||||
const VIA_POOL = argVal('--via-pool', null);
|
||||
const READONLY = !process.argv.includes('--read-write');
|
||||
const AWAIT_CLOSE = !process.argv.includes('--no-await-close');
|
||||
|
||||
if (VIA_POOL) {
|
||||
// These flags are consumed only by NATIVE mode; warn rather than ignore
|
||||
// silently so a VIA-POOL run is not misread as honoring them.
|
||||
const ignored = ['--rows', '--repos', '--read-write', '--no-await-close'].filter((f) =>
|
||||
process.argv.includes(f),
|
||||
);
|
||||
if (ignored.length) {
|
||||
console.error(
|
||||
`[fts-rss] NOTE: ${ignored.join(', ')} apply to NATIVE mode only; ignored in --via-pool.`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
if (typeof global.gc !== 'function') {
|
||||
console.error(
|
||||
'[fts-rss] WARNING: run with --expose-gc for clean RSS samples ' +
|
||||
'(`node --expose-gc <thisfile>`). Continuing without forced GC — results are noisier.',
|
||||
);
|
||||
}
|
||||
|
||||
const gc = () => {
|
||||
if (typeof global.gc === 'function') {
|
||||
global.gc();
|
||||
global.gc();
|
||||
}
|
||||
};
|
||||
const rssMb = () => Math.round(process.memoryUsage().rss / (1024 * 1024));
|
||||
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
|
||||
|
||||
// ── fixture: a minimal FTS-bearing .lbug ────────────────────────────────────
|
||||
const WORDS = [
|
||||
'login auth session token user password validate verify credential',
|
||||
'parse tree syntax node grammar lexer token ast traversal visitor',
|
||||
'graph query cypher match relation node edge pattern aggregate index',
|
||||
'memory pool buffer arena allocate reclaim evict cache resident heap',
|
||||
'search rank score bm25 fts index stem porter keyword document corpus',
|
||||
'worker fork process spawn kill reclaim isolate native binding addon',
|
||||
];
|
||||
|
||||
function buildFixture(dir) {
|
||||
fs.mkdirSync(dir, { recursive: true });
|
||||
const dbPath = path.join(dir, 'fixture.lbug');
|
||||
const db = new lbug.Database(dbPath, 0, false, false, LBUG_MAX_DB_SIZE);
|
||||
const conn = new lbug.Connection(db);
|
||||
return (async () => {
|
||||
await conn.query('LOAD EXTENSION fts');
|
||||
await conn.query(
|
||||
'CREATE NODE TABLE Doc(id STRING, name STRING, content STRING, PRIMARY KEY(id))',
|
||||
);
|
||||
// Batch-insert via UNWIND so large fixtures (`--rows`) build in seconds
|
||||
// instead of one round-trip per row. The fixture size drives the per-arena
|
||||
// FTS allocation, which is what makes a size-proportional leak observable.
|
||||
const rows = [];
|
||||
for (let i = 0; i < ROWS; i++) {
|
||||
const w = WORDS[i % WORDS.length];
|
||||
const name = `sym_${i}`;
|
||||
const content = `${w} ${name} block number ${i} ${WORDS[(i + 3) % WORDS.length]}`;
|
||||
rows.push({ id: `doc:${i}`, name, content });
|
||||
}
|
||||
const INSERT_CHUNK = 2000;
|
||||
for (let i = 0; i < rows.length; i += INSERT_CHUNK) {
|
||||
const chunk = rows.slice(i, i + INSERT_CHUNK);
|
||||
const stmt = await conn.prepare(
|
||||
'UNWIND $rows AS r CREATE (:Doc {id: r.id, name: r.name, content: r.content})',
|
||||
);
|
||||
await conn.execute(stmt, { rows: chunk });
|
||||
}
|
||||
await conn.query(
|
||||
"CALL CREATE_FTS_INDEX('Doc', 'doc_fts', ['name', 'content'], stemmer := 'porter')",
|
||||
);
|
||||
await conn.close();
|
||||
await db.close();
|
||||
return dbPath;
|
||||
})();
|
||||
}
|
||||
|
||||
const QUERIES = ['login token', 'parse node', 'memory arena', 'search index', 'worker reclaim'];
|
||||
|
||||
// ── NATIVE mode ─────────────────────────────────────────────────────────────
|
||||
async function runNative() {
|
||||
const root = fs.mkdtempSync(path.join(os.tmpdir(), 'fts-rss-'));
|
||||
console.error(
|
||||
`[fts-rss] NATIVE: ${REPOS} fixtures × ${ROWS} rows × ${CYCLES} cycles ` +
|
||||
`(readOnly=${READONLY}, awaitClose=${AWAIT_CLOSE})`,
|
||||
);
|
||||
console.error(`[fts-rss] building ${REPOS} FTS fixture(s) under ${root} …`);
|
||||
|
||||
const srcDb = await buildFixture(path.join(root, 'src'));
|
||||
const repoPaths = [];
|
||||
for (let k = 0; k < REPOS; k++) {
|
||||
const dst = path.join(root, `repo-${k}`);
|
||||
fs.cpSync(path.dirname(srcDb), dst, { recursive: true });
|
||||
repoPaths.push(path.join(dst, 'fixture.lbug'));
|
||||
}
|
||||
|
||||
// Mirror the pool's evict→reload: each visit opens a FRESH Database, makes a
|
||||
// Connection, LOADs fts, runs an FTS query, then closes — no caching, so every
|
||||
// visit is a reload. K>5 amplifies the LRU-thrash signal the pool would see.
|
||||
const series = [];
|
||||
gc();
|
||||
await sleep(50);
|
||||
const baseline = rssMb();
|
||||
console.error(`[fts-rss] baseline RSS=${baseline}MB`);
|
||||
|
||||
for (let cycle = 0; cycle < CYCLES; cycle++) {
|
||||
for (let k = 0; k < REPOS; k++) {
|
||||
const db = new lbug.Database(repoPaths[k], 0, false, READONLY, LBUG_MAX_DB_SIZE);
|
||||
const conn = new lbug.Connection(db);
|
||||
try {
|
||||
await conn.query('LOAD EXTENSION fts'); // the per-reload re-LOAD under test
|
||||
const q = QUERIES[(cycle + k) % QUERIES.length];
|
||||
const res = await conn.query(
|
||||
`CALL QUERY_FTS_INDEX('Doc', 'doc_fts', '${q}') RETURN node.id AS id, score ORDER BY score DESC LIMIT 20`,
|
||||
);
|
||||
// Drain so the query actually materializes results.
|
||||
if (res && typeof res.getAll === 'function') await res.getAll();
|
||||
} catch (e) {
|
||||
console.error(`[fts-rss] query error (cycle ${cycle}, repo ${k}): ${e?.message || e}`);
|
||||
} finally {
|
||||
// AWAIT_CLOSE (default) is the best case for reclamation. --no-await-close
|
||||
// mirrors the pool's fire-and-forget close (closeOne: db.close().catch())
|
||||
// so a leak that only manifests without awaiting is not hidden.
|
||||
if (AWAIT_CLOSE) {
|
||||
try {
|
||||
await conn.close();
|
||||
await db.close();
|
||||
} catch {
|
||||
/* ignore */
|
||||
}
|
||||
} else {
|
||||
conn.close().catch(() => {});
|
||||
db.close().catch(() => {});
|
||||
}
|
||||
}
|
||||
}
|
||||
gc();
|
||||
// Longer settle when not awaiting close, so fire-and-forget native teardown
|
||||
// has a chance to complete before the RSS sample (avoids a false PLATEAU).
|
||||
await sleep(AWAIT_CLOSE ? 20 : 200);
|
||||
const rss = rssMb();
|
||||
series.push(rss);
|
||||
console.error(`[fts-rss] cycle ${String(cycle + 1).padStart(3)}/${CYCLES} rssMB=${rss}`);
|
||||
}
|
||||
|
||||
fs.rmSync(root, { recursive: true, force: true });
|
||||
return { baseline, series, corpus: `${REPOS}×${ROWS} rows, native, awaitClose=${AWAIT_CLOSE}` };
|
||||
}
|
||||
|
||||
// ── VIA-POOL mode (real gitnexus pool from compiled dist) ───────────────────
|
||||
async function runViaPool(lbugPath) {
|
||||
if (!fs.existsSync(lbugPath)) {
|
||||
console.error(`[fts-rss] --via-pool path not found: ${lbugPath}`);
|
||||
process.exit(2);
|
||||
}
|
||||
// Compiled dist is required (the pool pulls the native addon + many modules).
|
||||
const distUrl = new URL('../../dist/core/lbug/pool-adapter.js', import.meta.url);
|
||||
let pool;
|
||||
try {
|
||||
pool = await import(distUrl.href);
|
||||
} catch (e) {
|
||||
console.error(
|
||||
`[fts-rss] could not import compiled pool-adapter (${e?.message}). ` +
|
||||
`Run \`node scripts/build.js\` first, or use NATIVE mode.`,
|
||||
);
|
||||
process.exit(2);
|
||||
}
|
||||
const { initLbug, executeParameterized, closeLbug } = pool;
|
||||
console.error(
|
||||
`[fts-rss] VIA-POOL on ${lbugPath} × ${CYCLES} cycles ` +
|
||||
`(explicit closeLbug+initLbug per cycle = forced evict→reload)`,
|
||||
);
|
||||
|
||||
// Probe ALL FTS indexes the analyzed graph carries (mirrors fts-schema.ts
|
||||
// FTS_INDEXES) so the per-cycle FTS arena load matches production, not a
|
||||
// 2-of-5 subset that would understate it.
|
||||
const FTS_INDEXES = [
|
||||
{ table: 'File', indexName: 'file_fts' },
|
||||
{ table: 'Function', indexName: 'function_fts' },
|
||||
{ table: 'Class', indexName: 'class_fts' },
|
||||
{ table: 'Method', indexName: 'method_fts' },
|
||||
{ table: 'Interface', indexName: 'interface_fts' },
|
||||
];
|
||||
|
||||
const series = [];
|
||||
gc();
|
||||
const baseline = rssMb();
|
||||
console.error(`[fts-rss] baseline RSS=${baseline}MB`);
|
||||
|
||||
for (let cycle = 0; cycle < CYCLES; cycle++) {
|
||||
try {
|
||||
await initLbug(lbugPath, lbugPath);
|
||||
const q = QUERIES[cycle % QUERIES.length];
|
||||
for (const { table, indexName } of FTS_INDEXES) {
|
||||
await executeParameterized(
|
||||
lbugPath,
|
||||
`CALL QUERY_FTS_INDEX('${table}', '${indexName}', $q) RETURN node.id AS id, score ORDER BY score DESC LIMIT 20`,
|
||||
{ q },
|
||||
).catch(() => []); // index may not exist for this graph — that's fine
|
||||
}
|
||||
await closeLbug(lbugPath); // force eviction → next cycle reopens + re-LOADs fts
|
||||
} catch (e) {
|
||||
console.error(`[fts-rss] pool cycle ${cycle} error: ${e?.message || e}`);
|
||||
}
|
||||
gc();
|
||||
// closeLbug fires a fire-and-forget native close (pool closeOne:
|
||||
// db.close().catch()), so settle longer than NATIVE's awaited close to let
|
||||
// native teardown finish before sampling — else a real leak reads PLATEAU.
|
||||
await sleep(200);
|
||||
const rss = rssMb();
|
||||
series.push(rss);
|
||||
console.error(`[fts-rss] cycle ${String(cycle + 1).padStart(3)}/${CYCLES} rssMB=${rss}`);
|
||||
}
|
||||
await closeLbug().catch(() => {});
|
||||
return { baseline, series, corpus: `via-pool ${path.basename(path.dirname(lbugPath))}` };
|
||||
}
|
||||
|
||||
// ── verdict ─────────────────────────────────────────────────────────────────
|
||||
function verdict({ baseline, series, corpus }) {
|
||||
const third = Math.max(1, Math.floor(series.length / 3));
|
||||
const firstMed = median(series.slice(0, third));
|
||||
const lastMed = median(series.slice(-third));
|
||||
const delta = lastMed - firstMed;
|
||||
const slope = slopeMbPerCycle(series);
|
||||
|
||||
// All label logic lives in the pure, unit-tested classifier (fts-rss-verdict.mjs):
|
||||
// epsilon-first flat→PLATEAU, decelerated→PLATEAU, sustained-sub-floor→INCONCLUSIVE,
|
||||
// ≥floor sustained→CLIMB, step→INCONCLUSIVE; floor scales with the working-set
|
||||
// growth (peak−baseline), not the pre-DB baseline RSS.
|
||||
const {
|
||||
verdict: label,
|
||||
firstHalfSlope,
|
||||
secondHalfSlope,
|
||||
decelRatio,
|
||||
floor,
|
||||
stepDiscontinuity,
|
||||
maxJump,
|
||||
peak,
|
||||
} = classifyVerdict(series, baseline);
|
||||
|
||||
console.log('\n==================== FTS evict→reload RSS verdict ====================');
|
||||
console.log(`corpus: ${corpus}`);
|
||||
console.log(`samples (MB): ${series.join(' ')}`);
|
||||
console.log(
|
||||
`baseline=${baseline} firstThirdMed=${firstMed} lastThirdMed=${lastMed} delta=${delta}MB ` +
|
||||
`peak=${peak} overallSlope=${slope.toFixed(2)} firstHalfSlope=${firstHalfSlope.toFixed(2)} ` +
|
||||
`secondHalfSlope=${secondHalfSlope.toFixed(2)}MB/cycle floor=${floor.toFixed(2)} decelRatio=${decelRatio.toFixed(2)} ` +
|
||||
`maxJump=${maxJump}MB step=${stepDiscontinuity} cycles=${series.length}`,
|
||||
);
|
||||
if (label === 'CLIMB') {
|
||||
console.log(
|
||||
'VERDICT: CLIMB — the per-cycle increment is SUSTAINED (second-half slope ≈ first-half),\n' +
|
||||
' i.e. RSS rises ~linearly with no decay. The native FTS arena is NOT reclaimed\n' +
|
||||
' by db.close(); the leak is real over a long-lived session.\n' +
|
||||
' → plan U4 (worker/process isolation of the FTS read path) is JUSTIFIED.',
|
||||
);
|
||||
} else if (label === 'PLATEAU') {
|
||||
console.log(
|
||||
`VERDICT: PLATEAU at this corpus (${corpus}) — the per-cycle increment DECAYS to flat\n` +
|
||||
' (second-half slope below the noise floor). db.close() reclaims the FTS arena;\n' +
|
||||
' footprint is bounded (and the pool further caps it at MAX_POOL_SIZE). No\n' +
|
||||
' unbounded leak. Caveat: synthetic fixture — confirm with a --via-pool run\n' +
|
||||
' against a real large analyzed repo before fully closing plan U4.',
|
||||
);
|
||||
} else {
|
||||
console.log(
|
||||
`VERDICT: INCONCLUSIVE at this corpus (${corpus}) — the run is noisy (step discontinuity)\n` +
|
||||
' or still decelerating without reaching flat, so neither a clean PLATEAU nor a\n' +
|
||||
' sustained linear CLIMB can be asserted. NATIVE synthetic runs do not resolve\n' +
|
||||
' this reliably at scale. The definitive test is a --via-pool run against a real\n' +
|
||||
' large analyzed repo over many cycles (with GITNEXUS_POOL_RSS_TRACE=1). Plan U4\n' +
|
||||
' stays GATED — neither closed nor built on this evidence.',
|
||||
);
|
||||
}
|
||||
console.log(
|
||||
`MACHINE: ${JSON.stringify({ mode: VIA_POOL ? 'via-pool' : 'native', corpus, baseline, firstMed, lastMed, delta, overallSlope: Number(slope.toFixed(3)), firstHalfSlope: Number(firstHalfSlope.toFixed(3)), secondHalfSlope: Number(secondHalfSlope.toFixed(3)), floor: Number(floor.toFixed(3)), decelRatio: Number(decelRatio.toFixed(3)), maxJump, stepDiscontinuity, peak, cycles: series.length, verdict: label })}`,
|
||||
);
|
||||
console.log('=====================================================================\n');
|
||||
}
|
||||
|
||||
// ── main ────────────────────────────────────────────────────────────────────
|
||||
(async () => {
|
||||
const result = VIA_POOL ? await runViaPool(VIA_POOL) : await runNative();
|
||||
verdict(result);
|
||||
process.exit(0);
|
||||
})().catch((e) => {
|
||||
console.error('[fts-rss] fatal:', e?.stack || e);
|
||||
process.exit(1);
|
||||
});
|
||||
105
gitnexus/scripts/bench/fts-rss-verdict.mjs
Normal file
105
gitnexus/scripts/bench/fts-rss-verdict.mjs
Normal file
|
|
@ -0,0 +1,105 @@
|
|||
// Pure, side-effect-free verdict classifier for the FTS evict→reload RSS bench
|
||||
// (fts-evict-reload-rss.mjs). Extracted so it can be unit-tested WITHOUT importing
|
||||
// the native LadybugDB addon or running the bench — this module has zero imports
|
||||
// and zero module-scope side effects. Do not add imports or top-level statements.
|
||||
//
|
||||
// The discriminant between a real leak and allocator warmup is SLOPE DECELERATION,
|
||||
// not total delta. A true per-reload leak (stranded FTS arena) rises ~linearly:
|
||||
// the second-half slope stays ≈ the first-half slope. Allocator working-set warmup
|
||||
// rises then flattens: the second-half slope decays to a fraction of the first.
|
||||
//
|
||||
// Thresholds:
|
||||
// EPSILON (~0.1 MB/cycle) — below this the tail is effectively flat (no leak).
|
||||
// SUSTAIN_FLOOR (0.5 MB/cycle) — the base noise floor.
|
||||
// The floor SCALES with the working-set growth (peak − baseline), NOT the pre-DB
|
||||
// `baseline` RSS: baseline is interpreter/addon overhead (and is LARGER in
|
||||
// --via-pool mode), so a baseline-keyed floor would inflate and HIDE leaks. A
|
||||
// bigger fixture has a bigger arena and bigger per-cycle noise, so the floor
|
||||
// rises with the working set: floor = SUSTAIN_FLOOR · max(1, (peak−baseline)/REF).
|
||||
|
||||
export const EPSILON_MB_PER_CYCLE = 0.1;
|
||||
export const SUSTAIN_FLOOR = 0.5;
|
||||
// Reference working-set (MB) at which the floor equals SUSTAIN_FLOOR; the floor
|
||||
// scales up linearly for larger arenas. ~200 MB ≈ a small FTS fixture's footprint.
|
||||
export const FLOOR_REF_WORKINGSET_MB = 200;
|
||||
|
||||
export function median(xs) {
|
||||
const s = [...xs].sort((a, b) => a - b);
|
||||
const m = Math.floor(s.length / 2);
|
||||
return s.length % 2 ? s[m] : Math.round((s[m - 1] + s[m]) / 2);
|
||||
}
|
||||
|
||||
export function slopeMbPerCycle(series) {
|
||||
// Least-squares slope of rss vs cycle index.
|
||||
const n = series.length;
|
||||
if (n < 2) return 0;
|
||||
const xs = series.map((_, i) => i);
|
||||
const xMean = xs.reduce((a, b) => a + b, 0) / n;
|
||||
const yMean = series.reduce((a, b) => a + b, 0) / n;
|
||||
let num = 0;
|
||||
let den = 0;
|
||||
for (let i = 0; i < n; i++) {
|
||||
num += (xs[i] - xMean) * (series[i] - yMean);
|
||||
den += (xs[i] - xMean) ** 2;
|
||||
}
|
||||
return den === 0 ? 0 : num / den;
|
||||
}
|
||||
|
||||
/**
|
||||
* Classify an RSS-per-cycle series into PLATEAU / CLIMB / INCONCLUSIVE.
|
||||
* Pure: no I/O, no globals. `baseline` is the pre-DB RSS; `peak` defaults to the
|
||||
* series max. Returns the label plus the diagnostics the bench prints.
|
||||
*/
|
||||
export function classifyVerdict(series, baseline, peak = Math.max(...series)) {
|
||||
const cycles = series.length;
|
||||
const half = Math.max(1, Math.floor(cycles / 2));
|
||||
const firstHalfSlope = slopeMbPerCycle(series.slice(0, half));
|
||||
const secondHalfSlope = slopeMbPerCycle(series.slice(-half));
|
||||
const decelRatio = secondHalfSlope / Math.max(firstHalfSlope, 1e-9);
|
||||
|
||||
// Step discontinuity: a single cycle-to-cycle jump far larger than the typical
|
||||
// per-cycle delta — a one-time allocator/arena reservation (then flat), not a
|
||||
// per-reload leak, but a noisy run we won't claim a clean result on.
|
||||
const deltas = series.slice(1).map((v, i) => v - series[i]);
|
||||
const absDeltas = deltas.map(Math.abs).sort((a, b) => a - b);
|
||||
const medAbsDelta = absDeltas.length ? absDeltas[Math.floor(absDeltas.length / 2)] : 0;
|
||||
const maxJump = deltas.length ? Math.max(...deltas) : 0;
|
||||
const stepDiscontinuity = maxJump > Math.max(30, 5 * Math.max(medAbsDelta, 1));
|
||||
|
||||
// Working-set-scaled floor (see header). Guard against a negative working set.
|
||||
const workingSet = Math.max(0, peak - baseline);
|
||||
const floor = SUSTAIN_FLOOR * Math.max(1, workingSet / FLOOR_REF_WORKINGSET_MB);
|
||||
|
||||
const SUSTAINED = 0.6; // decelRatio at/above which the tail is "not decaying"
|
||||
let verdict;
|
||||
if (stepDiscontinuity) {
|
||||
verdict = 'INCONCLUSIVE';
|
||||
} else if (secondHalfSlope < EPSILON_MB_PER_CYCLE) {
|
||||
// Effectively flat — no leak, regardless of decelRatio (a flat-from-start run
|
||||
// has decelRatio ≈ 1 but is still PLATEAU). This gate is what keeps a true
|
||||
// negative from being over-corrected into INCONCLUSIVE.
|
||||
verdict = 'PLATEAU';
|
||||
} else if (secondHalfSlope >= floor) {
|
||||
// Tail is still substantial: sustained → real leak; decelerating → unresolved.
|
||||
verdict = decelRatio >= SUSTAINED ? 'CLIMB' : 'INCONCLUSIVE';
|
||||
} else if (decelRatio < SUSTAINED) {
|
||||
// Below the floor AND decelerating — warmup converged toward flat → PLATEAU.
|
||||
verdict = 'PLATEAU';
|
||||
} else {
|
||||
// Below the floor but SUSTAINED — a slow steady creep RSS can't distinguish
|
||||
// from noise at this scale. The honest label is "not resolved", NEVER a clean
|
||||
// PLATEAU ("no leak"). This is the headline tri-review fix.
|
||||
verdict = 'INCONCLUSIVE';
|
||||
}
|
||||
|
||||
return {
|
||||
verdict,
|
||||
firstHalfSlope,
|
||||
secondHalfSlope,
|
||||
decelRatio,
|
||||
floor,
|
||||
stepDiscontinuity,
|
||||
maxJump,
|
||||
peak,
|
||||
};
|
||||
}
|
||||
|
|
@ -14,7 +14,7 @@ function parseLbugMaxDbSize(raw) {
|
|||
return Math.floor(parsed);
|
||||
}
|
||||
|
||||
async function installDuckDbExtension(extensionName) {
|
||||
async function installDuckDbExtension(extensionName, verifyOnly = false) {
|
||||
if (!extensionName || !EXTENSION_NAME_PATTERN.test(extensionName)) {
|
||||
throw new Error(`Invalid DuckDB extension name: ${extensionName ?? '<missing>'}`);
|
||||
}
|
||||
|
|
@ -22,9 +22,11 @@ async function installDuckDbExtension(extensionName) {
|
|||
const require = createRequire(import.meta.url);
|
||||
const lbugModule = require('@ladybugdb/core');
|
||||
const lbug = lbugModule.default ?? lbugModule;
|
||||
const lbugMaxDbSize = parseLbugMaxDbSize(
|
||||
process.argv[3] ?? process.env.GITNEXUS_LBUG_MAX_DB_SIZE,
|
||||
);
|
||||
// argv[3] is the optional positional size; ignore it when it is actually a
|
||||
// flag token (e.g. `--verify-only`) and fall back to the env default.
|
||||
const sizeArg =
|
||||
process.argv[3] && !process.argv[3].startsWith('--') ? process.argv[3] : undefined;
|
||||
const lbugMaxDbSize = parseLbugMaxDbSize(sizeArg ?? process.env.GITNEXUS_LBUG_MAX_DB_SIZE);
|
||||
|
||||
const tmpDir = await fs.mkdtemp(path.join(os.tmpdir(), 'gitnexus-ext-install-'));
|
||||
const dbPath = path.join(tmpDir, 'install.lbug');
|
||||
|
|
@ -34,7 +36,18 @@ async function installDuckDbExtension(extensionName) {
|
|||
try {
|
||||
db = new lbug.Database(dbPath, 0, false, false, lbugMaxDbSize);
|
||||
conn = new lbug.Connection(db);
|
||||
await conn.query(`INSTALL ${extensionName}`);
|
||||
if (verifyOnly) {
|
||||
// Prove a previously-baked extension is resolvable by a FRESH process
|
||||
// under the current HOME (the runtime `LOAD EXTENSION` path) — no INSTALL,
|
||||
// no network. Used as a Docker build-time gate so a HOME/extension-dir
|
||||
// mismatch fails the build instead of silently degrading search at runtime.
|
||||
await conn.query(`LOAD EXTENSION ${extensionName}`);
|
||||
console.log(
|
||||
`[install-ext] LOAD-only verify OK for '${extensionName}' (HOME=${process.env.HOME})`,
|
||||
);
|
||||
} else {
|
||||
await conn.query(`INSTALL ${extensionName}`);
|
||||
}
|
||||
} finally {
|
||||
if (conn) await conn.close().catch(() => {});
|
||||
if (db) await db.close().catch(() => {});
|
||||
|
|
@ -42,7 +55,10 @@ async function installDuckDbExtension(extensionName) {
|
|||
}
|
||||
}
|
||||
|
||||
installDuckDbExtension(process.argv[2] ?? process.env.GITNEXUS_LBUG_EXTENSION_NAME).catch((err) => {
|
||||
installDuckDbExtension(
|
||||
process.argv[2] ?? process.env.GITNEXUS_LBUG_EXTENSION_NAME,
|
||||
process.argv.includes('--verify-only'),
|
||||
).catch((err) => {
|
||||
console.error(err instanceof Error ? (err.stack ?? err.message) : String(err));
|
||||
process.exitCode = 1;
|
||||
});
|
||||
|
|
|
|||
|
|
@ -103,6 +103,19 @@ const IDLE_TIMEOUT_MS = 5 * 60 * 1000; // 5 minutes
|
|||
/** Max connections per repo (caps concurrent queries per repo) */
|
||||
const MAX_CONNS_PER_REPO = 8;
|
||||
|
||||
// Behavior-neutral RSS tracing for the FTS evict→reload memory repro
|
||||
// (gitnexus/scripts/bench/fts-evict-reload-rss.mjs). Two invariants keep it safe
|
||||
// in the pool init/close hot path: it writes ONLY to stderr (stdout is the MCP
|
||||
// JSON-RPC channel), and the GITNEXUS_POOL_RSS_TRACE gate makes it a no-op — one
|
||||
// env-var compare per call, nothing else — unless a harness explicitly enables it.
|
||||
function traceRss(event: 'init' | 'close', repoId: string): void {
|
||||
if (process.env.GITNEXUS_POOL_RSS_TRACE !== '1') return;
|
||||
const rssMb = Math.round(process.memoryUsage().rss / (1024 * 1024));
|
||||
process.stderr.write(
|
||||
`[pool-rss] ${event} repo=${repoId} pool=${pool.size} dbCache=${dbCache.size} rssMB=${rssMb}\n`,
|
||||
);
|
||||
}
|
||||
|
||||
let idleTimer: ReturnType<typeof setInterval> | null = null;
|
||||
|
||||
// Stdout-capture state lives in `gitnexus/src/mcp/stdio-capture.ts` — a leaf
|
||||
|
|
@ -240,6 +253,8 @@ function closeOne(repoId: string): void {
|
|||
// Isolate listener failures — teardown must complete.
|
||||
}
|
||||
}
|
||||
|
||||
traceRss('close', repoId);
|
||||
}
|
||||
|
||||
/**
|
||||
|
|
@ -611,6 +626,7 @@ async function doInitLbug(repoId: string, dbPath: string): Promise<void> {
|
|||
closed: false,
|
||||
});
|
||||
ensureIdleTimer();
|
||||
traceRss('init', repoId);
|
||||
}
|
||||
|
||||
/**
|
||||
|
|
@ -673,6 +689,7 @@ export async function initLbugWithDb(
|
|||
closed: false,
|
||||
});
|
||||
ensureIdleTimer();
|
||||
traceRss('init', repoId);
|
||||
}
|
||||
|
||||
/**
|
||||
|
|
|
|||
|
|
@ -177,6 +177,21 @@ function logQueryError(context: string, err: unknown): void {
|
|||
logger.error({ context, err: msg }, 'GitNexus query failed');
|
||||
}
|
||||
|
||||
/**
|
||||
* A "missing table/label/relation" prepare error is benign for the query tool's
|
||||
* best-effort enrichment: a repo analyzed without processes or communities simply
|
||||
* has no `Process`/`Community` tables, so the `STEP_IN_PROCESS` / `MEMBER_OF`
|
||||
* enrichment queries fail to prepare. That is a normal configuration, NOT a
|
||||
* degraded result — it must not raise the `partial` flag (which callers would
|
||||
* then learn to ignore). Real failures (timeouts, locks, native faults) do.
|
||||
*/
|
||||
function isBenignMissingTableError(err: unknown): boolean {
|
||||
const msg = err instanceof Error ? err.message : String(err ?? '');
|
||||
return /does not exist|no such (table|label|rel)|unknown (table|label)|not (defined|found)/i.test(
|
||||
msg,
|
||||
);
|
||||
}
|
||||
|
||||
const isReadOnlyDbError = (err: unknown): boolean => {
|
||||
// Walk the `cause` chain (bounded) so a wrapped read-only error (e.g. the
|
||||
// pool adapter's `{ cause }` wrapper) is still detected here — this is the
|
||||
|
|
@ -1112,6 +1127,112 @@ export class LocalBackend {
|
|||
>();
|
||||
const definitions: any[] = []; // standalone symbols not in any process
|
||||
|
||||
// Batch-fetch process participation, cohesion, and (optionally) content for
|
||||
// ALL matched symbols in 2-3 graph queries instead of 2-3 *per symbol*. The
|
||||
// previous per-symbol loop issued up to 3N sequential pool round-trips
|
||||
// (searchLimit symbols × {STEP_IN_PROCESS, MEMBER_OF, content}); on a warm
|
||||
// repo the IPC + query-setup overhead of those round-trips dominated query
|
||||
// latency. Collapsing to `WHERE n.id IN $nodeIds` preserves identical output
|
||||
// (the aggregation loop below is unchanged) while cutting the round-trips.
|
||||
// Array params bind through the pool exactly as bm25Search's
|
||||
// `WHERE n.id IN $nodeIds` already does. (Ported from gitnexus-enterprise
|
||||
// PR #222 — N+1 → 2-3 batched queries.)
|
||||
const nodeIds = merged.map(([, m]) => m.data?.nodeId).filter((id): id is string => !!id);
|
||||
|
||||
const processRowsByNode = new Map<string, any[]>();
|
||||
const cohesionByNode = new Map<string, { cohesion: number; module?: string }>();
|
||||
const contentByNode = new Map<string, string>();
|
||||
// Set when a batched enrichment query throws a REAL failure (timeout, lock,
|
||||
// native fault) — NOT the benign "no Process/Community table" case, which is
|
||||
// a normal config (a repo analyzed without processes/communities) and must
|
||||
// not raise a `partial` flag callers would learn to ignore. See
|
||||
// isBenignMissingTableError + the response build below.
|
||||
let enrichmentDegraded = false;
|
||||
|
||||
// Chunk the IN-list like the impact path (CHUNK_SIZE=100) so a large result
|
||||
// set never builds an unbounded `IN` parameter. Default batch is
|
||||
// processLimit*maxSymbolsPerProcess (≤ one chunk), but chunk for robustness.
|
||||
const QUERY_CHUNK_SIZE = 100;
|
||||
for (let i = 0; i < nodeIds.length; i += QUERY_CHUNK_SIZE) {
|
||||
const ids = nodeIds.slice(i, i + QUERY_CHUNK_SIZE);
|
||||
|
||||
// Processes each symbol participates in. `n.id AS nodeId` is prepended as
|
||||
// column 0 so rows from many symbols can be re-associated to their symbol.
|
||||
try {
|
||||
const rows = await executeParameterized(
|
||||
repo.lbugPath,
|
||||
`
|
||||
MATCH (n)-[r:CodeRelation {type: 'STEP_IN_PROCESS'}]->(p:Process)
|
||||
WHERE n.id IN $nodeIds
|
||||
RETURN n.id AS nodeId, p.id AS pid, p.label AS label, p.heuristicLabel AS heuristicLabel, p.processType AS processType, p.stepCount AS stepCount, r.step AS step
|
||||
`,
|
||||
{ nodeIds: ids },
|
||||
);
|
||||
for (const row of rows) {
|
||||
const nid = row.nodeId ?? row[0];
|
||||
let list = processRowsByNode.get(nid);
|
||||
if (!list) processRowsByNode.set(nid, (list = []));
|
||||
list.push(row);
|
||||
}
|
||||
} catch (e) {
|
||||
logQueryError('query:process-lookup', e);
|
||||
if (!isBenignMissingTableError(e)) enrichmentDegraded = true;
|
||||
}
|
||||
|
||||
// Cluster membership + cohesion. Keep the FIRST community row per node to
|
||||
// mirror the prior per-symbol `LIMIT 1` (each symbol keeps ITS community,
|
||||
// not one community for the whole batch).
|
||||
try {
|
||||
const rows = await executeParameterized(
|
||||
repo.lbugPath,
|
||||
`
|
||||
MATCH (n)-[:CodeRelation {type: 'MEMBER_OF'}]->(c:Community)
|
||||
WHERE n.id IN $nodeIds
|
||||
RETURN n.id AS nodeId, c.cohesion AS cohesion, c.heuristicLabel AS module
|
||||
`,
|
||||
{ nodeIds: ids },
|
||||
);
|
||||
for (const row of rows) {
|
||||
const nid = row.nodeId ?? row[0];
|
||||
if (!cohesionByNode.has(nid)) {
|
||||
cohesionByNode.set(nid, {
|
||||
cohesion: (row.cohesion ?? row[1]) || 0,
|
||||
module: row.module ?? row[2],
|
||||
});
|
||||
}
|
||||
}
|
||||
} catch (e) {
|
||||
logQueryError('query:cluster-info', e);
|
||||
if (!isBenignMissingTableError(e)) enrichmentDegraded = true;
|
||||
}
|
||||
|
||||
// Optionally fetch content for every matched symbol.
|
||||
if (includeContent) {
|
||||
try {
|
||||
const rows = await executeParameterized(
|
||||
repo.lbugPath,
|
||||
`
|
||||
MATCH (n)
|
||||
WHERE n.id IN $nodeIds
|
||||
RETURN n.id AS nodeId, n.content AS content
|
||||
`,
|
||||
{ nodeIds: ids },
|
||||
);
|
||||
for (const row of rows) {
|
||||
const nid = row.nodeId ?? row[0];
|
||||
contentByNode.set(nid, row.content ?? row[1]);
|
||||
}
|
||||
} catch (e) {
|
||||
logQueryError('query:content-fetch', e);
|
||||
if (!isBenignMissingTableError(e)) enrichmentDegraded = true;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Aggregation is unchanged from the per-symbol version — it now reads the
|
||||
// pre-fetched maps instead of issuing a query per symbol. Iterating `merged`
|
||||
// in the same (sorted) order preserves processMap insertion order, the
|
||||
// definitions order, and the item.score association exactly.
|
||||
for (const [_, item] of merged) {
|
||||
const sym = item.data;
|
||||
if (!sym.nodeId) {
|
||||
|
|
@ -1124,61 +1245,11 @@ export class LocalBackend {
|
|||
continue;
|
||||
}
|
||||
|
||||
// Find processes this symbol participates in
|
||||
let processRows: any[] = [];
|
||||
try {
|
||||
processRows = await executeParameterized(
|
||||
repo.lbugPath,
|
||||
`
|
||||
MATCH (n {id: $nodeId})-[r:CodeRelation {type: 'STEP_IN_PROCESS'}]->(p:Process)
|
||||
RETURN p.id AS pid, p.label AS label, p.heuristicLabel AS heuristicLabel, p.processType AS processType, p.stepCount AS stepCount, r.step AS step
|
||||
`,
|
||||
{ nodeId: sym.nodeId },
|
||||
);
|
||||
} catch (e) {
|
||||
logQueryError('query:process-lookup', e);
|
||||
}
|
||||
|
||||
// Get cluster membership + cohesion (cohesion used as internal ranking signal)
|
||||
let cohesion = 0;
|
||||
let module: string | undefined;
|
||||
try {
|
||||
const cohesionRows = await executeParameterized(
|
||||
repo.lbugPath,
|
||||
`
|
||||
MATCH (n {id: $nodeId})-[:CodeRelation {type: 'MEMBER_OF'}]->(c:Community)
|
||||
RETURN c.cohesion AS cohesion, c.heuristicLabel AS module
|
||||
LIMIT 1
|
||||
`,
|
||||
{ nodeId: sym.nodeId },
|
||||
);
|
||||
if (cohesionRows.length > 0) {
|
||||
cohesion = (cohesionRows[0].cohesion ?? cohesionRows[0][0]) || 0;
|
||||
module = cohesionRows[0].module ?? cohesionRows[0][1];
|
||||
}
|
||||
} catch (e) {
|
||||
logQueryError('query:cluster-info', e);
|
||||
}
|
||||
|
||||
// Optionally fetch content
|
||||
let content: string | undefined;
|
||||
if (includeContent) {
|
||||
try {
|
||||
const contentRows = await executeParameterized(
|
||||
repo.lbugPath,
|
||||
`
|
||||
MATCH (n {id: $nodeId})
|
||||
RETURN n.content AS content
|
||||
`,
|
||||
{ nodeId: sym.nodeId },
|
||||
);
|
||||
if (contentRows.length > 0) {
|
||||
content = contentRows[0].content ?? contentRows[0][0];
|
||||
}
|
||||
} catch (e) {
|
||||
logQueryError('query:content-fetch', e);
|
||||
}
|
||||
}
|
||||
const processRows = processRowsByNode.get(sym.nodeId) ?? [];
|
||||
const coh = cohesionByNode.get(sym.nodeId);
|
||||
const cohesion = coh?.cohesion ?? 0;
|
||||
const module = coh?.module;
|
||||
const content = includeContent ? contentByNode.get(sym.nodeId) : undefined;
|
||||
|
||||
const symbolEntry = {
|
||||
id: sym.nodeId,
|
||||
|
|
@ -1197,12 +1268,13 @@ export class LocalBackend {
|
|||
} else {
|
||||
// Add to each process it belongs to
|
||||
for (const row of processRows) {
|
||||
const pid = row.pid ?? row[0];
|
||||
const label = row.label ?? row[1];
|
||||
const hLabel = row.heuristicLabel ?? row[2];
|
||||
const pType = row.processType ?? row[3];
|
||||
const stepCount = row.stepCount ?? row[4];
|
||||
const step = row.step ?? row[5];
|
||||
// Positional fallbacks shift +1 because `n.id AS nodeId` is column 0.
|
||||
const pid = row.pid ?? row[1];
|
||||
const label = row.label ?? row[2];
|
||||
const hLabel = row.heuristicLabel ?? row[3];
|
||||
const pType = row.processType ?? row[4];
|
||||
const stepCount = row.stepCount ?? row[5];
|
||||
const step = row.step ?? row[6];
|
||||
|
||||
if (!processMap.has(pid)) {
|
||||
processMap.set(pid, {
|
||||
|
|
@ -1276,15 +1348,29 @@ export class LocalBackend {
|
|||
const timing = timer.summary();
|
||||
logQueryTiming(searchQuery, timing);
|
||||
|
||||
// Compose a single `warning` from all degraded conditions (FTS-missing
|
||||
// and/or a real enrichment failure) so neither overwrites the other, and
|
||||
// flag `partial` when enrichment was lost. Both are omitted on the clean
|
||||
// path, leaving the success-path response shape byte-identical.
|
||||
const warnings: string[] = [];
|
||||
if (!ftsUsed) {
|
||||
warnings.push(
|
||||
'FTS indexes missing — keyword search degraded. Run: gitnexus analyze --repair-fts (or gitnexus analyze --force) to rebuild indexes.',
|
||||
);
|
||||
}
|
||||
if (enrichmentDegraded) {
|
||||
warnings.push(
|
||||
'Symbol enrichment partially failed — some process/cohesion/content data may be missing from these results (see server logs).',
|
||||
);
|
||||
}
|
||||
|
||||
return {
|
||||
processes,
|
||||
process_symbols: dedupedSymbols,
|
||||
definitions: definitions.slice(0, 20), // cap standalone definitions
|
||||
timing,
|
||||
...(!ftsUsed && {
|
||||
warning:
|
||||
'FTS indexes missing — keyword search degraded. Run: gitnexus analyze --repair-fts (or gitnexus analyze --force) to rebuild indexes.',
|
||||
}),
|
||||
...(warnings.length > 0 && { warning: warnings.join(' ') }),
|
||||
...(enrichmentDegraded && { partial: true }),
|
||||
};
|
||||
}
|
||||
|
||||
|
|
|
|||
6
gitnexus/test/fixtures/local-backend-seed.ts
vendored
6
gitnexus/test/fixtures/local-backend-seed.ts
vendored
|
|
@ -35,6 +35,12 @@ export const LOCAL_BACKEND_SEED_DATA = [
|
|||
CREATE (a)-[:CodeRelation {type: 'STEP_IN_PROCESS', confidence: 1.0, reason: '', step: 1}]->(p)`,
|
||||
`MATCH (a:Function), (p:Process) WHERE a.id = 'func:validate' AND p.id = 'proc:login-flow'
|
||||
CREATE (a)-[:CodeRelation {type: 'STEP_IN_PROCESS', confidence: 1.0, reason: '', step: 2}]->(p)`,
|
||||
// func:validate is the terminalId of proc:beta-flow too — wiring its second
|
||||
// STEP_IN_PROCESS edge makes it a genuine MULTI-process symbol, which the
|
||||
// batched-query test uses to exercise the full row[1..6] positional shift
|
||||
// (a single-process symbol can't expose an off-by-one in those fallbacks).
|
||||
`MATCH (a:Function), (p:Process) WHERE a.id = 'func:validate' AND p.id = 'proc:beta-flow'
|
||||
CREATE (a)-[:CodeRelation {type: 'STEP_IN_PROCESS', confidence: 1.0, reason: '', step: 3}]->(p)`,
|
||||
`MATCH (h:Function), (t:Tool) WHERE h.id = 'func:alpha' AND t.id = 'Tool:alpha'
|
||||
CREATE (h)-[:CodeRelation {type: 'HANDLES_TOOL', confidence: 1.0, reason: 'tool-definition', step: 0}]->(t)`,
|
||||
`MATCH (h:Function), (t:Tool) WHERE h.id = 'func:beta' AND t.id = 'Tool:beta'
|
||||
|
|
|
|||
|
|
@ -111,6 +111,78 @@ withTestLbugDB(
|
|||
// At least one of the search phases must have fired for any
|
||||
// non-error response — bm25 and/or vector always runs.
|
||||
expect(result.timing.bm25 ?? result.timing.vector).toBeGreaterThanOrEqual(0);
|
||||
|
||||
// Success path (FTS present + Process/Community tables exist): no degraded
|
||||
// signal. Guards R6 — the response shape stays byte-identical when nothing
|
||||
// fails (the `warning`/`partial` fields appear only on degradation).
|
||||
expect(result).not.toHaveProperty('warning');
|
||||
expect(result).not.toHaveProperty('partial');
|
||||
});
|
||||
|
||||
// PR #222 port: the query tool batches per-symbol process/cohesion/content
|
||||
// lookups (N+1 → 2-3 `WHERE n.id IN $nodeIds` queries). These assertions
|
||||
// guard the batch-adaptation hazards that a naive cherry-pick would break:
|
||||
// (1) each symbol keeps ITS OWN community (the per-node first-row pick that
|
||||
// replaced the per-symbol `LIMIT 1`), and (2) content maps to the right
|
||||
// node — both depend on the +1 positional-index shift after prepending
|
||||
// `n.id AS nodeId`. func:login is MEMBER_OF comm:auth ("Authentication");
|
||||
// func:validate has no community, so it must NOT inherit login's.
|
||||
it('query batches per-symbol enrichment without cross-assigning community/content', async () => {
|
||||
const findSym = (res: any, id: string) =>
|
||||
(res.process_symbols ?? []).find((s: any) => s.id === id) ??
|
||||
(res.definitions ?? []).find((s: any) => s.id === id);
|
||||
|
||||
const loginRes = await backend.callTool('query', {
|
||||
query: 'login',
|
||||
include_content: true,
|
||||
});
|
||||
expect(loginRes).not.toHaveProperty('error');
|
||||
const login = findSym(loginRes, 'func:login');
|
||||
expect(login).toBeDefined();
|
||||
// Community correctly associated to its own node (not dropped, not leaked).
|
||||
expect(login.module).toBe('Authentication');
|
||||
// Content correctly mapped to its own node (positional [1] after nodeId).
|
||||
expect(login.content).toBe('function login() {}');
|
||||
|
||||
const validateRes = await backend.callTool('query', {
|
||||
query: 'validate',
|
||||
include_content: true,
|
||||
});
|
||||
expect(validateRes).not.toHaveProperty('error');
|
||||
const validate = findSym(validateRes, 'func:validate');
|
||||
expect(validate).toBeDefined();
|
||||
// validate has no MEMBER_OF edge — a flat batched `LIMIT 1` would have
|
||||
// leaked some other node's community onto it. It must have none.
|
||||
expect(validate.module).toBeUndefined();
|
||||
expect(validate.content).toBe('function validate() {}');
|
||||
});
|
||||
|
||||
// PR #222 port: a symbol in MULTIPLE processes is what fully exercises the
|
||||
// +1 positional shift in the batched STEP_IN_PROCESS aggregation — with a
|
||||
// single process row, `row.pid ?? row[1]` succeeds whether the shift is
|
||||
// right or wrong. func:validate is a step in BOTH proc:login-flow (step 2)
|
||||
// and proc:beta-flow (step 3), so both rows for the one node must be parsed
|
||||
// (pid=row[1], step=row[6]); an off-by-one would drop a process or mis-pair
|
||||
// pid↔step. Also pins process ranking (totalScore via the regroup-by-nodeId).
|
||||
it('query batches a multi-process symbol and ranks processes (positional shift across rows)', async () => {
|
||||
const res = await backend.callTool('query', { query: 'validate' });
|
||||
expect(res).not.toHaveProperty('error');
|
||||
const processIds = (res.processes ?? []).map((p: any) => p.id);
|
||||
// Both of validate's processes must appear — both STEP_IN_PROCESS rows
|
||||
// were parsed and grouped by the correct pid (row[1]).
|
||||
expect(processIds).toContain('proc:login-flow');
|
||||
expect(processIds).toContain('proc:beta-flow');
|
||||
|
||||
// process_symbols dedups by id, so validate appears once carrying the
|
||||
// pid+step of its top-ranked process — they must come from the SAME
|
||||
// shifted row: login-flow⇒step 2, beta-flow⇒step 3.
|
||||
const v = (res.process_symbols ?? []).find((s: any) => s.id === 'func:validate');
|
||||
expect(v).toBeDefined();
|
||||
expect(v.step_index).toBe(v.process_id === 'proc:beta-flow' ? 3 : 2);
|
||||
|
||||
// Ranking: 'login' surfaces proc:login-flow as the top process.
|
||||
const loginRes = await backend.callTool('query', { query: 'login' });
|
||||
expect((loginRes.processes ?? [])[0]?.id).toBe('proc:login-flow');
|
||||
});
|
||||
|
||||
it('tool_map returns per-tool flows without cross-attributing same-file tools', async () => {
|
||||
|
|
|
|||
76
gitnexus/test/unit/fts-rss-verdict.test.ts
Normal file
76
gitnexus/test/unit/fts-rss-verdict.test.ts
Normal file
|
|
@ -0,0 +1,76 @@
|
|||
import { describe, it, expect } from 'vitest';
|
||||
// Pure, side-effect-free classifier extracted from the FTS evict→reload bench.
|
||||
// Importing it must NOT load @ladybugdb/core, build fixtures, or run the bench
|
||||
// (the module has zero imports and zero module-scope side effects). If this
|
||||
// import ever pulled in the bench's native-addon require, this test file would
|
||||
// be slow / fail to load — so the import succeeding cheaply IS the no-side-effect
|
||||
// guard (R3).
|
||||
import { classifyVerdict, median, slopeMbPerCycle } from '../../scripts/bench/fts-rss-verdict.mjs';
|
||||
|
||||
// Build an exactly-linear series: start, start+slope, start+2·slope, …
|
||||
const linear = (start: number, slope: number, n: number): number[] =>
|
||||
Array.from({ length: n }, (_, i) => start + slope * i);
|
||||
|
||||
describe('classifyVerdict (FTS evict→reload bench)', () => {
|
||||
it('pure helpers behave', () => {
|
||||
expect(typeof classifyVerdict).toBe('function');
|
||||
expect(median([3, 1, 2])).toBe(2);
|
||||
// Least-squares slope of an exact linear series equals its step.
|
||||
expect(slopeMbPerCycle(linear(100, 2, 10))).toBeCloseTo(2, 6);
|
||||
});
|
||||
|
||||
it('truly flat run → PLATEAU (over-correction guard, R1)', () => {
|
||||
// No leak, no warmup: first ≈ second ≈ 0. Must stay PLATEAU, NOT INCONCLUSIVE.
|
||||
const series = [200, 200, 201, 200, 200, 201, 200, 200, 200, 201, 200, 200];
|
||||
const r = classifyVerdict(series, 190);
|
||||
expect(r.verdict).toBe('PLATEAU');
|
||||
});
|
||||
|
||||
it('sustained sub-floor positive slope → INCONCLUSIVE (the headline fix, R1)', () => {
|
||||
// ~0.4 MB/cycle, sustained (first ≈ second slope), below the absolute floor.
|
||||
// The OLD logic labeled this PLATEAU ("no leak"); it must now be INCONCLUSIVE.
|
||||
const series = linear(200, 0.4, 20); // peak ~207.6, baseline 190 → small WS → floor stays 0.5
|
||||
const r = classifyVerdict(series, 190);
|
||||
expect(r.secondHalfSlope).toBeGreaterThan(0.1); // above EPSILON
|
||||
expect(r.secondHalfSlope).toBeLessThan(r.floor); // below the floor
|
||||
expect(r.decelRatio).toBeGreaterThanOrEqual(0.6); // sustained, not decelerating
|
||||
expect(r.verdict).toBe('INCONCLUSIVE');
|
||||
});
|
||||
|
||||
it('decelerated-to-flat run → PLATEAU', () => {
|
||||
// Climbs then flattens: tail slope ≈ 0 (below EPSILON).
|
||||
const series = [200, 210, 218, 224, 228, 230, 231, 231, 231, 231, 231, 231, 231, 231, 231, 231];
|
||||
const r = classifyVerdict(series, 190);
|
||||
expect(r.verdict).toBe('PLATEAU');
|
||||
});
|
||||
|
||||
it('sustained linear above the floor → CLIMB', () => {
|
||||
const series = linear(200, 3, 20); // 3 MB/cycle sustained
|
||||
const r = classifyVerdict(series, 190);
|
||||
expect(r.decelRatio).toBeGreaterThanOrEqual(0.6);
|
||||
expect(r.secondHalfSlope).toBeGreaterThanOrEqual(r.floor);
|
||||
expect(r.verdict).toBe('CLIMB');
|
||||
});
|
||||
|
||||
it('step discontinuity → INCONCLUSIVE (existing guard preserved)', () => {
|
||||
const series = [200, 201, 202, 203, 204, 205, 265, 266, 267, 268, 269, 270];
|
||||
const r = classifyVerdict(series, 190);
|
||||
expect(r.stepDiscontinuity).toBe(true);
|
||||
expect(r.verdict).toBe('INCONCLUSIVE');
|
||||
});
|
||||
|
||||
it('floor scales with working-set growth, not baseline RSS (R2)', () => {
|
||||
// Identical 0.8 MB/cycle sustained tail, two different working sets.
|
||||
const series = linear(600, 0.8, 12); // peak ~608.8
|
||||
// Small working set (baseline near the series) → low floor → 0.8 clears it → CLIMB.
|
||||
const small = classifyVerdict(series, 590); // peak-baseline ~18.8 → floor 0.5
|
||||
expect(small.secondHalfSlope).toBeGreaterThanOrEqual(small.floor);
|
||||
expect(small.verdict).toBe('CLIMB');
|
||||
// Large working set (low baseline) → floor rises with arena size → 0.8 is now
|
||||
// sub-floor → the sustained-but-small slope is unresolved, not a clean CLIMB.
|
||||
const large = classifyVerdict(series, 0); // peak-baseline ~608 → floor ~1.5
|
||||
expect(large.floor).toBeGreaterThan(small.floor);
|
||||
expect(large.secondHalfSlope).toBeLessThan(large.floor);
|
||||
expect(large.verdict).toBe('INCONCLUSIVE');
|
||||
});
|
||||
});
|
||||
120
gitnexus/test/unit/query-degraded-signal.test.ts
Normal file
120
gitnexus/test/unit/query-degraded-signal.test.ts
Normal file
|
|
@ -0,0 +1,120 @@
|
|||
import { describe, it, expect, vi, beforeEach } from 'vitest';
|
||||
|
||||
// Mock the pool adapter (and its re-export shim) so executeParameterized is fully
|
||||
// controllable — the proven seam from impact-batching-grouping.test.ts. This is a
|
||||
// UNIT test: the integration suite runs the real executeParameterized against a
|
||||
// real DB, so it cannot make ONE enrichment query throw while the rest succeed.
|
||||
const executeParameterizedMock = vi.fn();
|
||||
|
||||
vi.mock('../../src/core/lbug/pool-adapter.js', async (importOriginal) => {
|
||||
const actual = await importOriginal<typeof import('../../src/core/lbug/pool-adapter.js')>();
|
||||
return {
|
||||
...actual,
|
||||
initLbug: vi.fn(),
|
||||
executeParameterized: (...args: any[]) => executeParameterizedMock(...args),
|
||||
closeLbug: vi.fn(),
|
||||
isLbugReady: vi.fn().mockReturnValue(true),
|
||||
};
|
||||
});
|
||||
vi.mock('../../src/mcp/core/lbug-adapter.js', async (importOriginal) => {
|
||||
const actual = await importOriginal<typeof import('../../src/mcp/core/lbug-adapter.js')>();
|
||||
return {
|
||||
...actual,
|
||||
initLbug: vi.fn(),
|
||||
executeParameterized: (...args: any[]) => executeParameterizedMock(...args),
|
||||
closeLbug: vi.fn(),
|
||||
isLbugReady: vi.fn().mockReturnValue(true),
|
||||
};
|
||||
});
|
||||
|
||||
import { LocalBackend } from '../../src/mcp/local/local-backend';
|
||||
|
||||
// A backend whose hybrid search yields exactly one matched symbol, so the
|
||||
// enrichment chunk loop runs and can be made to fail. `ftsUsed` is parameterized
|
||||
// so we can exercise the FTS-missing + enrichment-degraded composition.
|
||||
function makeBackend(ftsUsed = true): LocalBackend {
|
||||
const backend = new LocalBackend();
|
||||
const repoHandle = {
|
||||
id: 'repo1',
|
||||
name: 'repo1',
|
||||
repoPath: '/tmp/repo',
|
||||
storagePath: '/tmp/repo/.gitnexus',
|
||||
lbugPath: '/tmp/repo/.gitnexus/lbug',
|
||||
indexedAt: 'now',
|
||||
lastCommit: 'c',
|
||||
stats: {},
|
||||
} as any;
|
||||
(backend as any).repos.set(repoHandle.id, repoHandle);
|
||||
(backend as any).ensureInitialized = vi.fn().mockResolvedValue(undefined);
|
||||
const sym = {
|
||||
nodeId: 'func:x',
|
||||
name: 'x',
|
||||
type: 'Function',
|
||||
filePath: 'f.ts',
|
||||
startLine: 1,
|
||||
endLine: 2,
|
||||
};
|
||||
(backend as any).bm25Search = vi.fn().mockResolvedValue({ results: [sym], ftsUsed });
|
||||
(backend as any).semanticSearch = vi.fn().mockResolvedValue([]);
|
||||
return { backend, repoHandle } as any;
|
||||
}
|
||||
|
||||
const runQuery = (b: any, params: any = { query: 'x' }) =>
|
||||
(b.backend as any).query(b.repoHandle, params);
|
||||
|
||||
describe('query: degraded-enrichment signal', () => {
|
||||
beforeEach(() => vi.clearAllMocks());
|
||||
|
||||
it('a REAL enrichment failure surfaces warning + partial, and still returns the symbol', async () => {
|
||||
const b = makeBackend(true);
|
||||
executeParameterizedMock.mockImplementation(async (_repo: string, query: string) => {
|
||||
if (query.includes('STEP_IN_PROCESS'))
|
||||
throw new Error('Query execution timed out after 30000ms');
|
||||
return []; // MEMBER_OF / content succeed (empty)
|
||||
});
|
||||
|
||||
const result = await runQuery(b);
|
||||
|
||||
expect(result).not.toHaveProperty('error');
|
||||
expect(result.partial).toBe(true);
|
||||
expect(typeof result.warning).toBe('string');
|
||||
expect(result.warning.toLowerCase()).toContain('enrichment');
|
||||
// The matched symbol still comes back (degraded to definitions, not dropped).
|
||||
expect(result.definitions.map((d: any) => d.id)).toContain('func:x');
|
||||
});
|
||||
|
||||
it('a BENIGN missing-table error does NOT trip the signal', async () => {
|
||||
const b = makeBackend(true);
|
||||
executeParameterizedMock.mockImplementation(async (_repo: string, query: string) => {
|
||||
// A repo analyzed without processes/communities: prepare fails because the
|
||||
// table/label does not exist. This is normal, not degraded.
|
||||
if (query.includes('STEP_IN_PROCESS') || query.includes('MEMBER_OF'))
|
||||
throw new Error('Binder exception: Table Process does not exist.');
|
||||
return [];
|
||||
});
|
||||
|
||||
const result = await runQuery(b);
|
||||
|
||||
expect(result).not.toHaveProperty('error');
|
||||
expect(result.partial).toBeUndefined();
|
||||
expect(result.warning).toBeUndefined(); // ftsUsed=true and no real failure
|
||||
expect(result.definitions.map((d: any) => d.id)).toContain('func:x');
|
||||
});
|
||||
|
||||
it('composes the FTS-missing warning with the enrichment-degraded message', async () => {
|
||||
const b = makeBackend(false); // FTS unavailable
|
||||
executeParameterizedMock.mockImplementation(async (_repo: string, query: string) => {
|
||||
if (query.includes('STEP_IN_PROCESS'))
|
||||
throw new Error('Query execution timed out after 30000ms');
|
||||
return [];
|
||||
});
|
||||
|
||||
const result = await runQuery(b);
|
||||
|
||||
expect(result.partial).toBe(true);
|
||||
expect(typeof result.warning).toBe('string');
|
||||
// Both messages present in the single composed warning — neither overwrites the other.
|
||||
expect(result.warning).toMatch(/FTS indexes missing|repair-fts/i);
|
||||
expect(result.warning.toLowerCase()).toContain('enrichment');
|
||||
});
|
||||
});
|
||||
Loading…
Add table
Reference in a new issue