mirror of
https://github.com/abhigyanpatwari/GitNexus.git
synced 2026-10-03 02:21:44 +00:00
8 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Gergő Magyar
|
21a52af1d4
|
fix(lbug): ship FTS per-platform and recover in-place native aborts (#3274)
* fix(lbug): pin Ladybug core so Dependabot cannot ship a skewed FTS artifact The extension version is a separate upstream constant. Ignore daily core bumps and fail the pairing gate when the committed manifest does not name the installed core. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): make doctor and CI FTS gates resolve the packaged artifact Doctor and the REQUIRE_FTS file gates still treated an empty ~/.lbdb as unavailable, which would turn three CI jobs red once analyze stops installing into that tree. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): name native-abort and tuple-missing so analyze cannot mis-advise The CLI summary's trailing else treated every unknown skip reason as a missing extension. New crash and platform causes must get their own remedies, not a network-install hint. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): delete the dead read-path FTS index create ensureFTSIndex had no production callers and swallowed read-only CREATE_FTS_INDEX failures, which hid the only signal that a reader tried to write. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): vendor per-platform FTS artifacts so analyze needs no host install Keyword search depended on a CDN fetch into ~/.lbdb. Shipping the five published tuples inside the package makes air-gapped and ignore-scripts installs load the same artifact the publish gate checksums. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): load the packaged FTS artifact before any network install Analyze still required a CDN fetch into ~/.lbdb even when the package already shipped the file. FTS now path-loads the vendored tuple first and records source labels so a later truncated home copy cannot steal the diagnosis. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): diagnose a core/extension version skew instead of a missing runtime A structurally valid FTS artifact whose path version disagrees with the packaged pin must name both versions, not prescribe VC++ or OpenSSL. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): stamp an FTS phase so repair stays usable after an in-place abort A native CREATE_FTS_INDEX abort leaves no skip reason; the next run infers it from the dirty flag, and --repair-fts must not treat that phase as a half-written graph. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): park an in-place FTS crash WAL without wiping the graph An FTS abort after a successful checkpoint must reopen the live index on macOS, Windows, and Linux. Staging never parks the live WAL; readers keep today's large-WAL refusal. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): refuse read-only opens of an FTS-poisoned WAL MCP and serve cannot repair a leftover in-place abort. Fail before the native open and name --repair-fts, on macOS, Windows, and Linux. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): name a vendor-neutral Windows OpenSSL prerequisite OQ1 is unanswered here so GitNexus does not ship OpenSSL DLLs. Windows FTS now asks for a system OpenSSL 3 runtime instead of Git Bash PATH. Co-authored-by: Cursor <cursoragent@cursor.com> * test(lbug): inject the FTS vendor root and redact it on HTTP and MCP Path-loaded artifacts no longer vary with HOME. Tests pass an injected vendor tree and assert search warnings never leak a filesystem path. Co-authored-by: Cursor <cursoragent@cursor.com> * docs(lbug): document load-only as the global FTS install default Analyze still overrides to auto. Packaged per-platform artifacts load before any network install on macOS, Windows, and Linux. Co-authored-by: Cursor <cursoragent@cursor.com> * docs(lbug): format the FTS install-policy README table Prettier does not run on Markdown in pre-commit, so the U10 table wrap needs its own formatting commit. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): skip FTS CREATE after a persisted native abort A recovered analyze run was retrying CREATE_FTS_INDEX from skipReason alone. Keep that skip until --repair-fts, fail closed on unsupported tuples, and honor the checkpoint warrant for park/repair. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): honor checkpoint flushed warrant and align FTS tests with packaged vendor A no-op CHECKPOINT must not satisfy the FTS park warrant, and CI still asserted HOME-only FTS isolation after analyze started path-LOADing the packaged artifact. Co-authored-by: Cursor <cursoragent@cursor.com> * test(lbug): accept a nonempty incremental write set in the #2790 recovery check FTS-phase recovery can incremental-add files (changed=0, added=1). That is not the #2790 empty-diff wipe skip. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): compare FTS home versions to the core pin and tighten the publish filename gate Ladybug's ~/.lbdb/extension directory is the runtime/core version; treating it as the artifact version false-diagnosed skew. The publish guard now rejects a path-escaping filename the same way the fetch script does. Co-authored-by: Cursor <cursoragent@cursor.com> * test(lbug): seed FTS e2e fixtures from the packaged vendor artifact A machine with no ~/.lbdb copy should still run the vendor-survivorship cases; the seed no longer depends on HOME or a network install. Co-authored-by: Cursor <cursoragent@cursor.com> * Address PR review feedback (#3274) Keep in-place FTS abort evidence after persist so a second CREATE abort cannot fail-open readers, and close the CLI, loader, embed, and e2e gaps the review called out. Note: full npm test hit Ladybug worker-pool startup failures under memory pressure; tsc and 180 targeted unit tests passed. Co-authored-by: Cursor <cursoragent@cursor.com> * Address PR review feedback (#3274) Run the vendored-path symlink guard on the OS matrix, put e2e HOME fixtures on Ladybug's real extension layout, pin the embed crash-WAL gate before the writable open, and let analyze writers park through missing-shadow recovery. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): keep --repair-fts CI green after vendored-first FTS Never-installed warning fixtures must not inspect a packaged vendor binary, and a failed dirty restamp must not abort an otherwise successful --repair-fts run. Co-authored-by: Cursor <cursoragent@cursor.com> * test(cli): give the #1169 analyze e2e the same 90s Windows budget as its sibling The first #1169 persist-meta case was still on a 60s spawn/it budget and was killed banner-only on windows-latest after the FTS warning fixture no longer failed the shard first. Co-authored-by: Cursor <cursoragent@cursor.com> * test(ci): reweight Windows shards after the FTS e2e grew Vendored-first HOME fixtures pushed fts-extension-e2e to ~6 minutes on windows-latest, so the old 146s weight packed it with skills-e2e and blew the 20-minute watchdog. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
pcaceres-hazloagil
|
a8736a07d0
|
fix(lbug): checkpoint race in pool-adapter.ts + pin @ladybugdb/core to 0.18.3 (#3189)
* fix(lbug): await evict-then-reopen so it can't race the checkpoint
closeOne() closed the evicted repo's shared Database with a
fire-and-forget `db.close().catch(() => {})` (no await). Both call
sites that evict-then-reopen — evictLRU() right before doInitLbug
opens the new connection, and the "idle & changed" path in initLbug —
proceeded to open the next repo's connection immediately after,
without waiting for the evicted repo's close (and the checkpoint it
triggers) to finish. On the real engine the new open can then collide
with that still-in-flight checkpoint, surfacing on any read as:
Runtime exception: Cannot open database in read-only mode while
checkpoint is in progress. Please retry later.
This reproduces reliably once more than MAX_POOL_SIZE (5) distinct
repos are queried within a short window (self-hosted deployments with
more than a handful of active repos hit it routinely), and gets worse
under genuinely concurrent requests for different repos, since nothing
serialized pool mutations across callers either.
Fix:
- closeOne / evictLRU are now async and await their internal work
(closeOne's own close() call; evictLRU's call to closeOne), closing
the race within a single initLbug call.
- The exported initLbug is wrapped in a small async mutex
(initLbugInner does the real work) so concurrent initLbug calls for
different repos serialize instead of each racing their own
evict-then-reopen against the others.
- closeLbug's two closeOne() calls are now awaited too — closeOne
becoming async meant closeLbug could resolve before pool.delete()
had actually run, which a repo-pinning test caught (isLbugReady()
briefly still true right after a resolved closeLbug()).
- closeOne now deletes the pool entry (and clears its pin, and
notifies pool-close listeners) BEFORE the awaited db.close(), not
after. Review caught that the previous order left a "zombie" entry
reachable via pool.get(repoId) — closed=true, available emptied, but
still present — for the duration of that await; a same-repo
query/init landing in that window would see isLbugReady() as true
and hit a "Connection pool integrity error" in checkout() instead of
just reopening. Deleting first removes the entry entirely, so a
concurrent caller takes the normal fresh-open path instead.
Verified two ways:
- Against the compiled bundle (`ghcr.io/abhigyanpatwari/gitnexus`,
1.6.10/1.6.11 — pool-adapter.js is byte-identical between them): an
A/B docker build with 7 tiny local repos and genuinely concurrent
(parallel, not sequential) /api/graph requests goes from 7/7 failing
to 7/7 succeeding on a freshly-analyzed pool.
- Unit tests here (mocks @ladybugdb/core the same way as
lbug-pool-pinning.test.ts): one asserts the evicted repo's close()
completes before the initLbug call that triggered the eviction
settles; another asserts closeLbug's own promise doesn't resolve
before the underlying close() does. Both gate their mock's close()
on a real short delay and were confirmed to fail against code that
drops the corresponding await.
Note: a second, deeper issue was also observed in the docker A/B
setup — repeated rounds of concurrent access show a repo that has
gone through one evict+reopen cycle can become permanently unable to
reopen for reads, identically with and without this fix. That did not
reproduce with mocks and isn't understood yet; filed separately as
#3186, which stays open and untouched by this PR — this fix closes a
real, root-caused bug on its own but does not resolve #3186 by itself.
Second review round caught a follow-up: the idle-timeout sweep calls
closeOne(repoId) directly, outside of initLbug's poolLock. Now that
closeOne deletes the pool entry before its awaited close(), an
unsynchronized idle close racing a same-repo initLbug could let that
initLbug treat the repo as absent while the idle close (and its
checkpoint) is still in flight — reopening the same class of race this
PR exists to close, just via the idle path instead of LRU eviction.
Routed the idle sweep's closeOne call through withPoolLock too, so it
serializes against initLbug the same way evictLRU already does.
(Tried to add a mocked regression test for this specific interleaving;
dropped it — the mock's dbCache-reuse path masks the difference
regardless of the fix, so it could not be made to discriminate
reliably. Fixed by direct code review instead, same as the note below
already does for the native-engine-specific checkpoint collision.)
Also removed the initLbugInner per-repoId initPromises dedup map: with
every initLbug call now serialized through poolLock, a second call for
a repoId already being initialized cannot observe a pending promise in
initPromises (the first call always fully completes, including its
finally-block cleanup, before the lock releases) — the branch was dead
code the bot correctly flagged twice.
Third review round caught two more follow-ups on the same theme (both
introduced by making the idle sweep route through poolLock):
- closeLbug()'s no-arg ("close everything") branch still calls closeOne
directly in a loop over a snapshotted pool.keys(), without the lock —
an initLbug racing that loop could register a fresh entry the
snapshot never saw, leaving it resident after a call meant to empty
the pool. Wrapped the snapshot+loop in withPoolLock.
- The idle timer callback can now sit queued behind an in-progress
initLbug before its turn arrives, and that init (or a concurrent
touchRepo()) can refresh lastUsed in the meantime — so the pre-lock
idleness check taken when the timer fired can be stale by the time
it actually runs. Re-check lastUsed/checkedOut again inside the lock,
right before closing, instead of trusting the outer snapshot.
* fix(deps): pin @ladybugdb/core back to 0.18.3
Bisected the "checkpoint is in progress" symptom (root cause #2, not
touched by the pool-adapter.ts fix in the previous commit) down to a
single dependency-version-bump commit with zero application code
changes:
|
||
|
Gergő Magyar
|
7e993ab897
|
fix(group): fail ambiguous sync names and honor analyze --name (#3094)
* fix(group): fail sync when a member name is ambiguous Silent first-match bound the wrong clone when --allow-duplicate-name registered two paths under one alias. Refs #3028. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(analyze): apply --name on the already-up-to-date path A rename should not require --force when the index is already current. Register before the same-commit branch restamp. Refs #3028. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(group): hint member path when impact --repo is an alias $localRepo stays the yaml key; joining on the registry alias is a non-join. List matching keys so operators can retry. Refs #3028. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(group): keep injected sync and alias hints consistent Workspace-deps path maps reuse the resolved handle so duplicate names cannot throw after an injected resolver. Alias hints match case-insensitively. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
DuduPhudu
|
0f793558ad
|
fix(group)!: stop group sync claiming matching it never did (#3020)
* fix(group)!: remove the matching cascade that was advertised but never built `gitnexus group create` wrote `matching.bm25_threshold` and `matching.embedding_threshold` into every generated group.yaml, and no matcher ever read either one. That was not the whole of it — an entire feature surface described a BM25/embedding cascade that does not exist: - `matching.bm25_threshold` / `matching.embedding_threshold` — parsed, persisted, unread - `detect.embedding_fallback` — defaulted and templated, unread - `MatchType` declared `'bm25' | 'embedding'`; both variants unreachable - `SyncOptions.skipEmbeddings` — declared in sync.ts and never read - `gitnexus group sync --skip-embeddings` — accepted, threaded through GroupService, ignored - CLI help in en and zh-CN promised "Exact + BM25 only (no embedding fallback)" - the MCP `group_sync` schema exposed `skipEmbeddings`, described as "Exact + BM25 only (Demo PR: same as default exact path)" `sync.ts` imports exactly `buildProviderIndex`, `runExactMatch` and `runWildcardMatch`, and the printed cascade has one stage. An operator whose links do not match reaches for those thresholds first, and turning either knob changes nothing — config that silently does nothing is how people conclude a feature is broken. Evidence that the cascade should be deleted rather than implemented, from a real backend/frontend pair: of 165 consumer contracts, 149 link exactly and 16 do not. Nine of the sixteen are third-party APIs (Google OAuth, Apple public keys, PostHog, image annotation) with no in-group provider by construction — similarity matching cannot recover them, it can only invent false links. Two are verb mismatches: the frontend calls `POST /links` and `GET /links/check-exists` while the backend declares `GET /links` and eleven other `/links/*` routes but neither of those, so a fuzzy path match would link a POST consumer to a GET provider. The rest are path-extraction artifacts. Roughly none of the sixteen would be correctly recovered, and several would be actively mis-linked. BREAKING CHANGE: `gitnexus group sync --skip-embeddings` and the MCP `group_sync` `skipEmbeddings` parameter are removed. Both were accepted and ignored, so no behavior changes — but a script passing the flag now fails with `unknown option` instead of being silently misled. Existing group.yaml files keep loading: the removed keys are simply no longer part of the schema, and a regression test pins that a legacy config carrying all three still parses. Closes #3006 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(group)!: honour --exact-only, drop inert --allow-stale, report every matching stage Addresses the review findings on #3020, all of which are the same defect the PR itself is about: group-sync surface that describes behaviour the pipeline does not have. `exactOnly` was inert in exactly the way `skipEmbeddings` was — declared on `SyncOptions`, threaded through the CLI and the MCP tool, and read by nothing — and strictly worse, because the stage it promised to suppress DOES run and DOES write `matchType:'wildcard'` links into contracts.json and the bridge, which `group impact` and cross-repo `trace` then traverse. It is now honoured rather than deleted: unlike the never-built BM25/embedding stages, the stage it names exists, so the flag describes a real choice. The substituted result is `{ matched: [], remaining: unmatched }`, not an empty result — `wildcard.remaining` IS `SyncResult.unmatched`, so skipping the stage has to leave its input unmatched rather than dropping it from the count an operator reads. `allowStale` had no such stage to gate: `syncGroup` emits no stale warning at any point (the `checkStaleness` call lives in `groupStatus`, a different path), so it is removed under the same rationale as `skipEmbeddings`. `group sync` now prints every matching stage instead of `exact` alone. The old block printed a `Matching cascade:` header and counted only exact links while the next line reported `result.crossLinks.length` — which also includes `manifest` and `wildcard` — so for any group with those the two numbers disagreed with nothing on screen explaining why. Counting is an exhaustive `Record<MatchType, number>`, so a new MatchType fails the build here instead of going silently uncounted, and reads through `?? 0` so a legacy registry carrying a removed matchType prints an honest count rather than `NaN`. Also: the MCP `group_sync` description no longer omits the wildcard stage that always runs, `exactOnly`'s description no longer refers to a "cascade", and bench/cross-repo-trace/verify.mjs no longer generates the removed threshold keys into a fresh group.yaml. Tests: `sync-exact-only.test.ts` pins both directions of the gate (mutation-verified: removing the gate, or returning `remaining: []`, both go red). `group-tools.test.ts` pins that the MCP schema dropped `skipEmbeddings` and kept `exactOnly`. `group-cli.test.ts` pins that both removed flags are rejected, with `--exact-only` as an accepted-flag control. `config-parser.test.ts` now pins that legacy keys are PRESERVED (measured, not assumed) rather than only that parsing does not throw. The type narrowing's fallout in test files is cleared: `tsc -p tsconfig.test.json` is 987 errors at head against 987 measured on origin/main, with the two error sets identical — zero net, zero new, zero masked. Verification: `tsc --noEmit` exit 0; prettier clean; eslint 0 errors (2 warnings, both pre-existing on base); 69 test files / 1169 tests green across test/unit/group, test/integration/group, tools, cli-i18n and cli-index-help. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(group): reject malformed and retired group_sync parameters (U1) `GroupService.groupSync` read `exactOnly` off an untyped MCP payload with `Boolean(params.exactOnly)`. While the flag was inert that coercion was harmless; now that it gates the wildcard matching stage, the string "false" -- a routine shape for an LLM caller emitting JSON -- is truthy, so a caller that asked to KEEP wildcard matching got it suppressed and a registry with fewer cross-links persisted to disk. The opposite of the request, written down. Validate instead of coercing, at the service boundary: the MCP SDK does not enforce a tool's advertised inputSchema and `callTool` is reachable directly, so this method is the real gate. The validator mirrors `validateImpactMode`'s `{ value } | { error }` shape -- the established idiom for this boundary, and the one groupSync's other guards already return through. Also refuse `skipEmbeddings` and `allowStale` by name. The CLI rejects them outright because commander errors on an unknown option; the MCP path accepted and silently dropped them, so an agent working from a cached tool schema was never told. Removing them took away discoverability, not acceptance. Both guards run before the group is read off disk, so a rejected call performs no work. Every test asserts the sync did NOT run -- an error string alone cannot distinguish "refused" from "refused but synced anyway". The tool description gains the validation note AFTER the registryOutcome paragraph: `tools.test.ts` slices that description by ordinal position of the 'preserved' / 'superseded' / 'no-prior-registry' literals, so appending past all three leaves those slices intact (verified, 44/44). tsc clean; 1039/1039 group unit tests pass. * fix(group): record the matching stages a sync was told to skip (U2) An `--exact-only` sync wrote a contracts.json and bridge with fewer cross-links and nothing recorded that the wildcard stage had been suppressed by request. `group_impact` and cross-repo `trace` read that registry as authoritative, so a narrowed graph was indistinguishable from a complete one -- and because `group_sync` is MCP-exposed, one agent call durably narrowed the shared answer for every later reader with no signal at all. Add `suppressedMatchStages` to ContractRegistry and SyncResult, following the `unreadableRepos` tri-state end to end: absent means a registry written before the field existed, `[]` is the measurement "this run suppressed nothing", and a populated list names the stages. The writer always emits it, because omitting the empty case is what made "measured, none" unreachable for `unreadableRepos`. Two properties that are easy to get backwards, and are why the split matters: - SyncResult carries the marker on EVERY outcome. The sync genuinely did skip the stage whatever happened to the file afterwards, and the CLI summary (U3) renders from this rather than re-deriving it from the caller's options. - The PERSISTED registry stamps it only on the `written` outcome. The preserve path re-writes `{ ...prior }`, so a carried-forward registry keeps the marker of the sync that actually produced its contracts instead of being relabelled with this run's request. That holds by construction: the registry literal carrying the field is only reachable on the written path. `loadContractRegistryResilient` gains an explicit line, because it rebuilds the envelope field by field with no spread of the parsed root -- a new on-disk field is silently dropped unless named there. Its reader is `recordedMatchStages`, not the existing `recordedRepoList`: that one validates `string[]`, which is right for repo names and one notch too weak here. This repo has already retired MatchType members ('bm25', 'embedding'), so a stale value on disk is a real shape, and dropping non-members keeps an unknown stage name from reaching a caller typed as a live one. Surfaced on `group_contracts` and on `group_sync`'s own return -- deliberately kept separate from the truncated/truncationReason/riskEpistemic triple. That triple reports limits a run hit by accident, whose remedy is to fix the repo; a suppressed stage was asked for, and its remedy is to re-sync without the flag. Conflating them would tell an agent to retry something that returns identically. tsc clean; 1043/1043 group unit tests pass. * fix(group): name a skipped matching stage as skipped, and pin it (U3, U4) Two facts were printing as the same line. `wildcard: 0 cross-links` meant both "the stage ran and matched nothing" and "the stage never ran because you passed --exact-only" -- the same conflation this summary block was introduced to remove one line up, reintroduced by the flag that made the block necessary. Render a suppressed stage as `skipped (--exact-only)`, driven by the sync's own `suppressedMatchStages` rather than by `opts.exactOnly`. The renderer reports what the sync did, not what the caller asked for, so it stays correct on the outcomes where the run ended without writing a registry -- which is where the summary is least legible and a re-derivation from the options would have been wrong. Also drops the `?? 0` fallback and the comment justifying it. The comment claimed a legacy registry could carry a retired matchType into this loop. It cannot: `syncGroup` returns a freshly computed `crossLinks` array on every outcome, and even on the preserve path the prior links go to disk while the fresh array is returned. The code was harmless; the stated reason was false, and a comment that explains an unreachable path is worse than no comment. U4 pins both halves through the CLI. A manifest fixture is sufficient: the stage counts must sum to the total on the `Wrote contracts.json (…)` line, and the skipped rendering does not need a stage to have matched anything, because --exact-only records the suppression whatever the fixture holds. That is why this coverage did not need indexed gRPC/Thrift fixture repos. Verified by mutation, not assertion: removing the skipped-rendering branch turns `names a stage it was told to skip as skipped` red and leaves the other 22 green. A control case pins the opposite direction -- the same group without the flag still reports the stage as zero -- so `skipped` cannot be printed unconditionally and pass. U3 and U4 land together: the test has no value without the renderer, so one commit keeps a revert clean. Both depend on U2, which introduced the field they read. tsc clean; 1066/1066 across the group unit and CLI integration suites. * fix(group): make every description of --exact-only match what it does (U5) Two descriptions this branch wrote or touched still misstated behavior. The MCP `exactOnly` description carries "Manifest links still apply." The CLI help and both locale strings, rewritten in the same commit, omit it -- so the surface most operators read understated what still runs. Manifest cross-links are computed before the gate and are genuinely unaffected by the flag, so the caveat is the accurate half and the CLI now says it too. The `group_sync` tool description opened with "extract HTTP contracts". That clause was carried forward byte-identical while only the trailing cross-linking half was rewritten, and it is wrong: the detect config has six non-HTTP extraction toggles, and this branch's own new test fixture is Thrift. `help-i18n.ts` is deliberately untouched. It maps an option to its translation key and that key already exists; only the commander string and the two locale values carry text, so a text-only change does not reach it. The tool-description edit sits ahead of the registryOutcome paragraph, leaving the relative order of the 'preserved' / 'superseded' / 'no-prior-registry' literals intact -- `tools.test.ts` slices that description by their positions. tsc clean; 64/64 across the locale-parity, help-registration, tool-schema and group-tool suites. * fix(group)!: remove max_candidates_per_step and shared_libs (U6) Both keys were declared, defaulted, written into every generated group.yaml, and read by nothing -- the same three-station dead surface this PR removed for bm25_threshold, embedding_threshold and detect.embedding_fallback. Every other DetectConfig field gates a real extractor in sync.ts; shared_libs gates nothing, because 'lib' contracts come only from the operator-declared manifest extractor. MatchingConfig reaches matching.ts solely through buildNoisyContractFilter, which reads exclude_links_paths and exclude_links_param_only_paths and nothing else. Existing group.yaml files keep loading and keep their keys. parseGroupConfig spreads the raw block over its defaults, so a key the schema no longer knows about survives into the returned config -- which matters because `group add` and `group remove` round-trip the operator's file through loadGroupConfig -> yaml.dump -> write, so anything the parser dropped would be deleted from their checked-in file. The legacy-config test now pins both keys in the same cast form as its three siblings, and the fixture carries shared_libs so that assertion is not vacuous. Two stations that are easy to miss and are swept here: - gitnexus/bench/cross-repo-trace/verify.mjs GENERATES a fresh group.yaml. It is not a preserve-path fixture, so "leave YAML fixtures alone" does not cover it; the repo has two generators and both are updated. It is a .mjs file outside tsconfig's include, so no type gate would have caught it. - config-parser.test.ts asserted the removed default at runtime, which vitest DOES run. That assertion is gone from the defaults case (the key no longer has a default) and re-formed as a preserve assertion in the legacy case. Verification gate, corrected: "zero net new errors against origin/main" would have measured the whole branch delta and been red through no fault of this commit. Measured instead against the branch tip immediately before it -- tsc -p tsconfig.test.json --noEmit reports 989 before and 989 after. Twenty-four typed-literal sites across ten test files, none of them CI-gated, plus the two runtime sites above which are. Note the deliberate side effect: removing a key from the defaults also stops the group add round-trip from re-adding it to a file that never carried it. Nothing in src reads either key, so no behavior changes. BREAKING CHANGE: `matching.max_candidates_per_step` and `detect.shared_libs` are no longer part of the group.yaml schema and are no longer written into generated templates. Existing files carrying them continue to parse and retain them. src tsc clean; 1189/1189 across the group unit, group integration, locale-parity, help-registration and tool-schema suites. * docs(group): map PR #3020 review findings to the commits that close them Retitles the ledger to hold one section per reviewed PR and adds #3020's ten findings. Two things are stated rather than claimed away: `abda0d041` closes three findings because they are one code block plus the test that pins it, and the suppressed-stage marker is a coupled set because the renderer consumes the field the earlier commit introduces. Also records what is NOT closed here -- the PR description's false claim about `max_candidates_per_step` lives outside this branch. * refactor(group): apply simplify-pass findings Four cleanup agents (reuse, simplification, efficiency, altitude) over this run's diff. Efficiency was clean. The rest found five things worth fixing, two of which were real gaps rather than style. `recordedMatchStages` filtered unknown values instead of rejecting the list. That inverted the tri-state on the one field built to prevent exactly this conflation: a stale `['bm25']` -- the scenario its own comment cites as the motivation -- survived as `[]`, which on this field MEANS "measured, nothing was suppressed". A confident clean answer manufactured from a value we could not read. Now all-or-nothing, matching `recordedRepoList`. `gitnexus group contracts` showed nothing after an exact-only sync. The human renderer destructures a fixed field list and gates its incompleteness warning on `truncated`, so the marker reached the MCP payload and the JSON output but not the listing an operator actually reads. It now warns, separately from the `truncated` warning, because the remedies differ: one says fix the repo, this one says re-run without the flag. `verbose` was still coerced with `Boolean()` in the same call whose tool description this branch changed to promise "PARAMETERS ARE VALIDATED". Validated now, and added to the tool schema -- it was read by the backend and advertised nowhere. Reuse: the thrift wildcard-matchable pair existed twice, near-verbatim, in `sync-exact-only` and `registry-suppressed-stages`. Both now call a shared `makeWildcardPair` fixture, so the shape `runWildcardMatch` fires on is defined once. Simplification: dropped a `Set` built per sync over a list that only ever holds zero or one entries; iterating `Object.keys(STAGE_COUNTS) as MatchType[]` also keeps the exhaustiveness the `Record` was built for, which `Object.entries` had discarded. Deliberately not done, with reasons: a schema-driven unknown-parameter layer at the MCP chokepoint (five parameters are read by backends and declared in no schema, so a strict layer rejects working calls today, and it cannot produce the "was removed" message finding 3 is about); folding the marker into `GROUP_IMPACT_TRUNCATION_REASONS` (reverses a recorded plan decision and the bridge scope is an open question for the maintainer); a per-stage suppression cause `Record` (no second suppressor exists -- speculative); collapsing the six `detect` extractor branches into a table (a real generalization, but a refactor outside this diff); and converging an untouched pre-existing CLI test onto the new manifest helper (it captures a value the helper does not return, so the change risks more than the duplication costs). tsc clean; eslint 0 errors (1 pre-existing warning); 1085/1085. * docs(group): remove REVIEW-FINDINGS-MAP.md Removes the findings-to-commits ledger from the source tree. Note for anyone reading this in history: the file was introduced on main by #3012 and carried that PR's findings map; this branch had appended a #3020 section. Deleting it drops both. #3012's content is recoverable with `git show 2c0fb7753:gitnexus/src/core/group/REVIEW-FINDINGS-MAP.md`. * fix(group): stop cross-repo impact and trace claiming a narrowed graph is complete Closes the half of the suppressed-stage finding that was deferred. The reviewers were right that deferring it was the weak point: the motivating harm was named as `group_impact` and cross-repo `trace` traversing a graph missing real edges, and those were exactly the surfaces left uncovered. The deferral rested on an assumption that does not hold. "It is already blind to this, so we do not make it worse" is false: `--exact-only` was inert before this PR, so the number of narrowed registries in the world goes from zero to nonzero exactly when this lands. The blindness was harmless only while narrowing was impossible. And silence there is not neutral -- `cross-impact.ts` documents `truncated: false` as an affirmative completeness claim, so those tools were about to start asserting a complete answer over a knowingly short graph. `suppressedMatchStages` now rides the bridge the same way `unreadableRepos` does: persisted in meta.json (no BRIDGE_SCHEMA_VERSION bump -- meta fields have this precedent), read back all-or-nothing, and carried across the preserve path through `refreshPreservedBridgeMeta`'s diagnostics so a preserved bridge keeps the marker of the sync that actually built it. `crossRepoCompleteness` folds it in, which is what makes this one change reach all three surfaces -- that function is by design the ONE computation behind the truncation triple. Precedence is explicit: an unreadable or unaccounted repo outranks a suppressed stage, because it is the more serious structural gap and its remedy has to be the one reported. `'suppressed-stage'` is a new member of the truncation-reason union rather than a reuse of `'incomplete-sync'`. The earlier decision not to touch that union was about not conflating remedies -- telling an agent to repair a repo that read fine, for a narrowing it requested. A distinct member preserves that reasoning while letting the answer stop claiming completeness, which is what reusing the existing member would have destroyed. The union's guard test did its job: adding a member failed the check that every reason is explained on the agent-facing surface, so the impact tool description now names this one and its distinct remedy (re-run WITHOUT the flag; nothing failed to read). `group status` and its CLI renderer surface it too, on the populated case only -- absent is a registry predating the field and empty is the ordinary clean sync; neither earns a line. Deliberately still not done, and why: a repo-wide unknown-parameter layer for every MCP tool. Five parameters are read by backends and declared in no schema (`subgroupExact`, `unmatchedOnly`, `showClusters`, `showProcesses`, and `verbose` until this branch declared it), and three tools dispatch with no schema entry at all, so a strict layer rejects working calls until each is reconciled. That reconciliation is the work; the layer is the cheap part. It also cannot produce the "was removed and is no longer accepted" message the retired-parameter guard exists to give. tsc clean; eslint 0 errors (2 pre-existing warnings); 1159/1159 across the group unit, group integration and tool-schema suites. * fix(group): make the suppressed-stage signal actually reach its readers Applies the mechanical findings from the code review of the previous commit. That commit claimed cross-repo impact and trace stop reporting a narrowed graph as complete. Trace did; impact did not, and two operator-facing messages said something false. Four reviewers plus the cross-model pass converged on the same two defects, and the untested seams were exactly where they were. `runGroupImpact` recomputed the truncation reason and hardcoded its fallback, so it could never emit 'suppressed-stage' -- the value the previous commit added to the union and documented in the tool description. Every narrowed-but-readable bridge was reported as 'incomplete-sync', telling the caller to repair a repo that read fine. It now propagates the bridge's own reason, as cross-trace.ts already did. The preserve path stamped this run's request onto an older bridge. When no repo can be read the database and registry are kept from an earlier sync, so meta.json has to keep describing that sync; instead `{ ...existing, ...diagnostics }` overwrote its marker, leaving contracts.json, meta.json and bridge.lbug describing three different runs. Currently masked by unreadable-repo precedence, one loosened condition from a live wrong verdict. `group contracts` printed "the last sync did not record which repos it could read" after any exact-only sync: truncated was set with both repo lists empty, so the message fell through to the wrong branch. It is now gated on the reason, not the flag. `group impact` likewise blamed the local walk for a floor the flag caused. The tri-state reader is now defined once, in the leaf module whose own comment says it exists so this exact duplication cannot recur -- it had been copied into bridge-db.ts within one commit of that comment being true. Both agent-facing descriptions now name the field. The previous commit added it to three payloads and documented it on none. Tests cover what shipped green: the preserve path for both artifacts (verified by mutation -- reintroducing the stamp turns exactly one test red), and the reason's REACHABILITY. The existing guard only asserted each reason is described, which is why a documented-but-unemittable value passed it. Also corrects a comment that said the marker is deliberately not folded into the truncation triple. True when written; false one commit later. tsc clean; eslint 0 errors; 1163/1163 across the group unit, group integration and tool-schema suites. * fix(group): drop verbose from the MCP surface, fail a superseded bridge closed Two maintainer-directed findings from the review. verbose is removed from the group_sync MCP schema and from GroupService, and kept on the CLI. The parameter never did what either description claimed: the gates emit workspace-dependency discovery stats and one aggregate manifest line, not "each cross-link". Worse, they emit them through the server's logger, which an MCP caller cannot read at all -- so advertising it introduced precisely the kind of knob this PR exists to delete, in the PR that deletes them. SyncOptions keeps the field and the CLI keeps --verbose, because a CLI user really can see that output; its help now says "Show additional sync diagnostics", which is what it shows. It was added to the MCP schema earlier in this same PR, so there is no published compatibility burden in taking it back out. A caller that still sends it is ignored rather than refused: it was never a documented parameter, and the retired-name guard is reserved for ones this tool actually withdrew. The second fixes a split-brain the completeness work made materially worse. When contracts.json commits and the bridge write then fails, the previous database stays in place describing an EARLIER sync. Until now it kept vouching for itself, so group_impact could traverse the superseded graph and call its answer complete while group_contracts reported the advanced registry -- two public surfaces making contradictory epistemic claims out of one sync. That was tolerable when the disagreement was about counts. It is not, now that suppressed-stage makes completeness a correctness property. markBridgeProvenanceUnknown withdraws the claim without touching the database: bridgeMetaMatchesFile already gives provenanceUnknown highest precedence and refuses to vouch for the pair, so cross-repo answers downgrade to a floor until a sync succeeds. Deliberately not a re-stamp -- the metadata still describes the database it was written for, and saying otherwise recreates the mis-pairing the preserve path avoids. Deliberately not a delete -- the old graph is still worth having as a floor, it just stops being called complete. Best-effort, because it runs inside a failure handler and must not replace a reported bridge failure with an unrelated one; the warning now states which of the two happened. Shared registry+bridge generation identity is the architectural fix and is deliberately NOT attempted here. This is the PR-sized containment. Verified by mutation, both directions: neutering the withdrawal turns the new test red, and a control pins that a healthy sync does not withdraw provenance -- otherwise every successful run would report its own answers as a floor. tsc clean; eslint 0 errors; 1185/1185. * refactor(group): apply simplify-pass findings Four cleanup agents over the last five commits. Efficiency was clean and traced why: the containment helper is failure-path only, the reason ternary sits after the fan-out loop, and the tri-state readers run once per artifact read. The strongest finding was one the diff itself proved. `refreshPreservedBridgeMeta` enforced the never-persisted rule for `repoListsUnreadable` and `pairedWithDatabase` with two deletes in its own body, under a comment noting it was the only code that read metadata and wrote it back. That held exactly as long as there was one such caller. `markBridgeProvenanceUnknown` made it two, and inherited nothing. The strip now lives in `writeBridgeMeta`, so every writer gets it and no future one can forget; `pairedWithDatabase` is the dangerous one, because persisted it tells every later reader the pair was verified when nothing verified it. `group impact` still printed "fan-out stopped early" whenever `truncatedRepos` was non-empty — but the bridge's incomplete repos are unioned into that list even when zero crossings were attempted, so a structural gap was reported as a runtime one, with the only working remedy omitted. That is the same false-cause shape the contract listing was re-gated for one commit ago, left live one command over because the new reason was bolted in front of the old branch rather than replacing the thing it branched on. Now keyed on the reason. The `?? 'incomplete-sync'` arm in cross-impact was unreachable: reaching it needs `truncated` true with all three of its inputs false, which `truncated = runtimeTruncated || bridge.truncated` forbids. Flattened. Also: a `recordedMatchStages` insert had split `crossRepoCompleteness` from its own JSDoc; one new test was a strict subset of another; and the bridge-failure warning interleaved concatenation with a mid-chain ternary. The new invariant assertion was caught being VACUOUS by mutation before it shipped — seeded with a valid repo list, `readBridgeMeta` never sets the reader-only field, so it passed with or without the strip. The fixture now seeds an unreadable list, and both it and the pre-existing assertion go red when the strip is removed. Deliberately skipped, with reasons: a shared `firstTruncated` fold over `TruncationFields` (the right altitude, but it changes cross-trace's return assembly and that surface separately documents a 'timeout' rung it cannot emit — a behavior change, not a cleanup); a reason-keyed `explainFloor` helper across all four CLI renderers (real, but a four-site refactor); narrowing the persisted stage vocabulary to a `SuppressibleStage` alias (would be undone by the very extension the field was modelled as a list to allow); moving `verbose` to `logger.debug` and deleting `SyncOptions.verbose` (the maintainer explicitly directed keeping both); and merging the two tri-state readers behind a predicate (they are adjacent in one file now, so a tightening applies to both by inspection — the duplication the comment warned about was cross-FILE). tsc clean; eslint 0 errors; 1164/1164. * fix(group): address gitnexus-check findings Seven bot comments across two review rounds; five distinct after dedup. Four were valid and are fixed, two were already resolved by later commits the bot had not seen. The validator could throw from its own error path. `JSON.stringify` is the right renderer there — it is what distinguishes the string "false" from the boolean, which is the entire point of the message — but it throws on a BigInt and on a cyclic object. So a validator promising a structured `{ error }` instead rejected, and `callTool` is reachable directly, so neither input is hypothetical. Guarded, keeping the distinction and falling back for the shapes that cannot serialize. An unreadable suppression record read as "nothing was suppressed". `recordedMatchStages` is all-or-nothing by design, so garbage collapses to `undefined` — and the consumer treated `undefined` as an empty measurement, throwing that safety away and reporting a registry it could not parse as complete. Present-but-unreadable now forces the floor, while absent stays legitimate: a registry written before the field existed has no opinion and should not be dragged to a floor for it. Two test-side findings, both real and both invisible to CI because `tsconfig.json` is src-only. Three `mock.calls[0][1]` accesses did not type-check against a zero-arg mock, and four assertions read `truncationReason` / `riskEpistemic` straight off `CrossRepoCompleteness`, which is a discriminated union carrying them on one arm. Also removed a `StoredContract` import that went dead when those fixtures moved to `makeWildcardPair`. Worth recording: U6 set a test-config gate at 989 errors and later commits walked it to 994 without anyone re-measuring — the bot caught three of the five. Now 987, below the original baseline. Already fixed, not by this commit: the preserve-path stamp the bot flagged against |
||
|
DuduPhudu
|
2c0fb7753c
|
fix(group): stop reporting what could not be measured as a measurement of zero (#3012)
* fix: surface unreadable group indexes and escape raw NUL bytes in source Two independent diagnostics failures, both of which turn a real error into a confident, benign-looking answer. **Unreadable member repos (#3011).** `syncGroup` wrapped `initLbug` plus all contract extraction for each member in a bare `catch {}` that pushed the repo onto `missingRepos` and discarded the error. A LadybugDB storage-version mismatch therefore surfaced as "repo not found", `group sync` printed `0 contracts, 0 cross-links` and exited 0, and the existing contracts.json was overwritten with an empty registry. The two states need different answers from the operator — a missing repo must be indexed, an unreadable one is usually version skew or a lock — so they are now separate: - the caught error is logged with the repo, group path and lbug path - `unreadableRepos` is tracked alongside `missingRepos` on `SyncResult`, persisted (optionally, so older registries still parse) on `ContractRegistry`, and threaded through `GroupService` sync/status - `group sync` reports both before the cascade counts, since an unread repo is the likely explanation for a small or empty count - `group status` reports unreadable repos separately; calling them "missing" actively misdescribed them - when EVERY configured repo fails to open, the write is skipped: an extraction that read nothing is not evidence the group has no contracts, and replacing a good registry with an empty one loses data while reporting success **Raw NUL bytes (#3010).** `sync.ts` and `free-call-fallback.ts` each used a NUL as a join delimiter, written as a literal 0x00 instead of `\0`. Identical at runtime, but it makes the file test as binary: `file(1)` reports `data`, ugrep returns empty with exit 1 — indistinguishable from "no match", with no message — and BSD grep replaces matching lines with "Binary file ... matches". A search that should hit comes back as a confident "not present". Both now use the escape, and a unit test fails on any raw control byte in src/ so it cannot silently return. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(hygiene): guard every tracked source file against a raw NUL, not just src/ The guard added with the NUL escapes only scanned gitnexus/src for .ts/.tsx. Neither prior recurrence of this defect in this repo was in that scope: |
||
|
Gergő Magyar
|
7f7255aef8
|
fix(analyze): load VECTOR before the incremental writeback touches embedding rows (#2623) (#2624)
* feat(lbug): add ensureEmbeddingRowDmlSafe VECTOR gate for embedding-row DML LadybugDB refuses every mutation of a table carrying an HNSW index while the VECTOR extension is not loaded on that connection: DELETE and CREATE raise a Binder exception, DROP TABLE is refused while the index references it, and SET segfaults the process. Dropping the index is not an available recovery either — CALL DROP_VECTOR_INDEX is itself a VECTOR-extension function and is undefined in exactly that state. Add a single primitive that loads VECTOR under the analyze install policy and, only when that fails, reads CALL SHOW_INDEXES (which works without the extension) to decide whether an index actually exists to trip over. No call sites yet. Refs #2623 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(lbug): pin the #2623 VECTOR gate for embedding-row DML Three cases: no index + VECTOR unavailable stays safe (no needless escalation); index present + VECTOR unavailable is reported blocked AND the raw deleteNodesForFiles genuinely throws 'extension is not loaded' (proving the hazard is real, not theoretical); index present + VECTOR loadable is safe, the delete works, and the HNSW index survives — the invariant run-analyze relies on when it keeps the index across a surgical incremental run. Refs #2623 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(analyze): load VECTOR before the incremental writeback touches embedding rows Incremental analyze died on every content change once a repo had built code_embedding_idx: Analysis failed: Binder exception: Trying to delete from an index on table CodeEmbedding but its extension is not loaded. The surgical writeback's first statement is deleteNodesForFiles' CodeEmbedding join-delete, but nothing on that path loaded VECTOR until Phase 4 — so the engine refused the delete. This is an ordering defect, not an environment one: it reproduces on machines where VECTOR loads fine. The dirty-flag recovery then forced a full rebuild on the next run, which is why it read as 'just slow'. Call ensureEmbeddingRowDmlSafe() once, before the escalation gate and before any row is touched — the same 'index lifecycle before row DML' seam dropSearchFTSIndexes occupies for FTS (#2589). Unconditional, because a DB carrying the index from an earlier --embeddings run hits the same wall on a plain incremental run. When VECTOR truly cannot load the table is immutable (the index cannot be dropped without the extension either), so the run falls through to the existing wipe-and-COPY escalation with a message naming cause, consequence and remedy. Fixes #2623 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(analyze): pin the #2623 VECTOR-before-embedding-DML ordering end-to-end Sibling of the #2589 FTS drop-before-delete suite, same shape: drive the real runFullAnalysis incremental path over a real git repo and a real LadybugDB, seed real embedding rows, build the HNSW index, then assert the index state at the exact moment deleteNodesForFiles is invoked. Both cases were confirmed to discriminate — with the run-analyze change reverted they fail with the reported 'Trying to delete from an index on table CodeEmbedding but its extension is not loaded', and pass with it: - surgical path: the run completes, the index is still present AND extension_loaded at delete time, exactly one row per nodeId survives, and the untouched file's rows are preserved - blocked path: with GITNEXUS_LBUG_EXTENSION_INSTALL=never the run escalates to a full DB write and says so, instead of crashing Also applies prettier's reindent to the run-analyze log ternary. Refs #2623 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(lbug): cite the pinned LadybugDB version in the #2623 probe note The probe matrix behind ensureEmbeddingRowDmlSafe was first recorded on 0.18.0, but gitnexus/package-lock.json pins 0.18.2 (#2587). Re-ran every case on 0.18.2: refused DELETE, refused CREATE, SIGSEGV on SET, DROP_VECTOR_INDEX undefined, DROP TABLE refused, SHOW_INDEXES readable with extension_loaded intact. Identical on both, so the design is unchanged — only the citation was wrong. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(analyze): preserve embeddings across the VECTOR-blocked rebuild, and check the catalog before loading Three follow-ups from reviewing the fix itself. 1. Data loss on the blocked path. Escalating wipes the DB files, and Phase 3.5 restores embedding rows from cachedEmbeddings — which deriveEmbeddingMode only populates when meta.stats.embeddings > 0. A DB holding embedding rows that its meta does not account for therefore had every vector destroyed silently by a rebuild it never asked for. Probe on a 3-file repo: 3 rows before, 0 after, no warning. Read the rows before escalating (a plain MATCH, no extension needed) so the existing restore has something to restore, and say so in the log. The blocked-path test now asserts the seeded rows survive exactly once, and that assertion fails without this rescue. 2. Catalog before extension. ensureEmbeddingRowDmlSafe loaded VECTOR first and only read SHOW_INDEXES on failure, so every incremental analyze on a machine without VECTOR paid a bounded out-of-process INSTALL attempt plus an 'extension unavailable' warning — including repos that never built an embedding index and can never hit this bug. One local catalog read settles that case first; the load is attempted only when an index actually gates DML, or when the catalog cannot be read. 3. Dead branch. targetConn is always the module singleton there, so the isSharedSingletonConn ternary could never take its second arm. Collapsed to withConnLock. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(doctor): live-probe the VECTOR extension instead of printing the static platform capability Review finding on #2624 (MEDIUM), and exactly what #2623's reporter hit: doctor printed 'VECTOR index: available' — derived from a static platform check — while every incremental analyze on the same machine was dying on an unloaded VECTOR extension. The FTS line was switched to a live LOAD probe for the identical contradiction under #2374; VECTOR now gets the same treatment. probeVectorExtensionLoad shares the FTS probe's implementation (bounded, offline-safe, never runs the installer) and doctor's semantic-mode line now follows the probe, not the platform: without a loadable extension the vector index can be neither built nor queried, so search really is on exact scan. The load-error classifier's remedies are label-parameterized so the VECTOR row stops dispensing FTS-specific advice — 'run analyze --repair-fts' repairs FTS indexes only and was actively wrong for a missing vector extension. Default label stays 'FTS'; every existing caller and pinned remedy string is unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(lbug): remove the stale Windows VECTOR gate — the extension ships for win_amd64 The codebase categorically refused VECTOR on Windows (platform !== 'win32' in isVectorExtensionSupportedByPlatform, plus a hard early-return in loadVectorExtension) on the strength of an early-era report that in-process INSTALL VECTOR could SIGSEGV (#1365). That belief is stale, verified directly: - the extension server hosts win_amd64 VECTOR artifacts for every 0.18.x extension version — v0.18.0 and v0.18.1 both serve a real 14 MB PE32+ DLL (curl-probed; 'file' confirms PE32+ x86-64) - the pinned 0.18.2 core resolves its extension directory to 0.18.1 (strace-verified LOAD open()), so the pinned version's Windows artifact exists too - INSTALL now runs in a spawned child (installDuckDbExtensionOutOfProcess), so even a crashing installer kills only the child and degrades to unavailable — the original hazard cannot reach the parent process any more Windows now takes the same runtime path as every other OS: try LOAD, install out-of-process when policy allows, degrade to exact scan when it truly fails. The MCP semantic-search lane loses its static platform gate too — it always attempts the vector index and falls back to the exact scan on runtime failure, with a once-per-backend diagnostic naming the real error instead of a platform-policy message. isVectorExtensionSupportedByPlatform is deleted; getRuntimeCapabilities reports the platform capability as available everywhere and defers machine truth to the live probe. Windows CI is the enforcement: the vector suites skip visibly only when the extension genuinely cannot load, so green Windows lanes now actually exercise VECTOR instead of silently skipping by policy. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(lbug): pin the catalog-read-failure fallback in ensureEmbeddingRowDmlSafe Review finding on #2624 (LOW): the one branch where the gate cannot cheaply prove safety — SHOW_INDEXES itself erroring — was exercised only by inference. Force it with a Connection.prototype.query spy over the real DB: the catalog read fails, and the gate must fall through to actually attempting the extension load (asserted via the recorded statement stream) rather than guessing, returning true here because the extension is loadable. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(mcp): load VECTOR on the pool's shared Database so the semantic vector lane actually works Review finding on #2624 (MEDIUM): extension load scope is per-Database (probe-verified — LOAD on one connection enables QUERY_VECTOR_INDEX on every connection of the same Database), and the pool pre-warm loaded only FTS. So LocalBackend's vector lane has ALWAYS raised 'Catalog exception: function QUERY_VECTOR_INDEX is not defined' through the pool and silently fallen back to the exact scan — repos above the 10k exact-scan cap got empty semantic results. The serve path was unaffected (the embedding pipeline loads the extension itself). Mirror the FTS line at BOTH load sites — doInitLbug's pre-warm and initLbugWithDb's external-Database adoption — under the same load-only contract (the read pool never triggers a network install), tracked by a new SharedDB.vectorLoaded flag reset where ftsLoaded resets. The new pool test is discriminating and deliberately closes the writable core adapter before the pool opens: a shared/injected Database would inherit the VECTOR load from test seeding and pass either way, so the case forces the pool onto its OWN fresh read-only Database where only the pre-warm can make the lane legal. Verified: fails at the pre-fix tree with the exact Catalog exception, passes with the fix. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * ci: run the #2623 ordering suite on Windows/macOS and pre-install VECTOR alongside FTS Two review findings on #2624, both landing in existing seams: - scripts/cross-platform-tests.ts gains incremental-vector-extension-ordering .test.ts: the win32 VECTOR gate is gone in this PR, so the #2623 drop-ordering + blocked-path escalation must be proven on the windows-latest native addon, not just Ubuntu. (The review's claim that lbug-delete-nodes-for-files.test.ts was also missing was wrong — it has been on the roster since #2409.) - scripts/ensure-fts.ts now pre-installs VECTOR under the same best-effort auto-policy contract, so every sharded CI process LOADs from ~/.lbdb instead of racing its own bounded out-of-process INSTALL; the workflow's extension cache already covers it (path is the whole extension dir — key kept for cache continuity). The cross-platform job sets GITNEXUS_REQUIRE_VECTOR=1 beside GITNEXUS_REQUIRE_FTS so a genuinely unavailable VECTOR is a loud failure, never a silent skip. Windows/macOS cannot be executed locally; the PR's CI lanes are the proof for this commit. Linux smoke: ensure-fts.ts reports both extensions ready; all 79 roster entries resolve. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(pool): register loadVectorExtension in the pool unit-suite mocks The pool adapter's new loadVectorExtension import surfaced in four suites that mock lbug-adapter.js with explicit factories (vitest fails loudly on a missing mocked export). Register the export in each — resolving false where the suite's world assumes no vector, true where it mirrors FTS — and extend lbug-pool-fts-load.test.ts, the suite that owns pre-warm extension loading, with the vector pair: successful load cached per shared Database, failed load retried on the next open, both pinned to policy load-only. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(analyze): use POSIX literals for graph paths in the #2623 ordering suite First Windows CI run of this suite (it joined the cross-platform roster this PR) failed with 'Parser exception: Invalid input <MATCH (n:Function) WHERE n.filePath = '>' — path.join produces backslashes on Windows, and a backslash inside the seed helper's single-quoted Cypher literal breaks the parser. The graph stores repo-relative filePaths with forward slashes on every OS, so graph-side paths are POSIX literals now (the incremental-orchestration convention); path.join stays only for real filesystem access. The same Windows lane also proved the substance this suite exists for: lbug-vector-extension passed 7/7 on windows-latest — the extension installed, loaded, and built a real HNSW index there — and the pool vector-lane and DML gate suites passed too. This commit fixes the harness, not the fix. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Gergo Magyar <abhigyan1.patwari@gmail.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Gergő Magyar
|
b98f6e458f
|
fix(lbug): recognize Windows missing-shadow error so serve repo-switch recovers (#2382) (#2387) | ||
|
Gergő Magyar
|
fb068a9480
|
fix(group): pin repos during sync so large groups resolve cross-links (#2191)
* fix(lbug): pin repos to exempt them from automatic pool eviction [#2189] Add a pinnedRepos set and pinRepo/unpinRepo to the LadybugDB pool adapter. evictLRU and the idle-timeout sweep skip pinned repos; closeOne clears the pin on teardown so explicit close always wins and pins never leak across operations. Behavior is byte-identical when nothing is pinned. Bounded multi-repo callers (group sync) can now keep more than MAX_POOL_SIZE repos resident through deferred cross-repo resolution. * fix(group): pin repos during sync so >MAX_POOL_SIZE groups resolve [#2189] syncGroup now pins each repo immediately after initLbug and releases the pin (unpin then close) in the finally. This keeps every group member resident through the deferred manifest/workspace resolution that runs after the init loop, so cross-links anchor to real graph symbols instead of falling back to synthetic UIDs when a group has more than MAX_POOL_SIZE repos. Release is unpin-before-close plus closeOne's own pin-clear, so pins never leak across syncs in the long-lived MCP server even on error. * style(test): apply prettier formatting to #2189 test files * fix(review): apply autofix feedback Clarify the pinRepo docstring: the pin does not survive teardown (closeOne clears it) and the repoId must match the key passed to initLbug. Addresses a code-review finding that the prior 'or later holds' wording contradicted closeOne's unconditional pin-clear. * refactor(lbug): reference-count pool pins so overlapping holders are safe [#2189] Change pinnedRepos from Set<string> to Map<string,number>. pinRepo increments the lease count; unpinRepo decrements and deletes the key at 0 (flooring at zero, unknown-id no-op). evictLRU, the idle sweep, and closeOne are transparent to the swap (has()/delete() keep their semantics: skip while count>=1, force-clear on teardown). A boolean Set could not represent two simultaneous holders, so the first release wrongly cleared a pin another holder still needed — the concurrent overlapping group_sync teardown race from the PR #2191 review (Finding 1). Reference counts let two windows of one sync, or two concurrent syncs sharing a repo, coexist safely: the repo stays exempt until the last lease releases. * refactor(lbug): pinRepo returns a leak-proof release disposer [#2189] pinRepo now returns a release() disposer (mirroring addPoolCloseListener) that releases its own lease exactly once — a double-call is a guarded no-op, so it can never over-decrement a sibling holder's reference count. Callers can use the leak-proof pattern `const release = pinRepo(id); try { … } finally { release(); }`. unpinRepo stays exported for explicit pairing. Addresses the PR #2191 review's P3: the exported pin primitive had no built-in pairing, so a caller that forgot to unpin would disable eviction for a repo permanently. * refactor(group): windowed manifest resolution bounds sync pool residency [#2189] Replace whole-sync pinning with windowed deferred resolution. The init loop extracts contracts without pinning (repos evict naturally); manifest links are pre-sorted and partitioned into windows whose referenced in-group repos number <= getMaxResidentRepos(), and each window re-inits + leases only its own repos, resolves, then RELEASES the leases (not closeLbug — released repos stay evictable for the LRU, which avoids stomping a concurrent MCP reader). Peak per-sync pool residency is now bounded by getMaxResidentRepos() distinct repos regardless of group size, removing the unbounded-mmap crash risk the PR #2191 review flagged (Finding 3) — without a new magic-number threshold (it reuses MAX_POOL_SIZE via an intent-named accessor). #2189 stays fixed: each window resolves against live, freshly-leased pools, so cross-links anchor to real graph symbols. partitionManifestWindows is a pure, unit-tested function (every link in exactly one window — the contract-dedup invariant). New sync-windowed-resolution.test.ts asserts the partition bound and, through the real pool, that concurrently-open Databases never exceed the resident cap for a group larger than it. Rewrote the sync.test.ts pinning block (init loop no longer pins; per-window lease/release; release-not-close). |