scriptoriumadmin/GitNexus
1
0
Fork 0
mirror of https://github.com/abhigyanpatwari/GitNexus.git synced 2026-10-08 03:08:13 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 7
1822 commits 123 branches 1043 tags 254 MiB
Commit graph

5 commits

Author SHA1 Message Date
Livio Gamassia
d546fa3cce
fix(storage): rename index metadata to gitnexus.json with dual-write compatibility (#2363) 2026-07-03 19:32:59 +01:00
Sparsh
028bd11053
fix(group): cache read-only bridge handle to fix Windows @group reopen (#2274) (#2313) 
* fix(group): cache read-only bridge handle to fix Windows @group reopen (#2274)

A long-lived MCP server opened bridge.lbug read-only, queried, and closed
it on every @group trace/impact call. On Windows the in-process reopen of the
same file fails (the OS handle is not fully released before the next open races
in), so repeated @group calls broke. #2269 fixed Linux/macOS by skipping
CHECKPOINT on read-only handles; Windows stayed broken.

Instead of fighting LadybugDB's Windows close/reopen timing: cache one
read-only handle per groupDir and reuse it across calls (open-once-per-process
already works on Windows). getCachedBridgeReadOnly:
  - reuses a single handle keyed by resolved groupDir,
  - invalidates on mtime change (external writer / re-sync),
  - invalidates explicitly before same-process writes (writeBridge),
  - guards concurrent first-open with an in-flight promise (no handle leak),
  - closes all handles on process exit.

closeBridgeDb now no-ops for the cached handle (cache owns its lifetime);
uncached/writable handles are unaffected. ensureBridgeReady uses the cache.

The in-process write->read reopen of the same bridge.lbug file remains a known
LadybugDB Windows limitation, so the existing reopen tests stay win32-skipped.
A new cache-aware itCacheReopen gate applies to the 3 new tests whose setup
requires write-then-read in the same process (same class as itLbugReopen). The
cache itself exercises read->read reuse and is unaffected.

* fix(group): harden bridge RO-handle cache for concurrency, lifetime & Windows (#2313 review)

Addresses the tri-review + Copilot findings on the read-only bridge-handle cache:

- P1 (F2): serialize queryBridge per cached handle via a per-handle FIFO lock
  (the conn-lock.ts chain mechanic, keyed per cache entry, not the global lock).
  Two concurrent @group callers sharing one lbug.Connection can no longer
  dispatch two queries at once (the heap-corruption hazard). Uncached/writable
  handles skip the lock at zero cost.
- P1 (F3): refcount lease — getCachedBridgeReadOnly acquires, closeBridgeDb
  releases (no caller change). The native close is deferred until in-flight
  readers drain (refs===0) and runs exactly once (closeStarted guard).
  invalidateBridgeCache and the mtime-evict path share one evict/close path.
- Windows: bounded drain in evictBridgeEntry — a concurrent group_sync waits
  (<= WINDOWS_DRAIN_TIMEOUT_MS) for readers to release before the atomic rename
  on win32 so it stays clean; POSIX remains fully non-blocking; single-threaded
  sync still closes-before-rename on all platforms.
- P0 (F1/F6): gate the mtime cache test with itCacheReopen (win32-skipped) and
  drop the manual invalidate so writeBridge self-invalidation is under test;
  add an external-writer (fsp.utimes) reopen case.
- Windows coverage (F9): new cross-process integration test seeds bridge.lbug
  in a separate tsx process, so read->read handle reuse is proven on win32 CI
  (not skipped). Plus concurrent cold-open dedupe coverage.
- P2/P3: scope the Windows NOTE to read->read (F4); JSDoc the closeBridgeDb
  release/close contract (F5); drop the if-branch in the B2 probe (F7); revert
  incidental Prettier churn in cross-impact.ts (F14); fix the stale describe
  header (F15); document the beforeExit/signal and ENOENT-mtime behavior
  (F11/F13).

tsc clean; group unit + integration suites green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(group): run the B2 rename-clash probe on win32 via cross-process seed (#2313 review)

Moves the B2 "external rename while a cached RO handle is held" probe out of the
unit suite (where it was win32-skipped, because its in-process writeBridge->RO-open
is the unfixed Windows reopen) into the cross-process integration test, where a
separate-process seed makes the RO open clean. The probe now RUNS ON WIN32 CI and
empirically answers whether an open RO handle blocks an external atomic rename over
bridge.lbug — the assumption under writeBridge's invalidate-before-rename and the
win32 drain.

Hardened (per adversarial review) so a win32 RED is the real steady-state share-mode
signal, not an artifact:
- use production retryRename (not bare fsp.rename) so transient EBUSY/EPERM from the
  Windows AV/indexer scanning the fresh temp file is absorbed; a RED then means the
  rename is blocked even after retries (FILE_SHARE_DELETE absent -> invalidate-before-
  rename is load-bearing).
- stage the byte-identical replacement BEFORE opening the RO handle, so no second OS
  handle touches bridge.lbug while LadybugDB holds it (avoids a FILE_SHARE_READ red for
  the wrong question).
- drop the post-rename query (handle survival is covered by the reuse test); the probe's
  sole verdict is whether the rename is blocked.

Removes the old win32-skipped unit B2 (a strict subset of the new probe).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Gergő Magyar <gergomagyar@icloud.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-30 07:49:47 +01:00
Gergő Magyar
1a03c8527a
feat(group): cross-repo call trace using PDG (#2269) 
* refactor(group): extract shared resolveBridgeNeighbors from cross-impact

Lift the uid-filtered consumer<->provider ContractLink join (direction +
queryBridge + row normalization + confidence sort) out of runGroupImpact's
inline Phase-2 block into an exported resolveBridgeNeighbors helper. Behavior
is unchanged for impact; the helper becomes the single shared bridge join so
the upcoming cross-repo trace path never forks its own copy of the neighbor
Cypher. Empty uid sets short-circuit without a DB round-trip.

Adds direct coverage (real bridge via writeBridge/openBridgeDbReadOnly) for
both directions plus the empty-set and unknown-uid edges.

* feat(group): cross-repo trace stitching (groupTrace + runGroupTrace)

Add GroupService.groupTrace and the pure runGroupTrace engine that stitches
per-repo CALLS/HAS_METHOD trace segments across one ContractLink boundary in
the group bridge:

  from --(local trace)--> consumer --(ContractLink)--> provider --(local trace)--> to

- Resolves from/to across all members (symbol node id == bridge symbolUid);
  same-repo endpoints delegate to a single local trace with no crossing.
- Single boundary crossing (MAX_SUPPORTED_CROSS_DEPTH); deeper crossDepth is
  clamped with a note, mirroring cross-impact.
- Discriminated GroupTraceResult union (ok|not_found|ambiguous|error) with
  per-hop repo tags, a typed crossings[] entry, and centralized degraded-state
  note constants (TRACE_NOTES). No .
- Trace-specific pair query (keeps BOTH crossing endpoints) lives in this
  module; the uid-filtered neighbor join (resolveBridgeNeighbors) is reused
  where it fits. ensureBridgeReady exported for reuse.
- New GroupToolPort methods (trace/resolveSymbol/pdgFlows) are optional so
  existing port mocks keep type-checking; runGroupTrace guards on presence.

PDG enrichment is wired as an opt-in hook (enrichSegment) — the port method is
stubbed until U4. Covered by unit tests over a real bridge + mocked port.

* feat(group): route trace tool to groupTrace on @group syntax

Wire the cross-repo trace through the existing @group dispatch:
- callTool routes trace with an @-prefixed repo to callToolAtGroupRepo, which
  forwards from/to/uid/file/maxDepth/includeTests plus the experimental
  pdg/crossDepth flags to GroupService.groupTrace. Member path in @group/path
  is advisory for trace (resolution is whole-group).
- Port gains trace/resolveSymbol/pdgFlows adapters. resolveSymbolForGroup wraps
  the shared resolveSymbolCandidates so groupTrace can locate the member repo
  and recover each endpoint node id (== bridge symbolUid). pdgFlowsForGroup is
  a degraded stub here (call-level only); U4 implements the REACHING_DEF walk.
- trace tool schema documents the @group entry point, pdg, and crossDepth.

Single-repo trace is untouched. Covered by dispatch-routing tests (@group ->
groupTrace, non-group stays local) and tool-schema assertions.

* feat(group): opt-in PDG data-flow enrichment for cross-repo trace

Implement _pdgFlowsForGroupImpl: the real REACHING_DEF anchor walk that backs
the port pdgFlows adapter (replacing the U3 call-level stub). When pdg:true and
the segment repo has a flows PDG layer, the boundary-adjacent segments carry
their intra-procedural def->use hops:

- Anchors by the boundary symbol UID (precise; avoids the by-name ambiguity the
  resolveBlockAnchor path can hit), then reuses the same span-anchored,
  bind-param-only flows query as pdg_query (BasicBlock id-prefix + [start+1,
  end+1] line window; no rel-property index, so the anchor IS the bound).
- Stays intra-procedural: data flow never crosses the repo boundary.
- pdgStampForMode probe: false -> available:false (degrade with note); the
  trace stays ok. Any query failure is swallowed (enrichment is auxiliary).

Covered by runGroupTrace enrichment tests: dataFlow attached on opt-in,
degraded note when no layer, and no pdgFlows call when pdg is omitted.

* test(group): evaluation-first cross-repo trace e2e (two real indexes)

End-to-end gate for the cross-repo trace: stands up two real LadybugDB indexes
(consumer 'frontend' + provider 'backend'), a real ContractLink bridge, and a
real LocalBackend with both repos registered, then drives the public
callTool('trace', { repo: '@grp', pdg: true }) and asserts:
  - the stitched checkout -> callUsers -(CONTRACT_LINK)-> handleUsers -> getUsers
    path, each hop tagged with its member repo
  - real REACHING_DEF data-flow enrichment of the consumer segment (userId)
  - a degraded 'No PDG layer in app/backend' note (provider has no PDG layer)
  - single-repo trace against one member is unchanged (no crossings)

Hand-persists the minimal real graph (deterministic; a full two-repo analyze is
heavier than this gate needs) and exercises real Cypher across
resolveSymbolCandidates, _traceImpl, the bridge pair query, and
_pdgFlowsForGroupImpl. Windows-skipped (describeReopen) and registered in the
cross-platform native-lbug set.

Scoped to a single @group call: opening bridge.lbug read-only a SECOND time in
one process currently fails (shared bridge open/close lifecycle, also affects
impact @group) — the pdg-omitted/clamp variants are unit-covered.

* docs(group): document cross-repo trace + PDG enrichment

ARCHITECTURE.md: trace is now group-aware; describe the @group cross-repo
stitch over a single ContractLink boundary (CONTRACT_LINK hop, crossings[],
crossDepth clamp), the opt-in experimental PDG REACHING_DEF enrichment of
boundary-adjacent segments, the symbolUid-grain join between the two stores,
and the deferred full cross-program (SDG-like) data flow. PIPELINE.md: add the
cross-trace consumer of the bridge with its pair-query rationale.

Does not touch gitnexus/CHANGELOG.md (release-owned).

* fix(review): apply autofix feedback

Apply safe_auto findings from ce-code-review (run 20260622-094243):
- local-backend.ts: drop (r: any) in _pdgFlowsForGroupImpl row map; coerce
  hop line via Number() so a nullish LadybugDB cell can't surface NaN.
- tools.ts: advertise the forwarded  param in the trace schema and add
  crossDepth maximum:10 (schema now matches what groupTrace reads).
- cross-trace.ts: parallelize per-member resolveSymbol/resolveRepo with
  order-preserving Promise.all (matches groupContext/groupQuery); add a note
  when pdg:true is passed to a same-repo trace (PDG only enriches at a
  cross-repo boundary).
- tests: remove  / tighten  (no-any rule).

Residual gated_auto/manual findings (unbounded crossing query + loop,
whole-file PDG widening on absent span, error-vs-no_path masking, top-level
try/catch parity, helper dedupe, branch-coverage gaps) are recorded in the run
artifact for the PR body.

* fix(group): skip CHECKPOINT on read-only bridge close so it can reopen

Root cause of the in-process bridge.lbug reopen failure (which broke repeated
@group impact/trace calls in a long-lived MCP server): closeBridgeDb issued
CHECKPOINT on EVERY handle, including read-only ones. A CHECKPOINT on a
read-only connection has nothing to flush but leaves a WAL/shadow lock artifact
that makes the next read-only open of the same path fail (openBridgeDbReadOnly
returns null -> 'Could not open bridge.lbug read-only'). Reproduced: open ->
query -> closeBridgeDb -> open again returned null only when the close ran
CHECKPOINT; a non-checkpoint close reopened fine, and the raw native
open/close cycle was never the problem.

Fix: tag read-only handles (BridgeHandle._readOnly, set by openBridgeDbReadOnly)
and skip CHECKPOINT for them in closeBridgeDb. Writable handles are unchanged
(they still flush before close). This is the shared bridge-db close path, so
impact @group benefits identically.

- Regression test in bridge-db.test.ts: open/query/close/open/query/open in one
  process now succeeds.
- Re-enabled the second @group call in cross-trace-e2e.test.ts (was scoped to a
  single call for this very limitation).

* fix(group): bring bridge-db close to parity with the core adapter safeClose

The bridge open/close cycle was less robust than the main graph DB's: closeBridgeDb
closed the connection/database but skipped the post-close steps the core adapter's
safeClose performs, so a rapid in-process reopen could race the OS handle release
(Windows) or an orphaned WAL sidecar. That gap is why the close-then-reopen tests
had to skip Windows.

closeBridgeDb now mirrors safeClose after closing the handle:
- waitForWindowsHandleRelease(dbPath): probe the file (+ .wal) until the residual
  Windows lock clears, so the next open does not race (warns if the budget is
  exhausted, matching the core adapter).
- finalizeLbugSidecarsAfterClose(dbPath): quarantine an orphaned WAL (shadow
  missing) so the next open replays a consistent file.

Both helpers are the same ones safeClose uses (Windows-proven via the core adapter
CI), and the bridge read open already retries transient locks. Combined with the
read-only CHECKPOINT skip, the bridge reopen is now robust on every platform, so
the close-then-reopen tests run on all platforms (Windows CI exercises them via the
cross-platform subset). No write-path behavior change; Linux/macOS unaffected.

* fix(group): bound cross-repo crossing fan-out (LIMIT + segment memoization)

Address the top review residual: the bridge crossing query was unbounded and the
crossing-selection loop could run an O(2*N) sequential trace-BFS over every
ContractLink between a repo pair.

- CY_CROSSINGS_BETWEEN now ORDERs BY confidence DESC and LIMITs to
  MAX_CROSSINGS_TO_TRY + 1; listCrossingsBetween slices to the cap and reports
  truncation. Exceeding the cap surfaces a note (no silent truncation), keeping
  the highest-confidence crossings. Aligns with the repo's anchored+LIMIT-bounded
  query discipline (LadybugDB has no rel-property index).
- The home-repo segment (from -> consumer) depends only on the consumer uid and
  the target-repo segment (provider -> to) only on the provider uid, so each is
  memoized by that uid. Many crossings sharing a consumer/provider (one client
  call linked to several providers) now cost one trace per distinct endpoint
  instead of one per crossing. A consumer whose segment already failed is skipped
  for every later crossing that shares it.

Test: two links sharing a consumer (first provider unreachable, second reachable)
assert the from->consumer segment is traced exactly once and the second crossing
wins.

* fix(group): restore Windows skip for bridge reopen tests; drop ineffective close-side probe

The previous commit flipped the bridge close-then-reopen tests to run on Windows,
betting that a close-side waitForWindowsHandleRelease + finalizeLbugSidecarsAfterClose
probe (mirroring the core adapter safeClose) would make the in-process reopen work
there. Windows CI proved otherwise: 4 writeBridge->openBridgeDbReadOnly tests fail
('expected null not to be null' — the read open returns null). The writable-close ->
read-open handoff plus writeBridge's atomic sidecar rename does not release the OS
file handle before the read open races, and the existing open-side LBUG_OPEN_RETRY
only retries lock-pattern errors, not the post-rename sidecar database-id mismatch.
macOS passes; the core adapter's own reopen also passes — this is bridge+Windows
specific.

- Revert itLbugReopen to the Windows skip (the pre-existing, correct state).
- Remove the close-side probe + finalize from closeBridgeDb: it did NOT close the
  Windows gap, and reviewers flagged it for hot-path latency (finalize ran on every
  close, all platforms) and safeClose duplication.
- KEEP the load-bearing fix — skipping CHECKPOINT on read-only handles — which fixed
  the reproduced Linux/macOS in-process reopen artifact (the real bug).

Net: Linux/macOS repeated @group impact/trace works in-process; Windows in-process
bridge reopen remains a documented limitation (unchanged from before this PR).

* fix(group): surface degraded members + cap truncation; honest crossDepth schema

Address the cross-engine-corroborated tri-review findings (Codex + Claude):
- resolveAcrossMembers / runGroupTrace now track member repos that could NOT be
  queried (resolveRepo or resolveSymbol threw) and, when the result is not_found,
  attach a degraded-member note. A transient/corrupt member DB is no longer
  silently reported as a clean 'symbol absent' not_found. (Codex B1+B3 + ce-reliability.)
- The cross-repo not_found now carries a programmatic truncated:true flag (and a
  clearer suggestion) when the MAX_CROSSINGS_TO_TRY cap was hit, so a consumer can
  distinguish 'no path' from 'cap may have hidden a connecting ContractLink'.
  (Codex B3 + ce-adversarial + ce-api-contract.)
- trace tool schema: crossDepth maximum 10 -> 1 to match the implementation's
  single-hop clamp (the schema previously advertised an unsupported 2-10 range).
  (ce-api-contract, conf 100.)

Test: a member whose resolveSymbol throws yields not_found WITH a degraded note
naming the unreachable repo (if-free responder map).

* docs(group): clarify trace @group/memberPath is advisory (resolves all members)

Tri-review (Codex ce, conf 100) caught a doc/impl inconsistency: ARCHITECTURE.md
lumped trace with query/context/impact as honoring @group/memberPath member
scoping, but cross-repo trace resolves from/to across ALL members (the member
path is advisory). Clarify the behavior and point to from_uid/to_uid for
disambiguating same-named symbols across members.

* feat(group): file-level boundary fallback so cross-repo trace works on HTTP contracts

Benchmark (bench/cross-repo-trace/) running the REAL pipeline (runFullAnalysis
--pdg -> real syncGroup -> trace @group) found that cross-repo trace returned
not_found for real HTTP links even though sync built the correct ContractLinks:
HTTP (and other source-scan) contracts hardcode symbolUid:'' (http-route-extractor),
and both cross-trace AND cross-impact join crossings by Contract.symbolUid, which
never matches an empty uid. (Pre-existing — impact @group has the same gap.)

Fix: when a crossing's symbolUid is empty, fall back to the contract's FILE — if
the user's from/to resolves into the contract file, that endpoint anchors the
boundary. CY_CROSSINGS_BETWEEN now returns consumer/provider filePath; a crossing
is kept if it can be anchored by uid OR file on each side; a fileBoundaryFallback
note flags that the boundary is file-level, not symbol-precise. This makes the
common 'trace from=<calling fn> to=<handler fn>' case work end-to-end (verified:
fetchUsers -> listUsers stitches with a CONTRACT_LINK hop + PDG enrichment, 2/2).

Limits (documented in the bench README + the note): anonymous handlers have no
named target; when several contracts share files the file fallback may attach the
wrong contractId to a correct path. The proper upstream fix is to populate
symbolUid in the HTTP extraction (benefits impact too) — the bench is its gate.

Adds a unit test pinning the empty-symbolUid file-fallback stitch.

* fix(group): resolve HTTP contract symbolUid by containment (fixes cross-repo trace + impact)

Addresses the root cause behind the cross-repo trace file-fallback: HTTP
contracts hardcoded symbolUid:'' (http-route-extractor), so both cross-trace and
cross-impact — which join crossings on Contract.symbolUid — could not traverse
HTTP links. (Also found: the pre-existing graph-assisted resolution queried the
wrong edge, CONTAINS instead of DEFINES, so it never resolved a uid either.)

Now the extractor resolves each detection to a real symbol:
- HttpDetection carries the call-site line (node.ts sets it on every express/
  fetch/axios/jquery/nest detection; express also captures the handler arg).
- resolveDetectionSymbol resolves the named handler first, else the innermost
  Function/Method whose line span encloses the call (consumer = the function
  containing the fetch; provider = the named/inline handler), over the correct
  File-[DEFINES]->symbol edge. Base-tolerant (0- vs 1-based startLine).
- Wired into both source-scan and graph-assisted provider/consumer paths.

Verified end-to-end (bench/cross-repo-trace): all 4 contracts now carry real
uids, trace is symbol-precise (GET pair -> http::GET, POST -> http::POST, no
file-fallback note), and impact @group fans out (cross_repo_hits 0 -> 1). The
cross-trace file-level fallback remains as the secondary path for truly
anonymous handlers. Adds 2 containment unit tests; 738 group/integration pass.

Languages other than JS/TS still resolve providers by handler name; their
consumers fall through to the file fallback until their plugins set the line.

* fix(group): extend HTTP symbolUid containment to all languages + nested methods

Completes the symbolUid resolution across every bundled HTTP plugin: Python, Go,
PHP, Kotlin and Java now set the call-site line on their consumer (and Feign/
named) detections, so their HTTP contracts resolve to the containing function
the same way Node/TS already did.

Also generalizes the containment query: it now matches Function/Method/CodeElement
by filePath (UNION ALL) instead of File-[DEFINES]->symbol. The DEFINES edge only
reaches a file's TOP-LEVEL symbols, so methods nested in classes (Java/Kotlin —
File defines the class, the class defines the method) were invisible; matching by
filePath reaches them. Verified against a real index (LadybugDB supports the
UNION); JS/TS still fully symbol-precise (bench 2/2), 709 group tests pass.

Residual is now only the inherent case — a fully anonymous handler with no named
callee — which keeps the cross-trace file-level fallback.

* feat(group): destination trace — follow a consumer to an anonymous handler

Handles the one inherent residual: an anonymous route handler
(`router.get('/x', (req,res) => …)`) has no symbol node at all (the file holds
only a Const + PDG BasicBlocks), so it can never be named as a trace `to`.

Adds a DESTINATION TRACE: omit to/to_uid/to_file on an @group trace and
`trace from=<consumer>` follows the consumer's outgoing HTTP call across the
bridge and reports where it lands — by route + file:line, with a notes[] entry
flagging the handler as anonymous. Implemented as a new branch in runGroupTrace
(p.destination) backed by CY_CROSSINGS_FROM (all ContractLinks leaving the
consumer repo) + stitchToDestination; the provider endpoint is labelled
'<METHOD /path handler>' when its symbolName is a generic token/file basename.

The MCP routing already omitted an absent `to`, so only the schema docs changed.
parseTraceParams now treats a missing `to` as a destination trace instead of an
error. Verified end-to-end: anonymous fixture reports
'app/frontend:fetchUsers -> app/backend:<http::GET::/api/users handler>'; named
fixture lands at the real function. Adds 2 unit tests; 915 group tests pass.

* fix(group): tri-review fixes for cross-repo trace + symbolUid resolution

Two-engine tri-review (Claude swarm+ce + Codex GPT-5.5 swarm+ce+adversarial)
surfaced these; cross-engine-corroborated unless noted.

Correctness (P1, all four lanes): destination trace reported the WRONG endpoint
— an empty-uid consumer made trace(from->from) trivially succeed, so the highest-
confidence same-file crossing won regardless of which call `from` makes.
stitchToDestination now collects ALL connecting crossings, prefers symbol-precise
hits, and returns `ambiguous` (with candidates) when it cannot disambiguate.

Correctness (P1, Codex): resolveDetectionSymbol early-returned null when
d.line==null, blocking NAME resolution for named providers that set no line
(Spring/Go/etc.). Name resolution now runs first; only containment needs a line.

Correctness (P2): resolveContainingSymbol OR-ed `line` and `line-1`, which could
mis-pick a one-line sibling. It now probes the base-correct `line-1` first and
falls back to `line` only if nothing matches.

Correctness (Codex): anonymous Express handlers emitted name:'handler' and could
attach to an unrelated fn literally named `handler`. node.ts now emits name:null
for non-identifier handlers (containment-only).

Robustness: drop the first-symbol-in-file pickSymbolUid guess from the graph
consumer/provider paths (a wrong uid would win the contractId merge); remove the
dead CONTAINS_QUERY fallback (CONTAINS is File->Folder, never a symbol) + the now
-unused pickSymbolUid/handlerName; seed destination notes with degraded-member
notes so a successful trace still surfaces them; providerLabel takes providerUid
so a resolved fn named `handler` is not mislabeled anonymous, and only true file
basenames (known extensions) — not any dotted name — count as anonymous.

API contract: a single-repo trace with no `to` now returns an actionable error
(destination trace is @group-only) instead of "symbol 'undefined' not found".

Maintainability/tests: narrow asLocalTrace per-field (drop as-unknown-as); fix the
PR's lone as-any (vi.mocked); if-free e2e teardown; qualify the bench README.

Adds ambiguous-destination, anonymous-handler-no-false-name, and single-repo-no-to
tests; redirects graph mocks CONTAINS->UNION ALL. 918 group/integration pass.

* fix(group): carry degraded-member notes through SUCCESSFUL group traces

A reviewer (koriyoshi2041, PR #2269) correctly flagged that degraded-member
resolution was surfaced only on not_found, not on a successful ok result. Group
trace resolves names across ALL members, so an ok is 'unique among the members
we could query' — if a member that threw during resolveSymbol also holds from/to,
the real answer could be ambiguous. The destination path already seeded the note
(prior commit); this extends it to the same-repo and cross-repo success paths by
seeding the dispatch notes with degradedNotes([...fromRes.degraded, ...toRes.degraded]).

Adds a regression test: reg-be throws while a same-repo trace succeeds in reg-fe;
the ok result now carries the 'could not be queried' degraded note (app/backend).

* test(bench): cover all implemented cross-repo trace cases in one runner

Replace the single named-handler script with a self-contained verify.mjs that
generates each fixture inline and exercises every implemented end-to-end case
against the real analyze -> sync -> trace/impact pipeline, asserting PASS/FAIL
(exit non-zero on failure). 10 checks across 4 scenarios:
- named handlers: 4/4 symbolUid resolved; symbol-precise GET vs POST crossing
  selection; destination trace lands at the named handler.
- anonymous handler: empty symbolUid; destination trace reports it by route with
  the anonymous note.
- impact @group fan-out (cross_repo_hits >= 1).
- multi-language (Python Flask + requests): link built, cross-repo trace stitches,
  and the file-level boundary fallback is exercised when the provider has no uid.

Ambiguous-destination and degraded-member paths need synthetic inputs the real
analyzer cannot produce, so they stay in the unit suite (documented in the README
+ script header). Removes verify-named.mjs + fixtures-named/ (folded inline).

* test(group): pin destination degraded-success + precise-tier ambiguity

Adds the two regression guards koriyoshi2041 requested on PR #2269 after the
degraded-on-success fix:
- destination trace success with a degraded member: reg-fe resolves from and
  follows the link to an anonymous handler while reg-be throws; the ok result
  carries the anonymous endpoint AND the 'could not be queried' degraded note, so
  the no-to path stays aligned with explicit to traces.
- multiple PRECISE destination hits: one from reaches two consumers with resolved
  uids linked to different routes; the result is ambiguous (role: to) with both
  route candidates. Distinct from the existing file-level ambiguous test, this
  pins the stronger precise tier against a future change silently picking the
  highest-confidence destination.

Both already pass against current behavior; 716 group tests pass.
 2026-06-23 07:54:13 +01:00
Gergő Magyar
3f0c74fea0
fix(deps): upgrade @ladybugdb/core to 0.16.0 to resolve native segfaults (#1235) 
* fix(deps): upgrade @ladybugdb/core to 0.16.0 to resolve native segfaults

Resolves the SIGSEGV / access-violation (0xC0000005) / exit-139 crashes that
have been reported widely since 1.6.3. The native crashes originate in
@ladybugdb/core 0.15.x — primarily during FTS index creation, VECTOR
extension load, and concurrent query teardown — and are reproducible on
Linux, macOS and Windows. The maintainer-confirmed fix is to bump the
runtime to 0.16.0, which ships nodejs async + memory-management fixes,
extension ABI bump, and macOS Intel binaries.

Adopting 0.16.0 cleanly required three supporting changes; without them
the upgrade itself regresses other paths:

1. maxDBSize must be passed explicitly. 0.16.0 keeps the upstream JSDoc
   note that the default 0 is "introduced temporarily for now to get
   around with the default 8 TB mmap address space limit some
   environment". Constrained CI runners and laptops cannot reserve 8 TB
   and crash with "Buffer manager exception: Mmap for size
   8796093022208 failed." A new gitnexus/src/core/lbug/lbug-config.ts
   centralises a 16 GiB default (overridable via
   GITNEXUS_LBUG_MAX_DB_SIZE) and every Database() construction site
   now passes it.

2. enableCompression default flipped from false to true in 0.16.0. Every
   Database() call site is updated to pass false explicitly so existing
   GitNexus indexes keep the same wire format.

3. Bridge DB sidecar files (.wal, .shadow). 0.16.0 enforces a database-id
   check on .wal / .shadow sidecars and rejects opens whose sidecars
   belong to a different base name. writeBridge now (a) cleans the full
   sidecar set when removing the tmp slot, (b) renames .wal / .shadow
   alongside the main file during the atomic .tmp -> .lbug swap, and
   (c) wraps openBridgeDbReadOnly in a bounded retry on transient
   Win32-Error-33 lock errors. Eager db.init() / conn.init() forces the
   lazy native handle to surface lock contention at the retry site.

Known limitation (not a regression): on Windows the 0.16.0 native binary
does not release the OS file lock until the process exits, so the
close-then-reopen-same-process pattern raises Error 33 after the first
close. Production paths (analyze / serve / mcp each open the DB exactly
once per process) are unaffected, but eight tests that exercise the
pattern are guarded with a process.platform === 'win32' skip; CI's
Linux + macOS shards exercise them as before. Tracking upstream:
kuzudb/kuzu#3872 / #3883 / #4730.

Closes #1136 #1154 #1160 #1162 #1178 #1195 #1196 #1199 #1204 #1206
Refs #1209 (supersedes — Dependabot bump without the supporting fixes)

Made-with: Cursor

* fix(test): isolate LadybugDB native test state

Use per-suite LadybugDB databases in integration helpers so test forks do not reopen a database created by Vitest global setup, and centralize Windows-tolerant native temp cleanup for bridge tests.

* fix(lbug): avoid bridge existence reopen

Reuse the built LadybugDB config in the extension installer and avoid native close/reopen cycles when checking bridge existence on Windows.

Made-with: Cursor

* chore(docs): exclude local lbug plan

Keep the refactor planning note out of the PR while leaving the ignored local copy on disk.

Made-with: Cursor

* refactor(lbug): centralize database construction

Route LadybugDB opens through shared helpers so native constructor defaults stay consistent across core, pool, bridge, and extension install paths.

Made-with: Cursor

---------

Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com>
 2026-04-30 17:40:39 +01:00
ivkond
1ff324ca16
feat(group): bridge.lbug storage + contract matching expansion (1/4 of #606 split) (#795) 
* feat(group): bridge.lbug storage + contract matching expansion

Part 1 of 4 in the split of #606 (ticket: #791, closes #790 with a
revised plan per @magyargergo's request).

## What changed

Adds the LadybugDB-backed bridge storage infrastructure and extends
the contract matching algorithm with wildcard support. All changes are
additive: storage.ts, sync.ts, service.ts, cli/group.ts, mcp/tools.ts
are left on their upstream main versions and will migrate to the new
bridge in follow-up PRs (#792, #793, #794).

### Files

**New (844 LOC prod):**
- `gitnexus/src/core/group/bridge-db.ts` — atomic write-to-temp with
  `retryRename` for Windows EBUSY/EPERM, per-item write tolerance via
  `WriteBridgeReport`, `findContractNode` with three-tier symbol
  lookup (uid → filePath+name → filePath)
- `gitnexus/src/core/group/bridge-schema.ts` — schema DDL
- `gitnexus/src/core/group/normalization.ts` — contract ID
  canonicalization + `dedupeContracts` / `dedupeCrossLinks` helpers
  used by both matching and bridge write

**Modified (+137 LOC prod):**
- `gitnexus/src/core/group/matching.ts` — adds `runWildcardMatch` for
  `grpc::Service/*` wildcard consumers, `buildProviderIndex` helper,
  and canonical gRPC ID handling in `normalizeContractId`
- `gitnexus/src/core/group/types.ts` — `MatchType` gains `'wildcard'`;
  new `BridgeHandle` and `BridgeMeta` interfaces

**New tests (658 LOC):**
- `gitnexus/test/unit/group/bridge-db.test.ts` — core write/read round
  trip, `WriteBridgeReport` shape, dropped-links counter, retryRename
  behavior on EBUSY/ENOENT/EPERM/EACCES
- `gitnexus/test/unit/group/bridge-db-edge.test.ts` — edge cases
  (malformed meta, missing contract nodes, concurrent access)

**Modified tests (+225 LOC):**
- `gitnexus/test/unit/group/matching.test.ts` — wildcard consumer
  matching, gRPC canonical ID handling, same-service guard

### Self-review fixes folded in

Carried forward from the original #606 self-review:
- `writeBridge` try/finally handle lifecycle + `handleClosed` sentinel
- `openBridgeDbReadOnly` partial-handle cleanup
- `writeBridgeMeta` uses `retryRename` for Windows consistency
- `retryRename` unit tests (was zero coverage)
- Per-item try/catch around every CREATE loop so one malformed contract
  doesn't abort the whole write
- Dropped cross-link counter (`linksDroppedMissingNode`)

### Why now

magyargergo asked for the #606 PR to be split so we can iterate with
confidence (https://github.com/abhigyanpatwari/GitNexus/pull/606#issuecomment-4229612271).
This is the foundational layer — pure infra, no user-facing surface,
no callers of the new APIs in this PR. Later PRs wire it in.

### How to verify

- `cd gitnexus && npx tsc --noEmit`
- `cd gitnexus && npx vitest run test/unit/group/bridge-db.test.ts --pool=forks`
- `cd gitnexus && npx vitest run test/unit/group/bridge-db-edge.test.ts --pool=forks`
- `cd gitnexus && npx vitest run test/unit/group/matching.test.ts --pool=forks`
- Pre-commit hook runs clean

### Risk / rollback

**Low.** All new code sits under `src/core/group/` in new files plus a
minimal `+16/-1` diff to `types.ts` and a `+136/-0` diff to `matching.ts`
(both purely additive). No existing callers reference the new APIs
(bridge-db, openBridgeOrFallback, runWildcardMatch) — the PRs that wire
them in come later in the split chain. Rollback = `git revert` of the
merge commit; no state introduced, no schema migration triggered.

### Scope discipline (per GUARDRAILS.md)

- Only the 8 files listed above are touched; no drive-by refactors
- No CI/release/security config changes
- No secrets, tokens, or machine-specific paths
- Content is lifted from the #606 branch which already passed CI 11/11
  green on `d15b8cb` (before the split)

### Dependencies

- **Base:** `main` (no dependencies on other split PRs)
- **Blocks:** extractor expansion (#792), sync pipeline (#793),
  cross-impact feature (#794)
- **Related ticket:** #791

Co-authored-by: Claude <noreply@anthropic.com>

* fix(group): address @claude review on #795

Addresses the findings from the automated review on PR #795
(https://github.com/abhigyanpatwari/GitNexus/pull/795#issuecomment-4229770000
— posted by @magyargergo / claude-code Action run).

### Medium severity (reviewer flagged as blockers)

- **bridge-db.ts `openBridgeDbReadOnly` bak recovery** — the `.bak`
  recovery path used bare `fsp.rename(bakPath, dbPath)`, which is
  exactly the scenario most likely to hit Windows EBUSY/EPERM (an
  interrupted writer still holding the handle for a few ms). Switched
  to `retryRename` for consistency with the rest of the file's
  Windows-safe rename path.
- **bridge-db.ts `ensureBridgeSchema` error detection** — the inline
  `msg.includes('already exists')` substring match has been lifted
  into a named constant `LBUG_ALREADY_EXISTS_MSG` with a comment
  documenting the coupling to LadybugDB's error message wording and
  why we can't use `IF NOT EXISTS` (LadybugDB DDL doesn't support it)
  or typed errors (LadybugDB's JS driver doesn't expose error codes).
  Also tightened the `catch (err: any)` to `catch (err: unknown)`.
- **bridge-db.ts `findContractNode` — extracted out of writeBridge**
  — the 35-line async closure living inside `writeBridge` has been
  lifted to three module-level functions: `createContractLookupIndex`,
  `indexContract`, and `findContractNode`. `findContractNode` is now
  a pure synchronous function taking a prebuilt index instead of
  doing its own DB queries. The `writeBridge` cross-link loop is now
  ~25 lines instead of ~100.
- **bridge-db.ts `findContractNode` — N+1 query elimination** — the
  old inner-closure version issued up to 6 DB round-trips per
  cross-link (2 endpoints × up to 3 tiers of fallback queries). For a
  group with 1000 cross-links, that's up to 6000 DB queries just to
  resolve endpoints. The new version consults an in-memory
  `ContractLookupIndex` built incrementally as contracts are inserted
  (`indexContract` called AFTER each successful insert so failed
  inserts don't poison the index). Cross-link resolution is now
  O(1) per link instead of O(3) DB queries per link, with zero DB
  round-trips during the cross-link loop.

### Minor severity

- **bridge-db.ts `queryBridge` empty-array guard** — if LadybugDB
  ever returns an empty `QueryResult[]` at the top level (shouldn't
  happen with single-statement calls, but driver contract isn't
  explicit), the old code would call `.getAll()` on `undefined` and
  crash with a confusing stack. Added an `unwrapQueryResult` helper
  that throws an explicit `'empty QueryResult array'` error instead,
  making a potential driver regression visible immediately.
- **normalization.ts `contractRichness` weights** — added a
  block-level comment documenting the weight ordering (+3 for
  symbolUid, +2 for each symbol-identifying field, +1 for service
  tag or non-manifest origin) and explicitly noting that the
  absolute numbers don't matter, only the relative ordering. Matches
  the "comment for contributors" suggestion in the review.
- **bridge-schema.ts `BRIDGE_SCHEMA_VERSION` migration comment** —
  added a 4-point contract explaining what bumping the constant
  means ("discard and re-sync" strategy for V1, no in-place
  migration yet, new migration logic should live in a separate
  `bridge-migrations.ts` module when it becomes necessary).
- **test/unit/group/fixtures.ts** — extracted the `makeContract`
  helper previously copy-pasted between `bridge-db.test.ts` and
  `bridge-db-edge.test.ts` into a shared fixtures module. Both test
  files now import from `./fixtures.js`. Kept the scope minimal:
  fixtures is NOT a general-purpose factory module, just the shared
  baseline contract builder.

### New tests

Added 9 pure-function unit tests for the now-extracted
`findContractNode` in `bridge-db.test.ts`:
  - returns null on empty index
  - tier 1 (symbolUid) match, including repo-scope and role-scope
    isolation
  - tier 2 (filePath + symbolName) fallback when symbolUid is empty
    or mismatches
  - tier 3 (filePath only) when exactly one contract lives in the
    file, and refusal when multiple do
  - priority ordering when multiple tiers could resolve

These are fully isolated — no DB, no temp directories, no native
LadybugDB binding — so they run in &lt;10ms total and are
immediately trustworthy as a regression safety net.

### Deliberately deferred (reviewer marked as "fine for now")

- `BridgeHandle._db` / `._conn` typing to `unknown` with casts in
  `bridge-db.ts` — reviewer's note: "The typing is fine for now."
- Batch inserts via `UNWIND` — needs LadybugDB support confirmation,
  tracked as a follow-up; the per-item pattern remains.
- `queryBridge` prepared-statement lifecycle — the current pattern
  (prepare → execute → GC) relies on LadybugDB's internals, worth
  verifying against their docs in a separate audit.

### Scope discipline (per `GUARDRAILS.md`)

- Only files touched by this PR (`bridge-db.ts`, `bridge-schema.ts`,
  `normalization.ts`, both bridge test files, new `fixtures.ts`) —
  no drive-by refactors
- No CI/release/security config changes
- No secrets

### Test + typecheck status

- `npx tsc --noEmit` clean
- `bridge-db.test.ts`: added 9 `findContractNode` tests, all pass in
  isolation. The full-file run still hits the pre-existing native
  LadybugDB cleanup segfault that flakes the reported count — same
  as every prior commit on this branch, not a regression.
- `bridge-db-edge.test.ts`: 4/4 pass
- `matching.test.ts`: 28/28 pass
- `types.test.ts`: 5/5 pass
- `retryRename` tests (4/4) and `findContractNode` tests (9/9)
  verified in isolation via `-t` filter

Co-authored-by: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
 2026-04-11 19:46:12 +01:00