mirror of
https://github.com/abhigyanpatwari/GitNexus.git
synced 2026-08-28 05:25:25 +00:00
66 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
DuduPhudu
|
2c0fb7753c
|
fix(group): stop reporting what could not be measured as a measurement of zero (#3012)
* fix: surface unreadable group indexes and escape raw NUL bytes in source Two independent diagnostics failures, both of which turn a real error into a confident, benign-looking answer. **Unreadable member repos (#3011).** `syncGroup` wrapped `initLbug` plus all contract extraction for each member in a bare `catch {}` that pushed the repo onto `missingRepos` and discarded the error. A LadybugDB storage-version mismatch therefore surfaced as "repo not found", `group sync` printed `0 contracts, 0 cross-links` and exited 0, and the existing contracts.json was overwritten with an empty registry. The two states need different answers from the operator — a missing repo must be indexed, an unreadable one is usually version skew or a lock — so they are now separate: - the caught error is logged with the repo, group path and lbug path - `unreadableRepos` is tracked alongside `missingRepos` on `SyncResult`, persisted (optionally, so older registries still parse) on `ContractRegistry`, and threaded through `GroupService` sync/status - `group sync` reports both before the cascade counts, since an unread repo is the likely explanation for a small or empty count - `group status` reports unreadable repos separately; calling them "missing" actively misdescribed them - when EVERY configured repo fails to open, the write is skipped: an extraction that read nothing is not evidence the group has no contracts, and replacing a good registry with an empty one loses data while reporting success **Raw NUL bytes (#3010).** `sync.ts` and `free-call-fallback.ts` each used a NUL as a join delimiter, written as a literal 0x00 instead of `\0`. Identical at runtime, but it makes the file test as binary: `file(1)` reports `data`, ugrep returns empty with exit 1 — indistinguishable from "no match", with no message — and BSD grep replaces matching lines with "Binary file ... matches". A search that should hit comes back as a confident "not present". Both now use the escape, and a unit test fails on any raw control byte in src/ so it cannot silently return. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(hygiene): guard every tracked source file against a raw NUL, not just src/ The guard added with the NUL escapes only scanned gitnexus/src for .ts/.tsx. Neither prior recurrence of this defect in this repo was in that scope: |
||
|
Gergő Magyar
|
740f0a4e57
|
fix(skills): publish gitnexus-plan artifacts on macOS without an interpreter (#2905) (#2922)
* fix(skills): anchor gitnexus-plan safe writer on macOS (#2905) The safe generated-plan writer refused to run on anything but Linux. `requireDescriptorAnchoring()` hard-gated `process.platform !== 'linux'` because every name it resolves went through `/proc/self/fd/<fd>/<child>`, and publication went through `renameat2(RENAME_NOREPLACE)`. macOS has neither, so `write-plan` and `read-plan` failed on every input and `snapshot` failed whenever a materialized path was absent. Node cannot perform openat-style directory-relative resolution on macOS at all: `node:fs` exposes no dir_fd parameter, and `fcntl(F_GETPATH)` is a snapshot string that XNU reconstructs from the name cache, so using it would reintroduce the exact race this helper exists to prevent. Python does expose the *at() family via dir_fd, and macOS has renameatx_np with RENAME_EXCL, so the anchoring borrows the interpreter the writer already spawns for renameat2. Anchoring now goes through a backend with two implementations. The Linux one keeps the original expressions, flags, ordering and error strings. The Darwin one runs each operation in the integrity-checked python3: it re-walks the chain from the repository root with O_DIRECTORY|O_NOFOLLOW, asserting the caller's recorded device, inode and mode at every level before acting. A chain that fails that assertion reports a dedicated anchoring errno and never ENOENT, so a moved parent cannot be read as an absent file. Node holds an open descriptor on every chain element for the anchor's lifetime, which pins the inodes so their numbers cannot be recycled between spawns, and that coupling is re-checked on the way into every request rather than left implicit. A filesystem that answers ENOTSUP to RENAME_EXCL is a refusal, never a fallback to a replacing rename. Every other platform is still refused. The suite had silently skipped on every non-Linux runner, so it is now gated on linux-or-darwin and registered in the cross-platform test list, which puts it on the macos-latest CI matrix. Disclosed rather than papered over: operations that must hand Node a file descriptor are anchored in the helper and then opened lexically with O_NOFOLLOW and identity-compared. A racer can force a mismatch, which aborts, or land on the inode the anchored walk already found, which is harmless. A perfect ABA inside that window is impossible on Linux and detected in all but its narrowest form on macOS. The reference doc says so. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NtGZN6YSNU8chALSnKYHBp * test(skills): normalize the anchoring-gate fixture repo on Windows The two capability-gate tests are the only ones in this file that run on Windows, and both failed there: `createBaseRepo` returned the path `os.tmpdir()` gave it, which on Windows is the 8.3 short form (C:\Users\RUNNER~1\...). `assertRepository` compares fs.realpathSync of the caller's path against the realpath of `git rev-parse --show-toplevel`, and plain realpathSync does not expand short names while git always reports the long form, so the helper rejected its own fixture with "--repo must be the Git worktree root" before either platform gate was reached. Resolve the fixture with the native resolver, which returns the canonical long path. No-op on platforms where the two already agree. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NtGZN6YSNU8chALSnKYHBp * test(skills): skip the darwin backend gate on Windows Spoofing process.platform does not spoof fs.constants. Windows Node defines no O_DIRECTORY, so a darwin-spoofed run there refuses at the anchoring-flag check and returns that message instead of ever reaching the python3-backend branch the test exists to cover. Skip it on win32 rather than loosening the regex, which would also let a macOS run pass on the wrong message. The sibling test still asserts the Windows refusal on Windows. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NtGZN6YSNU8chALSnKYHBp * refactor(skills): tighten the macOS anchoring backend Quality pass over the Darwin backend. No behaviour change was intended on the success paths; the guarantees are the same or stronger. Structural: - openChildRead now proves identity inside the backend instead of by comment. It was returning a raw descriptor from a lexical open, with the "callers always compare against the preceding anchored stat" invariant enforced across four call sites in prose — and since the Linux predicate is a literal `return true`, a fifth caller that forgot would have been an unanchored open on macOS that Linux CI could not see. It routes through darwinAdoptAnchoredFile, which already did open-then-compare-then-close-on-mismatch for createChild. - recordAnchoredAbsence shares one prefix walk per snapshot instead of re-walking from the repository root for every absent cited path. With three absent paths under a three-deep prefix that is 12 helper spawns down to 6 and 12 retained descriptors down to 4. citedPaths is caller-supplied and unbounded, so the descriptor retention was the real problem; the cache is now the sole close owner. This does change Linux descriptor lifetime — prefixes stay open for the snapshot rather than only the tail, deduplicated across paths. - assertRepository and the sibling realpath comparisons use realpathSync.native. Windows hands back 8.3 short names that plain realpathSync preserves while git reports the long form, so `snapshot`, which is not platform-gated, could reject a worktree root by quoting that same directory back at the user. The fixture workaround that papered over this for the new gate tests is gone. Efficiency, all measured at ~13.5ms per helper spawn: - consume the identity mkdir already computed rather than re-stat it - act on renameNoReplace's return value rather than spending two stats re-deriving what it already reported - drop a duplicate anchored stat taken twice in a row in movePathToVault - import ctypes only where it is used; 19 of 20 spawns never touch it Simplification: pins folded into the descriptors the handle already carried, an unreachable refreshAnchorTail branch and the dead darwinHardenedOpen mode parameter removed, the four copies of the spawn options collapsed, the spawn-and-parse shared between the probe and the request path, the unreachable launch-path fallback and a redundant memo deleted, and the helper's dispatch made a real elif chain with leaf name and mode validated at one chokepoint rather than per operation. The two chain encodings were left alone deliberately: merging them would have grown triple fields on Linux for no Linux benefit and changed the Linux validatePlanParent comparison. The double re-stamp that motivated the merge is contained in one named helper with the hazard documented. Rejected candidate interpreters now say which dir_fd operations were missing instead of producing a generic refusal. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NtGZN6YSNU8chALSnKYHBp * refactor(skills): publish plans with link(2) and drop the interpreter The macOS backend spawned python3 for two jobs: openat-style resolution, which Node cannot do, and a no-replace rename. Only the first is actually unavoidable, and the second was carrying the whole dependency. link(2) is a no-replace publish. It is atomic, it fails EEXIST when the destination name is taken, and it refuses a symlinked destination without following it — the same guarantee renameat2(RENAME_NOREPLACE) and renameatx_np(RENAME_EXCL) give, reachable from plain fs.linkSync. The published file is the same inode as the verified temporary, so the downstream identity checks hold by construction rather than by argument. That removes the interpreter from Linux entirely, since /proc already did the resolving there, and it removes ctypes, libSystem, RENAME_EXCL and the ENOTSUP handling from macOS. Deleted with them: the trusted-executable validation, the held-descriptor exec and its two-tier probe, the capability probe, the JSON request protocol, and both embedded Python programs. The helper drops from 3047 to 2327 lines. macOS keeps the part that genuinely cannot be done in Node, and now does it without a subprocess: a lexical O_NOFOLLOW walk that holds an open descriptor on every directory in the chain and re-proves the chain either side of every step. Pinning is load-bearing — an open descriptor keeps its inode number from being recycled, which is what makes the recorded identities trustworthy across steps. The guarantees are no longer symmetric and the docs say so plainly. /dev/fd/<fd> is a devfs node, not a magic link: opening it works, resolving through it does not, open("/dev/fd/<fd>/child") returns ENOENT and realpath returns /dev/fd/<fd> — measured on macOS 26 rather than inferred. So Linux makes a parent swap impossible while macOS detects one and aborts. Also fixes the writer on 9p mounts, where renameat2(RENAME_NOREPLACE) returns EINVAL and publication failed every time; link(2) succeeds there. Tests 174 -> 154: dropped 29 fixtures that drove the deleted Python program directly, added coverage for the link publish, for a macOS parent swap caught through the pinned chain, and for a spoofed-darwin round trip that asserts no /proc path reaches the hooks, which the portable backend now makes runnable on Linux CI. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NtGZN6YSNU8chALSnKYHBp * fix(skills): drop O_NOFOLLOW_ANY, guard trailing slashes, handle link edge cases macOS CI rejected our hardened directory open with EINVAL on 30 tests. The flag O_NOFOLLOW_ANY was ORed into every open on the theory that XNU ignores unrecognized open bits, so it would be inert where unsupported. That theory is wrong, at least combined with O_DIRECTORY. The Python design never hit it because the walk ran inside the interpreter; once Node did the opening, every Darwin directory open went through it. Removed rather than probed. The per-component O_NOFOLLOW walk is what delivers the guarantee, and cap-std — the closest reference implementation of this problem — has not adopted O_NOFOLLOW_ANY either. A fixture now pins the exact flags of every directory open under a spoofed darwin, so the next failure names the flag instead of printing a stack trace. With the flag gone the two backends' directory open became identical, so it is no longer a platform concern at all. Three findings from researching the prior art, all now covered: Trailing slashes. CVE-2026-39822 escaped Go's os.Root because open(fd, path, O_NOFOLLOW) follows symlinks when the path ends in "/". It reproduces here: with docs a symlink, opening "docs" is ENOTDIR but "docs/" succeeds into the attacker's directory, and path.join preserves the slash. We were safe only by construction, and only for repo-derived names — the generated temporary and vault artifact names never passed through the validator. The guard now sits at anchoredChild, the single place a name becomes a path, so it holds for every caller. link() can lie on NFS. Per link(2) BUGS, the return code may be wrong if the server creates the link then dies before replying; open(2) NOTES gives the remedy, which is to stat the source and treat a link count of 2 as success. Implemented, with the man-page reasoning in the comment so it is not later removed as paranoia. Filesystems without hard links now fail loudly. EPERM, ENOTSUP and EMLINK say so and refuse to fall back to a replacing rename. Git falls back and accepts losing collision detection because its objects are content addressed; that reasoning does not transfer to a named plan destination. Durability was already correct — the temporary is fsynced before publication and the parent directory immediately after — but the comment now records why the parent fsync is required for link as it was for rename, and the honest limitation that fsync is not a write barrier on macOS while F_FULLFSYNC, which Node cannot reach, is. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NtGZN6YSNU8chALSnKYHBp * refactor(skills): shrink the anchoring seam and fix two CI breaks Four quality reviews over the pure-Node writer. Two real breaks, one drift that had already happened, and a seam that was sized for a design we deleted. The macOS round-trip fixture asserted that every observed path started with join(repo, 'docs/plans'). Reproduced on Linux by handing the helper a repo reached through a symlink, which is the shape macOS gives us via /var to /private/var: assertRepository realpaths the repo, so the handle builds paths from the resolved form while the fixture holds the form it passed in, and the prefix can never match. The assertion now proves the same thing without depending on the prefix — a lexical resolution always contains a docs/plans segment and /proc/self/fd/<fd>/<name> never does. Two publish fixtures sat in the capability-gate describe, the one block deliberately not skipped on unsupported platforms, while this PR added the file to the Windows matrix. They test link(2), not the gate, so they moved to SAFE_WRITE_FIXTURES. validatePlanParent restated verifyLexicalChain's loop without the try/catch that converts ENOENT and ENOTDIR into the parity message, so a raw errno could escape a function with a dozen call sites. It was masked on Darwin only because parentStillResolves catches first. It now calls the helpers, which also removes a second full chain walk per call there. openVerifiedFile adds O_NONBLOCK so a FIFO swapped in at the target name cannot wedge the process on open, and only Darwin was calling it. The operations are now shared, so Linux gets it by construction rather than by a per-backend decision. The backend is five methods rather than ten. The platform difference is two things — how a name becomes a path, and what guard wraps an operation — so the five operations became shared functions over a `verified` hook that is run() on Linux and the pinned-plus-lexical sandwich on Darwin. openChildRead always runs the identity adoption, so that proof is structural rather than a comment about what callers must remember. Selecting the backend is a registry that throws on an unknown platform instead of a ternary defaulting to Linux, which surfaced seven dead bindings that ran before the capability gate and made win32 report the registry error instead of the refusal. Snapshot capture no longer re-walks a prefix per record: 36,018 lstats to 6,384 and 162ms to 130ms on 2,000 dirty files across 100 directories, with a byte-identical global_dirty_digest. Absence anchoring is now bounded at 4096 pinned directories and refuses rather than evicting, because closing a cached descriptor would break the pinned chain of a guard already recorded — the inode-recycling hole the pins exist to close. The test suite no longer cache-busts its imports. That existed for the memoized python3 descriptor, the file's only mutable module binding, which is gone; the suite drops from 10.0s to 8.2s. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NtGZN6YSNU8chALSnKYHBp --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Gergő Magyar
|
e69d3c49c4
|
fix(analyze): gate FTS-indexed DML before the incremental writeback (#2841) (#2854)
* fix(lbug): never report a drop that could not happen, and gate FTS-indexed DML `CALL DROP_FTS_INDEX` is itself an FTS-extension function, so with the extension unloaded it fails with `Catalog exception: function DROP_FTS_INDEX is not defined`. `isBenignDropFtsIndexError` classifies that as "nothing to drop" — correct when the index does not exist, wrong when it does: the drop silently no-ops and the next write to that table dies at bind time with an engine message that never mentions FTS (#2841). The classifier stays pure (a message cannot tell you whether an index is live). Instead `dropFTSIndex` settles liveness with a catalog read on the ERROR path only and raises an FTS-named, remedy-bearing error when the index is present but undroppable. Adds `ensureFtsRowDmlSafe`, the FTS twin of `ensureEmbeddingRowDmlSafe` (#2623): catalog first, load FTS with the analyze policy only when an index actually gates DML. LadybugDB refuses that DML at BIND time — a DETACH DELETE matching zero rows fails exactly as hard as one matching thousands — and the indexes cannot be cleared in place, so a verdict is the only useful answer. Both gates now share one `SHOW_INDEXES` read via `readIndexCatalogRows`, so adding the FTS check costs no extra catalog round-trip. Refs #2841 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CxokkpfssUCxvBwRZMtRCB * fix(analyze): escalate instead of crashing when FTS blocks incremental DML The incremental writeback decided its write plan without ever asking whether row-level DML was legal. On a DB carrying FTS indexes with an unloadable FTS extension, `deleteNodesForFiles` then died mid-writeback: Binder exception: Trying to delete from an index on table File but its extension is not loaded. with no mention of FTS anywhere in the run — the only install-capable load happened in Phase 3, long after the writes (#2841). The incremental branch now reads the index catalog once and derives both extension verdicts before any DML. When FTS (or VECTOR) blocks in-place writes, the run falls through to the existing wipe-and-bulk-COPY escalation — the same answer #2623 gave for VECTOR, and the only one available, since the indexes cannot be dropped without the extension. Every blocked extension is named in the reason log, not just the first one checked: a DB can carry both a vector index and FTS indexes, and reporting half the cause is how this failure stayed mis-diagnosed. Refs #2841 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CxokkpfssUCxvBwRZMtRCB * test(analyze): cover the FTS DML gate, both-blocked escalation, and the drop guard New `incremental-index-extension-dml-gate.test.ts` drives the real `runFullAnalysis` against a real mini-repo and a real LadybugDB: - a DB carrying FTS indexes with FTS made unloadable escalates to a full DB write, names FTS in the log, ends with zero FTS indexes, and still has the newly committed content in the graph (pre-fix: Binder exception, exit 1); - FTS available keeps the surgical plan and the indexes; - a DB that never carried FTS indexes is not escalated (the catalog-first check must not tax FTS-less machines); - FTS and VECTOR both blocked produce ONE escalation naming both. `drop-fts-index-error-classification.test.ts` gains the two `dropFTSIndex` cases the #2841 guard turns on: live index + unloaded extension rejects with an FTS-named error, absent index still resolves. The existing classifier assertions are unchanged — it stays pure. The CLI e2e reproduces the reporter's exact journey (analyze with the extension, remove it, touch a file, analyze again) and asserts exit 0 plus an FTS-named reason. It skips visibly when the seeded extension cannot load on the host, so it can never report a false red about the fix. Mutation-verified: reverting the run-analyze gate fails the first scenario; reverting the dropFTSIndex guard fails the live-index case. Refs #2841 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CxokkpfssUCxvBwRZMtRCB * fix(lbug): make every catalog-gated path fail closed, and classify the drop remedy Review findings on #2854 (two-engine, 17 lanes). H3 — `ftsIndexExistsInCatalog` returned `false` when the catalog could not be read, i.e. "index absent", so `dropFTSIndex` swallowed the error and the caller proceeded as if the index were gone. That is the #2841 symptom the guard exists to make loud, and it contradicted the contract `readIndexCatalogRows` states two functions above. It now fails closed. §6.A — `ensureFtsRowDmlSafe` keyed on `index_type === 'FTS'`, which answers `undefined === 'FTS'` → false → *no gate* for a row whose shape cannot be read: fail-open, in the gate whose only job is preventing an unsafe write, while the VECTOR twin fails closed on the same input. Now only a positively-identified non-FTS index is waved through. Deliberately NOT the twin's `!== 'HASH'`: that is safe there only because it is scoped to the embedding table first, and this gate is table-agnostic — `!== 'HASH'` would let the HNSW index gate FTS DML. §5.A — `undefined` was overloaded: "caller passed nothing" and "caller tried and could not prove anything" shared one value, so a failed shared read silently became three reads and the two gates could decide from different snapshots. The failed snapshot is now representable (`INDEX_CATALOG_UNREADABLE`), leaving one unambiguous `??` in `resolveGateRows`. §5.B — both gates regained the unconditional null-connection precondition the refactor moved into the reader. §5.G — the throw's remedy now routes through `diagnoseExtensionLoad`, like `--repair-fts` and `ftsDegradedWarning`, so a missing runtime dependency is not told to reinstall. The message stays path-free (#2374/#2375). The dead positional row fallbacks are kept and marked `LADYBUGDB-CONTRACT`: removing them would turn a proven-inert hedge into a fail-open gate if a future engine returns unnamed tuples. Refs #2841 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CxokkpfssUCxvBwRZMtRCB * fix(analyze): never undo an explicit wipe, stage extension-forced rebuilds, report honestly Review findings on #2854 (two-engine, 17 lanes). H1 (P1, both engines) — `analyze --drop-embeddings` was silently reverted. The `--drop-embeddings` → `force` conversion sits inside the `embeddingCheckpoint` branch, so without a checkpoint the run stays incremental and reaches the gate; the flag then *deliberately* leaves `cachedEmbeddings` empty, which is exactly the rescue's trigger, so every row the operator asked to destroy was read back and restored, exit 0. Widening the rescue from `!embeddingRowDmlSafe` to `extensionForcedRebuild` moved that latent bug onto the dominant path, because every analyzed DB carries FTS indexes. Guarded on the flag itself — NOT on `shouldLoadCache`, which is false in the meta-under-reports case the rescue exists for and would have deleted the safeguard while fixing the wipe. The `--drop-embeddings --embeddings` variant is covered by the same guard. H2 — an extension-forced escalation wiped the LIVE index in place: `buildPath` was frozen ~440 lines earlier while the run was still classified incremental, so an interrupt or ENOSPC left no complete index, where main failed at bind time with it intact. Extension-forced rebuilds now build into a staging file and publish via the existing atomic swap; size-forced ones stay in place, since that trigger is the repo's own churn rather than a machine condition. H5 — the escalation log asserted a vector index "exists" and that the store "carries FTS indexes" in exactly the case the catalog read proved nothing, while the only truthful signal went to stderr rather than the IPC log. It now emits a distinct unreadable-catalog cause, and "this index carries" (which pointed at the vector index just named) reads "the graph store carries". §5.D — the write-set cause was dropped whenever an extension cause co-occurred; causes are appended now, not selected between. §5.C — after an FTS-forced rebuild stamped lastCommit, a plain rerun on the same commit hit the alreadyUpToDate fast path before Phase 3, so the CLI's "install … then rerun" advice could never restore FTS. The fast path is now bypassed when meta records FTS unavailable and the extension can load again, keyed on the persisted capabilities stamp rather than new state. §5.F (skip the escalation for a zero-change commit) is deliberately NOT implemented: `deleteSpringAutoConfigurationSyntheticClasses` and `deleteSpringAopEvidenceNodes` run unconditionally on the surgical branch and bind against FTS-indexed `Class`/`CodeElement`, and a zero-row DETACH DELETE fails at bind time exactly as hard as a large one — so the skip would restore the original crash. Refs #2841 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CxokkpfssUCxvBwRZMtRCB * perf(search): read the index catalog once per drop sweep, and state the real contract Review findings on #2854. H4 — on a machine where FTS cannot load and the DB carries no FTS index, the gate correctly returned early without loading the extension, but the surgical path still ran the full 20-entry drop sweep: every `CALL DROP_FTS_INDEX` raised "function DROP_FTS_INDEX is not defined", and the new liveness guard then fired a fresh catalog read per table — 20 reads every run, forever, for exactly the offline/load-only population, contradicting the "healthy path costs nothing" claim shipped with the guard. The sweep now reads the catalog once and skips entirely when no FTS-typed index exists. An unreadable catalog runs the sweep, so an unprovable catalog never skips real work. H8 — the docstring still promised `dropFTSIndex` "tolerates" an unloadable extension. Post-#2854 a live index plus an unloadable extension throws, and safety rests on caller ordering discipline rather than the type system — which is what would have talked the next caller out of that ordering. GUARDRAILS — the "switching to a full DB write" sign described exactly one trigger (write set >~50%). Since #2623 and #2841 an unloadable extension escalates regardless of write-set size; documented with its recovery steps. Refs #2841 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CxokkpfssUCxvBwRZMtRCB * test(analyze): cover the wipe guard, the staged rebuild, and the fail-closed branches Review findings on #2854. H1/H2 mutation-verified: removing `!options.dropEmbeddings` fails the new drop-embeddings case ("expected true to be false"); disabling the staging upgrade fails the staging case ("expected 0 to be greater than 0"), so both assert behaviour rather than describe it. Gate suite (7 cases): `--drop-embeddings` under an FTS-forced escalation ends at zero embedding rows and logs no "Preserving"; the escalation is one-shot — a third run on a healthy host returns to surgery and rebuilds every FTS index; an extension-forced rebuild is observed building into `lbug.staging.*` and leaves none behind; the rescue complement still preserves un-stamped rows when no wipe was requested; the never-built case now asserts the commit reached the graph. H6 — the both-blocked case hard-asserted `createVectorIndex()` while the suite probed FTS only, so it went red on any FTS-yes/VECTOR-no host. VECTOR is probed now and gates only that case, with a GITNEXUS_REQUIRE_VECTOR hard-fail. H7 — the fail-closed branches had no coverage although the VECTOR twin's test and interception technique were ready to copy: `ensureFtsRowDmlSafe` under an unreadable catalog now proves it routes to the load, and `dropFTSIndex` proves it rejects rather than silently tolerating. Plus a redaction case that forces a real path-bearing load failure — under policy `never` the assertion would have been vacuous, since that reason carries no path. §5.E/§6.B — the suite is registered in the cross-platform matrix (its sibling was; it wasn't, and GITNEXUS_REQUIRE_VECTOR is set only on that job) and moved into the sequential lbug-db project per TESTING.md:68, verified not to drop it from the sharded ubuntu job. A Windows shard weight is added as a labelled estimate — the 8s floor would skew the split it exists to protect. Refs #2841 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CxokkpfssUCxvBwRZMtRCB * refactor(analyze): make the FTS gate's fast path cheap, its claims provable, and its remedies classified Cleanup review of the #2841 work (four parallel angles: reuse, simplification, efficiency, altitude). Behaviour-preserving except where the previous behaviour was wrong. Correctness the review caught: - The fast-path probe keyed on `capabilities.fts.status === 'unavailable'`, which collapses "extension unavailable" and "index build failed". A deterministic build failure (an un-tokenizable row, #2544) therefore bypassed `alreadyUpToDate` on EVERY subsequent run, re-analyzed the whole repo, failed the same way, and restamped — a permanent loop where the run used to be one `stat`. Phase 3 already computes the discriminator; it is now persisted as `fts.skipReason` and the probe only runs for `extension-unavailable`. Metas written before this carry no field and keep today's behaviour. - `dropSearchFTSIndexes` skipped its sweep when no row read `index_type === 'FTS'`, while `ensureFtsRowDmlSafe` treats an unreadable type as "might be FTS". Opposite polarity, under a comment claiming they matched: a row-shape change would let the gate wave the surgical plan through while the sweep dropped nothing, putting DELETEs back on tables carrying live FTS indexes — #2589 again. The sweep now decides per configured index on identity, which is also strictly more precise. Its old justification (leftover indexes under other names) was unreachable — the loop only ever drops configured entries. - `dropFTSIndex` threw "FTS index X on table Y exists" on the one path where the catalog could not be read — a fabricated claim, on a DB the same run had just shown carries no FTS index. Presence is now `present | absent | unverifiable` and the message says which. - The remedy was hand-written for three of the four load-failure classes, discarding `missingFileRemedy`/`corruptFileRemedy`, so a corrupt extension file was told to retry an install — the misdirection #2383 fixed. Both the drop error and the escalation log now use the classified remedy. Cost, measured on a 391 MB index (cold open ~1 s, SHOW_INDEXES ~4 ms): - The probe opened the live index WRITABLE on the millisecond fast path, dragging in schema DDL, the cross-process write lock, sidecar reclaim and a CHECKPOINT on close. It is read-only now. That also closes an install trap: `doInitLbug`'s pre-load resolves the env policy on the writable branch, so an operator following our own `GITNEXUS_LBUG_EXTENSION_INSTALL=auto` advice paid a forked 15 s installer on every up-to-date run (memoized per process; the CLI is a fresh process each time). The read-only branch pins `load-only`. - A failed staged rebuild orphaned a full index-sized copy until the next lock sweep; the failure path now reclaims it. - The sweep re-read a catalog the run already held, defeating the invariant the snapshot type exists to enforce. Structure: row-shape accessors have one home, so the LADYBUGDB-CONTRACT grep claim is true by construction; staging now applies to both escalation causes, since recoverability is a property of the wipe-then-COPY plan, not of the trigger; `getExtensionCapability`/`getFtsCapability` replace hand-spelled lookups where the seam allows. Two lookups in run-analyze.ts deliberately keep the exported `getExtensionCapabilities()` form: the #2383 tests stub that export, and an ESM module mock does not intercept a helper's internal call — routing through it silently degraded the classified remedy to generic text. Recorded in-comment. Not taken, deliberately: extracting the escalation message and replacing the snapshot protocol with a connection-scoped catalog memo (both sound, both restructure code this PR just stabilised — they belong in their own change); an extension registry (premature at two instances, and the FTS/VECTOR polarity difference is exactly what it would have to parameterize back out). Refs #2841 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CxokkpfssUCxvBwRZMtRCB * test(analyze): pin both sides of the degraded-FTS fast-path bypass `healDegradedFts` (§5.C) had zero coverage — three separate review angles flagged it, and the cleanup pass then found it sat one conjunct away from a permanent full-re-analyze loop. Both sides are pinned now: - it re-analyzes past `alreadyUpToDate` when the stored meta says FTS is degraded and the extension loads again: run 1 analyzes with loads blocked (asserting the precondition — `status: 'unavailable'`, `skipReason: 'extension-unavailable'` — rather than assuming it), then a same-commit clean-tree rerun rebuilds every FTS index without a file changing; - it stands down when the degradation was a BUILD failure: the stored `skipReason` is rewritten to 'build-failed' and the rerun must take the fast path, because that rebuild would fail identically on every run forever. The build-failed state is reached by rewriting the stamped discriminator, not by provoking a real tokenizer failure: a genuine one needs a stored row the native tokenizer rejects (#2544/#2546), which is neither portable across the CI matrix nor deterministic, and §5.C reads only that field. Also folds the first escalation case into the one-shot case. The claim that it was fully subsumed did not hold on audit: `logs` containing 'FTS' was unique as expected, but so was the duplicate-File-node row count — every other reader goes through a Map keyed by path, which collapses a stale twin an appending rebuild would leave. Both assertions moved rather than one being dropped. Net suite runtime goes UP (two cycles removed, four added), against the cross-platform-matrix argument that motivated the dedup — recorded here because the shard weight is an estimate pending a real Windows measurement. Refs #2841 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CxokkpfssUCxvBwRZMtRCB * test(search): keep the whole-module adapter mock in step with the row accessors The cleanup pass moved the LadybugDB row-shape reads behind named accessors so the column contract has one home. `fts-indexes.test.ts` mocks the entire adapter module with a hand-written factory, which still exposed only the three exports the file imported before — so `verifySearchFTSIndexes` failed with "No `indexRowName` export is defined on the mock" while production was fine. The added accessors mirror the real implementations rather than returning stubs. A stub would have read `undefined` out of every catalog row and let the suite pass for the wrong reason — the failure mode a whole-module mock invites whenever the module under test grows an import. Refs #2841 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CxokkpfssUCxvBwRZMtRCB * revert(analyze): drop the degraded-FTS auto-heal, fix the advice it existed to justify §5.C's complaint was that the CLI tells users to "install the extension … then rerun" when a rerun lands on the up-to-date fast path and rebuilds nothing. The answer shipped for it was a probe that bypasses that fast path. Four independent problems later, the sentence is cheaper to fix than to make true: - it could not tell "extension was missing" from "index build failed" without a stamped discriminator, so a deterministic build failure (#2544/#2546) re-analyzed the entire repo on every invocation, forever, where the run used to be one `stat`; - it opened the live index on the millisecond fast path — writable at first, dragging in DDL, the cross-process lock and a CHECKPOINT (~1 s on a 391 MB index), and even read-only it is a full open; - `doInitLbug`'s pre-load resolves the env policy, so an operator following our own `GITNEXUS_LBUG_EXTENSION_INSTALL=auto` advice paid a forked 15 s installer per up-to-date run; - and it turns the fast path into a full re-analysis whenever an index authored where FTS was unavailable is later read where it loads — a legitimate, common state, and the invariant `analyzer-identity-cli.test.ts` pins. So: no probe. The degraded-search warning now points at `gitnexus analyze --repair-fts`, which rebuilds the search indexes without re-parsing the repo, instead of "then rerun". One line, no new failure modes, and it is what the issue actually asked for. `capabilities.fts.skipReason` stays in the meta stamp: it costs three lines, makes the two degradation causes distinguishable for support, and is what any future correct answer here would key on. Also gates the H2 staging assertion on the production predicate. It asserted staging unconditionally while the upgrade requires `posixSwap || windowsSwapOk`, and `windowsSwapOk` is opt-in (#2614) — so it failed on the Windows matrix for a reason unrelated to #2841. Registering this suite cross-platform is what exposed it; the assertion now mirrors the condition it is testing. Refs #2841 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CxokkpfssUCxvBwRZMtRCB * fix(analyze): never stage around a damaged index — escalate in place when the catalog is unreadable CI caught this on ubuntu and macOS: `analyze-wal-checkpoint-failure` stopped failing, which is worse than it sounds. That test plants a directory at `.gitnexus/lbug.wal.checkpoint` so the auto-checkpoint's rename target is blocked, and asserts analyze exits non-zero with the `--wal-checkpoint-threshold` hint. But LadybugDB cannot open that path at all, so `CALL SHOW_INDEXES()` now fails with `IO exception: … Is a directory`. The catalog read returns UNREADABLE, both DML gates correctly fail closed, both extension loads fail with the same IO error, and the run escalates — and since the escalation stages, it built a fresh index at `lbug.staging.<uuid>`, swapped it in, and exited 0. The blocked path was never touched. The run "succeeded" while the damage sat untouched on disk, waiting to break the next in-place writeback. So the staging upgrade is now conditional on the catalog having been READ. Staging exists to protect a healthy live index from a machine-level cause (an extension that will not load); it must not be used to route around a damaged one. When we are escalating out of ignorance, build in place so the underlying IO fault lands on the failure path where the operator gets a diagnosis. Verified against the real CLI, not just the suite: with a directory planted at the checkpoint path, analyze now exits 1 and prints `gitnexus analyze --wal-checkpoint-threshold 67108864`. The healthy extension-forced case still stages (gate suite 6/6). Refs #2841 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CxokkpfssUCxvBwRZMtRCB --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Gergő Magyar
|
9eaf2e6c4e
|
perf(mcp): cut the analyze-only language-provider closure out of MCP server startup (#2802) (#2806)
Some checks failed
CodeQL / Analyze (javascript-typescript) (push) Has been cancelled
CodeQL / Analyze (python) (push) Has been cancelled
Gitleaks / gitleaks (push) Has been cancelled
Publish / Classify release event (push) Has been cancelled
Scorecard / Scorecard analysis (push) Has been cancelled
Skill copy sync / shipped skills drift guard (push) Has been cancelled
Trivy Image Scan / Trivy (gitnexus-cli) (push) Has been cancelled
Trivy Image Scan / Trivy (gitnexus-web) (push) Has been cancelled
Publish / RC guard (marker + release-PR skip) (push) Has been cancelled
Publish / ci (push) Has been cancelled
Publish / Publish to npm (push) Has been cancelled
Publish / Build & Push RC Docker images (push) Has been cancelled
* fix(mcp): key the empty-ascent note on CALL_SUMMARY data, not language (#2802) `pdg-impact.ts` decided whether to append a "return-value ascent is TypeScript/JavaScript-only" caveat to the `impact(mode:'pdg')` note by looking up the criterion file's language. That put language-specific logic in a layer that must be language-agnostic, and it was a lossy proxy for a fact the graph already holds. Whether the ascent can fire is a property of the persisted CALL_SUMMARY edges. The descent already computes it, so thread the resolved-callee and return-flowing counts out of `interproceduralDescent` and key the note on those instead. Three defects the language proxy carried, all gone: - Wrong for `.mjs`/`.cjs`/`.mts`/`.cts`: the provider registry's extension arrays omit them while the ingestion pipeline parses them as TS/JS, so those files were harvested but the note claimed their ascent was empty. - Silently stale: any language whose harvester started recording formal indices would keep getting the caveat until someone edited the list. - Wrong in reverse: a TS/JS callee with no return-flow got no caveat, so an ascent that found nothing read like one that covered the slice. `pdg-impact.ts` now names no language and imports nothing from the language layer, which also drops the analyze-only provider closure from MCP server startup. Measured on overlayfs against a full build: import mcp/local/local-backend.js before 565-648 ms / 548 modules import mcp/local/local-backend.js after 458-463 ms / 170 modules Tests hold CALL_SUMMARY content fixed while varying the file extension across nine languages and assert the note text is identical, then hold the extension fixed and vary the summary to show the note tracks the data. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(mcp): guard MCP startup against the language-provider closure returning The eager `pdg-impact.ts -> core/ingestion/languages` edge was found and lost once already during #2793 before #2802 re-derived it, so it gets a test rather than a comment. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs(lbug): record why csv-generator is not lazy-imported #2802 proposed cutting `csv-generator.js` out of the adapter chain to shorten MCP server startup. Measured on a native filesystem, the marginal cost is small relative to the siblings this module already imports, and `core/search/bm25-index.ts` statically imports `normalizeFtsText` from the same module on a path `local-backend.ts` reaches dynamically for FTS — so deferring would relocate the cost to first query, not remove it. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(pdg): pin chained receiver calls reaching BasicBlock.calleeIds The PDG inter-procedural descent hops through `BasicBlock.calleeIds`, so it can only cross a call boundary the resolver resolved. Chained receiver calls reach `calleeIds` through the receiver-typing pass's own `calleeIdSink` — a separate path from plain calls. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs(analyze): drop the stale per-language cross-reference (#2802 review P3-4) `pdgModeMismatch`'s comment told readers to keep "the diagnostic per-language refinement in the impact CONSUMER (see pdg-impact.ts assemblePdgImpactResult)". That refinement is no longer per-language — removing it is the point of #2802, which now keys the empty-ascent note on the persisted CALL_SUMMARY data instead. The comment's real invariant is untouched and still correct: the values in `resolvePdgConfig` must stay scalar, because the comparison below is a shallow `!==` and an object would compare by reference. Only the cross-reference was stale. Comment-only; no executable line changes. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(mcp): probe the real module loader for the startup language closure (#2802 review P1-2) The previous guard hand-rolled a regex walk over TypeScript source to assert `core/ingestion/languages` was not statically reachable from MCP startup. Four bypasses were reproduced against it, any one of which let the exact 226-module regression return while the test stayed green: a. Wrong entry root. It walked from `mcp/local/local-backend.ts`, but the server module is `mcp/server.ts` — which imports LocalBackend as `import type`, so the guard's anchor was not even on server.ts's runtime closure. Ten real startup modules sat outside it. b. A top-level `await import(...)` executes during module evaluation, so it is eager at startup — but the walker skipped every `import(...)` by construction. c. The `import type` strip deleted a 16,445-character window of `pdg-impact.ts`: an `export type X =` matched lazily to the next `from "…"`, which lives inside a string literal. Any import in that window was invisible. d. The comment strip treated a `/*` inside a string literal as a comment opener. Replace the approximation with a real module-load probe: spawn a child node process per entry, import the built `dist/` entry, and report what the loader actually pulled in. Rooted at `dist/mcp/server.js` and `dist/cli/mcp.js` (the real startup entries) plus `dist/mcp/local/local-backend.js`. Syntax cannot fool it. One deviation from the two existing sibling probes is load-bearing: `dist/` is ESM, so a `require.cache` diff alone cannot see the first-party `dist/**` graph — it only catches CJS and native modules, which is why `import-closure.test.ts` gets away with it (it asserts on `@ladybugdb/core`). A pure cache diff here would have reported zero language modules unconditionally, i.e. a new vacuous guard. This probe unions `module.registerHooks({ load })` with the cache diff, and each entry carries a non-vacuity anchor and a module floor so an empty result fails loudly. Verified load-bearing: adding a top-level `await import('../core/ingestion/languages/index.js')` to `src/mcp/resources.ts` and rebuilding turns `dist/mcp/server.js` red with 70+ named offenders, while the `local-backend` and `cli/mcp` cases stay green — which is bypass (a) demonstrated directly. The old guard passed that poisoned tree entirely. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs(lbug): drop the unreproducible 9p multiplier from the csv-generator note (#2802 review P3-2) The comment justifying why `csv-generator.js` is NOT lazy-imported carried a hard "~40x" figure for how much a 9p mount inflates per-file ESM resolve. Three independent measurements during review produced ~40x, ~7.3x and ~30x, so the multiplier is not a reproducible quantity and had no business being stated as one in a durable comment. Reworked so the STRUCTURAL argument leads and the numbers only support it. That argument is what actually settles the question and it does not rot: `core/search/bm25-index.ts` statically imports `normalizeFtsText` from `csv-generator.js`, and `local-backend.ts` reaches bm25-index through a dynamic import on the FTS query path — so deferring here relocates the cost to first query rather than removing it. Both verified again at `bm25-index.ts:15` and `local-backend.ts:2756`. Remaining figures are re-measured, attributed to a date and issue, and labelled by filesystem: ~1.6 ms marginal (median of 45 cold imports on local disk) versus ~50 ms for the same import on a network mount, stated as environment-bound rather than as a property of the module. The provider-registry cost is given as "several hundred modules" — the static walk, the runtime hook, and the reviewer's probe each counted it differently (375 / 439 / 407), so no single number was picked to go stale. The old "226 modules" was real but counted only the `languages/` subtree and undercounted the win. Also repoints the trailing reference to the guard's new home at `test/integration/mcp/startup-language-closure.test.ts` (same comment block, inseparable from this rewrite). Comment-only; no executable line changes. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(mcp): stop the empty-ascent note asserting a fact an undecodable summary contradicts (#2802 review P2-2) The note claimed "this is a property of the persisted summaries" whenever the descent resolved callees and none carried a return-flow. But `decodeCallSummary` never throws by design: a version-skewed (`2|r:1`), corrupt (`1|r:zz`), or NULL `reason` yields no entry, which was indistinguishable from a cleanly-decoded empty summary. So the note could assert "no formal parameter is recorded as flowing to its return value" about a callee whose CALL_SUMMARY actually records `p0 -> return`. `meta.pdg.hasCallSummary` is a plain boolean and stores no codec version, so nothing else caught it. `calleesWithReturnFlow` now reports three outcomes instead of two — flowing, decoded-empty, and undecodable — and the undecodable count is threaded through the descent to the note. When it is non-zero the note says so and points at a re-index; when every summary decoded, the persisted-summaries claim is kept and now explicitly conditioned on that. Soundness is unchanged: an undecodable summary still licenses no ascent and never enters the return-flowing set, so the ascent path is byte-identical. Only the note's wording moves. Tests drive all three undecodable forms through the mock and assert the false claim is gone, the remedy is reported, and the ascent is still withheld. A companion assertion pins that the all-decoded case KEEPS the persisted-summaries claim, so the fix cannot degenerate into deleting the sentence. Verified load-bearing: reverting the source alone fails 6 of 34. Impact analysis: `calleesWithReturnFlow` upstream LOW (2 callers, both in this file); `assemblePdgImpactResult` upstream LOW (1 caller). Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(pdg): cover every chained-receiver shape and pin the inference gap (#2802 review P2-1, P3-1) The fixture proved chained receiver calls reach `BasicBlock.calleeIds` using exactly one receiver form — a local `const`. That is the shape that works, so a single-shape fixture implied general support the resolver does not have. This repo has been burned by that before: a drop-count gate blind to fixed shapes. Measuring nine forms against the real pipeline also corrects how the gap was originally characterised. It is NOT local-versus-field. An annotated field resolves fine, including the constructor-assigned variant: private p: Outer = new Outer(); -> both links private p: Outer; this.p = new Outer(); -> both links private p = new Outer(); -> EMPTY CELL private p; this.p = new Outer(); -> EMPTY CELL The discriminator is the type ANNOTATION. When a field's type must be inferred from its initializer the whole `calleeIds` cell empties — so even `Outer.inner`, an ordinary named-receiver call, is lost, and the inter-procedural descent cannot cross the boundary at all. Pre-existing; independent of #2802, which does not touch receiver resolution. The fixture is now table-driven over seven working forms (local const, local in a method, annotated field, ctor-assigned annotated, ctor-param assigned, call-result receiver, three-link chain) plus the two inference-typed forms, each row carrying its expected chain-link ids. Assertions moved from substring to exact id membership, split with the production `splitCalleeIds` reader — so `Inner.compute` can no longer be satisfied by `Inner.computeExtra` or `OtherInner.compute`, which matters because the descent keys on exact ids for span and CALL_SUMMARY lookup. The two known-gap rows are pinned with `it.fails` plus a hard assertion on the exact gap-row set, so a resolver fix turns them red instead of passing silently, and an anti-vacuity guard requires every shape to match exactly one block — without it a drifted fixture matching zero blocks would let `it.fails` pass for the wrong reason. Proven by mutation: relabelling a working row as a known gap fails both pins. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(mcp): qualify the empty-ascent note when the examined callee set is incomplete (#2802 review P2-4) The note asserted "none of the N resolved callees carry a CALL_SUMMARY return-flow", and on the all-decoded path that this is "a property of the persisted summaries". Both are universal claims over the callees the descent actually examined, and two mechanisms can leave that set incomplete without the note saying so: 1. Budget truncation. The descent stops on depth/limit/node-cap, so a callee that DOES carry a return-flow can sit in a hop never reached. A 4-deep chain reported "none of the 3 resolved callees" while link 4 held the only summary. 2. Emit-time capping. When a block's `calleeIds` cell was capped, `splitCalleeIds` strips CALLEES_TRUNCATED_SENTINEL, so the dropped callees are invisible to both the scan and the counters — even though the callgraph bridge in this same file already treats such a block as callee-incomplete. Add `calleeIdsWereTruncated`, the counterpart to the sentinel strip, read from the raw cell before splitting so a block whose entire list was capped away still raises the flag. Thread it through the descent to the note. Case 1 needs no new plumbing — the aggregate `truncated` is already on the input object. Using the aggregate rather than a descent-only flag is deliberate: seed truncation and intra-BFS depth truncation also shrink the initial slice, so their callees are never gathered either. It is a sound superset that never under-hedges. When either mechanism fired, one clause naming the reasons is appended and the whole-slice assertion softens to "every summary examined decoded … a property of those summaries". When the set is complete both branches stay byte-identical to before, so this does not become a blanket hedge. Tests pin truncated, untruncated, emit-capped-alone, both-mechanisms, and undecodable+truncated, asserting the truncation premise rather than assuming it. Verified load-bearing: reverting the source alone fails 6 of 42, and the HEAD note printed in those failures is the bug verbatim. Impact analysis: `assemblePdgImpactResult`, `calleeIdsByBlock`, `interproceduralDescent` all upstream LOW; every caller is in this file and `runImpactPDG`'s exported signature is unchanged. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(mcp): stop the empty-ascent note calling call-site references "resolved callees" (#2802 review P3-7) The note printed "none of the N resolved callees carry a CALL_SUMMARY return-flow (no formal parameter is recorded as flowing to its return value)". N counted the raw `BasicBlock.calleeIds` cell, which carries ids `resolveCalleeSpans` never enters — out-of-repo targets, interface methods, and the `Class:` id a `new X()` emits. On the chained-receiver fixture that inflated N from 1 to 3. Two defects, both in the wording rather than the arithmetic: "resolved" implies a symbol-table lookup that did not happen for those ids, and the parenthetical asserted a FORMALS-level property about symbols never resolved to a body. Reworded rather than re-seeded, deliberately. `calleesWithReturnFlow` scans the RAW id set, so the claim "none of these carries a return-flow" is exactly established for all N — the scan really did check the `Class:` id. Re-seeding N from the resolved spans would make the sentence quantify over a strict SUBSET of what was checked, silently dropping the un-enterable references from a claim that genuinely covers them, and would desync N from `calleesUndecodable`, which is derived from the same scan population. none of the N resolved callees carry ... none of the N call-site callee references carry ... and the formals parenthetical is dropped. The note gets shorter, not longer. `calleesResolved` is renamed `calleeReferences` end-to-end (file-local; nothing outside referenced it), and the descent's return-type doc — which called them "callee symbols the descent resolved" and reinforced the wrong reading — now states that un-enterable ids ride the same cell, are scanned, and are never entered. The `> 0` gate is unchanged, so no slice that previously produced the note stops producing one. A test pins that explicitly: an all-un-enterable cell resolves no span, takes no hop, and emits no ascent sentence despite a non-zero count — so a future re-seeding cannot silently move when the note fires. Tests also pin the quoted number and singular/plural against a mixed cell, with a discriminator asserting `reachableBlocks` is byte-identical while the count moves 1 -> 3. Verified load-bearing: reverting the source alone fails 6 of 7 new tests, printing the finding verbatim. Impact analysis: `assemblePdgImpactResult` and `interproceduralDescent` upstream LOW, sole caller `runImpactPDG` in the same file; exported signature unchanged. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(mcp): pin cross-hop callee accumulation and the mixed return-flow contract (#2802 review P2-5) Every case in this file drove a single hop, so the Set union the descent performs across hops (`calleeReferencesSeen` / `calleesReturnFlowingSeen`) was never proven to accumulate rather than overwrite — a one-hop descent cannot tell the two apart. And although a sibling commit added a three-id cell, none of those ids return-flowed, so the "some callees flow, some do not" boundary was entirely unpinned. Extends the mock with a `secondSummary` knob that drives a genuine second hop: `helper2` is named only in `helper`'s own body block, so the descent must cross a second boundary to reach it. Three mock handlers are made faithful to the parameters they already bind — `calleeIdsByBlock` now routes on the asked `$ids`, and the CALL_SUMMARY scan and span resolve answer per asked id — which is what makes a second callee answerable at all. Existing cases are behavior-identical. Five tests: the union count across two hops; a return-flow on hop 0 surviving a later empty hop; a return-flow found only on hop 1; mixed callees in one examined set going silent rather than partial; and a flowing callee alongside an undecodable sibling staying silent including the decode remedy. The mixed case pins a deliberate contract rather than proposing one. The production condition is `calleesReturnFlowing === 0`, so partial coverage is reported as silence. A reviewer considered and dropped "report partial coverage" as a product change; this makes flipping it a conscious edit instead of an accident. Verified load-bearing against three separate source mutations: accumulating only on hop 0 (2 fail), each hop overwriting instead of unioning (3 fail), and flipping the gate to partial-coverage reporting (4 fail). In all three every PRE-EXISTING test still passed — which is the finding restated as evidence. Test-only; `pdg-impact.ts` is byte-identical to HEAD. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs(mcp): consolidate the empty-ascent rationale to one canonical site (#2802 review P3-6) The "keyed on observed CALL_SUMMARY data, never on the criterion's language" rationale was restated in full at four comment sites. It exists because a reviewer asked "why not just look up the language?", so it has to stay findable — but not four times. The canonical explanation now lives in `interproceduralDescent`'s return-type doc, where the counters are actually computed, organised as POPULATION (why the raw `calleeIds` tally is the right set to quantify over) and OBSERVED DATA, NEVER THE CRITERION'S LANGUAGE (the full answer, including the producer-change argument and the no-language-naming rule). The other three sites keep only what is locally load-bearing and point here. Deliberately preserved, because each carries a non-obvious fact: why an undecodable summary licenses no ascent, why the aggregate `truncated` is used rather than a descent-only flag, and the raw-id-tally population argument. Net comment delta -11 lines. The reviewer also flagged the local/field naming asymmetry (`calleeReferencesSeen` vs `calleeReferences`). Keeping the suffix, with a comment recording why so it is not re-raised: the premise that every other local matches its field is true, but those locals are identity-returned, whereas these are `Set<string>` accumulators returned as `.size`. Dropping the suffix would give one identifier two types in one file — a `Set` at the accumulation site and a `number` where the note does arithmetic and pluralisation on it ~900 lines away. The Set-ness is also load-bearing: the dedup is why a callee invoked from two hops is not double-counted, which is what makes the note's count correct. Comment-only. Verified mechanically: every added and removed line in `git diff -U0` matches a comment pattern, so the note's template literals are untouched and its rendered text is byte-identical. 89 tests unchanged. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor(mcp): collapse the ascent plumbing accreted across 13 fix commits Quality cleanup, no behavior change. Four independent review passes converged on the same root cause: thirteen commits each fixed one review finding in isolation, and the ascent facts grew one loose field at a time until 62% of the changed region was comments explaining plumbing. Five changes: - `calleeIdsFromBlocks` deleted. Zero call sites anywhere in src/ or test/ — already dead on main, and this branch had edited it to keep it compiling. Its only reference was a stale `{@link}` in a neighbour's doc, now rewritten to stand alone. - `parseCalleeIdsCell` replaces the two-pass read. `calleeIdsWereTruncated` and `splitCalleeIds` were splitting the same cell on adjacent lines, which measured ~2x the parse cost (0.82 -> 1.59 ms at a realistic hop, 57.7 -> 92.7 ms at the per-statement site cap) and was a second independent encoding of the sentinel format — exactly what `splitCalleeIds` was extracted to prevent. One pass classifies as it walks; `splitCalleeIds` stays as a wrapper so its two external callers are untouched. The single-use `export` is gone. - `AscentCoverage` replaces four fields threaded through three signatures. ~12 declaration sites become 3, and the canonical rationale now lives on the type by construction — which is why the earlier doc-consolidation commit was needed at all. - `calleesReturnFlowing` becomes a boolean. Its only reads were `=== 0`, twice; it cost a Set sized to every callee in the slice plus a per-hop union loop. The flag is set inside the existing `returnFlowing.size > 0` branch — equivalent, since the cross-hop union is non-empty iff some hop's was. - The duplicated empty-ascent note head is collapsed to one gate and one head with per-arm tails. Both arms had been edited in lockstep twice in this branch's own history. The rendered note text is byte-identical. Verified structurally and then empirically: both expressions reconstructed standalone and diffed across the full cross product of references x returnFlowing x undecodable x truncated x listTruncated — 288 combinations, 0 mismatches. Net -53 lines. 102 tests pass unedited; the unused-symbol lint warning is gone. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(mcp): parallelise the startup probes, drop a redundant pin, name the mock knobs Quality cleanup from the same review passes. The set of verified behaviors is unchanged except where noted. **Startup probes run concurrently.** `spawnSync` blocks the event loop and vitest runs a file's tests in order, so the three probes strictly serialised. Launching all three with async `spawn` in `beforeAll` and asserting over the collected outcomes cuts the file from ~12.7 s to ~3.9 s wall (-69%). Every promise is caught before `Promise.all`, so all three children are reaped and failures report per entry rather than surfacing only the first rejection. Preserved and each proven by mutation: the missing-dist error names its entry, a raised module floor fails only its own row, and a bogus anchor still reports the loaded-module count. **The two `it.fails` rows are removed.** They pinned the inference-typed receiver gap that the strict `toEqual` pin beside them already covers — and they were the weaker of the two, because `it.fails` passes when the body throws for ANY reason, including `idsFor`'s own non-vacuity guard. A renamed fixture marker would have kept them green on a rotted premise. The strict pin is self-diffing and was verified load-bearing on its own: pointing a known-gap marker at a resolving shape fails it with the two newly-present ids listed. The file header now carries the gap's durable description. **The ascent-note mock takes options objects.** `descentExec` and `run` had grown to five and seven positional parameters in the order five agents added them, so call sites read `run(FILE, true, null, 3, false, undefined, null)` — several carrying `undefined` purely to reach a later argument. All 34 call sites are converted; nine that used only defaults are now bare `run(file)`. No knob renamed — they are orthogonal and correctly named. Code lines are exactly neutral (353 -> 353); the win is at the call sites. Also refreshes five comments that still described `calleesReturnFlowingSeen` and the two-branch note, both of which the preceding commit replaced. 102 unit and 10 integration tests pass; test count moves 9 -> 7 in the chained-receiver file, exactly the two redundant rows. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(mcp): publish return-value-ascent coverage on the PDG impact result `impact(mode:'pdg')` computed four facts about ascent coverage and used them exactly once — to interpolate an English sentence. They never reached the result object, so an agent consuming this MCP output could only ask "was the ascent complete, and if not why" by regexing prose. The cost was already demonstrated: a pure rewording commit earlier in this branch broke ~30 assertions and would have silently broken any consumer keying on the old phrase. Adds `pdgEvidence.ascent`: referencesScanned how many call-site callee references were scanned returnFlowFound did the ascent fire anywhere in this slice undecodableSummaryCount summaries the codec could not decode examinedComplete was the examined set the whole callee list incompleteReasons 'traversal-truncated' | 'callee-list-capped' callSummaryLayerPresent false => pre-FU-C (v3) index Nested under `pdgEvidence` because that is the established counts-and- classification namespace, and `composeUnifiedPdgImpactResult` already spreads it, so the member survives the unified compose untouched. `incompleteReasons` carries CODES, following the existing `truncatedByReasons: ('depth'|'limit')[]` precedent. The prose clause and the structured field now render from one array computed once, so an agent branching on codes and a human reading the note cannot disagree, and a third reason becomes a rendering decision rather than a contract change. Two shape decisions worth recording. `callSummaryLayerPresent` exists because without it a v3 index publishes `referencesScanned: N, returnFlowFound: false`, which reads as "these callees record no return-flow" when the truth is "the layer that records it is absent" — the note already distinguishes those, and the structured surface must not be less honest than the prose. And the field is ABSENT rather than zeroed when the descent never ran (upstream slices): "nothing was scanned" is a different fact from "we scanned and found nothing". `pdgResultVersion` stays 2. The documented trigger is a BREAKING change to the result shape; this removes nothing, renames nothing, and changes no existing field's meaning. Confirmed mechanically: zero top-level key drift across 2304 cases. The historical v2 bump was for changing an existing field's semantics (startLine 0- to 1-based). The note prose is byte-identical, proven across the same 2304 cases with a negative control — perturbing one character of the phrase table produces 60 drifts, so the harness demonstrably detects what it asserts. 14 new tests cover the structured surface and all 14 fail when the source is reverted, while the 54 prose tests pass unchanged. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(helpers): share one module-load probe, and fix two guards that passed on broken builds Three tests independently spawned a child node process to inspect what a built `dist/` entry loads, duplicating the REPO_ROOT derivation, the probe source, the missing-dist guard, the spawn with NODE_OPTIONS cleared, the status-vs-signal rendering, and the payload parse. The newest copy was also the only correct one, so the next author had 2-in-3 odds of copying a weaker probe. The two older probes diff `require.cache` only, which is structurally blind to the first-party ESM `dist/**` graph. That is not theoretical — both were demonstrated passing on genuinely broken builds: - Severing `dist/cli/mcp.js -> stdio-context.js` (a pure ESM change) leaves the require.cache diff EMPTY, so `import-closure.test.ts`'s two assertions reduce to `[].filter(...) === []`. It reported 2 passed on a severed graph. - Severing `registry -> swift/query.js` leaves 76 unrelated CJS entries, which satisfied `registry-import-closure.test.ts`'s indirect guard. The Swift half of its headline had gone vacuous and it reported 1 passed. Both now fail on those same builds, naming the missing anchor. `test/helpers/module-load-probe.ts` unions the ESM `registerHooks({ load })` channel with the cache diff, probes entries concurrently, and makes non-vacuity STRUCTURAL: `anchor` and `minModules` are required fields and the helper throws when either fails. A vacuous probe is a harness failure, not a silently green test, so it cannot be forgotten. Forbidden patterns and remedy text stay per-test — the harness is the shared part, the policy is not. Also fixes `toRepoRelativePosix` resolving non-absolute specifiers against `process.cwd()`, and dedupes modules a CJS-from-ESM import reported once per channel. Faster despite doing more: the registry file goes 12.4s -> 6.75s, because `spawnSync` burned the parent thread polling while the child loaded native grammars. `import-closure` drops to one spawn from two. The `local-backend.js` entry is kept although its closure is currently a strict subset of `server.js`'s: that is an observation, not an invariant. If `server.js` ever stops eagerly reaching the local backend, the server probe stays green while the module #2802 actually changed goes unobserved — and now that anchors are mandatory, that entry is what pins `pdg-impact.js`. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs(lbug): trim the csv-generator note and fix the claim it got wrong Two reviewers split on this comment: one wanted it cut to the structural argument, the other said a comment is the right depth for documenting a rejected change since there is no invariant to guard. Both are right, so it stays a comment and gets shorter — 13 lines to 6. Trimmed because it had already taken two corrections (an unreproducible "~40x" figure, and a pointer to a test file that no longer exists), and its tail had drifted from its own guard: the comment said "several hundred modules, ~150 ms" where `startup-language-closure.test.ts` says "~226 extra modules and ~130 ms". Two numbers for one fact. That tail is documented better in the guard's own header, so deleting it loses nothing. It also stated the load-bearing claim inaccurately. The old text said bm25-index imports `normalizeFtsText` "from here" — but `lbug-adapter.ts` neither exports nor re-exports it; the only occurrence of the identifier in this file WAS the comment. Anyone verifying would have grepped, found nothing, and concluded the note was stale. Now names `csv-generator.js` explicitly, re-verified at `bm25-index.ts:15` (static) and `local-backend.ts:2756` (dynamic, on the FTS query path). Comment-only, proven two ways: every changed line matches a comment pattern, and stripping all `//` lines from HEAD and from the working tree yields byte-identical text. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(helpers): extract the temp-repo lifecycle, collapsing five hand-rolled cleanups into one Four cfg integration tests each hand-rolled a `tmpDirs` array, a mkdtemp-and-register step, and an `afterAll` rmSync. It is actually five registrations across six creation sites — `pipeline-pdg.test.ts` keeps a second pool for its C-family fixtures. Seeding genuinely varies four ways (recursive cpSync, single copyFileSync, inline mkdir+writeFile, and nothing at all), so a fixture-copier helper would have fitted about half the sites and made things worse. Extracted the LIFECYCLE instead — mkdtemp, register, afterAll cleanup — which is byte-identical at all five registrations and is the correctness-critical part. `dir()` returns an empty registered directory for callers that seed themselves; `fromFixture()` covers the common case. That fits 6/6. The duplication had already produced a latent defect: `cFamilyTmpDirs` was cleaned by TWO `afterAll` blocks, harmless only because `rmSync` was called with `force: true`. Now one hook. `createTempDirPool` is a function called from each test file's module scope rather than a top-level hook in the helper, because under ESM caching a module-level `afterAll` would register once, against whichever file imported it first. That hazard is documented in the helper. Raw line count is roughly neutral (-44 across the tests, +62 for the helper, 29 of which are the rationale). The win is that a cleanup invariant went from five copies to one. Cleanup verified empirically, including the failure path: a throwaway suite whose `beforeAll` throws still has its directory removed, and every temp directory created by the four migrated files is gone after a run. 46 tests pass across the four files. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(resolvers): pin the inference-typed field receiver gap at the resolver level The gap was pinned only in a PDG test, asserting on `BasicBlock.calleeIds` behind the full `--pdg` pipeline. But it is a resolver fact: when a class field's type must be inferred from its initializer, chained receiver calls resolve to nothing. Whoever closes it will be working in the resolver suite and would have got a red CFG/PDG test with no resolver-side signal. Asserts CALLS edges directly, alongside `python-constructor-field-receiver.test.ts`. Nine receiver shapes run the identical statement; seven resolve, two do not: const o = new Outer() resolves private p: Outer = new Outer() resolves private p: Outer; this.p = new Outer() resolves private p: Outer; this.p = p (ctor arg) resolves constructor(private p: Outer) {} resolves makeOuter().inner().compute() resolves o.inner().mid().compute() (three links) resolves private p = new Outer() NO EDGES private p; this.p = new Outer() NO EDGES Two things the fixture establishes that the PDG-side pin could not. The discriminator is the type ANNOTATION, not local-versus-field — the parameter-property form resolves fine. And the initializer is NOT invisible to the resolver: `new Outer()` still emits its own constructor CALLS edge, byte-identical to the annotated twin. Only the initializer-to-field-type binding is missing, which narrows where a fix belongs. Assertions key on exact node ids rather than names, because `compute` is ambiguous across two classes and keying on the source name collides with `Object.prototype.constructor`. No `describe.skip` and no `it.fails` — the latter passes when the body throws for ANY reason, so it can go green on a rotted premise. The gap is pinned as its explicit current value, which self-diffs: simulating the fix fails one test showing the two newly-resolved ids, and renaming a fixture symbol fails the non-vacuity guard. Runtime is comparable to the PDG-side pin (~9-11s, both dominated by worker startup), so this is an altitude and scope win, not a speed one. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(mcp): replace the extension sweeps with a stronger language-agnosticism pin Two `it.each` sweeps over nine file extensions asserted that the empty-ascent caveat was present (or absent) for each. They looked like the pin for the property the whole change exists for — `pdg-impact.ts` must name no language and its output must not vary by extension — but they were the weakest available form of it. They asserted substring presence/absence, so a language dependence that ADDS text while leaving the caveat intact passes them. Demonstrated, not assumed: injecting a `.py`-only hedge inside the caveat sentence and replaying the two sweeps verbatim against that source gives 18 passed. The byte-identity test beside them caught it. So the sweeps are deleted and the identity test carries the property alone, hardened in two ways: - Two rows instead of one, covering BOTH sides of the caveat gate. The silent (return-flow present) branch previously had no identity counterpart at all — nine runs proving one fact, with nothing checking that its rendering was extension-invariant. - The fingerprint spans the note AND the reachable blocks, not just the note. Strictly more than the sweeps verified. Entailment is exact: identity across the extension set, plus the two existing single-extension content assertions, gives "every extension gets the caveat" and "no extension gets it". Reducing a sweep to one extension was rejected because it reproduces an assertion already present verbatim. Also converts the incompleteness block from six near-identical bodies to a 3-row premise table crossed with two assertions. Each row now names the exact phrase set its clause must contain, so presence and absence are asserted together — which adds three checks the longhand version lacked (the budget row now also proves the emit-cap phrase is absent). And three tests that re-rendered one fixture to make one assertion each are hoisted to a single render. 97 tests, down from 116: -18 sweep cases, -2 from the hoist, +1 identity row. No assertion was lost; several were added. Verified by injection: a `.py`-only note change fails the identity pin, and a dependence in the shared hop sentence fails BOTH rows, confirming the second row is load-bearing rather than decorative. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * perf(mcp): lazy-import syncGroup so MCP startup skips the group extractor closure `core/group/service.ts` statically imported `./sync.js`, which pulls all six contract extractors, five of which statically import the native `tree-sitter` binding. That put the whole parser stack on every MCP server start, for a server that never syncs. Only `groupSync` needs it. The other seven group tools — `group_list`, `group_impact`, `group_query`, `group_contracts`, `group_status`, `group_trace`, `group_context` — do not, and now never load it. `syncGroup` has a single call site, already inside an `async` method, so this is a lazy `await import(...)` at that call site and nothing else: no signature change, no async ripple, no change to `local-backend.ts`. The pattern is already established on this exact module — `cli/group.ts`'s sync command lazy-imports `sync.js` the same way. `service.ts` was the outlier. Measured on a native filesystem (overlayfs; /workspace is a 9p mount that inflates ESM resolve, so it is not a valid measurement surface), 5 cold runs, medians: dist/mcp/server.js 521 ms -> 133 ms (-75%) dist/mcp/local/local-backend.js 453 ms -> 66 ms (-85%) tree-sitter modules at both entries: 11 -> 0 Same defect class as #2802, which cut the language-provider registry from the same startup path; this is what remained. The cost is moved rather than deleted: the first `group_sync` call now pays the module load. That is the right trade — `group_sync` is already a long-running operation, and sessions that never sync pay nothing. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(mcp): guard MCP startup against the group extractor closure returning Sibling forbidden-pattern case in the #2802 startup guard, reusing the concurrent probes it already collects — no new spawn, no new harness. Asserts that none of `dist/mcp/server.js`, `dist/cli/mcp.js`, or `dist/mcp/local/local-backend.js` loads a `core/group/extractors/` module or the native `tree-sitter` package. The parser is matched by package prefix rather than a bare substring, so a source file that merely mentions the word can neither satisfy nor trip it. Verified load-bearing rather than assumed: restoring the static `import { syncGroup }` in `core/group/service.ts` and rebuilding turns `dist/mcp/server.js` red and names all seven offenders — http-route, grpc, thrift, topic, include, manifest and workspace extractors. Reverted and re-confirmed green. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * perf(mcp): keep the analyze-only CFG closure off MCP server startup (#2802 review) `mcp/local/pdg-impact.ts` imported `CALLEES_TRUNCATED_SENTINEL` and `CALLEE_ID_SEP` from `core/ingestion/cfg/emit.ts`. ESM evaluates a module to import any binding from it, so those two strings dragged the whole analyze-only CFG closure into every MCP server start. Measured against a clean build, per entry point: 8 modules — `emit`, `reaching-defs`, `reaching-defs-graph`, `control-dependence`, `post-dominators`, `synthetic-escape`, `call-site-harvest`, `reaching-def-reason-codec` — present at `dist/mcp/server.js`, `dist/mcp/local/local-backend.js` and `dist/mcp/http-transport.js`. Same defect class as the language-provider closure this branch already removed, and the guard could not see it: `FORBIDDEN_RE` covers `core/ingestion/languages/` and `FORBIDDEN_GROUP_RE` covers `core/group/extractors/|node_modules/tree-sitter`, neither of which matches `core/ingestion/cfg/`. The format constants move to a new LEAF module `cfg/callee-cell-format.ts` that imports nothing; `emit.ts` re-exports both names so every existing importer is untouched, and producer and consumer still resolve to one definition — the drift the shared constant exists to prevent stays impossible. Deleted, not deferred — the same bar #2802 held its own csv-generator proposal to. After: cfg modules at startup 8 -> 2, and both survivors (`callee-cell-format`, `reaching-def-reason-codec`) are leaves that import nothing. Totals: `server.js` 387 -> 380, `local-backend.js` 163 -> 156, `http-transport.js` 523 -> 516. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(mcp): stop pdgEvidence.ascent claiming a completeness it cannot have (#2802 review) `examinedComplete` is the field a consumer reads to decide whether `returnFlowFound: false` is a whole-slice claim. It could be published `true` over a callee set the descent never finished examining — the exact false all-clear the field was added to prevent. Root cause: `bfsReachableBlocks` sets `truncatedByDepth` when its frontier is still non-empty at the budget, but both call sites inside `interproceduralDescent` folded only the row-limit flag and dropped the depth flag. The top-level intra BFS's copy of that same flag was already propagated, so the asymmetry was unintended — one `if`-pair folding limit-but-not-depth, within a merge that already folds the node cap too. Reproduced at `maxDepth: 3`, the shipped default: a criterion calling a helper whose body is a 5-block dependence chain, with the return-flowing callee on the block past the clamp. Result reported `truncated: undefined`, `examinedComplete: true`, `incompleteReasons: []` and an unqualified universal note sentence. Fixed by propagating the dropped flags rather than inventing a parallel channel: `intraDepthBudget` is documented in-file as the SAME clamp the top-level intra BFS applies, and that one's depth truncation is already result-level. So the result's own `truncated`/`truncatedBy` were under-reporting for the same reason, and both surfaces are corrected together. Four further honesty fixes to the same published record: - Blocks reached only by the U-C4 ascent went into `reachable` but never `hopReached`, so their `calleeIds` cells were never scanned, never counted, and could not raise `callee-list-capped`. They are slice blocks; they now enter the hop set and get the same treatment as every other one. - `pdgEvidence.ascent` was absent on the empty-slice early return even though the descent had already run and scanned, contradicting the "present iff the descent ran" contract this branch itself added to `tools.ts`. Both exits now classify through one shared helper so they cannot disagree. - A block carrying call sites in `callees` but no resolved ids in `calleeIds` (the whole-file case where `emit.ts` has no fileMap) silently shrank the population while `examinedComplete` still reported `true`. That now raises a third reason, `callee-ids-unrecorded`. - `referencesScanned` is a distinct-callee tally and both surfaces described it as a call-site count. Field name kept — a rename is breaking at `pdgResultVersion: 2` — and the prose corrected instead. `PdgAscentIncompleteReason` gains a member, which is additive, so `pdgResultVersion` stays 2. Visible output change worth knowing: slices whose callee chain outruns `maxDepth` now report `truncatedBy: 'depth'` where they previously reported none, and a repo with id-less call sites now reports `examinedComplete: false`. Both are strictly more honest. Every behavioural change carries a mutation proof — revert the source, watch the new test go red, restore. One exception is documented inline rather than faked: the ascent-side fold cannot be observed independently, because the re-seed shares the caller's `visited` set and so can only reach past the budget when the traversal that covered that closure was already cut and had already raised a flag. Suite: 49 -> 59 tests. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(mcp): anchor each import-closure policy on the edge it polices (#2802 review) `module-load-probe.ts` makes non-vacuity structural via a required `anchor` — but the anchor was one per ENTRY while `startup-language-closure.test.ts` now runs TWO independent policies. The group-extractor policy added in |
||
|
Void Freud
|
7ee0df9e55
|
fix: serialize global registry transactions across processes (#2716)
* fix: serialize global registry mutations * fix: serialize global registry mutations * fix: keep registry reads lock-free * test(registry): document cross-platform lock coverage * fix(registry): isolate the registry lock's namespace, timeout and diagnostics Review follow-ups on the global registry lock (#2716): - The lock took `getGlobalDir()` itself, which is byte-identical to a repo's index slot when that repo is rooted at the user's home directory (a real dotfiles layout). `runFullAnalysis` holds the per-repo lock across its whole pipeline and `acquireIndexLock` is not reentrant, so `registerRepo` / `adoptFlatBranchLabel` self-deadlocked until the wait ceiling and then failed the analyze. The registry now locks a private `<globalDir>/registry-lock` namespace no index slot can ever resolve to. - The lock inherited the index lock's 10-minute default timeout, sized for multi-minute analyze runs. `gitnexus augment` — documented to cold-start in under 500ms and shelled out from editor tool-use hooks — reaches it through `listRegisteredRepos({ validate: true })`. Registry transactions are sub-second, so they now get their own 5s ceiling. - Contention was silent: no `log`/`onWaitStart` was wired, and the primitive's own texts attribute a wait to "another gitnexus analyze", which misnames a registry holder. A registry-specific line is emitted on wait start instead. - On timeout the transaction now proceeds unlocked with a warning rather than throwing. The lost-update race it guards was unguarded before this branch, so degrading to the old best-effort behaviour beats failing `analyze`/`list`/ `index` outright — none of which wrap these calls in a handler — on a wedged lock. - `adoptFlatBranchLabel`'s recursive `fs.rm` no longer runs inside the lock; only the closing re-read/mutate/write does, mirroring `clean.ts`, which deletes the branch directory before calling the locked `removeBranchIndex`. A slow delete no longer blocks every registry operation on the machine. * test(registry): cover the remaining locked mutators and the colliding layout Three of the five functions the registry lock wraps had no overlap coverage, so a future narrowing of the lock would go unnoticed. Adds: - overlapping `removeBranchIndex` calls on two branches of one entry, - an overlapping `unregisterRepo` / `registerRepo` pair on distinct repos, - a registration issued while an index lock is held on the global directory, which reproduces the home-rooted self-contention the lock namespace fix addresses. Each fails without the corresponding fix: the two overlap tests lose an update when `withRegistryLock` is bypassed, and the collision test sees the wait announcement and the degraded-write warning once the lock namespace is reverted to `getGlobalDir()`. The collision test asserts on those log records rather than on elapsed time, so it stays deterministic on a slow runner. * perf(registry): keep the validation walk out of the registry lock `listRegisteredRepos({ validate: true })` held the global lock across its read-only validation walk — an `fs.access` per entry, slow on a network mount or a large registry — even though the common case prunes nothing and writes nothing. That is the same lock `gitnexus augment` takes on every editor tool call, so unrelated registry work serialized behind a walk that never touched the file. The walk now runs unlocked; the lock is taken only when an entry is provably gone, and the prune is applied to a snapshot re-read inside it, so a registration that lands during the walk is no longer clobbered by a stale write. Same shape as the `adoptFlatBranchLabel` split. --------- Co-authored-by: Gergő Magyar <gergomagyar@icloud.com> Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> |
||
|
Gergő Magyar
|
7a064a1f2a
|
fix(storage): stop the Windows \\?\ long-path prefix from breaking repo path matching (#2667) (#2700)
* fix(lib): add stripWindowsLongPathPrefix for path comparisons (#2667) A caller can hand GitNexus a `\\?\`-prefixed path — the usual MAX_PATH workaround on Windows — and `path.resolve` preserves the prefix, so it reaches every string comparison GitNexus keys paths on. It also poisons relativization: `path.win32.relative` cannot express a relative path between a prefixed and an un-prefixed form of the same directory, so it returns the absolute target instead. That absolute string is the shape reported in #2667. The helper is deliberately scoped to the comparison domain. libuv's `fs__capture_path` does not re-add the prefix for over-MAX_PATH paths, so stripping a filesystem-facing path would break long-path access on hosts that have not opted into LongPathsEnabled. `\\?\Volume{GUID}\…` is left alone because the remainder is not a usable path. The test is fixture-free and takes an explicit `platform`, mirroring `normalizeAnalyzerRootPath`, and is registered on the cross-platform matrix since the whole transform is a POSIX no-op. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0119fkrRFdQDQKh58LY9Tqh2 * fix(storage): normalize the `\\?\` prefix in canonicalizePath (#2667) `canonicalizePath` is the single comparison key for the repo registry, MCP repo resolution and the server repo routes, and `registryPathEquals` compares its output as a plain string. A caller-supplied `\\?\` prefix therefore matched nothing: a repo registered as `D:\repo` was invisible to a caller passing `\\?\D:\repo`, which surfaces as "repo not found" or a duplicate registration from `analyze`, `remove`, `clean`, the MCP `repo` parameter and the server routes. Both branches are normalized. The realpath branch was already safe — libuv's `fs__realpath` strips the prefix itself — but the `catch` fallback returns `path.resolve(p)` untouched, and that is exactly the branch a path which is not on disk takes. Safe despite the CRITICAL blast radius (27 impacted, 12 direct dependents) because the result is only ever compared, never opened: all 23 call sites feed `registryPathEquals` or a string comparison. Both operands are canonicalized, so the equality relation is preserved and behaviour is unchanged for every un-prefixed input. The two regression assertions run only on windows-latest, where the file already runs via the cross-platform matrix. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0119fkrRFdQDQKh58LY9Tqh2 * docs(core): correct two false comments about Windows paths (#2667) Both comments assert the opposite of how the platform and the analyzer actually behave, and both would send the next investigator of #2667 the wrong way. `analyzer-identity.ts` claimed the `\\?\` prefix is one that `realpathSync.native` "can emit for paths over MAX_PATH". libuv's `fs__realpath_handle` strips the prefix unconditionally and rewrites `\\?\UNC\` back to `\\`, erroring if neither is present, so realpath never returns one. The prefix can only arrive from caller-supplied input. The optional group in the regex stays as a labelled defensive no-op, and the function's behaviour is unchanged on purpose: these identity fields are compared between an `analyze` and a later `status` run, so this is not the place to reshape a path. `include-extractor.ts` claimed "gitnexus analyze stores absolute paths in the File.filePath column". A full self-index at |
||
|
Gergő Magyar
|
2ec00b8952
|
fix(analyzer): reject cross-drive paths in the identity containment guard (#2688)
`isInside()` paired its `..` checks with no absolute-path rejection, so on
Windows it reported an unrelated drive as *inside* the parent. `path.relative`
cannot express a relative path between two drives and returns the absolute
target instead:
path.win32.relative('C:\\parent\\src', 'D:\\other\\file.js') // 'D:\\other\\file.js'
That string does not start with '..', so the guard passed it.
Impact, per call site:
- resolveInvokedArtifact: adopts `process.argv[1]` as the invoked analyzer
artifact whenever it merely sits on another drive. That file is then absent
from the validated build, so resolveAnalyzerRunnerIdentity throws — `analyze`
and `status` fail outright on a multi-drive Windows install (e.g. a launcher
on D: invoking a package installed on C:). This is how the bug surfaced: the
GitHub Windows runner keeps the repo on D: and temp fixtures on C:.
- cacheDirectory: the "trusted cache directory must be outside the package and
build roots" guard wrongly fires for a directory on another drive, rejecting a
legitimate configuration.
- validateIdentityCache / cachedBuildDigestForPath: a containment check that can
answer "inside" for a path on another drive is weaker than intended.
Fix: reject an absolute `path.relative` result. This is the idiom the repo's
other containment guards already use — server/api.ts, server/git-clone.ts and
group/extractors/fs-utils.ts all pair the '..' check with `path.isAbsolute`;
this function was the outlier.
`pathApi` is injectable (defaulting to the platform-bound `path`) so the win32
semantics are unit-testable from a POSIX runner. The new test is fixture-free
and registered on the cross-platform matrix; its cross-drive case fails without
the guard and the same-drive/POSIX cases pass either way, proving the fix is
narrow.
Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
|
||
|
Gergő Magyar
|
df0110b06f
|
fix: index staleness — false-stale status after analyze (#2668) + inline staleness in query/context/impact/cypher tools (#2655) (#2683)
* fix(analyzer): case-stabilize runner-identity path fields so status isn't false-stale (#2668) `gitnexus status` reported a freshly-analyzed, untouched repo as stale on Windows (econia/aptos-core, 1.6.10-aptos.0). `status`'s up-to-date check gates on `runnerIdentityIsCurrent`, which deep-compares the stamped runner identity against a freshly recomputed one. That comparison includes `build.rootPath`, `dependencyRuntime.manifestPath`/`lockfilePath`, and `runtime.executablePath` (only `invokedArtifact` is stripped), and `identityCacheKey` hashes packageRoot/buildRoot — all derived from paths that flow through `realpathSync.native`, which canonicalizes 8.3 names and symlinks but does NOT normalize the Windows drive-letter case. When `analyze` and `status` are launched under different drive-letter casing (`c:\...` vs `C:\...`, plausible across CLI shim / npx / server-worker entries), the two identities differ by that one byte and `status` reports stale. Fix: `normalizeAnalyzerRootPath(p, platform)` uppercases the Windows drive letter (POSIX no-op, platform-explicit for testability; preserves a `\\?\` extended-length prefix), applied at the single upstream source — `resolveBuildRoot`'s returned `{packageRoot, buildRoot}` — so every derived identity path field and the cache key inherit a case-stable root, plus at `runtime.executablePath` (process.execPath is the same compared class). The `runnerIdentityIsCurrent` gate is kept intact: a genuine analyzer change still differs in `build.digest`/`dependencyRuntime`, and analyze still rebuilds on real mismatch. Note: the drive-letter divergence was not reproduced on a Windows host (none available); the mechanical chain is verified in source and the fix is a correct defensive normalization that is a no-op on POSIX. If a `status --json` identity field-diff later shows `build.digest`/`dependencyRuntime`/`cliVersion` diverging instead, that indicates a genuinely different install (where "stale" is correct), not this bug. Migration: on Windows, an existing index stamped under the old (non-normalized) casing mismatches the normalized recompute once, triggering a single forced full re-analyze on first upgrade (and a one-time identity-cache recompute). One-time, Windows-only, POSIX no-op. Tests: pure `normalizeAnalyzerRootPath` unit tests (drive-letter uppercase, idempotence, drive-only scope, `\\?\` extended-length prefix, POSIX no-op). * feat(mcp): surface index staleness in query/context/impact/cypher tool responses (#2655) `checkStalenessAsync` already computes how many commits an index is behind the checkout's HEAD, and `list_repos` returns it as `staleness: {commitsBehind, hint}`. But the four hot read tools an agent actually calls in a session — `query`, `context`, `impact`, `cypher` — never surfaced it: `resolveRepo` only runs `maybeWarnSiblingDrift` (stderr, sibling-clone drift only), so a direct tool call gave zero indication the index might be behind HEAD. Thread the existing signal into those four tools at the single `callTool` dispatch chokepoint (after the one `resolveRepo`), reusing the `list_repos` `{commitsBehind, hint}` shape: - `stalenessForTool` computes `checkStalenessAsync` behind an in-flight-promise cache (5s TTL) keyed by lbugPath, so N concurrent tool calls share one `git rev-list` and flat/branch handles (same repoPath, different lastCommit) don't collide. The cache entry is evicted with the repo's other per-index state when the repo leaves the registry. - `withToolStaleness` skips the `git` spawn entirely for results that can't carry the field (via `canCarryStaleness`), so error-returning calls pay nothing. - `attachToolStaleness` adds a `staleness` field to an object result only when the index is behind HEAD. It NEVER changes an existing result's shape: raw-array results (non-tabular cypher rows) are returned untouched, because the CLI's `--limit` and other consumers branch on `Array.isArray`; error envelopes and already-annotated results are left as-is. Non-blocking: `checkStalenessAsync` swallows git failures to `{isStale:false}`, so a git error just omits the field — it never fails the tool. Deliberately out of scope: `@group`-targeted calls forward to `callToolAtGroupRepo` before the chokepoint (multi-repo, single-commit staleness is ill-defined); the legacy `search`/`explore` aliases; and `list_repos` / the `context` resource, which already carry the signal. Tests: `attachToolStaleness` branch matrix (stale object -> field; fresh -> unchanged; raw array -> unchanged; error envelope -> unchanged; idempotent; non-object -> unchanged; null-safe) and a flat-vs-branch cache-key regression test that fails when the cache is keyed by repoPath. * test(mcp): cover staleness tool-signal edge cases + harden the freshness boundary (#2655) Addresses the coverage gaps the review flagged on the #2655 staleness signal, plus one defensive guard so a failing freshness check can never fail a tool. Production (defense-in-depth, no behavior change on the happy path): - withToolStaleness now awaits stalenessForTool with a `.catch(() => undefined)` so a rejection degrades to no-staleness instead of failing query/cypher/ context/impact. - stalenessForTool wraps the check in `Promise.resolve(...).catch(...)` that evicts the cache entry on rejection — a transient failure isn't served as a permanently-rejecting promise for the rest of the TTL window, and the `Promise.resolve` wrap makes the boundary robust to a non-thenable return (a no-op for the real async checkStalenessAsync). A resolving promise is never evicted, so happy-path dedup is unchanged. Tests (gitnexus/test/unit/calltool-dispatch.test.ts): - F1: a rejecting checkStalenessAsync leaves the tool payload intact with no staleness field, and a later call recovers (proves the entry isn't poisoned). Written first and confirmed to fail without the guard. - F2: staleness attaches on query/context/impact object results and on cypher's tabular {markdown,row_count}; a raw-array cypher result keeps its shape. - F3: drift guard — exactly query/cypher/context/impact route through stalenessForTool; explain/pdg_query/detect_changes/check do not. - F4: the per-index cache dedupes within TOOL_STALENESS_TTL_MS and recomputes after it expires (driven via a Date.now spy, not fake timers). Tests (gitnexus/test/unit/analyzer-identity.test.ts): - F5: the produced identity's build.rootPath and runtime.executablePath are normalizer-stable, guarding that both call sites thread through normalizeAnalyzerRootPath (trivial on POSIX, a real regression guard on Windows CI). Plus a source comment noting the one-time Windows re-analyze on first upgrade. * test(mcp): run #2668 guard on Windows CI, document staleness field, cover staleness edge cases Addresses the review follow-ups on the staleness work: - Wire test/unit/analyzer-identity.test.ts into scripts/cross-platform-tests.ts (PLATFORM_LOGIC). Its "identity path fields are normalizer-stable" fixpoint is the Windows regression guard for the #2668 drive-letter normalization, but normalizeAnalyzerRootPath is a POSIX no-op, so the guard was only ever running (trivially green) on the Ubuntu full-suite and never on the windows-latest matrix where it actually bites. Now it runs where it matters. - Document the inline `staleness` field on query/context/impact/cypher responses in the gitnexus-guide skill (both the .claude source and the shipped gitnexus-claude-plugin mirror, kept in sync). - Add three staleness tests that pin behavior the prior tests only implied: * @group-routed calls never get the signal (forwarded before the wrapping switch) — locks the intentional skip so it can't silently flip. * one in-flight freshness check is shared across truly concurrent calls (two dispatched before checkStalenessAsync settles → a single spawn), not just sequential reuse of an already-resolved value. * a late rejection from a superseded cache entry does not evict the newer entry that replaced it after the TTL rolled over (the `=== entry` object-identity guard). The defensive stack in stalenessForTool/withToolStaleness (Promise.resolve wrap + guarded evict + outer catch) is retained deliberately: the wrap is load-bearing for the tests (a sibling describe's vi.resetAllMocks() makes the mock return undefined), and the guarded evict closes the superseded-entry edge now covered above. * fix(test): split the #2668 normalization guard into a portable cross-platform file Registering analyzer-identity.test.ts on the Windows/macOS matrix (previous commit) surfaced four pre-existing failures in that file on macOS 3/3 and windows 3/3. They are not new breakage: those fixture tests compare identity fields against the RAW temp-dir path while the identity resolves through realpathSync.native, so on macOS `/var/folders/...` is received as `/private/var/folders/...`. The file was simply never portable — it had only ever run in the Ubuntu full-suite. Reproduced locally by pointing TMPDIR at a symlink: the same four tests fail, and pass again without it. Move only the portable assertions — the pure `normalizeAnalyzerRootPath` cases (explicit `platform` argument) and the identity fixpoint guard (which compares each field against ITSELF normalized, never against the fixture path) — into test/unit/analyzer-identity-path-normalization.test.ts, and register that file on the matrix instead. The #2668 Windows regression guard still runs where it actually bites, without dragging four symlink-sensitive tests onto runners they were never written for. Verified: the new file passes with TMPDIR behind a symlink (the macOS condition); the heavy file is back to Ubuntu-only. * fix(test): keep the cross-platform #2668 file fixture-free so Windows stays green The split file still carried the fixture-based fixpoint guard, which fails on windows-latest: Invoked analyzer artifact is absent from the validated build: D:\a\...\node_modules\vitest\dist\workers\forks.js Cause is a pre-existing cross-drive defect in this module's `isInside()`, not the #2668 change. The GH Windows runner keeps the repo on D: and temp fixtures on C:. `path.win32.relative('C:\\...fixture', 'D:\\...forks.js')` cannot express a relative path across drives, so it returns the absolute target — which does not start with '..', so `isInside()` reports true. `resolveInvokedArtifact` therefore treats the vitest fork worker as the invoked artifact, it is absent from the fixture's validated build, and identity resolution throws. (Verified directly: `isInside` returns true cross-drive and false for the same-drive control.) Keep the cross-platform file strictly pure — only `normalizeAnalyzerRootPath` assertions with an explicit `platform` argument, no fixture and no filesystem — so it is green on every runner while still exercising the transform on real Windows. The fixture-based threading guard moves back to analyzer-identity.test.ts (Ubuntu-only), where the rest of that file's fixture tests already live, with a comment recording why it cannot be on the matrix. The underlying `isInside()` cross-drive bug is left untouched here (out of scope for this PR) but is worth its own fix: it also guards the trusted cache directory and the identity-cache path-escape check in validateIdentityCache, where a false "inside" verdict weakens validation on multi-drive Windows setups. --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> |
||
|
Gergő Magyar
|
1e764cd475
|
fix(analyze): single-writer lock for the index write path (#2658) (#2677) | ||
|
Gergő Magyar
|
7f7255aef8
|
fix(analyze): load VECTOR before the incremental writeback touches embedding rows (#2623) (#2624)
* feat(lbug): add ensureEmbeddingRowDmlSafe VECTOR gate for embedding-row DML LadybugDB refuses every mutation of a table carrying an HNSW index while the VECTOR extension is not loaded on that connection: DELETE and CREATE raise a Binder exception, DROP TABLE is refused while the index references it, and SET segfaults the process. Dropping the index is not an available recovery either — CALL DROP_VECTOR_INDEX is itself a VECTOR-extension function and is undefined in exactly that state. Add a single primitive that loads VECTOR under the analyze install policy and, only when that fails, reads CALL SHOW_INDEXES (which works without the extension) to decide whether an index actually exists to trip over. No call sites yet. Refs #2623 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(lbug): pin the #2623 VECTOR gate for embedding-row DML Three cases: no index + VECTOR unavailable stays safe (no needless escalation); index present + VECTOR unavailable is reported blocked AND the raw deleteNodesForFiles genuinely throws 'extension is not loaded' (proving the hazard is real, not theoretical); index present + VECTOR loadable is safe, the delete works, and the HNSW index survives — the invariant run-analyze relies on when it keeps the index across a surgical incremental run. Refs #2623 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(analyze): load VECTOR before the incremental writeback touches embedding rows Incremental analyze died on every content change once a repo had built code_embedding_idx: Analysis failed: Binder exception: Trying to delete from an index on table CodeEmbedding but its extension is not loaded. The surgical writeback's first statement is deleteNodesForFiles' CodeEmbedding join-delete, but nothing on that path loaded VECTOR until Phase 4 — so the engine refused the delete. This is an ordering defect, not an environment one: it reproduces on machines where VECTOR loads fine. The dirty-flag recovery then forced a full rebuild on the next run, which is why it read as 'just slow'. Call ensureEmbeddingRowDmlSafe() once, before the escalation gate and before any row is touched — the same 'index lifecycle before row DML' seam dropSearchFTSIndexes occupies for FTS (#2589). Unconditional, because a DB carrying the index from an earlier --embeddings run hits the same wall on a plain incremental run. When VECTOR truly cannot load the table is immutable (the index cannot be dropped without the extension either), so the run falls through to the existing wipe-and-COPY escalation with a message naming cause, consequence and remedy. Fixes #2623 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(analyze): pin the #2623 VECTOR-before-embedding-DML ordering end-to-end Sibling of the #2589 FTS drop-before-delete suite, same shape: drive the real runFullAnalysis incremental path over a real git repo and a real LadybugDB, seed real embedding rows, build the HNSW index, then assert the index state at the exact moment deleteNodesForFiles is invoked. Both cases were confirmed to discriminate — with the run-analyze change reverted they fail with the reported 'Trying to delete from an index on table CodeEmbedding but its extension is not loaded', and pass with it: - surgical path: the run completes, the index is still present AND extension_loaded at delete time, exactly one row per nodeId survives, and the untouched file's rows are preserved - blocked path: with GITNEXUS_LBUG_EXTENSION_INSTALL=never the run escalates to a full DB write and says so, instead of crashing Also applies prettier's reindent to the run-analyze log ternary. Refs #2623 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(lbug): cite the pinned LadybugDB version in the #2623 probe note The probe matrix behind ensureEmbeddingRowDmlSafe was first recorded on 0.18.0, but gitnexus/package-lock.json pins 0.18.2 (#2587). Re-ran every case on 0.18.2: refused DELETE, refused CREATE, SIGSEGV on SET, DROP_VECTOR_INDEX undefined, DROP TABLE refused, SHOW_INDEXES readable with extension_loaded intact. Identical on both, so the design is unchanged — only the citation was wrong. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(analyze): preserve embeddings across the VECTOR-blocked rebuild, and check the catalog before loading Three follow-ups from reviewing the fix itself. 1. Data loss on the blocked path. Escalating wipes the DB files, and Phase 3.5 restores embedding rows from cachedEmbeddings — which deriveEmbeddingMode only populates when meta.stats.embeddings > 0. A DB holding embedding rows that its meta does not account for therefore had every vector destroyed silently by a rebuild it never asked for. Probe on a 3-file repo: 3 rows before, 0 after, no warning. Read the rows before escalating (a plain MATCH, no extension needed) so the existing restore has something to restore, and say so in the log. The blocked-path test now asserts the seeded rows survive exactly once, and that assertion fails without this rescue. 2. Catalog before extension. ensureEmbeddingRowDmlSafe loaded VECTOR first and only read SHOW_INDEXES on failure, so every incremental analyze on a machine without VECTOR paid a bounded out-of-process INSTALL attempt plus an 'extension unavailable' warning — including repos that never built an embedding index and can never hit this bug. One local catalog read settles that case first; the load is attempted only when an index actually gates DML, or when the catalog cannot be read. 3. Dead branch. targetConn is always the module singleton there, so the isSharedSingletonConn ternary could never take its second arm. Collapsed to withConnLock. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(doctor): live-probe the VECTOR extension instead of printing the static platform capability Review finding on #2624 (MEDIUM), and exactly what #2623's reporter hit: doctor printed 'VECTOR index: available' — derived from a static platform check — while every incremental analyze on the same machine was dying on an unloaded VECTOR extension. The FTS line was switched to a live LOAD probe for the identical contradiction under #2374; VECTOR now gets the same treatment. probeVectorExtensionLoad shares the FTS probe's implementation (bounded, offline-safe, never runs the installer) and doctor's semantic-mode line now follows the probe, not the platform: without a loadable extension the vector index can be neither built nor queried, so search really is on exact scan. The load-error classifier's remedies are label-parameterized so the VECTOR row stops dispensing FTS-specific advice — 'run analyze --repair-fts' repairs FTS indexes only and was actively wrong for a missing vector extension. Default label stays 'FTS'; every existing caller and pinned remedy string is unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(lbug): remove the stale Windows VECTOR gate — the extension ships for win_amd64 The codebase categorically refused VECTOR on Windows (platform !== 'win32' in isVectorExtensionSupportedByPlatform, plus a hard early-return in loadVectorExtension) on the strength of an early-era report that in-process INSTALL VECTOR could SIGSEGV (#1365). That belief is stale, verified directly: - the extension server hosts win_amd64 VECTOR artifacts for every 0.18.x extension version — v0.18.0 and v0.18.1 both serve a real 14 MB PE32+ DLL (curl-probed; 'file' confirms PE32+ x86-64) - the pinned 0.18.2 core resolves its extension directory to 0.18.1 (strace-verified LOAD open()), so the pinned version's Windows artifact exists too - INSTALL now runs in a spawned child (installDuckDbExtensionOutOfProcess), so even a crashing installer kills only the child and degrades to unavailable — the original hazard cannot reach the parent process any more Windows now takes the same runtime path as every other OS: try LOAD, install out-of-process when policy allows, degrade to exact scan when it truly fails. The MCP semantic-search lane loses its static platform gate too — it always attempts the vector index and falls back to the exact scan on runtime failure, with a once-per-backend diagnostic naming the real error instead of a platform-policy message. isVectorExtensionSupportedByPlatform is deleted; getRuntimeCapabilities reports the platform capability as available everywhere and defers machine truth to the live probe. Windows CI is the enforcement: the vector suites skip visibly only when the extension genuinely cannot load, so green Windows lanes now actually exercise VECTOR instead of silently skipping by policy. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(lbug): pin the catalog-read-failure fallback in ensureEmbeddingRowDmlSafe Review finding on #2624 (LOW): the one branch where the gate cannot cheaply prove safety — SHOW_INDEXES itself erroring — was exercised only by inference. Force it with a Connection.prototype.query spy over the real DB: the catalog read fails, and the gate must fall through to actually attempting the extension load (asserted via the recorded statement stream) rather than guessing, returning true here because the extension is loadable. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(mcp): load VECTOR on the pool's shared Database so the semantic vector lane actually works Review finding on #2624 (MEDIUM): extension load scope is per-Database (probe-verified — LOAD on one connection enables QUERY_VECTOR_INDEX on every connection of the same Database), and the pool pre-warm loaded only FTS. So LocalBackend's vector lane has ALWAYS raised 'Catalog exception: function QUERY_VECTOR_INDEX is not defined' through the pool and silently fallen back to the exact scan — repos above the 10k exact-scan cap got empty semantic results. The serve path was unaffected (the embedding pipeline loads the extension itself). Mirror the FTS line at BOTH load sites — doInitLbug's pre-warm and initLbugWithDb's external-Database adoption — under the same load-only contract (the read pool never triggers a network install), tracked by a new SharedDB.vectorLoaded flag reset where ftsLoaded resets. The new pool test is discriminating and deliberately closes the writable core adapter before the pool opens: a shared/injected Database would inherit the VECTOR load from test seeding and pass either way, so the case forces the pool onto its OWN fresh read-only Database where only the pre-warm can make the lane legal. Verified: fails at the pre-fix tree with the exact Catalog exception, passes with the fix. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * ci: run the #2623 ordering suite on Windows/macOS and pre-install VECTOR alongside FTS Two review findings on #2624, both landing in existing seams: - scripts/cross-platform-tests.ts gains incremental-vector-extension-ordering .test.ts: the win32 VECTOR gate is gone in this PR, so the #2623 drop-ordering + blocked-path escalation must be proven on the windows-latest native addon, not just Ubuntu. (The review's claim that lbug-delete-nodes-for-files.test.ts was also missing was wrong — it has been on the roster since #2409.) - scripts/ensure-fts.ts now pre-installs VECTOR under the same best-effort auto-policy contract, so every sharded CI process LOADs from ~/.lbdb instead of racing its own bounded out-of-process INSTALL; the workflow's extension cache already covers it (path is the whole extension dir — key kept for cache continuity). The cross-platform job sets GITNEXUS_REQUIRE_VECTOR=1 beside GITNEXUS_REQUIRE_FTS so a genuinely unavailable VECTOR is a loud failure, never a silent skip. Windows/macOS cannot be executed locally; the PR's CI lanes are the proof for this commit. Linux smoke: ensure-fts.ts reports both extensions ready; all 79 roster entries resolve. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(pool): register loadVectorExtension in the pool unit-suite mocks The pool adapter's new loadVectorExtension import surfaced in four suites that mock lbug-adapter.js with explicit factories (vitest fails loudly on a missing mocked export). Register the export in each — resolving false where the suite's world assumes no vector, true where it mirrors FTS — and extend lbug-pool-fts-load.test.ts, the suite that owns pre-warm extension loading, with the vector pair: successful load cached per shared Database, failed load retried on the next open, both pinned to policy load-only. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(analyze): use POSIX literals for graph paths in the #2623 ordering suite First Windows CI run of this suite (it joined the cross-platform roster this PR) failed with 'Parser exception: Invalid input <MATCH (n:Function) WHERE n.filePath = '>' — path.join produces backslashes on Windows, and a backslash inside the seed helper's single-quoted Cypher literal breaks the parser. The graph stores repo-relative filePaths with forward slashes on every OS, so graph-side paths are POSIX literals now (the incremental-orchestration convention); path.join stays only for real filesystem access. The same Windows lane also proved the substance this suite exists for: lbug-vector-extension passed 7/7 on windows-latest — the extension installed, loaded, and built a real HNSW index there — and the pool vector-lane and DML gate suites passed too. This commit fixes the harness, not the fix. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Gergo Magyar <abhigyan1.patwari@gmail.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Gergő Magyar
|
8b5057f325
|
feat(skills): GitNexus Engineering Tool Kits (#2566)
* feat(skills): add ce-plan — GitNexus+PDG implementation-planning skill Adds .claude/skills/ce-plan: a planning-only skill that builds implementation-ready plans from GitNexus graph navigation (query/context/ impact/trace), bounded statement-level PDG slices (pdg_query, impact mode:pdg, explain), and targeted source verification, with a context ledger to prevent repeated reads and a machine-readable implementation context pack (stable contract for a future ce-implement). Whitelisted in .gitignore and registered in AGENTS.md and CLAUDE.md outside the auto-managed gitnexus block. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(skills): apply ce-plan validation findings (tool contract, consistency, conventions) Tool contract: impact mode:'pdg' shape now includes the schema-required direction param; CDG branch sense documented as the result 'label' field (reason is cypher/raw-edge only); explain caveats corrected to its real false-negative classes (cross-function TAINT_PATH is modeled). Consistency: PDG slice homed in working memory (ledger keeps one-liners); depth knob defined and category-overrides-baseline ordering stated; call_depth (consumed by nothing) and content-hash bookkeeping dropped; Never section folded into Hard rules; Phase 3 deduplicated to a pointer; allowed-repeat escalations defined; budget/discard accounting clarified; verification-commands gathering added to Phase 4; open_questions added to the context pack. From scenario runs: plans now pin the verified-at HEAD commit and index freshness in a header, tag claims [verified]/[graph]/[inferred]/[assumed], quote load-bearing tool output, prefer pre-hook-carrying npm scripts, and support an out:<path> destination override; output path defined as the Phase 1 target repo root. Conventions: AGENTS.md 1.9.0 / CLAUDE.md 1.4.0 changelog rows + metadata bumps; future ce-implement qualified as future. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(skills): rename ce-plan → gitnexus-plan; add cross-CLI (Codex) entrypoints Renames the skill dir, frontmatter, output filename convention, plan H1 (GitNexus Engineering Plan), the future executor handle (gitnexus-implement), the .gitignore whitelist entry, and all AGENTS.md/CLAUDE.md references. Follows the pr-swarm-review cross-CLI pattern: SKILL.md is the canonical CLI-neutral spec, AGENTS.md § Engineering planning is the Codex/any-agent entrypoint, and the README documents the optional user-level ~/.codex/prompts/gitnexus-plan.md slash command plus an invocation matrix. Skill prose de-branded from Claude Code (agent-neutral verification layer). Also fixes two post-review README contradictions: the anti-reread claim now names the ledger's allowed escalations, and 'read-only by contract' is now 'planning-only' (the skill writes exactly one repo file — the plan); the scope-creep rule and template §12 now agree on where deferred follow-ups land. Drops the stale plugin-collision limitation. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(skills): document Codex user-level install path for gitnexus-plan Codex discovers SKILL.md skills from ~/.agents/skills (same path the other gitnexus-* skills install to); README now documents the cp install plus the optional ~/.codex/prompts slash-command file, with the prompt body preferring the repo copy and falling back to the user-level install. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(skills): gitnexus-plan freshness gate + active PDG-layer refresh Freshness is now a Phase 1 gate, not advisory: under the default freshness:strict, a stale index is refreshed once per planning session via node .gitnexus/run.cjs analyze --index-only (appending --pdg when the task will reach the PDG phase), then the context resource is re-read. A missing PDG layer likewise triggers the one permitted --index-only --pdg refresh and re-probe instead of a passive recommendation. freshness:accept (or a failed/impractical refresh) preserves the old behavior: plan on the stale graph, source-weighted, labelled in the plan header. --index-only is the load-bearing flag choice — it suppresses all file generation, so the planning-only contract holds (only the .gitnexus store changes). Ledger gains an index_refresh record; plan header states fresh / refreshed / refresh-skipped-with-reason. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(skills): gitnexus-plan runner build check before freshness refresh When the target repo builds the analyzer from its own source (bin → dist/ mapping, as gitnexus/ does), the Phase 1 freshness gate now verifies dist/ is current before running the analyze refresh — rebuilding via the package's build script when any analyzer source file is newer than the built entrypoint — and prefers that freshly built CLI. Otherwise a stale dist re-indexes with outdated extraction logic and the 'fresh' index lies. Rebuilds are recorded in the ledger's index_refresh; the PDG-phase refresh inherits the same check. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(skills): add gitnexus-work executor and gitnexus-lfg pipeline gitnexus-work executes a gitnexus-plan as verified atomic commits: consumes the §11 implementation_context pack, drift-checks the plan's evidence pin against HEAD, re-verifies assumptions before relying on them, runs impact before every symbol edit and detect_changes before every commit (repo mandates), builds tests from the plan's scenarios, and routes structural drift back to gitnexus-plan Deepen mode instead of coding around it. gitnexus-lfg is a thin orchestrator: gitnexus-plan → blocking user gate (deepen / proceed / stop, deepen loops allowed) → gitnexus-work → review via the existing gitnexus-pr-review skill (open PR, else branch diff vs default). One bounded fix cycle for review findings; never pushes or opens a PR on its own. gitnexus-plan gains a Deepen mode (re-run freshness gate, escalate to depth:deep, re-verify graph/inferred/assumed claims toward verified, rewrite the same file); its 'future gitnexus-implement' placeholder is retired in favor of gitnexus-work. Registered via .gitignore whitelists, AGENTS.md 1.10.0 (section renamed to Engineering planning & execution), CLAUDE.md 1.5.0. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(skills): apply cross-skill review findings to the gitnexus skill family Two P1s: gitnexus-plan Deepen mode now re-anchors before re-pinning (diffs the old evidence pin over every [verified]-claim file and re-reads or downgrades before the header moves — moving the pin without this laundered stale claims as verified); the index-refresh budget is stated once in Phase 1 (one --index-only refresh plus at most one Phase 3 --pdg upgrade per session, Deepen = its own session) with ledger and pdg-slice deferring to it. Contract fixes: gitnexus-work's drift check now covers every file the pack cites (not just files_to_modify) and parses the full pack incl. primary/related symbols and acceptance_criteria (walked in Phase 4 alongside §13); a pre-completed check skips §7 steps already landed and Deepen gains a reconcile-execution-state step, closing the mid-execution route-back loop; pack assumptions must name what to check and how. lfg: Lane 4 passes the merge-base to detect_changes compare (two-dot diff misattributes upstream commits when default advanced), branch-diff is the stated normal case, oversized review findings route to the plan gate instead of overflowing direct mode, the one-fix-cycle cap is explicit on re-run, and headless runs end at the plan gate with the plan as deliverable. work: blank mode narrowed to *gitnexus-plan*.md with a re-execution guard, direct-mode discipline spelled out, branch meaningfulness defined against the plan slug, and the plan document is committed as the branch's docs commit (review diff includes it). Planning-only contract now names the dist/ rebuild as the second permitted state change; Phase 5.1 names the four claim tags; stale AGENTS.md anchors fixed. Known latent issue left untouched: gitnexus/gitnexus-pr-review pairs a three-dot example with a two-dot detect_changes compare — that skill is also shipped by the plugin, so fixing it here would drift the copies; lfg compensates by passing the merge-base. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(skills): ship the engineering skill family with the gitnexus package npm i -g gitnexus users now get gitnexus-plan / gitnexus-work / gitnexus-lfg: the three skills are added to gitnexus/skills/ in directory form (SKILL.md + references/), which installSkillsTo already enumerates dynamically and copies recursively to every editor target (~/.agents/skills for Codex, Cursor, OpenCode, Qoder, ...) on gitnexus setup — uninstall enumerates the same root, so removal stays clean. The Claude Code plugin channel (gitnexus-claude-plugin/skills/) carries the same copies plus the standard per-skill mcp.json. Global-install support in the skill text: gitnexus-plan Phase 1 now resolves the analyzer runner explicitly — node .gitnexus/run.cjs analyze when the project has a runner, else gitnexus analyze (installed CLI), else npx gitnexus analyze — and all analyze mentions route through it, satisfying the skills-steering policy (#1939/#1945) which sweeps the plugin copies. New drift guard test/unit/shipped-skills-sync.test.ts asserts the npm and plugin copies stay byte-identical to the canonical .claude/skills/ family (plugin = canonical + mcp.json), same discipline as run.cjs ↔ resolve-invocation.ts. skills-steering + shipped-skills-sync: 11/11 green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(eval): workflow_bench — measure the skill workflow's token savings Benchmarks gitnexus-plan → gitnexus-work against a baseline agent (--disallowedTools Skill) on identical tasks, in fresh detached worktrees, using real headless Claude Code sessions; every number comes from the CLI's --output-format json usage report (field names validated against a live 2.1.207 session). Reports per-arm medians (input/cache/output tokens, cost, wall time, turns), a savings row, and resolve status from a per-task verify command — savings on failed tasks are flagged, not celebrated. Per-task setup hook prepares fresh worktrees (deps); --permission-mode bypassPermissions (default) lets sessions run unattended in the throwaway trees. Free-model support: --base-url/--auth-token/--model route headless sessions through any Anthropic-compatible endpoint; free-model.litellm.yaml is a ready litellm-proxy template for OpenRouter :free variants or local Ollama, so benchmarking burns no paid tokens (README documents rate limits and the small-model skill-following caveat). Harness validated end-to-end with a stub CLI (worktree lifecycle, both arms, plan→work chaining, verify, aggregation, report) and 4 pytest units for the pure aggregation/savings/report helpers. AGENTS.md 1.11.0. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(eval): record first workflow_bench calibration run Trivial-task calibration (add -V alias): both arms resolved; workflow arm ~4.3x baseline cost — the documented overhead-dominated regime, recorded so the regime boundary is empirical rather than asserted. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(eval): workflow_bench scenario matrix — arm variants, task classes, churn Ground-base measurement across scenarios: tasks.scenarios.yaml spans four labeled classes (trivial → investigation-bug → investigation-feature → cross-module) with deterministic verifies (prescribed test files). New arms: workflow_direct (gitnexus-work direct mode — the middle option that locates the routing boundary lfg's gate and work's triage encode) and baseline_nomcp (no skills AND no graph tools — separates workflow-discipline value from GitNexus-tool value; off by default). Records now carry task class and diff churn (files/+ins/−del vs the starting commit) as an over-engineering proxy; the report renders a class column and per-arm savings rows vs baseline. 5 pytest units + stub-CLI e2e of the full three-arm matrix. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(eval): record workflow_bench ground base; fix churn measurement bias Ground base (3 classes x 3 arms, n=1/cell): every arm resolved every task — pass/fail quality saturates at this difficulty, making the comparison pure cost. Full plan→work never amortized its ~$9-11 fixed cost on tasks a baseline finishes in ≤35 turns (−211% to −333% cost); workflow_direct sits near baseline (−15% to −55%, once faster wall) with more test coverage. Routing implication recorded: direct mode/plain agent below this scale, full workflow for cross-module / multi-session / plan-as-deliverable work. The cross-module cell and multi-run variance are the next measurements. Churn fix: git add --intent-to-add -A before diffing (arms that never commit no longer undercount new files) and :(exclude)docs/plans (the committed plan doc no longer inflates workflow churn); this run's churn numbers predate the fix and are omitted from the recorded table. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * perf(skills): cost-optimize the workflow from measured ground base Every optimization targets a measured fixed-cost component (eval/workflow_bench ground base: workflow arm −211% to −333% vs baseline, all tasks resolved): - Plan form is category-priced: compact form (core sections w/ § anchors preserved, ≤80 lines excl. pack, mini-pack subset of the context pack) for narrow/default categories; the full 13 sections only for deep work (refactor/security/performance/concurrency/architecture). A compact plan outgrowing its cap reclassifies to full rather than overflowing. - Freshness gate is category-priced: compact categories default to accept (source-weighted, refresh only when a graph claim becomes load-bearing); strict stays the default for full-plan categories — the rebuild+re-index was the largest single fixed cost. - Turn economy: per-category tool-call budgets (~10 to ~45; architecture uncapped); budget exhaustion routes open questions to §12 instead of more digging. - gitnexus-work fast path: HEAD == evidence pin → skip all citation re-reading (the pin's entire point); mini-pack fields tolerated. - lfg Lane 1 boundary triage: tasks below the measured ~35-turn boundary get offered gitnexus-work direct mode before the plan lane is spent. Copies re-synced (npm skills/, plugin, ~/.agents); steering + sync guards green. Re-measurement of the workflow arm follows to verify the numbers actually improve. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(eval): record optimization re-measurement — inv-bug workflow cell −20% cost Same task, same conditions, post-830a0459 skills: $14.56→$11.70 (−20%), 83→72 turns, cache_read −24%; verified in-transcript that the compact form, turn budget, and skipped rebuild/re-index all fired. Wall +15% from a work- session test-debugging tail (n=1 variance). Regime unchanged (~3.5x baseline on this class) — routing rule stands. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(eval): per-arm clone isolation — worktree ref-namespace leak contaminated an arm The cross-module workflow_direct cell reported an impossible 28-turn solve with churn byte-identical to the workflow arm: git worktree add shares the repo's ref namespace, so the workflow arm's slug branch (created by gitnexus-work Phase 2) survived worktree removal and the direct arm found and adopted the completed work. Arms now get isolated git clone --shared copies (object store via alternates, refs clone-local — agent branches and stashes die with the clone; origin/<ref> fallback for non-default refs). Leaked branch deleted; baseline arm verified clean (0 branch references in its transcript); cell marked invalidated pending re-run. Records the valid cross-module cells: workflow $18.32 vs baseline $18.03 (premium −1.6%, vs −211%..−333% on smaller classes) — fixed costs amortize at this scale, with a less destructive diff and a plan artifact as bonus; resolve rate still tied. Churn fingerprinting is what caught the contamination — noted in the README as an integrity check. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(eval): complete cross-module cell — direct mode wins 47% cost / 56% wall Clean clone-isolated re-run: workflow_direct resolved the hardest class at $9.53/52 turns/15m vs $18.03/98/34m baseline and $18.32/107/37m full workflow. The measured story across all four classes: the execution discipline (gitnexus-work) is the consistent sweet spot and delivers real token savings on hard tasks; the planning pass buys its artifact, not same-session savings. Resolve rate tied everywhere (n=1/cell caveat). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(eval): add trajectory-gated skill evolution (#2431) - Pair prompt candidates with incumbent workflow arms - Gate promotions on pinned-model quality and efficiency - Expire router evidence and document its lifecycle * fix(eval): allow pr-review skill candidates * feat(skills): rename and generalize GitNexus review * feat(eval): external-comparator and review arms for workflow_bench - ce_workflow / ce_workflow_direct: compound-engineering ce-plan/ce-work arms prompted with the same structure as the gitnexus arms - review / ce_review: gitnexus-review vs ce-code-review on an identical diff applied by the task's setup - plan handoff is snapshot-based: committed example plans in docs/plans/ tie on clone mtimes and broke the name-glob pick (executed a stale plan) - verify output tail is recorded per run and the final working-tree patch is kept, so failed rows are diagnosable after the clone is destroyed Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(skills,eval): address #2431 review — data-safe rename migration, fail-closed bench evidence - setup: never delete a legacy renamed skill dir — the installer cannot prove ownership (users customize or hand-write skills under these names); warn with the path instead, and the test now asserts survival - workflow_bench: fail closed when a session's --output-format json report is empty, malformed, or missing usage fields — an exit-0 shell with no parseable usage no longer counts as measured evidence (5 parametrized regression tests) - workflow_bench: document the trust model prominently (task setup/verify are shell-executed, sessions run bypassPermissions with the parent env, candidate overlays are prompt injection surface) in README + docstring - free-model.litellm.yaml: master_key from LITELLM_MASTER_KEY env instead of a static token; loopback-binding warning - ci: run the eval workflow_bench pytest suite on ubuntu (pytest+pyyaml only — no full eval stack) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(eval): demand observed foreground verification in headless work-arm prompts In a headless -p session there is no later turn: a work arm backgrounded its slow test run, scheduled wakeups that can never fire, and reported done while two of its tests failed. All four work-arm prompts (both skill families, symmetric) now require verification output to be observed inside the session. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(skills): ask plan depth up front instead of offering deepen afterwards gitnexus-plan Phase 0 now asks one blocking question in interactive sessions — quick / standard / deep, mapped onto the existing depth/form/ freshness knobs — when the invocation carries no explicit depth signal. Explicit knobs and headless runs skip the question (category posture unchanged, so benchmarks and automation behave as before). gitnexus-lfg's plan gate slims to proceed/stop: depth was already the user's up-front choice, so deepening is no longer offered by default — an explicit deepen request at the gate and executor route-backs still run Deepen mode, which remains the mechanism for strengthening an existing plan document. All shipped copies resynced (npm skills/, Claude plugin); AGENTS.md 1.13.0 and CLAUDE.md 1.7.0 pointers updated, including the analyzer's regenerated index-stats block at this branch's head. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(skills): taint pass, expert lenses, and post-work index refresh gitnexus-review gains a PDG-backed taint-and-dependence pass (explain + pdg_query, --pdg folded into the stale refresh on trust-boundary diffs) and an Expert lenses section: domain reviewers derived from the graph's clusters plus four cross-cutting lenses (architectural fit, language conformance per the repo's own contract, Definition of Done, simplicity), dispatched once after the evidence-gathering steps and scaled to the diff. gitnexus-work Phase 4 now refreshes the knowledge graph after the DoD walk via the resolved-runner ladder with analyze --index-only, so the lfg review lane and later sessions query the finished work without dirtying the tree. lfg's threshold-governance paragraph moves to its README; eval citations are tagged as measured in the GitNexus repo. All shipped copies re-synced. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(cli): remove legacy gitnexus-pr-review on uninstall; cover the rename migration uninstall's removal set now includes LEGACY_SKILL_DIR_NAMES derived from RENAMED_SKILL_DIRS, so a pre-rename install is cleaned up instead of orphaned. The rename warning gains behavioral coverage (fires with a legacy dir present, silent without), and shipped-skills-sync asserts legacy names stay absent from every shipped tree. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(eval): metric provenance, error-kind rows, skill-invocation verification, gate noise floor The promotion gate defaults to cost_usd (the only metric that includes subagent spend); token metrics carry an explicit main-loop-only warning in the report and promotion.json. Rows are classified by error_kind (session-error / verify-failed / infra-error), excluded from efficiency medians, and the gate requires equal valid-run counts. Each session's transcript is scanned for the expected Skill invocation and fails closed on a verified miss; a one-run resolution edge no longer promotes (noise floor). Per-run timeouts and setup failures record an infra-error row instead of aborting the sweep. Overlays touching skills no candidate arm exercises are rejected up front. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: fix skill routing paths, version headers, and skill rosters Routing tables point at the tracked direct skill paths (matching the post-#2434 generator output), AGENTS.md/CLAUDE.md headers match their latest changelog rows, the 1.12.0 row describes what the migration actually does, package/cursor READMEs list the full shipped skill roster, and the swarm READMEs describe /gitnexus-review's expert lenses instead of calling it single-agent. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * ci: drift-guard workflow for skill copies; pin eval pip deps; track docs/plans ci.yml ignores '**.md', so an md-only skill edit would merge without the shipped-skills-sync test running — skill-sync.yml triggers exactly on the guarded trees. The eval job's pip install is version-pinned, and docs/plans/ is unignored so gitnexus-plan output can be committed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ci): keep the runner-invocation literal in gitnexus-review; add concurrency block to skill-sync skills-steering requires skills with a stale-index hint to carry the exact 'node .gitnexus/run.cjs analyze' form — restore it with the fallback ladder as a parenthetical instead of replacing it. skill-sync.yml gains the top-level concurrency block the workflow-convention check enforces. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(skills): token-economy guidance for expert lenses Merge lenses that ground in the same material into one reviewer, and use cheaper model/effort tiers for mechanical lenses where the harness offers them, reserving the strongest engine for adversarial judgment. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(eval): isolate transcript home on Windows Ensure workflow_bench transcript tests set USERPROFILE alongside HOME so Path.home() resolves to the temporary test home on Windows. * docs(skills): fold PR #2522 execution learnings into review/work/plan Eight incident-backed hardenings from running the full skill cycle (review -> plan -> work, 28-finding fix series) on PR #2522: gitnexus-review: - Expert lenses execute the code under review on candidate failing shapes (empirical probe outranks source reading — every HIGH the language lenses found came from a probe, not a read). - Step 7 re-runs the exact CI check for refreshed baselines/fingerprints (a stale committed artifact is invisible in the diff; caught a red benchmarks arm). - Step 8 treats version/invalidation constants as review surface (INCREMENTAL_SCHEMA_VERSION class recurred verbatim from #2494). gitnexus-work: - Step 4 proves regression tests discriminate against the pre-fix tree. - Step 5 rebuilds executed build output before every verification run (parse workers load dist/; a correct fix 'failed' until rebuilt). - Step 6 makes stage -> detect_changes -> commit one unbroken sequence. gitnexus-plan: - Phase 0 seeded-evidence mode: plan FROM a completed review's verified findings instead of re-running the graph ladder. - Template §7: fingerprint/golden-guarded output rebaselines once, at the series tip. All distribution copies resynced; shipped-skills-sync + skills-steering 24/24 locally. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(eval): close the skill-evolution loop with an automated proposer driver workflow_bench.evolve adds the three arrows the README described as manual: a proposer session that turns loser trajectories (results.jsonl rows, transcripts, patches, the learning queue) into ONE bounded candidate overlay, a driver that iterates propose -> paired benchmark -> deterministic gate up to --generations, and an --apply step that copies a promoted overlay onto the canonical skills and shipped mirrors as a working-tree diff. The trust boundary is unchanged: overlays re-validate through candidate_overlay_files before any benchmark or apply consumes them, and committing, CI, and the PR merge stay human. learnings.jsonl is gitignored: it is machine-local evidence, like the session transcripts it complements. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(skills): route live-task friction into the evolution learning queue Each family skill gains a short 'Skill feedback' section: on friction with the skill's own instructions, append one JSON line to eval/workflow_bench/learnings.jsonl (GitNexus repo only) — never self-edit the skill from a live task. The proposer in workflow_bench.evolve consumes the queue as hints; a learning reaches a shipped skill only by beating the incumbent on the paired benchmark. All shipped mirrors re-copied byte- identical. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * ci(tests): run the evolve helper tests in the eval pytest job test_evolve.py needs only pytest+pyyaml, same as the harness tests the job already runs — without this line the new module had no CI coverage. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(ci): comment-triggered GitNexus review agent for PRs '@gitnexus review' from a maintainer (OWNER/MEMBER/COLLABORATOR; the action re-validates write access) runs the repo's gitnexus-review skill headlessly against the PR and posts the review as a sticky comment — remote triggering with no local setup. Read-only by construction: contents: read token, Write/Edit and web tools disallowed, Bash allowlisted to git reads and the gitnexus CLI; analyze parses PR code with tree-sitter, never executes it. Requires the ANTHROPIC_API_KEY repository secret; activates once the file is on the default branch. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(ci): dispatch lane + existing OAuth secret for the review agent Align with claude.yml: same action pin and the CLAUDE_CODE_OAUTH_TOKEN secret the repo already carries — no new secret to configure. Add a workflow_dispatch lane (PR number input) so the agent can be triggered from the Actions UI and tested before the issue_comment trigger reaches the default branch. Allowlist gh pr view/diff and gh api, which the review skill uses to pin PR SHAs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ci): close a fork-PR RCE vector in the review agent's tool allowlist A live headless run of the exact workflow session against PR #2431 (66 turns, full gitnexus-review pass) surfaced a real HIGH-severity confused deputy: .gitnexus/ is gitignored, not blocked — a fork PR can commit its own .gitnexus/run.cjs, issue_comment checks out PR-head content, and the skill's runner ladder tries 'node .gitnexus/run.cjs analyze' first. That would execute fork-controlled JS inside a job holding CLAUDE_CODE_OAUTH_TOKEN and a write-scoped GITHUB_TOKEN — the opposite of the 'PR code is read, never executed' claim in the workflow's own header. Fix: drop the run.cjs allowlist entry so analyze always resolves through npx gitnexus (npm registry, not the checked-out tree); the skill's documented fallback mode covers the resulting graceful degradation. Also drop 'gh api' (not read-only — accepts -X POST/PATCH/DELETE) and downgrade pull-requests: write to read (comment posting only needs issues: write; the prompt already forbids formal review submission). Same session flagged a latent evolve.py bug: select_evidence's cost sort used dict.get's missing-key default, which doesn't cover an explicit JSON null in a foreign --seed-results row and crashes proposer setup with TypeError. Guarded with 'or 0.0' and added a regression test. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: harden PR review and evolution trust boundaries * ci: follow workflow concurrency convention * fix(eval): make terminating error paths explicit * fix: unblock hardened review runtime checks * test: make containment canaries deterministic * test: expose Claude canary tool failures * fix: adapt clean shell environment for Claude * fix(eval): accept the runner's transcript source key in evidence preflight The proposer evidence preflight required transcript-artifact metadata to be exactly {path, sha256, bytes}, but the runner stamps a fourth provenance key (source=parent-captured-stream-json). Any --seed-results or generation>=2 run therefore aborted with SandboxError before proposing or promoting. Pin the producer literal as PARENT_EVENT_STREAM_SOURCE and validate it in the metadata check, and round-trip real producer output through sum_sessions into the preflight so the schema can't drift again. * fix(eval): treat an unmeasured session cost as unavailable, not $0 well_formed validated only the nested usage block, so an otherwise-successful session missing total_cost_usd was recorded as cost_usd=0.0 — and cost_usd is the default promotion metric (lower wins), so a cost-less session scored as free and could win promotion it never earned. Extract cost via measured_cost() (None on absent/garbage, a measured 0.0 preserved), propagate None through sum_sessions/aggregate/savings/report, and have the gate refuse to rank on a metric that was not measured on every run in both arms. * fix(eval): warn when ranking on the main-loop-only num_turns metric num_turns comes from the CLI's top-level usage (main-loop session only), like output_tokens, but selecting it emitted no metric_warning — so a subagent-heavy candidate could look artificially efficient. Add num_turns to MAIN_LOOP_ONLY_METRICS and broaden the warning to cover turns. * fix(eval): fail closed when an overlay adds a file with no committed base An overlay adding a new .md under gitnexus-{plan,work} passes the structural overlay checks but has no committed base for committed_destination_base_digests to bind against, so it raised an uncaught ValueError that crashed the evolve driver (and runner --candidate-overlay) mid-run. Catch it at both call sites: evolve reports NOT PROMOTED and exits, runner routes it through parser.error. * feat(eval): circuit-break the runner sweep on a systemic outage A sustained upstream outage used to pay out every remaining --timeout window one session at a time. Track consecutive session/infra/cleanup failures via a pure systemic_outage_streak helper; after --outage-streak (default 5) in a row, stop the sweep, still write report.md/promotion.json from partial evidence, and exit non-zero so evolve.py halts instead of proposing from truncated evidence. A task's own resolved=False never trips the breaker. * fix(cli): report a dirty working tree as stale in gitnexus status status --json (and the human output) computed up-to-date from commit + runner identity + completeness only, so a repo with uncommitted source changes at a matching HEAD was reported up-to-date while analyze would still re-index it. A graph-backed agent gating on that JSON could skip re-analysis on a stale graph. Extract analyze's dirty-tree check into a shared isWorkingTreeDirty() in storage/git and fold it into the status freshness decision. * fix(ci): use single-slash deny globs in the review agent's disallowedTools github.workspace already expands to an absolute path, so Read(/${{ github.workspace }}/**) and Read(//proc/**),(//sys/**),(//dev/**) produced double-slash patterns that a normalizing matcher may not match — silently no-opping the deny layer. Not exploitable (the allowlist is the primary control and never grants those paths), but the globs should be well-formed. Update the pinned test strings. * ci: install gitnexus-shared with npm ci from the committed lockfile The gitnexus-shared build floated its deps via npm install in three workflows (skill-sync, ci-tests, and — most importantly — the release publish.yml) while every other install step uses npm ci. The lockfile is committed and in sync, so switch all three to npm ci for reproducible, locked installs. * test(cli): make the shipped-skills drift guard reject symlinks listFilesRecursive walked with readdirSync and snapshotDir read with readFileSync, both of which follow symlinks — so a mirror file symlinked to the canonical tree passed the byte-compare (and a symlinked mirror dir would be followed too). Reject a symlinked root via lstat and any symlinked entry via Dirent.isSymbolicLink, with negative tests (skipped on Windows). * test(eval): guard the candidate-skill vs mirror-root coverage invariant MIRROR_SKILL_ROOTS omits the Cursor tree, safe only because no candidate skill is cursor-shipped. Pin that invariant: every CANDIDATE_SKILLS entry must exist under canonical + every mirror root and must not ship to Cursor, so adding a cursor-shipped skill to the candidate set (the PR #2488 asymmetric-sync class) fails loudly instead of syncing three of four trees. * docs(ci): describe the review agent's staged post-merge rollout The DoD asked for a dry-run or triggered run before merge, but an issue_comment (or newly added workflow_dispatch) workflow only ever executes the default-branch copy, so it cannot be exercised from the PR that introduces it. Reword the DoD and the activation checklist to a staged rollout: merge registered-but-disabled, validate same-repo and fork execution post-merge, then enable the variable. * fix: pin plugin skill mcp.json to the release version via #2445 tooling The ten plugin skill mcp.json launched `npx -y gitnexus@latest mcp` on every skill connect — non-reproducible and a supply-chain surface, and (unlike the persisted setup config) never pinned. Extend sync-plugin-manifests.mjs with an mcp surface kind that stamps the gitnexus@<version> launch arg, pin all ten to 1.6.9 now, and keep them byte-identical so the drift guard stays green. The release lifecycle + publish.yml --check now re-stamp them like the four manifest surfaces; only READMEs stay on @latest as docs. * test(eval): prove the proposer's built-in file tools are confined The real-Claude canary only exercised Bash + MCP, so it proved process/MCP containment but not that the proposer's built-in file tools stay inside their mounts. Add a canary over the exact PROPOSER_ALLOWED_TOOLS surface and the same read-only /evidence mount as run_proposer (allowlist extracted to a shared constant so it can't drift): Read reaches /evidence, a Write into the read-only evidence mount is denied, and a Write lands in the output tree. * fix(eval): apply the candidate overlay after task setup for fair arms The candidate overlay was applied before the task's untrusted setup ran, so setup could observe candidate prose and the incumbent/candidate arms started from different pre-overlay state. Reorder within the sandbox: capture the base (pre-overlay) skill digest, run setup against the base skills, verify setup did not tamper them, then apply the overlay and capture the post-overlay digest the model must preserve. apply_candidate_overlay stages path-specific overlay files, so setup's uncommitted changes stay out of the baseline and churn is unchanged. Graph freshness for the review arm is handled by the status dirty-tree fix plus the review skill's stale-triggered re-index, not by reordering the cached per-task-sha graph materialization (which is mechanically blocked). * test(eval): end-to-end containment proof of the autonomous proposer Drives the real run_proposer through bubblewrap with a deterministic scripted model (no paid API): it reads the read-only evidence bundle and writes a candidate gitnexus-plan skill edit plus a rationale into the sandbox output tree; run_proposer enforces the trust boundary and copies only the validated overlay + proposal out. This exercises the autonomous-proposal stage of the self-evolution loop end-to-end in the eval/containment CI job (the gate and apply stages are covered by test_workflow_bench_evolution and test_promotion_apply). Env-gated on GITNEXUS_REQUIRE_CLAUDE_CANARY, so it runs only where the pinned Claude binary and user namespaces are available. * fix(eval): let the proposer author its overlay via Bash Running the end-to-end proposer canary in the containment CI job surfaced a real bug: run_proposer starts the session with --bare, which hard-disables the Write/Edit tools ("Write exists but is not enabled in this context"), yet allowlisted Edit/Write and omitted Bash. The proposer therefore had no working way to write its candidate overlay — the self-evolution loop could never produce a candidate. The sandbox settings already pre-authorize Bash (autoAllowBashIfSandboxed) and confine writes to workspace/tmp/home, so switch PROPOSER_ALLOWED_TOOLS to Read/Grep/Glob/Bash and tell the proposer to author files with Bash. The end-to-end test now drives the real run_proposer through bubblewrap and asserts a validated overlay + proposal are produced (this also replaces the earlier file-tool canary, whose Write/Edit premise was moot). * test(eval): author the proposer overlay with newline-free Bash content The nested shell-sandbox prefix mangles embedded newlines, so the multi-line overlay content never landed. Use single-line content for the deterministic proposer canary. * test(eval): drop the unverifiable end-to-end proposer canary The scripted proposer overlay never materialized in the containment job across runs, and the model tool-result content is not visible in CI logs, so the test cannot be finalized without an environment where the sandbox can actually run. Keep the verified production fix (Bash-authoring in run_proposer); the proposer sandbox/containment stays covered by the existing Bash+MCP and process-tree canaries. * test(cli): drop run-analyze.ts from the windowsHide spawn-family list U7 moved run-analyze.ts's only child_process call (the git status --porcelain dirty check) into storage/git.ts (already covered by this test, with windowsHide). run-analyze.ts no longer imports a spawn-family function, so the windowsHide-regression test's 'must have >=1 spawn call' invariant failed for it. Remove it from SRC_FILES. --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Zander Raycraft <zanderjraycraft@gmail.com> Co-authored-by: Azizur Rahman <azizur100389@gmail.com> |
||
|
Gergo Magyar
|
573a777ef5 |
ci(tests): widen the Windows shard watchdog and keep exit diagnostics (#2449)
The busiest Windows platform shard reached 14m57s against the 15 minute watchdog on the rc.19 green run and has timed out once since. CI now sets GITNEXUS_CROSS_PLATFORM_TIMEOUT_MINUTES=20 (the job timeout stays 25), the stale comfortably-under comment reflects reality, and the runner always logs status, signal, spawn code and elapsed time so the next status-null death is diagnosable. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Gergo Magyar
|
42de243e9a |
ci(release): sync plugin manifests on every version bump (#2445)
The RC path bumped only gitnexus/package.json, so every v1.6.10-rc tag through rc.28 shipped the four plugin manifest surfaces frozen at 1.6.9 and failed its own unit suite. The npm version lifecycle script now runs a fail-closed sync whenever npm version executes, in CI or on a maintainer's laptop; publish.yml verifies the result and stages the surfaces into the detached release commit, and the stable path refuses to publish a tag whose manifests drifted. The sync is textual so a release commit carries a one-line change per surface instead of reformatting churn. Design follows the proposal by @100yenadmin in #2445, moved onto the standard npm version hook. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Parafee41
|
737a8cdb18
|
fix(web): use repo path identity in switcher (#2420)
* fix(web): use repo path identity in switcher * keep repo URL project names stable * fix server repo path resolution * fix repo path miss resolution * fix(server): guard clone-dir deletion with path ownership check Deleting a registry entry derived its clone dir from the entry NAME with no ownership check, so deleting a local repo that shares a display name with a server-cloned sibling wiped the sibling's checkout. Gate the removal on cloneDirBelongsToEntry (canonicalized path equality), the same entry.path-driven rule the handler's step 2b already mandates. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(server): fail closed on relative repo params and rate-limit GET /api/repo Relative separator-containing ?repo= values (org/name, ./repo) were canonicalized against the server CWD — an attacker-influenced realpathSync probe on an un-rate-limited GET — before failing anyway. Reject them immediately without touching the filesystem, drop the redundant path.sep clause, document the resolver's two-tier contract, and wire createRouteLimiter on GET /api/repo like its DELETE sibling. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(server): lock repo resolver branches and register for Windows CI Lock in the resolver's remaining branches: first-wins for ambiguous bare names, Windows-shaped input as a fail-closed path claim, the repos[0] default, and the case-insensitive name fallback. Register the suite in cross-platform-tests.ts so windows-latest actually runs the path-shape logic it exists to protect. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(web): single repoIdentity helper with repoPath normalized end-to-end The identity fallback chain was copy-pasted in Header and RepoLanding while backend-client already owns BackendRepo and the repoPath normalization. Export one repoIdentity helper, normalize fetchRepos like fetchRepoInfo, and emit repoPath from GET /api/repos so the scheme no longer silently relies on /api/repo.repoPath equalling /api/repos.path. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(web): persist and restore repo path identity in the URL The URL persisted only ?project=<display name>, so refreshing after switching to a duplicate-name repo silently restored the first same-named sibling. Persist ?repo=<server-resolved path> alongside the readable ?project= at both write sites, prefer it on restore (legacy project-only URLs still work), keep failed path restores fail-visible (no name fallback), and strip stale identity params when deleting the active or last repo. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(web): analyze completion connects by path identity RepoAnalyzer's completion callback passed the display name, so analyzing a repo whose basename collides with an existing one reconnected the first same-named sibling. The SSE terminal payload now carries the job's repoPath (both emit sites), the analyzer passes that identity to onComplete while the done screen keeps showing the display name, and old servers without repoPath degrade to today's behavior. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(web): scope code-reference file reads to the active repo identity The code viewer passed the display name as the repo scope, so with duplicate-name repos it rendered the wrong repo's file contents under the right filename. Pass the active path identity (currentRepo) with the display name as fallback, and collapse the two dead repo fields that were already shadowed by the readFile spread. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(web): show display names instead of absolute paths in labels The path-identity switch leaked raw filesystem paths into three user-facing surfaces: the re-analyze progress label, the repo-switch overlay, and the agent prompt's project name via loadGraphAnyway. Resolve display names at render time (registry lookup, then basename fallback) while state keeps holding the identity; loadGraphAnyway passes the name explicitly because initializeAgent's empty-deps closure would otherwise fall through to the literal 'project'. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(web): stop initializeAgent from clobbering repo identity with display names initializeAgent fell back to writing overrideProjectName (a display name) into the repo identity, so any future name-only caller — the pre-PR idiom — would silently kill the Active badge and re-admit the duplicate-name ambiguity through the agent path. Only opts.repo may write the identity now. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(web): drop dead initializers flagged by CodeQL pNameStr's and repoIdentity's initial values were never read: both are assigned on the success path before any use and the catch returns early. Bare declarations resolve CodeQL alerts 825/826. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * style(web): fix tailwind class order per root prettier plugin The worktree pre-commit hook resolved prettier-plugin-tailwindcss through symlinked node_modules and sorted scrollbar-thin differently than CI's clean-room install. Re-formatted with the root lockfile environment; no behavior change. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(web): e2e coverage for every #2419 duplicate-name ambiguity Provision two live repos with the same basename under different parents via POST /api/analyze, then drive a real browser through each item of the issue's "Actual behavior" list: - duplicate rows render and the ACTIVE one is identifiable before and after switching (active-state must not compare repo.name) - switching between duplicates swaps the loaded graph, verified by per-repo marker files (onSwitchRepo must not receive repo.name) - re-analyze targets the clicked duplicate's exact path (POST body), tracks progress on that row only, and the completion reconnect requests that same path — never the same-named sibling - delete requests target exactly the chosen duplicate's path; the sibling stays registered and loaded - backend ?repo= resolution is path-first: landing selection loads the exact repo, ?repo= survives F5, and a stale path fails closed to the repo picker instead of retargeting the sibling Adds four data-testids to Header (switcher trigger/row/reanalyze/ delete, rows expose data-active) so the spec has stable selectors, and broadens the post-analyze reconnect retry in App to any BackendError: the server may still be reinitializing when the SSE complete event fires, and that surfaces as transient 5xx/binder errors, not only 404. The re-analyze and delete tests deliberately assert identity at the request level and tolerate two pre-existing server races that are unrelated to the #2419 identity contract (freshly-analyzed DB briefly unreadable after SSE complete; registry validate-prune clobbering a concurrent unregister) — see the in-test comments. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(autofix): apply prettier + eslint fixes via /autofix command * test(web): isolate repo-path-identity e2e onto a spec-owned backend The spec is the only e2e file doing write operations (analyze, re-analyze, delete). Running its force re-analysis against the shared CI backend while parallel workers held connections took the whole server down (run 29145679019: the jobId poll died with ECONNRESET and every later test in every file failed to connect). Spawn a dedicated `gitnexus serve` on port 4799 with an isolated GITNEXUS_HOME in beforeAll instead: writes can no longer perturb the other suites, a crash is contained to this spec (its output is captured and printed, which CI otherwise loses), and the registry is hermetic by construction — the previous leftover-purge and shared-registry cleanup are gone. Every page is pointed at the spec backend through useBackend's supported localStorage override, which covers both the probe-driven landing flow and the ?server= auto-connect. Verified self-sufficient (6/6 with no shared server running) and non-interfering (full suite 39/39 with the shared server up). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * stabilize repo path identity e2e * fix(server): don't report analyze complete before the index is settled The analyze worker reports `complete` over IPC before its on-disk finalization (LadybugDB checkpoint, native handle release, metadata write) is visible at the storage path — observed up to ~6.5s behind the IPC message. The launcher's "reinitialize backend BEFORE marking complete" ordering was meant to make the repo queryable by the time the client sees the SSE complete event, but it never verified that: clients reconnecting on that event read a database still being written. Locally that surfaces as "Binder exception: Table CodeRelation does not exist" or a silently empty graph, and the open can quarantine the in-flight WAL; on slow CI runners the native layer racing the rewrite has killed the whole server (signal exit, no output — run 29146867959). Gate the complete transition on the index actually settling: LadybugDB file and metadata both rewritten by THIS job (mtime >= job start — bare existence is not enough, a re-analysis leaves the previous index in place while it works) and no transient WAL/shadow/checkpoint sidecars remaining. Bounded (60s) and proceed-on-timeout, so a job whose analysis legitimately rewrites nothing cannot wedge. Also evict the server's cached DB handle before reinitializing — same invalidation DELETE /api/repo performs — so post-completion reads cannot be served from a pre-rewrite handle. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(web): assert re-analyze completion identity at the request level The strict form (Ready + marker on the re-analyzed duplicate) still trips a deeper pre-existing storage race that makes a freshly re-analyzed database transiently unreadable to the reconnect even with the settle gate in place — unrelated to the #2419 identity contract this test covers. Keep the identity assertions (the reconnect targets the exact duplicate's path and never the same-named sibling) and leave a pointer to tighten once the storage race is fixed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(server): resolve the settle-gate path from the registry, not the request CodeQL flagged the settle gate's stat/exists probes as js/path-injection: the probed path derived from the user-provided analyze `path`. Resolve it from the repo's registry entry instead — the user value is now only a comparison key, and the probes run against the server-owned storagePath record, which is also the authoritative path readers resolve through. Re-resolved each poll round because the worker registers the repo as part of the same finalization the gate is waiting out. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Gergő Magyar <gergomagyar@icloud.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> |
||
|
VL
|
117587d543
|
fix(cli): actionable diagnostics for non-4K page-size buffer manager failures (#2424)
Some checks are pending
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Scorecard / Scorecard analysis (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
|
||
|
Gergő Magyar
|
df1fc36094
|
fix: make large incremental writebacks commit reliably (#2409) (#2425) | ||
|
azizur100389
|
f236be05e0
|
feat: gate Icebug community engine prototype (#2376) | ||
|
Gergő Magyar
|
8402963198
|
fix(ci): shard platform-sensitive matrix + spawn built CLI to fix Windows cross-platform timeout (#2394)
* fix(ci): shard platform-sensitive matrix + spawn built CLI to fix Windows cross-platform timeout The `windows-latest (platform-sensitive)` job was hitting its 15-min internal vitest watchdog in run-cross-platform.ts. It's cumulative slowness, not a hang: the fixed 72-file suite is dominated by ~50 CLI/worker process spawns, and Windows is ~5x slower than macOS at process startup (macOS ran the same set in ~3min of tests). Two complementary changes bring it back under the watchdog with headroom, without touching any test assertion: - Shard the platform-sensitive matrix (windows/macos × shard [1,2]) and forward `--shard=i/2` through run-cross-platform.ts to vitest, which partitions the fixed file list deterministically (sha1, equal file-count) — halving each runner. macOS/Ubuntu were already under budget. - New test/helpers/cli-entry.ts (`CLI_SPAWN_PREFIX`): spawn the built `dist/cli/index.js` when `GITNEXUS_E2E_CLI=dist` (set on the cross-platform job, which already builds) instead of `node --import tsx src/cli/index.ts`, which re-transpiles the whole CLI on every spawn. Defaults to tsx-on-source so local runs always reflect current source; `GITNEXUS_E2E_CLI=dist` on an unbuilt tree throws an actionable "run npm run build" error. dist is opt-in only — never inferred from a generic `CI` env — so an ambient `CI=1` can't silently run a stale build. Converted 8 spawn-based e2e suites; added test/unit/cli-entry.test.ts. The Ubuntu coverage job leaves `GITNEXUS_E2E_CLI` unset, so the tsx-on-source path stays exercised in CI too (both entry points covered). Measured on Linux: cli-limit-e2e 121.5s→91s, cli-e2e 289s→217s (~25%); larger on Windows where the transpile is a bigger share of each spawn. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(ci): derive platform-sensitive shard count from one source (#2394) The shard total was hardcoded in three coupled, unenforced places (matrix length, job-name suffix, --shard denominator); editing one without the others silently dropped a shard's tests with green CI. Add a checkout-free shard-plan job whose single TOTAL generates both the shard index list (consumed via fromJSON) and the /N denominator (job name + --shard arg), so they cannot drift. Asserts TOTAL>=1 to rule out an empty-matrix silent skip. No behavior change — still 2 shards per OS. Addresses PR #2394 tri-review finding F2. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ci): 3 shards for real Windows headroom + honest sharding comments (#2394) vitest shards by file COUNT, not runtime, so the heaviest spawn suites cluster into one shard: live CI showed Windows shard 1/2 at 12m12s (~81% of the 15-min watchdog) vs shard 2/2 at 3m0s. The old comments claimed "comfortable/generous headroom", which the count-based split doesn't deliver at 2 shards. Bump TOTAL to 3 (one line, single source) so even the busiest Windows shard clears the watchdog, and reword the comments to describe count-based (not time-based) sharding. Addresses PR #2394 tri-review finding F1. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(ci): extract testable parseShardArg from run-cross-platform (#2394) The --shard parse/forward glue had no unit test. Extract it into a pure scripts/shard-arg.ts (mirroring the computeSpawnPrefix extraction precedent) so the branch logic is lockable without the script's top-level execFileSync, and add test/unit/shard-arg.test.ts (absent -> undefined, valid token -> passed through, found amid other args). Behavior unchanged; U4 adds the malformed fail-loud on top. Addresses PR #2394 tri-review finding F3. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ci): fail loud on a malformed --shard arg (#2394) A shard-shaped-but-malformed arg (--shard=1, --shard, --shard=abc) was silently ignored, dropping the shard flag so both legs ran the full unsharded ~50-spawn suite — re-arming the Windows watchdog timeout with no signal. parseShardArg now throws an actionable error on any --shard/--shard=… arg that fails the strict regex (unrelated flags like --shardx= pass through), and the call site in run-cross-platform.ts catches it into console.error + exit 1, kept outside the execFileSync try so the message isn't swallowed by that catch's watchdog-only branch. Addresses PR #2394 tri-review finding F4. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(test): fail loud on an unknown GITNEXUS_E2E_CLI value (#2394) computeSpawnPrefix silently degraded any unknown GITNEXUS_E2E_CLI value to tsx-on-source, so a typo (e.g. `dsit`) would make CI believe it tests the dist entry point while actually running src. Throw on any value other than 'dist'/'src'/unset (the safe tsx default is preserved for unset/''/'src', so it still never selects dist without an explicit opt-in). Flip the unknown-mode unit test to assert the throw and add the missing {mode:undefined, distExists:true} case. Only ci-tests.yml sets the var (=dist), so no existing suite is affected. Addresses PR #2394 tri-review findings minor-a/b. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(ci): run cli-entry.test.ts on the cross-platform matrix (#2394) cli-entry.test.ts resolves CLI_SPAWN_PREFIX from a real path, and its last assertion (cli[/\\]index) has a Windows backslash branch that only Ubuntu exercised. Register it in PLATFORM_LOGIC so it runs on the Windows/macOS matrix too. (shard-arg.test.ts stays out — pure string logic, OS-independent.) List grows 73 -> 74; the generated shard matrix keeps coverage complete. Addresses PR #2394 tri-review finding minor-c. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(test): share tsxLoaderUrl(), dedup the last tsx-loader boilerplate (#2394) bridge-cache-reopen.test.ts carried its own copy of the tsx-loader-resolution boilerplate (createRequire -> resolve('tsx/package.json') -> pathToFileURL) — the one site the PR's CLI_SPAWN_PREFIX migration didn't cover (it spawns a seed script, not the CLI). Export the existing tsxLoaderUrl() from cli-entry.ts and reuse it here; the resolved loader URL is byte-identical. Addresses PR #2394 tri-review finding minor-d. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(test): make skipUnlessFtsAvailable install FTS on miss so shards are self-sufficient (#2394) Sharding the platform-sensitive suite into 3 exposed a latent test-isolation bug: load-only FTS primitives (test/integration/lbug-core-adapter.test.ts) only passed because a sibling installer test happened to co-locate in the same shard and install FTS into the shared ~/.lbdb first. At 3 shards, lbug-core-adapter landed in a shard with no installer sibling, so its load-only loadFTSExtension() failed deterministically on macOS+Windows shard 2/3 under GITNEXUS_REQUIRE_FTS=1. Make the gate self-sufficient: on a load-only miss under REQUIRE_FTS, install FTS with `auto` (LOAD-first, then one bounded network INSTALL) before treating it as a hard failure — mirroring withTestIndexedDB. A pre-installed extension still costs no network (auto is LOAD-first); offline/local runs (no env var) still skip gracefully. Verified: with a fresh HOME (no pre-installed FTS) + REQUIRE_FTS=1, lbug-core-adapter now passes 15/15 (previously threw). Addresses the 3-shard CI failure surfaced while validating PR #2394's F1 fix. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * ci: warm-cache the LadybugDB FTS extension across platform shards (#2394) Follow-up to the FTS self-install fix: cache ~/.lbdb/extension per OS + lockfile so a warm run skips the network install entirely and the parallel shards share one download across runs. Pure reliability/speed — on a cache miss the tests still self-install FTS on demand (test/helpers/fts-availability.ts), so this is never a correctness dependency, just a way to cut the network-install surface that made the sharded FTS tests flaky. Keyed by lockfile hash (a LadybugDB version bump re-installs); per-OS since the extension is a native binary. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ci): pass shard via env to clear zizmor template-injection (#2394) Interpolating ${{ matrix.shard }} (now sourced from the shard-plan job output) directly into the run: shell tripped zizmor's template-injection audit (code-scanning alert #824, ci-tests.yml:147). Move the value into a SHARD env var — assigned via ${{ }} but referenced as "$SHARD" in the shell, which is not an injection sink — and set shell: bash so the expansion is uniform across the windows + macOS matrix (the default run shell is pwsh on Windows, where $SHARD would be empty and trip the new malformed-shard fail-loud). Verified locally with zizmor: the :147 template-injection finding is gone. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * ci: shard the ubuntu coverage job and merge blobs before the threshold gate (#2394) The coverage job ran the full suite unsharded (~16 min). Shard it like the cross-platform matrix, then merge the per-shard coverage before enforcing the threshold gate: - shard-plan now also single-sources the coverage shard count (cov_total / cov_shards), so the coverage matrix + /N denominator can't drift. - The `tests` job becomes a coverage shard matrix: each shard runs `vitest run --shard --coverage --reporter=blob` with thresholds forced to 0 (a single shard's partial coverage can never meet the gate) and uploads its blob. FTS self-installs per shard, so sharding the full suite is safe. - New `coverage-merge` job (needs: tests) reduces the blobs with `vitest --mergeReports`, enforcing the REAL config thresholds on the combined ('new') coverage — this is the gate. It also emits the merged test-results.json and runs the unsharded web + docker suites, so the `test-reports` artifact keeps the exact shape ci-report.yml consumes for its base-branch ('baseline') vs new coverage delta. The shard arg goes through a SHARD env var + shell: bash (no template-injection). Validated locally: shard blobs write and merge into a coverage-summary.json + merged test-results.json; the merge enforces thresholds on the union. CI Gate still aggregates the coverage-merge result via the reusable-workflow call. Note: the coverage check names change (ubuntu / coverage 1/3 … + merge) — update any pinned branch-protection required checks. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ci): include hidden files when uploading the coverage blob (#2394) The coverage shards write their blob to gitnexus/.vitest-reports/ (a dotdir). actions/upload-artifact excludes hidden files by default, so the coverage-blob-* artifacts uploaded empty — the merge job then downloaded 0 artifacts and vitest --mergeReports failed with ENOENT scandir '.vitest-reports'. Set include-hidden-files: true on the blob upload so the blobs actually ship. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ci): group shard-plan GITHUB_OUTPUT writes to satisfy shellcheck SC2129 (#2394) Adding the coverage shard outputs (cov_shards/cov_total) made the shard-plan gen step write four individual `>> "$GITHUB_OUTPUT"` redirects, which shellcheck (run by the actionlint check) flags as SC2129. Group the echoes into a single `{ …; } >> "$GITHUB_OUTPUT"` block. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(test): cost-balanced shard sequencer to cut CPU contention (#2394) vitest's default --shard hashes file paths and splits by file COUNT, which clustered the spawn-heavy suites onto one runner (Windows platform shard 1 ran ~4x the others). Add a custom sequence.sequencer that overrides only shard() and balances by estimated WORK instead: - specWeight() weights the fileParallelism:false spawn-heavy suites (cli-e2e, lbug-db — already isolated to run sequentially) far above the parallel default files, plus file size as a cheap finer signal. Deterministic per checkout. - assignShards() does greedy longest-processing-time bin-packing (heaviest file into the currently-lightest shard). The partition stays complete and disjoint — verified: on the 74-file cross-platform set the three shards weigh 7611/7610/8064 (the sequential-heavy files spread ~7/7/8) with zero overlap and no file dropped, vs the hash split's count-only balance. sort() is left to the base sequencer so project groupOrder / duration-cache ordering is untouched. Pure logic split into shard-balance.ts with a unit test locking the disjoint+complete, balance, and determinism properties. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ci): install + cache FTS up front on the coverage (and cross-platform) shards (#2394) coverage 3/3 failed on extension-binary-real.test.ts: it uses the file-path FTS gate (requireFtsResourceOrSkip), which resolves ~/.lbdb/extension at MODULE LOAD and cannot self-install the way the load-path gate (skipUnlessFtsAvailable, U8) does. The coverage job had no FTS cache and relied on an installer test running first in the shard — the balancing sequencer reshuffled the shards and dropped extension-binary-real into a shard with no installer, so FTS was absent. Remove the ordering dependency: add scripts/ensure-fts.ts (init a throwaway lbug db, loadFTSExtension with policy:auto → LOAD-first, INSTALL on miss) and run it up front on every coverage AND cross-platform shard, after restoring the per-OS FTS cache. The coverage job now shares that same cache key (it previously had none — this is the "share the cached FTS with coverage" the failure pointed at). Cold cache installs once; warm cache is a no-network load. Verified locally: ensure-fts installs FTS into a fresh HOME and is a no-op when already present. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Gergő Magyar
|
b98f6e458f
|
fix(lbug): recognize Windows missing-shadow error so serve repo-switch recovers (#2382) (#2387) | ||
|
Gergő Magyar
|
76a1c90b02
|
fix(fts): diagnose Windows FTS missing-dependency load failures (#2374, Phase 1) (#2383)
Some checks are pending
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Scorecard / Scorecard analysis (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
* feat(lbug): classify FTS extension load errors with Windows missing-dependency guard (#2374) Add classifyExtensionLoadError() — a pure-string, lbug-free four-way classifier (missing_file / corrupt_file / missing_dependency / unknown). The Windows catch-all guard keys missing_dependency strictly on the error-126 signal, never LadybugDB's generic 'Failed to load library … needed by extension' wrapper, so 127/5/1114 and truncated (193) files route correctly. * feat(fts): surface classified missing-dependency remedy in doctor, repair-fts, and degrade warnings (#2374) Route the FTS load reason through classifyExtensionLoadError at all four surfaces (doctor, --repair-fts error, analyze degrade log, ftsDegradedWarning). For the Windows missing-dependency class, emit the runtime-install remedy (VC++ redist, then OpenSSL) instead of the wrong reinstall-over-network guidance; other classes keep their existing routing. Path redaction preserved on the client-facing warning. * test(fts): assert doctor surfaces the classified remedy end-to-end (#2374) Extend the broken-file e2e: doctor now prints the corrupt-file re-download remedy through the real CLI, and the Windows missing-dependency remedy (VC++/OpenSSL) must not misfire on a corrupt file — the catch-all guard, verified end-to-end. Also assert the repair path does not misfire. * style(fts): apply prettier formatting to #2374 diagnosis files * feat(fts): language-independent hedged fallback for Windows load failures (#2374) The Windows OS-error tail is localized, so matching only en/zh 126 text left other locales on the generic 'run doctor' remedy. lbug's 'Failed to load library' wrapper is English on every platform and present for all load failures, so use it as a fallback: when the localized tail matches no specific class, emit a hedged remedy that points the user at their own OS error and offers both branches (install runtime / --repair-fts) without prescribing the wrong single fix. Precise en/zh 126 keeps its definite remedy. * feat(fts): language-independent structural classifier via binary inspection (#2374) Add diagnoseExtensionLoad: pull the extension's file path out of lbug's own English wrapper and inspect the binary header (PE/ELF/Mach-O magic + arch) directly, so corrupt-vs-valid is decided by the file itself, not the localized OS-error tail. A valid binary that still failed to load ⇒ missing_dependency (runtime dep), decided in any OS display language and on all three platforms. Falls back to the string classifier (with its hedged fallback) when the file can't be read. Wire all four surfaces to it. Event Viewer / GetLastError-via-FFI were dead ends (lbug catches the failure — no crash event; no native FFI dep). * test(fts): exercise the structural classifier on real binaries (#2374) Add an integration suite that runs inspectExtensionBinary/diagnoseExtensionLoad against genuine binaries — the running node executable, the real lbugjs.node addon, and the installed FTS extension (valid); a truncated real binary and a real text file (corrupt). Registered in cross-platform-tests PLATFORM_LOGIC so it runs on the Windows + macOS matrix, proving the PE and Mach-O header parsing on real PE/Mach-O files (ubuntu covers ELF). * fix(fts): honor a corrupt_file verdict over a structurally-valid header (#2374) The structural probe in diagnoseExtensionLoad inspects only the first 4 KB, so a download truncated after its header reads 'valid' and was routed to the "install VC++, reinstalling will NOT help" remedy — the exact loop #2374 exists to kill, for the truncated-download case the module docstring claims it handles. Honor the loader's own corruption report ("file too short" / Windows error 193 "not a valid Win32 application") before defaulting to the dependency remedy; localized corrupt tails stay hedged missing_dependency, preserving language-independence. Addresses PR #2383 review finding F1. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(fts): return indeterminate for a PE header beyond the read window (#2374) The structural probe reads only BINARY_HEADER_BYTES (4 KB). A valid PE with a large DOS stub whose e_lfanew points past that window was wrongly called 'corrupt', routing a fine DLL to "re-download". A garbage e_lfanew from a truly corrupt file is indistinguishable from here, so widen the header verdict with 'indeterminate' and return it in that case; the caller then defers to the loader's own report instead of asserting a false verdict. Fat Mach-O stays valid (LadybugDB ships thin per-arch binaries). Also covers the unmapped-arch and garbage-PE-signature branches. Addresses PR #2383 review finding F1-secondary. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(fts): drop contradictory reinstall guidance from the analyze degrade log (#2374) For a missing runtime dependency the extension file is present, so appending FTS_UNAVAILABLE_MESSAGE (which tells the user to install it "with network access") to the remedy ("reinstalling will NOT help") produced self-contradictory guidance on the main analyze surface. Lead the missing_dependency degrade log with the class-neutral sentence (FTS_UNAVAILABLE_LEAD) and append only the classified remedy; other classes keep FTS_UNAVAILABLE_MESSAGE unchanged. Addresses PR #2383 review finding F2. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(fts): cache the load diagnosis so the degraded warning does no per-request I/O (#2374) ftsDegradedWarning() runs on every degraded /api/search response and MCP query, and it was calling diagnoseExtensionLoad — a synchronous openSync/readSync of the extension file — on every call. Compute the diagnosis once at mark-unavailable time (the single load-failure sink, run per Database not per request), cache it on ExtensionCapability, and have the warning read the cached result (falling back to the pure, no-I/O string classifier if it is absent). Loader capability-shape assertions relax from toEqual to toMatchObject for the new optional field. Addresses PR #2383 review finding F3. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(fts): cover the missing_dependency remedy on the --repair-fts path (#2374) The repair-fts error interpolates the classified remedy, but no test reached the missing_dependency branch — only the corrupt/invalid-ELF path. Add a Windows error-126 case asserting the thrown error carries the VC++ redistributable remedy and omits the old "retry the network install" tail, and that no index is dropped. Addresses PR #2383 review finding F6a (--repair-fts surface). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(fts): share the VC++ redistributable install hint (#2374) The Microsoft Visual C++ redistributable name and aka.ms URL were duplicated verbatim in WINDOWS_MISSING_DEPENDENCY_REMEDY and STRUCTURAL_MISSING_DEPENDENCY_REMEDY. Factor a single VC_REDIST_INSTALL_HINT constant so the pointer cannot drift between them; the composed remedy strings are byte-identical (existing exact-text assertions unchanged). Also adds a test covering the previously-unexercised structural remedy branch. Addresses PR #2383 review finding F5a. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(fts): guard FILE_CORRUPTION_SIGNATURES parity with the installer script (#2374) The corruption-signature list is deliberately duplicated between extension-load-error.ts and scripts/install-duckdb-extension.mjs (the .mjs cannot import the .ts), with nothing guarding against drift — a one-sided edit would desync the FORCE-INSTALL verb from remedy classification. Export the array from both and add a parity test that compares regex source + flags element-wise. Addresses PR #2383 review finding F5b. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(test): run extension-binary-real in the sequential lbug-db vitest project (#2374) extension-binary-real.test.ts imports @ladybugdb/core but ran in the parallel `default` project, contrary to TESTING.md's rule that native-LadybugDB tests live in the sequential `lbug-db` project. Add it to the lbug-db include list and the default exclude list; it now runs under lbug-db and no longer under default. Addresses PR #2383 review finding F6c. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(fts): fail loud, not silent-skip, on missing FTS artifacts under REQUIRE_FTS=1 (#2374) The real-binary structural tests gated on raw .skipIf(!lbugNative) / .skipIf(!installedFts), so under GITNEXUS_REQUIRE_FTS=1 a missing artifact would silently vanish from a green CI run (the #2299 trap). These tests inspect the extension file directly and need its path, not a loaded connection — so skipUnlessFtsAvailable (which needs an initialized LadybugDB) does not fit. Add requireFtsResourceOrSkip: skip gracefully offline, throw under REQUIRE_FTS=1. The always-on process.execPath assertion still runs everywhere. Addresses PR #2383 review finding F6d. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * style(fts): apply prettier formatting to the #2383 fix files (#2374) Line-wrapping only; the quality/format CI check flagged three files. No behavior change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Gergő Magyar
|
177bbc89c3
|
fix: surface real FTS extension LOAD errors and self-heal broken extension files (#2374) (#2375) | ||
|
Gergő Magyar
|
cdad478c96
|
fix: proxy-blocked installs survive onnxruntime-node postinstall and self-heal embeddings (#2370) (#2372)
Some checks are pending
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Scorecard / Scorecard analysis (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
|
||
|
Gergő Magyar
|
6252aa745f
|
feat(setup): add CodeBuddy and Qoder coding-agent integrations (#2368)
* feat(setup): add CodeBuddy and Qoder coding-agent integrations Adds Tencent CodeBuddy and Alibaba Qoder to gitnexus setup/uninstall, fitted to the editor-targets registry and --coding-agent selection. - CodeBuddy: MCP entry written into the first existing file of its documented priority chain (~/.codebuddy/.mcp.json recommended, ~/.codebuddy/mcp.json deprecated, ~/.codebuddy.json legacy) so a populated deprecated config is never shadowed; skills to ~/.codebuddy/skills/ (https://www.codebuddy.ai/docs/cli/mcp) - Qoder: MCP entry in ~/.qoder.json, skills to ~/.qoder/skills/ (https://docs.qoder.com/cli/using-cli, /extensions/skills) - editor-targets gains optional legacyFiles; uninstall sweeps them - roster strings updated (CLI help, i18n en/zh-CN, READMEs); en/zh-CN setup descriptions were stale (missing Antigravity) and are refreshed Supersedes and credits PR #1030 by @zykai0302, re-fitted to the post-#2168 selective-agent architecture with documented config paths. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(cli): assert stable zh-CN setup-description fragment * fix(setup): surface non-ENOENT config read/stat failures instead of clobbering * fix(setup): report corrupt legacy MCP files informationally during uninstall * test(setup): cover multi-candidate uninstall sweep combinations * fix(setup): detect CodeBuddy/Qoder installs via existing MCP config files * fix(setup): skip empty and non-file candidates in the MCP config chain * docs: add CodeBuddy and Qoder manual MCP configuration sections * test(ci): run the setup-uninstall round-trip in the cross-platform matrix * fix(setup): never claim "not configured" when uninstall recorded errors * refactor(cli): share the isEnoent predicate via editor-targets * refactor(setup): share chain-file install detection between CodeBuddy and Qoder --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
ACT900
|
5aada28da5
|
fix(embeddings): use system-matched onnxruntime-node CUDA build so CUDA 13 hosts use the GPU (#2341)
* fix(embeddings): use system-matched onnxruntime-node CUDA build so CUDA 13 hosts use the GPU
transformers.js exact-pins a CUDA-12 onnxruntime-node while gitnexus' own dep floats to a CUDA-13 build. npm/pnpm cannot dedupe an exact pin against a range, so npm i -g installs two copies and the gitnexus overrides block (root-only) is inert. On a CUDA-13-only host the nested CUDA-12 provider cannot load libcublasLt.so.12, the CUDA EP fails, and embeddings silently fall back to CPU (isCudaAvailable() also only probed .so.12).
Add onnxruntime-node-resolver.ts (module.registerHooks redirect to the host-matching build, no-op elsewhere) mirroring onnxruntime-common-resolver.ts; probe libcublasLt .so.12 OR .so.13 against the copy that actually loads; unit test with 12 cases.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(embeddings): wire CUDA-13 build-match resolver into MCP query embedder
The MCP query-time embedder (src/mcp/core/embedder.ts) has its own,
separate initEmbedder() used for semantic search — it only called
ensureOnnxRuntimeCommonResolvable() before importing transformers.js, so
the CUDA-13 build-matching redirect added for the analyze/CLI embedder
never applied here. A CUDA-13 host running MCP search with
--embedding-device cuda still loaded the mismatched default onnxruntime-node
build.
Wire ensureOnnxRuntimeNodeMatchesSystem() into the same call site, mirroring
the core embedder's ordering (registered after the common-resolver fallback,
before the dynamic transformers import).
* fix(embeddings): gate CUDA redirect decision on registerHooks availability
decide() computed the CUDA-major redirect independent of whether Node's
module.registerHooks API actually exists — only ensureOnnxRuntimeNodeMatchesSystem()
checked that. On Node 22.0-22.14 (allowed by this package's engines
floor; registerHooks needs >=22.15), isCudaAvailable() could therefore
report a redirect target that ensureOnnxRuntimeNodeMatchesSystem() then
silently failed to install, so transformers.js loaded the mismatched
default onnxruntime-node build while the embedder still requested
device:'cuda' against it — reintroducing the uncatchable native crash
this probe exists to prevent.
Move the registerHooks check to the top of decide() so the probe and the
loader can never disagree, and skip CUDA-major probing entirely in that
case (a redirect could never install anyway).
Also fixes a related test-helper bug found while writing this unit's
tests: loadResolver's destructuring default (`registerHooks = vi.fn()`)
silently substituted a real mock function even when a test passed
`registerHooks: undefined` to simulate Node < 22.15 — meaning the
existing 'no-ops... when registerHooks is unavailable' test never
actually exercised that path. Distinguish 'omitted' from 'explicitly
undefined' via an 'in' check.
* test(embeddings): drive decide() -> redirect:true and assert the resolve() closure
The PR's actual shipped behavior — the installed registerHooks resolve()
closure, and the full redirect-active decision path — had zero executed
test coverage. All 4 prior ensureOnnxRuntimeNodeMatchesSystem tests avoided
driving decide() into redirect:true because require.resolve/createRequire
were never mocked, so the two-distinct-directory comparison decide()
depends on always resolved against whatever's actually installed in this
test's real node_modules (a single real copy, not the PR's two-copy
scenario).
Extend loadResolver()'s existing node:module mock to also fake createRequire,
keyed by call origin, so resolveOurOrtNodeDir/resolveDefaultOrtNodeDir can be
driven to two distinct fake directories with distinct CUDA majors — reaching
redirect:true without adding any injection points to production code. Then
capture the installed resolve() closure (mirroring the sibling
onnxruntime-common-resolver.test.ts's captureResolve() pattern) and assert
its three branches directly: onnxruntime-node redirect, onnxruntime-common
redirect, and passthrough for any other specifier.
* fix(embeddings): distinguish ldd detection-failure from no-CUDA-provider
ortCudaMajor treated any execFileSync('ldd', ...) failure with no usable
stdout (missing ldd binary, permission-denied .so, sandboxed exec)
identically to 'CUDA provider genuinely absent'. The pre-PR detection
(hasOrtCudaProvider) only used existsSync, never ldd, so this is a
regression: a CUDA-12 host that worked fine before this PR can now
silently fall back to CPU if ldd itself can't run, even though the
provider .so and system CUDA libs are both genuinely present.
readSoNeeded now reports whether ldd produced any usable output at all,
distinct from 'ldd ran and just found no matching NEEDED entry' (the
existing, already-handled '=> not found' case). When detection genuinely
fails, log a warning so an operator can tell 'CPU fallback because
detection itself failed' apart from 'CPU fallback because no CUDA build
shipped' — the return value stays null either way (the type can't
distinguish a third state), but the two cases are now observably
different via the log.
* fix(embeddings): check ourDir independently of whether defaultDir resolved
decide()'s ourDir fallback lookup was nested inside
'if (systemMajor != null && defaultDir)', so a null defaultDir (transformers'
own onnxruntime-node resolution failing outright, e.g. a partial/broken
install) skipped checking ourDir entirely — getEffectiveOnnxRuntimeNodeDir()
returned null even when gitnexus' own matching CUDA-13 copy would have
resolved fine and worked.
defaultDir resolving is not a precondition for the comparison: an
unresolvable default already counts as 'the default doesn't match', so the
ourDir check now runs whenever systemMajor is known, regardless of whether
defaultDir resolved.
* fix(embeddings): prefer CUDA 13 globally across the env-var directory scan
detectSystemCudaMajor's CUDA_PATH/LD_LIBRARY_PATH scan returned on the
first CUDA-major match within a single dir/sub pair, so a stale .so.12
found early (e.g. a leftover CUDA_PATH entry from a prior install) shadowed
a genuine .so.13 found later in the search path, even though the scan's
own ordering (checking 13 before 12 within each pair) was clearly intended
to prefer 13 wherever possible.
Keep scanning the full search space once a 12 is found, only returning
early once a 13 is found (the best possible answer) or the space is
exhausted.
* fix(embeddings): have onnxruntime-common-resolver defer to the effective onnxruntime-node dir
onnxruntime-common-resolver.ts independently re-derived transformers'
default onnxruntime-node dir (its own copy of the 'resolve transformers'
main entry, then onnxruntime-node' walk) to compute which onnxruntime-common
to pair with — duplicating onnxruntime-node-resolver.ts's own walk, and
capable of disagreeing with it: when the CUDA-major redirect is active,
this hook would still pair onnxruntime-common with transformers' default
(unredirected) onnxruntime-node, not the redirected copy the other hook
just switched onnxruntime-node itself to.
Have it call the already-exported getEffectiveOnnxRuntimeNodeDir() instead
— the same decision the CUDA-major redirect hook uses — so both hooks
always agree on which onnxruntime-node they're pairing onnxruntime-common
against, and the duplicated resolve-walk is removed entirely rather than
merely factored out.
* fix(embeddings): cache the effective CUDA major to remove redundant subprocess spawns
isCudaAvailable() in embedder.ts re-invoked ortCudaMajor/detectSystemCudaMajor
directly even though decide() (via getEffectiveOnnxRuntimeNodeDir) had
already computed both to make its redirect decision — a second, wasted
ldconfig + up to 2 ldd spawns on every initEmbedder() call.
Add effectiveMajor to the memoized Decision, computed once inside decide()
alongside effectiveDir/systemMajor, and export a single
isEffectiveCudaAvailable() that reads straight from the cached decision.
embedder.ts's local isCudaAvailable() wrapper (and its now-unused
getEffectiveOnnxRuntimeNodeDir/ortCudaMajor/detectSystemCudaMajor imports)
is replaced by this one exported function.
* fix(embeddings): surface CUDA redirect state at info level and in doctor
A successful CUDA-build redirect logged only at logger.debug (filtered
by the default 'info' level), and gitnexus doctor's embeddings section
never mentioned the redirect at all — leaving no diagnostic path for
'why is my CUDA-13 host still on CPU' after this PR ships.
Log the successful-redirect line at info (no-redirect/failure paths stay
at debug, since those are the common, expected case). Add
cudaRedirectDoctorStatus(), a pure summary of decide()'s already-computed
decision mirroring doctor.ts's existing localEmbeddingDoctorStatus shape,
and print it as a new literal (non-i18n) 'CUDA:' line in doctor's
embeddings section alongside the existing 'Support:' line, matching that
line's established convention.
* test(embeddings): register onnxruntime-node-resolver.test.ts in the cross-platform subset
The new test file guards on process.platform (linux/darwin cases) but was
absent from cross-platform-tests.ts's PLATFORM_LOGIC list, which
TESTING.md says platform-sensitive tests should be added to — so it never
ran on the Windows/macOS CI matrix, only Ubuntu.
Note: the sibling onnxruntime-common-resolver.test.ts has the identical,
pre-existing gap (it predates this PR) — left as-is here, since fixing
unrelated pre-existing test-registration debt is out of scope for this
PR's own follow-up fixes.
* test(embeddings): strengthen weak assertions, add garbled-output and CUDA_PATH coverage
Three of the four ensureOnnxRuntimeNodeMatchesSystem tests only asserted
'doesn't throw' rather than a concrete outcome — including one literally
named 'idempotent' that never asserted a call count on its own spy.
Strengthen each to assert real outcomes (module stays functional after a
no-op; spy call counts; return-value shape), while keeping the true
install-once idempotency proof in the redirect-active test added earlier
(this file's no-redirect scenario can't exercise it, since registerHooks
is never called either way).
Add the missing edge cases flagged in review: a CUDA_PATH-only fallback
scan test (mirroring the existing LD_LIBRARY_PATH one), and garbled/
unrecognized ldconfig and ldd output cases for both detectSystemCudaMajor
and ortCudaMajor, confirming neither falsely matches a CUDA major on
unparseable input. Also parameterize the non-linux platform test across
both darwin and win32 rather than darwin alone.
Not changed: the process.env reassignment vs. Object.defineProperty
'inconsistency' flagged in review — process.env, unlike process.platform,
has no getter-only restriction, so plain reassignment is already correct
and switching it to Object.defineProperty would be unnecessary ceremony.
* docs(embeddings): note the npm link/symlinked dev-checkout resolution caveat
resolveOurOrtNodeDir/resolveDefaultOrtNodeDir anchor to this module's own
real (post-symlink) location via import.meta.url, so a linked local dev
checkout may resolve against its own node_modules rather than the
consuming app's. Narrow, dev-only blast radius (regular npm/pnpm installs
are unaffected) — document-only, no structural fix warranted.
* fix(test): point the windowsHide spawn-family registry at the file that actually spawns
hooks.test.ts's windowsHide regression check still listed
gitnexus/src/core/embeddings/embedder.ts as a child_process-spawning
file, but this PR itself already moved all execFileSync usage out of
embedder.ts and into the new onnxruntime-node-resolver.ts — without
updating this registry. The check was silently failing at the PR's own
head commit (confirmed: 0 spawn-family calls found in embedder.ts,
'expected 0 to be greater than 0'), a pre-existing gap this fix-pass
surfaced via a full-suite run rather than something introduced by any of
the preceding follow-up commits.
Swap the registry entry to onnxruntime-node-resolver.ts, which does
import execFileSync (ldd + ldconfig, both already correctly passing
windowsHide: true).
* fix(test): make onnxruntime-node-resolver.test.ts path comparisons OS-agnostic
Registering this file in cross-platform-tests.ts's PLATFORM_LOGIC (a
prior commit in this series) means it now runs on the Windows CI matrix,
not just Ubuntu — and several of the fakeDirs-based tests (redirect:true,
ourDir-independent, subprocess-count, doctor-status) compared the
resolver's real join()/dirname() output against hardcoded forward-slash
fixture strings via exact-match or .startsWith().
Node's module is bound to path.win32 (or path.posix) based on the
REAL host OS at process start — stubbing process.platform later, as these
tests already do for the resolver's own platform branching, has no effect
on it. So on a genuine Windows runner, join(effectiveDir, 'package.json')
backslash-normalizes even under a faked platform:'linux', silently
breaking every forward-slash comparison in this file: the createRequire
dispatch would route to the wrong fake require, throw MODULE_NOT_FOUND,
get swallowed by ensureOnnxRuntimeNodeMatchesSystem's outer try/catch, and
registerHooks would never fire — the redirect-active tests would fail
outright on Windows CI.
Normalize every comparison point (the createRequire dispatcher, and the
shared execFileSync/existsSync mocks) with a single toPosix() helper.
Added a forceWin32Path test option (using path.win32's real join/dirname
behavior) to prove this holds without needing an actual Windows runner —
confirmed by temporarily reverting the fix and observing the new test
fail with the exact predicted mismatch before restoring it.
* chore(autofix): apply prettier + eslint fixes via /autofix command
* fix(embeddings): keep CUDA auto-detect working on Node < 22.15 when the default build already matches
The registerHooks guard in decide() returned effectiveMajor: null
unconditionally, so on Node 22.0-22.14 / 23.0-23.4 (engines floor is
>=22.0.0) isEffectiveCudaAvailable() was always false and a CUDA-12 host
whose default onnxruntime-node build already matched — which needs no
hook at all to use the GPU — silently regressed from CUDA to CPU on the
auto device path (pre-PR isCudaAvailable() behavior).
Probe the system and the default copy regardless of registerHooks
availability; only the ourDir redirect branch stays gated on it, so the
probe still never reports a redirect target that cannot be installed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Gergő Magyar <gergomagyar@icloud.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
|
||
|
Malik
|
859e4b75a4
|
fix(cli): --limit i18n, 0/negative guard, and correct truncation paths (#2310)
Some checks are pending
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Scorecard / Scorecard analysis (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
* fix: add --limit i18n, negative guard, correct property paths, and zh-CN translations - Add i18n keys for context/impact/cypher/detect-changes --limit options - Add zh-CN translations for all 4 --limit option descriptions - Add Math.max(0, parseInt()) guard to prevent negative --limit - Fix ALL property path mismatches discovered by audit: - context: callers/callees → incoming.calls/outgoing.calls+accesses - impact: upstream/downstream → affected_processes/affected_modules/byDepth - cypher: rows → row_count cap (rows embedded in markdown string) - detect-changes: affected_flows → affected_processes - Change query command from required to optional positional arg with -q alias - Update @ladybugdb/core from ^0.16.1 to ^0.17.1 - Update typescript from ^5.4.5 to ^5.9.3 * test: add E2E tests for --limit flag across all 5 CLI commands Tests context, impact, cypher, detect-changes, and query with --limit 1, baseline comparison, and --limit 0 (falsy/no-op). detect-changes output is formatted text (not JSON), so those tests count symbol lines matching 'Type name -> filePath' pattern. 14 tests, all passing. No regressions in 6455 existing tests. * fix: address Copilot review feedback on --limit guards - Add Math.max(0, ...) guard to queryCommand limit parsing - Change if(limit) to if(limit !== undefined) in all 5 commands (prevents --limit 0 from being treated as falsy/no-op) - Make queryText parameter optional (Commander may pass undefined) - Fix usage error strings: --search to -q, --query (en + zh-CN) * chore(autofix): apply prettier + eslint fixes via /autofix command * fix(cli): centralize --limit parsing, slice cypher markdown, fix usage text Address PR review feedback on --limit handling: - Add a shared parseLimit() helper (Number.isInteger(n) && n > 0), used by all 5 tool commands. Non-numeric / 0 / negative --limit now means "no limit" instead of the `options.limit ? Math.max(0, parseInt(...)) : undefined` path, where a string like "abc" is truthy and yields NaN -> slice(0, NaN) -> the guardrail commands (impact/context/detect-changes) silently emptied results with exit 0. - cypher: slice the markdown table to --limit data rows so the reported row_count matches what is actually printed (was capping row_count while printing every row). - Fix query usage string: [search_query] (optional positional) and `--query <text>` invocation form, not the option-definition `-q, --query <search_query>` syntax (en + zh-CN). - Add an E2E regression test for non-numeric --limit. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(cli): escape newlines in cypher markdown cells A multi-line cell value (e.g. a symbol's `content`) was rendered with raw newlines via String(v), so one logical row spanned multiple physical lines. That corrupts the markdown table and breaks `cypher --limit`'s line-based slice (it kept the wrong number of rows, often zero, while row_count over-claimed). Collapse newlines in formatCypherAsMarkdown so one physical line == one row; the existing CLI slice is now correct and the pre-existing un---limited corruption is fixed too. (#2310 review) * test(cli): de-vacuum the --limit truncation tests The truncation it()s used the repo-banned vacuous-pass pattern (early-return on status===null, assertions guarded by if(Array.isArray), bounds-only toBeLessThanOrEqual — DoD.md:82) against `validateInput`, which has only 1 caller, so context/impact/query --limit 1 compared 1>=1 and stayed green even if the slice were deleted. Rewrite with unconditional, exact assertions and target `logMessage` (2 callers, 4 processes) so the no-limit baseline truly exceeds the limit; detect-changes now mutates two real function bodies (two changed symbols). Adds a multi-line-cell cypher --limit regression. (#2310) * test(ci): run cli-limit-e2e in the cross-platform matrix The --limit E2E suite spawns the real CLI (child_process) but was not in SPAWN_CLI, so it ran only on Ubuntu — the cross-platform check only fails on listed-but-missing files, not the reverse (TESTING.md §Cross-platform). Register it so the --limit regression guard also runs on Windows/macOS, where path separators, CRLF and the formatted-output arrow differ. (#2310) * fix(cli): document impact --limit affected-list cap, drop dead byDepth re-slice `impact --limit` also caps affected_processes/modules, but the help only mentioned the per-depth cap — so JSON consumers reading the affected lists got a silently-truncated array. Update en + zh-CN + the command description to say so. Also remove the client-side byDepth re-slice: the backend already paginates byDepth to the same limit (paginationLimit = clamp(limit,1,10000), offset applied backend-side), so the client slice was a guaranteed no-op. (#2310) * fix(cli): reconcile detect-changes --limit summary, list, and overflow formatDetectChangesResult computed the "... and N more" overflow from the already---limit-sliced array length, so under `--limit` the header (true summary total), the listed rows, and the marker disagreed — e.g. "2 symbols" in the header but a list of 1 with no marker. Base the overflow on the true summary.changed_count / affected_count instead, and add the same marker to the affected-processes list, so header + list + marker stay consistent. (#2310) * feat(cli): add -l shorthand to impact --limit The PR added the -l alias to context/cypher/detect-changes but left impact on the long --limit only, so `impact -l 5` errored while `context -l 5` worked. Add -l for parity and update the help-i18n OPTION_DESCRIPTION_KEYS key to the new `-l, --limit <n>` flag string so the description still resolves. (#2310) * fix(cli): bound all context --limit array categories context --limit sliced only incoming.calls / outgoing.calls / outgoing.accesses / processes, leaving the other relType buckets unbounded — notably incoming.accesses (bounded on outgoing but not incoming) plus imports/extends/ uses/… and typed_properties. Replace the hardcoded slices with a generic loop over every array-valued bucket under incoming/outgoing, plus typed_properties and processes, so --limit caps the whole context payload. (#2310) * refactor(cli): parse --offset with a parseLimit-style helper impactCommand parsed --offset with the legacy parseInt/Number.isFinite idiom while --limit had moved to parseLimit, leaving two parsing styles side by side. Add a sibling parseOffset helper (non-negative — offset 0 is valid) and use it, so both options share one idiom; as a bonus it now rejects negative/fractional offsets instead of forwarding them to the backend. (#2310) --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Gergo Magyar <gergomagyar@icloud.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Gergő Magyar
|
35ebe37c42
|
fix(deps): pin Ladybug 0.18.0, validate the multi-writer deadlock fix (#2340)
* chore(deps): bump @ladybugdb/core to 0.18.0 Pins the release containing LadybugDB/ladybug#605 (TransactionManager lock-order-inversion deadlock fix). Checked for known post-release regressions specific to 0.18.0 via the Ladybug issue tracker — none found. * fix(lbug): re-validate version-coupled comments and regexes for 0.18.0 Extends the LADYBUGDB-CONTRACT re-validation to two spots the marker convention doesn't catch (bridge-db.ts's LBUG_OPEN_RETRY_PATTERNS, conn-lock.ts's serialization rationale). Confirms via upstream source diff (v0.16.1..v0.18.0) that every matched error-text string is unchanged; conn-lock.ts's rationale is unaffected by #612/#623 since neither addresses concurrent queries on one connection. Adds a stemmer-sweep test proving the bundled 0.18.0 FTS extension accepts every entry in SUPPORTED_FTS_STEMMERS, not just the default porter. A live-trigger test for isMissingShadowSidecarError was attempted but abandoned after empirical probing showed it isn't reliably reproducible (even a SIGKILL-simulated crash didn't reproduce the error on reopen) — documented as inspection-verified instead of overclaiming test coverage that doesn't exist. * test(lbug): add concurrent multi-connection deadlock stress test (#2338) Directly validates LadybugDB/ladybug#605 — the TransactionManager lock-order-inversion deadlock between a commit()-triggered checkpoint and a concurrent beginAutoTransaction() — under a shape close to GitNexus's real concurrent-writer load, independent of conn-lock.ts's app-level serialization. Comparison run against 0.17.1 (pre-fix): 1 of 4 runs hung for the full 60s timeout, a direct reproduction of the deadlock. 9 consecutive runs against 0.18.0 (post-fix) all passed cleanly. Production is unchanged — conn-lock.ts still serializes every write; this test validates the engine-level fix without shipping multi-writer as a default. * fix(test): address code review findings in multiwriter deadlock test - Reuse lbug-config.ts's createLbugDatabase (via GITNEXUS_WAL_CHECKPOINT_THRESHOLD) instead of a hand-duplicated 9-arg raw constructor call whose stated justification (needing to bypass createLbugDatabase for the threshold override) was incorrect — the env var already provides it. - Close every QueryResult via the existing closeQueryResults helper (write loop, read loop, verify query, setup query) instead of leaking native cursors, matching lbug-adapter.ts's established pattern. - Move all cleanup (timers, connections, db close, env var restore) into the outer finally block so it runs on every exit path, not just the happy path — a timeout or a writer exhausting its retry budget no longer leaves dangling timers/connections/abandoned query loops. Verified: 8 consecutive runs after the refactor, all passing cleanly. Found via 8-angle parallel code review (medium effort); the two other findings (isDbBusyError not recognizing LadybugDB's 'Only one write transaction' message, and shadow-file poll timing sensitivity) are noted in the PR description as residual — the first is a production-code change beyond this validation test's scope, the second is inherent to observing a transient native sidecar file and not cleanly fixable without overengineering. * fix(test): apply ce-code-review autofix findings Fixes from an 8-persona parallel review round (correctness/testing/ maintainability/project-standards/reliability/adversarial/agent-native/ learnings): - Extract the duplicated skipUnlessFtsAvailable/FTS_UNAVAILABLE_NOTE helper (previously copy-pasted between lbug-core-adapter.test.ts and fts-stemmer-sweep.test.ts) into a shared test/helpers/fts-availability.ts. - Fix a native connection leak: verifyConn in the deadlock test's final verification block is now pushed into the readers array the outer finally already closes, so it's cleaned up even if the count query throws. - Fix a latent TypeScript type error (tsconfig.test.json catches it, tsconfig.json doesn't): conn.query() types as QueryResult | QueryResult[]; narrow to the single-result case before calling .getAll() rather than assuming the array branch never happens. - Replace repeated inline InstanceType<typeof import(...)> expressions with local LbugDatabase/LbugConnection type aliases. Verified: 12 consecutive runs of the deadlock test all pass, full lbug-db project (336 tests) green. Cross-reviewer-confirmed but left as residual (design judgment calls, not mechanical fixes) for the PR description: isDbBusyError doesn't recognize LadybugDB's 'Only one write transaction' message (pre-existing production gap, confirmed independently by 3 reviewers); the deadlock test's timeout path doesn't cancel in-flight writer/reader loops before closing connections; the reader loop has no bounded retry for transient errors during the race window; pinning @ladybugdb/core with a caret range trades automatic patch updates for less re-validation certainty. * docs: trim task-referencing JSDoc artifacts, add operator notes The U2 re-validation pass left verbose 'Re-validated on the 0.17.0->0.18.0 bump (#2338): ...' paragraphs stacked onto 5 production files' docstrings, alongside the already-updated version numbers. That narrative (SIGKILL-probe methodology, diff commands run, issue cross-references) belongs in the PR description, not in code comments that will accumulate a new paragraph on every future bump and confuse readers who just want the current fact. Trimmed each to state only the durable, current-state fact: - lbug-config.ts, sidecar-recovery.ts, lbug-adapter.ts, bridge-db.ts: dropped the bump-narrative paragraphs; kept only genuinely durable notes (e.g., which matchers are inspection-verified vs live-tested, what upstream wording changed). - conn-lock.ts: compressed a 12-line, 3-issue-number enumeration into 2 lines stating the current conclusion (no upstream 0.18.0 fix addresses the same-connection-concurrent-query risk this lock guards against). Also added operator-facing notes to GUARDRAILS.md and RUNBOOK.md's existing 'LadybugDB lock' sections: an isDbBusyError gap found during this validation (LadybugDB's 'Only one write transaction...' message isn't recognized by our busy/lock retry matcher) means that specific error can surface unretried. Documented so it's recognized as the same single-writer conflict, not a new failure mode. * refactor(test): use gitnexus-shared's withRetry in multiwriter deadlock test Replaces the hand-rolled writeWithRetry/sleep loop with the existing gitnexus-shared retry helper (already used by embeddings/hf-env.ts) instead of duplicating the pattern. * fix(test): guarantee non-zero retry delay in deadlock test's writer loop withRetry's isRetryable previously returned {retry: bool} with no afterMs, so computeBackoffMs's exponential-jitter formula gave a deterministic zero-delay on the first retry (floor(random()*1) is always 0 at attempt=0). This contradicted the file's own documented tuning, which specifically needs a non-zero 1-3ms delay to avoid tripping a different native guard. Return an explicit afterMs override on the retryable branch instead. * docs(test): remove dangling doc references from deadlock test JSDoc The JSDoc pointed to a local-session-only docs/plans/2026-07-01-001-... path (docs/ is repo-gitignored, so this never existed for anyone but the implementing session) and to "the PR description" as a source of truth that stops being current once the PR merges. Replace both with self-contained prose and durable references (issue/PR numbers, commit SHAs, GUARDRAILS.md/RUNBOOK.md) that stay resolvable after merge. * fix(search): harden SUPPORTED_FTS_STEMMERS against external mutation Type as ReadonlySet<string> to match this codebase's established convention for exported validation allowlists (EVAL_SERVER_TOOLS, STRUCTURAL_LABELS). Type-only change — no behavior change; both the internal .has() check and the sweep test's spread-iterate pattern continue to work unchanged. * docs(guardrails): fold Known-gap note into the LadybugDB Sign's Why label GUARDRAILS.md's own convention is strictly Trigger/Do/Why per Sign entry (stated in the file's header, followed by all 5 other entries). The new isDbBusyError gap note introduced a 4th label; fold it into Why instead, which is what it's actually explaining. * fix(test): run the multi-writer deadlock test on Windows too itLbugMultiwriter mirrored lbug-core-adapter.test.ts's win32 skip, but that pattern exists for a close-then-reopen-same-path lock lingering bug (kuzudb/kuzu#3872). This test never reopens the database — it holds connections open for the whole run — so the skip excluded the one test validating issue #2338's deadlock fix from the platform conn-lock.ts actually ships native bindings for. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
azizur100389
|
8ad4469e96
|
fix(test): stabilize local Windows gate baselines (#2314) | ||
|
Gergő Magyar
|
576e81442e
|
fix(search): index description field for FTS so doc comments are keyword-searchable (#2300)
Some checks failed
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Scorecard / Scorecard analysis (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
Devcontainer Smoke / Config-transform unit tests (push) Has been cancelled
Devcontainer Smoke / Build devcontainer image (push) Has been cancelled
* fix(search): index description column for FTS so doc comments are keyword-searchable Closes #2299. descriptionExtractor (#2286) populates the `description` column for every symbol table, but FTS only indexed name+content on 5 tables, so doc-comment keywords (Javadoc/KDoc/godoc/Rust ///) were invisible to BM25 keyword search. - Add `description` to the Function/Class/Method/Interface FTS indexes (File has no description column, left as name+content). - Add FTS indexes for the remaining EMBEDDABLE_LABELS symbol tables (Struct, Enum, Trait, Impl, Macro, Namespace, Constructor, TypeAlias, Typedef, Const, Property, Record, Union, Static, Variable). - createSearchFTSIndexes now drops-then-creates each index so the schema change reaches existing DBs on incremental re-analyze and --repair-fts (createFTSIndex is idempotent-by-name and would otherwise skip stale indexes). Tests: fts-schema column-subset + coverage guards; drop-before-create order; e2e doc-comment keyword search (Java class + Rust struct found by description-only terms). bm25-search assertions derive from FTS_INDEXES. * fix(review): apply autofix feedback - Guard the --repair-fts path on FTS-extension availability before createSearchFTSIndexes drops-then-creates indexes (P1 regression: without the gate, an unavailable extension could drop existing indexes then fail to recreate them, leaving the DB index-less). Mirrors the analyze path's ftsAvailable gate and fails loudly first. - Add a re-analyze upgrade integration test: seed an old name+content-only DB (no Struct index), run the real createSearchFTSIndexes(), and assert description keyword search + the previously un-indexed Struct now resolve. Proves drop-then-create upgrades a live stale index end-to-end. * fix(ci): add loadFTSExtension to --repair-fts test mocks The R3 review fix added a loadFTSExtension availability gate to the --repair-fts path, but run-analyze-fts-repair.test.ts mocked the lbug adapter without that export, so both repair tests threw `No "loadFTSExtension" export`. Add loadFTSExtension to the two mocks (returning true to preserve their original intent) and add a dedicated test proving the guard fails loudly — and does NOT drop any index — when the extension is unavailable. * test(fts): run fts-description-search in the sequential lbug-db project It was the only FTS-index-creating integration test left in the parallel `default` vitest project; every other ftsIndexes-using test (search-core, search-pool, augmentation, …) runs in the `lbug-db` project, which forces fileParallelism: false to avoid LadybugDB native mmap file-lock conflicts in parallel forks (Windows). Add it to the lbug-db include list and the default exclude list to match the convention and remove the flake risk. * test(ci): fail loudly when FTS extension is unavailable, never silently skip FTS-dependent lbug integration suites (search-core, search-pool, augmentation, fts-description-search, …) self-skip via ctx.skip() when the LadybugDB FTS extension can't load, emitting only a console.warn while the job stays green. That means a broken/missing FTS extension in CI would make these integration tests silently vanish with no signal — false confidence. withTestLbugDB now honors GITNEXUS_REQUIRE_FTS=1: when set and the extension is unavailable, setup() throws instead of skipping, so the suite fails loudly. The CI test jobs (ubuntu coverage + windows/macOS cross-platform) set the flag; local/offline runs leave it unset and keep skipping gracefully. (Verified the extension currently loads on all three runners, so this is a guard against regression, not a behavior change today.) * test(ci): run fts-description-search on macOS/Windows cross-platform jobs The new FTS description-search suite was registered in the sequential lbug-db vitest project (ubuntu/coverage) but absent from LBUG_NATIVE, so the macOS/Windows platform-sensitive jobs (which run only the explicit ALL_CROSS_PLATFORM allowlist via run-cross-platform.ts) never executed it. The GITNEXUS_REQUIRE_FTS=1 hardening on those jobs guarded the old FTS fixtures but not the new 20-index/description path. Add the suite to LBUG_NATIVE so the new path is validated cross-platform too. Refs #2299. * fix(search): verify FTS indexes cover description, not just queryability verifySearchFTSIndexes probed each index with QUERY_FTS_INDEX and treated 'queryable' as 'present'. A stale name+content-only index left on a pre-#2299 DB stays queryable yet silently misses the description column, so verification would pass green while doc-comment search stayed broken. Switch to a single CALL SHOW_INDEXES() that exposes property_names per index, and report an index as missing when it is absent OR does not cover its configured columns. Return contract (string[] of table.indexName) is unchanged, so both run-analyze.ts call sites are untouched. The per-index string interpolation is gone, so the now-dead safeIdentifier helper is removed. The real caller of the live function in tests is bm25-search.test.ts (the repair test mocks verifySearchFTSIndexes wholesale); its two probe-shaped cases are rewritten to feed SHOW_INDEXES rows and now assert column coverage, plus an absent-index case. Refs #2299. * test(search): assert description search via the public query surface The #2299 integration suite only exercised the searchFTSFromLbug helper. Add a third block that drives the public LocalBackend.callTool('query') path — which resolves the repo via the registry and routes BM25 through the pool adapter (a different connection context than the core-adapter helper) — and asserts a description-only keyword returns the seeded class. Reuses the existing description-only SEED and production FTS_INDEXES; partial-mocks repo-manager so listRegisteredRepos points at the test DB while cleanupOldKuzuFiles and the rest stay real. Refs #2299. * test(search): make lbug-core-adapter FTS gate honor GITNEXUS_REQUIRE_FTS lbug-core-adapter.test.ts has its own per-test FTS gate (skipUnlessFtsAvailable) that called ctx.skip() whenever the extension could not load — bypassing the GITNEXUS_REQUIRE_FTS=1 hardening that withTestLbugDB already honors. Since this file is in LBUG_NATIVE it runs on the ubuntu/macOS/windows jobs that all set GITNEXUS_REQUIRE_FTS=1, so an FTS regression on a runner would have let these FTS-primitive tests silently vanish from a green run — the exact gap #2299's test-infra hardening set out to close. Make the helper mirror withTestLbugDB: when GITNEXUS_REQUIRE_FTS=1 and the extension is unavailable, throw (hard fail) instead of skipping. Offline/local runs (no env var) still skip gracefully. Refs #2299. |
||
|
Gergő Magyar
|
1a03c8527a
|
feat(group): cross-repo call trace using PDG (#2269)
* refactor(group): extract shared resolveBridgeNeighbors from cross-impact
Lift the uid-filtered consumer<->provider ContractLink join (direction +
queryBridge + row normalization + confidence sort) out of runGroupImpact's
inline Phase-2 block into an exported resolveBridgeNeighbors helper. Behavior
is unchanged for impact; the helper becomes the single shared bridge join so
the upcoming cross-repo trace path never forks its own copy of the neighbor
Cypher. Empty uid sets short-circuit without a DB round-trip.
Adds direct coverage (real bridge via writeBridge/openBridgeDbReadOnly) for
both directions plus the empty-set and unknown-uid edges.
* feat(group): cross-repo trace stitching (groupTrace + runGroupTrace)
Add GroupService.groupTrace and the pure runGroupTrace engine that stitches
per-repo CALLS/HAS_METHOD trace segments across one ContractLink boundary in
the group bridge:
from --(local trace)--> consumer --(ContractLink)--> provider --(local trace)--> to
- Resolves from/to across all members (symbol node id == bridge symbolUid);
same-repo endpoints delegate to a single local trace with no crossing.
- Single boundary crossing (MAX_SUPPORTED_CROSS_DEPTH); deeper crossDepth is
clamped with a note, mirroring cross-impact.
- Discriminated GroupTraceResult union (ok|not_found|ambiguous|error) with
per-hop repo tags, a typed crossings[] entry, and centralized degraded-state
note constants (TRACE_NOTES). No .
- Trace-specific pair query (keeps BOTH crossing endpoints) lives in this
module; the uid-filtered neighbor join (resolveBridgeNeighbors) is reused
where it fits. ensureBridgeReady exported for reuse.
- New GroupToolPort methods (trace/resolveSymbol/pdgFlows) are optional so
existing port mocks keep type-checking; runGroupTrace guards on presence.
PDG enrichment is wired as an opt-in hook (enrichSegment) — the port method is
stubbed until U4. Covered by unit tests over a real bridge + mocked port.
* feat(group): route trace tool to groupTrace on @group syntax
Wire the cross-repo trace through the existing @group dispatch:
- callTool routes trace with an @-prefixed repo to callToolAtGroupRepo, which
forwards from/to/uid/file/maxDepth/includeTests plus the experimental
pdg/crossDepth flags to GroupService.groupTrace. Member path in @group/path
is advisory for trace (resolution is whole-group).
- Port gains trace/resolveSymbol/pdgFlows adapters. resolveSymbolForGroup wraps
the shared resolveSymbolCandidates so groupTrace can locate the member repo
and recover each endpoint node id (== bridge symbolUid). pdgFlowsForGroup is
a degraded stub here (call-level only); U4 implements the REACHING_DEF walk.
- trace tool schema documents the @group entry point, pdg, and crossDepth.
Single-repo trace is untouched. Covered by dispatch-routing tests (@group ->
groupTrace, non-group stays local) and tool-schema assertions.
* feat(group): opt-in PDG data-flow enrichment for cross-repo trace
Implement _pdgFlowsForGroupImpl: the real REACHING_DEF anchor walk that backs
the port pdgFlows adapter (replacing the U3 call-level stub). When pdg:true and
the segment repo has a flows PDG layer, the boundary-adjacent segments carry
their intra-procedural def->use hops:
- Anchors by the boundary symbol UID (precise; avoids the by-name ambiguity the
resolveBlockAnchor path can hit), then reuses the same span-anchored,
bind-param-only flows query as pdg_query (BasicBlock id-prefix + [start+1,
end+1] line window; no rel-property index, so the anchor IS the bound).
- Stays intra-procedural: data flow never crosses the repo boundary.
- pdgStampForMode probe: false -> available:false (degrade with note); the
trace stays ok. Any query failure is swallowed (enrichment is auxiliary).
Covered by runGroupTrace enrichment tests: dataFlow attached on opt-in,
degraded note when no layer, and no pdgFlows call when pdg is omitted.
* test(group): evaluation-first cross-repo trace e2e (two real indexes)
End-to-end gate for the cross-repo trace: stands up two real LadybugDB indexes
(consumer 'frontend' + provider 'backend'), a real ContractLink bridge, and a
real LocalBackend with both repos registered, then drives the public
callTool('trace', { repo: '@grp', pdg: true }) and asserts:
- the stitched checkout -> callUsers -(CONTRACT_LINK)-> handleUsers -> getUsers
path, each hop tagged with its member repo
- real REACHING_DEF data-flow enrichment of the consumer segment (userId)
- a degraded 'No PDG layer in app/backend' note (provider has no PDG layer)
- single-repo trace against one member is unchanged (no crossings)
Hand-persists the minimal real graph (deterministic; a full two-repo analyze is
heavier than this gate needs) and exercises real Cypher across
resolveSymbolCandidates, _traceImpl, the bridge pair query, and
_pdgFlowsForGroupImpl. Windows-skipped (describeReopen) and registered in the
cross-platform native-lbug set.
Scoped to a single @group call: opening bridge.lbug read-only a SECOND time in
one process currently fails (shared bridge open/close lifecycle, also affects
impact @group) — the pdg-omitted/clamp variants are unit-covered.
* docs(group): document cross-repo trace + PDG enrichment
ARCHITECTURE.md: trace is now group-aware; describe the @group cross-repo
stitch over a single ContractLink boundary (CONTRACT_LINK hop, crossings[],
crossDepth clamp), the opt-in experimental PDG REACHING_DEF enrichment of
boundary-adjacent segments, the symbolUid-grain join between the two stores,
and the deferred full cross-program (SDG-like) data flow. PIPELINE.md: add the
cross-trace consumer of the bridge with its pair-query rationale.
Does not touch gitnexus/CHANGELOG.md (release-owned).
* fix(review): apply autofix feedback
Apply safe_auto findings from ce-code-review (run 20260622-094243):
- local-backend.ts: drop (r: any) in _pdgFlowsForGroupImpl row map; coerce
hop line via Number() so a nullish LadybugDB cell can't surface NaN.
- tools.ts: advertise the forwarded param in the trace schema and add
crossDepth maximum:10 (schema now matches what groupTrace reads).
- cross-trace.ts: parallelize per-member resolveSymbol/resolveRepo with
order-preserving Promise.all (matches groupContext/groupQuery); add a note
when pdg:true is passed to a same-repo trace (PDG only enriches at a
cross-repo boundary).
- tests: remove / tighten (no-any rule).
Residual gated_auto/manual findings (unbounded crossing query + loop,
whole-file PDG widening on absent span, error-vs-no_path masking, top-level
try/catch parity, helper dedupe, branch-coverage gaps) are recorded in the run
artifact for the PR body.
* fix(group): skip CHECKPOINT on read-only bridge close so it can reopen
Root cause of the in-process bridge.lbug reopen failure (which broke repeated
@group impact/trace calls in a long-lived MCP server): closeBridgeDb issued
CHECKPOINT on EVERY handle, including read-only ones. A CHECKPOINT on a
read-only connection has nothing to flush but leaves a WAL/shadow lock artifact
that makes the next read-only open of the same path fail (openBridgeDbReadOnly
returns null -> 'Could not open bridge.lbug read-only'). Reproduced: open ->
query -> closeBridgeDb -> open again returned null only when the close ran
CHECKPOINT; a non-checkpoint close reopened fine, and the raw native
open/close cycle was never the problem.
Fix: tag read-only handles (BridgeHandle._readOnly, set by openBridgeDbReadOnly)
and skip CHECKPOINT for them in closeBridgeDb. Writable handles are unchanged
(they still flush before close). This is the shared bridge-db close path, so
impact @group benefits identically.
- Regression test in bridge-db.test.ts: open/query/close/open/query/open in one
process now succeeds.
- Re-enabled the second @group call in cross-trace-e2e.test.ts (was scoped to a
single call for this very limitation).
* fix(group): bring bridge-db close to parity with the core adapter safeClose
The bridge open/close cycle was less robust than the main graph DB's: closeBridgeDb
closed the connection/database but skipped the post-close steps the core adapter's
safeClose performs, so a rapid in-process reopen could race the OS handle release
(Windows) or an orphaned WAL sidecar. That gap is why the close-then-reopen tests
had to skip Windows.
closeBridgeDb now mirrors safeClose after closing the handle:
- waitForWindowsHandleRelease(dbPath): probe the file (+ .wal) until the residual
Windows lock clears, so the next open does not race (warns if the budget is
exhausted, matching the core adapter).
- finalizeLbugSidecarsAfterClose(dbPath): quarantine an orphaned WAL (shadow
missing) so the next open replays a consistent file.
Both helpers are the same ones safeClose uses (Windows-proven via the core adapter
CI), and the bridge read open already retries transient locks. Combined with the
read-only CHECKPOINT skip, the bridge reopen is now robust on every platform, so
the close-then-reopen tests run on all platforms (Windows CI exercises them via the
cross-platform subset). No write-path behavior change; Linux/macOS unaffected.
* fix(group): bound cross-repo crossing fan-out (LIMIT + segment memoization)
Address the top review residual: the bridge crossing query was unbounded and the
crossing-selection loop could run an O(2*N) sequential trace-BFS over every
ContractLink between a repo pair.
- CY_CROSSINGS_BETWEEN now ORDERs BY confidence DESC and LIMITs to
MAX_CROSSINGS_TO_TRY + 1; listCrossingsBetween slices to the cap and reports
truncation. Exceeding the cap surfaces a note (no silent truncation), keeping
the highest-confidence crossings. Aligns with the repo's anchored+LIMIT-bounded
query discipline (LadybugDB has no rel-property index).
- The home-repo segment (from -> consumer) depends only on the consumer uid and
the target-repo segment (provider -> to) only on the provider uid, so each is
memoized by that uid. Many crossings sharing a consumer/provider (one client
call linked to several providers) now cost one trace per distinct endpoint
instead of one per crossing. A consumer whose segment already failed is skipped
for every later crossing that shares it.
Test: two links sharing a consumer (first provider unreachable, second reachable)
assert the from->consumer segment is traced exactly once and the second crossing
wins.
* fix(group): restore Windows skip for bridge reopen tests; drop ineffective close-side probe
The previous commit flipped the bridge close-then-reopen tests to run on Windows,
betting that a close-side waitForWindowsHandleRelease + finalizeLbugSidecarsAfterClose
probe (mirroring the core adapter safeClose) would make the in-process reopen work
there. Windows CI proved otherwise: 4 writeBridge->openBridgeDbReadOnly tests fail
('expected null not to be null' — the read open returns null). The writable-close ->
read-open handoff plus writeBridge's atomic sidecar rename does not release the OS
file handle before the read open races, and the existing open-side LBUG_OPEN_RETRY
only retries lock-pattern errors, not the post-rename sidecar database-id mismatch.
macOS passes; the core adapter's own reopen also passes — this is bridge+Windows
specific.
- Revert itLbugReopen to the Windows skip (the pre-existing, correct state).
- Remove the close-side probe + finalize from closeBridgeDb: it did NOT close the
Windows gap, and reviewers flagged it for hot-path latency (finalize ran on every
close, all platforms) and safeClose duplication.
- KEEP the load-bearing fix — skipping CHECKPOINT on read-only handles — which fixed
the reproduced Linux/macOS in-process reopen artifact (the real bug).
Net: Linux/macOS repeated @group impact/trace works in-process; Windows in-process
bridge reopen remains a documented limitation (unchanged from before this PR).
* fix(group): surface degraded members + cap truncation; honest crossDepth schema
Address the cross-engine-corroborated tri-review findings (Codex + Claude):
- resolveAcrossMembers / runGroupTrace now track member repos that could NOT be
queried (resolveRepo or resolveSymbol threw) and, when the result is not_found,
attach a degraded-member note. A transient/corrupt member DB is no longer
silently reported as a clean 'symbol absent' not_found. (Codex B1+B3 + ce-reliability.)
- The cross-repo not_found now carries a programmatic truncated:true flag (and a
clearer suggestion) when the MAX_CROSSINGS_TO_TRY cap was hit, so a consumer can
distinguish 'no path' from 'cap may have hidden a connecting ContractLink'.
(Codex B3 + ce-adversarial + ce-api-contract.)
- trace tool schema: crossDepth maximum 10 -> 1 to match the implementation's
single-hop clamp (the schema previously advertised an unsupported 2-10 range).
(ce-api-contract, conf 100.)
Test: a member whose resolveSymbol throws yields not_found WITH a degraded note
naming the unreachable repo (if-free responder map).
* docs(group): clarify trace @group/memberPath is advisory (resolves all members)
Tri-review (Codex ce, conf 100) caught a doc/impl inconsistency: ARCHITECTURE.md
lumped trace with query/context/impact as honoring @group/memberPath member
scoping, but cross-repo trace resolves from/to across ALL members (the member
path is advisory). Clarify the behavior and point to from_uid/to_uid for
disambiguating same-named symbols across members.
* feat(group): file-level boundary fallback so cross-repo trace works on HTTP contracts
Benchmark (bench/cross-repo-trace/) running the REAL pipeline (runFullAnalysis
--pdg -> real syncGroup -> trace @group) found that cross-repo trace returned
not_found for real HTTP links even though sync built the correct ContractLinks:
HTTP (and other source-scan) contracts hardcode symbolUid:'' (http-route-extractor),
and both cross-trace AND cross-impact join crossings by Contract.symbolUid, which
never matches an empty uid. (Pre-existing — impact @group has the same gap.)
Fix: when a crossing's symbolUid is empty, fall back to the contract's FILE — if
the user's from/to resolves into the contract file, that endpoint anchors the
boundary. CY_CROSSINGS_BETWEEN now returns consumer/provider filePath; a crossing
is kept if it can be anchored by uid OR file on each side; a fileBoundaryFallback
note flags that the boundary is file-level, not symbol-precise. This makes the
common 'trace from=<calling fn> to=<handler fn>' case work end-to-end (verified:
fetchUsers -> listUsers stitches with a CONTRACT_LINK hop + PDG enrichment, 2/2).
Limits (documented in the bench README + the note): anonymous handlers have no
named target; when several contracts share files the file fallback may attach the
wrong contractId to a correct path. The proper upstream fix is to populate
symbolUid in the HTTP extraction (benefits impact too) — the bench is its gate.
Adds a unit test pinning the empty-symbolUid file-fallback stitch.
* fix(group): resolve HTTP contract symbolUid by containment (fixes cross-repo trace + impact)
Addresses the root cause behind the cross-repo trace file-fallback: HTTP
contracts hardcoded symbolUid:'' (http-route-extractor), so both cross-trace and
cross-impact — which join crossings on Contract.symbolUid — could not traverse
HTTP links. (Also found: the pre-existing graph-assisted resolution queried the
wrong edge, CONTAINS instead of DEFINES, so it never resolved a uid either.)
Now the extractor resolves each detection to a real symbol:
- HttpDetection carries the call-site line (node.ts sets it on every express/
fetch/axios/jquery/nest detection; express also captures the handler arg).
- resolveDetectionSymbol resolves the named handler first, else the innermost
Function/Method whose line span encloses the call (consumer = the function
containing the fetch; provider = the named/inline handler), over the correct
File-[DEFINES]->symbol edge. Base-tolerant (0- vs 1-based startLine).
- Wired into both source-scan and graph-assisted provider/consumer paths.
Verified end-to-end (bench/cross-repo-trace): all 4 contracts now carry real
uids, trace is symbol-precise (GET pair -> http::GET, POST -> http::POST, no
file-fallback note), and impact @group fans out (cross_repo_hits 0 -> 1). The
cross-trace file-level fallback remains as the secondary path for truly
anonymous handlers. Adds 2 containment unit tests; 738 group/integration pass.
Languages other than JS/TS still resolve providers by handler name; their
consumers fall through to the file fallback until their plugins set the line.
* fix(group): extend HTTP symbolUid containment to all languages + nested methods
Completes the symbolUid resolution across every bundled HTTP plugin: Python, Go,
PHP, Kotlin and Java now set the call-site line on their consumer (and Feign/
named) detections, so their HTTP contracts resolve to the containing function
the same way Node/TS already did.
Also generalizes the containment query: it now matches Function/Method/CodeElement
by filePath (UNION ALL) instead of File-[DEFINES]->symbol. The DEFINES edge only
reaches a file's TOP-LEVEL symbols, so methods nested in classes (Java/Kotlin —
File defines the class, the class defines the method) were invisible; matching by
filePath reaches them. Verified against a real index (LadybugDB supports the
UNION); JS/TS still fully symbol-precise (bench 2/2), 709 group tests pass.
Residual is now only the inherent case — a fully anonymous handler with no named
callee — which keeps the cross-trace file-level fallback.
* feat(group): destination trace — follow a consumer to an anonymous handler
Handles the one inherent residual: an anonymous route handler
(`router.get('/x', (req,res) => …)`) has no symbol node at all (the file holds
only a Const + PDG BasicBlocks), so it can never be named as a trace `to`.
Adds a DESTINATION TRACE: omit to/to_uid/to_file on an @group trace and
`trace from=<consumer>` follows the consumer's outgoing HTTP call across the
bridge and reports where it lands — by route + file:line, with a notes[] entry
flagging the handler as anonymous. Implemented as a new branch in runGroupTrace
(p.destination) backed by CY_CROSSINGS_FROM (all ContractLinks leaving the
consumer repo) + stitchToDestination; the provider endpoint is labelled
'<METHOD /path handler>' when its symbolName is a generic token/file basename.
The MCP routing already omitted an absent `to`, so only the schema docs changed.
parseTraceParams now treats a missing `to` as a destination trace instead of an
error. Verified end-to-end: anonymous fixture reports
'app/frontend:fetchUsers -> app/backend:<http::GET::/api/users handler>'; named
fixture lands at the real function. Adds 2 unit tests; 915 group tests pass.
* fix(group): tri-review fixes for cross-repo trace + symbolUid resolution
Two-engine tri-review (Claude swarm+ce + Codex GPT-5.5 swarm+ce+adversarial)
surfaced these; cross-engine-corroborated unless noted.
Correctness (P1, all four lanes): destination trace reported the WRONG endpoint
— an empty-uid consumer made trace(from->from) trivially succeed, so the highest-
confidence same-file crossing won regardless of which call `from` makes.
stitchToDestination now collects ALL connecting crossings, prefers symbol-precise
hits, and returns `ambiguous` (with candidates) when it cannot disambiguate.
Correctness (P1, Codex): resolveDetectionSymbol early-returned null when
d.line==null, blocking NAME resolution for named providers that set no line
(Spring/Go/etc.). Name resolution now runs first; only containment needs a line.
Correctness (P2): resolveContainingSymbol OR-ed `line` and `line-1`, which could
mis-pick a one-line sibling. It now probes the base-correct `line-1` first and
falls back to `line` only if nothing matches.
Correctness (Codex): anonymous Express handlers emitted name:'handler' and could
attach to an unrelated fn literally named `handler`. node.ts now emits name:null
for non-identifier handlers (containment-only).
Robustness: drop the first-symbol-in-file pickSymbolUid guess from the graph
consumer/provider paths (a wrong uid would win the contractId merge); remove the
dead CONTAINS_QUERY fallback (CONTAINS is File->Folder, never a symbol) + the now
-unused pickSymbolUid/handlerName; seed destination notes with degraded-member
notes so a successful trace still surfaces them; providerLabel takes providerUid
so a resolved fn named `handler` is not mislabeled anonymous, and only true file
basenames (known extensions) — not any dotted name — count as anonymous.
API contract: a single-repo trace with no `to` now returns an actionable error
(destination trace is @group-only) instead of "symbol 'undefined' not found".
Maintainability/tests: narrow asLocalTrace per-field (drop as-unknown-as); fix the
PR's lone as-any (vi.mocked); if-free e2e teardown; qualify the bench README.
Adds ambiguous-destination, anonymous-handler-no-false-name, and single-repo-no-to
tests; redirects graph mocks CONTAINS->UNION ALL. 918 group/integration pass.
* fix(group): carry degraded-member notes through SUCCESSFUL group traces
A reviewer (koriyoshi2041, PR #2269) correctly flagged that degraded-member
resolution was surfaced only on not_found, not on a successful ok result. Group
trace resolves names across ALL members, so an ok is 'unique among the members
we could query' — if a member that threw during resolveSymbol also holds from/to,
the real answer could be ambiguous. The destination path already seeded the note
(prior commit); this extends it to the same-repo and cross-repo success paths by
seeding the dispatch notes with degradedNotes([...fromRes.degraded, ...toRes.degraded]).
Adds a regression test: reg-be throws while a same-repo trace succeeds in reg-fe;
the ok result now carries the 'could not be queried' degraded note (app/backend).
* test(bench): cover all implemented cross-repo trace cases in one runner
Replace the single named-handler script with a self-contained verify.mjs that
generates each fixture inline and exercises every implemented end-to-end case
against the real analyze -> sync -> trace/impact pipeline, asserting PASS/FAIL
(exit non-zero on failure). 10 checks across 4 scenarios:
- named handlers: 4/4 symbolUid resolved; symbol-precise GET vs POST crossing
selection; destination trace lands at the named handler.
- anonymous handler: empty symbolUid; destination trace reports it by route with
the anonymous note.
- impact @group fan-out (cross_repo_hits >= 1).
- multi-language (Python Flask + requests): link built, cross-repo trace stitches,
and the file-level boundary fallback is exercised when the provider has no uid.
Ambiguous-destination and degraded-member paths need synthetic inputs the real
analyzer cannot produce, so they stay in the unit suite (documented in the README
+ script header). Removes verify-named.mjs + fixtures-named/ (folded inline).
* test(group): pin destination degraded-success + precise-tier ambiguity
Adds the two regression guards koriyoshi2041 requested on PR #2269 after the
degraded-on-success fix:
- destination trace success with a degraded member: reg-fe resolves from and
follows the link to an anonymous handler while reg-be throws; the ok result
carries the anonymous endpoint AND the 'could not be queried' degraded note, so
the no-to path stays aligned with explicit to traces.
- multiple PRECISE destination hits: one from reaches two consumers with resolved
uids linked to different routes; the result is ambiguous (role: to) with both
route candidates. Distinct from the existing file-level ambiguous test, this
pins the stronger precise tier against a future change silently picking the
highest-confidence destination.
Both already pass against current behavior; 716 group tests pass.
|
||
|
Minidoracat
|
912285064a
|
perf(hooks): cmdline-first Linux db-lock scan, drop the lsof fallback (#2180) (#2183)
* perf(hooks): cmdline-first Linux db-lock scan, drop the lsof fallback (#2180) The probe's Linux scan was O(processes × fds) — stat every fd of every process — so on a busy host it blew its budget and fell through to lsof, which then timed out (~2 s) and fail-closed. Every Grep/Glob/Bash hook spent ~2 s of CPU to conclude 'couldn't tell'. Rewrite linuxProcScanFindGitNexusServer (name kept; return type now tri-state 'owned' | 'not-owned' | 'timeout') as three phases: 0. /proc/<pid>/comm prefilter — kernel task->comm, never touches the target's memory maps; truncation-safe whitelist match (comm is capped at 15 visible chars). Calibrated to what a real server reports: @ladybugdb/core's worker_threads rename the main thread to 'MainThread', so that is whitelisted alongside the launcher basenames — omitting it would blind the probe to every server. 1. bounded /proc/<pid>/cmdline read (openSync+readSync, default 16 KiB with a floor of 4 KiB and a bounded escalation up to a hard ceiling) so a D-state holder cannot stall the hook and the mcp/serve mode token is never clipped off a long interpreter path. 2. dev+ino fd match for the 0–2 survivors only. Dispatch: 'owned' and 'timeout' both map to true. Timeout is now fail-closed (overload self-throttle) instead of falling through to lsof; the Linux lsof fallback is removed entirely. End-to-end semantics on busy hosts are unchanged (the old lsof arm also fail-closed there) — the ~2 s of wasted work and the orphan-spawning lsof are what's gone. macOS lsof+ps and Windows Restart Manager paths are untouched. Also: fix the budget parse bug (Number(raw && trim()) treated '0' as 1200; now parseInt-then-validate, with <= 0 an explicit immediate timeout) and add GITNEXUS_HOOK_PROC_ROOT so the Linux scan can be unit tested against a fixture procfs instead of the host's real /proc. Measured on a 583-process host with 6 background gitnexus mcp servers: owner detection 6–12 ms (was ~1216 ms + lsof timeout), ~100x. Tests: new hook-db-lock-probe.test.ts drives all three phases against a fake procfs (comm-truncation safety, Phase 0 trap, 4 KiB-boundary owner-miss guard, budget=0 immediate timeout, EACCES fail-closed) plus a live-/proc e2e that pins the fd-visible lbug-handle property against a real subprocess holder. The lsof/ps owner-detection suites are relaned to macOS (Linux no longer takes that path); the lsof orphan-reaping suite is removed (no lsof is spawned on Linux now) with a rationale note. Note: pre-commit typecheck skipped; remaining tsc errors are pre-existing on main (none in files touched here). * fix(hooks): honest EACCES verdict + real escalation coverage (#2183 review) Addresses the tri-review (maintainer + Codex): - [P2] Phase-2 fd-dir EACCES no longer claims 'owned'. /proc/<pid>/fd is owner-only (0500), so a cross-user/root gitnexus server serving ANY repo cleared Phase 0+1 and hit EACCES here, and the old catch returned 'owned' — falsely claiming it locks THIS repo's lbug (dev+ino never compared) and permanently suppressing augment. Split the failure shapes: ENOENT -> continue (raced away); EACCES/EPERM and transient EIO/ESTALE -> 'timeout' (unverifiable -> fail-closed, but honest, not a false ownership claim); ENOTDIR/other structural errors -> continue (not a real fd dir). Same fail-closed dispatcher outcome, no false 'owned', plus a GITNEXUS_DEBUG diagnostic so an operator can tell this skip path from a real owner. - [P2] The escalation test now actually iterates the escalation loop: the gitnexus token sits under 4 KB while the mode token is padded past GITNEXUS_HOOK_PROC_CMDLINE_MAX=4096, and a readSync spy asserts >1 read (the old 9 KB-under-16 KB-cap shape read once and never escalated). - escalation loop now re-checks the budget each iteration and returns a distinct timeout sentinel (never '' — an empty string would read as 'not a candidate' and could drop a real owner -> fail-open); the caller maps it to 'timeout'. - GITNEXUS_HOOK_PROC_ROOT is gated to test context so a stray production env export can't disable Linux owner detection (fail-open). - New uid-agnostic spy tests pin every fd-readdir errno branch (EACCES/EPERM/EIO/ESTALE -> timeout, ENOTDIR -> not-owned) regardless of the runner's uid (the disk chmod-000 tests no-op under root). Note: pre-commit typecheck skipped; remaining tsc errors are pre-existing on main (none in files touched here). * fix(hooks): drop the always-true outOfBudget presence guard (CodeQL #2183) CodeQL flagged `typeof outOfBudget === 'function' && outOfBudget()` as unneeded defensive code: readLinuxCmdline has a single caller (linuxProcScanFindGitNexusServer) that always passes the callback, so the typeof guard is dead. Drop it, leaving `if (outOfBudget())`, and note the invariant in the comment. Mirrored in the byte-identical plugin copy. * fix(hooks): parse numeric hook env with Number() so scientific notation works (#2183 review) getCmdlineMaxBytes and resolveLinuxProcBudgetMs parsed their env via Number.parseInt(raw, 10), so a value like "16e3" silently became 16 (parseInt stops at 'e') instead of 16000. Switch both to Number(String(raw).trim()), which honors scientific notation and is stricter on trailing garbage ("123abc" -> NaN -> default) — matching the repo-majority Number()+isFinite env idiom (src/cli/analyze.ts, src/core/embeddings/hf-env.ts). The two functions had DIFFERENT guard skeletons, so a verbatim swap would regress the budget: resolveLinuxProcBudgetMs used `raw != null ?` with no empty-string short-circuit, and Number("")===0 (vs parseInt("")===NaN) would make a set-but-empty GITNEXUS_HOOK_LINUX_PROC_BUDGET_MS="" resolve to budget 0 => immediate fail-CLOSED timeout => augment permanently skipped. Added the `&& String(raw).trim()` guard so ''/whitespace fall to the 1200 default while "0" still parses to the deliberate #2180 immediate-timeout vector. Exported both helpers for white-box tests (the values are otherwise only observable indirectly through scan timing) and added platform-independent coverage: "16e3"->16000, ""/whitespace->1200 (the regression guard), "0"->0, "123abc"/unset->1200, cmdline "8e3"->8000, "2e3"/""/unset->16384. Both byte-identical hook-db-lock-probe.cjs copies updated together. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(hooks): allocUnsafe the per-chunk cmdline read buffer (#2183 review) readLinuxCmdline allocated each per-chunk read buffer with Buffer.alloc(chunkCap), zero-filling memory that readSync immediately and fully overwrites. Switch the hot read buffer to Buffer.allocUnsafe — safe because readSync initializes exactly [0, bytes), only buf.subarray(0, bytes) is consumed, and Buffer.concat deep-copies that slice into `collected`, so the uninitialized tail can never reach the decoded cmdline. The zero-length `collected = Buffer.alloc(0)` is left unchanged (allocUnsafe gains nothing on a 0-length buffer). The existing D3 multi-chunk decode tests cover the read path and stay green. Both byte-identical hook-db-lock-probe.cjs copies updated together. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(hooks): harden the live /proc owner-detection e2e against CI flake (#2183 review) Two flake mechanisms, fixed without weakening what the e2e proves: - Holder readiness (the genuine false-FAIL): the pid-file poll was 200x25ms=5s; a loaded runner can be slow to spawn the child, tripping expect(holderPid).toBeGreaterThan(0). Widened to ~10s and raised the per-test timeout 20s -> 40s. - Scan budget (kept the assertion honest): the live scan ran at the default 1200ms. Because the dispatcher maps a budget 'timeout' to owned=TRUE, a busy host exhausting 1200ms before reaching the holder would make the assertion pass for the WRONG reason (a hollow timeout, not real fd-visible detection). Set a generous explicit 10000ms budget via the existing setEnv() helper so the module afterEach restores it (replacing the raw `delete process.env...` that bypassed env tracking). Raised the coarse timing regression guard to sit ABOVE the budget (5000 -> 15000) so a legitimately-slow-but-correct scan can't trip it. The load-bearing asserts (dev+ino fd-visibility precheck, owned===true for our own lbug) are unchanged. Verified the e2e executes (not skipped) on Linux. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(changelog): empty the root CHANGELOG [Unreleased] section Per maintainer request, nothing should sit under [Unreleased] in the root CHANGELOG.md (the release-owned changelog is gitnexus/CHANGELOG.md, whose [Unreleased] is already empty). Removes all three accumulated blocks — Fixed (#2163), Performance (#2180), Changed (KuzuDB->LadybugDB) — leaving only the [Unreleased] header above [1.5.3]. Pure removal; no release sections touched. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Gergő Magyar <gergomagyar@icloud.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Gergő Magyar
|
2870aa6248
|
fix(grammars): load vendored tree-sitter grammars from vendor/ by absolute path (#2111) (#2144)
* fix(grammars): load vendored tree-sitter grammars from vendor/ by absolute path (#2111) The recurring Windows `EPERM: operation not permitted, symlink` (errno -4048) when adding the MCP server to Antigravity is NOT the #2101/#2110 module-load crash — it is an install-time arborist failure during the `_npx` reify that the MCP client triggers on every `npx gitnexus` launch. Root cause: the `postinstall` materialize step copied each vendored grammar (`vendor/tree-sitter-{c,dart,proto,swift,kotlin}`) into `node_modules/gitnexus/node_modules/tree-sitter-*` as a real package so runtime `require('tree-sitter-dart')` would resolve. Those packages are in no dependency graph, so every subsequent npm/npx reify treats them as **extraneous** and prunes/relocates them — on Windows the relocation goes through `@npmcli/move-file`'s symlink path and throws EPERM (symlinks need Developer Mode/admin), and on every OS the 2nd run silently deletes the grammars. This is the same class as #1728, which the materialize step itself claimed to have fixed. Fix (the prebuildify + node-gyp-build ecosystem pattern): never copy grammars into node_modules. Load each by absolute path from `vendor/<name>` via the new `requireVendoredGrammar` helper — the grammar's own `bindings/node` runs `node-gyp-build(<dir>)` and loads the committed `vendor/<name>/prebuilds/ <platform>-<arch>/…` directly (all 5 ship all 6 tuples). vendor/ is inside the package but not a node_modules subtree, so arborist never sees the grammars and the reify is idempotent — no EPERM, no silent deletion. - new src/core/tree-sitter/vendored-grammars.ts (requireVendoredGrammar / vendoredGrammarDir / VENDORED_GRAMMAR_PACKAGES; VENDOR_ROOT stable in dev+dist) - route all consumers through it: parser-loader, parse-worker, grpc proto, include-extractor (C), http-patterns kotlin, cli optional-grammars probe - postinstall drops the materialize step; build-tree-sitter-grammars.cjs builds in-place under vendor/ (gitignored) and deletes materialize-vendor-grammars.cjs - tests + grammar-introspection helper load grammars from vendor/ too (single source of truth); new vendored-grammars.test.ts guards against reintroducing a bare `require('tree-sitter-<vendored>')` Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(grammars): throw on a non-vendored name in requireVendoredGrammar Drift guard (PR #2144 review, P3): validate the argument against VENDORED_GRAMMAR_PACKAGES and fail loudly on an unknown name, so the three grammar lists (package set / CLI probe / build registry) drifting out of sync surfaces as a clear error instead of a confusing absolute-path require miss. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(grammars): prepack guard against stray vendor/<g>/build/ shadowing prebuilds Publish hygiene (PR #2144 review, P2). Now that build-tree-sitter-grammars.cjs source-builds into vendor/<name>/build/, a stray build dir would ship in the tarball (files:["vendor"] overrides .gitignore/.npmignore) AND shadow the committed prebuild — node-gyp-build resolves build/Release before prebuilds/. assert-publish-grammar-coverage.cjs (prepack) now fails `npm pack` if any vendor/*/build exists (findStrayBuildArtifacts), with a clear `rm -rf` fix hint. Adds unit coverage for the new pure function. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(grammars): harden the #2111 no-bare-require regression guard PR #2144 review (P2). The guard regex missed dynamic import(), side-effect `import 'x'`, /subpath, and backtick loads, and only scanned src/. It now covers every node_modules-forcing form (single/double/backtick quotes, optional subpath), scans test/ too (excluding fixtures and the guard file itself), drops the `//`-substring false-negative (leading-comment-only heuristic), and adds a self-test asserting every load form is caught while prose mentions and tree-sitter-cpp are ignored. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(grammars): correct stale vendored-grammar comments PR #2144 review (P3). kotlin/query.ts called tree-sitter-kotlin an "optionalDependency" — it is vendored and loaded from vendor/ by absolute path (#2111). proto.ts now states its remaining `_require` is only for the real `tree-sitter` dependency, not a vendored grammar (which goes through requireVendoredGrammar). Comment-only; no behavior change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Gergő Magyar
|
cef63dd044
|
feat(install): toolchain-free tree-sitter via vendored prebuilds (#2113)
* feat(install): toolchain-free tree-sitter via vendored GitNexus-built prebuilds
Eliminate the C/C++-toolchain requirement at install for the at-risk grammars
(dart, proto, kotlin) by generating + vendoring native prebuilds, mirroring the
existing vendored tree-sitter-swift. The 10 grammars that already ship 6 upstream
prebuilds stay npm dependencies (toolchain-free AND dependency-review-tracked).
- .github/workflows/build-tree-sitter-prebuilds.yml: a registry-parameterized
workflow that builds {dart,proto,kotlin} x {linux,darwin,win32}-{x64,arm64}
prebuilds natively, validates each loads + parses on its arch, and opens a PR
vendoring them. A `guard` job gates the heavy matrix to run ONLY on dispatch
or a real grammar-version change — ordinary code PRs cost zero matrix minutes.
- dart/proto: prefer a committed prebuild; fall back to today's source build
when none matches (no behavior change until prebuilds are vendored).
- kotlin: vendor it (Swift parity) instead of compiling the third-party
optionalDependency from source at the user's install — supersedes #2110's
optionalDependency mechanism. The ~23 MB parser.c is NOT vendored (the
workflow builds from the published package); only node-types + bindings +
prebuilds are. Removed from optionalDependencies; lock regenerated; probe,
parser-loader note, README/.devcontainer docs, and the #2110 tests updated.
DO NOT MERGE until vendor/tree-sitter-kotlin/prebuilds/ is populated by the
build-tree-sitter-prebuilds workflow: until then Kotlin is unavailable (vendored
with no source-build fallback). dart/proto remain fully functional throughout.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(install): guard 6/6 N-API prebuild coverage for every grammar
Regression guard so a toolchain-less install can never silently lose a tree-sitter
language on a supported platform-arch:
- Vendored grammars (vendor/tree-sitter-*): every one MUST ship a loadable N-API
prebuild for all 6 tuples {linux,darwin,win32}-{x64,arm64}. Asserts the
napi_register_module_v1 entry symbol in each .node (cross-platform, no need to
run the binary). Currently RED for dart/proto/kotlin until the
build-tree-sitter-prebuilds workflow populates their prebuilds/ — this is the
must-fill-before-merge gate (swift already passes 6/6).
- npm-dependency grammars: asserts upstream ships 6/6 N-API too, catching a
future platform drop. tree-sitter-c is allow-listed at 4/6 (missing
linux-arm64/win32-arm64) pending #2116; the guard also fails if that gap is
silently closed (prompting allow-list removal).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(install): vendor tree-sitter-c at 0.21.4 with GitNexus-built prebuilds (#2116)
tree-sitter-c is the one grammar dependency upstream ships incomplete prebuilds
for (4/6 — no linux-arm64/win32-arm64), AND it is a REQUIRED grammar: its own
`install` (node-gyp-build) compiles from source when no prebuild matches and
exits non-zero, so on a toolchain-less ARM host `npm install gitnexus` HARD-FAILS
at the c step — during npm's dependency phase, before any GitNexus postinstall
runs (so a postinstall "supplement" can't help).
Fix: vendor c prebuild-only at the pinned 0.21.4 (Kotlin pattern), with all six
prebuilds GitNexus-cross-built, and drop it from `dependencies`:
- vendor/tree-sitter-c/ (bindings + node-types + manifest + prebuilds); build
probe scripts/build-tree-sitter-c.cjs; added to the build workflow registry
(kind 'npm' — built from c@0.21.4 source).
- materialize-vendor-grammars.cjs: c is REQUIRED, so it is always materialized,
even under GITNEXUS_SKIP_OPTIONAL_GRAMMARS (it needs no toolchain).
- Removed from package.json dependencies + lockfile (nothing else needs npm c —
tree-sitter-cpp's dep on c is dev-only and not installed). Preserves the #1242
ABI pin: vendoring 0.21.4 keeps the good ABI while closing the ARM gap.
- parser-loader note + the prebuild-coverage guard + a cli-commands assertion
updated; c moves from the npm-gap allow-list into the vendored 6/6 cohort.
Verified: tsc clean, 31 unit tests pass, c loads/parses; the guard is RED for
c/dart/proto/kotlin until the workflow populates prebuilds (the must-fill gate).
Closes the operational risk in #2116.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(ci): source-build fallback for vendored c/kotlin so CI is healthy pre-prebuilds
The vendored prebuild-only grammars (c, kotlin) had empty prebuilds/ until the
build-tree-sitter-prebuilds workflow runs, so they could not load in CI — and
C is hard-required by cross-platform tests (tree-sitter-languages/parsing on
ubuntu+macos+windows), which I cannot pre-build for macos/windows locally. The
robust fix is a source-build fallback that works on every CI runner (all have a
toolchain), mirroring dart/proto:
- Vendor the grammar source (binding.gyp + src/) for c and kotlin; their build
scripts now PREFER a committed prebuild (toolchain-free) and fall back to
`node-gyp rebuild` from the vendored source when no prebuild matches. Verified
both compile against the hoisted node-addon-api@^8 and the runtime loads.
- prebuild-coverage guard is now bootstrap-tolerant: a grammar that vendors its
source (binding.gyp) may have an incomplete prebuild set (the workflow fills
it); a prebuild-only grammar (swift) still must ship all six. Any present
prebuild must still be N-API. Guard goes green; it re-tightens per-grammar as
the workflow populates prebuilds.
- actionlint: silence a false-positive SC2016 (JS template literals inside the
single-quoted `node -e` validate block).
Note: kotlin's generated parser.c is large (~23 MB on disk; compresses heavily
in git). Once the workflow populates all six kotlin prebuilds, the source serves
only as the fallback and could be slimmed if desired.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(docker): re-materialize+rebuild vendored grammars after npm prune
`npm prune --omit=dev` in the gitnexus CLI image drops anything not in
package.json's dependency tree — including the VENDORED tree-sitter grammars
(materialized by postinstall, not declared deps) and their built bindings. The
`serve` image analyzes/parses repos at runtime, so re-run the grammar postinstall
after the prune (in the toolchain-equipped builder) to restore them. Load-bearing
for tree-sitter-c, a core REQUIRED grammar now vendored (#2116): as a former
dependency it survived prune; vendored, it would not. Also restores
swift/dart/proto/kotlin, which were silently pruned from the image before.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(grammars): unify tree-sitter-swift with the vendored-source build pipeline
Swift was the last grammar handled differently — it shipped only upstream
prebuilds, while c/dart/proto/kotlin vendor their grammar source and use a
prefer-prebuild -> source-build-fallback activation script. Vendor swift's
source so all five are handled identically (one uniform build path).
- vendor/tree-sitter-swift: add binding.gyp (win-hardened), bindings/node/
binding.cc, src/parser.c (ABI-14 default, ~18 MB), src/scanner.c, and
src/tree_sitter/ headers. The 6/6 prebuilds are retained. The legacy
parser_abi13.c alternate is intentionally not vendored.
- build-tree-sitter-swift.cjs: rewrite the prebuild probe into the dart-style
prefer-prebuild then source-build fallback (keeps the GITNEXUS_SKIP gate and
the never-exit-non-zero postinstall invariant).
- build-tree-sitter-prebuilds.yml: register swift (kind 'vendored'); add its
package.json to the version-gated pull_request paths and a validate snippet.
- prebuild-coverage guard auto-moves swift into the source-fallback cohort
(binding.gyp now present); refresh the stale "swift is prebuild-only" comments.
- tests: add build-tree-sitter-swift-probe.test.ts; fix the pre-existing
build-tree-sitter-kotlin-probe.test.ts breakage (it still asserted the old
probe strings after kotlin's dart-style conversion); assert swift's vendored
source in cli-commands.test.ts.
- docs: README / .devcontainer / kotlin vendor README — swift's prebuilds are
now GitNexus-cross-built from vendored source like the rest, not upstream-only.
Verified: swift source-builds against node-addon-api@8 -> N-API binary -> loads
against the pinned tree-sitter@0.21.1 (ABI 14) -> parses cleanly.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(publish): gate a lean prebuilds-only npm tarball behind a coverage guard
Vendoring grammar source (parser.c) alongside the prebuilds means the npm
tarball now carries ~50 MB of generated source it almost never compiles (every
supported platform-arch has a prebuild). Prepare to drop it from the published
package once all prebuilds exist — safely.
- .npmignore: add a GATED, commented-out "lean publish" block that excludes the
source-build inputs (parser.c/scanner.c/tree_sitter/binding.gyp/binding.cc) but
keeps prebuilds/ + the runtime files. Uncommenting ships prebuilds-only.
- scripts/assert-publish-grammar-coverage.cjs: a prepack guard that refuses to
pack/publish if the source exclusion is active while any vendored grammar still
lacks 6/6 prebuilds (which would ship a grammar with no loadable binding). Wired
into `prepack` (runs on npm pack + publish, incl. the publish.yml dry-run) and
exposed as `npm run assert-publish-coverage`.
- test: pure-core decision cases + a real-repo publish-safety check that fails CI
if .npmignore is activated prematurely.
Net: the prebuilds already publish today (files: ["vendor"]); this makes the
future switch to a prebuilds-only tarball a one-line uncomment that can't ship a
dead grammar. The guard currently reports "source + prebuilds" (only swift has
6/6 prebuilds so far) and passes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(grammars): consolidate the 5 build-tree-sitter-*.cjs into one
The per-grammar activation scripts (c/dart/proto/swift/kotlin) were ~95%
identical — same prefer-prebuild → source-build → never-fail flow, differing only
in name, target_name, required-vs-optional, and the display label in warnings.
- scripts/build-tree-sitter-grammars.cjs: one registry-driven script. Bare call
builds all (postinstall); `... <name>` builds only the named grammars (so the
probe test can isolate one). c is `required: true` (ignores the opt-out gate);
the rest honor GITNEXUS_SKIP_OPTIONAL_GRAMMARS. Per-grammar try/catch + a final
process.exit(0) preserve the postinstall never-exit-non-zero invariant.
- package.json: postinstall is now `materialize && build-tree-sitter-grammars.cjs`
(was five chained `build-tree-sitter-<name>.cjs` calls).
- tests: replace the two near-identical *-probe.test.ts files with one
parameterized build-tree-sitter-grammars-probe.test.ts that also covers the
required-vs-optional opt-out split and an unknown-grammar arg.
- update cli-commands.test.ts postinstall assertions + the vendor c/kotlin/swift
README + swift provenance to reference the consolidated script.
Behavior is preserved (warnings normalized to one consistent format). Removes 5
scripts + 1 test file; adds 1 script + 1 test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(ingestion): lazy-load tree-sitter-c to prevent module-load crash
tree-sitter-c is now vendored prebuild-only (#2116) with 0/6 committed
prebuilds, so on a toolchain-less or `--ignore-scripts` install C has no native
binding. Three modules loaded it via a hard top-level `import C from
'tree-sitter-c'`, which throws ERR_MODULE_NOT_FOUND at module-load — crashing
`analyze` before parser-loader's optional/severity:error degradation can run.
This is the #2091/#2093 bug class (previously fixed for swift/dart/kotlin); C was
left static because it used to be an always-present npm dependency.
- languages/c/query.ts: load via the lazy guarded getLanguageGrammar(C), mirroring
swift/query.ts; the main-thread isLanguageAvailable filter ensures the getters
are reached only when C is present.
- workers/parse-worker.ts: guarded `_require('tree-sitter-c')` + conditional
languageMap spread, like swift/dart/kotlin.
- group/extractors/include-extractor.ts: guarded `_require`; getLanguageForFile
returns null for .c/.h when absent, so C include-extraction degrades to a no-op
(C++ unaffected).
- extend the registry-import-closure regression test (#2091/#2093) to assert C
also loads lazily at registry static-import time.
* fix(ci): repin attest-build-provenance to the real v2.4.0 SHA
The workflow pinned actions/attest-build-provenance@bd77c077… commented
`# v2.4.0`, but v2.4.0 is e8998f94… (verified via the GitHub API); bd77c077…
is an untagged mid-stream commit, so the SLSA-attestation step ran unvetted
action code and the comment misrepresented what runs. Repin to the real
v2.4.0 commit and drop the `# PLACEHOLDER-PIN` markers on both this line and
the setup-python pin (a26af69b… is already the correct v5.6.0 — only its
comment was stale). Update the header NOTE accordingly.
* fix(ci): skip the prebuild-PR aggregate when release App secrets are absent
The aggregate job mints a GitHub App token as its first step; with
RELEASE_APP_ID/RELEASE_APP_PRIVATE_KEY unset it hard-failed AFTER a full
(up-to-6-runner) native build. Since the `secrets` context isn't available in
a job-level `if:`, the guard job now computes a `release_app` boolean output
(a step can read secrets) and emits an actionable `::notice::`; aggregate
gates on it and skips cleanly, while the build job's artifacts still upload
(run with open_pr=false for artifacts-only).
* chore(ci): drop package-lock.json from the prebuild paths filter; widen build timeout
`gitnexus/package-lock.json` changes on nearly every dependency PR, so it
fired the prebuild workflow's guard job on unrelated churn (the matrix stayed
correctly skipped — `gitnexus/package.json` already covers the transition-window
pin, so removing the lock only drops guard noise). Also bump the native build
job timeout 30 -> 45 min for headroom compiling the 23 MB kotlin / 18 MB swift
parser.c, especially under arm emulation.
* fix(ci): event-gate the aggregate open-PR condition explicitly
`inputs.open_pr` is null on pull_request events, and the prior
`inputs.open_pr != false` leg relied on GHA's direction-ambiguous null
coercion (Codex F4) to decide whether to open the prebuild PR. Gate
explicitly on the event: a non-fork pull_request that bumped a grammar
version opens the prebuild PR (the documented flow), and `open_pr` is only
consulted on workflow_dispatch — so a manual run with open_pr=false stays
artifacts-only and no event's behavior rests on coercion.
* fix(publish): validate the effective npm-pack contents in the coverage guard
The publish guard inferred "is source shipped?" from a single .npmignore toggle
line, which a partial/out-of-order edit could defeat (exclude binding.gyp but
leave parser.c → unbuildable yet "source-shipping"). It now inspects the
EFFECTIVE tarball via `npm pack --dry-run --ignore-scripts --json` (the
--ignore-scripts avoids re-entering this guard through prepack): a grammar
"ships source" only when EVERY on-disk source-build input (binding.gyp +
binding.cc + parser.c + scanner.c when present + a tree_sitter header) is
actually in the packed file list.
This also surfaced that the gated lean-publish .npmignore block was inert:
package.json's `files: ["vendor"]` allow-list overrides .npmignore for the
vendored subtree, so those exclusion lines never dropped anything. Replace the
dead toggle with documentation of the real mechanism (narrow the `files` field)
and note the guard enforces safety on the effective pack regardless of how the
slim is done.
* test(prebuild): hard-gate declared-fully-prebuilt grammars on 6/6 coverage
The strict 6/6 prebuild assertion was dormant whenever a grammar vendors source
(binding.gyp) — which is every grammar — so a dropped prebuild passed CI
silently. Add a FULLY_PREBUILT allowlist of grammars GitNexus has committed 6/6
for (today: swift); those must keep all six even with a source fallback, so
losing one now fails CI. Grammars graduate into the set as the
build-tree-sitter-prebuilds workflow lands their binaries. (The static-import
degradation smoke is covered by the registry-import-closure regression test
extended in the C lazy-load commit.)
* chore(deps): promote node-gyp-build/node-addon-api to regular dependencies
Every vendored grammar's index.js does `require("node-gyp-build")` at runtime
to load even a prebuilt .node, so node-gyp-build is runtime-load-critical (and
node-addon-api is needed for the source-build fallback). They were
optionalDependencies, surviving `--omit=optional` only via the required
tree-sitter's transitive edge — correct today but fragile. Promote both to
regular dependencies so the contract is explicit (optionalDependencies is now
empty and removed). Lock the contract with a cli-commands assertion.
* chore(vendor): add Windows cflags parity block to tree-sitter-c/binding.gyp
c's binding.gyp used an unconditional `cflags_c: ["-std=c11"]`, while
kotlin/swift gate MSVC flags behind an `OS=='win'` condition (/std:c11 /utf-8).
Inert today (no non-ASCII bytes in c's parser.c, and node-gyp ignores cflags_c
on MSVC anyway), but align the three so a future source-build fallback on
Windows behaves consistently.
* docs(agents): correct stale optional-grammar / postinstall notes
AGENTS.md still said postinstall "patches tree-sitter-swift, builds
tree-sitter-proto" and that only kotlin/swift are "optional". Update to the
vendored-uniform model: postinstall materializes the vendored grammars and
prefers a committed prebuild (source-build only when none matches); c is
required while dart/proto/swift/kotlin are optional + skippable via
GITNEXUS_SKIP_OPTIONAL_GRAMMARS=1, with non-fatal warnings only on a
toolchain-less host with no matching prebuild.
* fix(install): preserve the backup and warn loudly on a failed materialize rollback
If renameSync(partial, dest) failed AND the rollback renameSync(backup, dest)
also failed, the grammar was left unmaterialized (node_modules/<name> missing)
with only a generic "could not materialize" warning — the recoverable backup at
<dest>.materialize-bak was unmentioned. Emit a CRITICAL warning naming the
backup path and the recovery command on that double-failure, and document that
the fail-soft catch removes only the scratch `partial`, never the `backup`
(which may be the sole recoverable copy). Never-throw / exit-0 contract intact.
* fix(publish): make the coverage guard's npm-pack inspection script-safe
The prepack guard shelled out to `npm pack --dry-run --ignore-scripts --json`,
but the `--ignore-scripts` flag is not reliably honored by npm pack's
prepare/prepack lifecycle on the CI npm — so build.js ran, polluted the --json
stdout with `[build] …`, and the guard's JSON.parse threw. That broke every
`npm pack` (packaged-install-smoke on ubuntu+windows) and failed the guard's own
real-repo unit test (the only coverage-job failure). Force script-skipping via
the reliable `npm_config_ignore_scripts` env config (also removes the prepack
re-entry/recursion risk) and parse defensively from the JSON-array start.
* fix(publish): make the coverage guard deterministic — read `files`, not `npm pack`
The npm-pack-based guard timed out in CI: `npm pack`'s prepare/prepack lifecycle
is not skipped by `--ignore-scripts` (flag or env config) on the CI npm, so the
inner pack ran the full build (~20s+) — fine for the slow smoke job, but it blew
past vitest's 30s test timeout in the coverage job (and risked re-entering this
prepack guard).
Replace it with a deterministic, fast (~0.1s) check that needs no subprocess:
since `files: ["vendor"]` OVERRIDES `.npmignore` for the vendored subtree (so
`.npmignore` can never drop vendored source — verified), the ONLY lever that can
exclude source is narrowing the package.json `files` field. The guard now reads
`files` directly: a grammar "ships source" iff `files` includes the vendor
subtree AND the grammar carries a buildable source set on disk. A lean publish
that narrows `files` while a grammar lacks 6/6 prebuilds still fails the gate.
* feat(ci): vendored tree-sitter grammar update monitor
Adds a weekly (+ dispatchable) workflow that checks each vendored grammar against
its source-of-origin (npm for swift/kotlin, the GitHub default branch for
dart/proto; c is excluded — held at 0.21.4 for ABI safety) and opens a PR
re-vendoring any update that is ABI-COMPATIBLE with the pinned tree-sitter@0.21.1
(LANGUAGE_VERSION 13-14).
ABI awareness is the point: most upstreams have moved to ABI 15 (newer
tree-sitter), so a blind "bump to latest" would open PRs that can't build. The
monitor fetches the candidate source, reads its parser.c LANGUAGE_VERSION, and
only re-vendors 13/14 — incompatible updates are reported (notice + job summary),
never applied. (Confirmed live: dart/proto upstreams are ABI 15 today and are
correctly held; swift/kotlin are current.)
The re-vendor refreshes only the source-build inputs + runtime entrypoints,
preserving the GitNexus-hardened binding.gyp / README / prebuilds; the version
bump then triggers build-tree-sitter-prebuilds.yml, whose ABI-validation is the
final safety net so a subtly-wrong re-vendor can't silently ship. PR creation is
gated on the RELEASE_APP secret (skips with a notice if absent), mirroring the
build aggregate. Unit test locks the ABI gate; the script is import-safe.
* feat(ci): monitor tree-sitter-c too (report-only, ABI-pinned)
c was excluded from the update monitor, so an upstream c update went unnoticed.
Include it, but as report-only via a `hold`: c is ABI-pinned at 0.21.4
(#1242/#858) and must not auto-bump without a tree-sitter runtime upgrade, so an
available c update is detected + surfaced (notice + job summary) but never
auto-PR'd — even if it were ABI-13/14. `--apply c` refuses defensively. (Live:
upstream c is 0.24.1 / ABI 15 today, so c is doubly held — reported, not applied.)
* fix(ci): drop the shell in the grammar monitor's github fetch (CodeQL)
CodeQL flagged the GitHub-tarball fetch — it used `bash -c "gh api …/tarball/$ref
> src.tgz && tar xzf src.tgz"`, interpolating the API-derived ref into a shell
command (the shell-command-injection family: "this shell command depends on an
uncontrolled file name"). Replace it with a shell-free path: capture `gh api`'s
binary tarball as a Buffer via execFileSync, write it to a fixed file, and
extract with execFileSync('tar', …). No shell, no injection surface. Verified the
dart/proto fetch + ABI read still work.
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
Gergő Magyar
|
f1151660b9
|
fix(install): graceful Kotlin optional-grammar install + accurate toolchain docs (#2110)
Some checks are pending
Devcontainer Smoke / Config-transform unit tests (push) Waiting to run
Devcontainer Smoke / Build devcontainer image (push) Waiting to run
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Scorecard / Scorecard analysis (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
* fix(install): document Kotlin optional-grammar toolchain behavior + graceful install probe tree-sitter-kotlin is a third-party npm optionalDependency that ships source-only (no upstream prebuilds) and compiles its native binding via node-gyp at install. It was the only optional grammar without a GitNexus install-time probe, and the README's GITNEXUS_SKIP_OPTIONAL_GRAMMARS "no toolchain needed" note omitted Kotlin entirely. This adds a fail-soft probe (mirroring the Swift one) that warns clearly and always exits 0 so install never breaks, wires it into postinstall, and corrects the optional-grammar docs in README.md and .devcontainer/README.md. Shipping prebuilt .node binaries (the literal request) needs an upstream/CI build matrix and is intentionally left as follow-up. Refs #2107 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: address PR #2110 tri-review findings (Kotlin optional-grammar install) Addresses the four P2 findings from the PR #2110 tri-review: - F1: docs no longer imply GITNEXUS_SKIP_OPTIONAL_GRAMMARS=1 skips Kotlin's toolchain. npm compiles tree-sitter-kotlin via its own node-gyp-build step regardless of that variable; point to `npm install --omit=optional` as the real lever (README.md + .devcontainer/README.md). - F2: the install probe now surfaces its "Kotlin unavailable" guidance on the dir-absent branch — the dominant toolchain-less case, where npm prunes the failed optional dependency so the package dir is gone at postinstall. Gated on npm_config_omit so a deliberate `--omit=optional` stays silent. Still never throws or exits non-zero. - F3: add a behavioral test that executes the probe across its skip / dir-absent-warn / dir-absent-omit-silent paths and asserts exit code 0 (guards the postinstall "never exit non-zero" invariant a static assertion cannot). - F4: reframe prebuilt Kotlin as deferred Swift-parity follow-up — GitNexus already vendors its own self-built Swift prebuilds and could do the same for Kotlin — tracked in #2107, not an upstream-only blocker. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Gergő Magyar
|
288b96f3e5
|
fix: batch query enrichment, bake FTS extension into CLI image, add FTS memory repro (#2108)
* perf(query): batch per-symbol process/cohesion/content lookups (N+1 -> 2-3) Port of the local-backend query-batching from gitnexus-enterprise PR #222 into the OSS local MCP backend. The query tool traced each matched symbol to its processes + cohesion (+ content) with up to 3N sequential pool round-trips; batch them into 2-3 'WHERE n.id IN $nodeIds' queries keyed back to each symbol by a prepended 'n.id AS nodeId' column. Output is identical: the aggregation loop is unchanged, iterates merged in the same order, and reads pre-fetched maps instead of issuing a query per symbol. Adaptations over a blind cherry-pick (would otherwise change output): - per-nodeId first-row community pick replaces the per-symbol LIMIT 1, so each symbol keeps its own community (not one for the whole batch); - batched rows regrouped to the originating merged item by nodeId so the JS-side RRF item.score still drives process ranking; - positional fallbacks shift +1 (process row[1..6], cohesion [1]/[2], content [1]); CodeRelation{type:...} relation form kept; IN-list chunked at 100 like the impact path. Adds a regression test asserting per-node community/content association (func:login keeps comm:auth; func:validate inherits no community). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(docker): bake LadybugDB FTS extension into the CLI/serve image The container runs `serve` under the default `load-only` extension policy (the read pool pins {policy:'load-only'}), so a runtime LOAD EXTENSION fts never INSTALLs. Dockerfile.cli copied the extension installer but never ran it, so the runtime user's HOME had no FTS extension: keyword search silently degraded (no FTS indexes written, ranking falls back to vector-only with only a warning field). Same class of footgun fixed for the Hub image in gitnexus-enterprise PR #222. Run install-duckdb-extension.mjs as the `node` user with the runtime HOME so INSTALL fts materializes the extension under $HOME/.lbdb/extension where the runtime LOAD resolves it offline. Pin ENV HOME=/home/node because Docker does not derive HOME from USER — without it the build-install and runtime-load would resolve different paths. Verified locally: INSTALL lands in $HOME/.lbdb/extension/0.17.0 and a fresh offline load-only `LOAD EXTENSION fts` resolves it. Dockerfile.web is unaffected (static frontend, no @ladybugdb backend). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(lbug): FTS evict->reload RSS repro + inert pool RSS tracing Settles the gitnexus-enterprise PR #222 root-cause hypothesis for OSS: does re-running LOAD EXTENSION fts on every pool evict->reload strand the native FTS arena (unbounded RSS growth in long-lived MCP serve), or does db.close() reclaim it (bounded by MAX_POOL_SIZE)? Static read could not decide — the native lbugjs.node binary documents no close->extension-unload contract. Adds gitnexus/scripts/bench/fts-evict-reload-rss.mjs: a NATIVE mode that reproduces the exact native sequence doInitLbug()+closeOne() perform (open Database -> Connection -> LOAD EXTENSION fts -> QUERY_FTS_INDEX -> close) across K self-built FTS fixtures, and a --via-pool mode that drives the real compiled pool (initLbug/executeParameterized/closeLbug) against an existing analyzed repo. Plus a behavior-neutral GITNEXUS_POOL_RSS_TRACE=1 stderr trace on pool init/close (stdout reserved for MCP JSON-RPC; single env read when disabled). RESULT (native, 24 and 40 cycles x 6 fixtures, --expose-gc): PLATEAU. RSS warms up to ~400 MB then flattens (40-cycle: +36 MB over cycles 1-10, +3 MB over 30-40; decelerating), not the linear climb a per-reload arena leak would produce (240 reloads x stranded arena = multi-GB). db.close() reclaims the FTS arena. The unbounded-leak hypothesis is NOT reproduced for the OSS path: the pool's LRU eviction + close-on-evict BOUNDS the footprint, which is exactly the protection the enterprise Hub supervisor lacked (it opened bridge DBs in-process without eviction -> 15 GB). => plan U4 (worker/process isolation) is NOT justified by this evidence; U1 + U2 are the only OSS-shared changes. Caveat: small fixtures + awaited close; a --via-pool run against a large analyzed repo over a long session is the production-faithful follow-up (instrumentation is in place for it). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(review): apply ce-code-review autofix feedback (#222 migration) Adversarial review found the U3 bench PLATEAU->no-leak conclusion was over-claimed from a 600-row fixture: a size-proportional FTS-arena leak would be sub-threshold at that scale. Strengthen the bench and make its verdict honest: - scale the fixture (--rows, UNWIND batch insert), probe ALL 5 FTS indexes in --via-pool (not 2 of 5), add a --no-await-close variant (the pool fire-and-forget close shape), and replace the absolute-delta gate with a SLOPE-DECELERATION 3-way verdict (PLATEAU / CLIMB / INCONCLUSIVE) plus step-discontinuity detection. At production-representative scale the synthetic runs are noisy/INCONCLUSIVE (deceleration argues against an UNBOUNDED leak but does not prove bounded), so plan U4 stays GATED on a --via-pool run against a real large analyzed repo -- not closed. - Dockerfile.cli: source the scratch-DB size from ENV GITNEXUS_LBUG_MAX_DB_SIZE (single source of truth) and add a build-time verify-only LOAD gate that fails the build on a HOME/extension-dir mismatch instead of silently degrading runtime keyword search. - install-duckdb-extension.mjs: additive verify-only mode (LOAD-only in a fresh process) + robust size parse; back-compatible with the runtime positional-size caller (validated). - tests: wire func:validate into a second process (proc:beta-flow) so the batched STEP_IN_PROCESS row[1..6] positional shift is exercised by a genuine multi-process symbol, and assert process ranking. No blast radius (75 seed-consuming tests pass). - pool-adapter.ts: trim the traceRss narrated-code comment (DoD 2.3). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(bench): classify a sustained sub-floor RSS slope as INCONCLUSIVE, not PLATEAU Tri-review P2: the FTS evict->reload verdict short-circuited to PLATEAU whenever secondHalfSlope < SUSTAIN_FLOOR, BEFORE the deceleration check — so a sustained (non-decelerating) linear leak below 0.5 MB/cycle was labeled PLATEAU ("no leak"), the label that would wrongly close plan U4. Extract median/slopeMbPerCycle/classifyVerdict into a pure, side-effect-free fts-rss-verdict.mjs (zero imports) so it is unit-testable without loading the native addon or running the bench, and fix the classifier: - epsilon-first gate: a truly flat tail (< 0.1 MB/cycle) is PLATEAU regardless of decelRatio (guards against over-correcting a real negative into INCONCLUSIVE); - a sustained sub-floor positive slope (>= epsilon, < floor, decelRatio >= 0.6) is INCONCLUSIVE — a slow creep RSS cannot distinguish from noise at this scale, so the honest label is "not resolved", never a clean PLATEAU; - the noise floor now scales with the WORKING-SET growth (peak-baseline), not the pre-DB baseline RSS (which is interpreter/addon overhead, larger in --via-pool mode, and would inflate the floor and HIDE leaks). Reconcile the stale "per-row-relative delta floor" docstring; add floor + decelRatio to the MACHINE line. New fts-rss-verdict.test.ts pins all label boundaries (flat->PLATEAU, sustained-sub-floor->INCONCLUSIVE, decelerated->PLATEAU, sustained-linear->CLIMB, step->INCONCLUSIVE, working-set floor, no import side effects). U1 does NOT add detection power for sub-floor leaks (RSS cannot attribute that magnitude) — it stops the false PLATEAU and routes that regime to the --via-pool run. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(query): signal partial/warning on a real enrichment failure (not benign missing-table) Tri-review P2: when a batched enrichment query (process/cohesion/content) threw, it was caught + logged and the chunk's symbols silently fell back to `definitions` with no signal — the caller could not tell "genuinely standalone" from "enrichment failed". Track an `enrichmentDegraded` flag in the three enrichment catch blocks and, at response build, compose a single `warning` (FTS-missing and/or the enrichment message, so neither overwrites the other) plus `partial: true`. Both fields are omitted on the clean path, so the success-path response shape is byte-identical. Crucially, the flag fires ONLY for a REAL failure (timeout / lock / native fault), NOT the benign "no Process/Community table" prepare error — a repo analyzed without processes/communities is a normal config, and firing `partial` on every such query would desensitize callers (isBenignMissingTableError gates it). New unit test test/unit/query-degraded-signal.test.ts (vi.mock pool-adapter, override hybrid search to feed one matched symbol, route STEP_IN_PROCESS -> throw): real failure -> warning+partial+symbol still returned; benign missing-table -> no signal; FTS-missing + enrichment failure -> both messages in one warning. Plus a success-path no-warning/no-partial assertion in the calltool integration test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Gergő Magyar
|
f2c9e69792
|
feat(ingestion): M0 — taint/PDG substrate (schema + seams + spikes) (#2080) (#2092) | ||
|
azizur100389
|
93e04b46d6
|
fix(lbug): load FTS in Windows read pool (#2040) | ||
|
Gergő Magyar
|
083aedbc41
|
refactor(ingestion): delete legacy call-resolution DAG + heritage processor (RING4-1, #942) (#2023)
* refactor(ingestion): delete legacy call-resolution DAG + heritage processor (#942) RING4-1: all 16 production languages (incl. Vue #940) are registry-primary, so the legacy resolution legs only ran under the now-removed CI parity gate. Calls and inheritance now resolve exclusively through scope-resolution (Registry.lookup, preEmitInheritanceEdges, emitHeritageEdges, buildMro → MethodDispatchIndex). Removed: - Call-resolution DAG: call-processor.ts legacy body (processCalls, processCallsFromExtracted, resolveCallTarget + all resolver/dispatch/chain helpers), model/resolve.ts MRO-via-HeritageMap, model/heritage-map.ts, type-env DAG types; inferImplicitReceiver/selectDispatch LanguageProvider hooks + Ruby impls; DispatchDecision/ImplicitReceiverOverride/ReceiverEnriched. - Legacy heritage path: heritage-processor.ts, heritage-types.ts, heritage-extractors/, @heritage.* tree-sitter queries, heritageExtractor/ heritageDefaultEdge/interfaceNamePattern wiring, worker + parse-impl heritage passes (parse-worker/parsing-processor lockstep), cross-file-impl DAG pass. - Scope-parity infrastructure entirely (no legacy↔registry parity left to run): scripts/run-parity.ts, scripts/ci-list-migrated-languages.ts, ci-scope-parity.yml, test:parity, and the scope-parity ci.yml gate. Resolver integration tests still run via the normal tests job. Kept (shared infra, NOT call-DAG-only): type-env.ts buildTypeEnv (field extraction / structure phase / embeddings), model/resolve.ts c3Linearize + gatherAncestors (mro-processor mroPhase), route/fetch/exported-type-map helpers in call-processor.ts, preEmitInheritanceEdges (legacy-edge dedup simplified). Acceptance: grep for resolveCallTarget/inferImplicitReceiver/selectDispatch/ buildHeritageMap/HeritageMap/processHeritage/heritageExtractor/@heritage. is zero across src + test. tsc clean (both packages); resolver integration suite green (bit-compatible EXTENDS/IMPLEMENTS/CALLS); scope-capture fingerprints unchanged (python re-baselined: removed redundant ignored captures). ARCHITECTURE.md updated to scope-resolution-only. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(review): apply autofix feedback (#942) ce-code-review autofix pass on the RING4-1 deletion: - parse-cache.ts: bump SCHEMA_BUMP 2→3 — ParseWorkerResult lost its `heritage` field, so stale on-disk caches must invalidate (prevents a rollback replaying a heritage-less cache into legacy code) [api-contract P2]. - parse-impl.ts: drop 3 now-unused type imports (ExtractedCall, ExtractedAssignment, FileConstructorBindings) left by the deferred-block removal — would fail the eslint CI gate [correctness+maintainability P1]. - AGENTS.md / CLAUDE.md / scope-resolver.ts contract doc: fix stale pointers to the deleted "§ Call-Resolution DAG" section + removed hooks; preserve the language-neutrality rule [project-standards P1]. - registry-primary-flag.ts / cross-file.ts / parse-impl.ts: refresh stale comments referencing deleted symbols (legacy DAG, runCrossFileBindingPropagation). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(ingestion): remove the vestigial isRegistryPrimary flag (#942) With the legacy call-resolution DAG deleted, the per-language `REGISTRY_PRIMARY_<LANG>` / `isRegistryPrimary` / `MIGRATED_LANGUAGES` flag had only one meaningful state — every production language resolves via scope-resolution — and an explicit `=0` override could only *disable* resolution with no fallback (a footgun the review flagged). Removing it. - Delete `registry-primary-flag.ts` and the now-dead `shadow-harness.ts` (legacy↔registry shadow-parity tool) + its test. - Collapse the three flag gates to their behavior-preserving outcome (`SCOPE_RESOLVERS == MIGRATED_LANGUAGES`, so this is a no-op): - scope-resolution phase now runs for every registered `SCOPE_RESOLVERS` entry (was `∩ MIGRATED_LANGUAGES`). - import-processor `addImportGraphEdge` + parse-impl `shouldAccumulate`: the legacy emit/accumulate paths were already inert for migrated languages (scope-resolution owns IMPORTS via the imports-to-edges bridge); drop the flag term. - Collapse flag-branching tests to the scope-resolution path and delete the csharp legacy-`=0`-leg describe blocks; remove the ruby/rust-scope env-forcing hooks (no-ops now). - Refresh docs/comments (ARCHITECTURE.md "one registration", scope-resolver cookbook, phase deps) — adding a language is now a single `SCOPE_RESOLVERS` registration. Verified: tsc clean (both packages); resolver integration tests green (747 assertions across cobol/csharp/ruby/rust/typescript/go, IMPORTS edges intact); grep for the flag symbols is zero across src + test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * style(format): prettier formatting on #942 changes Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ci): drop legacy heritage-capture tests + re-baseline scope-capture fingerprints (#942) Two CI failures from the #942 cleanup, surfaced by the tri-review + CI: - tree-sitter-languages.test.ts: two tests asserted `@heritage.*` captures (Rust trait-impl, Dart extends/implements/with) that this PR removed. The acceptance grep used `@heritage\.` (with `@`); these reference the runtime capture name `heritage.trait` (no `@`), so they slipped the earlier sweep. Inheritance is now covered by the resolver integration suite. (fixed macos-latest) - Re-baselined the scope-capture bench fingerprints for csharp/rust/ruby/java/ javascript/kotlin (baselines.json) + python (python-scope/baseline-fingerprint.txt). The earlier test-cleanup reworded comments inside the lang-resolution fixture files (Shapes.cs, child.rs, derived.rb, IA.java/Plain.java, Service.js, F.kt, app.py) to scrub deleted-symbol references for the acceptance grep; those are the bench corpus, so capture node positions shifted. Capture LOGIC is unchanged — verified `--check` passes for all 14 langs + python. (fixed benchmarks) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs/chore: scrub remaining REGISTRY_PRIMARY + deleted-symbol references (#942) Tri-review P3 follow-ups (verified): - TESTING.md: rewrite the "Scope-resolution parity" section — the legacy dual-leg (REGISTRY_PRIMARY_<LANG>=0/1) and `npm run test:parity` no longer exist; resolver tests run once on the sole scope-resolution path in the normal tests job. - scripts/bench-scope-resolution.ts: drop the inert `REGISTRY_PRIMARY_PYTHON=1` env set + usage hint (the flag is gone). - ruby/scope-resolver.ts, php/captures.ts: re-point doc-comments off the deleted heritage-map.ts / heritage-processor.ts to the current behavior. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ci): prettier format + regenerate scope-capture goldens (#942) Two more CI failures, same root cause as the bench re-baseline (the test-cleanup reworded comments in lang-resolution bench/golden-corpus fixtures): - quality/format: prettier on tree-sitter-languages.test.ts (blank line left by the deleted heritage-capture tests) + TESTING.md (the rewritten section). - tests/ubuntu/coverage: `csharp-captures-golden` (and python/ruby/rust) drifted because the edited fixtures feed the per-language capture-golden snapshots too (not just the bench). Regenerated via UPDATE_GOLDEN=1. Verified safe: only the edited-fixture entries changed; csharp `captureGroups` unchanged (38) — digest shifted from comment-position only; capture LOGIC untouched. 1168 scope- resolution tests pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(resolvers): drop createResolverParityIt wrapper, use vitest it directly The parity-aware `it` wrapper became a no-op when #942 removed the legacy call-resolution DAG (it just returned vitest's `it`). Remove it entirely so the resolver tests call vitest's `it` directly instead of shadowing it with a local `const it` (or `pit`/`rustParityIt`): - helpers.ts: delete createResolverParityIt + its now-unused vitestIt import and VitestIt type. - 16 files: drop `const it = createResolverParityIt('x')` and import `it` from vitest instead. - ruby.test.ts (pit) + rust.test.ts (rustParityIt): rename calls to `it`. - Scrub every comment that described the removed wrapper / dual-mode parity skip / legacy_skip gate (vue-scope, js/ts/dart/php/python headers, rust x2, cpp, swift x4, rust-coverage). Genuine test rationale is kept; only the vestigial two-leg framing is dropped. Accurate "legacy DAG (removed in #942)" historical notes are retained. No fixtures touched (no bench/golden re-baseline). tsc clean; rust+ruby resolver suites green (323 tests, incl. #1992 worker-path parity after a local dist build). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
DuduPhudu
|
c2b4ec6c31
|
feat(vue): migrate Vue SFC to scope-based resolution (RFC #909 Ring 3, closes #940) (#1950)
* feat(vue): migrate Vue SFC to scope-based resolution (RFC #909 Ring 3, closes #940) Adds `vueScopeResolver` and wires Vue into the scope-resolution pipeline (`SCOPE_RESOLVERS`, `MIGRATED_LANGUAGES`). Vue's `<script>` / `<script setup>` blocks are TypeScript — `emitVueScopeCaptures` extracts the script block via the existing `extractVueScript` utility and delegates to `emitTsScopeCaptures`, keeping grammar identity consistent with the cached tree the parse-worker already builds. - `languages/vue/captures.ts` — `emitVueScopeCaptures` - `languages/vue/import-target.ts` — `makeVueResolveImportTarget` (TS resolver + tsconfig path-alias support; explicit `.vue` imports resolve via the exact-path branch) - `languages/vue/scope-resolver.ts` — `vueScopeResolver` - `languages/vue/index.ts` — barrel + known-limitations doc - `languages/vue.ts` — `emitScopeCaptures` hooked up - `scope-resolution/pipeline/registry.ts` — Vue entry added - `registry-primary-flag.ts` — `SupportedLanguages.Vue` added to `MIGRATED_LANGUAGES` (production default → registry-primary) - `vue-composition-api` — `<script setup lang="ts">`, defineProps / defineEmits macros, cross-file TS imports, computed refs - `vue-options-api` — `defineComponent({methods, computed, data})`, this-based method calls, imported utility calls - `vue-cross-file` — composable functions returning class instances, multi-level import chains, UserModel/PostModel method calls - `fieldFallbackOnMethodLookup: true` — Options API `this.X()` calls may not resolve through the type-binding layer (no formal class); fallback catches common patterns via declared field names. - `allowGlobalFreeCallFallback: false` — Vue uses explicit imports; workspace-wide unique-name fallback would produce spurious edges for built-ins (ref, reactive, defineProps, …). - Template expression calls intentionally out of scope: component- reference CALLS edges are already emitted by the legacy template extractor. Remaining template gaps tracked in #1647. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(vue): address P0/P1 review findings from #1950 ## P0 #1 — missing scope-resolution hooks in vueProvider `pass3CollectImports` early-returns when `interpretImport` is undefined, producing zero IMPORTS and zero cross-file CALLS edges. Add the four hooks to `vueProvider` in `vue.ts`: - `interpretImport: interpretTsImport` - `interpretTypeBinding: interpretTsTypeBinding` - `bindingScopeFor: tsBindingScopeFor` - `importOwningScope: tsImportOwningScope` Also add `receiverBinding`, `mergeBindings`, `arityCompatibility`, and `resolveImportTarget` to complete the scope-resolution contract. ## P0 #2 — template-component CALLS dropped when Vue is registry-primary `isRegistryPrimary(Vue) → true` makes the main call-processor loop skip Vue files entirely, silencing the inline `vue-template-component` CALLS emitter at ≈L1506. Add a dedicated post-loop pass in `call-processor.ts` that emits template-component CALLS for Vue files whenever Vue is registry-primary. Update the stale `vue/index.ts` limitation comment to reflect the new emit site. ## P1 #3 — worker-mode double-extraction → zero captures In worker mode (≥15 files) the parse worker pre-extracts the `<script>` block and passes `scriptContent` as `sourceText`. `emitVueScopeCaptures` was calling `extractVueScript` a second time, getting null, and returning `[]`. Fix: if extraction returns null and the content has no SFC block- level markers (`<template`, `<style`), treat it as already-extracted script text and delegate directly to `emitTsScopeCaptures`. ## Test assertion strictness Replace all `toBeGreaterThanOrEqual(1)` assertions with exact `toBe(N)` counts. IMPORTS counts reflect per-symbol scope-based edges (value imports only; `import type` is not emitted as an IMPORTS edge). CALLS counts are 1 per single-call-site. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(vue): template-derived edges + pipeline benchmark (#1950 review) Addresses the reviewer's request for template edge attribution and a performance benchmark. ## Template event-handler CALLS (`vue-template-callback`) Add `extractTemplateEventHandlers` to `vue-sfc-extractor.ts`. Extracts bare single-identifier handlers from `@event="methodName"` and `v-on:event="methodName"` attributes. Inline expressions with arguments or operators (`@click="toggle(item)"`) are intentionally excluded. Wire into the dedicated registry-primary Vue template pass in `call-processor.ts`. For each extracted handler name, `ctx.resolve` finds the in-file Function/Method node and emits a CALLS edge with `reason: 'vue-template-callback'`. ## Template attribute-binding ACCESSES (`vue-template-attribute`) Add `extractTemplateAttributeBindings` to `vue-sfc-extractor.ts`. Extracts bare single-identifier values from `:prop="varName"` and `v-bind:prop="varName"` bindings. Member-access (`:key="post.id"`) and literals are excluded by the identifier-boundary regex. Wire into the same template pass. For each extracted variable, `ctx.resolve` finds the in-file node and emits an ACCESSES edge with `reason: 'vue-template-attribute'`. ## `vue/index.ts` limitations comment Updated to accurately describe all three categories of template-derived edges and explicitly document the complex-expression exclusions. ## Tests Add 6 new assertions in `vue-scope.test.ts`: - `@click="handleSave"` → CALLS `handleSave` (UserProfile.vue) - `@select="onPostSelected"` → CALLS `onPostSelected` (App.vue composition) - `@keyup.enter="addTodo"` → CALLS `addTodo` (TodoList.vue) - `@loaded="onUserLoaded"` → CALLS `onUserLoaded` (App.vue cross-file) - `:userId="currentUserId"` → ACCESSES `currentUserId` (App.vue composition) - `:posts="allPosts"` → ACCESSES `allPosts` (App.vue composition) Add `vue` entry to `LEGACY_RESOLVER_PARITY_EXPECTED_FAILURES` in `helpers.ts` documenting which assertions are registry-primary-only (IMPORTS cardinality, template-derived edges, `<script setup>` export). ## Benchmark Add `vue-pipeline-benchmark.test.ts` (gated by `GITNEXUS_BENCH=1`). Generates N-component synthetic repos (10 / 25 / 50 / 100) and asserts that wall-clock and node counts scale sub-quadratically with component count, guarding against O(n²) regressions in the template extraction or scope-resolution passes. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(vue): BINDS_EVENT_HANDLER/EMITS_EVENT edges via ScopeResolver hook Per maintainer feedback on PR #1950: - Do not edit call-processor.ts (will be removed when all languages migrate) - Model Vue component-event system with dedicated edge types to avoid CALLS noise in deep component hierarchies (per contributor discussion) Changes: - gitnexus-shared: add BINDS_EVENT_HANDLER and EMITS_EVENT to RelationshipType - vue-sfc-extractor: add extractComponentEventBindings, extractNativeElementEventHandlers, and extractScriptEmitCalls - ScopeResolver contract: add optional emitPostResolutionEdges hook - run.ts: wire emitPostResolutionEdges after emitImportEdges - vue/scope-resolver: implement emitPostResolutionEdges emitting: 1. CALLS (vue-template-component) — PascalCase component File refs 2. CALLS (vue-template-callback) — @event on native HTML elements 3. BINDS_EVENT_HANDLER (vue-event: @name) — @event on component elements; source = handler fn in parent, target = child component File (not CALLS) 4. EMITS_EVENT (vue-emit: name) — emit() calls; self-loop on component File, joinable with BINDS_EVENT_HANDLER via Cypher for impact tracing 5. ACCESSES (vue-template-attribute) — :prop="var" bindings - call-processor.ts: revert dedicated Vue post-loop pass; moved to scope resolver - Tests and parity expected-failures updated accordingly Co-authored-by: Cursor <cursoragent@cursor.com> * fix(vue): close review gaps in scope/parity extraction Resolve the new PR #1950 review findings by widening Vue scope context to include TS/JS import closures, fixing BINDS_EVENT_HANDLER endpoint assertions, hardening emit/event extraction to avoid comment/property false positives, supporting kebab-case component tags, and ensuring parity runs include vue-scope suites. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(vue): address second review round — regex safety, emit coverage, arch Closes items raised in the Jun 2 review comment on PR #1950. Correctness fixes: - ReDoS mitigation: bound attribute-capture spans to [^>]{0,512}? in all three template tag regexes to prevent pathological backtracking. - Kebab-case misclassified as native: added (?![A-Za-z0-9-]) negative lookahead to NATIVE_TAG_RE so <post-list> is no longer split as native tag `post` with attrs `-list ...`. - Hyphenated event names dropped: widened TAG_EVENT_RE from [\w:.]+ to [\w:.-]+ so @user-loaded and @update:model-value are captured. - this.$emit silently dropped: collectBareEmitEventNames now allows this.$emit(...) by looking back past the '.' to verify preceding token is exactly `this`; socket.emit etc. remain blocked. - Event names with colon rejected: extended validator to accept update:modelValue and update:model-value patterns. Architecture fix: - Moved collectVueScopeFilePaths out of shared phase.ts into a new collectScopeContextPaths optional hook on ScopeResolver, keeping shared pipeline code language-agnostic. vueScopeResolver implements the hook. - Fixed memory leak: preExtractedByPath cleanup now iterates filePaths (all context files) not just primaryFilePaths (only .vue files). Cleanup: - Removed unused extractTemplateEventHandlers and duplicate EVENT_HANDLER_RE. - Fixed skipped comment numbers in emitPostResolutionEdges (1,2,4,5,6 -> 1-6). - Updated vue/index.ts: four categories -> five (added EMITS_EVENT). - Fixed gitnexus-shared EMITS_EVENT JSDoc to reflect File->File reality. Tests: 7 new unit tests covering hyphenated events, this.$emit, kebab-case native-tag exclusion, and update:modelValue event name validation. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(vue): eliminate double file-read and per-file template re-scans Two performance fixes from the self-review pass: 1. **No more double read of .vue files in phase.ts**: primary files were previously read once for `collectScopeContextPaths` (via `entryFileContents`) and again in the blanket `readFileContents(filePaths)` call. Now the primary-file map is passed directly and only the extra context files (TS/JS import closure) require a second I/O round-trip. 2. **Single template parse per .vue file in emitPostResolutionEdges**: previously each of the five extractor functions (components, native handlers, component event bindings, emit calls, attribute bindings) ran `TEMPLATE_RE.exec(content)` independently — five full-file scans per `.vue` file. Replaced with a new `extractVueTemplateEdgeData` batching helper that parses the template and script blocks once and feeds all five extractors from the pre-extracted content. emitPostResolutionEdges now calls a single function and destructures the results. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(parity): exclude TypeScript HOC/HOF/JSX scope-resolver tests from legacy DAG parity gate Three test files introduced in prior PRs exercise scope-resolver-only correctness wins: HOC-wrapped const declarations, HOF-callback caller attribution, and JSX-as-call CALLS edges. The parity runner's ${slug}-*.test.ts glob now picks them up, causing typescript [legacy] failures in CI. Fix: convert each file to use createResolverParityIt('typescript') and register all 26 legacy-failing test names in LEGACY_RESOLVER_PARITY_EXPECTED_FAILURES.typescript with explanatory comments. Legacy mode: 11+11+4 tests skipped, zero failures. Registry-primary mode: all 37 tests pass as before. Co-authored-by: Cursor <cursoragent@cursor.com> * chore(test): remove registry-primary-flag unit tests after migration complete All languages are now in MIGRATED_LANGUAGES; the per-language flip tests are no longer needed. Addresses PR #1950 review feedback. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Gergő Magyar <gergomagyar@icloud.com> |
||
|
Gergő Magyar
|
f885330b34
|
fix(cli): steer docs, skills, and hooks through a CLI-neutral project-local runner (#1939) (#1945)
* fix(cli): steer npm 11 users away from npx install crash (#1939) Prefer global gitnexus or pnpm dlx in hooks and generated AI context, warn when npm 11.x would use the broken npx path, and document workarounds for the arborist node.target null failure mode. Co-authored-by: Cursor <cursoragent@cursor.com> * test(hooks): stage resolve-analyze-cmd.cjs for antigravity adapter; harden load checks The antigravity adapter gained a top-level require('./resolve-analyze-cmd.cjs') but stageAdapter() did not copy it, so the spawned adapter crashed with MODULE_NOT_FOUND. Three load-sensitive tests failed; four silent-path tests false-passed on empty stdout. Stage the helper alongside the other sibling helpers, and assert status===0 and no MODULE_NOT_FOUND on the four silent-path tests so a non-loading hook can never pass green again. Force a deterministic invocation mode in the stale-index test so the emitted analyze command no longer varies by CI-runner PATH. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(cli): standardize invocation hints on gitnexus@latest; single-source CJS helper NPX_REF becomes a literal `gitnexus@latest` in resolve-invocation.ts, dropping the package.json require and the module-load throw (a malformed/absent version can no longer crash any CLI command at import). The safety this PR delivers is the install method steered to (global / pnpm dlx), not a pinned gitnexus version, and the in-repo CJS mirror already degraded to `latest` once copied outside the package. Make the two resolve-analyze-cmd.cjs copies byte-identical and add a parity test that fails on drift. The separate, version-pinned NPX_REF that setup.ts writes into the MCP server registration is intentional and left unchanged. Co-authored-by: Cursor <cursoragent@cursor.com> * perf(cli): move npm-11 npx warning off module load; memoize invocation mode warnIfNpm11NpxRisk() ran at index.ts module load, so every CLI invocation (including the `gitnexus mcp` stdio hot path) paid which/where + npm --version spawns — against the lazy-startup/MCP-stdout discipline (#207, #1383). Move the call into analyzeCommand, after the ensureHeap() re-exec guard, so it fires once in the working process and only for `analyze`. Memoize the PATH-probe-derived invocation mode (the GITNEXUS_INVOCATION override stays uncached) so repeated callers don't re-probe, and add a test-only reset so the cache + once-only warning flag don't leak across the unit suite. Covers the mode!=='npx', npm<11, and npm-absent suppression branches. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(cli): detect .exe/extensionless global gitnexus shims on Windows The winGitnexusWrapper branch only matched .cmd/.bat, so a global gitnexus installed by Volta or scoop (a .exe or an extensionless shim) was missed and the hint fell back to pnpm/npx. Accept .exe and treat any non-empty `where` hit as on-PATH (the emitted hint is `gitnexus analyze` regardless of which shim resolves it). Mirror the change into both resolve-analyze-cmd.cjs copies so the TS source and the byte-identical hook mirrors stay in sync. Add Windows-mocked test cases (.exe-only, extensionless, .cmd preference, CRLF stripping) and register resolve-invocation.test.ts in cross-platform-tests.ts so the windows-latest runner exercises the branch. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(cli): emit fixed pnpm dlx analyze command in generated AGENTS.md/CLAUDE.md ai-context baked a machine-resolved command (formatAnalyzeCommand) into git-tracked AGENTS.md/CLAUDE.md, so the stale-index hint varied per machine and churned across branches (the #1706 class). Emit the fixed string `pnpm dlx gitnexus@latest analyze` instead: committed AI-context is the most authoritative instruction an agent reads, so it must name an install-free, crash-free method — never `npx`, the npm-11 path #1939 steers away from. formatAnalyzeCommand stays exported and unit-tested in resolve-invocation.ts (it still mirrors the two .cjs hook copies); ai-context just no longer calls it. Co-authored-by: Cursor <cursoragent@cursor.com> * refactor(cli): unify hook-helper copy into one non-silent routine installClaudeCodeHooks copied its four hook helpers in separate try/catch blocks that silently swallowed failures, while installAntigravityHooks recorded an error per failed copy. Extract one copyHookHelpers(srcDir, destDir, label, result) with a single canonical helper list (including resolve-analyze-cmd.cjs) and the antigravity loop's error-reporting policy, and use it from both paths so a missing helper surfaces as a setup error instead of a silent runtime crash. Assert both the Claude and Antigravity install paths co-locate resolve-analyze-cmd.cjs next to the adapter, and that a failed copy records an error rather than passing silently. Co-authored-by: Cursor <cursoragent@cursor.com> * docs(cli): reattach installClaudeCodeHooks JSDoc after helper extraction The extracted HOOK_HELPERS/copyHookHelpers block landed between the installClaudeCodeHooks JSDoc and its function, leaving the doc reading as if it described the helper list. Move the block above the doc so it documents the function again. No behavior change. Co-authored-by: Cursor <cursoragent@cursor.com> * test(cli): enforce TS<->CJS invocation parity and guard CLI startup posture Tier-2 review found two in-scope gaps in the #1945 follow-up: - The "mirrors resolve-invocation.ts / test enforces parity" comments overclaimed: the parity test only compared the two .cjs copies to each other, so the TS source and the CJS hook copies could silently drift (NPX_REF, the per-mode command, and the Windows shim regex were hand-edited in all three this PR). Add TS<->CJS value parity (NPX_REF + formatAnalyzeCommand for every forced mode) and a source-level shim-regex parity check, and make the mirror comments accurately describe what is enforced. - No test locked the R3/R4 startup posture, so re-adding warnIfNpm11NpxRisk() (or any resolve-invocation import) at index.ts module scope -- the #207/#1383 lazy-startup regression -- would pass CI. Add a guard asserting index.ts has no module-load invocation probe and the warning is wired into analyzeCommand. Co-authored-by: Cursor <cursoragent@cursor.com> * refactor(cli): collapse npx-invocation resolver to one source of truth PR #1945 carried the gitnexus/pnpm/npx selection in three hand-synced places — the canonical hook helper, its byte-identical plugin copy, and a full TypeScript re-implementation in resolve-invocation.ts — kept in lockstep by per-mode-command and regex-extracted-by-regex parity tests. The TS formatAnalyzeCommand had no production caller (ai-context emits a fixed string), and the module memoized + exposed a test-only reset for a "repeated callers" case that has exactly one caller. Make hooks/claude/resolve-analyze-cmd.cjs the single source: extract the Windows-shim line-picking into a pure, exported pickPathMatch() and add an injectable probe to resolveInvocationMode() so the shipped logic is testable without spawning or global mocks. resolve-invocation.ts (118 -> 59 lines) now consumes that cjs via createRequire for resolveInvocationMode/NPX_REF and adds only the CLI-only npm-version probe and warning; the relative path resolves identically from src/cli/ (tsx, vitest) and dist/cli/ (shipped, hooks/ is a published sibling of dist/). Tests exercise the real shipped artifact, the NPX_REF/mode-command parity scaffolding is dropped (one implementation can't drift), and parity narrows to the two cjs copies staying byte-identical. No behavior change: hook stale-index hints and the analyze warning are byte-identical; the pre-existing setup.ts resolveGitnexusBin is untouched. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(cli): bound stale-index hook PATH probe under the hook budget (U1) The PostToolUse stale-index hint calls formatAnalyzeCommand(), which probes which/where; named PROBE_TIMEOUT_MS=2000 keeps git rev-parse (~3s) + up to two probes well under Claude Code's 10s hook timeout while preserving the machine-correct hint. Byte-identical in the plugin copy. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(cli): steer generated cross-repo group commands off npx (#1939) (U2) The Cross-Repo Groups block in generated AGENTS.md/CLAUDE.md still emitted bare 'npx gitnexus group ...', funneling npm-11 users into the arborist crash; switch to fixed 'pnpm dlx gitnexus@latest group ...'. Export generateGitNexusContent and add a group-branch test asserting no 'npx gitnexus' literal survives. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: align steering guidance on pnpm dlx gitnexus@latest (U3) README troubleshooting uses gitnexus@latest; the repo's own committed CLAUDE.md/AGENTS.md stale-index hint now matches the generated output (pnpm dlx gitnexus@latest analyze) so the repo dogfoods the fix. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(hooks): assert exact @latest analyze command and pin invocation mode (U4) Drop dead PKG_VERSION/NPX_REF version-pinned constants; the cjs always emits gitnexus@latest, so assert exact toContain(...) instead of the /@\\S+/ wildcard; pin GITNEXUS_INVOCATION in the --embeddings tests for host-independent determinism. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(cli): cover resolver warn/edge branches; document probe seam (U5) Add coverage for the gitnexus-mode warn suppression, getNpmMajorVersion edge inputs (empty/pre-release/non-numeric), and the Windows non-wrapper pickPathMatch branch; widen the InvocationResolver interface to document the optional probe param. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(cli): lower hook PATH-probe timeout to 1000ms (U1) In a linked worktree the stale-index hook runs git rev-parse --git-common-dir (~2s) + rev-parse HEAD (~3s) before up to two PATH probes; PROBE_TIMEOUT_MS=1000 holds the worst case near ~7s under Claude Code's 10s hook budget (was 2000, ~1s headroom). Byte-identical in the plugin copy. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(cli): fail closed in gitnexus setup on missing required hook helper/adapter (U2) copyHookHelpers now returns the failed REQUIRED helpers (the .cjs trio; win-rm-list-json.ps1 stays best-effort since it fails open). Both install paths skip hook registration with an actionable error when a required helper failed; the Claude path also gains the adapter-existence guard the Antigravity path already had. Prevents registering a hook that crashes MODULE_NOT_FOUND on every tool event. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(skills): steer committed skill files off npx to pnpm dlx gitnexus@latest (U3) All 26 committed skill-file copies (gitnexus/skills, .claude, plugin, cursor) used 'npx gitnexus analyze', contradicting the generated freshness line and funneling npm-11 users into the arborist crash. Replace with 'pnpm dlx gitnexus@latest analyze'; add a regression guard (skills-steering.test.ts) that globs all four locations and fails if any reintroduces it. The cli skill's non-analyze npx subcommands (status/clean/list/wiki) are left as-is (out of the analyze-funnel scope). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(cli): guard resolver import shape; assert group-impact steering (U4) Add a load-time guard on the createRequire(resolve-analyze-cmd.cjs) cast so a drifted/renamed cjs export fails loudly at module load instead of as a late TypeError in warnIfNpm11NpxRisk. Add the missing 'group impact' assertion to the ai-context Cross-Repo Groups test, and a resolver-contract test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(cli): auto-select invocation path with pnpm --allow-build (#1939) Probe npm/pnpm versions and PATH to pick a working analyze command without user configuration: global gitnexus first, pnpm dlx with --allow-build on npm 11+ (Ladybug native scripts), npx on npm 10 and earlier. Update docs, skills, and tests to match the canonical install-free command. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(cli): place pnpm --allow-build before dlx, repair version-injection seam (#1939) The auto-selected install command emitted `pnpm dlx --allow-build=… analyze`, but pnpm < 10.14 keeps `dlx` in its argv escape list, so flags placed *after* `dlx` are parsed as package specs and rejected (ERR_PNPM_SPEC_NOT_SUPPORTED) on pnpm 10.2–10.13.x — strictly worse than the bare command. Move the flags before `dlx` (the position pnpm has honored since 10.2.0) in both byte-identical hook copies, the committed AGENTS.md / CLAUDE.md, and every skill tree. Also repairs the CI-red resolveInvocationMode seam: injecting `{ npmMajor: null }` to simulate an absent npm fell through `??` to the host's real `npm --version` (npm 10.x on the CI runners → routed 'npx' instead of 'pnpm'). Use an `'npmMajor' in deps` sentinel so an injected null is honored, drop the dead parseMajorVersion guard, and gate the flags on pnpm >= 10.2 via a single minor-aware probeVersion spawn (skipped for committed docs). Align the TS getNpmMajorVersion timeout to the 1s hook budget and strengthen the skills-steering guard with a pre-dlx positive assertion plus a post-dlx regression check. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: add npm-11 pnpm caveat to README Quick Starts (#1939) The root, package, and cursor-integration README Quick Starts still steered first-contact users to bare `npx gitnexus analyze` — the exact npm 11.x arborist install crash issue #1939 names as a funnel. Add a one-line pnpm `--allow-build … dlx` caveat (keeping the simple npx default for npm<=10 / pnpm / yarn users); the package README points to its existing npm-11 workaround section. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(skills): route every gitnexus-cli command off npx to pnpm dlx (#1939) The gitnexus-cli skill demonstrated analyze via `pnpm --allow-build … dlx` but still showed status/clean/wiki/list via bare `npx gitnexus` — the same package, the same npm-11 crash-prone install path — and its header claimed "all commands work via npx". Convert every subcommand to the pnpm form across all three skill copies and reconcile the header. Broaden the skills-steering guard to forbid any `npx gitnexus` command in the cli-skill copies. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(hook): probe pnpm once on the stale-index path (#1939) The stale-index hook resolved pnpm twice — `which pnpm` for mode selection then `pnpm --version` for the allow-build gate — two spawns for one tool in a ~9s/10s budget. Capture the version once in formatAnalyzeCommand and thread it through the existing deps seam (a successful `pnpm --version` proves presence), sharing a memoized PATH probe with resolveInvocationMode. Add explicit pnpm 10.0-suppress / 10.2-emit boundary tests and relabel the unknown-minor case. Both byte-identical cjs copies updated together. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(setup): single-quote POSIX hook command + assert cliPath patch applied (#1939) The hook `command` written into editor settings is shell-evaluated; the double-quoted `node "<path>"` form left `$`, backtick, and other metacharacters live in an adversarial $HOME. Single-quote the path on POSIX (Windows keeps the double-quoted form — those chars are illegal in Windows filenames). Also assert the cliPath source-literal replace() actually matched, recording an actionable error on drift instead of silently shipping a hook with an unresolved relative path. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(setup): normalize expected hook path for the Windows runner (#1939) The new POSIX-escaping test built its expected hook path with path.join, which emits backslashes on the Windows runner, while setup.ts forward-slash- normalizes the path before quoting — so `expect(cmd).toBe(node '<path>')` mismatched on tests/windows-latest. Normalize the expected path the same way. Production code was already correct; only the test's expected value was platform-fragile. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(cli): steer docs/skills via a project-local runner, not a pnpm default (#1939) The prior approach hardcoded `pnpm --allow-build=… dlx gitnexus@latest <cmd>` into every committed skill + the generated AGENTS.md/CLAUDE.md, which assumes pnpm is installed. Replace it with a CLI-neutral project-local runner: - `gitnexus analyze` drops `.gitnexus/run.cjs` (a copy of the canonical `resolve-analyze-cmd.cjs`, which gains `buildRunnerArgv` + a `require.main` exec tail) next to the index. Docs/skills reference `node .gitnexus/run.cjs <cmd>`, which auto-selects the runner (global `gitnexus` → `pnpm dlx` → `npx`) at call time — no package-manager assumption. README first-run + an inline bootstrap note stay universal `npx gitnexus analyze`. - The exec tail uses `shell` on Windows so `.cmd`/`.ps1`/`.exe` shims resolve (execFileSync can't otherwise; Node blocks `.cmd` without a shell, CVE-2024-27980), and prints a diagnostic instead of a silent exit 1. Tests: runner exec-tail (real spawn, exit-code propagation + ENOENT diagnostic), copy-failure graceful degradation, and per-subcommand routing + pnpm-fallback vacuity guards. The generated CLAUDE.md block stays under the #856 token budget. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(cli): resolve Windows .cmd version probes so pnpm steering fires (#1939) probeVersion (and the TS getNpmMajorVersion mirror) spawned npm/pnpm --version via execFileSync with no shell, so on Windows the .cmd shims ENOENT'd, the probe reported a present tool as absent, and the stale-index hook recommended the npx crash path #1939 exists to avoid. Add shell: process.platform === 'win32' to the version probes (the exec tail already does this). Parse the first version-shaped line so a Corepack/notice banner on stdout no longer defeats the parse. Carry pnpm presence separately from version so a present-but-unparseable pnpm still selects pnpm. Drop the dead probe ?? resolveOnPath coalesce. Cover resolve-analyze-cmd.cjs (+ plugin twin) with the shell-injection and windowsHide source-regression guards. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(cli): widen pnpm allow-build for the --embeddings=N equals form (#1945) buildRunnerArgv detected embeddings via gitnexusArgs.includes('--embeddings'), which missed the equals form (--embeddings=5000) that Commander also accepts, dropping --allow-build=onnxruntime-node on pnpm 10.2+. Match both forms. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(cli): cover the runner exec-tail Windows shell branch on CI (#1945) runner-exec-tail.test.ts was POSIX-only and unregistered in cross-platform-tests.ts, so the run.cjs Windows shell:true exec branch ran on no platform despite the file comment claiming windows-latest covered it. Add a .cmd-shim it.skipIf(onPosix) case and register the file in SPAWN_CLI so the windows-latest job runs it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: fix broken troubleshooting anchor in gitnexus README (#1945) The npm-11 quick-start note linked to #npx-gitnexus-crashes-with-nodetarget-is-null-npm-11, which matches no heading; the actual troubleshooting heading slugifies to #cannot-destructure-property-package-of-nodetarget-as-it-is-null. Repoint the link. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(hooks): guard resolve-analyze-cmd.cjs in antigravity e2e sanity check (#1945) The antigravity adapter top-level require()s resolve-analyze-cmd.cjs, but the beforeAll helper-presence loop did not check for it — a failed copy would surface as noisy MODULE_NOT_FOUND in downstream tests instead of the intended actionable 'Helper not installed' error. Add it to the loop. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(skills): tie a missing-runner Cannot-find-module error to recovery (#1945) Generated CLAUDE.md/AGENTS.md make `node .gitnexus/run.cjs` the primary command, but the runner is gitignored, so a fresh clone or git clean leaves an agent facing a raw MODULE_NOT_FOUND. The CLAUDE.md block is token-budget-capped (#856), so the recovery guidance lives in the cli skill (its documented home): the bootstrap note now names the `Cannot find module` error and points at `npx gitnexus analyze` to (re)generate the runner. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(cli): disambiguate the MCP-pinned ref from the @latest hint (#1945) setup.ts and resolve-analyze-cmd.cjs both exported a constant named NPX_REF with different values (version-pinned for the persisted MCP entry vs. gitnexus@latest for hints). Rename setup.ts's module-private constant to MCP_PINNED_REF (value and behavior unchanged — the MCP pin stays pinned), leaving the cjs hint ref and its re-export alone. Also route the createRequire cast through 'unknown' so it reads as an explicit narrowing to the subset this module uses rather than a claim about the cjs's full export shape. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Zander Raycraft
|
fca30c7e26
|
fix(audit): Centralize heritage supertype matching (#1921/#1922) (#1940)
* fix(audit): Centralizes heritage supertype matching so qualified, generic, scoped, and interface bases produce inheritance edges across all OO languages, with per-language configs and fixtures. * fix(audit): Harden parsing for #1922 with per-parse timeouts, ERROR/partial parse flags, tree-sitter pinned to 0.21.1, and CI ABI checks for every grammar. * fix: action lint passing * fix: feedback from triage review --------- Co-authored-by: Gergő Magyar <gergomagyar@icloud.com> |
||
|
Gergő Magyar
|
d4449b4ec8
|
fix(lbug): resolve non-ASCII paths for KuzuDB on Windows (#1811) (#1817)
* fix(lbug): resolve non-ASCII paths to 8.3 short form on Windows (#1811) KuzuDB's native C++ layer uses ANSI file APIs (fopen) on Windows. When the repo path contains CJK or other non-ASCII characters, the UTF-8 bytes from Node.js are misinterpreted as the system's Active Code Page (e.g. GBK), producing a garbled path — "Error 3: The system cannot find the path specified." Add `toNativeSafePath()` which converts non-ASCII paths to their Windows 8.3 short-name form (all-ASCII) before passing them to the native layer. Applied to both the database open path and the COPY CSV paths. No-ops on non-Windows and on all-ASCII paths. Closes #1811 * test(lbug): add unit + integration tests for non-ASCII path handling (#1811) - Unit tests for toNativeSafePath: ASCII passthrough, non-Windows no-op, Windows short-path conversion, nonexistent-path fallback - Integration test: full initLbug + loadGraphToLbug round-trip with CJK characters in the storage path — runs on all platforms - Fix toNativeSafePath to reject cmd.exe output containing '?' chars (replacement for unrepresentable Unicode in the console code page) - Register integration test in vitest lbug-db project and cross-platform-tests.ts matrix * chore(autofix): apply prettier + eslint fixes via /autofix command * feat(lbug): junction fallback, tmpdir CSV staging, pool-adapter coverage (#1811) U1+U4: toNativeSafePath now tries 8.3 short path → NTFS junction fallback → diagnostic warning. Junctions target path.dirname(p) and reconstruct the leaf. Handles EEXIST races. Registers cleanup on exit/SIGTERM/SIGINT. Orphan scan on first call removes stale junctions from prior crashes. U2: loadGraphToLbug redirects csvDir to os.tmpdir() when storagePath contains non-ASCII on Windows, avoiding non-ASCII characters in COPY FROM paths entirely. U3: All 4 createLbugDatabase call sites in pool-adapter.ts now wrap dbPath with toNativeSafePath. * fix(test): fix CI failures from toNativeSafePath addition (#1811) - Fix lbug-non-ascii-path integration test: use CodeRelation (actual relationship table name) instead of CALLS - Add toNativeSafePath to lbug-config.js mocks in pool-wal-recovery and lbug-pool-win-fts-probe tests — pool-adapter now imports it * fix(lbug): sanitize path before cmd.exe shell expansion (CodeQL) Reject paths containing cmd.exe metacharacters (" % | & < > ^) before interpolating into the `for %I` short-path command. Prevents command injection via crafted path names. * fix(lbug): address code review findings in non-ASCII path implementation - U1: Use process.exit(0) on Windows instead of process.kill re-raise (SIGTERM forcefully kills on Windows, handlers never fire) - U2: Pass safePath to openWithLockRetry so sidecar sweep targets the path KuzuDB actually opened, not the original non-ASCII path - U3: Skip junction creation in worker threads (isMainThread guard) to prevent junction leaks from pool-adapter workers - U4: Replace existsSync with lstatSync in orphan scan to avoid 30s blocking on unreachable UNC network targets * chore(autofix): apply prettier + eslint fixes via /autofix command * fix(lbug): correct SIGTERM exit code and run Prettier (#1811) - Use exit code 143 (SIGTERM) / 130 (SIGINT) on Windows instead of 0 so termination is not masked as success - Run Prettier to fix formatting (CI Gate blocker) * fix(lbug): eliminate CodeQL command-injection taint in tryShortPath Pass the path via GITNEXUS_SP environment variable instead of interpolating it into the cmd.exe command string. The FOR loop reads %GITNEXUS_SP% from the environment, so the command text is entirely static — no user-controlled data in the shell command. Also removes CMD_UNSAFE_RE since the env var approach makes character-level sanitization unnecessary. --------- Co-authored-by: Test <test@example.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> |
||
|
Gergő Magyar
|
a229e8e77b
|
fix(build): skip build.js when running outside the monorepo (#1795) (#1816)
`scripts/build.js` assumes the monorepo sibling `gitnexus-shared`
exists. When a user runs `npm install` from within a global install
directory, the `prepare` lifecycle fires `build.js`, which calls
`execSync(tsc, { cwd: nonExistentPath })` — Node reports this as
the misleading `spawnSync /bin/sh ENOENT`.
Add an early guard: if `gitnexus-shared` is absent and `dist/`
already exists (published package context), exit cleanly. If neither
exists, print a helpful error pointing to the monorepo checkout.
Co-authored-by: Test <test@example.com>
|
||
|
Nilotpal Kashyap
|
50c6acb108
|
feat(setup): implement antigravity integration setup and hook adapter… (#1730)
* feat(setup): implement antigravity integration setup and hook adapter for gitnexus * docs(readme): list Antigravity in supported editors * test(setup-antigravity): pin platform per-test to fix Windows CI failure The MCP entry assertion expected `npx` directly, but on Windows `getMcpEntry()` wraps it as `cmd /c npx ...`, which broke the Windows runner. Pin platform to darwin in beforeEach so the existing assertion is deterministic, restore the descriptor in afterEach, and add a parity test for the win32 cmd-wrapper shape. * fix(antigravity): align hook adapter to Gemini CLI schema + fix Windows CI Rebase the Antigravity integration on the canonical Gemini CLI hooks contract (https://geminicli.com/docs/hooks/reference/), which is the documented schema Antigravity 2.0 inherits: - Hook adapter: replace PreToolUse/PostToolUse with the single AfterTool event. BeforeTool has no documented context-injection channel in the Gemini contract, so augmentation runs in AfterTool where hookSpecificOutput.additionalContext is the documented way to append text to the tool result the agent reads. Stale-index hints land in the same channel (so the agent sees them) and are mirrored to stderr for terminal users. Tool-name matcher updated to Gemini CLI snake_case (search_file_content|glob|run_shell_command). - Setup: write hooks to ~/.gemini/settings.json under canonical hooks.AfterTool[] (replaces the ad-hoc hooks.json top-level group). Polite-neighbor merge preserves existing user hooks. Also copy win-rm-list-json.ps1 alongside hook-db-lock-probe.cjs so the Windows MCP server ownership probe doesn't silently fail open. - Tests: 17 regression tests covering MCP write, win32 shape, hook schema, polite-neighbor merge, idempotency, adapter context emission, stale-index hint, and skill layout. - README: footnote documenting the AfterTool design choice and a link to the Gemini CLI hooks reference. Windows CI fix: installSkillsTo previously used glob('*.md') + glob('*/SKILL.md'), which returned zero matches under the Windows runner's temp paths (8.3 short-name like RUNNER~1). Replace with fs.readdir + dirent type checks — same behavior, no path quirks. This fixes the only failing Windows job on the PR. * fix(antigravity): address PR review — windowsHide, stale docs, dead code Addresses the production-readiness review findings on PR #1730: - F1 (blocker): add windowsHide:true to all four spawnSync sites in the Antigravity hook adapter (findCanonicalRepoRoot, runGitNexusCli's two branches, buildStaleIndexHint) so they don't flash console windows on Windows. Matches the fix #1794 already on main for the Claude hook. - F2 (blocker): update gitnexus/README.md editor table to say AfterTool and link the Gemini CLI hooks reference. The published README had drifted to the pre-c1872b4 PreToolUse + PostToolUse schema. - F3: rewrite the stale ~/.gemini block comment in setup.ts. It still described the old hooks.json + gitnexus group + grep_search design. - F4: remove grep_search dead code from extractPattern and its doc comment. The registered matcher is search_file_content|glob|run_shell_command, so grep_search would never be invoked. - F5: annotate timeout:10000 with a ms-unit comment noting Gemini CLI uses milliseconds (Claude Code uses seconds). - F6: add the GITNEXUS_DEBUG branch to extractAugmentContext for parity with the Claude adapter, so suppressed augment stderr is recoverable. - F7: stageAdapter test helper now copies win-rm-list-json.ps1 alongside the .cjs helpers, so the adapter's Windows lock-probe path isn't a silent fail-open in child-process smoke tests. * test(antigravity): add integration tests and register in cross-platform matrix Adds end-to-end coverage on top of the unit-level tests, per maintainer request: - test/integration/setup-antigravity.test.ts (10 tests): exercises the real setupCommand() against a temp HOME with ~/.gemini/antigravity/ present. Verifies mcp_config.json shape, ~/.gemini/settings.json AfterTool entry, adapter + helpers + win-rm-list-json.ps1 copy, baked-in cliPath rewrite (issue #108 regression class), skill layout, polite-neighbor merge against existing user hooks, idempotency, skip-when-absent, corrupt-file safety, and key preservation. - test/integration/antigravity-hook-e2e.test.ts (19 tests): runs the full install-then-execute flow — invokes setupCommand to lay down the adapter + helpers, then spawns the INSTALLED adapter as a real child process against a temp git repo + .gitnexus/. The source adapter cannot be spawned directly (it requires sibling .cjs helpers that only live in hooks/claude/); install-then-spawn mirrors the production codepath. Covers staleness detection across all five git mutation types, --embeddings propagation, polite skip on toolResponse.error / exit_code !== 0, augment crash-free behavior, cwd validation, corrupted/missing meta.json, unknown event names, empty stdin, and the no-.gitnexus deep-nested case. - scripts/cross-platform-tests.ts: registers all three antigravity test files (unit in PLATFORM_LOGIC, two integration files in SPAWN_CLI) so Windows and macOS CI exercise them on every run. * fix(antigravity): review fixes — dedup, silent-failure guard, type coercion, glob filter - Delete mergeGeminiSettingsHooks (verbatim copy of mergeHooksJsonc), replace call site with the original - Unify geminiHasGitnexusHook into hasGitnexusHook with commandFragment parameter; delete the duplicate - Guard against silent adapter-copy failure: verify the adapter file exists before registering the AfterTool hook entry in settings.json; surface helper copy errors instead of swallowing - Fix toolSucceeded type coercion: use Number() so string exit_code values from Gemini CLI are handled correctly - Align glob tool extractPattern with Claude adapter's restrictive regex filter (/[*\/]([a-zA-Z][a-zA-Z0-9_-]{2,})/) - Remove bounds-only toBeGreaterThan(0) assertion (DoD §2.7) - Add antigravity adapter to HOOK_FILES windowsHide regression list * chore(autofix): apply prettier + eslint fixes via /autofix command * chore: trigger CI --------- Co-authored-by: Gergő Magyar <gergomagyar@icloud.com> Co-authored-by: Test <test@example.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> |
||
|
eddie.pan
|
5ce448a93a
|
feat(wiki): support local Claude and Codex providers (#1769)
* feat(wiki): support local Claude and Codex providers * chore(autofix): apply prettier + eslint fixes via /autofix command * fix(wiki): address local CLI provider review findings - Add subprocess timeout: LocalCLIConfig gains requestTimeoutMs, runLocalCLI sets a kill timer that rejects with an actionable error matching the HTTP timeout message format. --timeout is no longer silently ignored for claude/codex providers. - Add windowsHide: true to spawn() to prevent console window flash on Windows, matching cursor-client.ts behavior. - Skip GITNEXUS_MODEL env var for local providers so a user's OpenAI model name doesn't cross-contaminate claude/codex CLI invocations. Precedence for local providers: --model → savedLocalModel → ''. - Guard against empty stdout: reject with actionable error when CLI exits 0 but produces no output, preventing silent empty wiki pages. * fix(wiki): address deep-review findings in local CLI providers - Move empty-output guard from runLocalCLI to per-provider callers so Codex can read --output-last-message file even when stdout is empty - Merge existing config in interactive setup (local + Azure paths) to prevent saveCLIConfig from erasing previously saved API keys - Use StringDecoder for stdout/stderr to handle multi-byte UTF-8 chars split across pipe chunk boundaries - Distinguish ENOENT from non-zero exit in detectLocalCLI so users see auth guidance instead of misleading "CLI not found" when the binary exists but is not authenticated * test(wiki): add subprocess contract tests for local CLI providers Add 21 integration-level tests covering the Claude and Codex subprocess contracts that wiki-flags.test.ts mocks out: - Claude argv: -p, --output-format text, --no-session-persistence, --model conditional, stdin prompt content, CI=1, windowsHide:true - Codex argv: exec subcommand, --sandbox read-only, -c approval_policy, --output-last-message temp path, --cd, stdin marker, --model - Timeout: kill timer fires and rejects, no timer when unset - Codex file fallback: stdout used when file missing, error when both empty - detectLocalCLI: warn on non-ENOENT, silent on ENOENT - onChunk: cumulative byte count forwarded Also register the test in cross-platform-tests.ts SPAWN_CLI section and fix detectLocalCLI ENOENT detection logic (invert the check so non-ENOENT errors produce a warning). * fix(wiki): platform-aware process tree kill and Codex contract snapshot - Add killChildTree helper that uses taskkill /T /F /PID on Windows to terminate the entire process tree (including cmd.exe grandchildren), with fallback to child.kill() if taskkill fails or on non-Windows - Add Codex CLI flag contract snapshot test that locks the exact spawn args — any flag rename, reorder, or removal is caught immediately - Add Windows taskkill tests: success path asserts taskkill called with correct PID and /T /F flags, failure path verifies child.kill() fallback --------- Co-authored-by: eddie.pan2 <eddie.pan2@jtexpress.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Gergő Magyar <gergomagyar@icloud.com> Co-authored-by: Test <test@example.com> |
||
|
Gergő Magyar
|
ac9a2ee12f
|
chore(ci): consolidate parity shards and narrow cross-platform matrix (#1798)
* chore(ci): reduce CI runner-minutes by consolidating parity and narrowing cross-platform
Scope-resolution parity previously spawned 9 separate GitHub Actions jobs
(one per migrated language), each doing full checkout + npm ci + build for
a single test file. Consolidate into one job running scripts/run-parity.ts
which loops through all migrated languages sequentially — same coverage,
~45 fewer runner-minutes of redundant setup per PR.
Cross-platform (Windows/macOS) previously ran the full 373-file test suite.
Narrow to 45 platform-sensitive files (native LadybugDB, process spawning,
path separators, worker threads, filesystem behavior). Full suite still runs
on Ubuntu with coverage.
Also adds 2 missing lbug integration tests (lbug-orphan-sidecar-recovery,
lbug-readonly-init) to the sequential lbug-db vitest project where they
belong, and rewrites TESTING.md to document all test lanes.
* fix: address code review findings on parity and cross-platform scripts
- Capture stderr in run-parity.ts (vitest writes diagnostics to stderr)
- Lower per-invocation timeout from 5min to 60s to stay within CI job limit
- Add --language flag validation (error on missing value)
- Add timeout diagnostic to run-cross-platform.ts catch block
- Add analyze-wal-checkpoint-failure.test.ts to lbug-db sequential project
- Expand cross-platform list: parser-loader, pipeline, pipeline-graph-golden,
setup-skills, cli/tool-no-index-stderr (51 files, was 45)
* fix: add shell:true for Windows npx resolution and simplify fs import
execFileSync('npx', ...) fails with ENOENT on Windows because npx is
npx.cmd — shell:true resolves this. Also replaces dynamic await
import('fs') with static import, and fixes timeout detection to use
err.killed instead of err.code.
* fix(ci): raise parity per-invocation timeout to 120s and job timeout to 30min
TypeScript and C++ resolver tests take 60-90s on CI runners, exceeding
the 60s per-invocation timeout. Raise to 120s. Also bump the job-level
timeout from 25 to 30 minutes for margin (realistic total is ~11 min).
* fix(ci): raise parity per-invocation timeout to 180s for C++ resolver
C++ resolver tests take 130-150s on CI runners due to template
metaprogramming, ADL, and SFINAE fixture volume. 120s was still too
tight. Realistic total across all 9 languages is ~12 min, well under
the 30-min job timeout.
* fix(ci): use stdio inherit for parity — no per-invocation timeout
Switch from piped stdio with per-invocation timeouts to stdio: 'inherit'.
Vitest output streams to CI console in real time, making failures
immediately visible. The CI job-level timeout (30 min) is the only
guard — no more artificial per-invocation timeouts that cut off slow
resolver tests like C++ (which genuinely takes 3+ minutes).
---------
Co-authored-by: Test <test@example.com>
|
||
|
Gergő Magyar
|
d3de5fa5d5
|
fix(install): materialize vendored grammars to fix Windows EPERM (#1728) (#1729)
* fix(install): materialize vendored grammars to fix Windows EPERM (#1728) Stop using file: optionalDependencies for tree-sitter-dart/proto/swift, which made npm symlink vendor paths on install and fail on Windows without symlink privileges. Copy vendor trees into node_modules at postinstall instead; keep native builds and #836 vendor hygiene. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(install): atomic materialize swap + fail-soft tests (#1728, #836) Hardens PR #1729 against two issues the original implementation could still hit: 1. Torn-state on rmSync→cpSync. The previous loop deleted the destination before copying. If cpSync threw — the exact Windows EPERM scenario this PR targets — a previously-working grammar was silently wiped. Now we copy to {dest}.materialize-tmp first and renameSync into place, so an interrupted copy leaves the prior materialization intact. 2. Fail-soft try/catch had no test coverage. Adds two POSIX-only tests (chmod 0o555 to deterministically force cpSync to throw) that verify (a) a single grammar failure does not abort the other two, and (b) an existing materialization survives a partial-copy failure. Skipped on Windows where chmod doesn't enforce write restriction; runs on Linux CI. Other test improvements locking in the install-hygiene invariants: - All three vendored grammars (dart/proto/swift) checked, not just dart. - GITNEXUS_SKIP_OPTIONAL_GRAMMARS=1 short-circuit is exercised. - Vendor cleanliness (#836): no node_modules/build under vendor/. - Idempotent re-runs (clean overwrite verified via sentinel file). - Missing-vendor warn+continue path now has explicit coverage. - Vendored package manifests asserted to carry no install script or runtime dependencies. - package.json optionalDependencies asserted free of vendored grammars. - package-lock.json assertion tightened from `if (entry !== undefined) { expect(entry.link).not.toBe(true); }` (vacuous when entry is absent, i.e. the expected post-fix state) to `expect(...).toBeUndefined()`. Verified locally: - npx tsc --noEmit: clean - vitest test/unit/materialize-vendor-grammars.test.ts: 8 pass + 2 POSIX-only skipped on Windows - npm pack tarball: no vendor/*/node_modules or vendor/*/build entries - Isolated global install (clean + upgrade + SKIP env) into temp prefix: succeeds; gitnexus --version → 1.6.5; vendor stays clean post-install. * fix(install): address review feedback — Swift parity, atomicity, CI smoke Resolves all findings from the automated production-readiness review on verify/issue-1728-symlink. Swift warning parity (review #2): Add tree-sitter-swift to OPTIONAL_GRAMMARS in src/cli/optional-grammars.ts alongside Dart and Proto. Before this commit, Swift was materialized at postinstall and probed by build-tree-sitter-swift.cjs but the runtime warnMissingOptionalGrammars() never warned when it failed to load — users got silent Swift degradation from the optional-grammars surface (parser-loader's separate unavailableNote only fires on demand). Now the warning path matches the materialize path. README env-var table (review #1): Update the GITNEXUS_SKIP_OPTIONAL_GRAMMARS row at README.md line 248 to list all three vendored grammars (dart, proto, swift). The quick note earlier in the README already mentioned all three; only the table row was stale. Atomicity hardening (review #3): materialize-vendor-grammars.cjs now copies to {dest}.materialize-tmp, renames the existing dest to {dest}.materialize-bak (if present), then renames the partial into dest, then removes the backup. If the partial→dest rename fails (e.g. Windows AV scanner racing the swap), the catch block restores from backup so the previously-materialized grammar is preserved. Closes the narrow torn-state window where the prior implementation could leave dest deleted after rmSync succeeded but renameSync failed. Swift probe docs (review #4): build-tree-sitter-swift.cjs script header rewritten to describe what the script actually does — probe node-gyp-build at install time so missing-prebuild failures surface as install-time warnings instead of first-parse runtime errors. The script does not "activate" anything; the runtime require() in parser-loader does the actual load. Console warning text updated to match ("prebuild probe" not "activation"). Windows packaged-install smoke test (review #5): New CI job `packaged-install-smoke` in .github/workflows/ci-tests.yml matrices on windows-latest and ubuntu-latest. Runs npm pack, installs the produced tarball globally into RUNNER_TEMP, then asserts: * no vendor/*/node_modules or vendor/*/build (#836 invariant) * tree-sitter-{dart,proto,swift} in node_modules are real directories, not junctions/symlinks (#1728 invariant) * gitnexus --version runs against the installed CLI Closes the coverage gap where the existing windows-latest job only ran `npm ci` in the source checkout — exercising postinstall but not the tarball reify step that historically tripped EPERM. Verified locally: npx tsc --noEmit: clean vitest test/unit/materialize-vendor-grammars.test.ts test/unit/cli-commands.test.ts: 18 pass + 2 POSIX-only skipped on Windows prettier + eslint on all changed files: clean * fix(ci): disable credential persistence on packaged-install-smoke checkout GitHub Advanced Security (zizmor artipacked) flagged the new packaged-install-smoke job's actions/checkout step as a potential credential-persistence risk. The job runs `npm pack` + global install and never pushes back, so the GITHUB_TOKEN that checkout would persist in .git/config provides no value and only widens the leak surface (any future artifact-upload step in this job would carry the token). Disable persistence explicitly via `persist-credentials: false` on this job's checkout. Scoped to the new job — pre-existing checkouts above are left unchanged. * fix(ci): use find instead of ls for tarball lookup (SC2012) actionlint shellcheck SC2012 flagged `TARBALL=$(ls gitnexus-*.tgz | head -n1)`. Switch to `find . -maxdepth 1 -name 'gitnexus-*.tgz' -print -quit` which handles non-alphanumeric filenames safely. Also add an explicit empty-result check so the failure mode is a clear error message instead of a silent `npm install -g ""` later. * fix(tests): sabotage vendor src (not partial path) in POSIX fail-soft tests The fail-soft tests in materialize-vendor-grammars.test.ts pre-chmod'd the destination's .materialize-tmp partial directory to 0o555 to force cpSync to throw. After the atomicity rewrite (`fix(install): atomic materialize swap + fail-soft tests`), the materialize script now starts each grammar's loop with `fs.rmSync(partial, { force: true })`, which deletes the chmod'd sabotage before cpSync runs — so cpSync succeeds and the partial is then renamed into dest, leaving the test's `finally` block with no path to chmod back (ENOENT) and the assertion that proto remained unmaterialized failing because it materialized cleanly. Fix: sabotage the *vendor source* directory (which the script reads from but never modifies) by chmod'ing it to 0o000. cpSync then fails on readdir, the catch block fires per-grammar, dart and swift still materialize from their unaffected sources, and the existing-dest preservation test verifies that a sabotaged second-run leaves the prior materialization (and its sentinel file) intact. Tests now pass locally (8 pass + 2 POSIX-only skipped on Windows) and should pass on macOS/Ubuntu CI where the sabotage runs. * fix(tests): restrict fail-soft tests to Linux (macOS Node cpSync abort) Node 22 on macOS aborts the process with `libc++abi: terminating due to uncaught exception filesystem_error` when fs.cpSync hits a source directory it can't read — the abort happens at the C++ filesystem layer and bypasses Node's JS try/catch entirely (nodejs/node#51399). My chmod-0o000-the-source sabotage strategy triggers this SIGABRT on macOS CI before the production script's `try { cpSync } catch` ever runs, so the test sees a child-process crash instead of the fail-soft warning it's verifying. The production script's fail-soft is correct on Linux (where EACCES surfaces as a normal JS exception) and effectively untestable on macOS via permission sabotage. Real installs don't hit this — npm always ships vendor/ with readable permissions — so the macOS gap is a test artifact, not a behavior gap. Restrict the two chmod-based tests to Linux only by replacing `skipOnWin` with `linuxOnly`. Linux CI continues to verify both the one-grammar-fails-others-succeed and existing-materialization-preserved invariants. macOS and Windows runs skip these two scenarios; the other 8 tests still run on every platform. * fix(tests): remove materialize unit tests, rely on CI smoke job The materialize-vendor-grammars.test.ts file has been a recurring source of platform-specific CI noise: - Windows: chmod doesn't enforce read/write restrictions the way POSIX does, so the fail-soft tests had to be skipped there. - macOS Node 22: cpSync against an unreadable source aborts the process with a libc++ filesystem_error (nodejs/node#51399) that bypasses JS try/catch entirely — making the chmod-based fail-soft tests unrunnable on macOS too. - The "vendor-cleanliness" and "idempotency" tests on Windows intermittently flake due to fs.cpSync timing on the GitHub runner. The invariants these tests verified are now covered by stronger, more realistic surfaces: - packaged-install-smoke (ci-tests.yml): runs `npm pack` then `npm install -g ./gitnexus-*.tgz` on windows-latest and ubuntu-latest, then asserts no vendor/*/node_modules, no vendor/*/build (#836), no junctions/symlinks on the materialized grammar directories (#1728), and a working `gitnexus --version`. This is the actual end-user install path. - cli-commands.test.ts (kept, unmodified): asserts package.json declares no `file:` optionalDependencies for vendored grammars, the Swift vendor manifest carries no install script or dependencies, and the postinstall chain runs materialize-vendor-grammars.cjs + build-tree-sitter-swift.cjs. These are static manifest checks — deterministic, fast, no flake risk. Removing the dynamic script-execution tests trades unit-level coverage for end-to-end smoke coverage that actually exercises the `file:` → cpSync change against a real npm install lifecycle, on the platform the fix targets (windows-latest). --------- Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
ChamHerry
|
2a3d14057a
|
fix(analyze): prevent cache-hit native workers from aborting (#1751)
* fix(analyze): prevent cache-hit native workers from aborting Delay parse worker startup until a cache miss requires it, fall back to sequential parsing when initial worker readiness fails, and preserve analyzer diagnostics/progress when heap respawn captures child output. Constraint: Node 25 and tree-sitter/N-API worker initialization can abort before ready, while warm-cache analysis should not start workers at all. Rejected: Treating status-134/SIGABRT as heap OOM unconditionally | native worker aborts require distinct recovery guidance and stderr/stdout evidence. Rejected: cli-progress noTTYOutput for respawn progress | it appends newline frames instead of preserving one-line redraw UX. Confidence: high Scope-risk: moderate Directive: Keep parse-worker creation behind confirmed cache misses and preserve TTY-style progress when respawn pipes stderr for crash classification. Tested: GitNexus impact analysis for ensureHeap, runChunkedParseAndResolve, createWorkerPool, WorkerPool, walkRepositoryPaths; GitNexus detect_changes scoped to staged worktree; targeted vitest for analyze respawn, parse lazy cache, filesystem walker, worker pool; npx tsc --noEmit; npm run build; NODE_OPTIONS='--max-old-space-size=8192' npm test. Not-tested: Windows terminal rendering and published npm package install path. * ci(docker): tolerate slower arm64 TypeScript builds Docker PR builds run gitnexus prepare under QEMU for linux/arm64, where the fixed 120s TypeScript timeout can kill otherwise healthy builds. Increase the default timeout and allow GITNEXUS_BUILD_TIMEOUT_MS to tune slower environments without changing the build steps. Constraint: PR #1751 Docker Build & Push gitnexus failed with spawnSync /bin/sh ETIMEDOUT while running node_modules/.bin/tsc in scripts/build.js.\nRejected: Rerunning CI only | the failure was the build script's deterministic timeout boundary under arm64 emulation, not a code assertion.\nConfidence: high\nScope-risk: narrow\nDirective: Keep build timeout changes in scripts/build.js configurable; do not hide real compiler failures, only allow slower successful compiles to finish.\nTested: GitNexus impact for gitnexus/scripts/build.js reported LOW; gitnexus detect_changes reported 1 changed file, 0 affected processes, low risk; git diff --check; gitnexus npm run build.\nNot-tested: GitHub Docker arm64 build rerun before pushing; local Docker multi-platform build under QEMU. * fix(analyze): truncate respawn progress safely Preserve complete ANSI escape sequences and grapheme boundaries when the respawn progress terminal shim truncates wrapped output, so the shim does not emit dangling escape bytes or split surrogate pairs while keeping raw writes untouched. Constraint: Claude review on PR #1751 flagged `s.slice(0, width)` in createAnsiPipeTerminal.write() as a latent terminal-corruption risk. Rejected: Adding a display-width dependency | a local helper is sufficient for this narrow respawn terminal shim and avoids new dependency churn. Rejected: Changing silent status-134 classification | current tests already document the output-less 134 fallback as heap guidance. Confidence: high Scope-risk: narrow Directive: Keep respawn terminal writes ANSI-aware and preserve rawWrite bypass semantics for callers that intentionally write control sequences. Tested: GitNexus impact for createAnsiPipeTerminal reported LOW; GitNexus detect_changes reported 2 changed files, 3 affected processes, medium risk; targeted vitest for analyze respawn progress and heap respawn; gitnexus npx tsc --noEmit; prettier check for changed files; eslint for changed files. Not-tested: Full npm test suite; manual terminal rendering on Windows. --------- Co-authored-by: wangxc <wangxc_a_bj@si-tech.com.cn> |
||
|
GoGoLin
|
4cc4e9c84b
|
fix(build): use platform-aware tsc command for win32 (#1531) | ||
|
Gergő Magyar
|
de63418f7e
|
fix(mcp): close MCP server timeout — stdout discipline + cold-start friction (#1383)
* fix(lbug): route diagnostic logs to stderr to avoid MCP stdio corruption
Replace console.log/console.warn with console.error in core/lbug so
diagnostic messages reach stderr and never corrupt the JSON-RPC stream
on MCP stdio. Per spec, the server MUST NOT write anything to stdout
that is not a valid MCP message.
- lbug-adapter.ts:367 - schema creation warning (MCP-reachable via lazy
DB init from tool handlers)
- lbug-adapter.ts:1047,1054 - legacy embedding fallback diagnostics
(currently HTTP-only, but covered by upcoming no-console lint rule)
- extension-loader.ts:191 - default warn handler fallback used during
DuckDB extension loading
* feat(mcp): add stdout sentinel via AsyncLocalStorage transport-write tagging
Untagged process.stdout.write calls now redirect to stderr with a
[mcp:stdout-redirect] prefix instead of corrupting the JSON-RPC frame
stream. Identification is correctness-by-construction: the transport
wraps every send() in withMcpWrite() (AsyncLocalStorage) and the
sentinel checks isMcpWrite() per call. A byte-shape heuristic would
have falsely rejected Content-Length frames (start with C, end with })
and misclassified multi-chunk writes.
- gitnexus/src/mcp/stdio-context.ts: AsyncLocalStorage helpers + factory
- gitnexus/src/mcp/server.ts: install sentinel in safeStdout Proxy,
flush summary at process exit
- gitnexus/src/mcp/compatible-stdio-transport.ts: wrap send() write in
withMcpWrite so transport frames pass through cleanly
- gitnexus/test/unit/mcp-stdout-sentinel.test.ts: 17 cases covering
pass-through, redirect, prefix, truncation (default 200 / custom),
rate limit (default 10), one-shot warning, summary, mixed sequences
* feat(eslint): forbid console.log/warn and process.stdout.write in MCP-reachable code
Add a narrow ESLint override for gitnexus/src/mcp/**, gitnexus/src/core/lbug/**,
gitnexus/src/core/embeddings/**, and gitnexus/src/cli/mcp.ts that:
- sets no-console: ['error', { allow: ['error'] }] — only console.error
survives, since stderr is the only spec-safe channel for diagnostics
while the MCP stdio transport owns stdout for JSON-RPC frames
- adds no-restricted-syntax matching MemberExpression and CallExpression
forms of process.stdout.write to close the bypass path that the
AsyncLocalStorage sentinel cannot guarantee
Migrates 18 pre-existing console.log/warn call sites in core/embeddings/
(embedder.ts, embedding-pipeline.ts) to console.error; these are reached
from gitnexus_query semantic search and would have polluted MCP stdio
once a query triggered the embedding pipeline.
Adds eslint-disable-next-line comments in pool-adapter.ts at the four
legitimate process.stdout.write sites — they ARE the captured-real-write
infrastructure used by the sentinel and the silenceStdout/restoreStdout
mechanism.
The override is forward-compatible with feat/pino-logger (PR #1336)
which adds a broader no-console rule for gitnexus/src/; the narrow rule
here is a strict subset and rebases trivially when #1336 lands.
* feat(setup): pin setup-generated MCP config to installed version, keep static configs on @latest
The user-facing MCP config that 'gitnexus setup' writes into editor configs
now references gitnexus@<installed-version> instead of gitnexus@latest, read
dynamically from gitnexus/package.json#version at module load. This skips
the npm-registry metadata roundtrip on every MCP connect and stays
reproducible until the user explicitly upgrades.
Static example configs and quickstart docs intentionally keep @latest:
- .mcp.json, gitnexus-claude-plugin/.mcp.json
- gitnexus-claude-plugin/skills/*/mcp.json (6 files)
- README.md / gitnexus/README.md MCP examples
Pinning these would create per-release version-bump churn for marginal
(~100-500ms) savings. The dominant cold-cache cost is the native rebuild
addressed separately by the GITNEXUS_SKIP_OPTIONAL_GRAMMARS env var.
README adds a one-line steer above the @latest quickstart pointing
repeated users at 'gitnexus setup' for the absolute-path config that
bypasses npx entirely.
Tests refactored to assert against the dynamic version (createRequire of
package.json) so they don't break on every release bump:
- gitnexus/test/unit/setup.test.ts
- gitnexus/test/unit/setup-jsonc.test.ts
- gitnexus/test/unit/setup-codex.test.ts
- gitnexus/test/integration/setup-skills.test.ts (regex match)
* feat(install,mcp): GITNEXUS_SKIP_OPTIONAL_GRAMMARS opt-out + missing-grammar warnings
Postinstall scripts (build-tree-sitter-dart.cjs, build-tree-sitter-proto.cjs)
gain a strict 'process.env.GITNEXUS_SKIP_OPTIONAL_GRAMMARS === "1"'
early-exit so users without a C++ toolchain (or anyone wanting fast
'npm install gitnexus') can skip the native rebuild. Strict '=1' only —
'true', 'yes', '0' and any other value fall through to the rebuild.
Add gitnexus/src/cli/optional-grammars.ts: cheap require.resolve probe for
each optional grammar, with a stderr warning helper. The warning surfaces:
- At MCP server start (cli/mcp.ts) — unconditional, since the server
serves any indexed repo and we cannot pre-filter by language.
- At 'gitnexus analyze' start (cli/analyze.ts) — conditional on the
target repo containing .dart/.proto files (cheap glob), so users with
no relevant code don't see noise.
README documents the env var with the strict '=1' value and the trade-off
(faster install, no Dart/Proto parsing until reinstalled).
* test(mcp): child-process integration test asserts end-to-end stdout discipline
Spawns 'node dist/cli/index.js mcp' as a child, drives the MCP stdio
handshake (initialize -> initialized -> tools/list), reassembles every
stdout chunk into Content-Length-framed JSON-RPC messages, and asserts
zero stray bytes. Any byte outside a valid header-then-body window is
captured and surfaced in the failure message alongside the server's
stderr — this is the regression gate for U1 (no console.log/warn in
MCP-reachable code) and U3 (AsyncLocalStorage stdout sentinel).
Time budget: 5s local / 15s CI for first frame; 10s/30s total. Asserts
the published GitNexus tool surface (list_repos, query, context, impact,
detect_changes, rename) is reported by tools/list.
Adds 'pretest:integration': 'node scripts/build.js' so 'npm run
test:integration' rebuilds dist before the spawn — closes the
'stale dist masks regression' DX gap.
* fix(mcp): address PR #1383 review — sentinel scope, grammar detection, lint, contract
Blockers:
- B2: detectMissingOptionalGrammars now actually require()s each grammar
instead of require.resolve(). For 'file:' optional dependencies the
package directory is always installed regardless of postinstall outcome,
so resolve() never threw and the missing-grammar warning never fired
for the exact target users (those who set GITNEXUS_SKIP_OPTIONAL_GRAMMARS=1
or whose native rebuild soft-failed). require() loads the entry, which
triggers node-gyp-build and throws if .node is absent. Result memoized.
Should-fix:
- S1: Removed duplicate uncaughtException/unhandledRejection handlers from
cli/mcp.ts. server.ts:startMCPServer already registers handlers with
full stack traces; cli/mcp.ts handlers fired first with worse output and
never got a chance to exit because server.ts shuts down immediately.
- S2: Sentinel is now actually global. New setActiveStdoutWrite() in
pool-adapter so silenceStdout/restoreStdout cycles preserve a
registered wrapper instead of unwinding to raw realStdoutWrite. At
startMCPServer: install sentinel.write as process.stdout.write AND
register it as the active handler. Direct process.stdout.write calls
from anywhere (console.log, dependency banners, etc.) now route through
the sentinel instead of bypassing it. The transport's _safeStdout Proxy
remains as belt-and-suspenders.
- S3: ESLint no-restricted-syntax now also forbids destructuring of
process.stdout (covers both 'const { write } = process.stdout' shapes
and rest patterns).
Minor:
- M1: chunkToBuffer now handles plain Uint8Array (Buffer.from(u8)) instead
of falling through to String(chunk) which produced '1,2,3,...' garbage.
- M2: Untagged-write callbacks are now invoked on next tick per the
Node Writable.write contract — both within and beyond the rate-limit cap.
extractCallback handles the (chunk, cb) and (chunk, encoding, cb) overloads.
- M3: setup.ts throws early if package.json#version is missing/non-string
instead of emitting 'gitnexus@undefined'.
- M4: parser-loader.ts console.warn → console.error; ESLint scope extended
to gitnexus/src/core/tree-sitter/** so future violations are caught.
New tests cover:
- Plain Uint8Array redirect (asserts no String(chunk) garbage).
- Writable callback fired async (next-tick) for both normal and
past-rate-limit redirects.
Validation: cd gitnexus && npx tsc --noEmit clean; vitest run 7863 passed,
11 skipped; eslint clean on MCP-reachable scope; integration test green
against rebuilt dist/.
* fix(mcp): close pre-sentinel stdout window + tighten contracts
Address ce-code-review findings on PR #1383:
P1 — Sentinel install order (was: stdout corruption window during
mcpCommand pre-startup):
- Add idempotent installGlobalStdoutSentinel() to mcp/stdio-context.ts.
It captures realStdoutWrite/realStderrWrite, replaces process.stdout.write,
and registers with pool-adapter's setActiveStdoutWrite — exactly once.
- cli/mcp.ts now installs the sentinel as the FIRST line of mcpCommand,
before warnMissingOptionalGrammars (which after the B2 fix actually
require()s each native grammar binding and could emit node-gyp-build
banners to raw stdout in the pre-sentinel window).
- mcp/server.ts startMCPServer keeps a safety-net call to the same helper;
the second invocation is a no-op.
P1 — WriteFn type erasure:
- WriteFn now declared as instead of
, so the assignment
and the
setActiveStdoutWrite(sentinel.write) call don't silently cross a
type boundary.
P1 — extractCallback fragility:
- Replaced backward-scan-with-undefined-break heuristic with a strict
'last arg if function' check matching the documented Writable.write
contract. No longer breaks on a future (chunk, options, cb) overload.
P2 — _detectionCache premature memoization:
- Removed the explicit cache. Node's module cache already memoizes
require() — calling detectMissingOptionalGrammars multiple times is
cheap. Removing the module-level mutable state makes the helper
trivially testable (no need for a reset hatch).
P2 — Misleading 'reinstall' message on broken (not missing) grammars:
- detectMissingOptionalGrammars now distinguishes MODULE_NOT_FOUND /
node-gyp-build 'no native build' patterns from other errors
(SyntaxError, EACCES, native crash). Broken bindings get an
actionable stderr line naming the real failure instead of the
misleading 'reinstall to enable' hint.
Other:
- mcp/core/lbug-adapter.ts updated with a KEEP-THIS-FILE note. Tests
use the path as a vi.mock seam (calltool-dispatch.test.ts and 7
others); new non-test code may import core/lbug/pool-adapter.js
directly. The maintainability finding flagging the shim as
self-contradictory was incorrect — the shim has a real test purpose.
Validation: tsc clean, vitest 7863 passed (no regressions), eslint
clean on MCP-reachable scope, integration test green against rebuilt
dist/.
* fix(mcp): close import-time stdout corruption window
Codex's adversarial review on PR #1383 found that even though cli/mcp.ts
is loaded lazily by Commander, ITS static imports (startMCPServer,
LocalBackend, installGlobalStdoutSentinel, warnMissingOptionalGrammars)
evaluate synchronously when the module loads — well before mcpCommand's
function body runs. Three of those four imports transitively pulled in
core/lbug/pool-adapter.ts, which imports @ladybugdb/core at module top
level. The native binding's init can write to raw stdout in that
pre-sentinel window and corrupt the JSON-RPC frame stream.
Fix: shrink cli/mcp.ts's static-import closure to a single zero-dep
chain (mcp/stdio-context.js -> mcp/stdio-capture.js, both leaf-clean),
install the sentinel as the first executable statement of mcpCommand,
then dynamically import the heavy backend modules in parallel via
await Promise.all.
Per the plan at docs/plans/2026-05-06-002-fix-import-time-stdout-window-plan.md:
- U1: New leaf module gitnexus/src/mcp/stdio-capture.ts owns the
stdout-capture singleton state (realStdoutWrite, realStderrWrite,
activeStdoutWrite + setActiveStdoutWrite/getActiveStdoutWrite).
Zero non-node: imports — adding any would re-introduce the hazard.
- U2: pool-adapter.ts re-exports the relocated symbols under the
existing names so the test mock seam (8+ files use vi.mock on
mcp/core/lbug-adapter.ts which re-exports * from pool-adapter)
keeps working without churn. restoreStdout and the watchdog now
read the active handler via getActiveStdoutWrite(). stdio-context.ts
imports from stdio-capture directly.
- U3: cli/mcp.ts's static imports collapse to one
(installGlobalStdoutSentinel). startMCPServer / LocalBackend /
warnMissingOptionalGrammars become parallel await import()
inside mcpCommand, after the sentinel install.
- U4: New regression test gitnexus/test/integration/mcp/import-closure.test.ts
spawns a child Node process that imports dist/cli/mcp.js (without
invoking mcpCommand), inspects the CJS module cache via createRequire,
and asserts @ladybugdb/core (and tree-sitter native bindings) are
NOT in the static-import closure. Characterization-first: this test
was authored to fail against the pre-fix code and confirmed to do so
before U1-U3 landed.
Validation: tsc clean; vitest 7865 passed / 11 skipped (2 new U4 cases);
eslint clean on MCP-reachable scope; integration server-startup test
green against rebuilt dist/.
* fix(mcp): drop dead ESLint selector + suppress redundant grammar warning
Two minor PR #1383 review findings:
1. eslint.config.mjs: removed Selector 3 (`Property[key.name='write'].properties:has(...)`).
`.properties` is not a valid attribute on a Property node in the ESTree
AST, so the :has clause never matched — dead code. Selector 4 covers
the canonical `const { write } = process.stdout` shape; tightened its
comment to make that explicit.
2. cli/mcp.ts: removed the unconditional warnMissingOptionalGrammars call
at MCP startup. The analyze path already emits this warning at index
time with relevantExtensions filtered to the repo's actual file types,
and a repo can only be served by MCP after analyze has run. Repeating
the warning unconditionally on every MCP session was pure noise on
machines whose indexed repos don't use .dart/.proto.
* chore(mcp): address PR #1383 review nits
Three minor hygiene findings from the production-readiness review:
- cli/mcp.ts: rewrite stale comment that described
warnMissingOptionalGrammars as living inside mcpCommand. The call was
removed in
|
||
|
Gergő Magyar
|
3f0c74fea0
|
fix(deps): upgrade @ladybugdb/core to 0.16.0 to resolve native segfaults (#1235)
* fix(deps): upgrade @ladybugdb/core to 0.16.0 to resolve native segfaults Resolves the SIGSEGV / access-violation (0xC0000005) / exit-139 crashes that have been reported widely since 1.6.3. The native crashes originate in @ladybugdb/core 0.15.x — primarily during FTS index creation, VECTOR extension load, and concurrent query teardown — and are reproducible on Linux, macOS and Windows. The maintainer-confirmed fix is to bump the runtime to 0.16.0, which ships nodejs async + memory-management fixes, extension ABI bump, and macOS Intel binaries. Adopting 0.16.0 cleanly required three supporting changes; without them the upgrade itself regresses other paths: 1. maxDBSize must be passed explicitly. 0.16.0 keeps the upstream JSDoc note that the default 0 is "introduced temporarily for now to get around with the default 8 TB mmap address space limit some environment". Constrained CI runners and laptops cannot reserve 8 TB and crash with "Buffer manager exception: Mmap for size 8796093022208 failed." A new gitnexus/src/core/lbug/lbug-config.ts centralises a 16 GiB default (overridable via GITNEXUS_LBUG_MAX_DB_SIZE) and every Database() construction site now passes it. 2. enableCompression default flipped from false to true in 0.16.0. Every Database() call site is updated to pass false explicitly so existing GitNexus indexes keep the same wire format. 3. Bridge DB sidecar files (.wal, .shadow). 0.16.0 enforces a database-id check on .wal / .shadow sidecars and rejects opens whose sidecars belong to a different base name. writeBridge now (a) cleans the full sidecar set when removing the tmp slot, (b) renames .wal / .shadow alongside the main file during the atomic .tmp -> .lbug swap, and (c) wraps openBridgeDbReadOnly in a bounded retry on transient Win32-Error-33 lock errors. Eager db.init() / conn.init() forces the lazy native handle to surface lock contention at the retry site. Known limitation (not a regression): on Windows the 0.16.0 native binary does not release the OS file lock until the process exits, so the close-then-reopen-same-process pattern raises Error 33 after the first close. Production paths (analyze / serve / mcp each open the DB exactly once per process) are unaffected, but eight tests that exercise the pattern are guarded with a process.platform === 'win32' skip; CI's Linux + macOS shards exercise them as before. Tracking upstream: kuzudb/kuzu#3872 / #3883 / #4730. Closes #1136 #1154 #1160 #1162 #1178 #1195 #1196 #1199 #1204 #1206 Refs #1209 (supersedes — Dependabot bump without the supporting fixes) Made-with: Cursor * fix(test): isolate LadybugDB native test state Use per-suite LadybugDB databases in integration helpers so test forks do not reopen a database created by Vitest global setup, and centralize Windows-tolerant native temp cleanup for bridge tests. * fix(lbug): avoid bridge existence reopen Reuse the built LadybugDB config in the extension installer and avoid native close/reopen cycles when checking bridge existence on Windows. Made-with: Cursor * chore(docs): exclude local lbug plan Keep the refactor planning note out of the PR while leaving the ignored local copy on disk. Made-with: Cursor * refactor(lbug): centralize database construction Route LadybugDB opens through shared helpers so native constructor defaults stay consistent across core, pool, bridge, and extension install paths. Made-with: Cursor --------- Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> |