mirror of
https://github.com/abhigyanpatwari/GitNexus.git
synced 2026-10-28 06:22:22 +00:00
62 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Gergő Magyar
|
01af833875
|
fix(scope): preserve lexical bindings in HTTP wrapper detection (#3632)
* docs(scope): clarify lexical binding ownership and wrapper provenance Checkpoint for #3630. Publishing the implemented source, regression tests, and benchmark harness as a draft PR; full-suite and final scaling validation remain in progress. * fix(scope): publish #3630 docs/plans * fix(scope): publish #3630 gitnexus/bench/lexical-binding-provenance * fix(scope): publish #3630 gitnexus/bench/lexical-binding-provenance/fixtures * fix(scope): publish #3630 gitnexus/bench/lexical-binding-provenance/results * fix(scope): publish #3630 gitnexus/src/core/ingestion * fix(scope): publish #3630 gitnexus/src/core/ingestion/languages/typescript * fix(scope): publish #3630 gitnexus/src/core/ingestion/languages * fix(scope): publish #3630 gitnexus/src/core/ingestion/pipeline-phases * fix(scope): publish #3630 gitnexus/src/core/ingestion/route-extractors * fix(scope): publish #3630 gitnexus/src/core/ingestion/workers * fix(scope): publish #3630 gitnexus/test/fixtures/lang-resolution/fetch-wrapper-target-identity/lib * fix(scope): publish #3630 gitnexus/test/fixtures/lang-resolution/fetch-wrapper-target-identity * fix(scope): publish #3630 gitnexus/test/fixtures/lang-resolution/fetch-wrapper-target-identity/packages/http * fix(scope): publish #3630 gitnexus/test/fixtures/lang-resolution/fetch-wrapper-target-identity/packages/http/src * fix(scope): publish #3630 gitnexus/test/fixtures/lang-resolution/fetch-wrapper-target-identity/server * fix(scope): publish #3630 gitnexus/test/fixtures/lang-resolution/fetch-wrapper-target-identity/web * fix(scope): publish #3630 gitnexus/test/integration/resolvers * fix(scope): publish #3630 gitnexus/test/unit * fix(scope): publish #3630 gitnexus/test/unit/group * fix(scope): publish #3630 gitnexus/test/unit/scope-resolution * fix(scope): publish #3630 gitnexus/test/unit/scope-resolution/typescript * fix(scope): publish #3630 gitnexus-shared/src/scope-resolution * fix(scope): treat wrapped assignment targets as writes `(app) = new Map()`, `(app as T) = x` and `app! = x` assign the same binding as `app = x`, but route-fact invalidation and the shared pattern walker stopped at the wrapper, so Express provenance survived the write and produced a false route. Recurse through parenthesized, as/satisfies and non-null targets. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(routes): resolve wrappers through overloads, declarator lists and barrels Three wrapper shapes lost their identity: - Bodyless overload signatures were indexed as separate definitions sharing the implementation's binding, so the binding looked ambiguous and both local and imported calls were rejected. Signatures now declare no callable. - Facts for `const base = 0, request = (url) => fetch(url)` followed the first declarator. The captured name now selects its own declarator. - An import resolved to a workspace barrel stopped at index.ts. Workers now record named, star and imported-then-exported re-exports, and resolution follows them to the declaring module: local and named exports shadow `export *`, star exports never carry `default`, and conflicting star exports block the name. No repository-wide name fallback is restored. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(scope): assert closure ownership against the declaration The closure-write assertion compared one lookup with itself, which also passes when both are undefined. Compare it with the outer declaration and the shadowing parameter, and keep the memoization check explicit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(routes): share lexical helpers and release per-file HTTP contexts - Share binding keys, parameter patterns and transparent expression wrappers from lexical-bindings instead of four local copies; drop the unused moduleStem export; name the inferred-source predicates in lookup. - Key the per-file HTTP context by its tree rather than its ParsedFile. Workers retain ParsedFiles for the whole chunk, which kept every file's AST and traversal maps alive until the chunk flushed. - Index re-exports by name, memoize barrel walks per starting export, and keep facts without import targets as-is instead of copying them. - Move TS-only wrapper cases into the TS describe.each instead of branching inside the per-language loop. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(routes): keep wrapper identity for HOCs, CommonJS, object keys and fetch packages Four definition shapes that main linked by name lost their wrapper identity: - `const load = useCallback((url) => fetch(url))` and curried HOCs produced no facts because definition lookup stopped at the call; it now follows the callable argument, as the structural definition capture does. - `module.exports = { request }`, `exports.x = …` and `module.exports.x = …` were not exports, so destructured `require` callers missed the wrapper. CommonJS export assignments are recorded when `module`/`exports` are the unshadowed globals, including functions defined inline in the export object. - An object-literal key or method name was resolved as a lexical binding and collided with a same-named module wrapper. Only binding identifiers now name a definition's binding. - `import fetch from 'node-fetch'` no longer counted as fetch. Imports of known fetch implementations (node-fetch, cross-fetch, isomorphic-fetch, the fetch member of undici and whatwg-fetch) are fetch evidence again; parameters and other local names still are not. Reference identifiers are also resolved only when their name is bound to a callable or class region; other identifiers cannot add a reachability edge. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(routes): drive the forwarding fallback case from data and cover re-export merges Replace the boolean-switched forwarding fallback case with table rows instead of branching inside the test, and assert that barrel re-export facts survive worker sub-batch merges. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * chore(bench): reject stale or dirty call-graph exports The committed comparison named a candidate revision that is not on this branch, and nothing stopped it. Comparison now requires both exports to come from clean checkouts and the candidate export to come from the checkout running the comparison. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * chore(bench): regenerate the call-graph comparison on the reviewed head Clean baseline |
||
|
Gergő Magyar
|
62f86ea068
|
feat(mcp): reduce response size and query overhead (#3621)
* docs(mcp): document focused response filters * feat(mcp): add opt-in response section and relation filters * test(mcp): verify response filters and compatibility * test(mcp): add deterministic response-size benchmark * docs(skills): teach focused MCP calls and safe filter use * docs(skills): update debugging response-filter guidance * docs(skills): update exploration response-filter guidance * docs(skills): explain focused MCP response options * docs(plugin): mirror focused debugging guidance * docs(plugin): mirror focused exploration guidance * docs(plugin): mirror focused MCP tool guide * docs(cursor): mirror focused debugging guidance * docs(cursor): mirror focused exploration guidance * test(mcp): give filter cases unique identities Use named cases to keep array inputs intact and let the CI execution audit distinguish every malformed-filter test. * perf(mcp): batch BM25 symbol hydration * test(mcp): serialize native hydration coverage * test(eval): publish child PID atomically * test(mcp): cover hydration on native platforms --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Abhigyan Patwari <126312502+abhigyanpatwari@users.noreply.github.com> |
||
|
Gergő Magyar
|
17b056fc79
|
test(lbug): guard max DB size accounting across repeated checkpoints (#3529) (#3624)
On LadybugDB 0.18.3 (1.6.12, 1.6.13-rc.57) every CHECKPOINT left its shadow file handle's frame groups charged against max_db_size until the database closed (LadybugDB #924, fixed by #925 before 0.21.1): about 8-9 MiB per checkpoint, empty ones included. Analyze checkpoints every 5 s and per embedding window, so a long run can report "Maximum database size ... has been reached" while the file is small, and a retry in a fresh process succeeds. The new native test opens the database through initLbug with a 256 MiB ceiling and issues 128 driver-path checkpoints (tryFlushWAL). It fails on 0.18.3 with the reporter's message and passes on 0.21.1, so a future engine pin that reintroduces the leak is caught. Registered in the lbug-db project and the cross-platform native list. Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Gergő Magyar
|
bd20dde825
|
fix(taint): report inline and factory handler flows (#3612)
* fix(taint): cover inline and returned handlers (U8) Join expression-handler CFG summaries to distinct graph functions without changing existing anonymous-owner identities. Keep named function expressions private to their lexical scope. Related: #3489 * fix(ci): encode inline handler taint paths (U8) Use codec-safe generated display names while preserving positional function identity. Verify persisted hop continuity and remove the repaired inline-flow accuracy allowance. * fix: keep anonymous names unbound (U8) * test: pin inline callback captures (U8) * ci: run inline taint guard (U8) * test: align inline handler contracts (U8) Pin callback-owned call edges while preserving nested callable IDs and module-level HOC attribution. Assert repaired inline flows and the retained captured-closure limitation. * ci: run native inline proofs on all OSes (U8) Register real-worker identity/CFG and persisted explain cases on the existing Windows/macOS matrix. Charge explicit scheduling estimates without changing watchdogs or runtime behavior. * test(taint): reject all enclosing factory sources --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> |
||
|
Gergő Magyar
|
30676685e0
|
fix(rename): apply the exact reviewed edit plan (#3617)
* fix(rename): bind apply to reviewed plans (U7) Retain exact semantic edits behind single-use preview tokens and reject source, target, inventory and graph drift before application. Preserve explicit partial-write accounting and keep large unedited sources covered with hashes instead of retained text. Refs #2973 * fix(rename): cover unindexed sources (U7) Address PR #3617 review by binding every supported source to streamed byte hashes and canonical targets, including tracked and untracked files absent from the graph. Keep the exact reviewed edits and bound evidence memory. Use conventional even-sample medians and extend the existing CI benchmark to capture and validate unindexed sources without relaxing timing gates. Note: pre-existing packed Tree-sitter GLIBCXX_3.4.31 host failure in parse-cache-identity-package.test.ts is not addressed by this PR. * fix(rename): preserve lazy startup (U7) Keep language providers off the MCP startup path; load their existing classification only when rename inventory is requested. The existing 12 startup closure checks now pass. Address PR #3617 review by including token validation in reviewed apply timing and growing indexed evidence before preview, preserving the 20MiB controls under a 256MiB native pool. Migrate dispatch and retention tests to the preview token contract, keeping write-failure oracles. Note: pre-existing auto-sync /tmp fixture security and installed ARM64 grammar-header failures remain outside this PR; the packed Tree-sitter GLIBCXX_3.4.31 host failure also reproduces on unchanged main. * test(rename): always remove apply fixture (U7) Own the temporary directory from creation in an outer finally so setup and preview assertions cannot bypass cleanup. Keep spy restoration in its inner finally and preserve all write-failure assertions. Address PR #3617 cleanup feedback. * fix(rename): retain targets and config evidence * test(rename): verify approved physical output * test(rename): cover native filesystem matrix * test(rename): align preview contract fixtures --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> |
||
|
Gergő Magyar
|
f43834e814
|
fix(indexing): converge coverage around directory symlinks (#3616)
* fix(indexing): converge coverage around directory symlinks (U5) * style(bench): format symlink baseline (U4) * ci: run directory symlink coverage benchmark --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> |
||
|
Gergő Magyar
|
9b04fbe627
|
fix(embeddings): budget generated inputs and isolate rejections (#3615)
* fix(embeddings): budget generated inputs and isolate rejections (U4) * test(embeddings): require pending retry request * fix(embeddings): preserve Unicode in context * ci: run generated embedding input budget guard * test(embeddings): run input isolation on native OSes * fix: retain container identity after long prefixes * test: distinguish generated input cases --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> |
||
|
Gergő Magyar
|
e6ae04ed84
|
fix(embeddings): reject corrupt identities before mutation (#3614)
* fix(embeddings): reject corrupt identities before mutation (U2) * fix(embeddings): preserve exact Unicode keys (U2) * fix(storage): validate identities before opening CSVs * fix(embeddings): validate caches before mutation * ci: gate native embedding identity benchmark * test(embeddings): run identity proofs on native OSes * docs: describe CSV identity validation pass --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> |
||
|
Gergő Magyar
|
c8629f1b6c
|
fix(storage): preserve published graph on rebuild failure (#3613)
* fix(storage): preserve published graph on rebuild failure (U1) * test(storage): strengthen publication review coverage (U1) * test(storage): align publication evidence (U1) * fix(storage): retain completed recovery archives * fix(storage): preserve quarantines before cache reads * ci: run staged publication benchmark guard * fix(ci): declare WAL replay probe determinism Document that the bounded recovery read drains and discards its rows. Only WAL replay success is observed; row identity and query behavior are unchanged. The existing 21 query-determinism checks pass. * test(storage): cover publication across platforms * test(ci): exclude POSIX suite from Windows estimates * test: allow native publication retry to finish * test: verify Windows incremental recovery --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> |
||
|
Nguyễn Đăng Minh Lực
|
5c3961756b
|
fix(ingestion): preserve Go method ownership across files (#3559)
* fix(ingestion): re-point Go struct member edges when struct is declared in a sibling file (#3558) findEnclosingClassInfo mints a Go method's owner id from the file containing the method (Struct:<methodFile>:<Type>), while the Struct node is keyed by the file declaring the struct (parse-worker uses declFile). Same package, different file => the HAS_METHOD edge's sourceId points at a node that does not exist and the edge is silently dropped at COPY into LadybugDB. Add reconcileGoStructOwnerEdges, run at the end of the parse phase after ALL worker chunks are merged so every package's structs are visible. It rewrites only edges that would otherwise dangle, and only when exactly one Struct node shares the dangling id's package directory and simple name; 0 or >=2 candidates keep the old drop behavior (a wrong owner is worse than a dropped edge). Index build is lazy: dangling Struct sources are rare (struct fields never dangle — they live in the struct's own file), so the common non-Go or fully-same-file case pays two per-type edge scans and no whole-node pass. Regression test: test/integration/go-cross-file-struct-owner.test.ts (fails-before: 1 HAS_METHOD edge; passes-after: 3). * fix(ingestion): scope Go struct-owner reconcile by language and package Address PR #3559 review findings: - Scope the pass to Go: dangling HAS_METHOD/HAS_PROPERTY owner ids whose Struct:<file>:<name> file is not Go source (Rust, Swift, ...) are never re-pointed, and only Struct nodes stamped language 'go' (the marker the worker sets on every declared symbol) are candidates. Negative test pins a Rust-shaped dangling edge staying untouched beside a same-name Go struct. - Key candidate structs by (dir, declared package clause, name) instead of (dir, name): one directory can carry 'package pkg' and external-test 'package pkg_test' files declaring the same type name, which the old key saw as ambiguous and dropped fail-closed. Clauses are read via the shared inferGoPackageName derivation with a call-scoped cache. Integration test covers the 3-file layout (pkg decl, pkg_test decl, pkg_test methods file) and asserts the edge lands on the pkg_test.Widget node. - Keep the direct call in parse-impl with a TODO: the pass needs a graph-level post-merge hook, which no LanguageProvider/ScopeResolver hook provides today. * refactor(ingestion): remove Go graph repair from shared parse phase * fix(ingestion): resolve Go ownership during ingestion (language-provider.ts) * Add resolveDetachedOwnerName function to Go config * Refactor method owner population logic in Go * Refactor receiver binding to handle unnamed receivers * Enhance route handler method resolution logic Refactor route handler to improve ownership resolution and method indexing. * Add emitPostResolutionEdges to scope resolver * fix(ingestion): resolve Go ownership during ingestion (ownership.ts) * fix(ingestion): resolve Go ownership during ingestion (ast-helpers.ts) * Add resolveDetachedOwnerName to provider methods * Update SCHEMA_BUMP to 141 Increment schema version to 141 for new features and fixes. * fix(ingestion): resolve Go ownership during ingestion (decl.go) * fix(ingestion): resolve Go ownership during ingestion (external_decl_test.go) * fix(ingestion): resolve Go ownership during ingestion (external_methods_test.go) * fix(ingestion): resolve Go ownership during ingestion (foreign.rs) * fix(ingestion): resolve Go ownership during ingestion (go.mod) * fix(ingestion): resolve Go ownership during ingestion (local.go) * fix(ingestion): resolve Go ownership during ingestion (methods.go) * fix(ingestion): resolve Go ownership during ingestion (go-cross-file-struct-owner.test.ts) * Refactor comments and method registration in tests Updated comments for clarity and modified method registration to include qualified names and ownership. * fix(ingestion): resolve Go ownership during ingestion (has-method.test.ts) * Update SCHEMA_BUMP to 141 in tests * refactor(ingestion): remove obsolete Go struct owner reconciliation * fix(go): model build constraints for method ownership * test(go): cover build constraint grammar and platform variants * fix(go): resolve method owners across compatible build variants * test(go): verify variant ownership through ingestion pipeline This test verifies that build-variant owners are consistent across different workers, the semantic registry, and the graph. It sets up various scenarios to ensure that ownership is correctly resolved and that there are no dangling edges in the ownership graph. * fix(go): reject impossible known constraints when peer is unknown Address PR #3559 review feedback. Regression coverage follows in this update. * feat(ingestion): allow providers to group incremental resolution files Add the optional language-neutral hook used by the Go ownership writeback fix for PR #3559. * fix(go): group sibling files for incremental ownership resolution Use the existing Go package-directory key for the provider hook. * fix(incremental): collect sibling resolution groups for changed paths Keep group keys local to each provider and include current siblings of deleted files. * fix(analyze): persist ownership changes in unchanged Go siblings Expand changed and deleted resolution groups before importer traversal. Use the same effective file set for deletion and writeback so package-wide ownership changes reach the database. * test(incremental): cover provider-local resolution groups Cover changed, added, deleted and unrelated paths for the new sibling expansion helper. * test(go): compare incremental ownership writes with cold rebuilds Reproduce ownership changes after deleting and restoring a Windows declaration while the shared method file stays unchanged. Verify incremental writes and persisted ownership parity with forced rebuilds. * test(go): exclude impossible owners with unknown method constraints Use real parsing to cover impossible owner edges and valid owners hidden by impossible competitors. * test(go): pin filename rules and unknown constraint overlap Document Go first-dot filename behavior and verify impossible known constraints in both operand orders. * test(go): register incremental ownership coverage across platforms Include the new database-backed Go ownership suite in the cross-platform test list. * test(go): schedule incremental ownership suite with native database tests Complete PR #3559 review fixes. Local validation: 47 focused tests across five suites passed; build and source/test type checks passed. The full suite was interrupted without a result. * refactor(ingestion): centralize worker owner hook forwarding Pass the language provider into the private cached owner lookup instead of repeating four owner hooks at five call sites. Keep qualified-name selection explicit and preserve the exported helper signature. Validation: CLI/shared build and test typecheck pass; repository lint has no errors; 591 tests pass with one skipped across 13 ownership and worker suites. Web typecheck is blocked by uninstalled web dependencies. Graph checks are unavailable because this checkout has no registered index. * test(rename): add Go cross-file fixture module Part of the cross-file method ownership regression for PR #3559. The completed fixture uses real indexing, public rename preview/apply, warm cache replay and post-rename reindexing. All 14 Go rename integration cases pass locally. * test(rename): add cross-file fixture before/types.go * test(rename): add cross-file fixture before/methods.go * test(rename): add cross-file fixture before/callers.go * test(rename): add cross-file fixture after/go.mod * test(rename): add cross-file fixture after/types.go * test(rename): add cross-file fixture after/methods.go * test(rename): add cross-file fixture after/callers.go * Add test case for renaming pointer receiver method Complete the real persisted-index regression fixture for PR #3559. The type, pointer receiver method and callers are in separate Go files. Public rename preview/apply must make exactly four edits across the structural interface family while preserving an unrelated same-name method, comments and strings. The fixture checks six exact ownership, interface and call edges before rename, after warm parse-cache replay and after reindexing the edited sources. All 14 Go rename integration cases pass. Formatting and the CLI typecheck pass. Graph impact/detect-changes could not run because this checkout has no registered index. --------- Co-authored-by: System administrator <root@nixos-infra.home> Co-authored-by: Gergő Magyar <gergomagyar@icloud.com> |
||
|
Gergő Magyar
|
cae60c2bd4
|
fix(mcp): reject corrupt query identities and refresh index metadata (#3571)
Some checks are pending
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Scorecard / Scorecard analysis (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
* fix(mcp): reject corrupt query identities and refresh index metadata * fix(mcp): preserve reader and search integrity Keep replaced readers unknown until matching metadata is published. Defer shared-handle refresh until other consumers release the old graph to avoid rebuilding connections against the same stale generation. Propagate identity corruption after both hybrid search lanes settle, so the sibling cannot query after its database session is released. Select the regressions for the OS matrix and retain Python coverage without optional Swift grammars. Validation: 393 focused tests, nine native generation tests and ten race executions across five fresh processes pass. Build and source/test typechecks pass. The early full run passed 26,096 tests and failed eight: seven reproduce on baseline; the mixed-source publication case passes in clean reruns. Windows/macOS native execution remains for CI. Graph analysis maps all edited production symbols; coverage remains partial for the anonymous FTS test callback, manually reviewed. Refs #3571 * fix(lbug): bump @ladybugdb/core to 0.21.2 to stop doubled rows after delete and re-copy LadybugDB 0.21.0 and 0.21.1 return every row of a table that spans more than one native vector twice once the incremental writeback has DETACH DELETEd it and a COPY into an earlier table has run. NodeGroup::scan let a morsel that ended before the group's first live row wrap an unsigned subtraction and rescan to the end of the group (LadybugDB #1085, fixed in 0.21.2). No primary key error fires. The identity guard from #3442 stops the run with "unexpected or duplicate ID", so a Swift edit that pulls every importer into the write set failed on repos with more than 2048 structs. Only unreleased main was affected: 1.6.12 ships 0.18.3. Pin 0.21.2 in package.json, update the six @ladybugdb/core entries in the lockfile, and move coreVersion in the vendored FTS manifest. The FTS extension stays at 0.21.0 because no v0.21.1 or v0.21.2 artifact is published; the pairing assertion and its unit tests pass. Add a regression to the delete-nodes-for-files suite: 2049 structs with CONTAINS edges are loaded, every file is deleted, the graph is loaded again, and each id and edge must appear exactly once. Verification (Linux, Node 22.18): - lbug-delete-nodes-for-files: 11/11 on 0.21.2. With the 0.21.1 native packages swapped in, the new test fails with 4098 rows instead of 2049. - Reporter recipe end to end on the built CLI: incremental analyze exits 0 with no duplicate ids on a Swift-heavy fixture and on a 70-file Python fixture, with and without FTS. On 0.21.1 the Swift fixture exits 1. - A 0.21.1-written database opens and reads under 0.21.2. - prettier, eslint and typecheck:tests are clean on the changed files. - The full lbug-db project had not finished when this was committed. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * test(fts): say what the Windows warm-session branch covers On Windows the repair step calls poolClose, which removes the pool entry before the in-place writer opens. The step therefore checks the backend's cached capability state, not a live pool connection. The comment now says so instead of implying the pool stays warm. No behavior change. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * revert: take the @ladybugdb/core 0.21.2 bump out of this PR Reverts |
||
|
Gergő Magyar
|
8fa8e64388
|
fix(rename): extend safe refactoring and flow coverage across languages (#3557) | ||
|
Gergő Magyar
|
82347f0f07
|
fix(scope): resolve local imports by lexical ownership (#3532) | ||
|
Christian C. Berclaz
|
672fc72f56
|
fix(dart): capture package metadata portably during repository scan (#3465)
Some checks failed
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Scorecard / Scorecard analysis (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
Skill copy sync / shipped skills drift guard (push) Has been cancelled
|
||
|
Younes Beriane
|
047354c3f0
|
fix(community): run Leiden in a worker so its timeout can fire (#3478) | ||
|
Gergő Magyar
|
df49f90889
|
fix(ci): require every test to execute across the CI matrix (#3479) | ||
|
Gergő Magyar
|
526b6f249b
|
test(search): guard native lookup and Unicode FTS after COPY (#3477) | ||
|
Gergő Magyar
|
6bb99767ff
|
fix(auto-sync): HTTPS remotes, OpenSSH image, and rc embeddings (#3378)
* fix(docker): install OpenSSH in the CLI runtime image Auto-sync requires git SSH remotes, but the published image omitted openssh-client so every clone failed with ssh: not found. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(auto-sync): accept HTTPS remotes and reclone failed checkouts Allowlisted HTTPS URLs can clone without SSH keys, and a timed-out clone with no remote.origin is quarantined instead of blocking forever. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(auto-sync): honor .gitnexusrc embeddings and warn on empty vectors Auto-sync analyze now reads embeddings from the clone's project config, and query reports when an index has no vectors so keyword fallback is visible. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(review): warn when CodeEmbedding table is missing (U5) Co-authored-by: Cursor <cursoragent@cursor.com> * fix(ci): wrap long openssh-client test line for prettier Co-authored-by: Cursor <cursoragent@cursor.com> * fix(query): keep keyword-only indexes off query.warning Empty or missing CodeEmbedding is the default index. Put the #3372 notice in a once-per-backend log line so FTS-success query results stay warning-free. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(review): bound rc reads and quarantine only a missing origin Drop the implementation plan from the branch. Auto-sync reads .gitnexusrc through the bounded control-file reader, and a git config failure no longer relocates a live checkout. The same allowlisted repo can switch between SSH and HTTPS without a refused pull. Co-authored-by: Cursor <cursoragent@cursor.com> * refactor(auto-sync): share repo identity and skip a second origin read Co-authored-by: Cursor <cursoragent@cursor.com> * test(auto-sync): clean temp fixtures and cover nested embeddings precedence Co-authored-by: Cursor <cursoragent@cursor.com> * test(auto-sync): skip the symlink rc fixture on Windows Co-authored-by: Cursor <cursoragent@cursor.com> * test(auto-sync): reject symlink rc files on every platform Co-authored-by: Cursor <cursoragent@cursor.com> * test(ci): run auto-sync symlink and clone tests on Windows and macOS Co-authored-by: Cursor <cursoragent@cursor.com> * test(git-clone): keep Windows CI on file URLs and POSIX permission checks Co-authored-by: Cursor <cursoragent@cursor.com> * test(git-clone): keep the SSH-to-HTTPS origin check offline Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
Joseph Yared
|
b92c14cdd0
|
feat: add Factory AI (Droid) integration (#2543)
Some checks are pending
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Scorecard / Scorecard analysis (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
* feat(setup): add Factory Droid (MCP + skills) to gitnexus setup Register 'droid' in the editor-targets abstraction so `gitnexus setup -c droid` writes the MCP server to ~/.factory/mcp.json and installs skills to ~/.factory/skills/ from the single canonical skills/ source (no per-editor copies). uninstall.ts is target-driven, so removal is covered automatically. Adds unit + round-trip coverage. * feat(plugin): add gitnexus-factory-plugin for droid plugin install * docs: add Factory Droid to editor support table and setup docs * fix(factory-plugin): guard augment hook against fan-out and DB contention Reuse the Claude adapter's acquireHookSlot and LadybugDB owner probe (bundled byte-identical, kept in lockstep by a drift test) instead of running an unguarded augment. Add direct tests for the hook and manifests. * docs: align Factory row in editor support table * fix(factory-plugin): honor GITNEXUS_HOOK_CLI_PATH so augment runs on Windows * docs(hooks): point bundled guard copies at their drift tests * docs(factory-plugin): note the Execute tokenizer's quoting limit * docs(readme): clarify the Full tier and group the Factory row * docs(hooks): trim drift note to a single line * test(ci): run factory-plugin tests on the windows cross-platform lane * refactor(hooks): drop the drift-note comments, the tests already enforce it * fix(factory-plugin): pin CLI version and parse quoted shell patterns - Pin mcp.json and the hook's npx fallback to gitnexus@<version> from the plugin manifest, registered with the release sync script so a mutable @latest can never execute on MCP connect or augment fallback - Port the #2938 shell tokenizer (tokenizeShellWords + parseRgGrepPattern) so quoted, backslash-escaped, --regexp=, -eVALUE, and -- patterns survive - Add the #2938 regression matrix and pin assertions to factory-plugin.test.ts * docs: add Factory Droid to published npm README * fix(factory-plugin): wire marketplace so droid installs the Factory plugin Add .factory-plugin/marketplace.json sourcing ./gitnexus-factory-plugin. Droid reads it before .claude-plugin/marketplace.json, so `droid plugin install` now delivers the Factory plugin (Execute matcher, pinned mcp.json) instead of the translated Claude plugin (Bash matcher, gitnexus@latest). Register the surface in the version-sync script and cover the wiring in the factory and sync test suites. * fix(factory-plugin): use registry lookup for index resolution Bundle registry-query.cjs so external indexes resolve (#3060); re-pin to 1.6.12. * fix(factory-plugin): sync Execute parser with Cursor hook Fixes echo-rg and -f false positives; tighten test env isolation. * fix(factory-plugin): stop no-match augment from re-running via npx A PATH `gitnexus` that finds no match exits 0 with empty stderr, which fell through to a second `npx -y gitnexus@<pin> augment` with its own 8s timeout (16s worst case vs the 10s hook budget). Fall through to npx only when the PATH launcher is missing (ENOENT); any launched PATH binary, including a timeout or non-zero exit, now ends the augment. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix(factory-plugin): filter augment stderr to the [GitNexus] block runAugment returned raw child stderr, so npm/Node/LadybugDB warnings leaked into additionalContext and noise-only stderr counted as success. Port the Claude adapter's extractAugmentContext (verbatim, with isDebugEnabled) and apply it on every launch tier before the success decision. Adds a drift test against the Claude copy and PATH-tier noise/noise-only behavior tests. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix(factory-plugin): quote DROID_PLUGIN_ROOT in hook command An unquoted plugin root containing spaces (e.g. a Windows user profile path) split into multiple argv words, so the PostToolUse hook silently never ran. Quote it like the Claude plugin does, and pin the exact quoted command in the hooks.json wiring test. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix(release): stage Factory plugin manifests in the release commit The rc release job stages only the original four manifest surfaces in the detached release commit, so the v<version> tag tree carried the Factory plugin.json, mcp.json and marketplace.json at the previous version while --check (working tree) passed. Stage them too, and guard the git add block against the synced surfaces in sync-plugin-manifests.test.ts. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * refactor(factory-plugin): simplify hook gates, spawn tiers and tests - main(): resolve the repo only after the tool-name and pattern gates, matching the Claude/Cursor hook order (skips fs/git work on no-op calls). - runAugment(): share one spawnAugment helper between the GITNEXUS_HOOK_CLI_PATH and npx tiers; PATH tier ENOENT logic unchanged. - factory-plugin test: pre-filter comment lines instead of `continue`. - sync-plugin-manifests test: hoist EXECUTABLE_MCP_FILES and derive TOTAL_SURFACES from its length. - fnSource(): throw when the function or its closing brace is not found. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix(cli): list Factory Droid in localized setup help `localizeCliHelp` overwrites the `setup` command description with the `help.command.setup.description` i18n key, so the literal edited in index.ts never reached `gitnexus setup --help`. Add Factory Droid to the en and zh-CN keys. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Address PR review feedback (#2543) - factory hook: run every augment tier under the bundled Unix timeout guard (npx tier group-kills), keeping exactly-one-tier fall-through - hook-db-lock-probe: trim GITNEXUS_HOOK_{LSOF,PS}_PATH once so a padded override is used, not silently replaced (all 3 copies) - hook-lock: evict a stale slot via rename-to-tombstone + identity check, so a concurrently recreated fresh lock is never deleted (all 4 copies) - registry-query: a set-but-invalid storage override resolves no repo instead of falling back to the registry storagePath (all 4 copies) - publish.yml: stage the ten skill mcp.json manifests in the rc release commit; the staging test now requires every synced surface Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Address PR review feedback round 2 (#2543) - hook-lock: replace rename-to-tombstone eviction with an O_EXCL per-slot `.evicting` marker plus an identity re-check before unlink, so a live lock is never moved, and a crashed evictor leaves only a self-expiring marker (all 4 copies) - hook-db-lock-probe: clamp GITNEXUS_HOOK_PROC_CMDLINE_MAX to a named 256 KiB ceiling and require an integer, so an oversized override can no longer fail the buffer allocation and miss a live owner (all 3 copies) - registry-query: treat an empty GITNEXUS_STORAGE_PATH/ROOT as set but invalid, matching the CLI's `!== undefined` rule (all 4 copies); the factory test env now deletes those keys instead of blanking them Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Address PR review feedback round 3 (#2543) - hook-db-lock-probe: a capped /proc cmdline read stops early only once both the GitNexus token and the mcp/serve mode are present (or at EOF, the ceiling, or the budget), so a mode word such as `--require mcp` before the GitNexus path no longer hides a live owner (all 3 copies) - registry-query: correct the override comment; a filesystem root is invalid only for GITNEXUS_STORAGE_PATH, not GITNEXUS_STORAGE_ROOT (all 4 copies, comment only) Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Address PR review feedback round 4 (#2543) - hook-db-lock-probe: an fd-directory read error other than ENOENT or ENOTDIR on an identified server candidate now fails closed ('timeout') instead of reporting not-owned (EMFILE/ENFILE/ENOMEM/EINTR) - hook-db-lock-probe: resolve GITNEXUS_HOOK_TIMEOUT_PATH to an absolute path before validating and caching it, so callers that spawn with a request cwd can still execute the guard - hook-db-lock-probe: document the chunked cmdline read's actual stop conditions (all 3 copies) Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Harden hook-lock eviction marker lifecycle (#2543) Per the chosen option (B) for the stale-slot eviction race: - `.evicting` markers carry a per-call owner token (pid + random hex) - an evictor re-reads its token immediately before the slot identity check and unlink; a stalled evictor whose marker was broken backs off - `finally` removes the marker only while it still holds our token - an orphaned marker is broken only if, re-checked just before unlink, its bigint identity and token are unchanged from when judged stale - doc comment states the two remaining two-syscall windows (slot lstat->unlink, marker token->unlink); POSIX has no conditional unlink, and the worst case is one extra concurrent augment All four byte-identical hook-lock copies updated. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Fix CodeQL file-system race in hook-lock orphan-marker check (#2543) breakOrphanedMarker stat'd the marker by path and then read it by path, which CodeQL flags (js/file-system-race): the file could be replaced between the two calls. Take the stat and the token from one open descriptor (readMarkerSnapshot, O_NOFOLLOW where available) for both the "judged stale" snapshot and the pre-unlink re-check. All four hook-lock copies updated. The replaced-marker test injected its swap via a readFileSync(path) spy, which no longer fires; it now swaps the marker just before its second open, counting opens of the marker path only (a per-path counter fired early on slot-0 and let a mutant pass). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Unregister hook-lock exit listener on release (#2543) Each acquireHookSlot registered `release` as a process 'exit' listener that was never removed, so a long-lived process acquiring and releasing slots repeatedly would accumulate listeners (MaxListenersExceededWarning) and retain every closure. release() now removes itself. All four hook-lock copies updated; a test asserts 12 acquire/release cycles leave the 'exit' listener count unchanged. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Gergő Magyar <gergomagyar@icloud.com> Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
Nguyễn Đăng Minh Lực
|
5be80b1fc3
|
fix(lbug): self-heal read-only opens refused by an interrupted checkpoint (#3340)
Some checks are pending
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Scorecard / Scorecard analysis (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
* fix(lbug): self-heal read-only opens refused by an interrupted checkpoint Homelab repro 2026-09-19 (image 1.6.10-20260917, @ladybugdb/core 0.19.x): a wiki pod killed mid-CHECKPOINT left the engine's checkpoint artifacts on disk (lbug.wal.checkpoint / lbug.shadow / checkpoint intent+apply locks), and every later READ-ONLY open refused with 'Cannot open database in read-only mode while checkpoint is in progress' — permanently, until a writable open (any gitnexus analyze) happened to run. Two defects, both fixed: 1. The refusal was unclassified. ensureReadOnlyConnectionUsable (direct adapter) and openReadOnlyDatabase (pool) recovered missing-shadow and shadow-replay errors but rethrew this one raw, so 'LadybugDB unavailable for __wiki__' repeated forever. Add isReadOnlyCheckpointInProgressError (LADYBUGDB-CONTRACT, live-verified against 0.19.1) and route it through the same writable-open recovery — including at OPEN time, where the refusal fires before any probe can run (pool: init() moved inside the try; direct: doInitLbug catch). 2. The existing shadow-replay recovery was not durable. Reproduction matrix on 0.19.1: the writable probe replays the WAL in MEMORY only — without an explicit CHECKPOINT the engine drops the pages at close and the follow-up read-only open silently serves the pre-checkpoint state. Both recovery paths now CHECKPOINT after the probe, which applies the replay, consumes the sidecars, and clears the checkpoint locks. Verified end-to-end: real engine 0.19.1, killed-mid-CHECKPOINT state → exact refusal → pool adapter self-heals → 80,800 rows intact, sidecars consumed. The planted-signature integration test runs on any engine version (refusal asserted only where the engine emits it, 0.19+). * docs(architecture): list checkpoint-in-flight artifacts and the read-path self-heal * fix(lbug): address review findings — shared cursor closer, tighten structural guard Both findings from the gitnexus-check review on this PR: 1. pool-adapter.ts: the recovery CHECKPOINT closed its cursor with a bare unawaited result.close?.(). Use the shared best-effort closer (closeQueryResults) the repo already funnels both adapters through, so a cursor-close failure stays cleanup and cannot escape as an unhandled rejection. 2. sidecar-recovery.test.ts: the structural regex omitted the leading negation, so as a substring match it also accepted the inverted predicate (recover ONLY shadow-replay, exclude checkpoint) — the exact regression the guard exists to prevent. Pin the full '!isReadOnlyShadowReplayError(err) && !isReadOnlyCheckpointInProgressError(err)' throw-through shape. * test(lbug): wire the recovery plant into lbug-db/LBUG_NATIVE; force the refusal on any engine pin Address the tri-review findings (all four): P1 — the planted-signature integration test was collected by the parallel 'default' project and never by the serialized 'lbug-db' project, and Windows/macOS CI never ran it: register it next to its sibling in vitest.config.ts (lbug-db include + default exclude) and in cross-platform-tests.ts LBUG_NATIVE, per TESTING.md's rule for native @ladybugdb/core suites. Verified via 'vitest list --project lbug-db'. P2 — on the committed 0.18.3 pin the plant passes as 'pool opens and count(n)=300' without ever exercising the new classifier or recovery CHECKPOINT. Add forced-refusal behavioral tests for BOTH adapters: a mocked native layer whose first read-only Database refuses with the canonical 0.19 message, asserting the exact self-heal shape (ro-refused -> writable open -> CHECKPOINT -> ro retry) and that a healthy db never triggers a writable open. The direct adapter is lazy, so its refusal is scripted at the first probe query rather than init(). P2 — the LADYBUGDB-CONTRACT header on isReadOnlyCheckpointInProgressError claimed '^0.18.0' like its siblings; only 0.19.x emits this string (0.18.3 tolerates the state). State the first-observed version so a bump reviewer validates the right binary. P2 — the pool's writable replay recovery quarantined on missing-shadow even when the error came from the post-probe CHECKPOINT, where the main file has already changed: mirror the direct adapter's probeSucceeded guard (replaySucceeded) and fail closed instead of parking a live sidecar. Also assert walBuffer.byteLength > 0 in the plant so the fixture cannot silently degrade into an empty shell on tolerant engines. * test(lbug): fix hosted-CI failures — Windows handle release, version-gated plant, prettier Address the CHANGES_REQUESTED review of the hosted run: Windows blocker (Win32 Error 33, locked file region at the pooled reopen): the fixture now makes handle release explicit — waitForFixtureRelease probe-reads the db and its residual WAL with bounded retries after every native close (plant, raw refusal probe, pool close), mirroring the adapter's own Windows handle-release probing. The WAL is re-planted from the captured bytes instead of renamed: a close-time auto-checkpoint can consume the live .wal out from under the rename (ENOENT, second flake). Version-gate the plant itself: on < 0.19 engines that tolerate the planted signature, its synthetic sidecars are not a consistent staging state for the old engine (double-apply replays surfaced as 'Person already exists in catalog' — the third flake), and no refusal can be forced there anyway. The plant now skips below 0.19 with that rationale in-file; the behavioral contract on every pin stays with the forced-refusal units, and this native suite remains registered in lbug-db + LBUG_NATIVE so Win/macOS exercise it as soon as the pin moves off 0.18.3. Also: prettier on the two flagged files (format gate). * fix(lbug): keep failed checkpoint heals from re-entering CHECKPOINT Wrap recovery failures without repeating native refusal text, skip already-wrapped errors in doInitLbug, and pin constructor-time heal plus the Windows reopen skip. Co-authored-by: Cursor <cursoragent@cursor.com> * Address PR review feedback (#3340) - Clean each forced-heal tmpDir in afterEach so earlier cases do not leak - Compare major.minor when version-gating the interrupted-checkpoint plant Co-authored-by: Cursor <cursoragent@cursor.com> * Address PR review feedback (#3340) Reset the pool forced-refusal mock Database sequence in native.reset() so later tests can still script the first construction as the checkpoint victim. * Address PR review feedback (#3340) Count every MATCH on the pool forced-refusal mock and assert the writable replay probe so CHECKPOINT-without-probe cannot stay green. --------- Co-authored-by: Gergő Magyar <gergomagyar@icloud.com> Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
Abhinav Pandey
|
0ede6ae501
|
fix(rust): respect Cargo target boundaries in name fallback (#3294)
* fix(rust): respect Cargo target boundaries in name fallback * fix(rust): read Cargo sources through validated file descriptors * fix(rust): require matching imports for crate-root guesses * fix(rust): require Cargo root identity for cross-target imports * fix(rust): enforce Cargo identity across fallback imports * fix(rust): keep Cargo membership on typical derive/macros (#3294) Abort only genuine unknown expansion, ignore Cargo artifact layouts instead of every `target` path segment, and require a covering import for unanswered crate-root name fallback so #3253 still holds on ordinary Rust sources. Co-authored-by: Cursor <cursoragent@cursor.com> * test(rust): gate Cargo target membership in CI (#3253) Add a parse-dispatch-rounds-style bench so derive/macro abort, target-path globs, and superlinear membership walks fail in CI instead of staying graph-invisible. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(rust): verify Cargo macro and re-export evidence * style(bench): format Cargo membership benchmark * fix(rust): require imports for cross-file root guesses --------- Co-authored-by: Gergő Magyar <gergomagyar@icloud.com> Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
Gergő Magyar
|
21a52af1d4
|
fix(lbug): ship FTS per-platform and recover in-place native aborts (#3274)
* fix(lbug): pin Ladybug core so Dependabot cannot ship a skewed FTS artifact The extension version is a separate upstream constant. Ignore daily core bumps and fail the pairing gate when the committed manifest does not name the installed core. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): make doctor and CI FTS gates resolve the packaged artifact Doctor and the REQUIRE_FTS file gates still treated an empty ~/.lbdb as unavailable, which would turn three CI jobs red once analyze stops installing into that tree. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): name native-abort and tuple-missing so analyze cannot mis-advise The CLI summary's trailing else treated every unknown skip reason as a missing extension. New crash and platform causes must get their own remedies, not a network-install hint. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): delete the dead read-path FTS index create ensureFTSIndex had no production callers and swallowed read-only CREATE_FTS_INDEX failures, which hid the only signal that a reader tried to write. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): vendor per-platform FTS artifacts so analyze needs no host install Keyword search depended on a CDN fetch into ~/.lbdb. Shipping the five published tuples inside the package makes air-gapped and ignore-scripts installs load the same artifact the publish gate checksums. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): load the packaged FTS artifact before any network install Analyze still required a CDN fetch into ~/.lbdb even when the package already shipped the file. FTS now path-loads the vendored tuple first and records source labels so a later truncated home copy cannot steal the diagnosis. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): diagnose a core/extension version skew instead of a missing runtime A structurally valid FTS artifact whose path version disagrees with the packaged pin must name both versions, not prescribe VC++ or OpenSSL. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): stamp an FTS phase so repair stays usable after an in-place abort A native CREATE_FTS_INDEX abort leaves no skip reason; the next run infers it from the dirty flag, and --repair-fts must not treat that phase as a half-written graph. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): park an in-place FTS crash WAL without wiping the graph An FTS abort after a successful checkpoint must reopen the live index on macOS, Windows, and Linux. Staging never parks the live WAL; readers keep today's large-WAL refusal. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): refuse read-only opens of an FTS-poisoned WAL MCP and serve cannot repair a leftover in-place abort. Fail before the native open and name --repair-fts, on macOS, Windows, and Linux. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): name a vendor-neutral Windows OpenSSL prerequisite OQ1 is unanswered here so GitNexus does not ship OpenSSL DLLs. Windows FTS now asks for a system OpenSSL 3 runtime instead of Git Bash PATH. Co-authored-by: Cursor <cursoragent@cursor.com> * test(lbug): inject the FTS vendor root and redact it on HTTP and MCP Path-loaded artifacts no longer vary with HOME. Tests pass an injected vendor tree and assert search warnings never leak a filesystem path. Co-authored-by: Cursor <cursoragent@cursor.com> * docs(lbug): document load-only as the global FTS install default Analyze still overrides to auto. Packaged per-platform artifacts load before any network install on macOS, Windows, and Linux. Co-authored-by: Cursor <cursoragent@cursor.com> * docs(lbug): format the FTS install-policy README table Prettier does not run on Markdown in pre-commit, so the U10 table wrap needs its own formatting commit. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): skip FTS CREATE after a persisted native abort A recovered analyze run was retrying CREATE_FTS_INDEX from skipReason alone. Keep that skip until --repair-fts, fail closed on unsupported tuples, and honor the checkpoint warrant for park/repair. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): honor checkpoint flushed warrant and align FTS tests with packaged vendor A no-op CHECKPOINT must not satisfy the FTS park warrant, and CI still asserted HOME-only FTS isolation after analyze started path-LOADing the packaged artifact. Co-authored-by: Cursor <cursoragent@cursor.com> * test(lbug): accept a nonempty incremental write set in the #2790 recovery check FTS-phase recovery can incremental-add files (changed=0, added=1). That is not the #2790 empty-diff wipe skip. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): compare FTS home versions to the core pin and tighten the publish filename gate Ladybug's ~/.lbdb/extension directory is the runtime/core version; treating it as the artifact version false-diagnosed skew. The publish guard now rejects a path-escaping filename the same way the fetch script does. Co-authored-by: Cursor <cursoragent@cursor.com> * test(lbug): seed FTS e2e fixtures from the packaged vendor artifact A machine with no ~/.lbdb copy should still run the vendor-survivorship cases; the seed no longer depends on HOME or a network install. Co-authored-by: Cursor <cursoragent@cursor.com> * Address PR review feedback (#3274) Keep in-place FTS abort evidence after persist so a second CREATE abort cannot fail-open readers, and close the CLI, loader, embed, and e2e gaps the review called out. Note: full npm test hit Ladybug worker-pool startup failures under memory pressure; tsc and 180 targeted unit tests passed. Co-authored-by: Cursor <cursoragent@cursor.com> * Address PR review feedback (#3274) Run the vendored-path symlink guard on the OS matrix, put e2e HOME fixtures on Ladybug's real extension layout, pin the embed crash-WAL gate before the writable open, and let analyze writers park through missing-shadow recovery. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): keep --repair-fts CI green after vendored-first FTS Never-installed warning fixtures must not inspect a packaged vendor binary, and a failed dirty restamp must not abort an otherwise successful --repair-fts run. Co-authored-by: Cursor <cursoragent@cursor.com> * test(cli): give the #1169 analyze e2e the same 90s Windows budget as its sibling The first #1169 persist-meta case was still on a 60s spawn/it budget and was killed banner-only on windows-latest after the FTS warning fixture no longer failed the shard first. Co-authored-by: Cursor <cursoragent@cursor.com> * test(ci): reweight Windows shards after the FTS e2e grew Vendored-first HOME fixtures pushed fts-extension-e2e to ~6 minutes on windows-latest, so the old 146s weight packed it with skills-e2e and blew the 20-minute watchdog. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
Gergő Magyar
|
ceaff27c1e
|
fix(parse-cache): retire a chunk whose durable generation could not be reset (#3271)
* fix(parse-cache): retire a chunk whose durable generation could not be reset #3200 skipped the parse-cache write when `prepareDurableParsedFileChunk` failed, but the chunk hash was already in `usedKeys` from the lookup. When a previous generation existed on disk — reachable because the coherence gate re-dispatches a chunk whose `.v8` shard is live but whose durable shards are unreadable — `saveParseCache` copied that old shard forward, and the durable prune, which keeps exactly the saved keys, retained the mixed directory. The next run then served a warm hit out of a directory the previous run had already decided it could not account for. Retire the hash instead of only skipping the write: - `ParseCache.staleKeys` is a transient set that `saveParseCache` filters out of its key list. Filtering at save is what makes it survive the post-parse key merges in run-analyze (#2106 sibling fold, unreadable-meta retention), and it reaches both stores at once because the durable prune keeps exactly the keys `saveParseCache` returns. - The hash is retired at the reset-failure site, which runs unconditionally. The parse-cache write branch sits behind `rawResults.length > 0`, so a chunk whose worker round returns nothing would never have been retired there. - Worker-quarantined chunks get the same treatment for the same reason: they also reach the save with no in-memory entry, which is what triggers the copy-forward. That branch was previously unreachable when the worker died on the chunk and returned no results. - Guard the durable prune's non-survivor `fs.rm`. The causes that break the reset break that delete too, and it sat outside the validation try — one undeletable directory aborted the loop and cost every remaining chunk its index entry. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(review): apply review findings Retire a chunk only when a generation nobody cleared is still on disk. `prepareDurableParsedFileChunk` is rm-then-mkdir, and the catch could not tell the two apart: an rm that succeeded before a failing mkdir leaves NO directory, so the workers recreate it and write a clean generation. Retiring there discarded a good `.v8` for no safety gain — and under a correlated failure (an empty durable index turns every chunk into a re-dispatched miss, then a descriptor burst rejects the resets en masse) it would have wiped both shared stores for every branch, where the pre-#3204 posture cost only the writes. `durableChunkHasStaleShards` is the discriminator. Finish the delete guard on the path that runs before it. The staged→live overlay in `mergeStagedDurableParsedFileStore` awaited `replaceDurableChunkDir` unguarded, so on the cold-rebuild path one undeletable directory threw out of the merge before the prune ever ran — the durable index was never rewritten and a retired chunk kept its directory. Same log-and-continue treatment, plus best-effort handling of the two `.replacing` backup removals. Aggregate the prune's delete-failure warning: a store-wide cause hits every non-survivor, and one line per directory buries the message that matters. Tests: - Guard the chmod-based prune test with the repo's `skipIf` for root/Windows and assert the directory survived, so it cannot pass vacuously where the delete succeeds. - Add a two-chunk control: one chunk's reset fails, and the sibling must stay warm through the next run. One chunk plus a global spawn marker could not tell "retires the failing chunk" from "retires everything". - Add the rm-succeeded/mkdir-failed case, which must NOT retire. - Model both post-parse merges in the R4 test (the sibling fold re-adds the key, the unreadable-meta fallback unions `entries`), and move the in-memory `entries` assertion to a direct helper test — the sharded path never populates `entries`, so the old assertion proved nothing. - Type the cache factory as `ParseCache`; the `staleKeys` assertions were TS2339 and `?? false` read as a pass regardless. - Register the store test in the cross-platform filesystem list. Correct two comments that still described a quarantined chunk by the premise this fix disproves. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(review): stop swallowing the backup removal in replaceDurableChunkDir Swallowing that `fs.rm` manufactured the very hazard this PR removes. When a non-empty `${to}.replacing` survives, the following `fs.rename(to, backup)` cannot overwrite it and is suppressed as "dest was missing", so `backedUp` stays false and the `fs.cp(from, to)` fallback merges the staged generation INTO the live directory — old shards alongside new, which the prune then indexes as one valid survivor. Let it throw; the per-entry guard added to `mergeStagedDurableParsedFileStore` already stops one such chunk from costing the others their prune. The post-publish backup cleanup stays best-effort, where an undeletable leftover really is litter. Also: - Make the sibling-isolation test perform the run its title claims. It asserted index membership and stopped; an index entry does not exercise the warm-hit path, so it would have passed even if the sibling re-dispatched. Each chunk now runs alone so the single spawn marker names which one re-parsed. - Correct two comments that outran the implementation: retirement is gated on shards actually surviving, and an undeletable directory is dropped from the index rather than removed from disk. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
azizur100389
|
f8036ac349
|
feat(analyze): add explicit FTS opt-out (#3205)
* feat(analyze): add explicit FTS opt-out * chore(autofix): apply prettier + eslint fixes via /autofix command * fix(analyze): treat flag and env FTS opt-out as one mode Avoid a same-commit rebuild when only the skipReason discriminator changes, and advertise disablement on dirty meta before leftover indexes are wiped. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Gergő Magyar <gergomagyar@icloud.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
ClayLeee
|
ab25b86807
|
fix(typescript): resolve tsconfig paths aliases on Windows (#3203)
* fix(typescript): resolve tsconfig `paths` aliases on Windows `rebaseTarget` turns a `paths` target that `path.resolve` produced back into a repo-relative one, and it recognised the wildcard suffix only as `/*`. On Windows the resolved target is `C:\repo\src\*`, so the check never matched, the bare-`*` branch stripped the star, `path.relative` ate the trailing backslash, and every alias target came back as `src*`. `substituteStar` then built `srclib/date` from `@/lib/date`, nothing resolved, and every alias import was dropped as external — no IMPORTS/CALLS edge for anything reached through `@/`, so `impact` answered UNKNOWN for a repo's whole shared layer. Normalise the separator before looking at the suffix. `tsconfig-index.test.ts` already asserts the `src/*` shape and fails 4 of 13 cases on Windows without this; it only ever ran on Ubuntu, so add it to the cross-platform lane. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * test(typescript): make `paths` rebasing assertable from any runner `rebaseTarget` reads the platform separator, but the only suite covering it builds real directories — so on Ubuntu it sees `/` whatever the code does with `\`, and only the windows-latest lane can fail it. That is how an alias bug affecting every Windows install reached a green CI in the first place. Thread an injectable `pathApi` through `rebaseTarget` and `repoRelative` — the seam `isInside` and the `\\?\` prefix guard already use — and add a fixture-free suite that pins the win32 and POSIX branches explicitly. Deleting the separator normalisation now fails on every runner rather than only on windows-latest. Registered beside its fixture sibling in the cross-platform lane so the two halves of the same rule stay discoverable as one group. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(typescript): emit a bare `*` for a repo-root `paths` target A target naming the repo ROOT — `"*": ["./*"]` under `"baseUrl": "."` — rebases to an EMPTY repo-relative prefix, so only the suffix survived and the encoding came out as `/*`. `substituteStar` turns that into `/lib/date`, and `resolveFile` matches repo-relative keys without stripping a leading slash, so the indexed `lib/date.ts` misses and the alias is dropped as external. Emit the bare `*` for that case instead, which substitutes to `lib/date` and resolves. This is the shipped POSIX encoding as much as the Windows one, and it is what the Windows path emitted by accident before the separator was normalised — so the normalisation does not narrow what already resolved there. The common `"@/*": ["./src/*"]` is untouched: it has a real prefix and stays `src/*`. Pinned at all three levels the encoding passes through: the rebaser, on both path flavours; the loader, through a real fixture; and resolution, where `*` finds the file and `/*` does not. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs(test): say why the root-wildcard fixture declares no `baseUrl` The comment introduced the fixture as `"baseUrl": "."`, which the loader encodes as `''`, while the fixture passes `null` — a different scope, since `null` means the config declares no baseUrl at all and takes the paths arm alone. `null` is the right fixture and the comment was the wrong half: with `''` the baseUrl arm resolves `packages/utils/src/index` by itself, so the negative case would return null for a reason unrelated to the target encoding. Verified by substitution — `''` makes that assertion report the file instead. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> Co-authored-by: Gergő Magyar <gergomagyar@icloud.com> |
||
|
Gergő Magyar
|
9bf307123e
|
feat: notify users when a newer gitnexus version is available (#3175)
* feat(core): add cache-first npm update-check service Shared fail-open checker: validated 24h cache under GITNEXUS_HOME, acquireFileLock-guarded refresh, monotonic publication, hardened registry fetch (no credentials, private-address redirects refused, body-capped), strict x.y.z comparator, install-eligibility classification, and an unref'd refresh scheduler for long-lived processes. Extracts getGlobalDir into storage/global-dir.ts with a repo-manager re-export (no caller changes). Co-authored-by: Cursor <cursoragent@cursor.com> * feat(cli): notify on available updates via stderr and doctor One i18n'd stderr line on interactive invocations when the validated cache holds a newer version (TTY-gated, CI/opt-out/eligibility-gated, hook and help/version command identities excluded). Stale cache spawns a detached hidden __update-check refresh child so command exit latency is unchanged. doctor prints the cached latest version when known. Dockerfile.cli sets GITNEXUS_NO_UPDATE_NOTIFIER=1. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(mcp): emit one stderr update notice per process per version Process-scoped adapter in mcpCommand (stdio and --http), dynamically imported after the stdout sentinel, started only after connect, fully catch-isolated. Arms the shared refresh scheduler with cleanup on process exit. Protocol payloads stay free of update state (R15). Co-authored-by: Cursor <cursoragent@cursor.com> * feat(serve): expose update state on /api/info Serve-scoped controller owns an in-memory update snapshot: one staleness evaluation after listen, then the shared unref'd scheduler, stopped on close/shutdown. /api/info reads only the snapshot and gains optional latestVersion/updateAvailable fields for eligible installs; the existing three fields are byte-compatible. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(web): dismissible update-available banner from /api/info Fetches server info after backend connect and on reconnect, renders a fixed banner in the exploring view only when updateAvailable is true and the version is undismissed, hides while the reconnect banner is active, and fails open on fetch errors. role=status + aria-live with a keyboard-focusable dismiss; dismissal persists per version in localStorage. Copy in en/zh-CN common.json with version interpolation. Co-authored-by: Cursor <cursoragent@cursor.com> * docs(cli): document update notifications and opt-outs Co-authored-by: Cursor <cursoragent@cursor.com> * fix(review): apply review findings and simplify pass Review: gate the detached refresh spawn on a live lock-owner probe so parallel CLI invocations coalesce to one refresh child (validated P2, three-reviewer agreement); poll /api/info on a slow cadence while exploring so post-load server-side discoveries surface (validated P1); add a monotonic sequence guard so overlapping server-info fetches commit in order. Simplify (behavior-preserving): shared truthy-env/opt-out/freshness helpers in update-cache.ts, shared cachedUpdateNoticeLine for CLI and doctor, extracted install-eligibility core with per-process memo, memoized registry parsing, single evaluation per scheduler cycle, cache-only startup evaluate in serve, flattened MCP exit handler, shared bottom-banner shell, storage keys in ui-constants. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(update-notifier): address residual review tickets on this PR Stop the lock-busy 1ms scheduler spin, replace clock-skewed cache entries, move the outbound URL guard into core, and extract the serve update controller. Pin the startup/guard/single-flight/MCP/CLI contracts those tickets called out. Fixes #3167 #3168 #3169 #3170 #3171 #3172 #3173 #3174 Co-authored-by: Cursor <cursoragent@cursor.com> * Address PR review feedback (#3175) Fetch the npm /latest document instead of the full packument so the 64KiB cap can succeed, and treat reused lock PIDs as stale so refresh is not suppressed. Co-authored-by: Cursor <cursoragent@cursor.com> * Address PR review feedback (#3175) Register the CLI spawn suite on the OS matrix, pin MCP opt-out env, and compare versions without IEEE-754 rounding. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(cli): add gitnexus update install and versioned command banners Give an explicit Claude/Codex-style upgrade (`npm i -g gitnexus@version`) and print `GitNexus <Name> (version)` on every command so the running build is obvious without silent self-update. Co-authored-by: Cursor <cursoragent@cursor.com> * Address PR review feedback (#3175) - Document the pinned install as npm i -g gitnexus@<x.y.z>, not a copyable @version tag - Wait for wall-clock-future cache repair to publish before asserting - Restore the stdout spy if the TTY notice assertions fail Co-authored-by: Cursor <cursoragent@cursor.com> * fix(update-notifier): keep last known latestVersion on a failed refresh A later offline check was wiping the pin and hiding a known update for 24h. gitnexus update still treats a failed live fetch as checkFailed. Co-authored-by: Cursor <cursoragent@cursor.com> * Address PR review feedback (#3175) - Word update.current so a newer-than-latest install is not called the latest stable version. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(ci): hide the detached update-check spawn on Windows The refresh child was spawned without windowsHide, so Windows CI could stall before writing the cache and then fail cleanup with EBUSY. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
简律纯
|
5bad2d8b0b
|
fix(mcp): resolve omitted repo from cwd (#3085)
* fix(mcp): resolve omitted repo from cwd * test(mcp): cover cwd repository routing gaps * fix(mcp): harden cwd repository routing * docs(mcp): clarify cwd repository boundary * fix(mcp): preserve resolver compatibility * fix(mcp): align restricted repository routing --------- Co-authored-by: Gergő Magyar <gergomagyar@icloud.com> |
||
|
azizur100389
|
bf7dcf98ca
|
feat(analyze): add incremental watch mode (#3072)
* feat(analyze): add incremental watch mode * fix(watch): harden control file reads * fix(watch): contain refresh errors and bound reads * fix(watch): stream strict control file reads * fix(watch): harden refresh recovery and lifecycle * fix(watch): report ignored repository defaults * fix(analyze): preserve signal exit semantics * style(analyze): format signal exit helper * test(config): exercise descriptor growth guard * test(watch): await source event before rename * fix(watch): keep live-index retries honest and ignore analyzer writes Hold retry backoff when events merge, stop only after a live-index mutation, skip .gitnexus self-writes, and reject the remaining one-shot watch flags. Export impact-risk scoring from gitnexus-shared so consumers can share the same scale. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(watch): contain queue edge cases after review Preserve overflow-only refreshes, contain synchronous refresh failures, and mark successful atomic publication before later operations can fail. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Gergő Magyar <gergomagyar@icloud.com> |
||
|
azizur100389
|
f64cc8b7a8
|
feat(group): add GraphQL cross-repo contracts (#3070)
* feat(group): add GraphQL contract extraction * fix(group): tighten GraphQL contract guards * fix(group): complete GraphQL review hardening * fix(group): isolate bounded GraphQL reads * fix(group): harden GraphQL contract extraction |
||
|
DuduPhudu
|
2c0fb7753c
|
fix(group): stop reporting what could not be measured as a measurement of zero (#3012)
* fix: surface unreadable group indexes and escape raw NUL bytes in source Two independent diagnostics failures, both of which turn a real error into a confident, benign-looking answer. **Unreadable member repos (#3011).** `syncGroup` wrapped `initLbug` plus all contract extraction for each member in a bare `catch {}` that pushed the repo onto `missingRepos` and discarded the error. A LadybugDB storage-version mismatch therefore surfaced as "repo not found", `group sync` printed `0 contracts, 0 cross-links` and exited 0, and the existing contracts.json was overwritten with an empty registry. The two states need different answers from the operator — a missing repo must be indexed, an unreadable one is usually version skew or a lock — so they are now separate: - the caught error is logged with the repo, group path and lbug path - `unreadableRepos` is tracked alongside `missingRepos` on `SyncResult`, persisted (optionally, so older registries still parse) on `ContractRegistry`, and threaded through `GroupService` sync/status - `group sync` reports both before the cascade counts, since an unread repo is the likely explanation for a small or empty count - `group status` reports unreadable repos separately; calling them "missing" actively misdescribed them - when EVERY configured repo fails to open, the write is skipped: an extraction that read nothing is not evidence the group has no contracts, and replacing a good registry with an empty one loses data while reporting success **Raw NUL bytes (#3010).** `sync.ts` and `free-call-fallback.ts` each used a NUL as a join delimiter, written as a literal 0x00 instead of `\0`. Identical at runtime, but it makes the file test as binary: `file(1)` reports `data`, ugrep returns empty with exit 1 — indistinguishable from "no match", with no message — and BSD grep replaces matching lines with "Binary file ... matches". A search that should hit comes back as a confident "not present". Both now use the escape, and a unit test fails on any raw control byte in src/ so it cannot silently return. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(hygiene): guard every tracked source file against a raw NUL, not just src/ The guard added with the NUL escapes only scanned gitnexus/src for .ts/.tsx. Neither prior recurrence of this defect in this repo was in that scope: |
||
|
Gergő Magyar
|
740f0a4e57
|
fix(skills): publish gitnexus-plan artifacts on macOS without an interpreter (#2905) (#2922)
* fix(skills): anchor gitnexus-plan safe writer on macOS (#2905) The safe generated-plan writer refused to run on anything but Linux. `requireDescriptorAnchoring()` hard-gated `process.platform !== 'linux'` because every name it resolves went through `/proc/self/fd/<fd>/<child>`, and publication went through `renameat2(RENAME_NOREPLACE)`. macOS has neither, so `write-plan` and `read-plan` failed on every input and `snapshot` failed whenever a materialized path was absent. Node cannot perform openat-style directory-relative resolution on macOS at all: `node:fs` exposes no dir_fd parameter, and `fcntl(F_GETPATH)` is a snapshot string that XNU reconstructs from the name cache, so using it would reintroduce the exact race this helper exists to prevent. Python does expose the *at() family via dir_fd, and macOS has renameatx_np with RENAME_EXCL, so the anchoring borrows the interpreter the writer already spawns for renameat2. Anchoring now goes through a backend with two implementations. The Linux one keeps the original expressions, flags, ordering and error strings. The Darwin one runs each operation in the integrity-checked python3: it re-walks the chain from the repository root with O_DIRECTORY|O_NOFOLLOW, asserting the caller's recorded device, inode and mode at every level before acting. A chain that fails that assertion reports a dedicated anchoring errno and never ENOENT, so a moved parent cannot be read as an absent file. Node holds an open descriptor on every chain element for the anchor's lifetime, which pins the inodes so their numbers cannot be recycled between spawns, and that coupling is re-checked on the way into every request rather than left implicit. A filesystem that answers ENOTSUP to RENAME_EXCL is a refusal, never a fallback to a replacing rename. Every other platform is still refused. The suite had silently skipped on every non-Linux runner, so it is now gated on linux-or-darwin and registered in the cross-platform test list, which puts it on the macos-latest CI matrix. Disclosed rather than papered over: operations that must hand Node a file descriptor are anchored in the helper and then opened lexically with O_NOFOLLOW and identity-compared. A racer can force a mismatch, which aborts, or land on the inode the anchored walk already found, which is harmless. A perfect ABA inside that window is impossible on Linux and detected in all but its narrowest form on macOS. The reference doc says so. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NtGZN6YSNU8chALSnKYHBp * test(skills): normalize the anchoring-gate fixture repo on Windows The two capability-gate tests are the only ones in this file that run on Windows, and both failed there: `createBaseRepo` returned the path `os.tmpdir()` gave it, which on Windows is the 8.3 short form (C:\Users\RUNNER~1\...). `assertRepository` compares fs.realpathSync of the caller's path against the realpath of `git rev-parse --show-toplevel`, and plain realpathSync does not expand short names while git always reports the long form, so the helper rejected its own fixture with "--repo must be the Git worktree root" before either platform gate was reached. Resolve the fixture with the native resolver, which returns the canonical long path. No-op on platforms where the two already agree. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NtGZN6YSNU8chALSnKYHBp * test(skills): skip the darwin backend gate on Windows Spoofing process.platform does not spoof fs.constants. Windows Node defines no O_DIRECTORY, so a darwin-spoofed run there refuses at the anchoring-flag check and returns that message instead of ever reaching the python3-backend branch the test exists to cover. Skip it on win32 rather than loosening the regex, which would also let a macOS run pass on the wrong message. The sibling test still asserts the Windows refusal on Windows. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NtGZN6YSNU8chALSnKYHBp * refactor(skills): tighten the macOS anchoring backend Quality pass over the Darwin backend. No behaviour change was intended on the success paths; the guarantees are the same or stronger. Structural: - openChildRead now proves identity inside the backend instead of by comment. It was returning a raw descriptor from a lexical open, with the "callers always compare against the preceding anchored stat" invariant enforced across four call sites in prose — and since the Linux predicate is a literal `return true`, a fifth caller that forgot would have been an unanchored open on macOS that Linux CI could not see. It routes through darwinAdoptAnchoredFile, which already did open-then-compare-then-close-on-mismatch for createChild. - recordAnchoredAbsence shares one prefix walk per snapshot instead of re-walking from the repository root for every absent cited path. With three absent paths under a three-deep prefix that is 12 helper spawns down to 6 and 12 retained descriptors down to 4. citedPaths is caller-supplied and unbounded, so the descriptor retention was the real problem; the cache is now the sole close owner. This does change Linux descriptor lifetime — prefixes stay open for the snapshot rather than only the tail, deduplicated across paths. - assertRepository and the sibling realpath comparisons use realpathSync.native. Windows hands back 8.3 short names that plain realpathSync preserves while git reports the long form, so `snapshot`, which is not platform-gated, could reject a worktree root by quoting that same directory back at the user. The fixture workaround that papered over this for the new gate tests is gone. Efficiency, all measured at ~13.5ms per helper spawn: - consume the identity mkdir already computed rather than re-stat it - act on renameNoReplace's return value rather than spending two stats re-deriving what it already reported - drop a duplicate anchored stat taken twice in a row in movePathToVault - import ctypes only where it is used; 19 of 20 spawns never touch it Simplification: pins folded into the descriptors the handle already carried, an unreachable refreshAnchorTail branch and the dead darwinHardenedOpen mode parameter removed, the four copies of the spawn options collapsed, the spawn-and-parse shared between the probe and the request path, the unreachable launch-path fallback and a redundant memo deleted, and the helper's dispatch made a real elif chain with leaf name and mode validated at one chokepoint rather than per operation. The two chain encodings were left alone deliberately: merging them would have grown triple fields on Linux for no Linux benefit and changed the Linux validatePlanParent comparison. The double re-stamp that motivated the merge is contained in one named helper with the hazard documented. Rejected candidate interpreters now say which dir_fd operations were missing instead of producing a generic refusal. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NtGZN6YSNU8chALSnKYHBp * refactor(skills): publish plans with link(2) and drop the interpreter The macOS backend spawned python3 for two jobs: openat-style resolution, which Node cannot do, and a no-replace rename. Only the first is actually unavoidable, and the second was carrying the whole dependency. link(2) is a no-replace publish. It is atomic, it fails EEXIST when the destination name is taken, and it refuses a symlinked destination without following it — the same guarantee renameat2(RENAME_NOREPLACE) and renameatx_np(RENAME_EXCL) give, reachable from plain fs.linkSync. The published file is the same inode as the verified temporary, so the downstream identity checks hold by construction rather than by argument. That removes the interpreter from Linux entirely, since /proc already did the resolving there, and it removes ctypes, libSystem, RENAME_EXCL and the ENOTSUP handling from macOS. Deleted with them: the trusted-executable validation, the held-descriptor exec and its two-tier probe, the capability probe, the JSON request protocol, and both embedded Python programs. The helper drops from 3047 to 2327 lines. macOS keeps the part that genuinely cannot be done in Node, and now does it without a subprocess: a lexical O_NOFOLLOW walk that holds an open descriptor on every directory in the chain and re-proves the chain either side of every step. Pinning is load-bearing — an open descriptor keeps its inode number from being recycled, which is what makes the recorded identities trustworthy across steps. The guarantees are no longer symmetric and the docs say so plainly. /dev/fd/<fd> is a devfs node, not a magic link: opening it works, resolving through it does not, open("/dev/fd/<fd>/child") returns ENOENT and realpath returns /dev/fd/<fd> — measured on macOS 26 rather than inferred. So Linux makes a parent swap impossible while macOS detects one and aborts. Also fixes the writer on 9p mounts, where renameat2(RENAME_NOREPLACE) returns EINVAL and publication failed every time; link(2) succeeds there. Tests 174 -> 154: dropped 29 fixtures that drove the deleted Python program directly, added coverage for the link publish, for a macOS parent swap caught through the pinned chain, and for a spoofed-darwin round trip that asserts no /proc path reaches the hooks, which the portable backend now makes runnable on Linux CI. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NtGZN6YSNU8chALSnKYHBp * fix(skills): drop O_NOFOLLOW_ANY, guard trailing slashes, handle link edge cases macOS CI rejected our hardened directory open with EINVAL on 30 tests. The flag O_NOFOLLOW_ANY was ORed into every open on the theory that XNU ignores unrecognized open bits, so it would be inert where unsupported. That theory is wrong, at least combined with O_DIRECTORY. The Python design never hit it because the walk ran inside the interpreter; once Node did the opening, every Darwin directory open went through it. Removed rather than probed. The per-component O_NOFOLLOW walk is what delivers the guarantee, and cap-std — the closest reference implementation of this problem — has not adopted O_NOFOLLOW_ANY either. A fixture now pins the exact flags of every directory open under a spoofed darwin, so the next failure names the flag instead of printing a stack trace. With the flag gone the two backends' directory open became identical, so it is no longer a platform concern at all. Three findings from researching the prior art, all now covered: Trailing slashes. CVE-2026-39822 escaped Go's os.Root because open(fd, path, O_NOFOLLOW) follows symlinks when the path ends in "/". It reproduces here: with docs a symlink, opening "docs" is ENOTDIR but "docs/" succeeds into the attacker's directory, and path.join preserves the slash. We were safe only by construction, and only for repo-derived names — the generated temporary and vault artifact names never passed through the validator. The guard now sits at anchoredChild, the single place a name becomes a path, so it holds for every caller. link() can lie on NFS. Per link(2) BUGS, the return code may be wrong if the server creates the link then dies before replying; open(2) NOTES gives the remedy, which is to stat the source and treat a link count of 2 as success. Implemented, with the man-page reasoning in the comment so it is not later removed as paranoia. Filesystems without hard links now fail loudly. EPERM, ENOTSUP and EMLINK say so and refuse to fall back to a replacing rename. Git falls back and accepts losing collision detection because its objects are content addressed; that reasoning does not transfer to a named plan destination. Durability was already correct — the temporary is fsynced before publication and the parent directory immediately after — but the comment now records why the parent fsync is required for link as it was for rename, and the honest limitation that fsync is not a write barrier on macOS while F_FULLFSYNC, which Node cannot reach, is. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NtGZN6YSNU8chALSnKYHBp * refactor(skills): shrink the anchoring seam and fix two CI breaks Four quality reviews over the pure-Node writer. Two real breaks, one drift that had already happened, and a seam that was sized for a design we deleted. The macOS round-trip fixture asserted that every observed path started with join(repo, 'docs/plans'). Reproduced on Linux by handing the helper a repo reached through a symlink, which is the shape macOS gives us via /var to /private/var: assertRepository realpaths the repo, so the handle builds paths from the resolved form while the fixture holds the form it passed in, and the prefix can never match. The assertion now proves the same thing without depending on the prefix — a lexical resolution always contains a docs/plans segment and /proc/self/fd/<fd>/<name> never does. Two publish fixtures sat in the capability-gate describe, the one block deliberately not skipped on unsupported platforms, while this PR added the file to the Windows matrix. They test link(2), not the gate, so they moved to SAFE_WRITE_FIXTURES. validatePlanParent restated verifyLexicalChain's loop without the try/catch that converts ENOENT and ENOTDIR into the parity message, so a raw errno could escape a function with a dozen call sites. It was masked on Darwin only because parentStillResolves catches first. It now calls the helpers, which also removes a second full chain walk per call there. openVerifiedFile adds O_NONBLOCK so a FIFO swapped in at the target name cannot wedge the process on open, and only Darwin was calling it. The operations are now shared, so Linux gets it by construction rather than by a per-backend decision. The backend is five methods rather than ten. The platform difference is two things — how a name becomes a path, and what guard wraps an operation — so the five operations became shared functions over a `verified` hook that is run() on Linux and the pinned-plus-lexical sandwich on Darwin. openChildRead always runs the identity adoption, so that proof is structural rather than a comment about what callers must remember. Selecting the backend is a registry that throws on an unknown platform instead of a ternary defaulting to Linux, which surfaced seven dead bindings that ran before the capability gate and made win32 report the registry error instead of the refusal. Snapshot capture no longer re-walks a prefix per record: 36,018 lstats to 6,384 and 162ms to 130ms on 2,000 dirty files across 100 directories, with a byte-identical global_dirty_digest. Absence anchoring is now bounded at 4096 pinned directories and refuses rather than evicting, because closing a cached descriptor would break the pinned chain of a guard already recorded — the inode-recycling hole the pins exist to close. The test suite no longer cache-busts its imports. That existed for the memoized python3 descriptor, the file's only mutable module binding, which is gone; the suite drops from 10.0s to 8.2s. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NtGZN6YSNU8chALSnKYHBp --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Gergő Magyar
|
e69d3c49c4
|
fix(analyze): gate FTS-indexed DML before the incremental writeback (#2841) (#2854)
* fix(lbug): never report a drop that could not happen, and gate FTS-indexed DML `CALL DROP_FTS_INDEX` is itself an FTS-extension function, so with the extension unloaded it fails with `Catalog exception: function DROP_FTS_INDEX is not defined`. `isBenignDropFtsIndexError` classifies that as "nothing to drop" — correct when the index does not exist, wrong when it does: the drop silently no-ops and the next write to that table dies at bind time with an engine message that never mentions FTS (#2841). The classifier stays pure (a message cannot tell you whether an index is live). Instead `dropFTSIndex` settles liveness with a catalog read on the ERROR path only and raises an FTS-named, remedy-bearing error when the index is present but undroppable. Adds `ensureFtsRowDmlSafe`, the FTS twin of `ensureEmbeddingRowDmlSafe` (#2623): catalog first, load FTS with the analyze policy only when an index actually gates DML. LadybugDB refuses that DML at BIND time — a DETACH DELETE matching zero rows fails exactly as hard as one matching thousands — and the indexes cannot be cleared in place, so a verdict is the only useful answer. Both gates now share one `SHOW_INDEXES` read via `readIndexCatalogRows`, so adding the FTS check costs no extra catalog round-trip. Refs #2841 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CxokkpfssUCxvBwRZMtRCB * fix(analyze): escalate instead of crashing when FTS blocks incremental DML The incremental writeback decided its write plan without ever asking whether row-level DML was legal. On a DB carrying FTS indexes with an unloadable FTS extension, `deleteNodesForFiles` then died mid-writeback: Binder exception: Trying to delete from an index on table File but its extension is not loaded. with no mention of FTS anywhere in the run — the only install-capable load happened in Phase 3, long after the writes (#2841). The incremental branch now reads the index catalog once and derives both extension verdicts before any DML. When FTS (or VECTOR) blocks in-place writes, the run falls through to the existing wipe-and-bulk-COPY escalation — the same answer #2623 gave for VECTOR, and the only one available, since the indexes cannot be dropped without the extension. Every blocked extension is named in the reason log, not just the first one checked: a DB can carry both a vector index and FTS indexes, and reporting half the cause is how this failure stayed mis-diagnosed. Refs #2841 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CxokkpfssUCxvBwRZMtRCB * test(analyze): cover the FTS DML gate, both-blocked escalation, and the drop guard New `incremental-index-extension-dml-gate.test.ts` drives the real `runFullAnalysis` against a real mini-repo and a real LadybugDB: - a DB carrying FTS indexes with FTS made unloadable escalates to a full DB write, names FTS in the log, ends with zero FTS indexes, and still has the newly committed content in the graph (pre-fix: Binder exception, exit 1); - FTS available keeps the surgical plan and the indexes; - a DB that never carried FTS indexes is not escalated (the catalog-first check must not tax FTS-less machines); - FTS and VECTOR both blocked produce ONE escalation naming both. `drop-fts-index-error-classification.test.ts` gains the two `dropFTSIndex` cases the #2841 guard turns on: live index + unloaded extension rejects with an FTS-named error, absent index still resolves. The existing classifier assertions are unchanged — it stays pure. The CLI e2e reproduces the reporter's exact journey (analyze with the extension, remove it, touch a file, analyze again) and asserts exit 0 plus an FTS-named reason. It skips visibly when the seeded extension cannot load on the host, so it can never report a false red about the fix. Mutation-verified: reverting the run-analyze gate fails the first scenario; reverting the dropFTSIndex guard fails the live-index case. Refs #2841 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CxokkpfssUCxvBwRZMtRCB * fix(lbug): make every catalog-gated path fail closed, and classify the drop remedy Review findings on #2854 (two-engine, 17 lanes). H3 — `ftsIndexExistsInCatalog` returned `false` when the catalog could not be read, i.e. "index absent", so `dropFTSIndex` swallowed the error and the caller proceeded as if the index were gone. That is the #2841 symptom the guard exists to make loud, and it contradicted the contract `readIndexCatalogRows` states two functions above. It now fails closed. §6.A — `ensureFtsRowDmlSafe` keyed on `index_type === 'FTS'`, which answers `undefined === 'FTS'` → false → *no gate* for a row whose shape cannot be read: fail-open, in the gate whose only job is preventing an unsafe write, while the VECTOR twin fails closed on the same input. Now only a positively-identified non-FTS index is waved through. Deliberately NOT the twin's `!== 'HASH'`: that is safe there only because it is scoped to the embedding table first, and this gate is table-agnostic — `!== 'HASH'` would let the HNSW index gate FTS DML. §5.A — `undefined` was overloaded: "caller passed nothing" and "caller tried and could not prove anything" shared one value, so a failed shared read silently became three reads and the two gates could decide from different snapshots. The failed snapshot is now representable (`INDEX_CATALOG_UNREADABLE`), leaving one unambiguous `??` in `resolveGateRows`. §5.B — both gates regained the unconditional null-connection precondition the refactor moved into the reader. §5.G — the throw's remedy now routes through `diagnoseExtensionLoad`, like `--repair-fts` and `ftsDegradedWarning`, so a missing runtime dependency is not told to reinstall. The message stays path-free (#2374/#2375). The dead positional row fallbacks are kept and marked `LADYBUGDB-CONTRACT`: removing them would turn a proven-inert hedge into a fail-open gate if a future engine returns unnamed tuples. Refs #2841 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CxokkpfssUCxvBwRZMtRCB * fix(analyze): never undo an explicit wipe, stage extension-forced rebuilds, report honestly Review findings on #2854 (two-engine, 17 lanes). H1 (P1, both engines) — `analyze --drop-embeddings` was silently reverted. The `--drop-embeddings` → `force` conversion sits inside the `embeddingCheckpoint` branch, so without a checkpoint the run stays incremental and reaches the gate; the flag then *deliberately* leaves `cachedEmbeddings` empty, which is exactly the rescue's trigger, so every row the operator asked to destroy was read back and restored, exit 0. Widening the rescue from `!embeddingRowDmlSafe` to `extensionForcedRebuild` moved that latent bug onto the dominant path, because every analyzed DB carries FTS indexes. Guarded on the flag itself — NOT on `shouldLoadCache`, which is false in the meta-under-reports case the rescue exists for and would have deleted the safeguard while fixing the wipe. The `--drop-embeddings --embeddings` variant is covered by the same guard. H2 — an extension-forced escalation wiped the LIVE index in place: `buildPath` was frozen ~440 lines earlier while the run was still classified incremental, so an interrupt or ENOSPC left no complete index, where main failed at bind time with it intact. Extension-forced rebuilds now build into a staging file and publish via the existing atomic swap; size-forced ones stay in place, since that trigger is the repo's own churn rather than a machine condition. H5 — the escalation log asserted a vector index "exists" and that the store "carries FTS indexes" in exactly the case the catalog read proved nothing, while the only truthful signal went to stderr rather than the IPC log. It now emits a distinct unreadable-catalog cause, and "this index carries" (which pointed at the vector index just named) reads "the graph store carries". §5.D — the write-set cause was dropped whenever an extension cause co-occurred; causes are appended now, not selected between. §5.C — after an FTS-forced rebuild stamped lastCommit, a plain rerun on the same commit hit the alreadyUpToDate fast path before Phase 3, so the CLI's "install … then rerun" advice could never restore FTS. The fast path is now bypassed when meta records FTS unavailable and the extension can load again, keyed on the persisted capabilities stamp rather than new state. §5.F (skip the escalation for a zero-change commit) is deliberately NOT implemented: `deleteSpringAutoConfigurationSyntheticClasses` and `deleteSpringAopEvidenceNodes` run unconditionally on the surgical branch and bind against FTS-indexed `Class`/`CodeElement`, and a zero-row DETACH DELETE fails at bind time exactly as hard as a large one — so the skip would restore the original crash. Refs #2841 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CxokkpfssUCxvBwRZMtRCB * perf(search): read the index catalog once per drop sweep, and state the real contract Review findings on #2854. H4 — on a machine where FTS cannot load and the DB carries no FTS index, the gate correctly returned early without loading the extension, but the surgical path still ran the full 20-entry drop sweep: every `CALL DROP_FTS_INDEX` raised "function DROP_FTS_INDEX is not defined", and the new liveness guard then fired a fresh catalog read per table — 20 reads every run, forever, for exactly the offline/load-only population, contradicting the "healthy path costs nothing" claim shipped with the guard. The sweep now reads the catalog once and skips entirely when no FTS-typed index exists. An unreadable catalog runs the sweep, so an unprovable catalog never skips real work. H8 — the docstring still promised `dropFTSIndex` "tolerates" an unloadable extension. Post-#2854 a live index plus an unloadable extension throws, and safety rests on caller ordering discipline rather than the type system — which is what would have talked the next caller out of that ordering. GUARDRAILS — the "switching to a full DB write" sign described exactly one trigger (write set >~50%). Since #2623 and #2841 an unloadable extension escalates regardless of write-set size; documented with its recovery steps. Refs #2841 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CxokkpfssUCxvBwRZMtRCB * test(analyze): cover the wipe guard, the staged rebuild, and the fail-closed branches Review findings on #2854. H1/H2 mutation-verified: removing `!options.dropEmbeddings` fails the new drop-embeddings case ("expected true to be false"); disabling the staging upgrade fails the staging case ("expected 0 to be greater than 0"), so both assert behaviour rather than describe it. Gate suite (7 cases): `--drop-embeddings` under an FTS-forced escalation ends at zero embedding rows and logs no "Preserving"; the escalation is one-shot — a third run on a healthy host returns to surgery and rebuilds every FTS index; an extension-forced rebuild is observed building into `lbug.staging.*` and leaves none behind; the rescue complement still preserves un-stamped rows when no wipe was requested; the never-built case now asserts the commit reached the graph. H6 — the both-blocked case hard-asserted `createVectorIndex()` while the suite probed FTS only, so it went red on any FTS-yes/VECTOR-no host. VECTOR is probed now and gates only that case, with a GITNEXUS_REQUIRE_VECTOR hard-fail. H7 — the fail-closed branches had no coverage although the VECTOR twin's test and interception technique were ready to copy: `ensureFtsRowDmlSafe` under an unreadable catalog now proves it routes to the load, and `dropFTSIndex` proves it rejects rather than silently tolerating. Plus a redaction case that forces a real path-bearing load failure — under policy `never` the assertion would have been vacuous, since that reason carries no path. §5.E/§6.B — the suite is registered in the cross-platform matrix (its sibling was; it wasn't, and GITNEXUS_REQUIRE_VECTOR is set only on that job) and moved into the sequential lbug-db project per TESTING.md:68, verified not to drop it from the sharded ubuntu job. A Windows shard weight is added as a labelled estimate — the 8s floor would skew the split it exists to protect. Refs #2841 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CxokkpfssUCxvBwRZMtRCB * refactor(analyze): make the FTS gate's fast path cheap, its claims provable, and its remedies classified Cleanup review of the #2841 work (four parallel angles: reuse, simplification, efficiency, altitude). Behaviour-preserving except where the previous behaviour was wrong. Correctness the review caught: - The fast-path probe keyed on `capabilities.fts.status === 'unavailable'`, which collapses "extension unavailable" and "index build failed". A deterministic build failure (an un-tokenizable row, #2544) therefore bypassed `alreadyUpToDate` on EVERY subsequent run, re-analyzed the whole repo, failed the same way, and restamped — a permanent loop where the run used to be one `stat`. Phase 3 already computes the discriminator; it is now persisted as `fts.skipReason` and the probe only runs for `extension-unavailable`. Metas written before this carry no field and keep today's behaviour. - `dropSearchFTSIndexes` skipped its sweep when no row read `index_type === 'FTS'`, while `ensureFtsRowDmlSafe` treats an unreadable type as "might be FTS". Opposite polarity, under a comment claiming they matched: a row-shape change would let the gate wave the surgical plan through while the sweep dropped nothing, putting DELETEs back on tables carrying live FTS indexes — #2589 again. The sweep now decides per configured index on identity, which is also strictly more precise. Its old justification (leftover indexes under other names) was unreachable — the loop only ever drops configured entries. - `dropFTSIndex` threw "FTS index X on table Y exists" on the one path where the catalog could not be read — a fabricated claim, on a DB the same run had just shown carries no FTS index. Presence is now `present | absent | unverifiable` and the message says which. - The remedy was hand-written for three of the four load-failure classes, discarding `missingFileRemedy`/`corruptFileRemedy`, so a corrupt extension file was told to retry an install — the misdirection #2383 fixed. Both the drop error and the escalation log now use the classified remedy. Cost, measured on a 391 MB index (cold open ~1 s, SHOW_INDEXES ~4 ms): - The probe opened the live index WRITABLE on the millisecond fast path, dragging in schema DDL, the cross-process write lock, sidecar reclaim and a CHECKPOINT on close. It is read-only now. That also closes an install trap: `doInitLbug`'s pre-load resolves the env policy on the writable branch, so an operator following our own `GITNEXUS_LBUG_EXTENSION_INSTALL=auto` advice paid a forked 15 s installer on every up-to-date run (memoized per process; the CLI is a fresh process each time). The read-only branch pins `load-only`. - A failed staged rebuild orphaned a full index-sized copy until the next lock sweep; the failure path now reclaims it. - The sweep re-read a catalog the run already held, defeating the invariant the snapshot type exists to enforce. Structure: row-shape accessors have one home, so the LADYBUGDB-CONTRACT grep claim is true by construction; staging now applies to both escalation causes, since recoverability is a property of the wipe-then-COPY plan, not of the trigger; `getExtensionCapability`/`getFtsCapability` replace hand-spelled lookups where the seam allows. Two lookups in run-analyze.ts deliberately keep the exported `getExtensionCapabilities()` form: the #2383 tests stub that export, and an ESM module mock does not intercept a helper's internal call — routing through it silently degraded the classified remedy to generic text. Recorded in-comment. Not taken, deliberately: extracting the escalation message and replacing the snapshot protocol with a connection-scoped catalog memo (both sound, both restructure code this PR just stabilised — they belong in their own change); an extension registry (premature at two instances, and the FTS/VECTOR polarity difference is exactly what it would have to parameterize back out). Refs #2841 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CxokkpfssUCxvBwRZMtRCB * test(analyze): pin both sides of the degraded-FTS fast-path bypass `healDegradedFts` (§5.C) had zero coverage — three separate review angles flagged it, and the cleanup pass then found it sat one conjunct away from a permanent full-re-analyze loop. Both sides are pinned now: - it re-analyzes past `alreadyUpToDate` when the stored meta says FTS is degraded and the extension loads again: run 1 analyzes with loads blocked (asserting the precondition — `status: 'unavailable'`, `skipReason: 'extension-unavailable'` — rather than assuming it), then a same-commit clean-tree rerun rebuilds every FTS index without a file changing; - it stands down when the degradation was a BUILD failure: the stored `skipReason` is rewritten to 'build-failed' and the rerun must take the fast path, because that rebuild would fail identically on every run forever. The build-failed state is reached by rewriting the stamped discriminator, not by provoking a real tokenizer failure: a genuine one needs a stored row the native tokenizer rejects (#2544/#2546), which is neither portable across the CI matrix nor deterministic, and §5.C reads only that field. Also folds the first escalation case into the one-shot case. The claim that it was fully subsumed did not hold on audit: `logs` containing 'FTS' was unique as expected, but so was the duplicate-File-node row count — every other reader goes through a Map keyed by path, which collapses a stale twin an appending rebuild would leave. Both assertions moved rather than one being dropped. Net suite runtime goes UP (two cycles removed, four added), against the cross-platform-matrix argument that motivated the dedup — recorded here because the shard weight is an estimate pending a real Windows measurement. Refs #2841 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CxokkpfssUCxvBwRZMtRCB * test(search): keep the whole-module adapter mock in step with the row accessors The cleanup pass moved the LadybugDB row-shape reads behind named accessors so the column contract has one home. `fts-indexes.test.ts` mocks the entire adapter module with a hand-written factory, which still exposed only the three exports the file imported before — so `verifySearchFTSIndexes` failed with "No `indexRowName` export is defined on the mock" while production was fine. The added accessors mirror the real implementations rather than returning stubs. A stub would have read `undefined` out of every catalog row and let the suite pass for the wrong reason — the failure mode a whole-module mock invites whenever the module under test grows an import. Refs #2841 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CxokkpfssUCxvBwRZMtRCB * revert(analyze): drop the degraded-FTS auto-heal, fix the advice it existed to justify §5.C's complaint was that the CLI tells users to "install the extension … then rerun" when a rerun lands on the up-to-date fast path and rebuilds nothing. The answer shipped for it was a probe that bypasses that fast path. Four independent problems later, the sentence is cheaper to fix than to make true: - it could not tell "extension was missing" from "index build failed" without a stamped discriminator, so a deterministic build failure (#2544/#2546) re-analyzed the entire repo on every invocation, forever, where the run used to be one `stat`; - it opened the live index on the millisecond fast path — writable at first, dragging in DDL, the cross-process lock and a CHECKPOINT (~1 s on a 391 MB index), and even read-only it is a full open; - `doInitLbug`'s pre-load resolves the env policy, so an operator following our own `GITNEXUS_LBUG_EXTENSION_INSTALL=auto` advice paid a forked 15 s installer per up-to-date run; - and it turns the fast path into a full re-analysis whenever an index authored where FTS was unavailable is later read where it loads — a legitimate, common state, and the invariant `analyzer-identity-cli.test.ts` pins. So: no probe. The degraded-search warning now points at `gitnexus analyze --repair-fts`, which rebuilds the search indexes without re-parsing the repo, instead of "then rerun". One line, no new failure modes, and it is what the issue actually asked for. `capabilities.fts.skipReason` stays in the meta stamp: it costs three lines, makes the two degradation causes distinguishable for support, and is what any future correct answer here would key on. Also gates the H2 staging assertion on the production predicate. It asserted staging unconditionally while the upgrade requires `posixSwap || windowsSwapOk`, and `windowsSwapOk` is opt-in (#2614) — so it failed on the Windows matrix for a reason unrelated to #2841. Registering this suite cross-platform is what exposed it; the assertion now mirrors the condition it is testing. Refs #2841 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CxokkpfssUCxvBwRZMtRCB * fix(analyze): never stage around a damaged index — escalate in place when the catalog is unreadable CI caught this on ubuntu and macOS: `analyze-wal-checkpoint-failure` stopped failing, which is worse than it sounds. That test plants a directory at `.gitnexus/lbug.wal.checkpoint` so the auto-checkpoint's rename target is blocked, and asserts analyze exits non-zero with the `--wal-checkpoint-threshold` hint. But LadybugDB cannot open that path at all, so `CALL SHOW_INDEXES()` now fails with `IO exception: … Is a directory`. The catalog read returns UNREADABLE, both DML gates correctly fail closed, both extension loads fail with the same IO error, and the run escalates — and since the escalation stages, it built a fresh index at `lbug.staging.<uuid>`, swapped it in, and exited 0. The blocked path was never touched. The run "succeeded" while the damage sat untouched on disk, waiting to break the next in-place writeback. So the staging upgrade is now conditional on the catalog having been READ. Staging exists to protect a healthy live index from a machine-level cause (an extension that will not load); it must not be used to route around a damaged one. When we are escalating out of ignorance, build in place so the underlying IO fault lands on the failure path where the operator gets a diagnosis. Verified against the real CLI, not just the suite: with a directory planted at the checkpoint path, analyze now exits 1 and prints `gitnexus analyze --wal-checkpoint-threshold 67108864`. The healthy extension-forced case still stages (gate suite 6/6). Refs #2841 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CxokkpfssUCxvBwRZMtRCB --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Gergő Magyar
|
9eaf2e6c4e
|
perf(mcp): cut the analyze-only language-provider closure out of MCP server startup (#2802) (#2806)
Some checks failed
CodeQL / Analyze (javascript-typescript) (push) Has been cancelled
CodeQL / Analyze (python) (push) Has been cancelled
Gitleaks / gitleaks (push) Has been cancelled
Publish / Classify release event (push) Has been cancelled
Scorecard / Scorecard analysis (push) Has been cancelled
Skill copy sync / shipped skills drift guard (push) Has been cancelled
Trivy Image Scan / Trivy (gitnexus-cli) (push) Has been cancelled
Trivy Image Scan / Trivy (gitnexus-web) (push) Has been cancelled
Publish / RC guard (marker + release-PR skip) (push) Has been cancelled
Publish / ci (push) Has been cancelled
Publish / Publish to npm (push) Has been cancelled
Publish / Build & Push RC Docker images (push) Has been cancelled
* fix(mcp): key the empty-ascent note on CALL_SUMMARY data, not language (#2802) `pdg-impact.ts` decided whether to append a "return-value ascent is TypeScript/JavaScript-only" caveat to the `impact(mode:'pdg')` note by looking up the criterion file's language. That put language-specific logic in a layer that must be language-agnostic, and it was a lossy proxy for a fact the graph already holds. Whether the ascent can fire is a property of the persisted CALL_SUMMARY edges. The descent already computes it, so thread the resolved-callee and return-flowing counts out of `interproceduralDescent` and key the note on those instead. Three defects the language proxy carried, all gone: - Wrong for `.mjs`/`.cjs`/`.mts`/`.cts`: the provider registry's extension arrays omit them while the ingestion pipeline parses them as TS/JS, so those files were harvested but the note claimed their ascent was empty. - Silently stale: any language whose harvester started recording formal indices would keep getting the caveat until someone edited the list. - Wrong in reverse: a TS/JS callee with no return-flow got no caveat, so an ascent that found nothing read like one that covered the slice. `pdg-impact.ts` now names no language and imports nothing from the language layer, which also drops the analyze-only provider closure from MCP server startup. Measured on overlayfs against a full build: import mcp/local/local-backend.js before 565-648 ms / 548 modules import mcp/local/local-backend.js after 458-463 ms / 170 modules Tests hold CALL_SUMMARY content fixed while varying the file extension across nine languages and assert the note text is identical, then hold the extension fixed and vary the summary to show the note tracks the data. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(mcp): guard MCP startup against the language-provider closure returning The eager `pdg-impact.ts -> core/ingestion/languages` edge was found and lost once already during #2793 before #2802 re-derived it, so it gets a test rather than a comment. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs(lbug): record why csv-generator is not lazy-imported #2802 proposed cutting `csv-generator.js` out of the adapter chain to shorten MCP server startup. Measured on a native filesystem, the marginal cost is small relative to the siblings this module already imports, and `core/search/bm25-index.ts` statically imports `normalizeFtsText` from the same module on a path `local-backend.ts` reaches dynamically for FTS — so deferring would relocate the cost to first query, not remove it. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(pdg): pin chained receiver calls reaching BasicBlock.calleeIds The PDG inter-procedural descent hops through `BasicBlock.calleeIds`, so it can only cross a call boundary the resolver resolved. Chained receiver calls reach `calleeIds` through the receiver-typing pass's own `calleeIdSink` — a separate path from plain calls. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs(analyze): drop the stale per-language cross-reference (#2802 review P3-4) `pdgModeMismatch`'s comment told readers to keep "the diagnostic per-language refinement in the impact CONSUMER (see pdg-impact.ts assemblePdgImpactResult)". That refinement is no longer per-language — removing it is the point of #2802, which now keys the empty-ascent note on the persisted CALL_SUMMARY data instead. The comment's real invariant is untouched and still correct: the values in `resolvePdgConfig` must stay scalar, because the comparison below is a shallow `!==` and an object would compare by reference. Only the cross-reference was stale. Comment-only; no executable line changes. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(mcp): probe the real module loader for the startup language closure (#2802 review P1-2) The previous guard hand-rolled a regex walk over TypeScript source to assert `core/ingestion/languages` was not statically reachable from MCP startup. Four bypasses were reproduced against it, any one of which let the exact 226-module regression return while the test stayed green: a. Wrong entry root. It walked from `mcp/local/local-backend.ts`, but the server module is `mcp/server.ts` — which imports LocalBackend as `import type`, so the guard's anchor was not even on server.ts's runtime closure. Ten real startup modules sat outside it. b. A top-level `await import(...)` executes during module evaluation, so it is eager at startup — but the walker skipped every `import(...)` by construction. c. The `import type` strip deleted a 16,445-character window of `pdg-impact.ts`: an `export type X =` matched lazily to the next `from "…"`, which lives inside a string literal. Any import in that window was invisible. d. The comment strip treated a `/*` inside a string literal as a comment opener. Replace the approximation with a real module-load probe: spawn a child node process per entry, import the built `dist/` entry, and report what the loader actually pulled in. Rooted at `dist/mcp/server.js` and `dist/cli/mcp.js` (the real startup entries) plus `dist/mcp/local/local-backend.js`. Syntax cannot fool it. One deviation from the two existing sibling probes is load-bearing: `dist/` is ESM, so a `require.cache` diff alone cannot see the first-party `dist/**` graph — it only catches CJS and native modules, which is why `import-closure.test.ts` gets away with it (it asserts on `@ladybugdb/core`). A pure cache diff here would have reported zero language modules unconditionally, i.e. a new vacuous guard. This probe unions `module.registerHooks({ load })` with the cache diff, and each entry carries a non-vacuity anchor and a module floor so an empty result fails loudly. Verified load-bearing: adding a top-level `await import('../core/ingestion/languages/index.js')` to `src/mcp/resources.ts` and rebuilding turns `dist/mcp/server.js` red with 70+ named offenders, while the `local-backend` and `cli/mcp` cases stay green — which is bypass (a) demonstrated directly. The old guard passed that poisoned tree entirely. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs(lbug): drop the unreproducible 9p multiplier from the csv-generator note (#2802 review P3-2) The comment justifying why `csv-generator.js` is NOT lazy-imported carried a hard "~40x" figure for how much a 9p mount inflates per-file ESM resolve. Three independent measurements during review produced ~40x, ~7.3x and ~30x, so the multiplier is not a reproducible quantity and had no business being stated as one in a durable comment. Reworked so the STRUCTURAL argument leads and the numbers only support it. That argument is what actually settles the question and it does not rot: `core/search/bm25-index.ts` statically imports `normalizeFtsText` from `csv-generator.js`, and `local-backend.ts` reaches bm25-index through a dynamic import on the FTS query path — so deferring here relocates the cost to first query rather than removing it. Both verified again at `bm25-index.ts:15` and `local-backend.ts:2756`. Remaining figures are re-measured, attributed to a date and issue, and labelled by filesystem: ~1.6 ms marginal (median of 45 cold imports on local disk) versus ~50 ms for the same import on a network mount, stated as environment-bound rather than as a property of the module. The provider-registry cost is given as "several hundred modules" — the static walk, the runtime hook, and the reviewer's probe each counted it differently (375 / 439 / 407), so no single number was picked to go stale. The old "226 modules" was real but counted only the `languages/` subtree and undercounted the win. Also repoints the trailing reference to the guard's new home at `test/integration/mcp/startup-language-closure.test.ts` (same comment block, inseparable from this rewrite). Comment-only; no executable line changes. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(mcp): stop the empty-ascent note asserting a fact an undecodable summary contradicts (#2802 review P2-2) The note claimed "this is a property of the persisted summaries" whenever the descent resolved callees and none carried a return-flow. But `decodeCallSummary` never throws by design: a version-skewed (`2|r:1`), corrupt (`1|r:zz`), or NULL `reason` yields no entry, which was indistinguishable from a cleanly-decoded empty summary. So the note could assert "no formal parameter is recorded as flowing to its return value" about a callee whose CALL_SUMMARY actually records `p0 -> return`. `meta.pdg.hasCallSummary` is a plain boolean and stores no codec version, so nothing else caught it. `calleesWithReturnFlow` now reports three outcomes instead of two — flowing, decoded-empty, and undecodable — and the undecodable count is threaded through the descent to the note. When it is non-zero the note says so and points at a re-index; when every summary decoded, the persisted-summaries claim is kept and now explicitly conditioned on that. Soundness is unchanged: an undecodable summary still licenses no ascent and never enters the return-flowing set, so the ascent path is byte-identical. Only the note's wording moves. Tests drive all three undecodable forms through the mock and assert the false claim is gone, the remedy is reported, and the ascent is still withheld. A companion assertion pins that the all-decoded case KEEPS the persisted-summaries claim, so the fix cannot degenerate into deleting the sentence. Verified load-bearing: reverting the source alone fails 6 of 34. Impact analysis: `calleesWithReturnFlow` upstream LOW (2 callers, both in this file); `assemblePdgImpactResult` upstream LOW (1 caller). Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(pdg): cover every chained-receiver shape and pin the inference gap (#2802 review P2-1, P3-1) The fixture proved chained receiver calls reach `BasicBlock.calleeIds` using exactly one receiver form — a local `const`. That is the shape that works, so a single-shape fixture implied general support the resolver does not have. This repo has been burned by that before: a drop-count gate blind to fixed shapes. Measuring nine forms against the real pipeline also corrects how the gap was originally characterised. It is NOT local-versus-field. An annotated field resolves fine, including the constructor-assigned variant: private p: Outer = new Outer(); -> both links private p: Outer; this.p = new Outer(); -> both links private p = new Outer(); -> EMPTY CELL private p; this.p = new Outer(); -> EMPTY CELL The discriminator is the type ANNOTATION. When a field's type must be inferred from its initializer the whole `calleeIds` cell empties — so even `Outer.inner`, an ordinary named-receiver call, is lost, and the inter-procedural descent cannot cross the boundary at all. Pre-existing; independent of #2802, which does not touch receiver resolution. The fixture is now table-driven over seven working forms (local const, local in a method, annotated field, ctor-assigned annotated, ctor-param assigned, call-result receiver, three-link chain) plus the two inference-typed forms, each row carrying its expected chain-link ids. Assertions moved from substring to exact id membership, split with the production `splitCalleeIds` reader — so `Inner.compute` can no longer be satisfied by `Inner.computeExtra` or `OtherInner.compute`, which matters because the descent keys on exact ids for span and CALL_SUMMARY lookup. The two known-gap rows are pinned with `it.fails` plus a hard assertion on the exact gap-row set, so a resolver fix turns them red instead of passing silently, and an anti-vacuity guard requires every shape to match exactly one block — without it a drifted fixture matching zero blocks would let `it.fails` pass for the wrong reason. Proven by mutation: relabelling a working row as a known gap fails both pins. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(mcp): qualify the empty-ascent note when the examined callee set is incomplete (#2802 review P2-4) The note asserted "none of the N resolved callees carry a CALL_SUMMARY return-flow", and on the all-decoded path that this is "a property of the persisted summaries". Both are universal claims over the callees the descent actually examined, and two mechanisms can leave that set incomplete without the note saying so: 1. Budget truncation. The descent stops on depth/limit/node-cap, so a callee that DOES carry a return-flow can sit in a hop never reached. A 4-deep chain reported "none of the 3 resolved callees" while link 4 held the only summary. 2. Emit-time capping. When a block's `calleeIds` cell was capped, `splitCalleeIds` strips CALLEES_TRUNCATED_SENTINEL, so the dropped callees are invisible to both the scan and the counters — even though the callgraph bridge in this same file already treats such a block as callee-incomplete. Add `calleeIdsWereTruncated`, the counterpart to the sentinel strip, read from the raw cell before splitting so a block whose entire list was capped away still raises the flag. Thread it through the descent to the note. Case 1 needs no new plumbing — the aggregate `truncated` is already on the input object. Using the aggregate rather than a descent-only flag is deliberate: seed truncation and intra-BFS depth truncation also shrink the initial slice, so their callees are never gathered either. It is a sound superset that never under-hedges. When either mechanism fired, one clause naming the reasons is appended and the whole-slice assertion softens to "every summary examined decoded … a property of those summaries". When the set is complete both branches stay byte-identical to before, so this does not become a blanket hedge. Tests pin truncated, untruncated, emit-capped-alone, both-mechanisms, and undecodable+truncated, asserting the truncation premise rather than assuming it. Verified load-bearing: reverting the source alone fails 6 of 42, and the HEAD note printed in those failures is the bug verbatim. Impact analysis: `assemblePdgImpactResult`, `calleeIdsByBlock`, `interproceduralDescent` all upstream LOW; every caller is in this file and `runImpactPDG`'s exported signature is unchanged. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(mcp): stop the empty-ascent note calling call-site references "resolved callees" (#2802 review P3-7) The note printed "none of the N resolved callees carry a CALL_SUMMARY return-flow (no formal parameter is recorded as flowing to its return value)". N counted the raw `BasicBlock.calleeIds` cell, which carries ids `resolveCalleeSpans` never enters — out-of-repo targets, interface methods, and the `Class:` id a `new X()` emits. On the chained-receiver fixture that inflated N from 1 to 3. Two defects, both in the wording rather than the arithmetic: "resolved" implies a symbol-table lookup that did not happen for those ids, and the parenthetical asserted a FORMALS-level property about symbols never resolved to a body. Reworded rather than re-seeded, deliberately. `calleesWithReturnFlow` scans the RAW id set, so the claim "none of these carries a return-flow" is exactly established for all N — the scan really did check the `Class:` id. Re-seeding N from the resolved spans would make the sentence quantify over a strict SUBSET of what was checked, silently dropping the un-enterable references from a claim that genuinely covers them, and would desync N from `calleesUndecodable`, which is derived from the same scan population. none of the N resolved callees carry ... none of the N call-site callee references carry ... and the formals parenthetical is dropped. The note gets shorter, not longer. `calleesResolved` is renamed `calleeReferences` end-to-end (file-local; nothing outside referenced it), and the descent's return-type doc — which called them "callee symbols the descent resolved" and reinforced the wrong reading — now states that un-enterable ids ride the same cell, are scanned, and are never entered. The `> 0` gate is unchanged, so no slice that previously produced the note stops producing one. A test pins that explicitly: an all-un-enterable cell resolves no span, takes no hop, and emits no ascent sentence despite a non-zero count — so a future re-seeding cannot silently move when the note fires. Tests also pin the quoted number and singular/plural against a mixed cell, with a discriminator asserting `reachableBlocks` is byte-identical while the count moves 1 -> 3. Verified load-bearing: reverting the source alone fails 6 of 7 new tests, printing the finding verbatim. Impact analysis: `assemblePdgImpactResult` and `interproceduralDescent` upstream LOW, sole caller `runImpactPDG` in the same file; exported signature unchanged. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(mcp): pin cross-hop callee accumulation and the mixed return-flow contract (#2802 review P2-5) Every case in this file drove a single hop, so the Set union the descent performs across hops (`calleeReferencesSeen` / `calleesReturnFlowingSeen`) was never proven to accumulate rather than overwrite — a one-hop descent cannot tell the two apart. And although a sibling commit added a three-id cell, none of those ids return-flowed, so the "some callees flow, some do not" boundary was entirely unpinned. Extends the mock with a `secondSummary` knob that drives a genuine second hop: `helper2` is named only in `helper`'s own body block, so the descent must cross a second boundary to reach it. Three mock handlers are made faithful to the parameters they already bind — `calleeIdsByBlock` now routes on the asked `$ids`, and the CALL_SUMMARY scan and span resolve answer per asked id — which is what makes a second callee answerable at all. Existing cases are behavior-identical. Five tests: the union count across two hops; a return-flow on hop 0 surviving a later empty hop; a return-flow found only on hop 1; mixed callees in one examined set going silent rather than partial; and a flowing callee alongside an undecodable sibling staying silent including the decode remedy. The mixed case pins a deliberate contract rather than proposing one. The production condition is `calleesReturnFlowing === 0`, so partial coverage is reported as silence. A reviewer considered and dropped "report partial coverage" as a product change; this makes flipping it a conscious edit instead of an accident. Verified load-bearing against three separate source mutations: accumulating only on hop 0 (2 fail), each hop overwriting instead of unioning (3 fail), and flipping the gate to partial-coverage reporting (4 fail). In all three every PRE-EXISTING test still passed — which is the finding restated as evidence. Test-only; `pdg-impact.ts` is byte-identical to HEAD. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs(mcp): consolidate the empty-ascent rationale to one canonical site (#2802 review P3-6) The "keyed on observed CALL_SUMMARY data, never on the criterion's language" rationale was restated in full at four comment sites. It exists because a reviewer asked "why not just look up the language?", so it has to stay findable — but not four times. The canonical explanation now lives in `interproceduralDescent`'s return-type doc, where the counters are actually computed, organised as POPULATION (why the raw `calleeIds` tally is the right set to quantify over) and OBSERVED DATA, NEVER THE CRITERION'S LANGUAGE (the full answer, including the producer-change argument and the no-language-naming rule). The other three sites keep only what is locally load-bearing and point here. Deliberately preserved, because each carries a non-obvious fact: why an undecodable summary licenses no ascent, why the aggregate `truncated` is used rather than a descent-only flag, and the raw-id-tally population argument. Net comment delta -11 lines. The reviewer also flagged the local/field naming asymmetry (`calleeReferencesSeen` vs `calleeReferences`). Keeping the suffix, with a comment recording why so it is not re-raised: the premise that every other local matches its field is true, but those locals are identity-returned, whereas these are `Set<string>` accumulators returned as `.size`. Dropping the suffix would give one identifier two types in one file — a `Set` at the accumulation site and a `number` where the note does arithmetic and pluralisation on it ~900 lines away. The Set-ness is also load-bearing: the dedup is why a callee invoked from two hops is not double-counted, which is what makes the note's count correct. Comment-only. Verified mechanically: every added and removed line in `git diff -U0` matches a comment pattern, so the note's template literals are untouched and its rendered text is byte-identical. 89 tests unchanged. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor(mcp): collapse the ascent plumbing accreted across 13 fix commits Quality cleanup, no behavior change. Four independent review passes converged on the same root cause: thirteen commits each fixed one review finding in isolation, and the ascent facts grew one loose field at a time until 62% of the changed region was comments explaining plumbing. Five changes: - `calleeIdsFromBlocks` deleted. Zero call sites anywhere in src/ or test/ — already dead on main, and this branch had edited it to keep it compiling. Its only reference was a stale `{@link}` in a neighbour's doc, now rewritten to stand alone. - `parseCalleeIdsCell` replaces the two-pass read. `calleeIdsWereTruncated` and `splitCalleeIds` were splitting the same cell on adjacent lines, which measured ~2x the parse cost (0.82 -> 1.59 ms at a realistic hop, 57.7 -> 92.7 ms at the per-statement site cap) and was a second independent encoding of the sentinel format — exactly what `splitCalleeIds` was extracted to prevent. One pass classifies as it walks; `splitCalleeIds` stays as a wrapper so its two external callers are untouched. The single-use `export` is gone. - `AscentCoverage` replaces four fields threaded through three signatures. ~12 declaration sites become 3, and the canonical rationale now lives on the type by construction — which is why the earlier doc-consolidation commit was needed at all. - `calleesReturnFlowing` becomes a boolean. Its only reads were `=== 0`, twice; it cost a Set sized to every callee in the slice plus a per-hop union loop. The flag is set inside the existing `returnFlowing.size > 0` branch — equivalent, since the cross-hop union is non-empty iff some hop's was. - The duplicated empty-ascent note head is collapsed to one gate and one head with per-arm tails. Both arms had been edited in lockstep twice in this branch's own history. The rendered note text is byte-identical. Verified structurally and then empirically: both expressions reconstructed standalone and diffed across the full cross product of references x returnFlowing x undecodable x truncated x listTruncated — 288 combinations, 0 mismatches. Net -53 lines. 102 tests pass unedited; the unused-symbol lint warning is gone. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(mcp): parallelise the startup probes, drop a redundant pin, name the mock knobs Quality cleanup from the same review passes. The set of verified behaviors is unchanged except where noted. **Startup probes run concurrently.** `spawnSync` blocks the event loop and vitest runs a file's tests in order, so the three probes strictly serialised. Launching all three with async `spawn` in `beforeAll` and asserting over the collected outcomes cuts the file from ~12.7 s to ~3.9 s wall (-69%). Every promise is caught before `Promise.all`, so all three children are reaped and failures report per entry rather than surfacing only the first rejection. Preserved and each proven by mutation: the missing-dist error names its entry, a raised module floor fails only its own row, and a bogus anchor still reports the loaded-module count. **The two `it.fails` rows are removed.** They pinned the inference-typed receiver gap that the strict `toEqual` pin beside them already covers — and they were the weaker of the two, because `it.fails` passes when the body throws for ANY reason, including `idsFor`'s own non-vacuity guard. A renamed fixture marker would have kept them green on a rotted premise. The strict pin is self-diffing and was verified load-bearing on its own: pointing a known-gap marker at a resolving shape fails it with the two newly-present ids listed. The file header now carries the gap's durable description. **The ascent-note mock takes options objects.** `descentExec` and `run` had grown to five and seven positional parameters in the order five agents added them, so call sites read `run(FILE, true, null, 3, false, undefined, null)` — several carrying `undefined` purely to reach a later argument. All 34 call sites are converted; nine that used only defaults are now bare `run(file)`. No knob renamed — they are orthogonal and correctly named. Code lines are exactly neutral (353 -> 353); the win is at the call sites. Also refreshes five comments that still described `calleesReturnFlowingSeen` and the two-branch note, both of which the preceding commit replaced. 102 unit and 10 integration tests pass; test count moves 9 -> 7 in the chained-receiver file, exactly the two redundant rows. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(mcp): publish return-value-ascent coverage on the PDG impact result `impact(mode:'pdg')` computed four facts about ascent coverage and used them exactly once — to interpolate an English sentence. They never reached the result object, so an agent consuming this MCP output could only ask "was the ascent complete, and if not why" by regexing prose. The cost was already demonstrated: a pure rewording commit earlier in this branch broke ~30 assertions and would have silently broken any consumer keying on the old phrase. Adds `pdgEvidence.ascent`: referencesScanned how many call-site callee references were scanned returnFlowFound did the ascent fire anywhere in this slice undecodableSummaryCount summaries the codec could not decode examinedComplete was the examined set the whole callee list incompleteReasons 'traversal-truncated' | 'callee-list-capped' callSummaryLayerPresent false => pre-FU-C (v3) index Nested under `pdgEvidence` because that is the established counts-and- classification namespace, and `composeUnifiedPdgImpactResult` already spreads it, so the member survives the unified compose untouched. `incompleteReasons` carries CODES, following the existing `truncatedByReasons: ('depth'|'limit')[]` precedent. The prose clause and the structured field now render from one array computed once, so an agent branching on codes and a human reading the note cannot disagree, and a third reason becomes a rendering decision rather than a contract change. Two shape decisions worth recording. `callSummaryLayerPresent` exists because without it a v3 index publishes `referencesScanned: N, returnFlowFound: false`, which reads as "these callees record no return-flow" when the truth is "the layer that records it is absent" — the note already distinguishes those, and the structured surface must not be less honest than the prose. And the field is ABSENT rather than zeroed when the descent never ran (upstream slices): "nothing was scanned" is a different fact from "we scanned and found nothing". `pdgResultVersion` stays 2. The documented trigger is a BREAKING change to the result shape; this removes nothing, renames nothing, and changes no existing field's meaning. Confirmed mechanically: zero top-level key drift across 2304 cases. The historical v2 bump was for changing an existing field's semantics (startLine 0- to 1-based). The note prose is byte-identical, proven across the same 2304 cases with a negative control — perturbing one character of the phrase table produces 60 drifts, so the harness demonstrably detects what it asserts. 14 new tests cover the structured surface and all 14 fail when the source is reverted, while the 54 prose tests pass unchanged. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(helpers): share one module-load probe, and fix two guards that passed on broken builds Three tests independently spawned a child node process to inspect what a built `dist/` entry loads, duplicating the REPO_ROOT derivation, the probe source, the missing-dist guard, the spawn with NODE_OPTIONS cleared, the status-vs-signal rendering, and the payload parse. The newest copy was also the only correct one, so the next author had 2-in-3 odds of copying a weaker probe. The two older probes diff `require.cache` only, which is structurally blind to the first-party ESM `dist/**` graph. That is not theoretical — both were demonstrated passing on genuinely broken builds: - Severing `dist/cli/mcp.js -> stdio-context.js` (a pure ESM change) leaves the require.cache diff EMPTY, so `import-closure.test.ts`'s two assertions reduce to `[].filter(...) === []`. It reported 2 passed on a severed graph. - Severing `registry -> swift/query.js` leaves 76 unrelated CJS entries, which satisfied `registry-import-closure.test.ts`'s indirect guard. The Swift half of its headline had gone vacuous and it reported 1 passed. Both now fail on those same builds, naming the missing anchor. `test/helpers/module-load-probe.ts` unions the ESM `registerHooks({ load })` channel with the cache diff, probes entries concurrently, and makes non-vacuity STRUCTURAL: `anchor` and `minModules` are required fields and the helper throws when either fails. A vacuous probe is a harness failure, not a silently green test, so it cannot be forgotten. Forbidden patterns and remedy text stay per-test — the harness is the shared part, the policy is not. Also fixes `toRepoRelativePosix` resolving non-absolute specifiers against `process.cwd()`, and dedupes modules a CJS-from-ESM import reported once per channel. Faster despite doing more: the registry file goes 12.4s -> 6.75s, because `spawnSync` burned the parent thread polling while the child loaded native grammars. `import-closure` drops to one spawn from two. The `local-backend.js` entry is kept although its closure is currently a strict subset of `server.js`'s: that is an observation, not an invariant. If `server.js` ever stops eagerly reaching the local backend, the server probe stays green while the module #2802 actually changed goes unobserved — and now that anchors are mandatory, that entry is what pins `pdg-impact.js`. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs(lbug): trim the csv-generator note and fix the claim it got wrong Two reviewers split on this comment: one wanted it cut to the structural argument, the other said a comment is the right depth for documenting a rejected change since there is no invariant to guard. Both are right, so it stays a comment and gets shorter — 13 lines to 6. Trimmed because it had already taken two corrections (an unreproducible "~40x" figure, and a pointer to a test file that no longer exists), and its tail had drifted from its own guard: the comment said "several hundred modules, ~150 ms" where `startup-language-closure.test.ts` says "~226 extra modules and ~130 ms". Two numbers for one fact. That tail is documented better in the guard's own header, so deleting it loses nothing. It also stated the load-bearing claim inaccurately. The old text said bm25-index imports `normalizeFtsText` "from here" — but `lbug-adapter.ts` neither exports nor re-exports it; the only occurrence of the identifier in this file WAS the comment. Anyone verifying would have grepped, found nothing, and concluded the note was stale. Now names `csv-generator.js` explicitly, re-verified at `bm25-index.ts:15` (static) and `local-backend.ts:2756` (dynamic, on the FTS query path). Comment-only, proven two ways: every changed line matches a comment pattern, and stripping all `//` lines from HEAD and from the working tree yields byte-identical text. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(helpers): extract the temp-repo lifecycle, collapsing five hand-rolled cleanups into one Four cfg integration tests each hand-rolled a `tmpDirs` array, a mkdtemp-and-register step, and an `afterAll` rmSync. It is actually five registrations across six creation sites — `pipeline-pdg.test.ts` keeps a second pool for its C-family fixtures. Seeding genuinely varies four ways (recursive cpSync, single copyFileSync, inline mkdir+writeFile, and nothing at all), so a fixture-copier helper would have fitted about half the sites and made things worse. Extracted the LIFECYCLE instead — mkdtemp, register, afterAll cleanup — which is byte-identical at all five registrations and is the correctness-critical part. `dir()` returns an empty registered directory for callers that seed themselves; `fromFixture()` covers the common case. That fits 6/6. The duplication had already produced a latent defect: `cFamilyTmpDirs` was cleaned by TWO `afterAll` blocks, harmless only because `rmSync` was called with `force: true`. Now one hook. `createTempDirPool` is a function called from each test file's module scope rather than a top-level hook in the helper, because under ESM caching a module-level `afterAll` would register once, against whichever file imported it first. That hazard is documented in the helper. Raw line count is roughly neutral (-44 across the tests, +62 for the helper, 29 of which are the rationale). The win is that a cleanup invariant went from five copies to one. Cleanup verified empirically, including the failure path: a throwaway suite whose `beforeAll` throws still has its directory removed, and every temp directory created by the four migrated files is gone after a run. 46 tests pass across the four files. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(resolvers): pin the inference-typed field receiver gap at the resolver level The gap was pinned only in a PDG test, asserting on `BasicBlock.calleeIds` behind the full `--pdg` pipeline. But it is a resolver fact: when a class field's type must be inferred from its initializer, chained receiver calls resolve to nothing. Whoever closes it will be working in the resolver suite and would have got a red CFG/PDG test with no resolver-side signal. Asserts CALLS edges directly, alongside `python-constructor-field-receiver.test.ts`. Nine receiver shapes run the identical statement; seven resolve, two do not: const o = new Outer() resolves private p: Outer = new Outer() resolves private p: Outer; this.p = new Outer() resolves private p: Outer; this.p = p (ctor arg) resolves constructor(private p: Outer) {} resolves makeOuter().inner().compute() resolves o.inner().mid().compute() (three links) resolves private p = new Outer() NO EDGES private p; this.p = new Outer() NO EDGES Two things the fixture establishes that the PDG-side pin could not. The discriminator is the type ANNOTATION, not local-versus-field — the parameter-property form resolves fine. And the initializer is NOT invisible to the resolver: `new Outer()` still emits its own constructor CALLS edge, byte-identical to the annotated twin. Only the initializer-to-field-type binding is missing, which narrows where a fix belongs. Assertions key on exact node ids rather than names, because `compute` is ambiguous across two classes and keying on the source name collides with `Object.prototype.constructor`. No `describe.skip` and no `it.fails` — the latter passes when the body throws for ANY reason, so it can go green on a rotted premise. The gap is pinned as its explicit current value, which self-diffs: simulating the fix fails one test showing the two newly-resolved ids, and renaming a fixture symbol fails the non-vacuity guard. Runtime is comparable to the PDG-side pin (~9-11s, both dominated by worker startup), so this is an altitude and scope win, not a speed one. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(mcp): replace the extension sweeps with a stronger language-agnosticism pin Two `it.each` sweeps over nine file extensions asserted that the empty-ascent caveat was present (or absent) for each. They looked like the pin for the property the whole change exists for — `pdg-impact.ts` must name no language and its output must not vary by extension — but they were the weakest available form of it. They asserted substring presence/absence, so a language dependence that ADDS text while leaving the caveat intact passes them. Demonstrated, not assumed: injecting a `.py`-only hedge inside the caveat sentence and replaying the two sweeps verbatim against that source gives 18 passed. The byte-identity test beside them caught it. So the sweeps are deleted and the identity test carries the property alone, hardened in two ways: - Two rows instead of one, covering BOTH sides of the caveat gate. The silent (return-flow present) branch previously had no identity counterpart at all — nine runs proving one fact, with nothing checking that its rendering was extension-invariant. - The fingerprint spans the note AND the reachable blocks, not just the note. Strictly more than the sweeps verified. Entailment is exact: identity across the extension set, plus the two existing single-extension content assertions, gives "every extension gets the caveat" and "no extension gets it". Reducing a sweep to one extension was rejected because it reproduces an assertion already present verbatim. Also converts the incompleteness block from six near-identical bodies to a 3-row premise table crossed with two assertions. Each row now names the exact phrase set its clause must contain, so presence and absence are asserted together — which adds three checks the longhand version lacked (the budget row now also proves the emit-cap phrase is absent). And three tests that re-rendered one fixture to make one assertion each are hoisted to a single render. 97 tests, down from 116: -18 sweep cases, -2 from the hoist, +1 identity row. No assertion was lost; several were added. Verified by injection: a `.py`-only note change fails the identity pin, and a dependence in the shared hop sentence fails BOTH rows, confirming the second row is load-bearing rather than decorative. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * perf(mcp): lazy-import syncGroup so MCP startup skips the group extractor closure `core/group/service.ts` statically imported `./sync.js`, which pulls all six contract extractors, five of which statically import the native `tree-sitter` binding. That put the whole parser stack on every MCP server start, for a server that never syncs. Only `groupSync` needs it. The other seven group tools — `group_list`, `group_impact`, `group_query`, `group_contracts`, `group_status`, `group_trace`, `group_context` — do not, and now never load it. `syncGroup` has a single call site, already inside an `async` method, so this is a lazy `await import(...)` at that call site and nothing else: no signature change, no async ripple, no change to `local-backend.ts`. The pattern is already established on this exact module — `cli/group.ts`'s sync command lazy-imports `sync.js` the same way. `service.ts` was the outlier. Measured on a native filesystem (overlayfs; /workspace is a 9p mount that inflates ESM resolve, so it is not a valid measurement surface), 5 cold runs, medians: dist/mcp/server.js 521 ms -> 133 ms (-75%) dist/mcp/local/local-backend.js 453 ms -> 66 ms (-85%) tree-sitter modules at both entries: 11 -> 0 Same defect class as #2802, which cut the language-provider registry from the same startup path; this is what remained. The cost is moved rather than deleted: the first `group_sync` call now pays the module load. That is the right trade — `group_sync` is already a long-running operation, and sessions that never sync pay nothing. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(mcp): guard MCP startup against the group extractor closure returning Sibling forbidden-pattern case in the #2802 startup guard, reusing the concurrent probes it already collects — no new spawn, no new harness. Asserts that none of `dist/mcp/server.js`, `dist/cli/mcp.js`, or `dist/mcp/local/local-backend.js` loads a `core/group/extractors/` module or the native `tree-sitter` package. The parser is matched by package prefix rather than a bare substring, so a source file that merely mentions the word can neither satisfy nor trip it. Verified load-bearing rather than assumed: restoring the static `import { syncGroup }` in `core/group/service.ts` and rebuilding turns `dist/mcp/server.js` red and names all seven offenders — http-route, grpc, thrift, topic, include, manifest and workspace extractors. Reverted and re-confirmed green. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * perf(mcp): keep the analyze-only CFG closure off MCP server startup (#2802 review) `mcp/local/pdg-impact.ts` imported `CALLEES_TRUNCATED_SENTINEL` and `CALLEE_ID_SEP` from `core/ingestion/cfg/emit.ts`. ESM evaluates a module to import any binding from it, so those two strings dragged the whole analyze-only CFG closure into every MCP server start. Measured against a clean build, per entry point: 8 modules — `emit`, `reaching-defs`, `reaching-defs-graph`, `control-dependence`, `post-dominators`, `synthetic-escape`, `call-site-harvest`, `reaching-def-reason-codec` — present at `dist/mcp/server.js`, `dist/mcp/local/local-backend.js` and `dist/mcp/http-transport.js`. Same defect class as the language-provider closure this branch already removed, and the guard could not see it: `FORBIDDEN_RE` covers `core/ingestion/languages/` and `FORBIDDEN_GROUP_RE` covers `core/group/extractors/|node_modules/tree-sitter`, neither of which matches `core/ingestion/cfg/`. The format constants move to a new LEAF module `cfg/callee-cell-format.ts` that imports nothing; `emit.ts` re-exports both names so every existing importer is untouched, and producer and consumer still resolve to one definition — the drift the shared constant exists to prevent stays impossible. Deleted, not deferred — the same bar #2802 held its own csv-generator proposal to. After: cfg modules at startup 8 -> 2, and both survivors (`callee-cell-format`, `reaching-def-reason-codec`) are leaves that import nothing. Totals: `server.js` 387 -> 380, `local-backend.js` 163 -> 156, `http-transport.js` 523 -> 516. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(mcp): stop pdgEvidence.ascent claiming a completeness it cannot have (#2802 review) `examinedComplete` is the field a consumer reads to decide whether `returnFlowFound: false` is a whole-slice claim. It could be published `true` over a callee set the descent never finished examining — the exact false all-clear the field was added to prevent. Root cause: `bfsReachableBlocks` sets `truncatedByDepth` when its frontier is still non-empty at the budget, but both call sites inside `interproceduralDescent` folded only the row-limit flag and dropped the depth flag. The top-level intra BFS's copy of that same flag was already propagated, so the asymmetry was unintended — one `if`-pair folding limit-but-not-depth, within a merge that already folds the node cap too. Reproduced at `maxDepth: 3`, the shipped default: a criterion calling a helper whose body is a 5-block dependence chain, with the return-flowing callee on the block past the clamp. Result reported `truncated: undefined`, `examinedComplete: true`, `incompleteReasons: []` and an unqualified universal note sentence. Fixed by propagating the dropped flags rather than inventing a parallel channel: `intraDepthBudget` is documented in-file as the SAME clamp the top-level intra BFS applies, and that one's depth truncation is already result-level. So the result's own `truncated`/`truncatedBy` were under-reporting for the same reason, and both surfaces are corrected together. Four further honesty fixes to the same published record: - Blocks reached only by the U-C4 ascent went into `reachable` but never `hopReached`, so their `calleeIds` cells were never scanned, never counted, and could not raise `callee-list-capped`. They are slice blocks; they now enter the hop set and get the same treatment as every other one. - `pdgEvidence.ascent` was absent on the empty-slice early return even though the descent had already run and scanned, contradicting the "present iff the descent ran" contract this branch itself added to `tools.ts`. Both exits now classify through one shared helper so they cannot disagree. - A block carrying call sites in `callees` but no resolved ids in `calleeIds` (the whole-file case where `emit.ts` has no fileMap) silently shrank the population while `examinedComplete` still reported `true`. That now raises a third reason, `callee-ids-unrecorded`. - `referencesScanned` is a distinct-callee tally and both surfaces described it as a call-site count. Field name kept — a rename is breaking at `pdgResultVersion: 2` — and the prose corrected instead. `PdgAscentIncompleteReason` gains a member, which is additive, so `pdgResultVersion` stays 2. Visible output change worth knowing: slices whose callee chain outruns `maxDepth` now report `truncatedBy: 'depth'` where they previously reported none, and a repo with id-less call sites now reports `examinedComplete: false`. Both are strictly more honest. Every behavioural change carries a mutation proof — revert the source, watch the new test go red, restore. One exception is documented inline rather than faked: the ascent-side fold cannot be observed independently, because the re-seed shares the caller's `visited` set and so can only reach past the budget when the traversal that covered that closure was already cut and had already raised a flag. Suite: 49 -> 59 tests. Refs #2802 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(mcp): anchor each import-closure policy on the edge it polices (#2802 review) `module-load-probe.ts` makes non-vacuity structural via a required `anchor` — but the anchor was one per ENTRY while `startup-language-closure.test.ts` now runs TWO independent policies. The group-extractor policy added in |
||
|
Void Freud
|
7ee0df9e55
|
fix: serialize global registry transactions across processes (#2716)
* fix: serialize global registry mutations * fix: serialize global registry mutations * fix: keep registry reads lock-free * test(registry): document cross-platform lock coverage * fix(registry): isolate the registry lock's namespace, timeout and diagnostics Review follow-ups on the global registry lock (#2716): - The lock took `getGlobalDir()` itself, which is byte-identical to a repo's index slot when that repo is rooted at the user's home directory (a real dotfiles layout). `runFullAnalysis` holds the per-repo lock across its whole pipeline and `acquireIndexLock` is not reentrant, so `registerRepo` / `adoptFlatBranchLabel` self-deadlocked until the wait ceiling and then failed the analyze. The registry now locks a private `<globalDir>/registry-lock` namespace no index slot can ever resolve to. - The lock inherited the index lock's 10-minute default timeout, sized for multi-minute analyze runs. `gitnexus augment` — documented to cold-start in under 500ms and shelled out from editor tool-use hooks — reaches it through `listRegisteredRepos({ validate: true })`. Registry transactions are sub-second, so they now get their own 5s ceiling. - Contention was silent: no `log`/`onWaitStart` was wired, and the primitive's own texts attribute a wait to "another gitnexus analyze", which misnames a registry holder. A registry-specific line is emitted on wait start instead. - On timeout the transaction now proceeds unlocked with a warning rather than throwing. The lost-update race it guards was unguarded before this branch, so degrading to the old best-effort behaviour beats failing `analyze`/`list`/ `index` outright — none of which wrap these calls in a handler — on a wedged lock. - `adoptFlatBranchLabel`'s recursive `fs.rm` no longer runs inside the lock; only the closing re-read/mutate/write does, mirroring `clean.ts`, which deletes the branch directory before calling the locked `removeBranchIndex`. A slow delete no longer blocks every registry operation on the machine. * test(registry): cover the remaining locked mutators and the colliding layout Three of the five functions the registry lock wraps had no overlap coverage, so a future narrowing of the lock would go unnoticed. Adds: - overlapping `removeBranchIndex` calls on two branches of one entry, - an overlapping `unregisterRepo` / `registerRepo` pair on distinct repos, - a registration issued while an index lock is held on the global directory, which reproduces the home-rooted self-contention the lock namespace fix addresses. Each fails without the corresponding fix: the two overlap tests lose an update when `withRegistryLock` is bypassed, and the collision test sees the wait announcement and the degraded-write warning once the lock namespace is reverted to `getGlobalDir()`. The collision test asserts on those log records rather than on elapsed time, so it stays deterministic on a slow runner. * perf(registry): keep the validation walk out of the registry lock `listRegisteredRepos({ validate: true })` held the global lock across its read-only validation walk — an `fs.access` per entry, slow on a network mount or a large registry — even though the common case prunes nothing and writes nothing. That is the same lock `gitnexus augment` takes on every editor tool call, so unrelated registry work serialized behind a walk that never touched the file. The walk now runs unlocked; the lock is taken only when an entry is provably gone, and the prune is applied to a snapshot re-read inside it, so a registration that lands during the walk is no longer clobbered by a stale write. Same shape as the `adoptFlatBranchLabel` split. --------- Co-authored-by: Gergő Magyar <gergomagyar@icloud.com> Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> |
||
|
Gergő Magyar
|
7a064a1f2a
|
fix(storage): stop the Windows \\?\ long-path prefix from breaking repo path matching (#2667) (#2700)
* fix(lib): add stripWindowsLongPathPrefix for path comparisons (#2667) A caller can hand GitNexus a `\\?\`-prefixed path — the usual MAX_PATH workaround on Windows — and `path.resolve` preserves the prefix, so it reaches every string comparison GitNexus keys paths on. It also poisons relativization: `path.win32.relative` cannot express a relative path between a prefixed and an un-prefixed form of the same directory, so it returns the absolute target instead. That absolute string is the shape reported in #2667. The helper is deliberately scoped to the comparison domain. libuv's `fs__capture_path` does not re-add the prefix for over-MAX_PATH paths, so stripping a filesystem-facing path would break long-path access on hosts that have not opted into LongPathsEnabled. `\\?\Volume{GUID}\…` is left alone because the remainder is not a usable path. The test is fixture-free and takes an explicit `platform`, mirroring `normalizeAnalyzerRootPath`, and is registered on the cross-platform matrix since the whole transform is a POSIX no-op. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0119fkrRFdQDQKh58LY9Tqh2 * fix(storage): normalize the `\\?\` prefix in canonicalizePath (#2667) `canonicalizePath` is the single comparison key for the repo registry, MCP repo resolution and the server repo routes, and `registryPathEquals` compares its output as a plain string. A caller-supplied `\\?\` prefix therefore matched nothing: a repo registered as `D:\repo` was invisible to a caller passing `\\?\D:\repo`, which surfaces as "repo not found" or a duplicate registration from `analyze`, `remove`, `clean`, the MCP `repo` parameter and the server routes. Both branches are normalized. The realpath branch was already safe — libuv's `fs__realpath` strips the prefix itself — but the `catch` fallback returns `path.resolve(p)` untouched, and that is exactly the branch a path which is not on disk takes. Safe despite the CRITICAL blast radius (27 impacted, 12 direct dependents) because the result is only ever compared, never opened: all 23 call sites feed `registryPathEquals` or a string comparison. Both operands are canonicalized, so the equality relation is preserved and behaviour is unchanged for every un-prefixed input. The two regression assertions run only on windows-latest, where the file already runs via the cross-platform matrix. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0119fkrRFdQDQKh58LY9Tqh2 * docs(core): correct two false comments about Windows paths (#2667) Both comments assert the opposite of how the platform and the analyzer actually behave, and both would send the next investigator of #2667 the wrong way. `analyzer-identity.ts` claimed the `\\?\` prefix is one that `realpathSync.native` "can emit for paths over MAX_PATH". libuv's `fs__realpath_handle` strips the prefix unconditionally and rewrites `\\?\UNC\` back to `\\`, erroring if neither is present, so realpath never returns one. The prefix can only arrive from caller-supplied input. The optional group in the regex stays as a labelled defensive no-op, and the function's behaviour is unchanged on purpose: these identity fields are compared between an `analyze` and a later `status` run, so this is not the place to reshape a path. `include-extractor.ts` claimed "gitnexus analyze stores absolute paths in the File.filePath column". A full self-index at |
||
|
Gergő Magyar
|
2ec00b8952
|
fix(analyzer): reject cross-drive paths in the identity containment guard (#2688)
`isInside()` paired its `..` checks with no absolute-path rejection, so on
Windows it reported an unrelated drive as *inside* the parent. `path.relative`
cannot express a relative path between two drives and returns the absolute
target instead:
path.win32.relative('C:\\parent\\src', 'D:\\other\\file.js') // 'D:\\other\\file.js'
That string does not start with '..', so the guard passed it.
Impact, per call site:
- resolveInvokedArtifact: adopts `process.argv[1]` as the invoked analyzer
artifact whenever it merely sits on another drive. That file is then absent
from the validated build, so resolveAnalyzerRunnerIdentity throws — `analyze`
and `status` fail outright on a multi-drive Windows install (e.g. a launcher
on D: invoking a package installed on C:). This is how the bug surfaced: the
GitHub Windows runner keeps the repo on D: and temp fixtures on C:.
- cacheDirectory: the "trusted cache directory must be outside the package and
build roots" guard wrongly fires for a directory on another drive, rejecting a
legitimate configuration.
- validateIdentityCache / cachedBuildDigestForPath: a containment check that can
answer "inside" for a path on another drive is weaker than intended.
Fix: reject an absolute `path.relative` result. This is the idiom the repo's
other containment guards already use — server/api.ts, server/git-clone.ts and
group/extractors/fs-utils.ts all pair the '..' check with `path.isAbsolute`;
this function was the outlier.
`pathApi` is injectable (defaulting to the platform-bound `path`) so the win32
semantics are unit-testable from a POSIX runner. The new test is fixture-free
and registered on the cross-platform matrix; its cross-drive case fails without
the guard and the same-drive/POSIX cases pass either way, proving the fix is
narrow.
Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
|
||
|
Gergő Magyar
|
df0110b06f
|
fix: index staleness — false-stale status after analyze (#2668) + inline staleness in query/context/impact/cypher tools (#2655) (#2683)
* fix(analyzer): case-stabilize runner-identity path fields so status isn't false-stale (#2668) `gitnexus status` reported a freshly-analyzed, untouched repo as stale on Windows (econia/aptos-core, 1.6.10-aptos.0). `status`'s up-to-date check gates on `runnerIdentityIsCurrent`, which deep-compares the stamped runner identity against a freshly recomputed one. That comparison includes `build.rootPath`, `dependencyRuntime.manifestPath`/`lockfilePath`, and `runtime.executablePath` (only `invokedArtifact` is stripped), and `identityCacheKey` hashes packageRoot/buildRoot — all derived from paths that flow through `realpathSync.native`, which canonicalizes 8.3 names and symlinks but does NOT normalize the Windows drive-letter case. When `analyze` and `status` are launched under different drive-letter casing (`c:\...` vs `C:\...`, plausible across CLI shim / npx / server-worker entries), the two identities differ by that one byte and `status` reports stale. Fix: `normalizeAnalyzerRootPath(p, platform)` uppercases the Windows drive letter (POSIX no-op, platform-explicit for testability; preserves a `\\?\` extended-length prefix), applied at the single upstream source — `resolveBuildRoot`'s returned `{packageRoot, buildRoot}` — so every derived identity path field and the cache key inherit a case-stable root, plus at `runtime.executablePath` (process.execPath is the same compared class). The `runnerIdentityIsCurrent` gate is kept intact: a genuine analyzer change still differs in `build.digest`/`dependencyRuntime`, and analyze still rebuilds on real mismatch. Note: the drive-letter divergence was not reproduced on a Windows host (none available); the mechanical chain is verified in source and the fix is a correct defensive normalization that is a no-op on POSIX. If a `status --json` identity field-diff later shows `build.digest`/`dependencyRuntime`/`cliVersion` diverging instead, that indicates a genuinely different install (where "stale" is correct), not this bug. Migration: on Windows, an existing index stamped under the old (non-normalized) casing mismatches the normalized recompute once, triggering a single forced full re-analyze on first upgrade (and a one-time identity-cache recompute). One-time, Windows-only, POSIX no-op. Tests: pure `normalizeAnalyzerRootPath` unit tests (drive-letter uppercase, idempotence, drive-only scope, `\\?\` extended-length prefix, POSIX no-op). * feat(mcp): surface index staleness in query/context/impact/cypher tool responses (#2655) `checkStalenessAsync` already computes how many commits an index is behind the checkout's HEAD, and `list_repos` returns it as `staleness: {commitsBehind, hint}`. But the four hot read tools an agent actually calls in a session — `query`, `context`, `impact`, `cypher` — never surfaced it: `resolveRepo` only runs `maybeWarnSiblingDrift` (stderr, sibling-clone drift only), so a direct tool call gave zero indication the index might be behind HEAD. Thread the existing signal into those four tools at the single `callTool` dispatch chokepoint (after the one `resolveRepo`), reusing the `list_repos` `{commitsBehind, hint}` shape: - `stalenessForTool` computes `checkStalenessAsync` behind an in-flight-promise cache (5s TTL) keyed by lbugPath, so N concurrent tool calls share one `git rev-list` and flat/branch handles (same repoPath, different lastCommit) don't collide. The cache entry is evicted with the repo's other per-index state when the repo leaves the registry. - `withToolStaleness` skips the `git` spawn entirely for results that can't carry the field (via `canCarryStaleness`), so error-returning calls pay nothing. - `attachToolStaleness` adds a `staleness` field to an object result only when the index is behind HEAD. It NEVER changes an existing result's shape: raw-array results (non-tabular cypher rows) are returned untouched, because the CLI's `--limit` and other consumers branch on `Array.isArray`; error envelopes and already-annotated results are left as-is. Non-blocking: `checkStalenessAsync` swallows git failures to `{isStale:false}`, so a git error just omits the field — it never fails the tool. Deliberately out of scope: `@group`-targeted calls forward to `callToolAtGroupRepo` before the chokepoint (multi-repo, single-commit staleness is ill-defined); the legacy `search`/`explore` aliases; and `list_repos` / the `context` resource, which already carry the signal. Tests: `attachToolStaleness` branch matrix (stale object -> field; fresh -> unchanged; raw array -> unchanged; error envelope -> unchanged; idempotent; non-object -> unchanged; null-safe) and a flat-vs-branch cache-key regression test that fails when the cache is keyed by repoPath. * test(mcp): cover staleness tool-signal edge cases + harden the freshness boundary (#2655) Addresses the coverage gaps the review flagged on the #2655 staleness signal, plus one defensive guard so a failing freshness check can never fail a tool. Production (defense-in-depth, no behavior change on the happy path): - withToolStaleness now awaits stalenessForTool with a `.catch(() => undefined)` so a rejection degrades to no-staleness instead of failing query/cypher/ context/impact. - stalenessForTool wraps the check in `Promise.resolve(...).catch(...)` that evicts the cache entry on rejection — a transient failure isn't served as a permanently-rejecting promise for the rest of the TTL window, and the `Promise.resolve` wrap makes the boundary robust to a non-thenable return (a no-op for the real async checkStalenessAsync). A resolving promise is never evicted, so happy-path dedup is unchanged. Tests (gitnexus/test/unit/calltool-dispatch.test.ts): - F1: a rejecting checkStalenessAsync leaves the tool payload intact with no staleness field, and a later call recovers (proves the entry isn't poisoned). Written first and confirmed to fail without the guard. - F2: staleness attaches on query/context/impact object results and on cypher's tabular {markdown,row_count}; a raw-array cypher result keeps its shape. - F3: drift guard — exactly query/cypher/context/impact route through stalenessForTool; explain/pdg_query/detect_changes/check do not. - F4: the per-index cache dedupes within TOOL_STALENESS_TTL_MS and recomputes after it expires (driven via a Date.now spy, not fake timers). Tests (gitnexus/test/unit/analyzer-identity.test.ts): - F5: the produced identity's build.rootPath and runtime.executablePath are normalizer-stable, guarding that both call sites thread through normalizeAnalyzerRootPath (trivial on POSIX, a real regression guard on Windows CI). Plus a source comment noting the one-time Windows re-analyze on first upgrade. * test(mcp): run #2668 guard on Windows CI, document staleness field, cover staleness edge cases Addresses the review follow-ups on the staleness work: - Wire test/unit/analyzer-identity.test.ts into scripts/cross-platform-tests.ts (PLATFORM_LOGIC). Its "identity path fields are normalizer-stable" fixpoint is the Windows regression guard for the #2668 drive-letter normalization, but normalizeAnalyzerRootPath is a POSIX no-op, so the guard was only ever running (trivially green) on the Ubuntu full-suite and never on the windows-latest matrix where it actually bites. Now it runs where it matters. - Document the inline `staleness` field on query/context/impact/cypher responses in the gitnexus-guide skill (both the .claude source and the shipped gitnexus-claude-plugin mirror, kept in sync). - Add three staleness tests that pin behavior the prior tests only implied: * @group-routed calls never get the signal (forwarded before the wrapping switch) — locks the intentional skip so it can't silently flip. * one in-flight freshness check is shared across truly concurrent calls (two dispatched before checkStalenessAsync settles → a single spawn), not just sequential reuse of an already-resolved value. * a late rejection from a superseded cache entry does not evict the newer entry that replaced it after the TTL rolled over (the `=== entry` object-identity guard). The defensive stack in stalenessForTool/withToolStaleness (Promise.resolve wrap + guarded evict + outer catch) is retained deliberately: the wrap is load-bearing for the tests (a sibling describe's vi.resetAllMocks() makes the mock return undefined), and the guarded evict closes the superseded-entry edge now covered above. * fix(test): split the #2668 normalization guard into a portable cross-platform file Registering analyzer-identity.test.ts on the Windows/macOS matrix (previous commit) surfaced four pre-existing failures in that file on macOS 3/3 and windows 3/3. They are not new breakage: those fixture tests compare identity fields against the RAW temp-dir path while the identity resolves through realpathSync.native, so on macOS `/var/folders/...` is received as `/private/var/folders/...`. The file was simply never portable — it had only ever run in the Ubuntu full-suite. Reproduced locally by pointing TMPDIR at a symlink: the same four tests fail, and pass again without it. Move only the portable assertions — the pure `normalizeAnalyzerRootPath` cases (explicit `platform` argument) and the identity fixpoint guard (which compares each field against ITSELF normalized, never against the fixture path) — into test/unit/analyzer-identity-path-normalization.test.ts, and register that file on the matrix instead. The #2668 Windows regression guard still runs where it actually bites, without dragging four symlink-sensitive tests onto runners they were never written for. Verified: the new file passes with TMPDIR behind a symlink (the macOS condition); the heavy file is back to Ubuntu-only. * fix(test): keep the cross-platform #2668 file fixture-free so Windows stays green The split file still carried the fixture-based fixpoint guard, which fails on windows-latest: Invoked analyzer artifact is absent from the validated build: D:\a\...\node_modules\vitest\dist\workers\forks.js Cause is a pre-existing cross-drive defect in this module's `isInside()`, not the #2668 change. The GH Windows runner keeps the repo on D: and temp fixtures on C:. `path.win32.relative('C:\\...fixture', 'D:\\...forks.js')` cannot express a relative path across drives, so it returns the absolute target — which does not start with '..', so `isInside()` reports true. `resolveInvokedArtifact` therefore treats the vitest fork worker as the invoked artifact, it is absent from the fixture's validated build, and identity resolution throws. (Verified directly: `isInside` returns true cross-drive and false for the same-drive control.) Keep the cross-platform file strictly pure — only `normalizeAnalyzerRootPath` assertions with an explicit `platform` argument, no fixture and no filesystem — so it is green on every runner while still exercising the transform on real Windows. The fixture-based threading guard moves back to analyzer-identity.test.ts (Ubuntu-only), where the rest of that file's fixture tests already live, with a comment recording why it cannot be on the matrix. The underlying `isInside()` cross-drive bug is left untouched here (out of scope for this PR) but is worth its own fix: it also guards the trusted cache directory and the identity-cache path-escape check in validateIdentityCache, where a false "inside" verdict weakens validation on multi-drive Windows setups. --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> |
||
|
Gergő Magyar
|
1e764cd475
|
fix(analyze): single-writer lock for the index write path (#2658) (#2677) | ||
|
Gergő Magyar
|
7f7255aef8
|
fix(analyze): load VECTOR before the incremental writeback touches embedding rows (#2623) (#2624)
* feat(lbug): add ensureEmbeddingRowDmlSafe VECTOR gate for embedding-row DML LadybugDB refuses every mutation of a table carrying an HNSW index while the VECTOR extension is not loaded on that connection: DELETE and CREATE raise a Binder exception, DROP TABLE is refused while the index references it, and SET segfaults the process. Dropping the index is not an available recovery either — CALL DROP_VECTOR_INDEX is itself a VECTOR-extension function and is undefined in exactly that state. Add a single primitive that loads VECTOR under the analyze install policy and, only when that fails, reads CALL SHOW_INDEXES (which works without the extension) to decide whether an index actually exists to trip over. No call sites yet. Refs #2623 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(lbug): pin the #2623 VECTOR gate for embedding-row DML Three cases: no index + VECTOR unavailable stays safe (no needless escalation); index present + VECTOR unavailable is reported blocked AND the raw deleteNodesForFiles genuinely throws 'extension is not loaded' (proving the hazard is real, not theoretical); index present + VECTOR loadable is safe, the delete works, and the HNSW index survives — the invariant run-analyze relies on when it keeps the index across a surgical incremental run. Refs #2623 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(analyze): load VECTOR before the incremental writeback touches embedding rows Incremental analyze died on every content change once a repo had built code_embedding_idx: Analysis failed: Binder exception: Trying to delete from an index on table CodeEmbedding but its extension is not loaded. The surgical writeback's first statement is deleteNodesForFiles' CodeEmbedding join-delete, but nothing on that path loaded VECTOR until Phase 4 — so the engine refused the delete. This is an ordering defect, not an environment one: it reproduces on machines where VECTOR loads fine. The dirty-flag recovery then forced a full rebuild on the next run, which is why it read as 'just slow'. Call ensureEmbeddingRowDmlSafe() once, before the escalation gate and before any row is touched — the same 'index lifecycle before row DML' seam dropSearchFTSIndexes occupies for FTS (#2589). Unconditional, because a DB carrying the index from an earlier --embeddings run hits the same wall on a plain incremental run. When VECTOR truly cannot load the table is immutable (the index cannot be dropped without the extension either), so the run falls through to the existing wipe-and-COPY escalation with a message naming cause, consequence and remedy. Fixes #2623 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(analyze): pin the #2623 VECTOR-before-embedding-DML ordering end-to-end Sibling of the #2589 FTS drop-before-delete suite, same shape: drive the real runFullAnalysis incremental path over a real git repo and a real LadybugDB, seed real embedding rows, build the HNSW index, then assert the index state at the exact moment deleteNodesForFiles is invoked. Both cases were confirmed to discriminate — with the run-analyze change reverted they fail with the reported 'Trying to delete from an index on table CodeEmbedding but its extension is not loaded', and pass with it: - surgical path: the run completes, the index is still present AND extension_loaded at delete time, exactly one row per nodeId survives, and the untouched file's rows are preserved - blocked path: with GITNEXUS_LBUG_EXTENSION_INSTALL=never the run escalates to a full DB write and says so, instead of crashing Also applies prettier's reindent to the run-analyze log ternary. Refs #2623 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(lbug): cite the pinned LadybugDB version in the #2623 probe note The probe matrix behind ensureEmbeddingRowDmlSafe was first recorded on 0.18.0, but gitnexus/package-lock.json pins 0.18.2 (#2587). Re-ran every case on 0.18.2: refused DELETE, refused CREATE, SIGSEGV on SET, DROP_VECTOR_INDEX undefined, DROP TABLE refused, SHOW_INDEXES readable with extension_loaded intact. Identical on both, so the design is unchanged — only the citation was wrong. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(analyze): preserve embeddings across the VECTOR-blocked rebuild, and check the catalog before loading Three follow-ups from reviewing the fix itself. 1. Data loss on the blocked path. Escalating wipes the DB files, and Phase 3.5 restores embedding rows from cachedEmbeddings — which deriveEmbeddingMode only populates when meta.stats.embeddings > 0. A DB holding embedding rows that its meta does not account for therefore had every vector destroyed silently by a rebuild it never asked for. Probe on a 3-file repo: 3 rows before, 0 after, no warning. Read the rows before escalating (a plain MATCH, no extension needed) so the existing restore has something to restore, and say so in the log. The blocked-path test now asserts the seeded rows survive exactly once, and that assertion fails without this rescue. 2. Catalog before extension. ensureEmbeddingRowDmlSafe loaded VECTOR first and only read SHOW_INDEXES on failure, so every incremental analyze on a machine without VECTOR paid a bounded out-of-process INSTALL attempt plus an 'extension unavailable' warning — including repos that never built an embedding index and can never hit this bug. One local catalog read settles that case first; the load is attempted only when an index actually gates DML, or when the catalog cannot be read. 3. Dead branch. targetConn is always the module singleton there, so the isSharedSingletonConn ternary could never take its second arm. Collapsed to withConnLock. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(doctor): live-probe the VECTOR extension instead of printing the static platform capability Review finding on #2624 (MEDIUM), and exactly what #2623's reporter hit: doctor printed 'VECTOR index: available' — derived from a static platform check — while every incremental analyze on the same machine was dying on an unloaded VECTOR extension. The FTS line was switched to a live LOAD probe for the identical contradiction under #2374; VECTOR now gets the same treatment. probeVectorExtensionLoad shares the FTS probe's implementation (bounded, offline-safe, never runs the installer) and doctor's semantic-mode line now follows the probe, not the platform: without a loadable extension the vector index can be neither built nor queried, so search really is on exact scan. The load-error classifier's remedies are label-parameterized so the VECTOR row stops dispensing FTS-specific advice — 'run analyze --repair-fts' repairs FTS indexes only and was actively wrong for a missing vector extension. Default label stays 'FTS'; every existing caller and pinned remedy string is unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(lbug): remove the stale Windows VECTOR gate — the extension ships for win_amd64 The codebase categorically refused VECTOR on Windows (platform !== 'win32' in isVectorExtensionSupportedByPlatform, plus a hard early-return in loadVectorExtension) on the strength of an early-era report that in-process INSTALL VECTOR could SIGSEGV (#1365). That belief is stale, verified directly: - the extension server hosts win_amd64 VECTOR artifacts for every 0.18.x extension version — v0.18.0 and v0.18.1 both serve a real 14 MB PE32+ DLL (curl-probed; 'file' confirms PE32+ x86-64) - the pinned 0.18.2 core resolves its extension directory to 0.18.1 (strace-verified LOAD open()), so the pinned version's Windows artifact exists too - INSTALL now runs in a spawned child (installDuckDbExtensionOutOfProcess), so even a crashing installer kills only the child and degrades to unavailable — the original hazard cannot reach the parent process any more Windows now takes the same runtime path as every other OS: try LOAD, install out-of-process when policy allows, degrade to exact scan when it truly fails. The MCP semantic-search lane loses its static platform gate too — it always attempts the vector index and falls back to the exact scan on runtime failure, with a once-per-backend diagnostic naming the real error instead of a platform-policy message. isVectorExtensionSupportedByPlatform is deleted; getRuntimeCapabilities reports the platform capability as available everywhere and defers machine truth to the live probe. Windows CI is the enforcement: the vector suites skip visibly only when the extension genuinely cannot load, so green Windows lanes now actually exercise VECTOR instead of silently skipping by policy. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(lbug): pin the catalog-read-failure fallback in ensureEmbeddingRowDmlSafe Review finding on #2624 (LOW): the one branch where the gate cannot cheaply prove safety — SHOW_INDEXES itself erroring — was exercised only by inference. Force it with a Connection.prototype.query spy over the real DB: the catalog read fails, and the gate must fall through to actually attempting the extension load (asserted via the recorded statement stream) rather than guessing, returning true here because the extension is loadable. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(mcp): load VECTOR on the pool's shared Database so the semantic vector lane actually works Review finding on #2624 (MEDIUM): extension load scope is per-Database (probe-verified — LOAD on one connection enables QUERY_VECTOR_INDEX on every connection of the same Database), and the pool pre-warm loaded only FTS. So LocalBackend's vector lane has ALWAYS raised 'Catalog exception: function QUERY_VECTOR_INDEX is not defined' through the pool and silently fallen back to the exact scan — repos above the 10k exact-scan cap got empty semantic results. The serve path was unaffected (the embedding pipeline loads the extension itself). Mirror the FTS line at BOTH load sites — doInitLbug's pre-warm and initLbugWithDb's external-Database adoption — under the same load-only contract (the read pool never triggers a network install), tracked by a new SharedDB.vectorLoaded flag reset where ftsLoaded resets. The new pool test is discriminating and deliberately closes the writable core adapter before the pool opens: a shared/injected Database would inherit the VECTOR load from test seeding and pass either way, so the case forces the pool onto its OWN fresh read-only Database where only the pre-warm can make the lane legal. Verified: fails at the pre-fix tree with the exact Catalog exception, passes with the fix. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * ci: run the #2623 ordering suite on Windows/macOS and pre-install VECTOR alongside FTS Two review findings on #2624, both landing in existing seams: - scripts/cross-platform-tests.ts gains incremental-vector-extension-ordering .test.ts: the win32 VECTOR gate is gone in this PR, so the #2623 drop-ordering + blocked-path escalation must be proven on the windows-latest native addon, not just Ubuntu. (The review's claim that lbug-delete-nodes-for-files.test.ts was also missing was wrong — it has been on the roster since #2409.) - scripts/ensure-fts.ts now pre-installs VECTOR under the same best-effort auto-policy contract, so every sharded CI process LOADs from ~/.lbdb instead of racing its own bounded out-of-process INSTALL; the workflow's extension cache already covers it (path is the whole extension dir — key kept for cache continuity). The cross-platform job sets GITNEXUS_REQUIRE_VECTOR=1 beside GITNEXUS_REQUIRE_FTS so a genuinely unavailable VECTOR is a loud failure, never a silent skip. Windows/macOS cannot be executed locally; the PR's CI lanes are the proof for this commit. Linux smoke: ensure-fts.ts reports both extensions ready; all 79 roster entries resolve. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(pool): register loadVectorExtension in the pool unit-suite mocks The pool adapter's new loadVectorExtension import surfaced in four suites that mock lbug-adapter.js with explicit factories (vitest fails loudly on a missing mocked export). Register the export in each — resolving false where the suite's world assumes no vector, true where it mirrors FTS — and extend lbug-pool-fts-load.test.ts, the suite that owns pre-warm extension loading, with the vector pair: successful load cached per shared Database, failed load retried on the next open, both pinned to policy load-only. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(analyze): use POSIX literals for graph paths in the #2623 ordering suite First Windows CI run of this suite (it joined the cross-platform roster this PR) failed with 'Parser exception: Invalid input <MATCH (n:Function) WHERE n.filePath = '>' — path.join produces backslashes on Windows, and a backslash inside the seed helper's single-quoted Cypher literal breaks the parser. The graph stores repo-relative filePaths with forward slashes on every OS, so graph-side paths are POSIX literals now (the incremental-orchestration convention); path.join stays only for real filesystem access. The same Windows lane also proved the substance this suite exists for: lbug-vector-extension passed 7/7 on windows-latest — the extension installed, loaded, and built a real HNSW index there — and the pool vector-lane and DML gate suites passed too. This commit fixes the harness, not the fix. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Gergo Magyar <abhigyan1.patwari@gmail.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Parafee41
|
737a8cdb18
|
fix(web): use repo path identity in switcher (#2420)
* fix(web): use repo path identity in switcher * keep repo URL project names stable * fix server repo path resolution * fix repo path miss resolution * fix(server): guard clone-dir deletion with path ownership check Deleting a registry entry derived its clone dir from the entry NAME with no ownership check, so deleting a local repo that shares a display name with a server-cloned sibling wiped the sibling's checkout. Gate the removal on cloneDirBelongsToEntry (canonicalized path equality), the same entry.path-driven rule the handler's step 2b already mandates. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(server): fail closed on relative repo params and rate-limit GET /api/repo Relative separator-containing ?repo= values (org/name, ./repo) were canonicalized against the server CWD — an attacker-influenced realpathSync probe on an un-rate-limited GET — before failing anyway. Reject them immediately without touching the filesystem, drop the redundant path.sep clause, document the resolver's two-tier contract, and wire createRouteLimiter on GET /api/repo like its DELETE sibling. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(server): lock repo resolver branches and register for Windows CI Lock in the resolver's remaining branches: first-wins for ambiguous bare names, Windows-shaped input as a fail-closed path claim, the repos[0] default, and the case-insensitive name fallback. Register the suite in cross-platform-tests.ts so windows-latest actually runs the path-shape logic it exists to protect. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(web): single repoIdentity helper with repoPath normalized end-to-end The identity fallback chain was copy-pasted in Header and RepoLanding while backend-client already owns BackendRepo and the repoPath normalization. Export one repoIdentity helper, normalize fetchRepos like fetchRepoInfo, and emit repoPath from GET /api/repos so the scheme no longer silently relies on /api/repo.repoPath equalling /api/repos.path. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(web): persist and restore repo path identity in the URL The URL persisted only ?project=<display name>, so refreshing after switching to a duplicate-name repo silently restored the first same-named sibling. Persist ?repo=<server-resolved path> alongside the readable ?project= at both write sites, prefer it on restore (legacy project-only URLs still work), keep failed path restores fail-visible (no name fallback), and strip stale identity params when deleting the active or last repo. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(web): analyze completion connects by path identity RepoAnalyzer's completion callback passed the display name, so analyzing a repo whose basename collides with an existing one reconnected the first same-named sibling. The SSE terminal payload now carries the job's repoPath (both emit sites), the analyzer passes that identity to onComplete while the done screen keeps showing the display name, and old servers without repoPath degrade to today's behavior. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(web): scope code-reference file reads to the active repo identity The code viewer passed the display name as the repo scope, so with duplicate-name repos it rendered the wrong repo's file contents under the right filename. Pass the active path identity (currentRepo) with the display name as fallback, and collapse the two dead repo fields that were already shadowed by the readFile spread. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(web): show display names instead of absolute paths in labels The path-identity switch leaked raw filesystem paths into three user-facing surfaces: the re-analyze progress label, the repo-switch overlay, and the agent prompt's project name via loadGraphAnyway. Resolve display names at render time (registry lookup, then basename fallback) while state keeps holding the identity; loadGraphAnyway passes the name explicitly because initializeAgent's empty-deps closure would otherwise fall through to the literal 'project'. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(web): stop initializeAgent from clobbering repo identity with display names initializeAgent fell back to writing overrideProjectName (a display name) into the repo identity, so any future name-only caller — the pre-PR idiom — would silently kill the Active badge and re-admit the duplicate-name ambiguity through the agent path. Only opts.repo may write the identity now. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(web): drop dead initializers flagged by CodeQL pNameStr's and repoIdentity's initial values were never read: both are assigned on the success path before any use and the catch returns early. Bare declarations resolve CodeQL alerts 825/826. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * style(web): fix tailwind class order per root prettier plugin The worktree pre-commit hook resolved prettier-plugin-tailwindcss through symlinked node_modules and sorted scrollbar-thin differently than CI's clean-room install. Re-formatted with the root lockfile environment; no behavior change. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(web): e2e coverage for every #2419 duplicate-name ambiguity Provision two live repos with the same basename under different parents via POST /api/analyze, then drive a real browser through each item of the issue's "Actual behavior" list: - duplicate rows render and the ACTIVE one is identifiable before and after switching (active-state must not compare repo.name) - switching between duplicates swaps the loaded graph, verified by per-repo marker files (onSwitchRepo must not receive repo.name) - re-analyze targets the clicked duplicate's exact path (POST body), tracks progress on that row only, and the completion reconnect requests that same path — never the same-named sibling - delete requests target exactly the chosen duplicate's path; the sibling stays registered and loaded - backend ?repo= resolution is path-first: landing selection loads the exact repo, ?repo= survives F5, and a stale path fails closed to the repo picker instead of retargeting the sibling Adds four data-testids to Header (switcher trigger/row/reanalyze/ delete, rows expose data-active) so the spec has stable selectors, and broadens the post-analyze reconnect retry in App to any BackendError: the server may still be reinitializing when the SSE complete event fires, and that surfaces as transient 5xx/binder errors, not only 404. The re-analyze and delete tests deliberately assert identity at the request level and tolerate two pre-existing server races that are unrelated to the #2419 identity contract (freshly-analyzed DB briefly unreadable after SSE complete; registry validate-prune clobbering a concurrent unregister) — see the in-test comments. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(autofix): apply prettier + eslint fixes via /autofix command * test(web): isolate repo-path-identity e2e onto a spec-owned backend The spec is the only e2e file doing write operations (analyze, re-analyze, delete). Running its force re-analysis against the shared CI backend while parallel workers held connections took the whole server down (run 29145679019: the jobId poll died with ECONNRESET and every later test in every file failed to connect). Spawn a dedicated `gitnexus serve` on port 4799 with an isolated GITNEXUS_HOME in beforeAll instead: writes can no longer perturb the other suites, a crash is contained to this spec (its output is captured and printed, which CI otherwise loses), and the registry is hermetic by construction — the previous leftover-purge and shared-registry cleanup are gone. Every page is pointed at the spec backend through useBackend's supported localStorage override, which covers both the probe-driven landing flow and the ?server= auto-connect. Verified self-sufficient (6/6 with no shared server running) and non-interfering (full suite 39/39 with the shared server up). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * stabilize repo path identity e2e * fix(server): don't report analyze complete before the index is settled The analyze worker reports `complete` over IPC before its on-disk finalization (LadybugDB checkpoint, native handle release, metadata write) is visible at the storage path — observed up to ~6.5s behind the IPC message. The launcher's "reinitialize backend BEFORE marking complete" ordering was meant to make the repo queryable by the time the client sees the SSE complete event, but it never verified that: clients reconnecting on that event read a database still being written. Locally that surfaces as "Binder exception: Table CodeRelation does not exist" or a silently empty graph, and the open can quarantine the in-flight WAL; on slow CI runners the native layer racing the rewrite has killed the whole server (signal exit, no output — run 29146867959). Gate the complete transition on the index actually settling: LadybugDB file and metadata both rewritten by THIS job (mtime >= job start — bare existence is not enough, a re-analysis leaves the previous index in place while it works) and no transient WAL/shadow/checkpoint sidecars remaining. Bounded (60s) and proceed-on-timeout, so a job whose analysis legitimately rewrites nothing cannot wedge. Also evict the server's cached DB handle before reinitializing — same invalidation DELETE /api/repo performs — so post-completion reads cannot be served from a pre-rewrite handle. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(web): assert re-analyze completion identity at the request level The strict form (Ready + marker on the re-analyzed duplicate) still trips a deeper pre-existing storage race that makes a freshly re-analyzed database transiently unreadable to the reconnect even with the settle gate in place — unrelated to the #2419 identity contract this test covers. Keep the identity assertions (the reconnect targets the exact duplicate's path and never the same-named sibling) and leave a pointer to tighten once the storage race is fixed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(server): resolve the settle-gate path from the registry, not the request CodeQL flagged the settle gate's stat/exists probes as js/path-injection: the probed path derived from the user-provided analyze `path`. Resolve it from the repo's registry entry instead — the user value is now only a comparison key, and the probes run against the server-owned storagePath record, which is also the authoritative path readers resolve through. Re-resolved each poll round because the worker registers the repo as part of the same finalization the gate is waiting out. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Gergő Magyar <gergomagyar@icloud.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> |
||
|
VL
|
117587d543
|
fix(cli): actionable diagnostics for non-4K page-size buffer manager failures (#2424)
Some checks are pending
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Scorecard / Scorecard analysis (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
|
||
|
Gergő Magyar
|
df1fc36094
|
fix: make large incremental writebacks commit reliably (#2409) (#2425) | ||
|
Gergő Magyar
|
8402963198
|
fix(ci): shard platform-sensitive matrix + spawn built CLI to fix Windows cross-platform timeout (#2394)
* fix(ci): shard platform-sensitive matrix + spawn built CLI to fix Windows cross-platform timeout The `windows-latest (platform-sensitive)` job was hitting its 15-min internal vitest watchdog in run-cross-platform.ts. It's cumulative slowness, not a hang: the fixed 72-file suite is dominated by ~50 CLI/worker process spawns, and Windows is ~5x slower than macOS at process startup (macOS ran the same set in ~3min of tests). Two complementary changes bring it back under the watchdog with headroom, without touching any test assertion: - Shard the platform-sensitive matrix (windows/macos × shard [1,2]) and forward `--shard=i/2` through run-cross-platform.ts to vitest, which partitions the fixed file list deterministically (sha1, equal file-count) — halving each runner. macOS/Ubuntu were already under budget. - New test/helpers/cli-entry.ts (`CLI_SPAWN_PREFIX`): spawn the built `dist/cli/index.js` when `GITNEXUS_E2E_CLI=dist` (set on the cross-platform job, which already builds) instead of `node --import tsx src/cli/index.ts`, which re-transpiles the whole CLI on every spawn. Defaults to tsx-on-source so local runs always reflect current source; `GITNEXUS_E2E_CLI=dist` on an unbuilt tree throws an actionable "run npm run build" error. dist is opt-in only — never inferred from a generic `CI` env — so an ambient `CI=1` can't silently run a stale build. Converted 8 spawn-based e2e suites; added test/unit/cli-entry.test.ts. The Ubuntu coverage job leaves `GITNEXUS_E2E_CLI` unset, so the tsx-on-source path stays exercised in CI too (both entry points covered). Measured on Linux: cli-limit-e2e 121.5s→91s, cli-e2e 289s→217s (~25%); larger on Windows where the transpile is a bigger share of each spawn. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(ci): derive platform-sensitive shard count from one source (#2394) The shard total was hardcoded in three coupled, unenforced places (matrix length, job-name suffix, --shard denominator); editing one without the others silently dropped a shard's tests with green CI. Add a checkout-free shard-plan job whose single TOTAL generates both the shard index list (consumed via fromJSON) and the /N denominator (job name + --shard arg), so they cannot drift. Asserts TOTAL>=1 to rule out an empty-matrix silent skip. No behavior change — still 2 shards per OS. Addresses PR #2394 tri-review finding F2. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ci): 3 shards for real Windows headroom + honest sharding comments (#2394) vitest shards by file COUNT, not runtime, so the heaviest spawn suites cluster into one shard: live CI showed Windows shard 1/2 at 12m12s (~81% of the 15-min watchdog) vs shard 2/2 at 3m0s. The old comments claimed "comfortable/generous headroom", which the count-based split doesn't deliver at 2 shards. Bump TOTAL to 3 (one line, single source) so even the busiest Windows shard clears the watchdog, and reword the comments to describe count-based (not time-based) sharding. Addresses PR #2394 tri-review finding F1. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(ci): extract testable parseShardArg from run-cross-platform (#2394) The --shard parse/forward glue had no unit test. Extract it into a pure scripts/shard-arg.ts (mirroring the computeSpawnPrefix extraction precedent) so the branch logic is lockable without the script's top-level execFileSync, and add test/unit/shard-arg.test.ts (absent -> undefined, valid token -> passed through, found amid other args). Behavior unchanged; U4 adds the malformed fail-loud on top. Addresses PR #2394 tri-review finding F3. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ci): fail loud on a malformed --shard arg (#2394) A shard-shaped-but-malformed arg (--shard=1, --shard, --shard=abc) was silently ignored, dropping the shard flag so both legs ran the full unsharded ~50-spawn suite — re-arming the Windows watchdog timeout with no signal. parseShardArg now throws an actionable error on any --shard/--shard=… arg that fails the strict regex (unrelated flags like --shardx= pass through), and the call site in run-cross-platform.ts catches it into console.error + exit 1, kept outside the execFileSync try so the message isn't swallowed by that catch's watchdog-only branch. Addresses PR #2394 tri-review finding F4. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(test): fail loud on an unknown GITNEXUS_E2E_CLI value (#2394) computeSpawnPrefix silently degraded any unknown GITNEXUS_E2E_CLI value to tsx-on-source, so a typo (e.g. `dsit`) would make CI believe it tests the dist entry point while actually running src. Throw on any value other than 'dist'/'src'/unset (the safe tsx default is preserved for unset/''/'src', so it still never selects dist without an explicit opt-in). Flip the unknown-mode unit test to assert the throw and add the missing {mode:undefined, distExists:true} case. Only ci-tests.yml sets the var (=dist), so no existing suite is affected. Addresses PR #2394 tri-review findings minor-a/b. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(ci): run cli-entry.test.ts on the cross-platform matrix (#2394) cli-entry.test.ts resolves CLI_SPAWN_PREFIX from a real path, and its last assertion (cli[/\\]index) has a Windows backslash branch that only Ubuntu exercised. Register it in PLATFORM_LOGIC so it runs on the Windows/macOS matrix too. (shard-arg.test.ts stays out — pure string logic, OS-independent.) List grows 73 -> 74; the generated shard matrix keeps coverage complete. Addresses PR #2394 tri-review finding minor-c. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(test): share tsxLoaderUrl(), dedup the last tsx-loader boilerplate (#2394) bridge-cache-reopen.test.ts carried its own copy of the tsx-loader-resolution boilerplate (createRequire -> resolve('tsx/package.json') -> pathToFileURL) — the one site the PR's CLI_SPAWN_PREFIX migration didn't cover (it spawns a seed script, not the CLI). Export the existing tsxLoaderUrl() from cli-entry.ts and reuse it here; the resolved loader URL is byte-identical. Addresses PR #2394 tri-review finding minor-d. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(test): make skipUnlessFtsAvailable install FTS on miss so shards are self-sufficient (#2394) Sharding the platform-sensitive suite into 3 exposed a latent test-isolation bug: load-only FTS primitives (test/integration/lbug-core-adapter.test.ts) only passed because a sibling installer test happened to co-locate in the same shard and install FTS into the shared ~/.lbdb first. At 3 shards, lbug-core-adapter landed in a shard with no installer sibling, so its load-only loadFTSExtension() failed deterministically on macOS+Windows shard 2/3 under GITNEXUS_REQUIRE_FTS=1. Make the gate self-sufficient: on a load-only miss under REQUIRE_FTS, install FTS with `auto` (LOAD-first, then one bounded network INSTALL) before treating it as a hard failure — mirroring withTestIndexedDB. A pre-installed extension still costs no network (auto is LOAD-first); offline/local runs (no env var) still skip gracefully. Verified: with a fresh HOME (no pre-installed FTS) + REQUIRE_FTS=1, lbug-core-adapter now passes 15/15 (previously threw). Addresses the 3-shard CI failure surfaced while validating PR #2394's F1 fix. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * ci: warm-cache the LadybugDB FTS extension across platform shards (#2394) Follow-up to the FTS self-install fix: cache ~/.lbdb/extension per OS + lockfile so a warm run skips the network install entirely and the parallel shards share one download across runs. Pure reliability/speed — on a cache miss the tests still self-install FTS on demand (test/helpers/fts-availability.ts), so this is never a correctness dependency, just a way to cut the network-install surface that made the sharded FTS tests flaky. Keyed by lockfile hash (a LadybugDB version bump re-installs); per-OS since the extension is a native binary. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ci): pass shard via env to clear zizmor template-injection (#2394) Interpolating ${{ matrix.shard }} (now sourced from the shard-plan job output) directly into the run: shell tripped zizmor's template-injection audit (code-scanning alert #824, ci-tests.yml:147). Move the value into a SHARD env var — assigned via ${{ }} but referenced as "$SHARD" in the shell, which is not an injection sink — and set shell: bash so the expansion is uniform across the windows + macOS matrix (the default run shell is pwsh on Windows, where $SHARD would be empty and trip the new malformed-shard fail-loud). Verified locally with zizmor: the :147 template-injection finding is gone. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * ci: shard the ubuntu coverage job and merge blobs before the threshold gate (#2394) The coverage job ran the full suite unsharded (~16 min). Shard it like the cross-platform matrix, then merge the per-shard coverage before enforcing the threshold gate: - shard-plan now also single-sources the coverage shard count (cov_total / cov_shards), so the coverage matrix + /N denominator can't drift. - The `tests` job becomes a coverage shard matrix: each shard runs `vitest run --shard --coverage --reporter=blob` with thresholds forced to 0 (a single shard's partial coverage can never meet the gate) and uploads its blob. FTS self-installs per shard, so sharding the full suite is safe. - New `coverage-merge` job (needs: tests) reduces the blobs with `vitest --mergeReports`, enforcing the REAL config thresholds on the combined ('new') coverage — this is the gate. It also emits the merged test-results.json and runs the unsharded web + docker suites, so the `test-reports` artifact keeps the exact shape ci-report.yml consumes for its base-branch ('baseline') vs new coverage delta. The shard arg goes through a SHARD env var + shell: bash (no template-injection). Validated locally: shard blobs write and merge into a coverage-summary.json + merged test-results.json; the merge enforces thresholds on the union. CI Gate still aggregates the coverage-merge result via the reusable-workflow call. Note: the coverage check names change (ubuntu / coverage 1/3 … + merge) — update any pinned branch-protection required checks. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ci): include hidden files when uploading the coverage blob (#2394) The coverage shards write their blob to gitnexus/.vitest-reports/ (a dotdir). actions/upload-artifact excludes hidden files by default, so the coverage-blob-* artifacts uploaded empty — the merge job then downloaded 0 artifacts and vitest --mergeReports failed with ENOENT scandir '.vitest-reports'. Set include-hidden-files: true on the blob upload so the blobs actually ship. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ci): group shard-plan GITHUB_OUTPUT writes to satisfy shellcheck SC2129 (#2394) Adding the coverage shard outputs (cov_shards/cov_total) made the shard-plan gen step write four individual `>> "$GITHUB_OUTPUT"` redirects, which shellcheck (run by the actionlint check) flags as SC2129. Group the echoes into a single `{ …; } >> "$GITHUB_OUTPUT"` block. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(test): cost-balanced shard sequencer to cut CPU contention (#2394) vitest's default --shard hashes file paths and splits by file COUNT, which clustered the spawn-heavy suites onto one runner (Windows platform shard 1 ran ~4x the others). Add a custom sequence.sequencer that overrides only shard() and balances by estimated WORK instead: - specWeight() weights the fileParallelism:false spawn-heavy suites (cli-e2e, lbug-db — already isolated to run sequentially) far above the parallel default files, plus file size as a cheap finer signal. Deterministic per checkout. - assignShards() does greedy longest-processing-time bin-packing (heaviest file into the currently-lightest shard). The partition stays complete and disjoint — verified: on the 74-file cross-platform set the three shards weigh 7611/7610/8064 (the sequential-heavy files spread ~7/7/8) with zero overlap and no file dropped, vs the hash split's count-only balance. sort() is left to the base sequencer so project groupOrder / duration-cache ordering is untouched. Pure logic split into shard-balance.ts with a unit test locking the disjoint+complete, balance, and determinism properties. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ci): install + cache FTS up front on the coverage (and cross-platform) shards (#2394) coverage 3/3 failed on extension-binary-real.test.ts: it uses the file-path FTS gate (requireFtsResourceOrSkip), which resolves ~/.lbdb/extension at MODULE LOAD and cannot self-install the way the load-path gate (skipUnlessFtsAvailable, U8) does. The coverage job had no FTS cache and relied on an installer test running first in the shard — the balancing sequencer reshuffled the shards and dropped extension-binary-real into a shard with no installer, so FTS was absent. Remove the ordering dependency: add scripts/ensure-fts.ts (init a throwaway lbug db, loadFTSExtension with policy:auto → LOAD-first, INSTALL on miss) and run it up front on every coverage AND cross-platform shard, after restoring the per-OS FTS cache. The coverage job now shares that same cache key (it previously had none — this is the "share the cached FTS with coverage" the failure pointed at). Cold cache installs once; warm cache is a no-network load. Verified locally: ensure-fts installs FTS into a fresh HOME and is a no-op when already present. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Gergő Magyar
|
b98f6e458f
|
fix(lbug): recognize Windows missing-shadow error so serve repo-switch recovers (#2382) (#2387) | ||
|
Gergő Magyar
|
76a1c90b02
|
fix(fts): diagnose Windows FTS missing-dependency load failures (#2374, Phase 1) (#2383)
Some checks are pending
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Scorecard / Scorecard analysis (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
* feat(lbug): classify FTS extension load errors with Windows missing-dependency guard (#2374) Add classifyExtensionLoadError() — a pure-string, lbug-free four-way classifier (missing_file / corrupt_file / missing_dependency / unknown). The Windows catch-all guard keys missing_dependency strictly on the error-126 signal, never LadybugDB's generic 'Failed to load library … needed by extension' wrapper, so 127/5/1114 and truncated (193) files route correctly. * feat(fts): surface classified missing-dependency remedy in doctor, repair-fts, and degrade warnings (#2374) Route the FTS load reason through classifyExtensionLoadError at all four surfaces (doctor, --repair-fts error, analyze degrade log, ftsDegradedWarning). For the Windows missing-dependency class, emit the runtime-install remedy (VC++ redist, then OpenSSL) instead of the wrong reinstall-over-network guidance; other classes keep their existing routing. Path redaction preserved on the client-facing warning. * test(fts): assert doctor surfaces the classified remedy end-to-end (#2374) Extend the broken-file e2e: doctor now prints the corrupt-file re-download remedy through the real CLI, and the Windows missing-dependency remedy (VC++/OpenSSL) must not misfire on a corrupt file — the catch-all guard, verified end-to-end. Also assert the repair path does not misfire. * style(fts): apply prettier formatting to #2374 diagnosis files * feat(fts): language-independent hedged fallback for Windows load failures (#2374) The Windows OS-error tail is localized, so matching only en/zh 126 text left other locales on the generic 'run doctor' remedy. lbug's 'Failed to load library' wrapper is English on every platform and present for all load failures, so use it as a fallback: when the localized tail matches no specific class, emit a hedged remedy that points the user at their own OS error and offers both branches (install runtime / --repair-fts) without prescribing the wrong single fix. Precise en/zh 126 keeps its definite remedy. * feat(fts): language-independent structural classifier via binary inspection (#2374) Add diagnoseExtensionLoad: pull the extension's file path out of lbug's own English wrapper and inspect the binary header (PE/ELF/Mach-O magic + arch) directly, so corrupt-vs-valid is decided by the file itself, not the localized OS-error tail. A valid binary that still failed to load ⇒ missing_dependency (runtime dep), decided in any OS display language and on all three platforms. Falls back to the string classifier (with its hedged fallback) when the file can't be read. Wire all four surfaces to it. Event Viewer / GetLastError-via-FFI were dead ends (lbug catches the failure — no crash event; no native FFI dep). * test(fts): exercise the structural classifier on real binaries (#2374) Add an integration suite that runs inspectExtensionBinary/diagnoseExtensionLoad against genuine binaries — the running node executable, the real lbugjs.node addon, and the installed FTS extension (valid); a truncated real binary and a real text file (corrupt). Registered in cross-platform-tests PLATFORM_LOGIC so it runs on the Windows + macOS matrix, proving the PE and Mach-O header parsing on real PE/Mach-O files (ubuntu covers ELF). * fix(fts): honor a corrupt_file verdict over a structurally-valid header (#2374) The structural probe in diagnoseExtensionLoad inspects only the first 4 KB, so a download truncated after its header reads 'valid' and was routed to the "install VC++, reinstalling will NOT help" remedy — the exact loop #2374 exists to kill, for the truncated-download case the module docstring claims it handles. Honor the loader's own corruption report ("file too short" / Windows error 193 "not a valid Win32 application") before defaulting to the dependency remedy; localized corrupt tails stay hedged missing_dependency, preserving language-independence. Addresses PR #2383 review finding F1. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(fts): return indeterminate for a PE header beyond the read window (#2374) The structural probe reads only BINARY_HEADER_BYTES (4 KB). A valid PE with a large DOS stub whose e_lfanew points past that window was wrongly called 'corrupt', routing a fine DLL to "re-download". A garbage e_lfanew from a truly corrupt file is indistinguishable from here, so widen the header verdict with 'indeterminate' and return it in that case; the caller then defers to the loader's own report instead of asserting a false verdict. Fat Mach-O stays valid (LadybugDB ships thin per-arch binaries). Also covers the unmapped-arch and garbage-PE-signature branches. Addresses PR #2383 review finding F1-secondary. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(fts): drop contradictory reinstall guidance from the analyze degrade log (#2374) For a missing runtime dependency the extension file is present, so appending FTS_UNAVAILABLE_MESSAGE (which tells the user to install it "with network access") to the remedy ("reinstalling will NOT help") produced self-contradictory guidance on the main analyze surface. Lead the missing_dependency degrade log with the class-neutral sentence (FTS_UNAVAILABLE_LEAD) and append only the classified remedy; other classes keep FTS_UNAVAILABLE_MESSAGE unchanged. Addresses PR #2383 review finding F2. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(fts): cache the load diagnosis so the degraded warning does no per-request I/O (#2374) ftsDegradedWarning() runs on every degraded /api/search response and MCP query, and it was calling diagnoseExtensionLoad — a synchronous openSync/readSync of the extension file — on every call. Compute the diagnosis once at mark-unavailable time (the single load-failure sink, run per Database not per request), cache it on ExtensionCapability, and have the warning read the cached result (falling back to the pure, no-I/O string classifier if it is absent). Loader capability-shape assertions relax from toEqual to toMatchObject for the new optional field. Addresses PR #2383 review finding F3. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(fts): cover the missing_dependency remedy on the --repair-fts path (#2374) The repair-fts error interpolates the classified remedy, but no test reached the missing_dependency branch — only the corrupt/invalid-ELF path. Add a Windows error-126 case asserting the thrown error carries the VC++ redistributable remedy and omits the old "retry the network install" tail, and that no index is dropped. Addresses PR #2383 review finding F6a (--repair-fts surface). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(fts): share the VC++ redistributable install hint (#2374) The Microsoft Visual C++ redistributable name and aka.ms URL were duplicated verbatim in WINDOWS_MISSING_DEPENDENCY_REMEDY and STRUCTURAL_MISSING_DEPENDENCY_REMEDY. Factor a single VC_REDIST_INSTALL_HINT constant so the pointer cannot drift between them; the composed remedy strings are byte-identical (existing exact-text assertions unchanged). Also adds a test covering the previously-unexercised structural remedy branch. Addresses PR #2383 review finding F5a. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(fts): guard FILE_CORRUPTION_SIGNATURES parity with the installer script (#2374) The corruption-signature list is deliberately duplicated between extension-load-error.ts and scripts/install-duckdb-extension.mjs (the .mjs cannot import the .ts), with nothing guarding against drift — a one-sided edit would desync the FORCE-INSTALL verb from remedy classification. Export the array from both and add a parity test that compares regex source + flags element-wise. Addresses PR #2383 review finding F5b. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(test): run extension-binary-real in the sequential lbug-db vitest project (#2374) extension-binary-real.test.ts imports @ladybugdb/core but ran in the parallel `default` project, contrary to TESTING.md's rule that native-LadybugDB tests live in the sequential `lbug-db` project. Add it to the lbug-db include list and the default exclude list; it now runs under lbug-db and no longer under default. Addresses PR #2383 review finding F6c. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(fts): fail loud, not silent-skip, on missing FTS artifacts under REQUIRE_FTS=1 (#2374) The real-binary structural tests gated on raw .skipIf(!lbugNative) / .skipIf(!installedFts), so under GITNEXUS_REQUIRE_FTS=1 a missing artifact would silently vanish from a green CI run (the #2299 trap). These tests inspect the extension file directly and need its path, not a loaded connection — so skipUnlessFtsAvailable (which needs an initialized LadybugDB) does not fit. Add requireFtsResourceOrSkip: skip gracefully offline, throw under REQUIRE_FTS=1. The always-on process.execPath assertion still runs everywhere. Addresses PR #2383 review finding F6d. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * style(fts): apply prettier formatting to the #2383 fix files (#2374) Line-wrapping only; the quality/format CI check flagged three files. No behavior change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Gergő Magyar
|
177bbc89c3
|
fix: surface real FTS extension LOAD errors and self-heal broken extension files (#2374) (#2375) | ||
|
Gergő Magyar
|
cdad478c96
|
fix: proxy-blocked installs survive onnxruntime-node postinstall and self-heal embeddings (#2370) (#2372)
Some checks are pending
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Scorecard / Scorecard analysis (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
|
||
|
Gergő Magyar
|
6252aa745f
|
feat(setup): add CodeBuddy and Qoder coding-agent integrations (#2368)
* feat(setup): add CodeBuddy and Qoder coding-agent integrations Adds Tencent CodeBuddy and Alibaba Qoder to gitnexus setup/uninstall, fitted to the editor-targets registry and --coding-agent selection. - CodeBuddy: MCP entry written into the first existing file of its documented priority chain (~/.codebuddy/.mcp.json recommended, ~/.codebuddy/mcp.json deprecated, ~/.codebuddy.json legacy) so a populated deprecated config is never shadowed; skills to ~/.codebuddy/skills/ (https://www.codebuddy.ai/docs/cli/mcp) - Qoder: MCP entry in ~/.qoder.json, skills to ~/.qoder/skills/ (https://docs.qoder.com/cli/using-cli, /extensions/skills) - editor-targets gains optional legacyFiles; uninstall sweeps them - roster strings updated (CLI help, i18n en/zh-CN, READMEs); en/zh-CN setup descriptions were stale (missing Antigravity) and are refreshed Supersedes and credits PR #1030 by @zykai0302, re-fitted to the post-#2168 selective-agent architecture with documented config paths. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(cli): assert stable zh-CN setup-description fragment * fix(setup): surface non-ENOENT config read/stat failures instead of clobbering * fix(setup): report corrupt legacy MCP files informationally during uninstall * test(setup): cover multi-candidate uninstall sweep combinations * fix(setup): detect CodeBuddy/Qoder installs via existing MCP config files * fix(setup): skip empty and non-file candidates in the MCP config chain * docs: add CodeBuddy and Qoder manual MCP configuration sections * test(ci): run the setup-uninstall round-trip in the cross-platform matrix * fix(setup): never claim "not configured" when uninstall recorded errors * refactor(cli): share the isEnoent predicate via editor-targets * refactor(setup): share chain-file install detection between CodeBuddy and Qoder --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
ACT900
|
5aada28da5
|
fix(embeddings): use system-matched onnxruntime-node CUDA build so CUDA 13 hosts use the GPU (#2341)
* fix(embeddings): use system-matched onnxruntime-node CUDA build so CUDA 13 hosts use the GPU
transformers.js exact-pins a CUDA-12 onnxruntime-node while gitnexus' own dep floats to a CUDA-13 build. npm/pnpm cannot dedupe an exact pin against a range, so npm i -g installs two copies and the gitnexus overrides block (root-only) is inert. On a CUDA-13-only host the nested CUDA-12 provider cannot load libcublasLt.so.12, the CUDA EP fails, and embeddings silently fall back to CPU (isCudaAvailable() also only probed .so.12).
Add onnxruntime-node-resolver.ts (module.registerHooks redirect to the host-matching build, no-op elsewhere) mirroring onnxruntime-common-resolver.ts; probe libcublasLt .so.12 OR .so.13 against the copy that actually loads; unit test with 12 cases.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(embeddings): wire CUDA-13 build-match resolver into MCP query embedder
The MCP query-time embedder (src/mcp/core/embedder.ts) has its own,
separate initEmbedder() used for semantic search — it only called
ensureOnnxRuntimeCommonResolvable() before importing transformers.js, so
the CUDA-13 build-matching redirect added for the analyze/CLI embedder
never applied here. A CUDA-13 host running MCP search with
--embedding-device cuda still loaded the mismatched default onnxruntime-node
build.
Wire ensureOnnxRuntimeNodeMatchesSystem() into the same call site, mirroring
the core embedder's ordering (registered after the common-resolver fallback,
before the dynamic transformers import).
* fix(embeddings): gate CUDA redirect decision on registerHooks availability
decide() computed the CUDA-major redirect independent of whether Node's
module.registerHooks API actually exists — only ensureOnnxRuntimeNodeMatchesSystem()
checked that. On Node 22.0-22.14 (allowed by this package's engines
floor; registerHooks needs >=22.15), isCudaAvailable() could therefore
report a redirect target that ensureOnnxRuntimeNodeMatchesSystem() then
silently failed to install, so transformers.js loaded the mismatched
default onnxruntime-node build while the embedder still requested
device:'cuda' against it — reintroducing the uncatchable native crash
this probe exists to prevent.
Move the registerHooks check to the top of decide() so the probe and the
loader can never disagree, and skip CUDA-major probing entirely in that
case (a redirect could never install anyway).
Also fixes a related test-helper bug found while writing this unit's
tests: loadResolver's destructuring default (`registerHooks = vi.fn()`)
silently substituted a real mock function even when a test passed
`registerHooks: undefined` to simulate Node < 22.15 — meaning the
existing 'no-ops... when registerHooks is unavailable' test never
actually exercised that path. Distinguish 'omitted' from 'explicitly
undefined' via an 'in' check.
* test(embeddings): drive decide() -> redirect:true and assert the resolve() closure
The PR's actual shipped behavior — the installed registerHooks resolve()
closure, and the full redirect-active decision path — had zero executed
test coverage. All 4 prior ensureOnnxRuntimeNodeMatchesSystem tests avoided
driving decide() into redirect:true because require.resolve/createRequire
were never mocked, so the two-distinct-directory comparison decide()
depends on always resolved against whatever's actually installed in this
test's real node_modules (a single real copy, not the PR's two-copy
scenario).
Extend loadResolver()'s existing node:module mock to also fake createRequire,
keyed by call origin, so resolveOurOrtNodeDir/resolveDefaultOrtNodeDir can be
driven to two distinct fake directories with distinct CUDA majors — reaching
redirect:true without adding any injection points to production code. Then
capture the installed resolve() closure (mirroring the sibling
onnxruntime-common-resolver.test.ts's captureResolve() pattern) and assert
its three branches directly: onnxruntime-node redirect, onnxruntime-common
redirect, and passthrough for any other specifier.
* fix(embeddings): distinguish ldd detection-failure from no-CUDA-provider
ortCudaMajor treated any execFileSync('ldd', ...) failure with no usable
stdout (missing ldd binary, permission-denied .so, sandboxed exec)
identically to 'CUDA provider genuinely absent'. The pre-PR detection
(hasOrtCudaProvider) only used existsSync, never ldd, so this is a
regression: a CUDA-12 host that worked fine before this PR can now
silently fall back to CPU if ldd itself can't run, even though the
provider .so and system CUDA libs are both genuinely present.
readSoNeeded now reports whether ldd produced any usable output at all,
distinct from 'ldd ran and just found no matching NEEDED entry' (the
existing, already-handled '=> not found' case). When detection genuinely
fails, log a warning so an operator can tell 'CPU fallback because
detection itself failed' apart from 'CPU fallback because no CUDA build
shipped' — the return value stays null either way (the type can't
distinguish a third state), but the two cases are now observably
different via the log.
* fix(embeddings): check ourDir independently of whether defaultDir resolved
decide()'s ourDir fallback lookup was nested inside
'if (systemMajor != null && defaultDir)', so a null defaultDir (transformers'
own onnxruntime-node resolution failing outright, e.g. a partial/broken
install) skipped checking ourDir entirely — getEffectiveOnnxRuntimeNodeDir()
returned null even when gitnexus' own matching CUDA-13 copy would have
resolved fine and worked.
defaultDir resolving is not a precondition for the comparison: an
unresolvable default already counts as 'the default doesn't match', so the
ourDir check now runs whenever systemMajor is known, regardless of whether
defaultDir resolved.
* fix(embeddings): prefer CUDA 13 globally across the env-var directory scan
detectSystemCudaMajor's CUDA_PATH/LD_LIBRARY_PATH scan returned on the
first CUDA-major match within a single dir/sub pair, so a stale .so.12
found early (e.g. a leftover CUDA_PATH entry from a prior install) shadowed
a genuine .so.13 found later in the search path, even though the scan's
own ordering (checking 13 before 12 within each pair) was clearly intended
to prefer 13 wherever possible.
Keep scanning the full search space once a 12 is found, only returning
early once a 13 is found (the best possible answer) or the space is
exhausted.
* fix(embeddings): have onnxruntime-common-resolver defer to the effective onnxruntime-node dir
onnxruntime-common-resolver.ts independently re-derived transformers'
default onnxruntime-node dir (its own copy of the 'resolve transformers'
main entry, then onnxruntime-node' walk) to compute which onnxruntime-common
to pair with — duplicating onnxruntime-node-resolver.ts's own walk, and
capable of disagreeing with it: when the CUDA-major redirect is active,
this hook would still pair onnxruntime-common with transformers' default
(unredirected) onnxruntime-node, not the redirected copy the other hook
just switched onnxruntime-node itself to.
Have it call the already-exported getEffectiveOnnxRuntimeNodeDir() instead
— the same decision the CUDA-major redirect hook uses — so both hooks
always agree on which onnxruntime-node they're pairing onnxruntime-common
against, and the duplicated resolve-walk is removed entirely rather than
merely factored out.
* fix(embeddings): cache the effective CUDA major to remove redundant subprocess spawns
isCudaAvailable() in embedder.ts re-invoked ortCudaMajor/detectSystemCudaMajor
directly even though decide() (via getEffectiveOnnxRuntimeNodeDir) had
already computed both to make its redirect decision — a second, wasted
ldconfig + up to 2 ldd spawns on every initEmbedder() call.
Add effectiveMajor to the memoized Decision, computed once inside decide()
alongside effectiveDir/systemMajor, and export a single
isEffectiveCudaAvailable() that reads straight from the cached decision.
embedder.ts's local isCudaAvailable() wrapper (and its now-unused
getEffectiveOnnxRuntimeNodeDir/ortCudaMajor/detectSystemCudaMajor imports)
is replaced by this one exported function.
* fix(embeddings): surface CUDA redirect state at info level and in doctor
A successful CUDA-build redirect logged only at logger.debug (filtered
by the default 'info' level), and gitnexus doctor's embeddings section
never mentioned the redirect at all — leaving no diagnostic path for
'why is my CUDA-13 host still on CPU' after this PR ships.
Log the successful-redirect line at info (no-redirect/failure paths stay
at debug, since those are the common, expected case). Add
cudaRedirectDoctorStatus(), a pure summary of decide()'s already-computed
decision mirroring doctor.ts's existing localEmbeddingDoctorStatus shape,
and print it as a new literal (non-i18n) 'CUDA:' line in doctor's
embeddings section alongside the existing 'Support:' line, matching that
line's established convention.
* test(embeddings): register onnxruntime-node-resolver.test.ts in the cross-platform subset
The new test file guards on process.platform (linux/darwin cases) but was
absent from cross-platform-tests.ts's PLATFORM_LOGIC list, which
TESTING.md says platform-sensitive tests should be added to — so it never
ran on the Windows/macOS CI matrix, only Ubuntu.
Note: the sibling onnxruntime-common-resolver.test.ts has the identical,
pre-existing gap (it predates this PR) — left as-is here, since fixing
unrelated pre-existing test-registration debt is out of scope for this
PR's own follow-up fixes.
* test(embeddings): strengthen weak assertions, add garbled-output and CUDA_PATH coverage
Three of the four ensureOnnxRuntimeNodeMatchesSystem tests only asserted
'doesn't throw' rather than a concrete outcome — including one literally
named 'idempotent' that never asserted a call count on its own spy.
Strengthen each to assert real outcomes (module stays functional after a
no-op; spy call counts; return-value shape), while keeping the true
install-once idempotency proof in the redirect-active test added earlier
(this file's no-redirect scenario can't exercise it, since registerHooks
is never called either way).
Add the missing edge cases flagged in review: a CUDA_PATH-only fallback
scan test (mirroring the existing LD_LIBRARY_PATH one), and garbled/
unrecognized ldconfig and ldd output cases for both detectSystemCudaMajor
and ortCudaMajor, confirming neither falsely matches a CUDA major on
unparseable input. Also parameterize the non-linux platform test across
both darwin and win32 rather than darwin alone.
Not changed: the process.env reassignment vs. Object.defineProperty
'inconsistency' flagged in review — process.env, unlike process.platform,
has no getter-only restriction, so plain reassignment is already correct
and switching it to Object.defineProperty would be unnecessary ceremony.
* docs(embeddings): note the npm link/symlinked dev-checkout resolution caveat
resolveOurOrtNodeDir/resolveDefaultOrtNodeDir anchor to this module's own
real (post-symlink) location via import.meta.url, so a linked local dev
checkout may resolve against its own node_modules rather than the
consuming app's. Narrow, dev-only blast radius (regular npm/pnpm installs
are unaffected) — document-only, no structural fix warranted.
* fix(test): point the windowsHide spawn-family registry at the file that actually spawns
hooks.test.ts's windowsHide regression check still listed
gitnexus/src/core/embeddings/embedder.ts as a child_process-spawning
file, but this PR itself already moved all execFileSync usage out of
embedder.ts and into the new onnxruntime-node-resolver.ts — without
updating this registry. The check was silently failing at the PR's own
head commit (confirmed: 0 spawn-family calls found in embedder.ts,
'expected 0 to be greater than 0'), a pre-existing gap this fix-pass
surfaced via a full-suite run rather than something introduced by any of
the preceding follow-up commits.
Swap the registry entry to onnxruntime-node-resolver.ts, which does
import execFileSync (ldd + ldconfig, both already correctly passing
windowsHide: true).
* fix(test): make onnxruntime-node-resolver.test.ts path comparisons OS-agnostic
Registering this file in cross-platform-tests.ts's PLATFORM_LOGIC (a
prior commit in this series) means it now runs on the Windows CI matrix,
not just Ubuntu — and several of the fakeDirs-based tests (redirect:true,
ourDir-independent, subprocess-count, doctor-status) compared the
resolver's real join()/dirname() output against hardcoded forward-slash
fixture strings via exact-match or .startsWith().
Node's module is bound to path.win32 (or path.posix) based on the
REAL host OS at process start — stubbing process.platform later, as these
tests already do for the resolver's own platform branching, has no effect
on it. So on a genuine Windows runner, join(effectiveDir, 'package.json')
backslash-normalizes even under a faked platform:'linux', silently
breaking every forward-slash comparison in this file: the createRequire
dispatch would route to the wrong fake require, throw MODULE_NOT_FOUND,
get swallowed by ensureOnnxRuntimeNodeMatchesSystem's outer try/catch, and
registerHooks would never fire — the redirect-active tests would fail
outright on Windows CI.
Normalize every comparison point (the createRequire dispatcher, and the
shared execFileSync/existsSync mocks) with a single toPosix() helper.
Added a forceWin32Path test option (using path.win32's real join/dirname
behavior) to prove this holds without needing an actual Windows runner —
confirmed by temporarily reverting the fix and observing the new test
fail with the exact predicted mismatch before restoring it.
* chore(autofix): apply prettier + eslint fixes via /autofix command
* fix(embeddings): keep CUDA auto-detect working on Node < 22.15 when the default build already matches
The registerHooks guard in decide() returned effectiveMajor: null
unconditionally, so on Node 22.0-22.14 / 23.0-23.4 (engines floor is
>=22.0.0) isEffectiveCudaAvailable() was always false and a CUDA-12 host
whose default onnxruntime-node build already matched — which needs no
hook at all to use the GPU — silently regressed from CUDA to CPU on the
auto device path (pre-PR isCudaAvailable() behavior).
Probe the system and the default copy regardless of registerHooks
availability; only the ourDir redirect branch stays gated on it, so the
probe still never reports a redirect target that cannot be installed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Gergő Magyar <gergomagyar@icloud.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
|
||
|
Malik
|
859e4b75a4
|
fix(cli): --limit i18n, 0/negative guard, and correct truncation paths (#2310)
Some checks are pending
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Scorecard / Scorecard analysis (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
* fix: add --limit i18n, negative guard, correct property paths, and zh-CN translations - Add i18n keys for context/impact/cypher/detect-changes --limit options - Add zh-CN translations for all 4 --limit option descriptions - Add Math.max(0, parseInt()) guard to prevent negative --limit - Fix ALL property path mismatches discovered by audit: - context: callers/callees → incoming.calls/outgoing.calls+accesses - impact: upstream/downstream → affected_processes/affected_modules/byDepth - cypher: rows → row_count cap (rows embedded in markdown string) - detect-changes: affected_flows → affected_processes - Change query command from required to optional positional arg with -q alias - Update @ladybugdb/core from ^0.16.1 to ^0.17.1 - Update typescript from ^5.4.5 to ^5.9.3 * test: add E2E tests for --limit flag across all 5 CLI commands Tests context, impact, cypher, detect-changes, and query with --limit 1, baseline comparison, and --limit 0 (falsy/no-op). detect-changes output is formatted text (not JSON), so those tests count symbol lines matching 'Type name -> filePath' pattern. 14 tests, all passing. No regressions in 6455 existing tests. * fix: address Copilot review feedback on --limit guards - Add Math.max(0, ...) guard to queryCommand limit parsing - Change if(limit) to if(limit !== undefined) in all 5 commands (prevents --limit 0 from being treated as falsy/no-op) - Make queryText parameter optional (Commander may pass undefined) - Fix usage error strings: --search to -q, --query (en + zh-CN) * chore(autofix): apply prettier + eslint fixes via /autofix command * fix(cli): centralize --limit parsing, slice cypher markdown, fix usage text Address PR review feedback on --limit handling: - Add a shared parseLimit() helper (Number.isInteger(n) && n > 0), used by all 5 tool commands. Non-numeric / 0 / negative --limit now means "no limit" instead of the `options.limit ? Math.max(0, parseInt(...)) : undefined` path, where a string like "abc" is truthy and yields NaN -> slice(0, NaN) -> the guardrail commands (impact/context/detect-changes) silently emptied results with exit 0. - cypher: slice the markdown table to --limit data rows so the reported row_count matches what is actually printed (was capping row_count while printing every row). - Fix query usage string: [search_query] (optional positional) and `--query <text>` invocation form, not the option-definition `-q, --query <search_query>` syntax (en + zh-CN). - Add an E2E regression test for non-numeric --limit. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(cli): escape newlines in cypher markdown cells A multi-line cell value (e.g. a symbol's `content`) was rendered with raw newlines via String(v), so one logical row spanned multiple physical lines. That corrupts the markdown table and breaks `cypher --limit`'s line-based slice (it kept the wrong number of rows, often zero, while row_count over-claimed). Collapse newlines in formatCypherAsMarkdown so one physical line == one row; the existing CLI slice is now correct and the pre-existing un---limited corruption is fixed too. (#2310 review) * test(cli): de-vacuum the --limit truncation tests The truncation it()s used the repo-banned vacuous-pass pattern (early-return on status===null, assertions guarded by if(Array.isArray), bounds-only toBeLessThanOrEqual — DoD.md:82) against `validateInput`, which has only 1 caller, so context/impact/query --limit 1 compared 1>=1 and stayed green even if the slice were deleted. Rewrite with unconditional, exact assertions and target `logMessage` (2 callers, 4 processes) so the no-limit baseline truly exceeds the limit; detect-changes now mutates two real function bodies (two changed symbols). Adds a multi-line-cell cypher --limit regression. (#2310) * test(ci): run cli-limit-e2e in the cross-platform matrix The --limit E2E suite spawns the real CLI (child_process) but was not in SPAWN_CLI, so it ran only on Ubuntu — the cross-platform check only fails on listed-but-missing files, not the reverse (TESTING.md §Cross-platform). Register it so the --limit regression guard also runs on Windows/macOS, where path separators, CRLF and the formatted-output arrow differ. (#2310) * fix(cli): document impact --limit affected-list cap, drop dead byDepth re-slice `impact --limit` also caps affected_processes/modules, but the help only mentioned the per-depth cap — so JSON consumers reading the affected lists got a silently-truncated array. Update en + zh-CN + the command description to say so. Also remove the client-side byDepth re-slice: the backend already paginates byDepth to the same limit (paginationLimit = clamp(limit,1,10000), offset applied backend-side), so the client slice was a guaranteed no-op. (#2310) * fix(cli): reconcile detect-changes --limit summary, list, and overflow formatDetectChangesResult computed the "... and N more" overflow from the already---limit-sliced array length, so under `--limit` the header (true summary total), the listed rows, and the marker disagreed — e.g. "2 symbols" in the header but a list of 1 with no marker. Base the overflow on the true summary.changed_count / affected_count instead, and add the same marker to the affected-processes list, so header + list + marker stay consistent. (#2310) * feat(cli): add -l shorthand to impact --limit The PR added the -l alias to context/cypher/detect-changes but left impact on the long --limit only, so `impact -l 5` errored while `context -l 5` worked. Add -l for parity and update the help-i18n OPTION_DESCRIPTION_KEYS key to the new `-l, --limit <n>` flag string so the description still resolves. (#2310) * fix(cli): bound all context --limit array categories context --limit sliced only incoming.calls / outgoing.calls / outgoing.accesses / processes, leaving the other relType buckets unbounded — notably incoming.accesses (bounded on outgoing but not incoming) plus imports/extends/ uses/… and typed_properties. Replace the hardcoded slices with a generic loop over every array-valued bucket under incoming/outgoing, plus typed_properties and processes, so --limit caps the whole context payload. (#2310) * refactor(cli): parse --offset with a parseLimit-style helper impactCommand parsed --offset with the legacy parseInt/Number.isFinite idiom while --limit had moved to parseLimit, leaving two parsing styles side by side. Add a sibling parseOffset helper (non-negative — offset 0 is valid) and use it, so both options share one idiom; as a bonus it now rejects negative/fractional offsets instead of forwarding them to the backend. (#2310) --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Gergo Magyar <gergomagyar@icloud.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |