mirror of
https://github.com/abhigyanpatwari/GitNexus.git
synced 2026-09-22 00:31:17 +00:00
1272 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
dependabot[bot]
|
4d9318e2ee
|
chore(deps)(deps): bump hono from 4.12.23 to 4.12.26 in /gitnexus (#2244)
Bumps [hono](https://github.com/honojs/hono) from 4.12.23 to 4.12.26. - [Release notes](https://github.com/honojs/hono/releases) - [Commits](https://github.com/honojs/hono/compare/v4.12.23...v4.12.26) --- updated-dependencies: - dependency-name: hono dependency-version: 4.12.26 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
Parafee41
|
e33f908775
|
ci: keep tree-sitter readiness summary counts current (#2196) | ||
|
Parafee41
|
78e5ff3b9b
|
fix(wiki): keep graph DB pinned during generation (#2232) | ||
|
azizur100389
|
72876ab69a
|
fix(cpp): rank homogeneous braced-init overloads (#2214)
Some checks failed
CodeQL / Analyze (javascript-typescript) (push) Has been cancelled
CodeQL / Analyze (python) (push) Has been cancelled
Gitleaks / gitleaks (push) Has been cancelled
Publish / Classify release event (push) Has been cancelled
Scorecard / Scorecard analysis (push) Has been cancelled
Trivy Image Scan / Trivy (gitnexus-cli) (push) Has been cancelled
Trivy Image Scan / Trivy (gitnexus-web) (push) Has been cancelled
Publish / RC guard (marker + release-PR skip) (push) Has been cancelled
Publish / ci (push) Has been cancelled
Publish / Publish to npm (push) Has been cancelled
Publish / Build & Push RC Docker images (push) Has been cancelled
|
||
|
Gergő Magyar
|
b895a20415
|
perf(lbug): overlap node COPY with relationship emit (#2203) (#2226)
* test(lbug): lock PARALLEL=false as a tested correctness invariant (#2203)
The parallel CSV reader (Kuzu-derived, default PARALLEL=true) cannot parse
quoted fields with embedded newlines (kuzudb/kuzu#5778); our content/text
columns hold source code, so PARALLEL=false is mandatory for correctness.
Add a live-DB multiline-quoted round-trip that fails if it is ever flipped,
plus a static guard on the generated COPY queries. Export COPY_CSV_OPTS /
getCopyQuery for the static assertion; document the invariant at the source.
* feat(lbug): expose node/rel phase boundary via onNodePhaseComplete hook (#2203)
streamAllCSVsToDisk now fires an optional onNodePhaseComplete(nodeFiles)
callback right after node CSVs are flushed and before the relationship pass
writes any rel_*.csv — the boundary the COPY-overlap leg needs. The node-file
manifest construction is hoisted above the rel pass and reused in the return,
so output is byte-for-byte identical when no callback is supplied (verified by
the emit bench fingerprint and the splitRelCsvByLabelPair differential oracle).
The callback is not awaited, so the rel pass runs concurrently with the
caller's node COPY.
* perf(lbug): overlap node COPY with relationship emit (#2203)
The deferred parallelism leg of #2203. LadybugDB is single-writer and its
parallel CSV reader is unsafe for our multiline content (kuzudb/kuzu#5778), so
the only safe parallelism is pipeline-overlap: node COPY (uses conn, never the
rel files) runs concurrently with the relationship emit pass (writes rel_*.csv,
never conn). Node COPY now starts at streamAllCSVsToDisk's onNodePhaseComplete
boundary while the rel pass keeps writing; the relationship COPY still waits for
node COPY (FK precondition), so DB load order and content are unchanged.
- Extract copyNodeCSVs; start it in the hook (overlap) or after emit (serial).
- GITNEXUS_SERIAL_LBUG_LOAD=1 forces the legacy strictly-sequential path
(operator escape hatch + differential-test oracle).
- Settle the in-flight node-COPY promise on emit failure (no unhandled
rejection); rethrow node-COPY errors at the FK barrier.
- Preserve the PDG manifest merge + collision guards (node merge at the hook,
rel merge before rel COPY) and all retry/fallback/cleanup behavior.
- PROF_LBUG_LOAD gains mode=overlap|serial; copy-nodes becomes the residual
node-COPY time after emit (trends to 0 as overlap hides it).
* test(lbug): differential gate — overlap load === serial load (#2203)
Loads one fixture (multiple node tables, multiple edge pairs, multiline File
content + BasicBlock text) into two fresh DBs — once via the default node-COPY
‖ rel-emit overlap, once via GITNEXUS_SERIAL_LBUG_LOAD=1 — and asserts the two
databases are content-equivalent: identical per-table node counts, per-type
edge counts, byte-for-byte multiline content/text, and identical
insertedRels/skippedRels/warnings. This is the issue's byte-identical-content
acceptance gate for the parallelism leg.
* fix(review): apply autofix feedback
- csv-generator: onNodePhaseComplete doc-contract now matches reality (a sync
throw is allowed and is how loadGraphToLbug surfaces the manifest collision
guard) — drops the inaccurate 'must not throw synchronously' line.
- lbug-adapter: copyNodeCSVs totalSteps is the node-table count (drop the +1
rel-step holdover; the rel COPY has its own progress line).
- lbug-adapter: on emit+node-COPY double-failure, log the swallowed node-COPY
error before rethrowing the emit error (diagnosability).
- lbug-load-prof test: assert mode=overlap on the default path.
* fix(test): use mkdtemp for secure temp dirs (CodeQL js/insecure-temporary-file)
CodeQL flagged lbug-load-overlap.test.ts writing a file into a predictable
os.tmpdir() path. Create the base temp dir with fs.mkdtemp (atomic, random
suffix) in both new live-DB tests, and switch to the gitnexus-lbug- prefix that
TEST_FIXTURE_PREFIXES recognizes so the Windows stale-sidecar sweep covers
these fixtures.
* fix(lbug): check PDG manifest rel-pair collision before node COPY (#2203)
Found by Codex in tri-review. The manifest rel-pair collision guard ran after
the FK barrier (after node COPY committed), so on that should-never-happen
error branch the overlap path left orphan node rows AND the
GITNEXUS_SERIAL_LBUG_LOAD escape hatch diverged from the legacy 'validate
manifest before any COPY' behavior. Move the rel merge + collision check ahead
of beginNodeCopy/the barrier: the serial path now detects a collision before
committing any node rows (legacy parity restored — the escape hatch is a
faithful oracle again), and the overlap path detects it as early as csvResult
is available. The node-collision guard already ran before node COPY (in the
hook).
* test(lbug): cover rel-emit failure with node COPY in flight (#2203)
Resolves a P1 review gap on PR #2226: the overlap's catch(emitErr) branch
(settle the in-flight node-COPY promise, then rethrow the emit error) was
untested. Fault-injects via a vi.mock of streamAllCSVsToDisk that fires
onNodePhaseComplete (starting a real node COPY on a live DB) then throws,
asserting loadGraphToLbug rejects with the emit error and no unhandled
rejection leaks. Also covers the both-fail case (node COPY error is logged,
emit error still wins). Listener removed in finally; macrotask queue flushed
before the assertion so it can't pass vacuously.
* test(lbug): cover node-COPY hard-failure rethrow at the FK barrier (#2203)
Resolves the second P1 review gap on PR #2226. Mocks emit to fire
onNodePhaseComplete with a nodeFiles entry pointing at a missing CSV (a
bind-time COPY error that IGNORE_ERRORS does not suppress) and otherwise
succeed, so copyNodeCSVs throws, the error is captured in nodeCopyError, and
loadGraphToLbug rethrows it at the FK barrier — asserted via rejects /COPY
failed for File/.
* test(lbug): cover PDG manifest rel-pair collision in overlap + serial (#2203)
Resolves the P2 gap behind the Codex tri-review finding: the manifest rel-pair
collision guard (moved ahead of node COPY in
|
||
|
azizur100389
|
ff0124e067
|
feat(cpp): parse CUDA source extensions (#2213)
* feat(cpp): parse CUDA source extensions * test(cpp): characterize CUDA parser limitations --------- Co-authored-by: Gergő Magyar <gergomagyar@icloud.com> |
||
|
dependabot[bot]
|
4c7e0661f1
|
chore(deps)(deps): bump the npm_and_yarn group across 1 directory with 3 updates (#2220)
Bumps the npm_and_yarn group with 3 updates in the /gitnexus-web directory: [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite), [@babel/core](https://github.com/babel/babel/tree/HEAD/packages/babel-core) and [form-data](https://github.com/form-data/form-data). Updates `vite` from 8.0.11 to 8.0.16 - [Release notes](https://github.com/vitejs/vite/releases) - [Changelog](https://github.com/vitejs/vite/blob/main/packages/vite/CHANGELOG.md) - [Commits](https://github.com/vitejs/vite/commits/v8.0.16/packages/vite) Updates `@babel/core` from 7.29.0 to 7.29.7 - [Release notes](https://github.com/babel/babel/releases) - [Changelog](https://github.com/babel/babel/blob/main/CHANGELOG.md) - [Commits](https://github.com/babel/babel/commits/v7.29.7/packages/babel-core) Updates `form-data` from 4.0.5 to 4.0.6 - [Release notes](https://github.com/form-data/form-data/releases) - [Changelog](https://github.com/form-data/form-data/blob/master/CHANGELOG.md) - [Commits](https://github.com/form-data/form-data/compare/v4.0.5...v4.0.6) --- updated-dependencies: - dependency-name: vite dependency-version: 8.0.16 dependency-type: direct:development dependency-group: npm_and_yarn - dependency-name: "@babel/core" dependency-version: 7.29.7 dependency-type: indirect dependency-group: npm_and_yarn - dependency-name: form-data dependency-version: 4.0.6 dependency-type: indirect dependency-group: npm_and_yarn ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Abhigyan Patwari <126312502+abhigyanpatwari@users.noreply.github.com> Co-authored-by: Gergő Magyar <gergomagyar@icloud.com> |
||
|
Gergő Magyar
|
7d12ea8fd9
|
feat(analyze): private GitHub repos via PAT + Azure DevOps Server support (#2076, #2210) (#2223) | ||
|
Gergő Magyar
|
3c82361b66
|
perf(cfg): streaming/chunked PDG graph emit for full-kernel-scale repos (#2202) (#2216) | ||
|
dependabot[bot]
|
067c73b6b2
|
chore(deps)(deps-dev): bump @types/node in /gitnexus (#2222)
Bumps [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node) from 25.9.2 to 25.9.3. - [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases) - [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/node) --- updated-dependencies: - dependency-name: "@types/node" dependency-version: 25.9.3 dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
dependabot[bot]
|
526194bf42
|
chore(deps)(deps): bump protobufjs from 7.5.8 to 7.6.4 in /gitnexus (#2219)
Bumps [protobufjs](https://github.com/protobufjs/protobuf.js) from 7.5.8 to 7.6.4. - [Release notes](https://github.com/protobufjs/protobuf.js/releases) - [Changelog](https://github.com/protobufjs/protobuf.js/blob/protobufjs-v7.6.4/CHANGELOG.md) - [Commits](https://github.com/protobufjs/protobuf.js/compare/protobufjs-v7.5.8...protobufjs-v7.6.4) --- updated-dependencies: - dependency-name: protobufjs dependency-version: 7.6.4 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
dependabot[bot]
|
fbbda5a19b
|
chore(deps)(deps): bump tar from 7.5.13 to 7.5.16 in /gitnexus (#2218)
Bumps [tar](https://github.com/isaacs/node-tar) from 7.5.13 to 7.5.16. - [Release notes](https://github.com/isaacs/node-tar/releases) - [Changelog](https://github.com/isaacs/node-tar/blob/main/CHANGELOG.md) - [Commits](https://github.com/isaacs/node-tar/compare/v7.5.13...v7.5.16) --- updated-dependencies: - dependency-name: tar dependency-version: 7.5.16 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
dependabot[bot]
|
2d389a7d1b
|
chore(deps-dev): bump js-yaml (#2217)
Bumps the npm_and_yarn group with 1 update in the / directory: [js-yaml](https://github.com/nodeca/js-yaml). Updates `js-yaml` from 4.1.1 to 4.2.0 - [Changelog](https://github.com/nodeca/js-yaml/blob/master/CHANGELOG.md) - [Commits](https://github.com/nodeca/js-yaml/commits) --- updated-dependencies: - dependency-name: js-yaml dependency-version: 4.2.0 dependency-type: indirect dependency-group: npm_and_yarn ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
Gergő Magyar
|
df08ecc397
|
perf(lbug): cut graph-DB emit/persistence wall time (#2203) (#2215)
* perf(lbug): add PROF_LBUG_LOAD persistence-path timing breakdown (#2203 U1) loadGraphToLbug is un-timed today; the analyze 'emit' number is the scope-resolution emit bucket, not the CSV->COPY persistence path. Add a zero-cost-when-off per-stage breakdown (csv-emit/copy-nodes/rel-split/ copy-rels/fallback/total + node/rel counts) gated by PROF_LBUG_LOAD=1, mirroring the PROF_SCOPE_RESOLUTION pattern. Document the flag in README. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(lbug): route relationships to per-pair CSVs in the emit pass (#2203 U2) Relationships were written once to a monolithic relations.csv, then re-read line-by-line (regex per edge) and re-split into per-FROM->TO-label-pair files before COPY — writing and reading the entire ~1M-edge set twice. Route each edge to its pair file directly during the single emit pass via a shared RelPairRouter, eliminating the monolithic write + re-read + per-edge regex. The router applies the SAME getNodeLabel + validTables filter as the legacy splitRelCsvByLabelPair, which is retained as a differential oracle. A new differential test asserts the direct-emit per-pair files are byte-for-byte identical to the oracle's, with identical skip/total accounting. The prof line (U1) drops its rel-split stage (routing now folds into csv-emit). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(lbug): skip per-row microtask tick in BufferedCSVWriter (#2203 U3) addRow awaited an already-resolved promise on every buffered row, scheduling a microtask per node even when nothing flushed (millions at scale). It now returns a promise ONLY when it flushes; the node-emit loop awaits once per iteration after the switch. Flush/drain semantics are unchanged, so backpressure on the rows that actually write is preserved and the emitted CSV bytes are byte-identical (covered by the determinism + differential tests). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * bench(lbug): emit throughput + byte-identity gate for the persistence path (#2203 U4) Build-free bench (bench/emit-persistence/measure.mjs) times streamAllCSVsToDisk on a synthetic graph at two scales and gates: (1) an order-independent sha256 fingerprint over every emitted CSV line — the byte-identity guard for the U2/U3 emit optimisations — and (2) a scaling-ratio budget catching an O(n^2) emit re-regression. Wired into ci-tests.yml alongside the cfg/scope-capture benches. The LadybugDB COPY half needs a real DB, so its timing stays in PROF_LBUG_LOAD + the integration round-trip tests (documented in the bench README, with the deferred COPY-parallelism follow-up). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(review): apply autofix feedback (#2203) - P1: router backpressure drain-await rejected with a generic AbortError, masking the real EMFILE/disk-full error. Expose RelPairRouter.lastError and rethrow it in the emit catch — mirrors the oracle's throw streamError ?? err. - P1: cover RelPairRouter error + backpressure + teardown paths with a new unit test (test/unit/rel-pair-routing.test.ts) using an injected mock stream. - P2: wrap streamAllCSVsToDisk body in try/finally so the setMaxListeners bump is always restored (the U2 rel-routing throw path could leak it). - P2: dedup WriteStreamFactory — re-export the canonical type from rel-pair-routing instead of a second identical declaration. - P2: annotate splitRelCsvByLabelPair @internal as the retained differential oracle so a future dead-code sweep doesn't delete the byte-identity guard. - P3: differential test now covers the proc_ prefix + clears GITNEXUS_SORT_GRAPH_OUTPUT to prevent env-leak desync. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(lbug): scope byte-identity to quote-free ids + lock the quote-in-id divergence (#2215 review) The 'byte-identical' claim was unconditional, but the router derives labels from the raw id while the retained splitRelCsvByLabelPair oracle re-derives them via a regex over the escaped row — so for an id containing a double-quote they diverge (the router is the more-correct path). Soften the wording in rel-pair-routing.ts, the bench README, and the differential-test comment to document the exception, and add a differential test asserting the intended divergence (router routes the quote-in-id edge; oracle drops it) so a future change can't silently revert to the buggy regex semantics. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * bench(lbug): per-file fingerprint so the gate catches pair-file mis-routing (#2215 review) fingerprintEmit flattened every line of every per-pair file into one array, sorted globally, and hashed — losing file boundaries, so a row routed to the WRONG pair file produced an identical fingerprint. Hash a per-file digest (filename + sha256(file bytes)) and combine the sorted entry list, so mis-routing (and within-file row reordering) now changes the fingerprint. Baseline regenerated; the new scheme yields a different hash on byte-identical emit, confirming it is sensitive to file structure the old flatten ignored. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * bench(lbug): add absolute large-scale wall-time backstop to the emit gate (#2215 review) The scaling-ratio gate only compares large/small, so a uniform Nx slowdown at both scales passes with ratio ~1.0. Add an opt-in max_ms_large ceiling (1000ms vs observed ~200ms — generous, host-noise-tolerant) that --check enforces alongside the ratio, catching a gross absolute regression the ratio misses. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(lbug): cover the sorted-output path in the byte-identity differential (#2215 review) The differential test only exercised the default insertion-order emit path. Add a case under GITNEXUS_SORT_GRAPH_OUTPUT=1 that feeds the oracle the same id-sorted order orderedRelationships() uses and asserts per-pair byte-identity, so within-pair row reordering on the sorted path can't slip past the gate. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(lbug): cover the invalid-TO-label skip branch (#2215 review) Only an invalid-FROM label was exercised; the validTables skip is an OR over both endpoints, so the invalid-TO branch was untested (an inverted && would have slipped through). Add a valid-FROM/invalid-TO edge to the differential test and the router unit test, asserting it's skipped identically by router and oracle. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(lbug): exercise the BufferedCSVWriter FLUSH_EVERY boundary in vitest (#2215 review) The U3 addRow change (returns a flush promise only on flush; undefined when buffered) and the loop's `if (pending) await pending` were only crossed by the bench, never vitest (all fixtures are <500 nodes). Add a 600-node graph through streamAllCSVsToDisk asserting all rows land exactly once across the 500-row flush boundary — no drops, dups, or corruption. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(lbug): drop redundant step cast in buildRelRow (#2215 review) GraphRelationship.step is already typed number?, so (rel as { step?: number }).step was a no-op structural cast that obscured the shared-type coupling. Use rel.step directly. Byte-identical — bench fingerprint unchanged, differential test green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(lbug): make the unknown-label node drop explicit (#2215 review) With the U3 `let pending` switch idiom, a node whose label matches neither codeWriterMap nor multiLangWriters left `pending` undefined and was silently dropped — a footgun for a future node type. Add an explicit else with a comment documenting that unknown labels are intentionally not persisted and that a new type must be wired into a writer map. No behavior change (byte-identity + tests unchanged). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(lbug): drop the unused WriteStreamFactory re-export (#2215 review) The type was re-exported from lbug-adapter 'to preserve this module's surface,' but no external code imports it by name from here (the only test reference is a comment). Keep the import from rel-pair-routing.ts (its canonical home, still used by splitRelCsvByLabelPair's signature) and drop the dead re-export. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Gergő Magyar
|
cdb07289a4
|
perf(cfg): SSA-sparse reaching-defs to replace the dense-set worklist (#2201) (#2212)
* test(cfg): retain dense reaching-defs as differential oracle + fuzz harness (#2201 U1) * refactor(cfg): extract shared harvest/adjacency/sweep + swappable in-set computer (#2201 U2) * perf(cfg): sparse change-driven reaching-defs solver + canonical truncation (#2201 U3,U4) * perf(cfg): switch production reaching-defs to the sparse solver (#2201 U5) * perf(cfg): true SSA-sparse reaching-defs solver with auto-dispatch (#2201 U3) Replace the per-variable worklist (correct but no faster — it still walks pass-through blocks per binding) with Cytron SSA: CHK dominators + dominance frontiers + phi-placement + stack renaming over a synthetic entry, answering block-entry reaching queries by walking the SSA def-use graph (SCC-condensed, cycle-safe). Pass-through blocks carry the dominating def via the rename stack and phi-nodes statically capture loop merges, so dense-bindings drops from O(n^2) to O(n) (5-23x faster, asymptotic) and deep nests are depth-independent. The sweep now queries a lazy reachingAt accessor with a sparse intra-block overlay (no full per-block lattice copy). Production auto-dispatches: SSA for looping functions >=16 blocks (where it pays off, incl. the deep nests the dense ceiling used to truncate -> ceiling stops firing), dense elsewhere (small / loop-free functions, 1.0x — no regression). Throw-edge and unreachable-block functions fall back to dense (byte-identical). Held byte-identical to the dense oracle across a 300k-CFG (~1.2M-comparison) differential fuzz. * test(cfg): R5 contrast — dense ceiling fires, SSA solver converges (#2201 U6) * bench(cfg): deep-nest scenario + tighten dense-bindings rd budget 10->2 (#2201 U7) dense-bindings rd_scaling drops 5.2->0.86 (SSA linear); budget tightened to 2.0. New deep-nest scenario (N nested loops, one carried var) measures rd under the production blocks×64 ceiling and asserts the SSA solver still COMPUTES full facts (facts_large_min) where the dense worklist would truncate — the ceiling-stops-firing acceptance. CFG fingerprints unchanged. * docs(cfg): document SSA-sparse solver + resolve the WTO no-go note (#2201 U8) * fix(review): apply autofix feedback (#2201) - Close the production SSA-dispatcher fuzz-coverage gap: the generator's maxBlocks=14 was below SSA_MIN_BLOCKS=16, so the auto-dispatcher's SSA branch was never differentially fuzzed. Raise to 36, add a hadLargeLoop coverage assertion + a back-edge-into-entry canonical CFG. Validated byte-identical on 100k random CFGs incl. >=16-block looping shapes via both entry points. - Correct stale function JSDocs + @internal annotations (dispatch/fallback roles). - Add an independent rd_all_computed bench gate (catches partial truncation). - maxBlockVisits comment, SSA_MIN_BLOCKS calibration note, nx->next rename. * fix(cfg): gate out-of-range binding indices to the dense fallback (#2201 review) Tri-review (adversarial lane, reproduced) found the SSA path less tolerant than the dense oracle it replaced: an out-of-range binding index in defs/uses/mayDefs (a corrupted/stale durable store) crashed the nBindings-sized arrays (defBlocks[v]/stacks[u]), where dense tolerated it as a Map key. The throw escaped the unguarded taint/harvest call sites and lost a whole file's taint layer. Add a malformed-input gate that falls back to the dense solver (which handles any index), preserving byte-identity AND the graceful per-function degradation. Add an OOB canonical CFG to the differential fuzz + a production- entry no-throw unit test (the generator only ever emitted in-range indices, so this divergent input was structurally invisible). * perf(cfg): bound the SSA value-graph, fall back to dense when oversized (#2201 review R1) maxFacts bounds fact materialization in sweepFacts, but nothing bounded the SSA-sparse solver's φ/value-graph construction. A high-binding-density deep loop routed to SSA (≥16 blocks + a reachable loop) builds an O(blocks×bindings) value graph the dense path would have truncated at its maxBlockVisits ceiling (~1.5 GB measured on a 3000-block × 300-binding function). Cap the value graph: after φ-placement (where nodeKeys.length == the φ count, the input-superlinear term) plus a 2×Σgen bound on the renaming nodes, fall back to computeInSetsDense before paying for renaming + Tarjan SCC. The fallback is byte-identical (dense is the equivalence oracle) and bounded (dense honors maxBlockVisits). Mirrors the existing throw/unreachable/OOB-binding gates. The ceiling is DEFAULT_MAX_SSA_VALUE_GRAPH_NODES (1e6 — far above any real or benchmarked function; dense-bindings/deep-nest build <1e4), overridable per call via ReachingDefsLimits.maxSsaValueGraphNodes. The new unit test makes the otherwise-invisible routing flip observable by pairing the cap with a tight maxBlockVisits (dense truncates, SSA computes). Equivalence fuzz unchanged (byte-identical, 20k CFGs green); tsc clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(cfg): alias single-source SCC reaching-sets in reachByScc (#2201 review R2) The SCC-condensation pass built a fresh Set for every SCC and copied each cross-SCC operand's reaching-set element-by-element — O(defs²) at wide-fan-in φ merges (a φ over many predecessors, each carrying a large reaching-set). Add an alias fast path: an SCC with no own leaf keys whose cross-SCC operands all resolve to ONE source SCC has exactly that source's reaching-set, so share it by reference instead of copying. This is the common shape (pass-through φ / single-operand value node). The full union is still built when an SCC has own keys or genuinely merges ≥2 distinct sources. Safe to share: reachByScc sets are read-only after construction (operand SCCs are numbered before s in Tarjan's reverse-topological order and are only iterated), and contents are identical — set iteration order is irrelevant because sweepFacts sorts each use's keys before emission (KTD6). Byte-identical to the dense oracle (30k-CFG fuzz green); tsc clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(cfg): fold the SSA reachability gate into the RPO pass (#2201 review R8) computeInSetsSparse ran a standalone reachability BFS to gate unreachable-block functions to the dense oracle, then immediately computed a reverse-post-order over the synthetic-entry graph — two traversals of the same successor structure. reversePostOrder now returns the reachability bitmap its DFS already builds, and the sparse path reuses it for the unreachable-block gate (S→entry is S's only edge, so reachX[b] for b<n is exactly "reachable from entry" — identical to the removed BFS). One traversal instead of two on every SSA-dispatched function. The dispatcher's hasReachableLoop pass is left in place: it decides SSA-vs-dense BEFORE the solver is entered, and computeInSetsSparse must stay self-contained (the equivalence fuzz drives it directly, bypassing the dispatcher), so the two cannot share a traversal without coupling the InSetsComputer contract. Routing and facts unchanged — byte-identical to the dense oracle (30k-CFG fuzz, including unreachable-block shapes, green); tsc clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(cfg): trim per-statement/per-use/per-block allocations (#2201 review R9) Three transient allocations in the hot paths, all behavior-preserving: - sweepFacts: replace the per-statement `new Set([...defs, ...mayDefs])` with a direct `includes()` scan over the (1–3 element) def/mayDef arrays, guarded by a cheap hasSelfDefs flag that short-circuits pure-use statements. - sweepFacts: reuse a single scratch array for each use's reaching def-keys instead of spreading a fresh array per use. The KTD6 pre-sort still runs in place (load-bearing for truncated byte-identity). - computeInSetsSparse: build dPredsX by skipping consecutive-equal `from` values (preds[b] is pre-sorted by buildAdjacency, so duplicates are adjacent) instead of a per-block Set + spread + sort; the synthetic entry S = n exceeds every block index so it appends in order. The sweep is shared with the dense oracle, so these stay byte-identical on both paths — 50k-CFG fuzz (incl. maxFacts truncation, the order-sensitive case) green; tsc clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(cfg): correct the sweepFacts truncation byte-identity mechanism (#2201 review R6) The outer sweepFacts JSDoc attributed a truncated result's cross-solver byte-identity to the two solvers producing "identical inSets — insertion order included". That is wrong: the dense (RPO fixpoint) and SSA (renaming/SCC) solvers deliberately build a loop-carried use's reaching set in DIFFERENT insertion orders — same set, different order. The actual mechanism is the KTD6 per-use sort that canonicalizes each use's keys by defKey BEFORE the maxFacts cutoff (already documented correctly on the inner comment). Rewrite the outer doc to say so. Documentation only. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(cfg): extract pure graph sub-stages to reaching-defs-graph.ts (#2201 review R4) reaching-defs.ts had grown to ~1190 lines with the #2201 SSA rewrite. Move the self-contained, pure (plain-array) algorithms into a sibling module: - reversePostOrder - buildDominators (Cooper-Harvey-Kennedy) - buildDominanceFrontiers (Cytron) - tarjanScc + condenseReachingSets (SCC condensation, alias fast path) - hasReachableLoop (dispatcher loop check) - unionSets / latticeEquals (def-set / lattice primitives) The new module has a STRICT one-way dependency (it imports nothing from reaching-defs.ts — every helper is parameterized over plain arrays/Sets), so there is no import cycle and each stage is independently testable. reaching-defs.ts now holds the orchestrator, the two solver bodies, harvest, adjacency, the statement sweep, and the dispatcher: 1190 → 988 lines. Pure mechanical extraction — behavior is preserved by the differential equivalence fuzz (40k CFGs byte-identical) + the reaching-defs unit/snapshot suites; tsc clean. The helpers are @internal (kept out of the shipped .d.ts by the stripInternal change). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(pdg): stamp the reaching-defs solver identity for incremental re-analysis (#2201 review R3) The SSA-sparse rewrite computes full REACHING_DEF facts for deep-loop functions the old dense worklist truncated to empty at the blocks×64 ceiling. But an existing `--pdg` index carries those stale-truncated rows, and nothing forced a re-analysis: RepoMeta.pdg had no solver-identity key, so an upgraded run over an unchanged file kept the incremental fast path and never recomputed. Add a constant `reachingDefSolver: 'ssa-sparse-v1'` to the resolved pdg stamp (and to the RepoMeta['pdg'] type). It rides the existing key-union pdgModeMismatch comparator: a pre-#2201 stamp lacks the key, so 'ssa-sparse-v1' !== undefined trips one full writeback that recomputes the fuller coverage — no `--force` needed — exactly like the M2 REACHING_DEF cap and M5 CDG cap upgrade paths. A matching post-#2201 stamp compares equal, so there is no spurious re-analysis churn on steady-state re-runs. Tests: new pre-#2201→SSA upgrade block in pdg-mode-flip.test.ts (stamp present, absent-key mismatch, identical-stamp no-churn) + the persisted-stamp shape assertions and resolvePdgConfig DEFAULTS updated for the new key. tsc clean; pdg-mode-flip + run-analyze suites green (55/55). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * build(ts): stripInternal so @internal test-only exports stay out of the shipped .d.ts (#2201 review R5) computeReachingDefsDense/computeReachingDefsSparse are exported only for the equivalence fuzz and tagged @internal, but `declaration: true` emitted them into the public dist/**/*.d.ts. stripInternal removes any @internal-tagged export from the declaration output. This is repo-wide, which is the intended behavior: the same applies to every other test-only @internal export (hf-env's withDownloadTimeout etc., worker-pool's buildDispatchMessage/crashSignature, parse-impl's handleWorkerStartupFailure, the logger/safe-parse test resets, and the new reaching-defs-graph SSA helpers) — all of which are documented as not-public. Verified: - declaration emit succeeds with no TS4094/TS9006 ("cannot be named") errors; - the @internal functions are gone from the emitted .d.ts (reaching-defs-graph.d.ts is now `export {};`), while public symbols (computeReachingDefs) remain; - gitnexus-web — the only cross-package consumer — typechecks clean and imports only from gitnexus-shared, never from gitnexus internals; - runtime .js and the vitest/tsx tests are source-based, so unaffected. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(bench): add wide-merge scenario + tighten deep-nest facts floor (#2201 review R7) wide-merge: N bindings, each assigned in a 3-way branch (a wide multi-operand φ per binding) inside a loop, then all used after the merge. Unlike dense-bindings (one chained redef per `if`), every binding fans into its own wide φ, so the scenario exercises φ-placement + renaming + the reachByScc condensation across many independent wide merges. N bindings × constant arms ⇒ O(N) facts, so the gate is rd_scaling LINEARITY (measured ~1.07; budget 2.0 catches a regression to the per-binding-rescan O(N²) class the reachByScc alias path guards against). It runs the production SSA path (10007 blocks + a loop) and computes all facts under the blocks×64 budget (facts_large_min 24000 of a measured 26008 + the rd_all_computed gate). deep-nest: tighten facts_large_min 100 → 150 (measured 164) so a partial- truncation regression that still cleared 100 — but lost facts — now fails, with ~9% headroom for noise. bench --check PASS (9 scenarios) under --expose-gc; all existing CFG fingerprints unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * style(cfg): drop trailing blank line in reaching-defs.ts (prettier) Whitespace-only — a stray trailing newline left by the U4 extraction. `prettier --check` (the root format CI gate) now passes on every changed file. No behavior change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Gergő Magyar
|
5e96a99b0d
|
feat(cfg): model value-position branches as control dependence (#2205, #2207) (#2211)
Some checks are pending
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Scorecard / Scorecard analysis (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
* feat(cfg): model Java value-position switch as control flow (#2207) A value-position `switch` expression with ≥2 arms is now modeled as a CFG dispatch in the two highest-value carriers, instead of collapsing the owning statement to a single inline block: - `var x = switch (k) { … }` — the arms become real blocks reached by `switch-case` edges and rejoin at a binding continuation that carries the declared name's def (uses stay on the arm blocks). - `return switch (k) { … }` — each arm returns the function result, threading every active finalizer. This makes the arms control-dependent on the dispatch (the point of #2207 — they previously produced zero CDG), mirroring the Kotlin / Rust value-position binding pattern. `breaksBlock` routes a value-switch declaration out of `visitSeq` coalescing; `visitReturn` and `visitStmt` gain the carrier handling; `java-harvest` gains `bindingDefFacts`. An assignment RHS (`x = switch …`), a call argument, and a multi- declarator decl remain inline (documented gap). Java has no value- position `if` (the ternary is excluded, like Kotlin's elvis). Verified: 51 java-visitor tests (4 new), full CFG unit+integration suites (661) green, CDG snapshot byte-identical, bench --check PASS. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(cfg): model C# value-position switch expression as control flow (#2207) A C# `switch_expression` (`k switch { p => v, … }`) with ≥2 arms is now modeled as a CFG `switch-case` dispatch — a discriminant block, each arm value a block reached by a dispatch edge, all arms rejoining at one exit — in the three value-position carriers, instead of collapsing the owning construct to a single inline block: - `var x = k switch { … }` — arms rejoin at a binding continuation that carries the declared name's def (discriminant + arm uses on the arms). - `return k switch { … }` — each arm returns the function result, threading every active finalizer. - `=> k switch { … }` expression-bodied member — each arm returns. The arms are now control-dependent on the discriminant (the point of #2207 — they previously produced zero CDG). Arm patterns / `when` guards are harvested as conditional uses on the dispatch; an unguarded `_`/`var` arm is the exhaustive catch-all (a non-exhaustive switch keeps EXIT reachable via a no-match edge). `switch_expression` is distinct from `switch_statement`, so this adds a dedicated `visitSwitchExpr`. An assignment RHS (`x = k switch …`), a call argument, and a multi- declarator decl remain inline (documented gap). `csharp-harvest` gains `bindingDefFacts`. Verified: 47 csharp-visitor tests (4 new), full CFG unit+integration suites (664) green, CDG snapshot byte-identical, bench --check PASS. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(cfg): model PHP value-position match expression as control flow (#2207) A PHP `match($v) { c => v, default => v }` with ≥2 arms is now modeled as a CFG `switch-case` dispatch — a discriminant block, each arm value a block reached by a dispatch edge, all arms rejoining at one exit (no fallthrough) — in the two value-position carriers, instead of collapsing the owning statement to one inline block: - `$x = match($v) { … }` — the dominant PHP idiom (no typed local decl): arms rejoin at a binding continuation carrying the assignment target's def (condition + arm uses on the arms). - `return match($v) { … }` — each arm returns the function result, threading every active finally. The arms are now control-dependent on the discriminant (the point of #2207). Arm `match_condition_list`s are harvested as conditional uses on the dispatch; a `default` arm is the catch-all (a defaultless `match` throws UnhandledMatchError, kept EXIT-reachable via a no-match edge). `php-harvest` gains `assignmentDefFacts`. A `match` in a call argument / nested subexpression stays inline; the ternary `?:` is excluded by design (a micro-branch, like elvis). Verified: 37 php-visitor tests (3 new), CDG snapshot byte-identical, bench --check PASS. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(cfg): model Dart value-position switch expression as control flow (#2207) A Dart 3 value-position `switch (v) { p => e, _ => e }` with ≥2 arms is now modeled as a CFG `switch-case` dispatch — a discriminant block, each arm value a block reached by a dispatch edge, all arms rejoining at one exit (no fallthrough) — in the two value-position carriers, instead of collapsing the owning statement to one inline block: - `var x = switch (v) { … }` — single-binding decl; arms rejoin at a binding continuation carrying the declared name's def. - `return switch (v) { … }` — each arm returns the function result, threading every active finalizer. The arms are now control-dependent on the discriminant (the point of #2207). A Dart call value parses as `identifier` + `selector` (multiple children, not one node), so the arm-value facts come from a dedicated `switchExprArmValueFacts`; arm patterns harvest conditionally onto the dispatch; a `_` arm is the catch-all (a non-exhaustive switch keeps EXIT reachable via a no-match edge). `dart-harvest` gains `bindingDefFacts` + the arm value/pattern fact helpers. A `switch_expression` in a call argument / multi-binding decl stays inline (its conditional arm sub-evaluation remains the #2206 harvest may-def path, re-pointed in the regression test). `?:`/`??`/`?.` excluded by design. Verified: 39 dart-visitor tests (4 new), CDG snapshot byte-identical, bench --check PASS. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(cfg): model Swift value-position if/switch as control flow (#2207) A Swift 5.9 value-position `if`/`switch` is now modeled as control flow in the two value-position carriers, instead of collapsing the owning statement to one inline block: - `let x = if … else … / switch v { … }` — arms rejoin at a binding continuation carrying the declared name's def (condition + arm uses on the branch blocks). - `return if … / switch …` — each arm returns the function result, threading every active finalizer. The arms are now control-dependent on the branch (the point of #2207). tree-sitter-swift reuses `if_statement` / `switch_statement` for the value form (no separate `if_expression`/`switch_expression`), so the existing `visitIf`/`visitSwitch` are reused — this mirrors the Kotlin carrier exactly. `swift-harvest` gains `bindingDefFacts`. A value-position `if` requires an `else`; a value `switch` needs ≥2 entries. A value branch in a call argument / interpolation stays inline; `?:`/`??` are excluded by design. Verified: 31 swift-visitor tests (4 new), CDG snapshot byte-identical, bench --check PASS. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(cfg): model Kotlin assignment-RHS and value-position try as control flow (#2205) Completes the value-position branch carriers #2205 left deferred after the initial val/var-binding + return + expr-body work: - `x = when (k) { … }` / `x = if (c) a else b` / `x = try { … }` — a plain `=` assignment whose RHS is a modelable branch now models the arms as control flow and binds the LHS target at the rejoin (a compound `+=` and a plain-call RHS stay inline). - `val x = try { … } catch { … }` — a value-position `try` is now a modelable value branch (reusing visitTry), so the binding/assignment carriers route it through control flow too. The arms are now control-dependent on the branch (the point of #2205). `isModelableValueBranch` gains `try_expression`; `visitBranchExpr` routes it to `visitTry`; `isControlFlow`/`visitStmt` gain the `assignment` carrier; `kotlin-harvest` gains `assignmentDefFacts`. A branch nested in a call argument (`f(when …)`) stays inline (the direct value is the call); `?:`/`?.` micro-branches excluded by design. The `return try { … }` carrier is intentionally left out (finalizer-threading in return position is risky and was not requested). Verified: 43 kotlin-visitor tests (5 new), full CFG unit+integration suites (676) green, CDG snapshot byte-identical, bench --check PASS. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(cfg): Java colon-form value switch — yield ends the arm, no fallthrough (#2211) Tri-review (adversarial + correctness lanes) found that a value-position colon-form switch expression — `int x = switch(k){ case 1: yield a(); case 2: yield b(); }` (valid Java 14+) — reused the statement `visitSwitch` fallthrough logic, wiring a spurious `fallthrough` edge between the yield-terminated colon groups. A switch EXPRESSION never falls through between arms; the false edge dropped an arm's control-dependence edge and added a false reaching-defs propagation edge (verified by a real-parser probe). Arrow-form value switches were already correct. Root cause: `visitYield` modeled `yield e;` as a block that CONTINUES to the next statement. Semantically `yield` produces the switch-expression's value and EXITS the switch. Fix: `visitYield` now terminates the arm, jumping to the enclosing switch's exit and threading any finalizer it crosses — exactly like a `break` out of the switch, but carrying the yielded value's facts. Adds `ControlFlowContext.resolveYield()` (nearest SWITCH frame, never an intervening loop). `yield` is Java-only here (C# `yield return` is iterator semantics, untouched). Tests: a colon-form value switch asserting NO `fallthrough` edge and that BOTH arms are control-dependent on the dispatch (specific controller→ dependent pairs), plus a `return switch(…)` inside `try/finally` asserting `finally-return` threading per arm. Verified: full CFG unit+integration suites green, CDG snapshot byte- identical, bench --check PASS. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(cfg): Dart value switch — guarded `_` is not a catch-all; guard is a dispatch test (#2211) Tri-review (adversarial + correctness lanes) found two issues in the Dart `visitSwitchExpr` value-position modeling: 1. Catch-all detection was `pattern.text === '_'`, ignoring guards. A guarded `_ when c => …` is NOT exhaustive (Dart throws at runtime if no arm + guard matches), so falsely treating it as a catch-all suppressed the conservative no-match edge — asserting an exhaustive switch that isn't. The sibling C# visitor already gated catch-all on `!guard`. 2. A `when` guard parses as a bare sibling between the pattern and the value (no wrapper node), so it fell into the arm-VALUE children and was harvested as an unconditional arm-value use instead of a conditional dispatch test. Fix: new `armParts()` splits a `switch_expression_case` at the `=>` token into pattern / guard(s) / value(s). The pattern AND guard are harvested conditionally onto the dispatch block (they evaluate before the body, only when earlier arms missed); the arm-value facts come from the post-`=>` children only; the catch-all is gated on an unguarded `_`. Removes the now- unused dart-harvest `switchExprArm{Value,Pattern}Facts` (the visitor harvests per-child via the existing `facts`/`factsConditional`). Tests: a guarded value switch asserting the no-match edge is present (3 switch-case successors from the dispatch) and EXIT stays reachable, plus a test that the guard's use is recorded on the dispatch block, not an arm. Verified: full CFG suites green, CDG snapshot byte-identical, bench --check PASS. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(cfg): Kotlin return try {…} models the value-position try (#2205, #2211) Tri-review (maintainability + testing lanes) caught a doc-vs-code mismatch: the visitor header documents a `return <branch>` carrier that includes `try`, but `visitReturn` only matched `when_expression`/`if_expression`, so `return try { … } catch { … }` fell through to the single inline-block path — its arms were not modeled. Fix: `visitReturn` now also matches `try_expression`, making the `return` carrier uniform with the binding / assignment / expression-body carriers (all route a value-position `try` through `isModelableValueBranch` → `visitTry`, threading the active finalizers per arm). No new machinery — just completes the carrier set the docstring already claimed. Tests: `return try {…} catch {…}` (throw + return edges, CDG-bearing, EXIT reachable) and `x = try {…} catch {…}` assignment-RHS (the assignment carrier's try path, previously untested). Verified: full CFG suites green, CDG snapshot byte-identical, bench --check PASS. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(cfg): cover the no-match edge of non-exhaustive C#/PHP value switches (#2211) The testing lane noted the `hasCatchAll`/`hasDefault === false` branch — where a value-position `switch`/`match` with no catch-all/default arm adds a conservative no-match edge so EXIT stays reachable — was untested (every existing fixture used a `_`/`default` arm). Adds a C# `x switch { 1 => …, 2 => … }` and a PHP `match($x){ 1 => …, 2 => … }` (no default) test, each asserting the dispatch fans to (arms + 1) `switch-case` successors and `isExitReachableFromAllBlocks` holds. Verified: full CFG suites green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(cfg): clarify Kotlin value-position try gate covers the finally-only path (#2211) Tri-review (maintainability lane) noted the inline comment at the `try_expression` branch of `isModelableValueBranch` said only "a catch's value", but the gate fires on `catch_block || finally_block`. Reword to acknowledge that a value-position `try` with a `catch` OR a `finally` is a modelable branch. Comment-only; no behavior change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(cfg): document the deliberate C# declaratorInit duplication (#2211) Tri-review (maintainability lane) flagged the byte-identical `declaratorInit` helper in `csharp.ts` (visitor) and `csharp-harvest.ts` (harvester). The two are standalone classes with no shared base (repo convention) and the only module both import is the generic `utils/ast-helpers` (types only) — not a home for a C#-grammar-specific helper. Resolve the lowest-risk way: a cross-reference comment at each definition noting the deliberate duplication and the keep-in-sync requirement. No new shared module; no behavior change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(cfg): unwrap the paren in Dart visitSwitch for dispatch consistency (#2211) Tri-review (maintainability lane) noted `visitSwitch` (statement form) used the raw parenthesized `condition` (dispatch text `switch (x)`) while the new `visitSwitchExpr` unwraps it (`switch x`). Probed the vendored tree-sitter-dart: the `switch_statement` condition IS a `parenthesized_expression`, so apply `unwrapParen` in `visitSwitch` too. The harvest walks into the paren either way, so the discriminant's def/use facts are unchanged — only the dispatch block's text string normalizes. Verified byte-identical (cdg-snapshot + bench --check unchanged; the Dart unit tests assert topology, not block text). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(cfg): Swift single-entry value switch stays inline (#2211) Tri-review (testing lane) noted the existing Swift "stays inline" test used a plain call (`let x = g()`), which never exercises the single-entry switch gate. Add a real one-entry value switch (`let x = switch v { default: g() }`) asserting it coalesces (no switch-case edge), pinning the `>= 2` switch_entry threshold in `isModelableValueBranch`. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(cfg): cover the Kotlin expression-body try carrier (#2205, #2211) Tri-review (testing lane) noted the `fun f() = try { … } catch { … }` expression-body carrier (visitExprBody -> isModelableValueBranch accepting try_expression) existed but was untested. Add a regression asserting the expr-body try is modeled (throw + return edges, CDG-bearing, EXIT reachable). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(cfg): pin the value-branch carriers never throw on a truncated AST (R4) (#2211) Tri-review (adversarial lane) noted the new value-position branch carriers must return undefined / never throw on a malformed AST, or a single bad function would drop the whole file's CFG group (the R4 invariant). Add a per-language regression feeding a TRUNCATED value-branch carrier (an unterminated `var x = switch/match/if/when (…)`) through the existing `collectFunctions` + `buildFunctionCfg(...).not.toThrow()` graceful-undefined harness, for all six languages whose value-branch path is new (Java/C#/PHP/Dart/Swift/Kotlin). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Gergő Magyar
|
6932e7a9fd
|
feat(cfg): PDG/CFG visitors for all supported languages (#2195) (#2197)
* test(cfg): validate cfg/visitors literals + drop 3 dead TS node types
Extend the grammar-literal CI gate (test/helpers/literal-collectors.ts)
to scan cfg/visitors/*.ts, mapping each visitor file to its grammar via
the existing basename rule (c-cpp -> C/C++, csharp -> C#, java -> Java,
go -> Go, typescript -> TS). Closes the gap where the gate never
validated CFG visitor node-type literals -- the prerequisite for adding
C-family visitors safely (#2195 U1).
The newly-scanned TS visitor surfaced 3 dead literals absent from every
grammar it serves (typescript/javascript/tsx all = 0): for_of_statement
(for-of parses as for_in_statement), async_function_declaration and
async_arrow_function (async functions are function_declaration /
arrow_function + an async child). Removed them; behavior-preserving --
the cases never matched, bench --check fingerprints unchanged, TS
visitor unit tests green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(cfg): language-agnostic CFG unit-test harness (#2195 U1)
Extract the grammar-agnostic engine from ts-cfg-harness into
makeCfgHarness(grammar, visitor, filePath) at test/helpers/cfg-harness.ts.
Function discovery delegates to visitor.isFunction, so the harness carries
no language-specific node-type knowledge -- each C-family visitor's unit
tests can drive the real worker-side builder against real source.
ts-cfg-harness becomes a thin TS binding re-exporting the same
parse/collectFunctions/cfgOf/cfgsOf (behavior-preserving: all 5 existing
consumers -- taint propagate/model-match/summary-harvest/taint-emit + cfg
harvest -- pass unchanged, 223 tests green). New harness.test.ts proves
TS-faithfulness and isFunction-delegation via a stub visitor.
The bench parameterization (measure.mjs) is sequenced into U7, where the
first C-family scaling scenario makes the {grammar, visitorFactory} seam
validatable against a real non-TS language.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(cfg): C and C++ CFG visitor + def/use harvest (#2195 U2)
Add createCCfgVisitor/createCppCfgVisitor over a shared CCfgWalk core.
Grammar introspection confirmed tree-sitter-c and tree-sitter-cpp share
every control-flow node type/field, so CppCfgWalk extends CCfgWalk with
only the C++-only nodes (try/catch/throw/for_range_loop/lambda) via a
visitExtra hook -- no language conditionals (AGENTS no-language-naming).
Wire both into c-cpp.ts providers.
Harvest (c-cpp-harvest.ts): two-phase binding table + per-statement
defs/uses/mayDefs (no sites[] yet -- U6). Edge kinds match the TS
contract; functionStartColumn populated; non-terminating loops (for(;;),
while(1)) emit the structural exit-escape edge so EXIT stays
reverse-reachable and CDG is not silently skipped -- verified against the
production post-dominator + control-dependence solvers (for(;;) -> 3 CDG
edges). buildFunctionCfg returns undefined rather than throwing.
23 real-parser regression tests; grammar-literal gate green (literals
validated against both grammars). Documented gaps: C++ RAII destructors,
setjmp/longjmp, computed goto (route to EXIT + warn).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(cfg): C# CFG visitor + def/use harvest (#2195 U3)
Add createCsharpCfgVisitor + csharp-harvest over the shared CfgBuilder /
ControlFlowContext, modeling the C# statement taxonomy: if/else,
for/foreach/while/do, switch_section (+ switch_expression arms),
try/catch/catch_filter/finally, using + lock (deterministic finalizers --
dispose/release runs on normal AND exception exit, finally-* completion
edges on crossing jumps), goto/labeled, yield (surface only), return/
throw/break/continue. Wire into csharpProvider.
Every literal validated against tree-sitter-c-sharp via the introspection
probe (record_declaration, no else_clause, switch_section, positional
access where no field exists). Edge kinds match the contract;
functionStartColumn populated; while(true) keeps EXIT reverse-reachable
(production CDG probe: 3 edges). buildFunctionCfg returns undefined
rather than throwing.
34 real-parser regression tests; grammar-literal gate green; no
regression (cfg unit dir 256/256, tsc clean). Documented gaps: yield
iterator state machine, goto case/default, async suspension points.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(cfg): Java CFG visitor + def/use harvest (#2195 U4)
Add createJavaCfgVisitor + java-harvest over the shared CfgBuilder /
ControlFlowContext: if/else, classic for, enhanced-for, while, do-while,
classic-vs-arrow switch (switch_block_statement_group fallthrough vs
switch_rule no-fallthrough), try/catch/finally + try-with-resources
(auto-close synthesized as a finalizer, closes on normal AND exception
exit) + synchronized (monitor-release finalizer), labeled break/continue
to the labeled frame, yield, return/throw/break/continue. Wire into
javaProvider.
Every literal validated against tree-sitter-java via the probe
(switch_expression covers both switch forms, generic_type, line_comment,
for init field). Edge kinds match the contract; functionStartColumn
populated; while(true)/for(;;) keep EXIT reverse-reachable (production
CDG probe: 3 edges; hazard fixture: 34 CDG edges). buildFunctionCfg
returns undefined rather than throwing.
43 real-parser regression tests; grammar-literal gate green; no
regression (cfg unit suite 304, tsc clean). Documented gaps: switch-as-
expression-value inline, yield state machine, async/field-write defs.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(cfg): Go CFG visitor + def/use harvest (#2195 U5)
Add createGoCfgVisitor + go-harvest, the highest-divergence target:
for_statement (all four shapes -- for_clause C-style, while-style,
range_clause, bare for{}), expression/type switch (no implicit
fallthrough) + explicit fallthrough_statement, select_statement, defer
(LIFO finalizer legs at function exit), go (call is straight-line; the
closure body is its own CFG via isFunction), labeled break/continue/goto,
multiple-return assigns (a, b := f() defines each LHS). Wire into
goProvider.
CRITICAL (review A2): every non-terminating shape -- for{}, for cond{},
select{} with no default -- emits a structural exit-escape edge so EXIT
stays reverse-reachable and the production CDG is not silently skipped.
Verified: for{} -> CDG=3, select{} -> CDG=1, for-range -> CDG=2, all
exitReachable=true.
Every literal validated against tree-sitter-go via the probe. 32
real-parser regression tests; grammar-literal gate green; no regression
(186 across all 5 visitors + gate, full cfg unit 331, tsc clean).
Documented gaps: panic/recover unwind, goroutine happens-before.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(cfg): call-site sites[] taint substrate for C-family (#2195 U6)
Extend the C/C++/C#/Java/Go harvests with the call-site sites[] taint
substrate (SiteRecord/SiteArgOccurrence), mirroring the TS shape so the
shared taint matcher consumes all languages uniformly. Extract the
grammar-agnostic site machinery into cfg/visitors/call-site-harvest.ts
(CallSiteFactAccumulator -- names no language); each harvest adds only its
per-grammar visitCall/walkChain over its call node (C/C++ call_expression,
C# invocation_expression, Java method_invocation, Go call_expression).
INERT BY DESIGN: no C-family taint model exists (registerBuiltinTaintModels
is TS/JS only), so getSourceSinkConfig returns undefined for these
languages and the harvested sites produce ZERO TAINTED edges -- the
positive source->sink->TAINTED path is deferred with the model authoring.
sites emitted only when non-empty; facts-only attachment, block/edge
topology unchanged (pre-existing topology + def/use tests byte-identical).
23 new substrate tests; 574 green across the cfg/taint/emit suites; gate
green; tsc clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(cfg): worker-mode PDG integration + bench parameterization (#2195 U7)
Prove the five C-family visitors build PDG through the REAL worker
pipeline. pipeline-pdg.test.ts: per-language (C/C++/C#/Java/Go) temp repo
run with pdg:true asserts BasicBlock+CFG+REACHING_DEF+CDG all > 0 (CDG>0
proves EXIT stays reverse-reachable end-to-end through the worker, incl.
each fixture's non-terminating loop/select); a paired run with pdg off
asserts == 0, the two flag-off graphs byte-identical (R3), no PDG types
leak, pinned by a golden snapshot. Counts e.g. Go 151 BB / 56 CDG.
Parameterize bench/cfg/measure.mjs by a per-language LANGS registry
resolved generically via getLanguageGrammar + getProvider(X).cfgVisitor
(no static import table). Default TS byte-identical -- all 6 TS
fingerprints unchanged under --check; taint-dense stays TS-only
(TS_JS_TAINT_MODEL never runs against model-less C-family CFGs). Add a
go:branchy scenario+baseline (namespaced) -- its fingerprint shape
(32 blocks/46 edges) matches TS branchy, cross-validating the Go visitor.
15 pipeline tests + bench --check PASS (7 scenarios); 354 unit cfg green;
dist rebuilt clean. Absorbs the bench parameterization deferred from U1.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(cfg): Python CFG visitor + def/use harvest (#2195 U8)
Add createPythonCfgVisitor + python-harvest -- the most structurally
divergent target (indentation blocks, elif, for/while-else, with, try/
except/except-group/else/finally, match/case, comprehensions, walrus),
confirming the shared CfgBuilder/ControlFlowContext core carries no
brace-family assumptions. for/while else-clause sits on the normal-
completion edge (not break); with modeled as try/finally dispose; match
has no fallthrough. Wire into pythonProvider.
Every literal validated against tree-sitter-python via the probe.
while True: keeps EXIT reverse-reachable (production CDG probe: 3 edges;
fixture: 42 CDG edges). 37 real-parser tests; gate green; no regression
(cfg unit 391, tsc clean). Gaps: async/generator suspension, comprehension
scope over-approximation. No sites[] (taint substrate, separate).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(cfg): PHP CFG visitor + def/use harvest (#2195 U9)
Add createPhpCfgVisitor + php-harvest: if/elseif/else (+ alt colon
syntax), for/foreach/while/do-while, switch (fallthrough) + match (no
fallthrough), try/catch/finally, break N/continue N (N-th enclosing
loop), goto, return/throw. Wire into phpProvider.
Every literal validated against tree-sitter-php (php_only) via the probe
(for_statement initialize/condition/update; throw_expression not
throw_statement; break/continue integer child). while(true) keeps EXIT
reverse-reachable (production CDG probe: 3 edges; break 2 escapes the
outer loop). 35 real-parser tests.
Also repoint worker-roundtrip's "non-CFG language" gate test from Python
(which now has a cfgVisitor) to COBOL (the permanent non-goal of the
rollout) -- a stale assertion the Python commit invalidated. Full
in-process sweep green (452 across 18 files). Gaps: match inline value,
goto plain-block.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(cfg): Ruby CFG visitor + def/use harvest (#2195 U10)
Add createRubyCfgVisitor + ruby-harvest: if/unless/elsif/else +
statement-modifier forms (x if c, x while c), while/until/for (until
inverts the sense), case/when + case/in (pattern, no fallthrough),
begin/rescue/else/ensure (ensure=finally, rescue=catch) + retry
(loop-back into begin), return/break/next/redo, blocks/lambdas as their
own closure CFGs. Wire into rubyProvider.
Every literal validated against tree-sitter-ruby via the probe (case vs
case_match, modifier nodes, typed rescue/ensure children). loop do /
while true keep EXIT reverse-reachable (production CDG probe: 3 edges).
34 real-parser tests; comprehensive sweep green (486). Gaps: yield,
expression-position if/case/begin inline, ivar/gvar non-local defs.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(cfg): Rust CFG visitor + def/use harvest (#2195 U11)
Add createRustCfgVisitor + rust-harvest for the expression-oriented Rust:
if/else + if-let, loop (infinite -- structural escape edge), while/
while-let/for, match (no fallthrough) + guards, labeled break/continue
('outer), break-with-value, ? operator (try_expression) as an
early-return throw edge to EXIT, let-else (diverging else). visitLet
handles control-flow in value position (let x = loop/if/match). Wire into
rustProvider.
Every literal validated against tree-sitter-rust via the probe (label is
a named child not a field; line_comment; _ pattern). loop {} keeps EXIT
reverse-reachable (production CDG probe: 3 edges). 33 real-parser tests;
comprehensive sweep green (519). Gaps: panic, async/.await, macro bodies.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(cfg): Swift CFG visitor + def/use harvest (#2195 U12)
Add createSwiftCfgVisitor + swift-harvest (vendored tree-sitter-swift via
requireVendoredGrammar): if/else + optional binding (if let), guard...else
(diverging early exit), for-in/while/repeat-while (bottom-test), switch
(no implicit fallthrough; explicit fallthrough keyword; where guards),
do/catch + try/try?/try!, defer (LIFO finalizer at scope exit), labeled
break/continue, control_transfer_statement (one node for break/continue/
return/throw). Wire into swiftProvider.
Every literal validated against the vendored grammar via the probe (no
block node; if-let folds into condition+bound_identifier; defer parses as
a call_expression with trailing closure). while true keeps EXIT
reverse-reachable (production CDG probe: 3 edges). 24 real-parser tests;
comprehensive sweep green (543). Gaps: computed properties, defer
block-scope approx, fatalError traps.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(cfg): Kotlin CFG visitor + def/use harvest (#2195 U13)
Add createKotlinCfgVisitor + kotlin-harvest (vendored tree-sitter-kotlin):
if/else, when (subject + subjectless, no fallthrough), for/while/do-while,
try/catch/finally, jump_expression (return/return@/break/break@/continue/
continue@/throw), labeled loops, control_structure_body unwrapping,
expression-body functions. The grammar is field-less for control flow, so
the visitor navigates by child type+position. Wire into kotlinProvider.
Every literal validated against the vendored grammar via the probe
(line_comment/multiline_comment, not comment). while (true) keeps EXIT
reverse-reachable (production CDG probe: 3 edges; worker-mode fixture:
BB=82, CDG=41). 28 real-parser tests; comprehensive sweep green (571).
Gaps: value-position if/when/try inline, inline-fun non-local return,
getters/setters.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(cfg): Dart CFG visitor + def/use harvest (#2195 U14)
Add createDartCfgVisitor + dart-harvest (vendored tree-sitter-dart):
if/else, C-for/for-in/while/do-while, switch (empty-case fallthrough +
explicit continue-label) + switch_expression, try/on/catch/finally +
rethrow + assert (throw edges), return/break/continue/throw, labeled
loops, arrow bodies, closures. Dart splits a function into sibling
signature + function_body nodes, so the body (or function_expression) is
the CFG-bearing node. Wire into dartProvider.
Every literal validated against the vendored grammar via the probe (only
constant_pattern exists; removed speculative relational/logical pattern
names). while (true) keeps EXIT reverse-reachable (production CDG probe:
3 edges). 34 real-parser tests; comprehensive sweep green (605). Gaps:
labeled-loop grammar quirk (read via ERROR sibling), async straight-line,
value-position if/switch inline.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(cfg): Vue (reuse TS visitor) + worker-mode proof for all langs (#2195 U15)
Vue SFC <script> blocks are extracted and parsed with the TS grammar
(parse-worker languageMap[Vue] = TypeScript.typescript), so wire
vueProvider.cfgVisitor = createTypeScriptCfgVisitor() -- pure reuse, no
Vue-specific visitor. vue-visitor.test.ts replicates the worker path
(extractVueScript -> TS parse -> CFG) and confirms branch edges + EXIT
reverse-reachable + CDG>0.
Extend pipeline-pdg.test.ts with a worker-mode block covering all eight
remaining languages (Python/PHP/Ruby/Rust/Swift/Kotlin/Dart/Vue): per-
language temp repo, real worker pool, BasicBlock+CFG+REACHING_DEF+CDG all
> 0 with --pdg (CDG>0 proves EXIT reverse-reachable end-to-end through the
worker despite each fixture's non-terminating loop), == 0 without. Counts
e.g. Ruby 122 BB/45 CDG, Vue 49 BB/11 CDG. 30 pipeline tests green.
COBOL: documented as the deliberate PDG non-goal (no grammar, exotic
PERFORM/GO-TO control flow) in cobol.ts + the worker-roundtrip gate.
This completes PDG language coverage: every supported language except
COBOL now builds CFG/REACHING_DEF/CDG under --pdg.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(cfg): surface skippedUnsoundFunctions in per-language stats (#2195 U2)
emitFileCdg computes skippedUnsoundFunctions (functions whose CDG is
withheld because EXIT isn't reverse-reachable from all blocks) but run.ts
dropped it on the floor — only cdgEdges/cdgDropped were aggregated. Add
the aggregation + a stats-line segment so CDG coverage gaps are an
explicit signal, not silent. Establishes the baseline skip count that
makes the U1 synthetic-escape pass's effect (the drop to genuine
anomalies only) measurable.
Additive; no emit-logic change. The emit-side field is covered by
cfg-emit.test.ts (asserts skippedUnsoundFunctions===1 + the warn on a
disconnected-block CFG); the run.ts aggregation is a thin pass-through.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(cfg): synthetic-escape pass restores CDG for exit-unreachable cycles (#2195 U1)
Unconditional goto-cycles (C/C++/C#/Go) wire a backward seq edge with no
structural exit-escape edge, so EXIT becomes non-reverse-reachable and
emitFileCdg silently skipped ALL control-dependence for the function.
New cfg/synthetic-escape.ts: a pure deterministic SCC routine (iterative
Tarjan, sorted adjacency) + augmentForPostDom(cfg). No-op when EXIT is
already reverse-reachable (terminating fns + visitor-escaped loops are
byte-identical — returns the same object). Otherwise it batch-bridges
every exit-less SCC by adding an ANALYSIS-ONLY escape edge from the SCC's
controlling block (highest out-degree branch; lowest-index tie-break) to
EXIT, on a shallow-cloned FunctionCfg — never mutating persisted
cfg.edges. emitFileCdg threads that augmented view through BOTH
isExitReachableFromAllBlocks AND computeControlDependence (the Ferrante
walk re-reads cfg.edges, so a tree-only augmentation would be wrong).
Precision (anti-masking): only a trapped region containing a control
point (>=2-successor block) is bridged — a branch-less trapped region
carries no recoverable control-dependence and is indistinguishable from a
genuine construction anomaly, so it stays on the skip path (the existing
disconnected-block skip test still skips, skippedUnsoundFunctions===1). A
residual non-cycle dangling block is never bridged.
repro `void handler(int a){ start: if(a>0){work();} goto start; }`:
before exitReachable=false/CDG=0 → after one synthetic 2->1 edge,
exitReachable=true, exact CDG = {2->2:T,2->2:F,2->3:T,2->4:T,2->4:F}
(pinned exactly, not CDG>0 — catches a wrong representative). AC2 property
test extended to the augmented graph; per-language goto-cycle regressions
(C/C++/C#/Go). 199 cfg tests green; bench --check fingerprints unchanged
(analysis-only, zero persisted drift).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(cfg): isolate the non-terminating-loop hazard in worker CDG asserts (#2195 U3)
The pipeline-pdg worker-mode blocks asserted a whole-fixture cdg>0
aggregate (satisfied by any branching fn) while the comment claimed it
proved the non-terminating-loop EXIT-reachability end-to-end. Add a per-
language `hazard` marker + isolate the assertion: locate the hazard
function's BasicBlocks by its anchor and assert >=1 CDG edge is sourced
within it (a marker mutation now fails the test — non-vacuous). C# keeps
the aggregate (its fixture has no infinite loop). Comments corrected.
Switch the 7 visitor unit tests (java/csharp/dart/kotlin/php/swift/c-cpp)
from the local exitReachableFromAll CFG-shape helper to the production
isExitReachableFromAllBlocks + computeControlDependence on the hazard
function, matching go/python/ruby/rust/vue. 241 unit + 30 pipeline green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(cfg): gate vendored-grammar worker assertions on isLanguageAvailable (#2195 U4)
The Swift/Kotlin/Dart worker-mode pipeline-pdg cases require a vendored
grammar prebuild that may be absent on a CI platform — they'd go red
there. Mark those three REMAINING_LANGS entries `vendored` and gate both
the --pdg-on and --pdg-off `it`s on isLanguageAvailable(SupportedLanguages
[lang]) → it.skip when the grammar can't load. Installed-grammar
languages stay unconditional. Grammars present here, so all 30 run green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(cfg): remove dead useCount() from swift + rust harvests (#2195 U5)
useCount() was declared on the local FactAccumulator in swift-harvest.ts
and rust-harvest.ts but never called (a copy-paste artifact; ruby's copy
IS used in an emit guard, so it stays). Pure deletion — the swift/rust
visitor suites stay green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(cfg): standardize harvester API table()->bindingTable() (#2195 U9)
The binding-table accessor was named table() in the C/C++/C#/Go harvests
but bindingTable() in the other 7. Rename the 4 (definitions + their
visitor call sites) to the majority name bindingTable(). Pure rename; the
4 visitor suites stay green and tsc confirms no call site was missed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(cfg): consolidate scope-tree substrate into ScopeTreeHarvester (#2195 U6)
The Go/Java/C#/C-C++ def/use harvesters each carried a byte-identical copy
of the lexical scope-tree machinery (Scope record, two-phase resolution
cache, openScope/nearestScopeOf/resolve/def/use/conditional/bindingTable,
~270 lines total). Extract it into an abstract ScopeTreeHarvester base; the
four harvesters now extend it and supply only their genuine per-language
variation (the prescan switch, plus Go's _-blank-identifier overrides of
declare/def/use). Net -422 lines. Mechanical and byte-equivalent: cfg unit
suite 613 passed, bench --check fingerprints unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(cfg): consolidate no-site def/use accumulator into DefUseAccumulator (#2195 U7)
The Kotlin/Python/Ruby/Rust/Dart/Swift harvesters each carried a
byte-identical copy of the no-site def/use accumulator (~270 lines total;
only Ruby's adds the live useCount() emit-guard helper). Extract it as an
exported DefUseAccumulator beside CallSiteFactAccumulator in
call-site-harvest.ts (the PR's own model for the with-site superset); the six
harvesters import it under their existing local FactAccumulator name. Pure
byte-equivalent move, no logic change: cfg unit suite 613 passed, tsc clean,
bench --check fingerprints unchanged (TS/Go paths untouched).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(cfg): consolidate copied visitor-test helpers into cfg-harness (#2195 U8)
The 13 *-visitor.test.ts files each copied a byte-identical set of CFG-shape
helpers (edgeKinds/block/reaches/reachable/bindingIdx/allSites/hasAnySites,
~380 lines total). Export them once from test/helpers/cfg-harness.ts and import
per file (only the subset each references). Also drop each file's local
exitReachableFromAll — a re-implementation of the production
isExitReachableFromAllBlocks (semantically identical: false iff some
entry-reachable non-EXIT block can't reach EXIT) — and point its live call
sites at the already-imported production function. Pure test-only mechanical
move, behavior-preserving: tsc clean, test/unit/cfg/ 613 passed unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(cfg): pin *-harvest.ts literals to their own grammar in the gate (#2195 U10)
The grammar-literal validation gate scans cfg/visitors/, but a <lang>-harvest.ts
basename was not in BASENAME_LANGS, so fileLanguages() fell it through to the
weak ALL_LANGS valid-if-any bucket — a node-type literal dead in its own grammar
but valid in some other grammar would pass undetected. Strip the -harvest suffix
and reuse the visitor basename map so go-harvest -> Go, c-cpp-harvest -> C+C++,
typescript-harvest -> TS, etc. The two language-agnostic harvesters
(call-site-harvest, scope-tree-harvest) name no grammar and stay valid-if-any.
Also corrects the now-inaccurate mode2Files comment. Adds a fileLanguages unit
test; the existing gate stays green (no harvest file has a dead literal), and a
scratch probe confirmed a bogus go-harvest literal is now caught.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(cfg): defensive per-statement cap on harvested taint sites (#2195 U11)
A statement's harvested sites[] had no explicit bound — a pathological or
machine-generated statement (hundreds of nested calls) could grow it without
limit. Add DEFAULT_PDG_MAX_SITES_PER_STATEMENT (512, mirroring the PDG edge/fact
cap style): openCallSite/addMemberRead check-before-push and stop at the cap,
keeping the first 512 sites fully intact and setting an observable
sitesTruncated flag. A cap-dropped openCallSite returns a -1 sentinel that
pushFrame/setSite*/the occurrence fan-out all tolerate (no dangling parent/via,
no clobber of kept sites). Generous enough that no real statement is affected:
bench --check fingerprints unchanged, cfg unit suite 617 passed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* ci(codeql): exclude nested test fixtures from the CodeQL gate (#2195)
The CodeQL results gate failed on test/integration/cfg/fixtures/python-hazards.py
('total' may be used before init, unused vars) — but that file is an intentional
CFG/PDG hazard fixture, exactly the synthetic broken-code the existing
'**/test/fixtures/**' exclusion is meant to skip. That glob does not match the
deeper test/integration/cfg/fixtures/ path, so the hazard fixtures leaked into
the scan. Add '**/test/**/fixtures/**' to cover fixtures nested anywhere under a
test tree. Analyze (python) and Analyze (javascript-typescript) both already pass
— production code is clean; this only silences fixture noise.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* style(cfg): apply prettier + drop unused imports across the PDG files (#2195)
The merge of main into this branch pulled in the stricter quality gates
(prettier --check . and eslint .), which surfaced pre-existing formatting in the
PDG/CFG rollout (line-width wrapping across the visitor + harvest files, bench,
tests) plus 9 no-unused-imports errors. Mechanical autofix only — npm run
format + lint:fix equivalent, scoped to gitnexus/: removes unused FunctionCfg/
SiteRecord type imports left by the U8 helper consolidation and stale
FinalizerFrame imports in python.ts/ruby.ts. No behavior change: tsc clean, cfg
unit suite 617 passed, eslint 0 errors. (gitnexus-web class-order noise is a
local tailwind-plugin artifact CI does not flag — left untouched.)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(cfg): harvest C++ structured-binding defs (auto [a,b]=e) (#2195)
The C++ def/use harvester only recorded a def when an init_declarator's
declarator was a plain identifier, so a structured binding (auto [a,b] = mk(),
incl. the auto& reference form whose binding sits under a reference_declarator)
declared only the first name in phase 1 and emitted ZERO defs in phase 2 — a,b
were walked as spurious uses and later use(a)/use(b) resolved to a synthetic
module binding, silently corrupting REACHING_DEF/taint for an idiomatic C++17
shape. Unwrap the structured_binding_declarator in both phases and def every
identifier leaf; result-of-initializer flows to the whole list. Inert for C
(no structured bindings). Characterization tests added (plain + reference form).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(cfg): model C++ co_return as a return terminator to EXIT (#2195)
co_return_statement was neither in CPP_CONTROL_FLOW_TYPES nor dispatched, so a
coroutine's co_return coalesced into a straight-line block and emitted a
spurious seq fallthrough to the following statement instead of an edge to EXIT
— statements after co_return looked reachable and the terminator edge was
missing, corrupting CFG/CDG for coroutines. Add the node type to the C++
control-flow set and dispatch it through visitReturn (block -> EXIT 'return',
no fallthrough). C path untouched; co_await/co_yield remain plain expressions.
Characterization test added; c-cpp suite + grammar-literal gate green, bench
--check unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(cfg): harvest C# out-var and deconstruction-declaration defs (#2195)
Two idiomatic C# write shapes recorded ZERO defs, silently breaking
REACHING_DEF/taint:
- out-var (G(out var n) / G(out int n)) parses as a declaration_expression;
it was neither declared (phase 1) nor def'd (phase 2), so n resolved to a
synthetic module binding and the callee-written value had no reaching def.
- deconstruction declaration (var (a, b) = T()) has a variable_declarator whose
name slot is a tuple_pattern (null name field), so declareVariableDeclaration
+ the variable_declaration walk skipped it entirely (only the assignment form
(a,b)=T() was handled). Both a and b were dropped.
Declare + def the declaration_expression's identifier (must-def: out params are
definitely-assigned), and route a null-name variable_declarator through the
tuple_pattern via the existing declareForeachTarget/defTupleTargets helpers.
Characterization tests added; csharp suite 42 passed, grammar gate + bench
--check green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(cfg): put embedded-script CFGs in file coordinates via lineOffset (#2195)
A Vue SFC <script> block parses at row 0 but lives at lineOffset in the .vue
file. Every other worker-emitted graph node adds lineOffset to reach file
coordinates, but collectFunctionCfgs built FunctionCfgs from the extracted
script's raw rows and never offset them. Two consequences for .vue files:
- inter-procedural taint silently resolved NOTHING — the summary-harvest join
keys graph Function/Method nodes by their (offset) startLine but looked up the
CFG's (unoffset) functionStartLine, missing by exactly lineOffset, so no
FunctionSummary was ever produced;
- persisted BasicBlock startLine/endLine (and the id's functionStartLine
segment) pointed at the wrong .vue line, breaking source mapping.
Thread lineOffset into collectFunctionCfgs and shift every CFG source-line field
(functionStartLine/End, block start/end, statement + non-synthetic binding
lines) into file coordinates at the one production chokepoint. A 0 offset
returns the CFG unchanged, so .ts/.js/etc. stay byte-identical (bench --check
fingerprints unchanged; worker-roundtrip + pipeline-pdg green). Unit tests for
the shift + the 0-offset no-op added.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(cfg): surface CDG soundness skips at warn, not just debug (#2195)
skippedUnsoundFunctions (a function whose EXIT is not reverse-reachable from
all blocks, so control dependence is withheld) was only reported inside the
per-language logger.debug stats line — while the taint/RD coverage-gap and
cap-drop counts surface unconditionally at warn. A language that systematically
trapped EXIT (an unmodeled non-terminating / multi-terminal shape the
synthetic-escape pass can't bridge) would silently lose all CDG. Add a parallel
unconditional warn (R8) alongside the R4 taint-gap warn. Observability only —
no graph change; emit-layer skip counting stays covered by cfg-emit's
skippedUnsoundFunctions test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(cfg): harvest Kotlin x++/--x as a def (#2195)
The Kotlin harvester had no postfix_expression/prefix_expression case, so an
increment/decrement fell to the default descent and recorded its operand as a
use only — never a def. Every sibling harvester (Java/C#/C++/Dart/TS/PHP) models
inc/dec, so a Kotlin counting loop (while/for using i++) silently dropped the
loop-carried reaching-def of the counter. Add the case: def AND use the operand
when it is a plain simple_identifier and the operator is ++/-- (other pre/postfix
forms — -x, !x, x!!, x? — stay pure reads, byte-identical to the old descent).
Characterization tests for postfix + prefix added; kotlin suite 30 passed,
grammar gate + bench --check green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(cfg): harvest Go select channel-receive binding as a def (#2195)
walkValue had no receive_statement case, so a select receive (case v := <-ch:)
fell to the default descent: v was recorded as a USE of an uninitialized var
and the channel-sourced definition was invisible to REACHING_DEF/taint —
channels are a primary taint source in Go. Add the case mirroring
short_var_declaration: def each left identifier, use the <-ch right, attach
resultDefs for the := short form. prescan already declared the binding; this
completes the phase-2 fact. go:branchy bench fingerprint unchanged; go suite
40 passed, grammar gate green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(cfg): harvest all names of a Dart multi-variable declaration (#2195)
`var a = 1, b = 2;` is one initialized_variable_definition whose first binding
is the name/value field pair and whose subsequent bindings are trailing
initialized_identifier children. Both prescan (declareInitializedVar) and the
walkValue case read only the name/value fields, so every name after the first
was never declared or def'd — `b` resolved to a synthetic module binding and
its REACHING_DEF/taint flow was lost. Iterate the trailing initialized_identifier
nodes in both phases. (Dart-3 record/list pattern declarations `var (a,b)=pair`
remain a separate follow-up.) dart suite 35 passed, tsc + grammar gate green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(cfg): bind Swift switch-case value patterns (case let n) (#2195)
A switch value-binding (case let n where …, case .some(let v)) was never
declared in prescan, so n/v resolved to a synthetic module binding and a body
use(n) did not link to any def — a very common Swift idiom silently lost its
data dependence. Declare the switch_pattern's bindings (prescan, reusing
declarePattern) and emit them as MAY-defs on the dispatch block (a case may not
match) via a new switchPatternFacts, propagated into the case body. swift suite
25 passed, tsc + grammar gate green. (The rare ?? / ternary-arm may-def — Swift
assignment-as-expression — remains a separate follow-up.)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(cfg): wire Swift multi-catch throw edges to every handler (#2195)
visitDo routed the protected body's throw edge only to handlerEntries[0], so a
do { try r() } catch A {} catch {} left the 2nd..Nth catch handlers UNREACHABLE
from ENTRY — orphaned blocks whose error bindings + def/use facts were stranded
in a dead component (a soundness gap for idiomatic Swift typed multi-catch).
Swift tries the catch clauses in order and the thrown type is unknown at CFG
time, so every protected block may reach ANY clause: edge each protected block
to every handlerEntry. Found by the per-language CFG/CDG verification swarm
(reproduced: 2-catch=1, 3-catch=3 unreachable blocks). swift suite 26 passed,
tsc clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(cfg): synthesize a protected block for an empty Kotlin try {} (#2195)
An empty `try {}` body produced zero protected blocks, so visitTry's throw-edge
loop wired nothing to the catch and the try's entry fell through to the finally
— leaving the catch handler block + its error binding orphaned (unreachable from
ENTRY), a malformed CFG with stranded def/use facts. Mirror the existing
empty-`catch` synthesis: when the try body is empty and there is a catch or
finally, synthesize one protected block so the catch handler(s) are wired and
the try entry is the body, not the finally. Found by the per-language CFG
verification swarm. Non-empty try is byte-identical; kotlin suite 31 passed,
tsc clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* perf(cfg): bound the reaching-defs fixpoint with a per-block visit ceiling (#2195)
The per-language verification swarm reproduced, AT PRODUCTION DEFAULTS, a
reaching-defs blow-up: a machine-generated ~2000-line all-loops function (under
DEFAULT_PDG_MAX_FUNCTION_LINES) reaches ~10k basic blocks because loops emit ~5
blocks/line, and the dataflow fixpoint is O(blocks^2.3) on deep loop nests —
measured 62s (C/C++) and 2.05s + 810MB (Go) for ONE function. maxFacts does not
help: the fact count stays LINEAR, so it never fires.
Iterative reaching-defs on a reducible CFG converges in O(loop-nesting-depth)
passes, so a worklist re-visits each block a small multiple of times for real
code. Add a maxBlockVisits ceiling (emit passes blocks.length × 64 — far beyond
any hand-written nesting depth, ~15) that bails when the fixpoint has not
converged. An unconverged fixpoint's in/out sets are not sound, so it returns
NO facts (status 'truncated', like the existing 'overflow' guard) — a per-
function coverage gap, never wrong facts. Real code is byte-identical: full cfg
suites 725 passed, bench --check fingerprints unchanged.
NOTE: computeControlDependence's O(N²) up-walk on deep post-dom chains is the
sibling concern but stays ~13ms in production (bounded by the line cap + the
CDG materialization cap); a CDG work-budget is a documented follow-up.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(cfg): raise the parse-worker stack limit for deep CFG recursion (#2195)
The CFG visitors build per-function control-flow graphs by recursive descent
over the tree-sitter AST, so deeply-nested source overflows the worker thread's
call stack (~1.5k nesting levels) — caught per-function (R4 try/catch) but the
function silently gets no PDG. A worker thread's stack is governed by
resourceLimits.stackSizeMb (Node default 4 MB); the main process's
--stack-size=4096 flag does NOT propagate to worker threads (confirmed by prior-
art research on Node worker_threads). Raise it to 16 MB, pushing the overflow
threshold to several-thousand nesting levels — far beyond any hand-written code,
so only machine-generated/obfuscated nesting can still hit it (and that stays a
caught per-function skip, never a crash). Complements a future proactive depth
guard. pipeline-pdg worker tests 30 passed, tsc clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* perf(cfg): compute control dependence as a reverse-CFG dominance frontier (#2195)
The Ferrante §3.1.1 up-walk re-climbed the ipdom chain once per CFG edge,
which is Θ(N²) on a deep post-dom chain (a single branch fanning into a
shared spine took ~7.1s at 16k blocks). Replace it with the reverse-CFG
post-dominance-frontier formulation (Cytron, Ferrante, Rosen, Wegman &
Zadeck 1991): control dependence IS the dominance frontier of the reverse
CFG, computed bottom-up over the post-dom tree (PDF_local from a node's CFG
in-edges + PDF_up from its post-dom-tree children) in O(N + E + output).
LLVM (ReverseIDFCalculator), Joern (CdgPass) and WALA use the same form.
Output is the IDENTICAL deduped/sorted (controller, dependent, label) set:
verified byte-identical across all cfg unit+integration suites, the
cdg-snapshot oracle, and bench --check fingerprints (unchanged). The PDF
unions a label SET per (controller, dependent) pair, preserving the
multi-label rows the old per-row dedup kept on opposite-sense (goto-cycle)
arms. buildArmSenses, labelFor, the final sort and the maxEdges truncation
cap are kept verbatim; the post-order walk is iterative so a chain-deep
post-dom forest cannot overflow the stack.
Adds three regressions: multi-label-per-pair preservation, the literal
self-edge / NO_IPDOM seed guard (a !== x), and a fan-into-chain perf
tripwire (linear vs the former quadratic up-walk).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(cfg): record the reaching-defs WTO no-go decision (#2195)
Weak-topological-order / loop-aware iteration (Bourdoncle 1993) was
evaluated as the fix for the O(blocks²) deep-loop-nest blow-up and
rejected: a faithful WTO solver was 104/104 byte-identical to the RPO
worklist but 0% faster — the cost is inherent dense-set propagation +
lattice merges, not visitation order, and the loop-body-skip shortcut is
unsound on irreducible (goto) CFGs. Document this at the RPO-order site
and the emit.ts revisit-ceiling constant so the shipped blocks×64 bound
reads as the sound backstop it is, with SSA-sparse reaching-defs named as
the deferred real fix. Comment-only; no behavior change.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(cfg): proactive visitor nesting-depth guard + observable CFG skips (#2195)
The CFG visitors are recursive-descent with no shared base, so a
pathologically nested function (machine-generated / adversarial) could
overflow the worker's native stack — a nondeterministic RangeError that
escaped to the language-group catch and silently dropped EVERY remaining
file's CFG.
Guard it proactively: CfgBuilder tracks live recursive-descent nesting
depth via enterNesting/exitNesting, called at each visitor's visitBody and
visitSeq choke points (visitBody covers nested control constructs incl.
else-if ladders; visitSeq covers deeply-nested bare blocks). Exceeding
MAX_CFG_NESTING_DEPTH (500, far below the ~1.2k+ native limit and far above
real code's ≤~50) throws a typed, DETERMINISTIC CfgNestingDepthError instead
of waiting for the engine's nondeterministic overflow.
collectFunctionCfgs now isolates the build PER FUNCTION: the depth bail or
any other throw is caught, counted, and skipped — one bad function no longer
loses the whole file's CFGs. CollectedCfgs.skipped widens from a bare number
to reason-counted buckets (tooManyLines / tooDeeplyNested / buildError). The
worker stops discarding that count (parse-worker.ts), aggregates it
per-language onto ParseWorkerResult.cfgSkipped (survives the parse cache via
slim's `...result`), and mergeChunkResults merges + warns per-language so a
CFG coverage gap is observable, not silent.
Behavior-preserving on normal code: the guard never fires below 500 nesting,
so the cfg unit+integration suites (731), the CDG/RD/CFG snapshots and the
bench --check fingerprints are all byte-identical. The worker stackSizeMb
4→16MB bump shipped earlier (
|
||
|
ChunxueLi
|
bd1d446baa
|
fix: detect single-ancestor method overrides in MRO processor (#2199) | ||
|
Goutham Krishna Mandati
|
4c73b18387
|
feat(mcp): add trace tool for shortest call path between symbols (#2173)
Some checks are pending
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Scorecard / Scorecard analysis (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
* feat(mcp): add trace tool for shortest call path between symbols (#1821) Implement the \ race\ MCP tool and \gitnexus trace\ CLI command that finds the shortest directed call path between two symbols using BFS over CALLS + HAS_METHOD edges. - MCP tool definition in tools.ts with READ_ONLY annotations - Directed BFS in local-backend.ts with parent-map path reconstruction - Symbol resolution via resolveSymbolCandidates (name/UID/file-hint) - Gap reporting with furthest reachable node and depth tracking - CLI wiring: gitnexus trace <from> <to> [--from-uid] [--to-uid] [--depth] - i18n keys in en.ts and zh-CN.ts + help-i18n.ts registration - ARCHITECTURE.md tools table entry - 16 unit tests (11 BFS core + 5 CLI wiring) * test(mcp): account for trace tool in tools.test.ts count The trace tool makes GITNEXUS_TOOLS length 15; update the hardcoded count, add 'trace' to the expected-names list, and refresh the stale "13 tools" comment and it() title. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(mcp): sanitize trace maxDepth to reject 0/NaN/negative `Math.min(params.maxDepth ?? 10, 30)` had no lower bound and `??` does not recover 0 or NaN, so `--depth 0|-5|abc` made the BFS loop run zero iterations and return a false `no_path`. Clamp at the real boundary with a `Number.isInteger && > 0` guard (the MCP inputSchema minimum is advisory only), and reject a non-numeric `--depth` in the CLI up front rather than forwarding NaN. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(mcp): check trace target before applying test-file filter The `isTestFilePath` filter ran before the target-equality check, but resolveSymbolCandidates does not exclude test-file symbols. A target (or a required hop) that lives in a test file was therefore skipped under the default includeTests=false and produced a false no_path with a misleading dynamic-dispatch suggestion. Match the explicitly-requested target first; non-target test-file nodes are still filtered. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(mcp): bound trace BFS with per-level LIMIT and visited cap The per-level query had no LIMIT and the visited set was uncapped, so a high-fanout hub could materialize an unbounded frontier. Cap per-level rows (interpolated LIMIT — Kuzu does not bind LIMIT) and the total visited set; either cap sets a `truncated` flag so a resulting no_path reports that the search was cut short rather than implying the graph was exhausted. Note: the sibling impact BFS shares the same unbounded pattern; applying the cap there is deferred (out of scope for this PR). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(mcp): clarify trace traverses call + class-member edges trace was advertised as a "shortest call path" but also traverses HAS_METHOD (class→member) containment edges so a class-rooted trace can descend into its methods. Keep that capability (consistent with impact/ context) and make the docs honest: rename EDGE_TYPES→TRAVERSAL_EDGE_TYPES, state the call + class-member traversal in the MCP/CLI/i18n/ARCHITECTURE descriptions, and note each hop's edge type is reported in edges[]. No behavior change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(mcp): set status:'error' on trace failure responses Every trace return path sets a `status` discriminator except the caught-error path, so a consumer switching on `result.status` saw undefined on failure. Add status:'error' to both the backend trace() catch and the CLI traceCommand catch. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(mcp): return a friendly error for non-string trace from/to A non-string from/to reaching resolveSymbolCandidates surfaced a low-level "x.includes is not a function" via name.includes. Guard the four name/uid params at the top of _traceImpl and return a structured status:'error' with a clear message instead. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(mcp): single row-decode + drop dead field in trace BFS Decode each BFS row once into named locals instead of repeating `(row.x ?? row[N])` across the two parent.set calls and the furthest-tracking. Drop the `type` field from the parent map value (it was written but never read), and rename the internal `deepestInfo` to `lastReached` for accuracy (the output field `furthest` is unchanged). Pure refactor — no behavior change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(cli): dedicated trace includeTests i18n key + guard coverage `trace|--include-tests` reused the impact help key, so rewording the impact option would silently change trace's help text. Add a dedicated help.option.trace.includeTests key in en + zh-CN and repoint it. Add CLI coverage for the (already symmetric) --from-uid/--to-uid flag-value guard and for --include-tests forwarding. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(mcp): faithful BFS mock + expand trace coverage Fix makeResolveMock: concatenate neighbors across ALL frontier ids (it returned only the first node's, so a multi-node frontier was unmodelled) and key the UID branch on params.uid (the old query-text match never fired). Add coverage: shortest path through the second frontier node (proves the mock fix), confidence floor fallback, HAS_METHOD traversal with a mixed edge-type chain, no_path furthest:null, and from_file disambiguation. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * style(trace): apply root prettier formatting to trace files The root `quality / format` gate (prettier --check, printWidth 100) runs on the full repo and flagged the trace sources/tests (the local config masks it). Reformat to root style — no behavior change; trace + tools suites and tsc stay green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(skills): document the trace tool for AI agents Add `trace` to the GitNexus skill docs so agents reach for it instead of hand-chaining context/impact hops. The guide gains a Tools Reference row and a "shortest path between two symbols" subsection (params, result shape, status/furthest/truncated semantics); the debugging skill gains a "how does A reach B?" pattern row and a trace tool example. Mirrored to the .claude and claude-plugin copies (byte-identical) and the cursor copy (compact style). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(mcp): drop unused trace test fixtures (CodeQL js/unused-local-variable) CodeQL flagged two unused locals in the trace BFS tests: the top-level SYMBOL_C and a SYMBOL_D inside the maxDepth test (both defined, never referenced). Remove them. No behavior change — 58 trace/tools tests stay green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Gergő Magyar <gergomagyar@icloud.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Parafee41
|
1967512211
|
fix(cli): preserve trailing spaces in git roots (#2192) | ||
|
Gergő Magyar
|
fb068a9480
|
fix(group): pin repos during sync so large groups resolve cross-links (#2191)
* fix(lbug): pin repos to exempt them from automatic pool eviction [#2189] Add a pinnedRepos set and pinRepo/unpinRepo to the LadybugDB pool adapter. evictLRU and the idle-timeout sweep skip pinned repos; closeOne clears the pin on teardown so explicit close always wins and pins never leak across operations. Behavior is byte-identical when nothing is pinned. Bounded multi-repo callers (group sync) can now keep more than MAX_POOL_SIZE repos resident through deferred cross-repo resolution. * fix(group): pin repos during sync so >MAX_POOL_SIZE groups resolve [#2189] syncGroup now pins each repo immediately after initLbug and releases the pin (unpin then close) in the finally. This keeps every group member resident through the deferred manifest/workspace resolution that runs after the init loop, so cross-links anchor to real graph symbols instead of falling back to synthetic UIDs when a group has more than MAX_POOL_SIZE repos. Release is unpin-before-close plus closeOne's own pin-clear, so pins never leak across syncs in the long-lived MCP server even on error. * style(test): apply prettier formatting to #2189 test files * fix(review): apply autofix feedback Clarify the pinRepo docstring: the pin does not survive teardown (closeOne clears it) and the repoId must match the key passed to initLbug. Addresses a code-review finding that the prior 'or later holds' wording contradicted closeOne's unconditional pin-clear. * refactor(lbug): reference-count pool pins so overlapping holders are safe [#2189] Change pinnedRepos from Set<string> to Map<string,number>. pinRepo increments the lease count; unpinRepo decrements and deletes the key at 0 (flooring at zero, unknown-id no-op). evictLRU, the idle sweep, and closeOne are transparent to the swap (has()/delete() keep their semantics: skip while count>=1, force-clear on teardown). A boolean Set could not represent two simultaneous holders, so the first release wrongly cleared a pin another holder still needed — the concurrent overlapping group_sync teardown race from the PR #2191 review (Finding 1). Reference counts let two windows of one sync, or two concurrent syncs sharing a repo, coexist safely: the repo stays exempt until the last lease releases. * refactor(lbug): pinRepo returns a leak-proof release disposer [#2189] pinRepo now returns a release() disposer (mirroring addPoolCloseListener) that releases its own lease exactly once — a double-call is a guarded no-op, so it can never over-decrement a sibling holder's reference count. Callers can use the leak-proof pattern `const release = pinRepo(id); try { … } finally { release(); }`. unpinRepo stays exported for explicit pairing. Addresses the PR #2191 review's P3: the exported pin primitive had no built-in pairing, so a caller that forgot to unpin would disable eviction for a repo permanently. * refactor(group): windowed manifest resolution bounds sync pool residency [#2189] Replace whole-sync pinning with windowed deferred resolution. The init loop extracts contracts without pinning (repos evict naturally); manifest links are pre-sorted and partitioned into windows whose referenced in-group repos number <= getMaxResidentRepos(), and each window re-inits + leases only its own repos, resolves, then RELEASES the leases (not closeLbug — released repos stay evictable for the LRU, which avoids stomping a concurrent MCP reader). Peak per-sync pool residency is now bounded by getMaxResidentRepos() distinct repos regardless of group size, removing the unbounded-mmap crash risk the PR #2191 review flagged (Finding 3) — without a new magic-number threshold (it reuses MAX_POOL_SIZE via an intent-named accessor). #2189 stays fixed: each window resolves against live, freshly-leased pools, so cross-links anchor to real graph symbols. partitionManifestWindows is a pure, unit-tested function (every link in exactly one window — the contract-dedup invariant). New sync-windowed-resolution.test.ts asserts the partition bound and, through the real pool, that concurrently-open Databases never exceed the resident cap for a group larger than it. Rewrote the sync.test.ts pinning block (init loop no longer pins; per-window lease/release; release-not-close). |
||
|
Gergő Magyar
|
7c3d4e6862
|
feat(pdg): control dependence — post-dominators + CDG (Ferrante) [M5 #2085] (#2188)
* feat(pdg): add CDG + POST_DOMINATE edge types (M5 #2085) * feat(pdg): post-dominator tree on reverse CFG (M5 #2085) * feat(pdg): Ferrante control-dependence over the post-dom tree (M5 #2085) * feat(pdg): emitFileCdg + optional POST_DOMINATE debug edges (M5 #2085) * feat(pdg): wire CDG emission in-phase + pdgModeMismatch CDG-cap stamp (M5 #2085) * test(pdg): CDG snapshot + end-to-end pipeline answerability (M5 #2085) * fix(review): apply autofix feedback (M5 #2085) * fix(pdg): label CDG edges by controller arm sense, not edge kind (#2188 F1/F2/F4) Tri-review (with Codex as the independent engine) found the CDG 'T'/'F' label was wrong for the commonest control flow: the M1 TS visitor wires a condition's fall-through FALSE arm as `seq`/`loop-back`, but `branchSense` mapped both to 'T', so guard clauses, if-no-else, and loop `break` got 'T' instead of 'F' (F1, P1). The structural CDG edges were correct; only the label — the AC3 "under what condition does X run?" answer — was wrong. - F1: replace edge-kind `branchSense` with controller-arm-sense `labelFor`. An ambiguous fall-through edge (seq/loop-back) takes the COMPLEMENT of its source block's explicit cond-true/cond-false sibling arm. This correctly handles do/while (loop-back = TRUE arm) and inner-if-in-loop (loop-back = FALSE arm) — the ambiguity a kind→label table cannot resolve. Adds real-parser regression tests (the hand-built tests used a fictional cond-false edge and missed it). - F2: correct the false "sound over-approximation that never drops a real dependence" claim in post-dominators.ts — exit-unreachable regions both drop and invent control dependences (latent for the current TS visitor, which keeps EXIT reverse-reachable). Reframe the exit-less-loop test to characterize, not bless, the degenerate behavior. - F4: make the AC2 property-test reference compute post-dominance INDEPENDENTLY (node-removal reachability, no shared code with post-dominators.ts), so a post-dom direction bug can no longer pass both the impl and the reference. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ci): root-prettier format + run-analyze pdg stamp gains maxCdgEdgesPerFunction (#2085) Two deterministic CI failures from the M5 CDG work: - quality/format: basicblock-roundtrip.test.ts failed CI's root `prettier --check .` (the pre-commit hook uses the gitnexus-local prettier config, which differs); reformatted with the root config. - tests/ubuntu/coverage: run-analyze.test.ts pinned the resolved RepoMeta.pdg shape (DEFAULTS) and the all-zero cap override without the new maxCdgEdgesPerFunction key (default 5000); added it so resolvePdgConfig toEqual and pdgModeMismatch(DEFAULTS) pass. (The stale-test sweep missed this file in PR #2188 — same trap M2 hit.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(mcp): add pdg_query tool definition (controls/flows modes) [M6 #2086] * feat(mcp): pdg_query backend — controls (CDG) + flows (REACHING_DEF) + e2e test [M6 #2086] * feat(mcp): document PDG edges + pdg_query (schema, cypher, skill, --pdg-gated ai-context) [M6 #2086] * fix(mcp): correct pdg_query symbol-anchor lower bound + harden inputs [PR #2188 review] Tri-review (Codex + adversarial + correctness lanes) of the M6 pdg_query surface found the symbol-anchor window over-includes a neighbor function's block. The upper bound was widened to the 1-based BasicBlock basis (symEnd+1) but the lower bound was left 0-based, so a block on the line directly above the target function leaked into the result. Shift both bounds +1 ([symStart+1, symEnd+1]) so the window is the function's true block span. Also from the same review: - pdg_query no longer throws on a no-arguments MCP call: the dispatch passes raw `params`, so default it to {} → a clean mode-validation error instead of a TypeError. (`explain` shares this latent pattern — pre-existing follow-up.) - tools.ts: the controls-mode description no longer hard-codes the 'F' branch sense for guards — `if (!ok) return;` rides the predicate's 'T' arm; the guard:true flag is label-agnostic (regex on the dependent block text). Tests: a hand-seeded adjacency regression (verified failing without the lower-bound +1) + a no-arguments validation test. Skill doc updated to document the two-sided [symStart+1, symEnd+1] window. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(mcp): drop always-true anchor conditional in pdg_query [CodeQL #2188] CodeQL alert 756 flagged `...(anchor ? { anchor } : {})` in _pdgQueryImpl as a useless conditional: `anchor` is unconditionally assigned in both the file-path and symbol branches before the return (the not-found/ambiguous/no-layer paths return earlier), so it is always truthy. Drop `| undefined` from the declaration (TypeScript definite-assignment holds across both branches) and emit `anchor` directly. No runtime change — the `anchor` field was already present on every result. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(cli): add hasPdg to the noStats bridge expectation [#2188] The M6 work threaded `hasPdg: options.pdg === true` into the AIContextOptions passed to generateAIContextFiles on the --skills regeneration path, but this test's strict .toEqual expectation predated it (4 keys vs 3 → CI failure). Add `hasPdg: false` (the value on this non---pdg path). The assertion stays strict; the #1477 noStats bridging it guards is unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(cli): collapse generateGitNexusContent params to an options bag [#2188] The function had grown to 9 positional params; reaching `hasPdg` meant passing six `undefined`s (the M6 review's maintainability flag). Collapse params 3-9 (generatedSkills, groupNames, noStats, skipSkills, runnerPath, defaultBranch, hasPdg) into a `GitNexusContentOptions` object with the defaults moved to destructuring. The body is unchanged (same local names); the single production caller and the test calls become self-documenting named fields. Pure refactor — generated AGENTS.md/CLAUDE.md content is byte-identical. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(cfg): skip CDG for exit-unreachable CFGs (unsound post-dominance) [#2188] M5 review P2: computePostDominators roots only at cfg.exitIndex and nothing enforced that EXIT is reachable from every block. For an entry-reachable region that cannot reach EXIT (a non-terminating loop, or a multi-terminal CFG a future visitor might emit) the EXIT-rooted reverse walk degenerates — it both drops real control dependences and invents spurious ones. Add a pure precondition predicate `isExitReachableFromAllBlocks` (co-located with the algorithm it guards) and gate it in emitFileCdg: a CFG that violates it is skipped for CDG (counted as skippedUnsoundFunctions + one onWarn), while its CFG and REACHING_DEF projections — which do not depend on post-dominance — are kept. A CDG-specific gate, not a widening of isEmitSafeCfg, so the blast radius is exactly the unsound CDG. The current TS visitor always satisfies the precondition (every loop gets a structural header→loopExit edge), so CDG output for real fixtures is unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(cfg): bound computeControlDependence materialization (heap parity) [#2188] M5 review P2: unlike computeReachingDefs (maxFacts) and the emit-side edge cap, computeControlDependence materialized the full deduped seen/out before emitFileCdg's per-function cap could trim it — O(edges × post-dom depth) heap for a deeply nested function. Add a `maxEdges` ceiling (default 0 = unbounded) returning {edges, truncated}, mirroring computeReachingDefs's {facts, truncated}. The ceiling is checked before pushing a new unique edge, so `truncated` means a genuine overflow (not merely "reached cap"). emitFileCdg passes a FIXED materialization ceiling (8× the default edge cap) — deliberately NOT derived from the runtime edge cap, because CDG's materialization IS the deduped-edge quantity the cap reports on (deriving it would pre-truncate that set and lose the exact dropped count). A ceiling hit is surfaced via onWarn + the truncated flag — never silent. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(mcp): share resolveBlockAnchor; fix explain's anchor off-by-one [#2188] M6 review P2 (duplication) + the flagged pre-existing _explainImpl correctness follow-up. _pdgQueryImpl and _explainImpl each carried a near-identical symbol↔block anchor resolver that had DRIFTED: pdg_query used the corrected [symStart+1, symEnd+1] window (BasicBlock startLine is 1-based, the symbol span 0-based) while _explainImpl still used [symStart, symEnd] — dropping a taint source on the function's final line AND leaking a neighbor's block on the line directly above. Extract one `resolveBlockAnchor` helper, used by both, that applies the correct window and a single (bare) clause convention (callers compose their own WHERE). This removes ~50 duplicated lines and fixes explain's anchor in one place. A hand-seeded characterization test (taint-explain Block 4) pins both bounds — verified to FAIL on the pre-fix window (it returned the line-10 neighbor instead of the line-15 final-line source). Existing taint-explain + pdg-query suites are unchanged (their fixtures have interior sources/sinks). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(mcp): pdg_query reports "status unknown" when the layer can't be confirmed [#2188] M6 review P3 (Codex): when meta is UNREADABLE and the bounded global existence probe returns zero rows of the edge type, _pdgQueryImpl asserted "no PDG layer" — but a genuinely edge-free layer (all-linear functions) is indistinguishable from a missing one via that probe. Soften only that fallback path to an inconclusive "PDG layer status unknown — was this repo indexed with --pdg?" note. The meta-stamped path (stamp present, cap absent ⇒ layer truly missing) keeps the definitive "no PDG layer" wording. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(mcp): cover pdg_query ambiguous / pagination / Windows-path gaps [#2188] M6 review test-gap follow-ups, all hand-seeded with controlled data: - ambiguous symbol name → status:'ambiguous' + ranked candidates shape (uid/name/filePath/score), never a silent guess; - total/truncated page boundary in both directions (limit below the match count sets truncated with the full total; limit above it omits truncated); - a Windows-style filePath containing ':' resolves and fnLineOf decodes the function-line segment correctly (split-from-right past the drive letter). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(skills): ship gitnexus-pdg-query skill mirrors + add pdg_query to the guide [#2086] M6 bundled pdg_query into this PR, but the skill shipped only in the canonical gitnexus/skills/ root. Mirror it (byte-identical) to the two hand-maintained roots the sibling taint skill uses — .claude/skills/gitnexus/ and the plugin — so Claude Code + plugin users get it too. Also extend the gitnexus-guide tool reference (all 3 copies, now byte-identical): add a `pdg_query` row + a "Control & data dependence" section mirroring the taint/`explain` section, and reconcile the pre-existing drift where only the .claude copy carried the `check` tool row (a real registered tool) — all three now list it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(architecture): refresh CFG/PDG section for the full M1–M6 stack [#2086] The PR body had deferred the "ARCHITECTURE docs refresh" to #2086; now that M6 ships here, do it: - MCP tools table gains `explain` and `pdg_query` (were absent). - "Optional CFG/PDG emission" was M1-only; rewrite to cover the whole opt-in stack — M1 CFG, M2 REACHING_DEF, M3/M4 taint, M5 CDG (Ferrante over CHK post-dominators, with the exit-unreachable skip), M6 read surface (pdg_query + explain, anchored + LIMIT-bounded, shared resolveBlockAnchor) — and note the no-Function→BasicBlock-edge join. - LadybugDB schema notes the `--pdg` additions: the `BasicBlock` node table and the CFG/REACHING_DEF/CDG/TAINTED/SANITIZES/TAINT_PATH relation types, kept out of the default VALID_RELATION_TYPES / web schema. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Gergő Magyar
|
96dc368d96
|
fix(ci): align tree-sitter readiness + grammar-update workflows on a shared manifest (#858) (#2187)
Some checks are pending
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Scorecard / Scorecard analysis (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
* chore(ci): add shared vendored-grammars manifest; monitor reads it .github/vendored-grammars.json is the single source of truth for the vendored tree-sitter grammars (c/swift/kotlin/dart/proto): name, upstream coords, and policy holds. update-vendored-grammars.mjs now builds its GRAMMARS map from the manifest (behavior-preserving — same exported shape). Adds manifest-agreement tests so the loader can't silently skew from the file. * fix(ci): classify vendored grammars from manifest, drop bare "?" (#858) The readiness report decided "is this vendored?" via is_vendored_pin (a file: package.json spec) — but the 5 vendored grammars aren't in package.json, so they were misrouted through the npm path and rendered bare "?" for ABI (read from an empty node_modules), plus a spurious "? (fetch failed)" for github-only proto. Now vendored grammars are classified by membership in the shared manifest and their ABI is read from gitnexus/vendor/<name>/src/parser.c (always in a checkout). github-only vendored grammars skip the npm peer-dep fetch; the tree-sitter-c hold is surfaced from the manifest (held, not plain "Ready"); and every remaining unintrospectable value renders a labeled token, never a bare "?". --assert-current now covers the vendored grammars too instead of skipping them. Adds a stdlib unittest suite incl. a manifest⇄vendor-dir consistency guard. * docs(ci): document the shared vendored-grammars manifest Both tree-sitter workflow headers now point at .github/vendored-grammars.json as the shared source of truth; the readiness workflow gains a PR-path trigger on the manifest + test, and runs the readiness unit tests on validation events. CONTRIBUTING.md documents the manifest contract under CI automation contracts. * fix(review): apply autofix feedback - Guard manifest reads in both scripts with a clear error (was an opaque module-import traceback that crashed the script and test collection). - Never render a bare "?": relabel the npm-path ABI/version/peer sentinels and the vendored upstream-ABI miss to labeled tokens; the report is now ?-free regardless of node_modules/network, and the test is hermetic. - Add a VENDORED_NAMES ⊆ GRAMMARS guard + manifest-missing error test. - Drop now-dead is_vendored_pin/is_vendored/_(vendored)_. - Compose held + out-of-range vendored blocker reasons instead of overwriting. - Reword the shared-manifest docs to not over-claim shared upstream coords. * fix(ci): apply root prettier formatting to mjs + ts test The quality/format gate runs root `prettier --check .` (printWidth 100, the gitnexus-local config differs and falsely passed locally). * test(ci): make both tree-sitter scripts testable offline The scripts hit live npm/GitHub, which makes the report run flaky and the monitor's detect/apply logic untestable. Add hermetic seams: - readiness: --offline flag (+ GITNEXUS_TS_READINESS_OFFLINE env) no-ops the npm registry + upstream fetches; the report renders deterministically (vendored ABIs from the repo, npm columns marked 'offline', no bare '?'). 3 tests assert an offline run touches ZERO network (urlopen patched to raise). - monitor: detect() and apply() accept injected deps (vendoredVersion/ resolveUpstream/fetchSource/readAbi) so the newer/ABI/hold gating runs offline with fixtures; apply gains --dry-run (validates but writes nothing). 6 tests cover newer/same-version/held-c/ABI-15/applicable + a no-mutation dry-run. * fix(review): keep --assert-current hermetic + harden the no-bare-? invariant Tri-review findings (PR #2187): - P2 REGRESSION: --assert-current (documented 'hermetic and offline', run in CI without --offline) routed the 5 vendored grammars through vendored_drift_summary, which fetches upstream parser.c + commit sha — 10 discarded network calls per run. Fix: read the vendored ABI locally via a new vendored_abi_from_repo() helper (also used by vendored_drift_summary). Now verifiably network-free. - Unify the upstream-ABI miss sentinel: prose said 'n/a (generated at build)' while the matrix said 'n/a' — and 'generated at build' is a wrong cause (swift HAS a committed parser.c). Both now render neutral 'n/a'. - Fix the stale assert_current docstring claiming swift is prebuilt-only/no parser.c. - Guard the last latent bare-? path (vendor package.json missing 'version'). Tests: AssertCurrent (network-free guard + out-of-range via the new injection point), malformed-JSON manifest, detect() error-path, explicit npm/github undefined assertions. 17 Python + 15 vitest, all hermetic. * fix(review): use a single unittest import style (CodeQL 753) CodeQL py/import-and-import-from flagged `import unittest` + `from unittest import mock`. Collapse to `from unittest import TestCase, main, mock`. * fix(review): explicit raise in _matrix_row (CodeQL 754) CodeQL py/mixed-returns flagged the implicit fall-through after self.fail() (which it doesn't model as NoReturn). End with an explicit raise AssertionError. * test(review): replace non-null assertions with a must() guard @typescript-eslint/no-non-null-assertion flagged 4 `!` operators. Add a narrowing must<T>(value, message) helper (throws on undefined) and a named baseResolveUpstream, removing every non-null assertion. * fix(review): unguessable heredoc delimiter for the report output The report embeds the manifest `hold` field (fork-PR-editable); a fixed DRIFT_EOF delimiter in a hold value could close the $GITHUB_OUTPUT heredoc early and inject output keys. Use DRIFT_EOF_$(openssl rand -hex 16) — a value the report cannot contain. (Randomized delimiter over base64: keeps REPORT raw markdown, no consumer-side decode.) * fix(review): scope issues:write to scheduled runs (two-job split) GitHub Actions has no step-level permissions, so the only way to keep PR runs (incl. forks) from receiving `issues: write` is to split the job. A `report` job (contents:read, all events) renders the report + the PR `:⚠️:` and exposes report/exit_code as job outputs; a schedule-only `upsert-issue` job (needs: report, issues:write, no checkout) consumes them for the issue upsert + close. The 'Check upgrade readiness' check name is preserved. * fix(review): launder npm-version '?' in disposition prose The disposition bucket prose interpolated r['npm_version'] raw, so a successful 200 npm /latest response lacking a 'version' key would render a bare '?' (the matrix cell already laundered it). Add npm_version_label ('unknown' for '?') and use it in all five bucket renderers. Test a version-less npm response. * refactor(review): load_vendored_manifest returns only the consumed 'hold' The readiness script reads only the grammar names + 'hold'; the 'key' and 'upstream' fields were phantom data (upstream-drift coords live in the script's own GRAMMARS map). Narrow the return to {hold}. * fix(review): unify detect()/apply() 'newer' check for github grammars detect() compared the bare sha7 while apply() compared up.version (the full <base>-g<sha7> provenance string apply() also writes). After the bot re-vendored a github grammar once, detect() reported a perpetual false 'update available' while apply() correctly saw 'already current' — a noisy job summary + wasted --apply subprocess (the PR-exists guard absorbed it before any duplicate PR). Extract a shared isNewer(up, have) helper used by both. Tests cover equal- provenance (false), first-vendoring plain-version (true, not suppressed), and sha-advanced (true). Coupled with U12 (the detect⇄apply agreement assertion lives there once apply()'s not-newer path returns instead of process.exit). * test(review): cover main()'s out-of-range + prebuilt-only vendored ABI branches main()'s vendored-ABI classification reads through vendored_abi_from_repo (the local-read seam --assert-current uses), so patching it drives the 'Vendored (ABI out of range)' blocker branch and the prebuilt-only (vendored_abi None → 'prebuilt' cell, not '?') branch — neither reachable today since all 5 vendor dirs ship parser.c at ABI 14. * test(review): monitor-side manifest⇄vendor-dir consistency guard Mirror the Python consistency guard on the monitor side — the monitor consumes the same manifest and is the side that WRITES files from manifest `name`, so manifest/vendor-dir drift must fail CI here too. * fix(review): validate grammar names at manifest load (path-traversal guard) The manifest `name` is joined into gitnexus/vendor/<name> paths in both scripts (and apply() WRITES there), so reject any name not matching tree-sitter-[a-z0-9-]+ at the single load chokepoint — defense-in-depth even though the live trust boundary already prevents exploitation. loadManifestGrammars gains an injectable `raw` arg + export for testing; tests reject a '../etc' name in both scripts. * refactor(review): apply() throws ApplyExit; CLI maps to exit codes apply()'s 4 process.exit calls killed the vitest worker, blocking in-process tests of its error branches. Replace them with a thrown ApplyExit{code}; the not-newer (already-current) path returns `have` instead of exit(0). The isMain CLI block try/catches and maps ApplyExit.code → process.exit, so the monitor's subprocess contract (exit 0/2/3) is byte-identical (verified via subprocess smoke). Tests cover unknown-key=2, held=3, ABI-reject=3, and not-newer (returns current, no throw, no write). * refactor(review): extract vendored render helper; trim docstrings (<1000 lines) Extract the 'Vendored parsers' prose render into _render_vendored_section() so main() coordinates named phases rather than inlining a ~450-line monolith, and condense the most verbose docstrings/comments. The script drops from 1092 to 999 lines (under the 1000 bar the maintainability review flagged). Behavior-preserving: the deterministic --offline render is byte-identical before/after (verified in-place), --assert-current still passes, and the full unit suite is green. * fix(review): row-diff regex captures only the Status cell The change-detection regex captured the whole row tail as group 2, so any non-status cell drift (e.g. an upstream-ABI bump) emitted a false-positive 'change' line. Capture only the Status cell ([^|]+? before the final |$). The workflow parseRows regex and the Python _ROW_DIFF_RE stay byte-identical; the stability test now asserts group 2 is the status string (e.g. c → 'Vendored — held') and contains no pipe. * fix(ci): hoist intro string out of the list literal (CodeQL 755) The U13 extraction moved the 'Vendored parsers' intro paragraph (implicitly concatenated string literals) INTO a list literal, tripping CodeQL py/implicit-string-concatenation-in-list (reads as a possibly-missing comma between elements). Hoist it into a parenthesized `intro` variable. Render is byte-identical. |
||
|
bluerose
|
89ffa71a52
|
feat(cli): add --embeddings-baseurl/-model/-auth-token/-dims flags to analyze (#2140)
* feat(cli): add --embeddings-baseurl/-model/-auth-token/-dims flags to analyze Add four CLI flags to `gitnexus analyze` that configure a custom OpenAI-compatible HTTP embedding endpoint by setting the GITNEXUS_EMBEDDING_URL / _MODEL / _API_KEY / _DIMS env vars the HTTP embedding client already reads. Flags override env vars; env vars keep working as before. URLs are validated (http/https) and dims must be a positive integer. Prints "Using custom embedding endpoint: <url>" when a URL+model pair is configured, and warns when the flags are passed without --embeddings. The new env keys are added to the analyze snapshot/restore set so programmatic callers don't leak state. The non-secret flags are also accepted from .gitnexusrc; the auth token is intentionally CLI/env-only. * fix(analyze): set GITNEXUS_EMBEDDING_DIMS from CLI flags before module import schema.ts reads EMBEDDING_DIMS at module-load time via the static-import chain (analyze.ts -> run-analyze.ts -> schema.ts). The previous approach of setting the env var inside analyzeCommandImpl ran AFTER schema.ts had already loaded with the default 384, causing "Expected: 384, Actual: 4096" errors when using --embeddings-dims 4096. Fix: use Commander's preAction hook to set GITNEXUS_EMBEDDING_* env vars before the lazy import of analyze.ts triggers the schema.ts module load. * fix(analyze): use hook callback arg instead of this in preAction Commander v14 passes the command as first argument, not as this binding. * refactor(cli): rename --embeddings-* analyze flags to singular --embedding-* Aligns the custom embedding endpoint flags with the existing singular tuning flags (--embedding-threads/--embedding-device): --embedding-base-url, --embedding-model, --embedding-auth-token, --embedding-dims. Renames the derived AnalyzeOptions fields and the .gitnexusrc KEY_SPECS keys to match. Behavior-preserving; the GITNEXUS_EMBEDDING_* env vars are unchanged. Refs #2140 review. * fix(cli): validate and normalize --embedding-dims before module-load reads it The preAction hook wrote GITNEXUS_EMBEDDING_DIMS unvalidated, so an invalid value (abc/0/-5/0x10) threw from schema.ts during the lazy import — surfacing as a raw unhandled rejection on the synchronous program.parse path instead of a friendly error. And '1e3' slipped through: schema.ts parseInt froze the vector column at FLOAT[1] while the impl's Number-based check accepted 1000, so http-client requested 1000-dim vectors against a 1-dim column. Extract a dependency-free normalizeEmbeddingDims helper (strict /^\d+$/ + positive, trim-then-validate, canonicalized) shared by both the hook (CLI path, before module-load) and analyzeCommandImpl (direct-call path). All three readers — schema.ts, http-client, and this helper — now agree on one value, and invalid input gets a clean message instead of a crash or a silent mismatch. Refs #2140 review. * fix(cli): mask credentials in the custom embedding endpoint confirmation A base URL with userinfo (http://user:pass@host/v1) or a query token (?api_key=…) passed the new-URL + http/https validation and was printed verbatim in the 'Using custom embedding endpoint:' line, leaking the secret to terminal scrollback and CI logs. Route it through the existing safeUrl() (now exported from http-client) which strips userinfo + query, keeping protocol/host/path. Single source of truth — no second sanitizer. Refs #2140 review. * fix(cli): drop the ineffective embeddingDims .gitnexusrc key embeddingDims as a .gitnexusrc key silently did nothing: .gitnexusrc loads in analyzeCommandImpl, AFTER the lazy import already ran schema.ts's module-load read of GITNEXUS_EMBEDDING_DIMS, so a config value never sized the vector column. Remove it (config now fails closed on the key, like the auth token); URL/MODEL stay as config keys because they're read lazily at runtime. Dims remains available via --embedding-dims or GITNEXUS_EMBEDDING_DIMS. Refs #2140 review. * refactor(cli): narrow the analyze preAction hook to GITNEXUS_EMBEDDING_DIMS Only DIMS is read at module-load (schema.ts), so only it must be set before the lazy import. URL/MODEL/API_KEY are read lazily at runtime, so analyzeCommandImpl is their sole setter — and because the impl's env snapshot is taken AFTER this hook ran, leaving those three in the hook leaked them past restore. Drop them from the hook (the impl already sets+restores them), and capture/restore the pre-hook DIMS baseline via a postAction hook so a CLI --embedding-dims override no longer leaks into a later in-process program.parseAsync. Refs #2140 review. * fix(cli): gate the custom-endpoint confirmation on the embedding flags The confirmation collapsed into one if/else chain that emits at most one message reflecting the run's intent. Gating on embeddingsEnabled stops the 'Using custom embedding endpoint' line from printing on every analyze run when GITNEXUS_EMBEDDING_URL+MODEL merely happen to be set in the environment, and ordering the '--embeddings absent' note first removes the contradiction where it printed alongside 'Using custom embedding endpoint'. Refs #2140 review. * test(cli): cover the custom embedding endpoint flags Adds direct-call (analyzeCommandImpl path) coverage the original PR lacked: URL validation (empty/invalid/non-http), model/token emptiness, dims validation incl. the 1e3 regression, credential masking in the confirmation line, confirmation gating (absent --embeddings; ambient env must not trigger it), CLI-over-env precedence, and the GITNEXUS_EMBEDDING_* snapshot/restore round-trip. Complements embedding-dims.test.ts and http-client-safe-url.test.ts. Refs #2140 review. * test(cli): e2e-cover the --embedding-dims crash path on the real CLI The dims-validation fix lives in the commander preAction hook, which only fires on the program.parse path; the direct analyzeCommand() unit tests bypass it. Add a subprocess e2e (run via tsx, no build) asserting that invalid --embedding-dims (abc/0/-5/1e3/3.5) produces the friendly flag-named error and exit 1 — NOT the raw schema.ts module-load throw that the original bug surfaced. Cases exit inside the hook (no repo/import/pipeline), so they're deterministic and fast. Updates the unit-suite comment to point at it. Refs #2140 review. --------- Co-authored-by: Gergo Magyar <gergomagyar@icloud.com> |
||
|
Minidoracat
|
912285064a
|
perf(hooks): cmdline-first Linux db-lock scan, drop the lsof fallback (#2180) (#2183)
* perf(hooks): cmdline-first Linux db-lock scan, drop the lsof fallback (#2180) The probe's Linux scan was O(processes × fds) — stat every fd of every process — so on a busy host it blew its budget and fell through to lsof, which then timed out (~2 s) and fail-closed. Every Grep/Glob/Bash hook spent ~2 s of CPU to conclude 'couldn't tell'. Rewrite linuxProcScanFindGitNexusServer (name kept; return type now tri-state 'owned' | 'not-owned' | 'timeout') as three phases: 0. /proc/<pid>/comm prefilter — kernel task->comm, never touches the target's memory maps; truncation-safe whitelist match (comm is capped at 15 visible chars). Calibrated to what a real server reports: @ladybugdb/core's worker_threads rename the main thread to 'MainThread', so that is whitelisted alongside the launcher basenames — omitting it would blind the probe to every server. 1. bounded /proc/<pid>/cmdline read (openSync+readSync, default 16 KiB with a floor of 4 KiB and a bounded escalation up to a hard ceiling) so a D-state holder cannot stall the hook and the mcp/serve mode token is never clipped off a long interpreter path. 2. dev+ino fd match for the 0–2 survivors only. Dispatch: 'owned' and 'timeout' both map to true. Timeout is now fail-closed (overload self-throttle) instead of falling through to lsof; the Linux lsof fallback is removed entirely. End-to-end semantics on busy hosts are unchanged (the old lsof arm also fail-closed there) — the ~2 s of wasted work and the orphan-spawning lsof are what's gone. macOS lsof+ps and Windows Restart Manager paths are untouched. Also: fix the budget parse bug (Number(raw && trim()) treated '0' as 1200; now parseInt-then-validate, with <= 0 an explicit immediate timeout) and add GITNEXUS_HOOK_PROC_ROOT so the Linux scan can be unit tested against a fixture procfs instead of the host's real /proc. Measured on a 583-process host with 6 background gitnexus mcp servers: owner detection 6–12 ms (was ~1216 ms + lsof timeout), ~100x. Tests: new hook-db-lock-probe.test.ts drives all three phases against a fake procfs (comm-truncation safety, Phase 0 trap, 4 KiB-boundary owner-miss guard, budget=0 immediate timeout, EACCES fail-closed) plus a live-/proc e2e that pins the fd-visible lbug-handle property against a real subprocess holder. The lsof/ps owner-detection suites are relaned to macOS (Linux no longer takes that path); the lsof orphan-reaping suite is removed (no lsof is spawned on Linux now) with a rationale note. Note: pre-commit typecheck skipped; remaining tsc errors are pre-existing on main (none in files touched here). * fix(hooks): honest EACCES verdict + real escalation coverage (#2183 review) Addresses the tri-review (maintainer + Codex): - [P2] Phase-2 fd-dir EACCES no longer claims 'owned'. /proc/<pid>/fd is owner-only (0500), so a cross-user/root gitnexus server serving ANY repo cleared Phase 0+1 and hit EACCES here, and the old catch returned 'owned' — falsely claiming it locks THIS repo's lbug (dev+ino never compared) and permanently suppressing augment. Split the failure shapes: ENOENT -> continue (raced away); EACCES/EPERM and transient EIO/ESTALE -> 'timeout' (unverifiable -> fail-closed, but honest, not a false ownership claim); ENOTDIR/other structural errors -> continue (not a real fd dir). Same fail-closed dispatcher outcome, no false 'owned', plus a GITNEXUS_DEBUG diagnostic so an operator can tell this skip path from a real owner. - [P2] The escalation test now actually iterates the escalation loop: the gitnexus token sits under 4 KB while the mode token is padded past GITNEXUS_HOOK_PROC_CMDLINE_MAX=4096, and a readSync spy asserts >1 read (the old 9 KB-under-16 KB-cap shape read once and never escalated). - escalation loop now re-checks the budget each iteration and returns a distinct timeout sentinel (never '' — an empty string would read as 'not a candidate' and could drop a real owner -> fail-open); the caller maps it to 'timeout'. - GITNEXUS_HOOK_PROC_ROOT is gated to test context so a stray production env export can't disable Linux owner detection (fail-open). - New uid-agnostic spy tests pin every fd-readdir errno branch (EACCES/EPERM/EIO/ESTALE -> timeout, ENOTDIR -> not-owned) regardless of the runner's uid (the disk chmod-000 tests no-op under root). Note: pre-commit typecheck skipped; remaining tsc errors are pre-existing on main (none in files touched here). * fix(hooks): drop the always-true outOfBudget presence guard (CodeQL #2183) CodeQL flagged `typeof outOfBudget === 'function' && outOfBudget()` as unneeded defensive code: readLinuxCmdline has a single caller (linuxProcScanFindGitNexusServer) that always passes the callback, so the typeof guard is dead. Drop it, leaving `if (outOfBudget())`, and note the invariant in the comment. Mirrored in the byte-identical plugin copy. * fix(hooks): parse numeric hook env with Number() so scientific notation works (#2183 review) getCmdlineMaxBytes and resolveLinuxProcBudgetMs parsed their env via Number.parseInt(raw, 10), so a value like "16e3" silently became 16 (parseInt stops at 'e') instead of 16000. Switch both to Number(String(raw).trim()), which honors scientific notation and is stricter on trailing garbage ("123abc" -> NaN -> default) — matching the repo-majority Number()+isFinite env idiom (src/cli/analyze.ts, src/core/embeddings/hf-env.ts). The two functions had DIFFERENT guard skeletons, so a verbatim swap would regress the budget: resolveLinuxProcBudgetMs used `raw != null ?` with no empty-string short-circuit, and Number("")===0 (vs parseInt("")===NaN) would make a set-but-empty GITNEXUS_HOOK_LINUX_PROC_BUDGET_MS="" resolve to budget 0 => immediate fail-CLOSED timeout => augment permanently skipped. Added the `&& String(raw).trim()` guard so ''/whitespace fall to the 1200 default while "0" still parses to the deliberate #2180 immediate-timeout vector. Exported both helpers for white-box tests (the values are otherwise only observable indirectly through scan timing) and added platform-independent coverage: "16e3"->16000, ""/whitespace->1200 (the regression guard), "0"->0, "123abc"/unset->1200, cmdline "8e3"->8000, "2e3"/""/unset->16384. Both byte-identical hook-db-lock-probe.cjs copies updated together. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(hooks): allocUnsafe the per-chunk cmdline read buffer (#2183 review) readLinuxCmdline allocated each per-chunk read buffer with Buffer.alloc(chunkCap), zero-filling memory that readSync immediately and fully overwrites. Switch the hot read buffer to Buffer.allocUnsafe — safe because readSync initializes exactly [0, bytes), only buf.subarray(0, bytes) is consumed, and Buffer.concat deep-copies that slice into `collected`, so the uninitialized tail can never reach the decoded cmdline. The zero-length `collected = Buffer.alloc(0)` is left unchanged (allocUnsafe gains nothing on a 0-length buffer). The existing D3 multi-chunk decode tests cover the read path and stay green. Both byte-identical hook-db-lock-probe.cjs copies updated together. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(hooks): harden the live /proc owner-detection e2e against CI flake (#2183 review) Two flake mechanisms, fixed without weakening what the e2e proves: - Holder readiness (the genuine false-FAIL): the pid-file poll was 200x25ms=5s; a loaded runner can be slow to spawn the child, tripping expect(holderPid).toBeGreaterThan(0). Widened to ~10s and raised the per-test timeout 20s -> 40s. - Scan budget (kept the assertion honest): the live scan ran at the default 1200ms. Because the dispatcher maps a budget 'timeout' to owned=TRUE, a busy host exhausting 1200ms before reaching the holder would make the assertion pass for the WRONG reason (a hollow timeout, not real fd-visible detection). Set a generous explicit 10000ms budget via the existing setEnv() helper so the module afterEach restores it (replacing the raw `delete process.env...` that bypassed env tracking). Raised the coarse timing regression guard to sit ABOVE the budget (5000 -> 15000) so a legitimately-slow-but-correct scan can't trip it. The load-bearing asserts (dev+ino fd-visibility precheck, owned===true for our own lbug) are unchanged. Verified the e2e executes (not skipped) on Linux. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(changelog): empty the root CHANGELOG [Unreleased] section Per maintainer request, nothing should sit under [Unreleased] in the root CHANGELOG.md (the release-owned changelog is gitnexus/CHANGELOG.md, whose [Unreleased] is already empty). Removes all three accumulated blocks — Fixed (#2163), Performance (#2180), Changed (KuzuDB->LadybugDB) — leaving only the [Unreleased] header above [1.5.3]. Pure removal; no release sections touched. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Gergő Magyar <gergomagyar@icloud.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Gergő Magyar
|
6dc6544365
|
fix(web): chat-only mode for large projects to prevent WebUI hang (#2178) (#2185)
* feat(web): add graph-load skip decision helper and node threshold (#2178) * feat(web): skip graph download in connectToServer for chat-only mode (#2178) * feat(web): add graphMode state and empty-graph chat-only handling (#2178) * feat(web): read and thread ?skipGraph URL param through connect flow (#2178) * feat(web): chat-only empty state with load-graph-anyway escape hatch (#2178) * style(web): apply prettier formatting to graph-load files (#2178) * fix(review): apply autofix feedback - Fail-safe confirm + authoritative node count (P1: prevent re-triggering the hang via Load-graph-anyway when count unknown) - In-flight guard on loadGraphAnyway (P1: double-fire) - Honor explicit ?skipGraph in onAnalyzeComplete and DropZone (R6/U4) - Extract buildGraphFromConnectResult shared helper (DRY across 3 connect sites) - Add tests: switchRepo skip path, threshold config override, loadGraphAnyway error path, confirm fail-safe, in-flight guard * fix(review): address tri-review findings - P1 (correctness+adversarial+risk): stop the cross-repo / F5 chat-only leak. loadGraphAnyway no longer persists ?skipGraph=0, and onAnalyzeComplete + DropZone no longer inherit a stale ?skipGraph for a different repo — both could bypass auto-detect and re-trigger the #2178 hang. ?skipGraph is now a bookmark hint honored only by the initial auto-connect; in-session repo changes auto-detect. - P2 (performance): auto-detect now also skips on edge count (edge-driven force-layout cliff), not just nodes; LARGE_GRAPH_EDGE_THRESHOLD default 50K. - P2 (julik): reset graphMode/chatOnlyNodeCount at the top of switchRepo so a failed switch can't leave a stale chat-only overlay. - P2 (julik): set serverBaseUrl before awaiting handleServerConnect in auto-connect so the Load-graph-anyway button isn't briefly a no-op. - P2 (risk): hide the misleading '0 nodes / 0 edges' stats in chat-only mode (Header + StatusBar). - P2 (performance): guard the GraphCanvas layout effect against the empty chat-only graph. - Tests: edge-threshold decision + connectToServer edge-trigger; load-anyway no longer asserts URL persistence. * fix(web): make Load-graph-anyway cancellable, unmount-safe, fail-safe confirm (#2178) - AbortController + mountedRef: cancel the in-flight download on unmount and guard every post-await setState by the mounted ref (an abort surfaces as a BackendError, not a DOMException AbortError, so name-checks would miss it) - Stale-result guard: a load-anyway that resolves after a concurrent switchRepo no longer clobbers the new repo's graph/mode/count - GraphCanvas confirm fails SAFE (treat as declined) when window.confirm is unavailable or throws, instead of silently proceeding into a large download * fix(web): make the AI agent and chat surface aware of chat-only mode (#2178) - buildDynamicSystemPrompt + createGraphRAGAgent take a chatOnly flag and append a note (both prompt branches) that supersedes VISUAL GROUNDING: the graph isn't loaded, [[Type:Name]] node citations won't highlight, prefer [[path:START-END]] - initializeAgent resolves chatOnly = opts ?? graphModeRef.current==='chatOnly': connect-flow callers (handleServerConnect, switchRepo, loadGraphAnyway re-init) pass it explicitly; lazy/settings re-inits fall back to live mode via the ref - loadGraphAnyway re-inits the agent (chatOnly:false) after a full load so the prompt drops the note - RightPanel shows a chat-only banner so the degradation is visible where AI output renders (en + zh-CN) * fix(web): streaming circuit breaker for graphs with missing size stats (#2178) - GraphTooLargeError + a mid-stream breaker in parseNdjsonGraphResponse: count nodes/relationships as they arrive and abort (cancel reader in try/finally, then throw) the moment either crosses its limit — reusing the existing node/ edge thresholds, no new magic constant. Throwing right after the offending push means a later error record in the same chunk can't pre-empt it. - fetchGraph gains optional maxNodes/maxEdges (off by default → existing callers unchanged). connectToServer arms them only for auto-detect downloads (skipGraph !== false) and catches GraphTooLargeError → chat-only, re-throwing every other error. This backstops the no-stats fail-open path that could otherwise re-trigger the original hang. * chore(autofix): apply prettier + eslint fixes via /autofix command --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> |
||
|
Gergő Magyar
|
9ff7337f1e
|
fix(mcp): rename query/cypher params so Claude Code can call them (#2186)
* fix(mcp): advertise search_query/statement params for query/cypher tools (#2175) Claude Code drops a tool-call argument named exactly 'query', making the query and cypher tools unusable from it. Rename the advertised required parameters to search_query and statement so the client transmits them. Handler-side backward-compat for the legacy 'query' key follows in the next commit. * fix(mcp): accept search_query/statement with legacy query fallback (#2175) Resolve the new advertised param names in the backend while still accepting the legacy 'query' key, so curl/HTTP, other MCP clients, the CLI, the group path, and the internal executeCypher() all keep working. Alias is normalized once at the callTool chokepoint (covers group-forward + search alias); query() and cypher() dual-read defensively. New name wins when both are supplied. Updates the required-error message and adds dual-accept unit + integration coverage. * fix(cli): pass canonical search_query/statement params to query/cypher tools (#2175) Stop the CLI from depending on the deprecated 'query' alias. No user-facing change — the positional args are unchanged and the backend accepts both keys. * fix(mcp): generators advertise search_query in query() examples (#2175) Update the three doc/example generators (ai-context AGENTS/CLAUDE block, skill-gen community skills, resources repo hint) so future analyze runs emit query({search_query: ...}) — the param name Claude Code actually transmits. Tests assert the new form is present and the legacy query({query: form is absent (the #2059 generator-test pattern). * docs(mcp): advertise search_query/statement in skill & guidance examples (#2175) Sync the committed agent-facing docs to the renamed params so a Claude Code agent following them emits the transmittable key: AGENTS.md/CLAUDE.md gitnexus block, the canonical gitnexus/skills/* source and its installed/plugin/cursor mirrors, and the README examples. Scoped rewrite of the two call prefixes only (query({query: -> search_query, cypher({query: -> statement). * style(mcp): prettier line-wrap for #2175 alias-resolution edits * fix(review): uniform search_query precedence + cypher empty guard (#2175) Code-review findings (correctness/adversarial/api-contract/maintainability consensus): - Group-mode query inverted the 'new name wins' rule: the callTool chokepoint backfilled params.query only when empty and the @group-forward read params.query directly, so a both-keys (or whitespace-legacy) group call let the legacy value win — unlike the local path. Replace the hidden param mutation with a self-contained 'search_query ?? query' resolve at the group-forward; precedence is now uniformly new-wins at every consumer site. - cypher() now returns the same friendly required-param error as query() when neither statement nor query is supplied, instead of a raw DB prepare error. - Document the legacy alias as permanent (third-party clients may send query=). Adds group-forward alias tests (both-keys + legacy-only), empty/whitespace search_query, the search-alias path, and the cypher empty-statement guard. * fix(review): non-string alias safety + drop stale chokepoint comment (#2175) Tri-review findings (correctness/adversarial/security + maintainability): - Non-string statement/search_query/query (the MCP envelope is not schema-validated) hit .trim() and threw TypeError to the server boundary instead of a friendly required-param error. Introduce resolveAliasString() (new name wins; non-string -> undefined) used by query(), cypher(), and the group-forward, so all three return the structured error. Empirically verified (123 ?? '' -> 123, (123).trim() throws) — this overrides a critic refutation that mis-read ?? as a string coercion. - Remove the stale query() comment claiming alias resolution happens at a callTool chokepoint; that mutation was removed earlier in this PR — each site resolves the alias itself. - Document GroupToolPort.query's intentionally-narrower required type vs the wider LocalBackend impl. Adds non-string and empty-new-key precedence tests. * fix(mcp): alias falls back to legacy value when new key is blank (#2175) PR #2186 review finding: resolveAliasString used `canonical ?? legacy` (nullish), so an explicitly empty/whitespace new-name value (e.g. {search_query:'', query:'real'}) won and was rejected — discarding a valid legacy value, contradicting the 'new name wins when both supplied' intent. Resolve to the first NON-BLANK string instead (new preferred when it carries a real value, else legacy). Covers query(), cypher(), and the group-forward (all route through the helper); non-string still resolves to a friendly error. Flips the presence-based test and adds whitespace/cypher/group fallback cases. * fix(mcp): drop legacy "query" mention from query/cypher schema descriptions (#2175) PR #2186 review finding: the search_query/statement inputSchema descriptions named the legacy "query" key — the exact arg Claude Code drops — and description text is read by an LLM choosing arguments, weakly nudging it to send "query". Trim the descriptions to their clean form and move the legacy-alias note to a code comment next to the schema (preserved for maintainers / non-CC clients). properties/required unchanged (no `query`). |
||
|
bluerose
|
cab63b508e
|
feat(mcp): add gitnexus mcp --http server with Streamable HTTP and legacy SSE transports (#2141) | ||
|
Copilot
|
60752de3e9
|
fix(ip): Scope write-route origin guard to server's own bound host (#2172)
* Initial plan
* Allow RFC1918 LAN origins in requireLocalhostOrigin
* Harden LAN origin parsing in middleware tests
* Refactor private IPv4 checks into shared server helper
* fix: scope origin guard to server's bound host, fix [::1], guard all write routes
- P1: Replace blanket RFC1918 trust with same-host check — only the server's
own bound host is allowed (via `createLocalhostOriginGuard(host)`), not
every device on the LAN.
- P2: Fix dead `::1` branch — compare against `'[::1]'` (with brackets) as
returned by WHATWG URL parser.
- P3: Update 403 message to "same-host origins" and doc comments.
- Out-of-scope: Add `requireLocalhostOrigin` to `DELETE /api/repo`,
`POST /api/embed`, `DELETE /api/embed/:jobId`, `DELETE /api/analyze/:jobId`.
- Tests: Add [::1] regression, ftp://, null origin, direct private-ip.ts
unit tests, and createLocalhostOriginGuard bound-host tests.
* fix: cast route params to string when middleware breaks type inference
* chore(autofix): apply prettier + eslint fixes via /autofix command
* fix(test): update rate-limit test regex to match multi-line embed route registration
* fix(ip): normalize boundHost and keep wildcard binds loopback-only
The same-host write guard compared the raw `--host` string to the WHATWG
`URL.hostname` of the Origin, so it silently 403'd legitimate same-host
browser writes for several bind forms:
- mixed-case hostnames (`MyHost.local` vs lowercased `myhost.local`)
- non-loopback IPv6 (`fe80::1` vs bracketed `[fe80::1]`, and non-canonical
forms like `fe80:0:0:0:0:0:0:1` / `::ffff:127.0.0.1`)
- wildcard binds (`0.0.0.0` / `::`), the CLI-advertised remote-access config
Canonicalize boundHost once at guard construction through `new URL().hostname`
(provably the same form the Origin is parsed into), and treat wildcard binds as
having no single host identity → writes stay loopback-only. We deliberately do
NOT fall through to RFC1918 for wildcards (that would re-open whole-LAN reach).
`createServer` now warns when bound to a wildcard so a remote-access deployment
is not silently write-blocked.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(ip): tag origin-block 403 with a machine-readable code and surface it in the web client
The write-route Origin guard returned a 403 with only a human-readable
`error` string, so clients could not distinguish an origin block from any
other 403. The hosted web client (gitnexus.vercel.app driving a local
backend) swallowed the resulting failure: the repo delete button caught the
error and only `console.error`'d it, so it silently no-op'd.
- Server: add a stable `code: 'origin_not_allowed'` discriminator to the 403 body.
- Web client: `assertOk` reads `body.code` and maps `origin_not_allowed` to a new
`BackendError` code `origin_blocked`; `formatBackendError` renders an actionable
i18n message (en + zh-CN) instead of the generic client message.
- Header: surface the delete failure inline instead of swallowing it to console.
Scope note: the embedding-status badge (EmbeddingStatus.tsx) hides in backend
mode (its `serverBaseUrl` guard), so it is not the surface where an origin-block
embed error appears; a dedicated backend-mode embedding-error surface is deferred
with the broader hosted-UI mode-awareness follow-up.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(ip): remove unused isValidIpv4Address export
`isValidIpv4Address` had no `src/` consumer — only its own test imported it.
It was a leftover from the reverted RFC1918-middleware approach (the same-host
guard now compares against a canonicalized bound host, not an IPv4 validity
check). Remove the export and its orphaned test block. `parseIpv4Octets` stays
(it feeds `isRfc1918PrivateIpv4`, which CORS `isAllowedOrigin` still uses).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Gergő Magyar <gergomagyar@icloud.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
azizur100389
|
50cda61a43
|
feat(setup): select coding agent integrations (#2168)
* feat(setup): select coding agent integrations * style: format setup agent selection * fix(setup): validate explicit agent selection --------- Co-authored-by: Gergő Magyar <gergomagyar@icloud.com> |
||
|
dependabot[bot]
|
6bd2d119fc
|
chore(deps)(deps-dev): bump esbuild from 0.28.0 to 0.28.1 in /gitnexus (#2182)
Bumps [esbuild](https://github.com/evanw/esbuild) from 0.28.0 to 0.28.1. - [Release notes](https://github.com/evanw/esbuild/releases) - [Changelog](https://github.com/evanw/esbuild/blob/main/CHANGELOG.md) - [Commits](https://github.com/evanw/esbuild/compare/v0.28.0...v0.28.1) --- updated-dependencies: - dependency-name: esbuild dependency-version: 0.28.1 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
Gergő Magyar
|
129bc84c0d
|
feat(taint): interprocedural taint via function summaries over resolved CALLS (#2084) (#2179) | ||
|
Minidoracat
|
0054496323
|
fix(hooks): wrap the augment CLI child in the orphan guard (#2163) (#2169)
Some checks failed
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Scorecard / Scorecard analysis (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
Devcontainer Smoke / Config-transform unit tests (push) Has been cancelled
Devcontainer Smoke / Build devcontainer image (push) Has been cancelled
* fix(hooks): wrap the augment CLI child in the orphan guard (#2163) Follow-up invited by the maintainer on #2165: the augment child (7s local / 12s npx) was the longest-lived unwrapped subprocess, exposed to the same SIGKILL-orphan mechanism fixed for lsof/ps. - Export resolveUnixGuardTimeout from the probe module (both copies, byte-identical); adapters share the same module instance, so the memo and lazy self-test still run at most once per hook process. - Wrap every CLI-executing branch of runGitNexusCli in the three probe-equipped adapters with the guard: budget ceil(inner/1000)+1 seconds with -k 1, strictly above each branch's inner spawnSync timeout, so the supervised path is unchanged and the wrapper only matters once the hook itself is SIGKILLed. Windows and no-guard hosts keep byte-identical argv. The plugin adapter's PATH-direct gitnexus branch (its most common production path) is wrapped too; the cheap which/where probe is not. - Cursor integration: debug-gated 'augment skipped: hook slots saturated' on the slot-starved early return. Its augment child stays unwrapped for now — that integration does not install the probe sibling (the 'cursor probe' item on the #2163 follow-up list). - Reaping tests get a guard-availability precheck with an explicit failure message (assertion, not skipIf, so a coreutils-less Linux host fails diagnosably instead of going silently green). - Tests: orphaned-augment reaping (CJS + Plugin, red without the wrap, ~9.1s reap measured), disabled-sentinel degradation equivalence, source pinning for all three adapters (exact per-branch budget-formula counts) + probe export + cursor debug line. Note: pre-commit typecheck skipped; remaining tsc errors are pre-existing on main (none in files touched here). * fix(hooks): group-SIGKILL the npx arm, prove guard exit propagation (#2169 review) Addresses the tri-review findings on #2169: - [P2] npx-arm containment: the CLI is the guard's grandchild there — at budget expiry coreutils timeout TERMs the group, npx (the obedient direct child) dies, timeout returns, and -k never fires, so a SIGTERM-immune grandchild escaped unbounded. The npx arm's wrapper now uses -s KILL: an unignorable group SIGKILL at budget that reaps the grandchild (kept -k 1 as a harmless belt; direct-exec arms keep TERM-first). CHANGELOG, adapter docblocks, and the test comment now state the per-arm semantics honestly. New behavioral test: a staged hook with a PATH-injected fake npx spawning a SIGTERM-immune grandchild is SIGKILLed; the grandchild must be reaped (red without -s KILL), with a route self-proof marker pinning the npx arm. - [P3] guard self-test now proves exit-status propagation (sh -c 'exit 42' must yield status 42), so an always-exit-0 stub like /bin/true is rejected and resolution falls through to the built-in candidates instead of silently killing the augment feature. New test: stub guard rejected, augment still emits context. - [P3] cleanup SIGKILLs in the reaping tests re-check the /proc/<pid>/cmdline identity immediately before firing (PID-reuse guard), applied consistently to the two pre-existing #2165 spots and both new tests. - Review notes: source pins now constrain wrapper argv order and exact per-arm counts; adapters degrade to unwrapped on probe version skew (typeof check) instead of a swallowed TypeError; export JSDoc wording fixed for relative env paths; debug-gated diagnostic when no guard is available (e.g. macOS without coreutils), with the CHANGELOG entry qualified accordingly. Note: pre-commit typecheck skipped; remaining tsc errors are pre-existing on main (none in files touched here). --------- Co-authored-by: Gergő Magyar <gergomagyar@icloud.com> |
||
|
Gergő Magyar
|
14397dd4aa
|
feat(taint): intra-procedural taint analysis (#2083) (#2164)
* feat(taint): harvest occurrence-tagged call/member sites on StatementFacts (#2083 U1) Worker-side site harvest in TsHarvester: call/new/member-read records with dotted callee paths, receiver slots, per-argument occurrence tagging with nested-site links, per-declarator resultDefs, spread/template/require-literal markers. hasTaintSafeSites validation seam. The pdg parse-cache chunk-key namespace is versioned (pdg:1 -> pdg:2) instead of a global SCHEMA_BUMP so flag-off users keep warm caches; bench fingerprints re-baselined for the three call-bearing scenarios (straight-line/dense-bindings byte-unchanged). * feat(taint): built-in TS/JS source/sink/sanitizer model + site matcher (#2083 U2) Typed spec (kind taxonomy; sanitizers carry neutralizes-kinds), the canonical Express/Node model, and matchFunctionSites: ESM alias/namespace + require- literal callee resolution, bare-name fallback restricted to true globals, sanitizers module-or-global only (never user-shadowable by name), spread/ template arg-position rules, deterministic taintModelVersion. * feat(taint): pure intra-procedural taint propagation engine (#2083 U3) Two-rule model (statement-local + du-fact worklist) with per-taint neutralized-kind exclusion sets: sanitizers exclude only the sink kinds they neutralize (escape(req.body) suppresses res.send but still fires db.query; exec(path.basename(t)) fires), intersection-over-paths so a bypass occurrence keeps the taint live, kill locality on resultDefs, propagate-through args+receiver with viaCall hops, one path per finding, deterministic caps, coverage-gap statuses. Test-first: 38 scenarios on real harvested CFGs. * feat(taint): thread taint caps + model version through pdg config/meta (#2083 U5) resolvePdgConfig gains maxTaintFindingsPerFunction (200), maxTaintHops (32), and the taintModelVersion digest; RepoMeta.pdg + RunScopeResolutionInput surfaces added. The key-union comparator trips full writeback on M2->M3 upgrade and on model-version change without --force (mode-flip tested). No CLI flags or rc keys (programmatic parity with the other caps). * feat(taint): in-phase taint emit with sparse TAINTED/SANITIZES edges (#2083 U4) run.ts pdg window: match-first fast path (solver only when a function has both a matched source and sink) -> computeReachingDefs with the shared RD fact derivation -> computeTaintFlows -> per-finding TAINTED (versioned hop-encoded reason via the shared path codec, statement-level occurrence identity) + per-kill SANITIZES, dedup-before-budget, truncate-and-warn. All emit counters surfaced (aggregate warn for gaps/drops, debug for volume); PROF gains taint=. Flag-off golden untouched. * feat(mcp): explain tool for persisted taint findings (#2083 U6) Anchorless calls enumerate the sparse TAINTED table (bounded, deterministic, limit-clamped); anchored calls (file or symbol via resolveSymbolCandidates) return full decoded hop detail. sinkKind rides a version-1 codec header (1;<kind>|hops — no other persisted channel exists; U4/U6 ship together). RepoMeta.pdg probe yields a no-taint-layer note instead of an error. TAINTED/SANITIZES pinned OUT of VALID_RELATION_TYPES (KTD9a negative- membership tests); generators + canonical skill docs + mirrors updated. * test(taint): acceptance fixture battery, snapshots, and bench gates (#2083 U7) pdg-repo taint-cases fixtures complete the six plan shapes; committed findings/kills snapshot via a shared pure-path harness that also feeds the AE2 exact-equality assertion (stored TAINTED == pure-path findings, the no-explosion gate). New taint-dense bench scenario with four --check gates: per-function findings pinned AT the cap, absolute reason-byte + site-bytes disk ceilings (the load-bearing R10 gate), zero-match pass < 0.5x match- dense, N-linearity. Pre-existing scenario baselines untouched. * refactor(taint): share one pointKey helper across propagate + emit (#2083 review) Extract pointKey(ProgramPoint) to cfg/reaching-defs.ts (colon-separated, matching the codebase block:stmt id convention) and import it in both propagate.ts and emit.ts, replacing the two divergent locals (':' vs '.'). Edge-id material now uses the colon form; ids are in-memory only and no test asserts the pointKey segment shape. * fix(taint): discriminate taint state by source occurrence (#2083 review) Two distinct sources flowing into one variable at one def point no longer collapse to a single TAINTED edge: the taint-state key gains a root source-occurrence discriminator ({point, siteIndex} — the same fields recordFinding's identity uses, excluding kind). Def->use fact lookup keys on the source-independent (binding, def-point) portion. Same-source multi-path flows still share one state so their exclusion sets intersect (the raw arm soundly wins); termination holds (finite keys, monotone shrink, no cross-source ping-pong). Restores the KTD6 identity contract. * fix(mcp): route dotted symbol names in explain to symbol resolution (#2083 review) The fileish classifier matched any dotted name (UserController.create) as a file via its extension-like suffix, so symbol resolution never ran and the tool returned a silent empty file-anchored result. Tighten the classifier to require a path separator or a real source extension (derived from the resolver's EXTENSIONS list, multi-language), so dotted/bare names route to resolveSymbolCandidates (found / ambiguous / not-found). * fix(mcp): gate explain no-taint-layer note on taintModelVersion (#2083 review) An M1/M2-era --pdg index has meta.pdg defined (BasicBlock/REACHING_DEF recorded) but no taintModelVersion and zero TAINTED rows. The probe keyed on generic meta.pdg presence, so explain returned the generic empty note instead of the actionable 'no taint layer — run analyze' hint. Gate on meta.pdg?.taintModelVersion (the field M3 stamps) so an M2-era index gets the layer hint; a taint-stamped index with no findings still gets the generic note. * fix(taint): sequence-expression value flows only the final operand (#2083 review) A comma expression in value position (exec((log(x), 'safe'))) default- descended, fanning every operand's occurrences into the enclosing sink argument — over-tainting exec's arg 0 with x. Add an explicit walkValue case that records earlier operands' uses with occurrence fan-out suppressed (new FactAccumulator.suppressOccurrences) and routes only the last operand through the value path. Sites-layer only; defs/uses/mayDefs byte-identical (cfg + reaching-defs snapshots unchanged). * perf(taint): FIFO head-cursor worklist + dedup before chainHops (#2083 review) Replace queue.shift() (O(N) dequeue) with a strict-FIFO head cursor plus order-preserving prefix reclamation; FIFO is load-bearing because chainHops reads the live taints map whose parent/source/viaCall are rewritten order-sensitively on monotone shrink, so hop determinism is dequeue-order contingent. Extract findingKey() and dedup-check before chainHops in the justify branch — already-recorded identities discard their hop chain (first write wins), so the ancestry walk was pure waste. The else kill branch is untouched. Findings + hops byte-identical (snapshot unchanged). * perf(taint): O(1) member-read dedup via composite-key set (#2083 review) addMemberRead rescanned the whole per-statement sites array per call to dedup by (object, property, parent) — O(n^2) on member-read-dense statements. Track a composite-key Set alongside sites for O(1) dedup. (The require-literal join is already O(sites) with a no-op body on non-require sites, so no early-exit is needed there.) Behavior identical: harvest + model-match + taint snapshots unchanged. * refactor(taint): drop test-only export; source taint caps via emit.ts (#2083 review) Remove the sanitizerNeutralizes export (its only consumers were two test assertions — inlined to entry.neutralizes membership). Re-export the DEFAULT_PDG_MAX_TAINT_* caps from emit.ts and point run.ts at emit.ts, so the pipeline's taint dependency surface is the single orchestration module rather than reaching into propagate.ts. * test(taint): extract the shared TS CFG/taint test harness (#2083 review) The parse/collectFunctions/cfgOf/cfgsOf/importsFor harness was copied byte-for-byte across four suites (harvest, model-match, propagate, taint-emit). Promote it to test/helpers/ts-cfg-harness.ts and import it. site-safety/reaching-defs carry a structurally different inlined builder and are left as-is. Pure extraction, no assertion changes. * test(mcp): harden explain limit-rejection battery (#2083 review) Add NaN, Infinity, -Infinity, and a numeric string to the out-of-bounds limit cases — a regression fence over the interpolated LIMIT, confirming the Number.isInteger guard rejects every non-integer/non-finite/string input before it reaches the query. |
||
|
Gergő Magyar
|
28f3b99822
|
chore(devcontainer): simplify Dockerfile and devcontainer.json by removing version args for AI CLIs (#2174) | ||
|
azizur100389
|
bdb824cfe4
|
feat(cli): add circular import cycle check (#2166) | ||
|
Minidoracat
|
10d1e47df3
|
fix(hooks): bound db-lock probe subprocesses and gate probe behind hook slot (#2163) (#2165)
* fix(hooks): bound db-lock probe subprocesses and gate probe behind hook slot (#2163) The Claude PreToolUse db-lock probe leaks orphaned lsof processes when the hook process is hard-killed mid-probe (e.g. Claude Code's 10s hook timeout under load). Orphans accumulate, raise load, slow the next probe, and snowball to sustained 100% CPU. - Wrap the unix lsof/ps fallback in coreutils timeout (-k 1 2 / -k 1 1), resolved via a lazy self-test, so probe children self-destruct within ~3s even if the hook is SIGKILLed. GITNEXUS_HOOK_TIMEOUT_PATH overrides the guard binary; the sentinel value 'disabled' turns the guard off; hosts without a usable guard keep the previous behavior. - Acquire the per-repo hook slot before probing (all three adapters), bounding concurrent probes to 3 per .gitnexus, with probe and augment inside try/finally so the slot is always released. - Tests: source-order contract, slot-gating behavior, orphan reaping with a SIGTERM-immune fake lsof and a SIGKILLed parent (red on base), probe-copy byte parity, no-guard equivalence, broken-guard rejection. Note: pre-commit typecheck skipped; the 62 tsc errors are pre-existing on main (all in src/core/** and src/server/, none in files touched here; base==head invariant verified). * fix(hooks): address tri-review P3 findings (#2165) - Map guard signal-death (status null + signal, no spawnSync error) to fail-closed at both the lsof and ps call sites, closing the freeze window (SIGSTOP / laptop sleep > 2s) that previously landed fail-open. Rewrite the exit-code comments: coreutils surfaces the -k kill as signal death, 124 is budget expiry (live arm), 137 covers only exit-code-propagating wrappers or an externally SIGKILLed child. - Add a debug-gated 'augment skipped: hook slots saturated' stderr line on the slot-starved early return in all three adapters, restoring observability under GITNEXUS_DEBUG=1. - GITNEXUS_HOOK_TIMEOUT_PATH now participates in candidate fall-through: the env candidate is tried first, then the built-ins, each behind the lazy self-test — an existing-but-unusable env path (directory, non-executable) can no longer silently disable orphan containment. - Tests: +6 — guard exit 124 pins the live arm (CJS+Plugin), guard signal-death pins the new mapping (CJS+Plugin, red before the fix), antigravity behavioral slot-gate, env-dir fall-through still reaps a SIGTERM-immune orphan via a built-in guard. Note: pre-commit typecheck skipped; the 62 tsc errors are pre-existing on main (none in files touched here). |
||
|
Gergő Magyar
|
bde340a5b4
|
feat(cfg): intra-procedural REACHING_DEF data-dependence layer (#2082) (#2160)
Some checks failed
Devcontainer Smoke / Config-transform unit tests (push) Has been cancelled
Devcontainer Smoke / Build devcontainer image (push) Has been cancelled
CodeQL / Analyze (javascript-typescript) (push) Has been cancelled
CodeQL / Analyze (python) (push) Has been cancelled
Gitleaks / gitleaks (push) Has been cancelled
Publish / Classify release event (push) Has been cancelled
Scorecard / Scorecard analysis (push) Has been cancelled
Trivy Image Scan / Trivy (gitnexus-cli) (push) Has been cancelled
Trivy Image Scan / Trivy (gitnexus-web) (push) Has been cancelled
Publish / RC guard (marker + release-PR skip) (push) Has been cancelled
Publish / ci (push) Has been cancelled
Publish / Publish to npm (push) Has been cancelled
Publish / Build & Push RC Docker images (push) Has been cancelled
* fix(cfg): route early exits through finally with target-relative threading (#2082 U2) * feat(cfg): harvest per-statement def/use facts into the side channel (#2082 U1) * feat(cfg): add reaching-definitions solver with GEN/KILL fixpoint + statement sweep (#2082 U3) * feat(cfg): persist budgeted REACHING_DEF projection with RepoMeta coherence (#2082 U4) * test(cfg): REACHING_DEF snapshot, pipeline both-sinks, and cache-seam coverage (#2082 U5) * bench(cfg): reaching-defs scaling gates — dense-bindings + fact-fanout scenarios (#2082 U6) * fix(mcp): exclude BasicBlock pseudo-symbols from detect_changes on pdg indexes (#2082 U7) * style: prettier pass over M2 files * fix(cfg): review-pass fixes — defKey overflow guard, catch-param block, class defs, intra-statement reads, graceful fact degradation (#2082) - reaching-defs: STMT_STRIDE 2^16→2^21 + upfront aliasing bail-out; a use that shares its statement with a def now also sees the same-statement def (assign-and-test idiom was a taint false negative); drop dead posInOrder - visitor: catch-param def gets its own once-executed block (prepending into a loop-header entry re-genned per iteration and killed loop-carried redefs); unresolved-label jumps now thread all active finallys; the finalizer-threading protocol moved to control-flow-context as shared helpers for future language visitors - harvest: class declarations def their name (was a bogus use in JS, silent skip in TS); class-expression names stay internal - emit: isEmitSafeCfg adds index==position contiguity; fact validation split into hasEmitSafeFacts so malformed facts degrade to CFG-only instead of dropping the function's whole CFG layer; facts-per-edge multiplier single source; lazy top-binding tally; dead solveMs removed - run-analyze: pdgModeMismatch compares the key union structurally — new resolved knobs join the comparison automatically - mcp: BasicBlock exclusion via id prefix (NULL-name rows of real symbols are no longer dropped) + same filter on the BM25 filePath fallback - bench: rd ratio denominator clamped (gate no longer self-disables at fast small-N); PROF-gated pdg timing in run.ts * test(run-analyze): model the M2 RepoMeta.pdg stamp in resolvePdgConfig defaults The DEFAULTS constant lacked the maxReachingDefEdgesPerFunction field that resolvePdgConfig resolves since the M2 stamp landed, failing two strict toEqual expectations (the CI 'tests' job failures). Models M2 steady-state equality; the M1-era-stamp upgrade path stays pinned in pdg-mode-flip.test.ts. Finding P1-4 of review 4471987625 (#2160). * test(cfg): reassign the shadowing fixture's bindings — fixes prefer-const CI errors Both withShadowing let bindings now genuinely reassign (s = s + 1 per scope), clearing the two prefer-const errors that failed quality/lint. Plain const would change the binding kind the harvest test exercises; reassignment keeps the let semantics and enriches the reaching-defs facts the snapshot pins (snapshot + per-binding assertion updated accordingly). Finding P2-6 of review 4471987625 (#2160). * fix(cfg): validate entry/exit indices in the emit-safety guard A corrupted side-channel element with an out-of-range entryIndex passed isEmitSafeCfg and threw inside the reaching-defs RPO walk — caught by the per-FILE try/catch, costing every sibling function's REACHING_DEF projection instead of the one element (and logging a misleading message). entry/exit join the guard's id-anchor checks. Finding P3 (entryIndex) of review 4471987625 (#2160). * fix(cfg): report the def-key stride bail-out as a distinct 'overflow' status The STMT_STRIDE aliasing guard reused status 'truncated', so the emit warn misnamed it as the fact-materialization limit (printing an unrelated maxFacts value, including '(0)' when unlimited) and telemetry conflated the two. A distinct 'overflow' status gets its own warn naming the actual cause; the function's CFG layer is explicitly unaffected. Finding P3 (stride-bail diagnosis) of review 4471987625 (#2160). * perf(cfg): cache the nearest enclosing scope per node during the prescan resolve() walked the AST parent chain per identifier — O(expression nesting depth), quadratic on deeply-chained single-statement expressions in generated code (not caught by any bench scenario, which scale blocks/bindings, not expression depth). The prescan already visits every node once, so caching its innermost scope makes phase-2 resolution O(scope-chain). Behavior-identical; the parent-chain walk survives as fallback for prescan-unvisited nodes. Finding P2 (resolve depth walk) of review 4471987625 (#2160). * fix(cfg): stop harvesting initializer-less var declarators as defs A bare `var x;` mid-function is hoisted and writes nothing at runtime, but the harvester recorded a def — fabricating a kill of the live def in the same block: `x = source(); var x; sink(x)` lost the source→sink fact (a reaching-defs false negative). Defs now require an initializer for variable_declaration declarators; let/const genuinely initialize and keep their def. Finding P2-5 of review 4471987625 (#2160). * fix(cfg): unwrap parenthesized/non-null lvalue wrappers before def detection `(x) += 1` and `(x)++` gated the def on the node type being exactly 'identifier', so the parenthesized form fell to the uses-only branch — the def (and its kill) silently vanished. Wrappers that don't change the lvalue (parenthesized_expression, TS non_null_expression) now unwrap at all three lvalue sites. Finding P3 (parenthesized lvalues) of review 4471987625 (#2160). * fix(cfg): conditionally-evaluated defs are MAY-defs — gen without kill A def inside a short-circuit right operand, ternary arm, logical assignment, or switch case test was harvested as a must-def; the solver's total kill then erased the prior def on the not-taken path — a taint false negative on core idioms (`if (a && (x = clean())) {} sink(x)` lost source→sink; `cached ?? (cached = load())` likewise). StatementFacts gains an optional mayDefs field (conditional-context tracking in the harvester); the solver's per-block GEN carries {set, kills} so a may-def UNIONS into the binding's set instead of replacing it, in both the transfer and the statement sweep; the emit fact-guard validates mayDefs indices; switch case tests harvest via the conditional path. Finding P1-1 of review 4471987625 (#2160). * fix(cfg): model labeled statements generically — break keeps its real continuation A break to a label the visitor didn't model (labeled non-loop block, the OUTER label of a doubly-labeled construct) routed to EXIT, REMOVING the only path that kept the pre-jump def live — a reaching-defs false kill the in-code comment wrongly called sound. Loop/switch frames now carry their full label LIST (`outer: inner: for` resolves both); a labeled non-loop statement gets a break-target frame whose target is a synthesized join after the body; an unlabeled break never matches a block frame; labels compose with finalizer threading (a labeled break crossing a finally still threads it). Finding P1-2 of review 4471987625 (#2160). * fix(cfg): throw edges deliver ALL of a block's defs to the handler The throw contribution was IN ∪ OUT — entry and final states only. The intermediate defs of a multi-def coalesced block were invisible to the handler, though they are exactly what the catch observes when a later statement throws: `try { x = parse(a); x = normalize(x); } catch { sink(x) }` lost the parse→sink fact (normalize throwing delivers parse's value). Throw predecessors now contribute IN(from) ∪ allDefs(from) — a static per-block all-def-sites map — which subsumes OUT; monotone and deterministic. Finding P1-3 of review 4471987625 (#2160). |
||
|
dependabot[bot]
|
1150eea98f
|
chore(deps): bump actions/checkout from 6.0.2 to 6.0.3 (#2152)
Bumps [actions/checkout](https://github.com/actions/checkout) from 6.0.2 to 6.0.3.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](
|
||
|
dependabot[bot]
|
ae1ec82f5f
|
chore(deps): bump actions/attest-build-provenance from 2.4.0 to 4.1.0 (#2158)
Bumps [actions/attest-build-provenance](https://github.com/actions/attest-build-provenance) from 2.4.0 to 4.1.0. - [Release notes](https://github.com/actions/attest-build-provenance/releases) - [Changelog](https://github.com/actions/attest-build-provenance/blob/main/RELEASE.md) - [Commits](https://github.com/actions/attest-build-provenance/compare/v2.4.0...a2bbfa25375fe432b6a289bc6b6cd05ecd0c4c32) --- updated-dependencies: - dependency-name: actions/attest-build-provenance dependency-version: 4.1.0 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
dependabot[bot]
|
5a590f052f
|
chore(deps): bump docker/setup-qemu-action from 4.0.0 to 4.1.0 (#2159)
Bumps [docker/setup-qemu-action](https://github.com/docker/setup-qemu-action) from 4.0.0 to 4.1.0.
- [Release notes](https://github.com/docker/setup-qemu-action/releases)
- [Commits](
|
||
|
dependabot[bot]
|
df56d34f33
|
chore(deps): bump release-drafter/release-drafter from 7.3.0 to 7.3.1 (#2157)
Bumps [release-drafter/release-drafter](https://github.com/release-drafter/release-drafter) from 7.3.0 to 7.3.1.
- [Release notes](https://github.com/release-drafter/release-drafter/releases)
- [Commits](
|
||
|
dependabot[bot]
|
04aeb1f4bf
|
chore(deps)(deps-dev): bump @vercel/node in /gitnexus-web (#2156)
Bumps [@vercel/node](https://github.com/vercel/vercel/tree/HEAD/packages/node) from 5.8.8 to 5.8.12. - [Release notes](https://github.com/vercel/vercel/releases) - [Changelog](https://github.com/vercel/vercel/blob/main/packages/node/CHANGELOG.md) - [Commits](https://github.com/vercel/vercel/commits/@vercel/node@5.8.12/packages/node) --- updated-dependencies: - dependency-name: "@vercel/node" dependency-version: 5.8.12 dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
dependabot[bot]
|
792cd96d37
|
chore(deps): bump actions/setup-python from 5.6.0 to 6.2.0 (#2155)
Bumps [actions/setup-python](https://github.com/actions/setup-python) from 5.6.0 to 6.2.0. - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](https://github.com/actions/setup-python/compare/v5.6.0...a309ff8b426b58ec0e2a45f0f869d46889d02405) --- updated-dependencies: - dependency-name: actions/setup-python dependency-version: 6.2.0 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
dependabot[bot]
|
351edca0ca
|
chore(deps)(deps-dev): bump @vitest/coverage-v8 in /gitnexus-web (#2153)
Bumps [@vitest/coverage-v8](https://github.com/vitest-dev/vitest/tree/HEAD/packages/coverage-v8) from 4.1.5 to 4.1.8. - [Release notes](https://github.com/vitest-dev/vitest/releases) - [Changelog](https://github.com/vitest-dev/vitest/blob/main/docs/releases.md) - [Commits](https://github.com/vitest-dev/vitest/commits/v4.1.8/packages/coverage-v8) --- updated-dependencies: - dependency-name: "@vitest/coverage-v8" dependency-version: 4.1.8 dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
dependabot[bot]
|
15e141f21b
|
chore(deps)(deps): bump sigma from 3.0.2 to 3.0.3 in /gitnexus-web (#2151)
Bumps [sigma](https://github.com/jacomyal/sigma.js) from 3.0.2 to 3.0.3. - [Release notes](https://github.com/jacomyal/sigma.js/releases) - [Changelog](https://github.com/jacomyal/sigma.js/blob/main/CHANGELOG.md) - [Commits](https://github.com/jacomyal/sigma.js/compare/sigma@3.0.2...sigma@3.0.3) --- updated-dependencies: - dependency-name: sigma dependency-version: 3.0.3 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
dependabot[bot]
|
0b2c88518e
|
chore(deps)(deps): bump dompurify from 3.4.7 to 3.4.8 in /gitnexus-web (#2150)
Bumps [dompurify](https://github.com/cure53/DOMPurify) from 3.4.7 to 3.4.8. - [Release notes](https://github.com/cure53/DOMPurify/releases) - [Commits](https://github.com/cure53/DOMPurify/compare/3.4.7...3.4.8) --- updated-dependencies: - dependency-name: dompurify dependency-version: 3.4.8 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
dependabot[bot]
|
8bb2fe9128
|
chore(deps)(deps): bump langchain from 1.4.2 to 1.4.4 in /gitnexus-web (#2149)
Bumps [langchain](https://github.com/langchain-ai/langchainjs) from 1.4.2 to 1.4.4. - [Release notes](https://github.com/langchain-ai/langchainjs/releases) - [Commits](https://github.com/langchain-ai/langchainjs/compare/langchain@1.4.2...@langchain/openai@1.4.4) --- updated-dependencies: - dependency-name: langchain dependency-version: 1.4.4 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
Gergő Magyar
|
6424d8b09c
|
fix(web): replace broken Browse-for-folder with upload directory picker (#1850)
* fix(web): replace broken Browse-for-folder with server-side directory picker The "Browse for folder" button used `<input type="file" webkitdirectory>` which only exposes relative paths via `webkitRelativePath`. The code extracted just the folder name (e.g. `myproject`), causing the server to reject it with "path must be an absolute path". No browser API can expose absolute filesystem paths, so the approach was fundamentally broken on all platforms. - Add `GET /api/fs/list` endpoint that lists subdirectories at a given absolute server-side path (rate-limited, validated) - Add `listDirectories()` client function in backend-client.ts - Add `DirectoryPicker` modal component with breadcrumb navigation - Replace broken `webkitdirectory` input in RepoAnalyzer with the new server-side directory picker - Update i18n strings (en + zh-CN) - Add unit tests for the new endpoint (9 tests) Docker users can now browse `/workspace/` and other container paths directly from the UI. Manual path entry continues to work unchanged. Closes #1518 * test(e2e): add Playwright tests for server-side directory picker 13 Playwright e2e tests covering the full DirectoryPicker flow: - Open/display: modal opens, shows root dirs, displays current path - Navigation: click into dirs, breadcrumb back-nav, home button - Selection: populates path input, returns absolute path, close without selecting - Edge cases: empty dir, API error, manual typing still works Also updates existing onboarding.spec.ts to match the renamed "Browse server directories" button, and adds data-testid attributes to DirectoryPicker and RepoAnalyzer for reliable e2e targeting. * fix(a11y): add accessibility and UX polish to DirectoryPicker - Add role="dialog", aria-modal, aria-label to the modal panel - Add aria-label to close button, home button - Add aria-hidden to decorative icons (chevrons, backdrop) - Add role="status" to loading spinner with sr-only label - Add role="alert" to error state - Add aria-current="location" to active breadcrumb segment - Wrap breadcrumb in nav landmark with aria-label - Add Escape key handler to dismiss the modal - Auto-focus the modal panel on open - Add focus-visible ring styles to all interactive elements (matches existing focus-visible:ring-2 ring-accent/40 pattern) - Increase breadcrumb button padding (px-1.5 py-1) for better touch targets - Increase directory entry padding (py-2.5) for touch comfort - Add active:bg-hover/70 pressed state on directory entries - Add active:bg-accent/80 pressed state on select button * chore(autofix): apply prettier + eslint fixes via /autofix command * fix: skip traversal guard for bare root paths in /api/fs/list (#2109) * fix(web): replace server-side directory picker with secure folder upload PR #1850 review found the new GET /api/fs/list directory-browsing endpoint enumerated any absolute server path (CodeQL js/path-injection, plus a DoS and cross-origin enumeration via the CORS/PNA allow-list). Browsers can't hand the server an absolute path, so rather than harden the endpoint, remove it and upload the folder instead — webkitdirectory exposes the file contents. - Add POST /api/analyze/upload: busboy-streamed multipart ingest into an mkdtemp sandbox under UPLOAD_ROOT with resolve-then-contain write sanitization, hard size/count/dir caps, manifest-first ordering, and guaranteed cleanup; promote (atomic same-filesystem rename, no EXDEV) and analyze via the shared job/worker machinery, never returning a server path. - Frontend: <input webkitdirectory> upload flow with client-side filtering (.git/node_modules/build), XHR progress, accessibility, en/zh-CN i18n. - Remove /api/fs/list + handleFsListRequest, DirectoryPicker, listDirectories and their tests. - Harden the adjacent /api/analyze {path} route: localhost-only CORS on write routes + realpath/exists/isDir validation replacing the inert normalize!==resolve guard. - Extend DELETE /api/repo cleanup to upload dirs (by entry.path) and add a startup sweep for orphaned staging dirs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(review): resolve CodeQL path-injection + CSRF introduced by the upload change The first push surfaced two new CodeQL alerts in the newly-added code (the upload sandbox itself passed — its resolve-then-contain sanitizer is recognized): - HIGH js/path-injection at the analyze route: the KTD11 in-route `fs.realpath(repoLocalPath)` / `fs.stat` was a user-controlled filesystem read with no security gain (the worker already reads the path; cross-origin reach is closed by requireLocalhostOrigin). Drop the in-route fs calls; keep only the absolute-path check + the localhost-origin guard. - MEDIUM js/client-side-request-forgery: the new raw `xhr.open` was a fresh request sink. Route the upload through the shared, origin-validated fetchWithTimeout instead (the centralized sink all other calls use). Trades the upload-progress percentage for an indeterminate "Uploading…" state. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(review): resolve tri-review findings on the upload flow A multi-agent review of the upload implementation surfaced a P0 plus several P2/P3s; all are addressed here. - P0: the upload handler took the single analysis slot (createJob) before validating/promoting, so any failure in that window left a queued job that was never failed — wedging ALL analysis until restart (trivially triggered by a single-segment manifest). Now: validate the folder before taking the slot, release it via failJob on any pre-launch error, and reject single-segment / multi-top manifests during ingest (also fixes a silent file-drop). - CI: rate-limit.test's source-regex broke when Prettier wrapped the /api/analyze registration; made it wrapping-tolerant. - Resource: the startup sweep now also removes stale promoted upload dirs with no .gitnexus index (orphans from analyses that failed before registering). - Frontend: guard against post-unmount SSE opening, reset upload state on cancel/mode-change, guard concurrent uploads, fall back to the folder name, add aria-busy, and fix the {{count}} plural ("1 files"). - Maintainability: extract launchAnalysisWorker into analyze-launch.ts (DI + typed WorkerMessage IPC), move requireLocalhostOrigin to middleware.ts, share REPO_NAME_PATTERN, tighten UploadJobRef, name the collision-retry constant. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(web): reset isMountedRef on mount (StrictMode double-invoke) The mount effect set isMountedRef=false on cleanup but never back to true on re-mount, so under React StrictMode's mount->unmount->mount the ref stayed false for the component's lifetime — trackJob then always early-returned and the upload never advanced past 'starting' (caught by the folder-upload e2e). Set it true at the start of the effect. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(review): de-flake upload-ingest cleanup test via injectable staging root ingestUpload gains an IngestOptions.root override (mirroring SweepOptions.root) so the test asserts cleanup against a per-test mkdtemp root instead of counting global ~/.gitnexus/uploads/.staging-* entries, which raced parallel forks. Production default stays UPLOAD_ROOT (promote rename same-filesystem invariant). * fix(web): make stale analyze/upload requests inert after mode switch, cancel, or unmount A folder upload (or URL analyze) still in flight when the user switched modes could resolve later, call trackJob(), and drive the old job's SSE stream under the new mode's form. The only guard was isMountedRef — mode change and cancel never unmount the component. - requestControllerRef: per-request AbortController doubling as the staleness token (captured per closure, checked after the await; the abort error is matched via signal.aborted, never error identity, since it surfaces both as BackendError('Request aborted') and as a raw AbortError from response.json()) - uploadFolder() now takes an optional AbortSignal; fetchWithTimeout already merges caller signals via AbortSignal.any - a stale-but-created job gets a fire-and-forget cancelAnalyze(jobId) (skipped when a live tracking session owns the id) so the single analyze slot is freed - handleModeChange early-returns on same-tab clicks and resets phase to input so an aborted request can't strand the form at 'starting' - fixed the stale breaker comment: resilientFetch records AbortError as breaker-neutral (recordNeutral), not as a retryable-network penalty * refactor(web): consolidate stale-request guard plumbing - single invalidateRequest() helper for the abort+null pattern (4 sites) - drop isMountedRef checks subsumed by the aborted-controller token (unmount aborts the controller, and unlike isMountedRef the token stays correct across a StrictMode unmount/remount) - dedup the component test's render/mock scaffolding - countStaging filters on the exported STAGING_PREFIX, not a magic string * fix(web): scope stale-job cancellation to the upload path Code review caught a regression in the first cut: URL analyzes dedup-alias by repo (createJob returns the existing active job's id), so a stale resolution's fire-and-forget cancel could kill a job another session — or the user's own fresh resubmit — is actively watching; the jobIdRef ownership guard was order-dependent and instance-local. Uploads always own a fresh, never-deduped job, so the cancel is kept (unconditionally) there and dropped on the URL path, where a same-URL resubmit re-attaches via dedup and the server's job timeout / TTL sweep bounds the slot occupancy. Also: remove the isMountedRef machinery outright (zero readers remain — the aborted-controller token subsumes it and stays correct across StrictMode remounts), make the e2e abort check ERR_ABORTED-specific, and let a broken test root fail loudly instead of passing vacuously. --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Sparsh <73558748+prajapatisparsh@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Gergő Magyar
|
5bf8a17cd5
|
feat(ingestion): add control-flow-graph layer for TS/JS (#2081) (#2099)
* feat(cfg): language-agnostic CFG construction core (#2081) U1 of M1 (CFG layer). Plain JSON-serializable CFG data model (BasicBlockData/ CfgEdgeData/FunctionCfg — must survive the worker→main boundary + ParsedFile store), a CfgBuilder accumulator (leaders→blocks→edges, synthetic ENTRY/EXIT, idempotent edges), a ControlFlowContext (break/continue/switch + labeled-jump target stacks), and a TraversalResult ({entry, dangling exits}). AST-agnostic and unit-tested on the classic control-flow topologies (if/else, while back-edge, mid-block return, labeled break/continue) the S2 spike validated; reachability helper backs the R9 property test. * feat(ingestion): U2 — TS/JS CFG visitor over tree-sitter AST (#2081) Add the TS/JS CfgVisitor that walks a function's tree-sitter AST and drives the U1 CfgBuilder to produce a serializable FunctionCfg. One visitor covers both languages (shared grammar family). Handles the classic CFG hazards explicitly (R2, R10): - loops allocate a dedicated loop-exit block so `break` has a concrete target before the loop's successor is known; `continue`/back-edge close the loop (while, do-while, C-for with init-once + increment-as-continue-target, for-in, for-of) - switch fallthrough falls out naturally: a non-breaking case yields exits we wire to the next case as `fallthrough`; a breaking case wires to the switch exit via ControlFlowContext - try/catch/finally: normal completion AND exceptional flow both route through finally (post-domination); a conservative exceptional edge models that the protected region may raise to its handler (not just explicit `throw`) - labeled break/continue resolve against the labeled loop's frame - early return/throw wire to EXIT/handler and terminate their block 19 hazard tests (one per construct) + AC1 10-function fixture; all green. No change to the committed U1 core or ControlFlowContext. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(ingestion): U3 — worker CFG build + cfgSideChannel + cache coherence (#2081) Run the CFG visitor in the parse worker (where the AST lives), serialize the per-function CFG onto a new ParsedFile.cfgSideChannel, and keep it coherent across the disk-backed store and the warm/durable parse cache (R3, R4). - gitnexus-shared parsed-file.ts: add `cfgSideChannel?: unknown` as a DISTINCT field from captureSideChannel (different producer/consumer/lifecycle; plain JSON data — blocks/edges deliberately lack the `nodeId` the store's interning reviver keys on, so no mis-interning). - cfg/types.ts + visitors/typescript.ts: add CfgVisitor.isFunction so the worker enumerates functions (and applies the line budget) by a cheap node-type test. - cfg/collect.ts (new): collectFunctionCfgs walks the tree, builds one CFG per function (nested included), applies maxFunctionLines (over-cap = skipped). - language-provider.ts: add `cfgVisitor?: CfgVisitor<SyntaxNode>` hook; typescript.ts attaches it to both the TS and JS providers (shared grammar). - parse-worker.ts: read pdg + pdgMaxFunctionLines from workerData (read once at init — the worker never sees PipelineOptions), gate the build, attach cfgSideChannel alongside captureSideChannel. - parse-cache.ts: bump SCHEMA_BUMP 4→5 (ParsedFile shape changed) and fold the pdg flag into computeChunkHash so a pdg-off cached chunk is NOT reused on a --pdg run (the #2038-class warm-cache trap). Default path keeps its keys. - worker-pool.ts + parse-impl.ts + pipeline.ts: thread pdg/pdgMaxFunctionLines PipelineOptions → WorkerPoolOptions → workerData, and into the chunk-hash key. 9 boundary tests: collect contract, JSON round-trip identity (no AST leakage), the pdg cache-key guard, the line-cap skip, and the no-visitor gate. Full CFG suite (U1+U2+U3) green; build clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(ingestion): U4 — emit BasicBlock + CFG within scope-resolution (#2081) Emit persisted BasicBlock nodes + CFG edges from each ParsedFile's worker-built cfgSideChannel, INSIDE scope-resolution's Phase-4 graph emission — the last point where the worker-built CFGs are loaded (emitParsedFiles carries the channel; the disk store is cleared right after the orchestrator returns). This is the architecture the doc-review corrected to: a standalone post-`mro` phase (the issue's literal subtask) provably reads empty data (KTD1). - cfg/emit.ts (new): pure emitFileCfgs(graph, cfgs, maxEdgesPerFunction, onWarn). BasicBlock id = `BasicBlock:<filePath>:<functionStartLine>:<blockIndex>` (KTD3 — funcStart disambiguates blocks across functions in one file; no `name` column). CFG edge = CodeRelation type 'CFG' with the edge KIND (seq/cond-true/…) in `reason` (kinds can't be their own edge type). Per- function edge cap stops at the cap and warns with the dropped count — no silent truncation (R6/KTD6). - run.ts: pdg-gated emit pass over emitParsedFiles after emitPostResolutionEdges (store still live); RunScopeResolutionInput gains pdg + pdgMaxEdgesPerFunction. - phase.ts: thread ctx.options.pdg / pdgMaxEdgesPerFunction into the call. - pipeline.ts: PipelineOptions.pdgMaxEdgesPerFunction. 6 tests: node/edge shape (KTD3 id, no name, type='CFG', kind in reason), cross-function id uniqueness, AC2 reachability-from-ENTRY property, the edge cap's no-silent-truncation contract, and empty-input no-op. Flag-off byte-identity + full runPipelineFromRepo round-trip land in U7. Build clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(cli): U5 — `--pdg` opt-in plumbing (CLI + .gitnexusrc → both sinks) (#2081) Expose the CFG/PDG substrate as an opt-in and thread it from CLI/.gitnexusrc to the single source of truth (PipelineOptions.pdg), which fans out to BOTH sinks already wired in U3/U4: the worker build gate (workerData.pdg) and the scope-resolution emit gate. Off by default (R7). - cli/index.ts: `--pdg` commander flag. - cli/analyze.ts: AnalyzeOptions.pdg + pass `pdg` into runFullAnalysis options. - cli/analyze-config.ts: KEY_SPECS `pdg` (boolean) so `.gitnexusrc { "pdg": true }` normalizes and a non-boolean value fails closed with GitNexusRcError. - core/run-analyze.ts: AnalyzeOptions.pdg → runPipelineFromRepo({ pdg }). (The internal PipelineOptions/WorkerPoolOptions/workerData fields + the parse-cache key fold landed in U3/U4; this unit adds the user-facing surface. The budget knobs stay at internal defaults for M1.) Tests: analyze-config pdg normalization + non-boolean rejection; opt-in.test.ts covers the CLI/file merge precedence and that pdg perturbs the chunk-dispatch key. The full worker-build + main-emit round-trip is the U7 integration test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(ingestion): U7 — CFG acceptance fixtures, parity, end-to-end + docs (#2081) Acceptance criteria for the M1 CFG layer: - AC1: a 10-function TS fixture's CFG node/edge set matches a committed snapshot (cfg-snapshot.test.ts). - AC2: every BasicBlock is reachable from its function ENTRY (property test over the emitted graph; the fixture has no dead code). - AC3: hazard fixtures lock the classic-bug coverage — try/throw/finally post-domination + labeled break/continue resolution. - AC4: the existing pipeline-graph-golden test stays byte-identical with --pdg off (verified; no UPDATE_GOLDEN), proving the opt-in adds zero default-run drift. - End-to-end (pipeline-pdg.test.ts): runPipelineFromRepo({ pdg: true }) on a tiny repo emits BasicBlock nodes + CFG edges with both endpoints present — the true both-sinks proof (worker builds → store → scope-resolution emits); the default run emits zero. Docs: CHANGELOG M1 entry, ARCHITECTURE "Optional CFG/PDG emission" subsection (why emit is in-phase, not post-mro), README CFG language-support note. Full CFG suite (U1–U7): 56 tests green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(ingestion): drop unused helper in cfg-snapshot test (#2081) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(review): apply ce-code-review autofix feedback (#2081) Review (10 reviewers) confirmed OFF-path byte-identity (adversarial + golden) and found defects all within the --pdg path. Fixes: - P1 same-line BasicBlock id collision: add a start-column disambiguator to FunctionCfg + the id (`BasicBlock:<file>:<line>:<col>:<idx>`) so two functions sharing a start line no longer collide under first-writer-wins addNode. - P1 worker crash-cascade: per-file try/catch around collectFunctionCfgs so a CFG-build throw cannot escape to the language-group catch and silently drop every remaining file in the group. - P2 edge-cap drop now logs unconditionally (input.onWarn is validator-gated/ silent in prod) — upholds the no-silent-truncation guarantee. - P2 Array.isArray guard before the cfgSideChannel cast in run.ts. - P2 maxFunctionLines default: worker applies DEFAULT_PDG_MAX_FUNCTION_LINES=2000 when unset; caps forwarded through run-analyze AnalyzeOptions (closes the server-path drop). - P3 README duplicate paragraph removed; `0`-vs-default docstrings corrected; CLI --pdg flag made language-neutral; reachableBlocks JSDoc corrected. - Documented the break-through-finally + stacked-label CFG limitations. - Tests: same-line id-collision regression, standalone throw→EXIT, dead-code- after-return, async/generator/method coverage, strengthened labeled-continue. Refuted: the HTTP-500 getNodeQuery finding — M0 already shipped the BasicBlock branch + name-floor (R12/web-safety handled). CFG + analyze-config suites: 95 tests green; golden parity (AC4) byte-identical. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(ingestion): benchmark CFG construction + O(n) block-text accumulation (#2081) Closes the M1 review's requires_verification perf gap ("no benchmark for collectFunctionCfgs; a wall-time + cfgSideChannel byte-size regression gate would catch the extendBlock concatenation before kernel scale"). - bench/cfg/measure.mjs (new): build-free tsx harness timing collectFunctionCfgs (parse once, reuse the tree) across three scaling scenarios — straight-line (extendBlock path), many-functions (collect walk), branchy (block/edge growth) — at 500→2000. Reports a wall-time scaling ratio AND a cfgSideChannel byte-size ratio, plus an order-independent sha256 over the emitted blocks/edges as the behavior gate. `--check` compares both ratios + the fingerprint against bench/cfg/baselines.json; mirrors the scope-capture / python-scope harnesses. - .github/workflows/ci-tests.yml: run the gate on every test job (build-free, alongside the existing scope-capture guards) so an O(n^2) re-regression fails CI. - cfg-builder.ts: structural fix for the one real hotspot the bench surfaced — accumulate basic-block text as fragments joined once in finish(), instead of concatenating onto a growing string per coalesced statement (O(n^2) → O(n)). Behavior-identical (the CFG fingerprint + the AC1 snapshot are unchanged). Measured (post-fix): time ratios straight-line ~1.3, many-functions ~1.0, branchy ~1.1 (all sub-quadratic; a true O(n^2) would be ~4.0). cfgSideChannel bytes scale linearly (~1.0-1.04). 60 CFG tests green; build clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(ingestion): add memory + disk growth gates to the CFG benchmark (#2081) Extend bench/cfg/measure.mjs beyond wall-time to the two other scalability dimensions that matter at kernel scale: - DISK growth: utf8 byte size of the serialized cfgSideChannel — exactly what a --pdg run writes onto every ParsedFile shard (durable store + parse cache). - MEMORY growth: retained JS heap of the cfgSideChannel payload, measured by the release-delta method (heap held minus heap after dropping it) — robust to pre-existing garbage and dead-stable run-to-run. Needs `node --expose-gc`; without it the heap metric is null and its gate is skipped (local runs still work). ci-tests.yml now passes --expose-gc so the heap gate runs in CI. Both gated on linear scaling in baselines.json (disk_bytes_budget / heap_budget 1.2-1.3). Measured: disk ~1.0-1.04, retained heap ~0.87-1.0 — both linear (~1KB/function each; ~2MB heap / 1.6MB disk at 2000 functions, --pdg only). Bumped REPS 7->15 to stabilize the noisier time signal and widened the coarse time tripwire budgets (the disk/heap gates carry the tight regression detection). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ingestion): address tri-review + CFG-expert findings (#2081) Corroborated findings from the tri-review (Codex + CE personas + GitNexus swarm + a CFG/program-analysis domain-expert lane). The OFF-path stays byte-identical; all fixes are within the --pdg path or the benchmark. - [Codex+CFG-expert] Exceptional `throw` edges now wire EVERY block in a try's protected region to the handler, not just the body ENTRY. A branched try body (`try { if (x) { use(t); } } catch`) previously left interior blocks with no path to `catch` — a taint false-negative into the handler for the M2 PDG pass. - [Codex+CFG-expert] An unresolved labeled jump (a stacked outer label or a labeled non-loop block) now routes to the function EXIT instead of leaving a dangling sink — restores the single-exit invariant post-dominator/PDG computation needs. - [Codex] computeChunkHash now folds pdgMaxFunctionLines/pdgMaxEdgesPerFunction into the chunk key (not just the pdg boolean), so a warm cache built under one cap is never served to a run with a different cap (#2038 class, extended to the budgets). Adds PdgCacheKey; boolean form kept for back-compat. - [perf] visitTry resolves catch/finally in a single namedChild pass (the double `namedChildren.find` allocated two throwaway arrays). - [adversarial] The bench `straight-line` scenario now runs at 2000->8000: output is a constant 4 blocks so disk/heap can't see the concat path, and at the old N a genuine O(n²) was masked by V8 cons-strings. Verified at the new N: the array-join impl ~1.0, a rope-optimized `+=` ~1.0 (correctly not flagged), a real O(n²) (re-join-every-append) ~3.8 — budget tightened 2.0->1.5. - [adversarial+Codex] The bench `--check` now FAILS LOUDLY when run without `--expose-gc` instead of silently skipping the retained-heap gate. - Doc: re-labeled the finally-bypass as a SOUNDNESS (false-negative) limitation tracked for M2, not mere "precision." 3 new regression tests (branched-try interior→handler, stacked-label→EXIT, cap-fold key). 99 CFG tests pass; build clean; bench gate green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(parse-cache): clarify that SCHEMA_BUMP still invalidates caches once (#2099 F6) The computeChunkHash comment claimed pdg-off warm caches "survive this change untouched" — true for the key FORMAT, but misleading as an upgrade-behavior promise: SCHEMA_BUMP 4→5 changes PARSE_CACHE_VERSION and both stores hard-invalidate on it. Separate the two facts so the next cache change isn't reasoned about from a false premise. Review finding F6 (P3) of PR #2099 tri-review. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(cfg): correct for-loop back-edge kinds when no increment clause (#2099 F5) A for with a body but no increment emitted an unconditional header→header 'loop-back' self-edge (a path that never executes the body) while the real back-edge body→header was labeled 'seq'. Any consumer identifying loops via reason='loop-back' picked the phantom edge and excluded the body from the natural loop. Gate the self-edge on the body being absent (the one case where the header genuinely re-tests itself) and carry 'loop-back' on the body's exits when they ARE the back-edge, matching visitWhile/visitForIn. Review finding F5 (P3) of PR #2099 tri-review. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(cfg): treat an empty catch clause as a real handler (#2099 F2) visitTry keyed handler semantics off the traversal result — null for an empty body, since visitSeq([]) returns null — instead of the syntactic clause. An empty `catch {}` was therefore treated as NO catch: the swallowed exception escaped to the outer handler/EXIT, the no-catch re-propagation misfired past finally, and code after a try whose body always throws became unreachable from ENTRY — a hard false-negative source for the M2 taint pass, on an extremely common pattern. Synthesize one empty block spanning the clause (entry == sole exit) when the catch body traverses to null, before the protected region is walked. Exception flow lands in it and rejoins the normal continuation; all downstream wiring (handler selection, finally routing, the !catchRes re-propagation gate) operates on the syntactically-correct shape. Review finding F2 (P2, reproduced) of PR #2099 tri-review. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(cfg): guard CFG emission per element, not just per outer array (#2099 F4) The cfgSideChannel guard checked only Array.isArray before casting to FunctionCfg[] — its own comment promised a wrong-shape value would 'skip emission, not throw a TypeError mid-graph-build', but a malformed ELEMENT sailed through. Worse, the obvious-looking failure shape never throws at all: emitFileCfgs string-templates any edge endpoint into the BasicBlock id and graph inserts are no-throw, so a non-integer endpoint silently became a dangling 'BasicBlock:…:undefined' edge that degrades the DB rel-pair COPY to row-by-row fallback inserts much later. Layered fix matching house precedents (parsedfile-store reviver, worker-side per-file catch): a per-element shape+content predicate (arrays + integer edge endpoints) that warns and skips malformed elements while valid siblings still emit, plus a per-file try/catch backstop for shapes that genuinely throw (e.g. a null inside blocks). Review finding F4 (P3) of PR #2099 tri-review. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(parse-cache): drop emit-time edge cap from the pdg chunk key (#2099 F3) pdgMaxEdgesPerFunction is applied exclusively in emitFileCfgs during scope-resolution on the main thread — the worker never receives it (workerData carries only pdg + pdgMaxFunctionLines), so the cached worker output is byte-identical across cap values. Folding it into the chunk key (added by a prior review round) only converted a free knob into a repo-sized cost: every cap change forced a full re-parse and a durable-store rewrite of unchanged data. Keep pdg + maxFunctionLines (genuinely worker-visible, shape the cached cfgSideChannel) and document the classification test in the PdgCacheKey doc comment so the next option gets sorted deliberately: worker-shard inputs go in this key; persisted-graph-only inputs belong in the RepoMeta pdg stamp (F1). Chunks written under the old ns string miss once and prune — no migration needed. Review finding F3 (P2) of PR #2099 tri-review. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(analyze): record pdg config in RepoMeta; force full writeback on mode flip (#2099 F1) Running --pdg against an already-indexed repo silently persisted ~zero CFG: incremental eligibility had no pdg term, RepoMeta recorded no mode, and extractChangedSubgraph keeps only changed-file nodes — on a no-change --pdg re-run every freshly built BasicBlock was dropped from the written subgraph ('Incremental: changed=0', run succeeds, zero rows). The converse flip left zombie mixed-coverage blocks only --force could clean. Worse, a clean-tree flip hit the alreadyUpToDate fast path and never ran the pipeline at all. - RepoMeta gains an additive-optional pdg stamp ({maxFunctionLines, maxEdgesPerFunction}, resolved values; absent ≡ pdg-off, which covers every legacy meta). No INCREMENTAL_SCHEMA_VERSION bump — that would force a one-time full rebuild for everyone. The end-of-run meta is a fresh literal, so omitting the field on a pdg-off run is what clears the stamp after an on→off flip. - pdgModeMismatch (pure, exported) compares the resolved triple; the flip check sits before the fast path and always logs its notice (not gated on options.force — --skills implies force with no message of its own), naming the .gitnexusrc pdg key that pins the mode. - The full-rebuild branch now writes the incrementalInProgress dirty flag (toWriteCount: 0 sentinel) before the wipe whenever a prior meta exists, mirroring the incremental branch. This closes the crash window where a rebuild dying between the bulk load and saveMeta left meta/DB inconsistent and the fast path certified zombie (or missing) CFG rows indefinitely — and incidentally closes the same pre-existing hole for user --force runs. Recovery log reworded accordingly. Tests: pdg-mode-flip.test.ts (real git + LadybugDB; primary assertion is a direct BasicBlock table count — meta.stats aggregates nondeterministic Community/Process rows) covering off→on, steady-state fast path, on→off zombie cleanup, cap-change rebuild, and dirty-flag + flip composition; pure-helper tests for default resolution and the 0=unlimited carve-out. Review finding F1 (P1) of PR #2099 tri-review. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |