From b3ef08b149a46e613b721eeef30bc0c50f43836f Mon Sep 17 00:00:00 2001 From: Gergo Magyar Date: Thu, 18 Jun 2026 16:31:07 +0000 Subject: [PATCH] docs(impact): correct axis-3 figures after the seed-line fix Re-measured the statement-precise reach on the live index after the seed-line proven fix (and the related closure-pin / order-independence fixes). The corrected downstream numbers: the statement-precise subset is strictly tighter than callgraph on 52/90 with-slice functions (median proven 1 vs callgraph 2, median statement-precision 0.67); localization median 0.26 (down) / 0.21 (up). Upstream stays all-proven (callgraph-equal) by design. Full inter-procedural reach remains identical to callgraph (240/240). The earlier figures (43/90, 0.30/0.22) predated the seed-block union, which moved seed-only-call functions from a false all-proven into correct statement-precise discrimination. Addresses PR #2227 tri-review headline follow-up (re-measure + doc correction). Co-Authored-By: Claude Opus 4.8 (1M context) --- gitnexus/bench/impact-pdg/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/gitnexus/bench/impact-pdg/README.md b/gitnexus/bench/impact-pdg/README.md index 2cef1a780..535208908 100644 --- a/gitnexus/bench/impact-pdg/README.md +++ b/gitnexus/bench/impact-pdg/README.md @@ -484,10 +484,10 @@ blocks). | Claim | Verdict | Evidence | |---|---|---| -| **Tighter / fewer false alarms** | ✅ strongly confirmed | *Correctness:* the line-seeded slice equals the curated intra dependence exactly — intra & mixed PDG F1 = 1.000, FPIS = FNIS = 0. *Magnitude:* the slice is a median **0.30** (downstream) / **0.22** (upstream) of the function body; **240/240** functions localize below whole-body — a ~70–78% cut in the intra-procedural inspection set, with no proven dropped dependency. | +| **Tighter / fewer false alarms** | ✅ strongly confirmed | *Correctness:* the line-seeded slice equals the curated intra dependence exactly — intra & mixed PDG F1 = 1.000, FPIS = FNIS = 0. *Magnitude:* the slice is a median **0.26** (downstream) / **0.21** (upstream) of the function body; **240/240** functions localize below whole-body — a ~74–79% cut in the intra-procedural inspection set, with no proven dropped dependency. | | **Catches impact callgraph misses** | ✅ confirmed (new axis) | Callgraph emits *no* statement-level output (unified intra-line CIS = 0, recall 0 on every fixture); PDG recovers every true dependent statement (intra recall = 1.000). PDG answers a def→use / control-dependence question callgraph cannot represent at all. | | **Finds *more* callers/callees** | ❌ refuted (tie, by design) | Full PDG inter-procedural reach is **identical** to callgraph on 240/240 real functions (0 pdg-only, 0 callgraph-only). PDG bridges inter-procedural reach *through* the call graph, so it never finds reach the call graph misses. | -| **Tighter cross-function reach (statement-precise)** | ✅ confirmed (precision, additive) | `mode:'pdg'` now also exposes `statementPreciseByDepth` — the callees actually invoked from the changed line's dependence slice (`BasicBlock.callees`), dropping symbols only reachable from independent statements. Strictly tighter than callgraph on **43/90** with-slice functions (median proven **1** vs callgraph **2** symbols); the full reach stays available alongside it. `statementPrecision` reports the cut. | +| **Tighter cross-function reach (statement-precise)** | ✅ confirmed (precision, additive) | `mode:'pdg'` now also exposes `statementPreciseByDepth` — the callees actually invoked from the changed line's dependence slice (`BasicBlock.callees`), dropping symbols only reachable from independent statements. Strictly tighter than callgraph on **52/90** with-slice functions (median proven **1** vs callgraph **2** symbols, median statement-precision **0.67**); the full reach stays available alongside it. `statementPrecision` reports the cut. Upstream seeds have no statement discriminator, so they stay all-proven (callgraph-equal) by design. | | **Faster / cheaper** | ❌ refuted | PDG carries ~**1.2–1.6×** callgraph latency (the slice query + the slice-callees lookup). It buys precision, not speed. | **Headline.** PDG makes `impact` *much* better at the localization/precision