GitNexus/gitnexus/bench/impact-pdg/baselines.json

{
  "_doc": "U7 impact-PDG accuracy baselines. THREE NON-byte-identity gates (KTD10 + U9): (1) annotationFingerprint — an order-independent digest over the curated ground-truth set (INCLUDING criterion.line, the PDG slice seed, AND the U9 `idBridge` block); any unreviewed edit to a ground-truth.json trips it (re-baseline deliberately after review). (2) f1Bands — a ONE-SIDED F1 regression band per mode per scope: a DROP below (band - epsilon) fails; improvements pass freely. A `null` band means F1 is genuinely undefined for that (scope,mode) on this corpus — the gate skips it (nothing to regress against). (3) U9 resolved-id soundness axis — every fixture carrying an `idBridge` ground-truth block must prove EXACTLY its `idBridge.idProven` set statement-precise (via the resolved symbol id) while the leaf-NAME match strictly over-attributes the eliminated collision id (`idBridge.fpEliminated`). The expected sets live in the ground-truth `idBridge` block, covered by Gate 1's fingerprint, so Gate 3 has no separate baseline number. measure.mjs --check applies median-of-K across GN_IMPACT_PDG_K runs before comparing, so substrate flakiness (the real-analyze lane, F5) cannot trip the band. Re-baseline: `node --import tsx bench/impact-pdg/measure.mjs --json` → copy annotationFingerprint and strata[scope][mode].f1 here, bump analyzerVersion if the analyzer moved.",
  "analyzerVersion": "1.6.7",
  "epsilon": 0.05,
  "annotationFingerprint": "f5792b91c0e2da2b80dbac1cbbee0219aa532806234646c35fa3edfc17d4e700",
  "f1Bands": {
    "intra": {
      "callgraph": null,
      "pdg": 1.0
    },
    "inter": {
      "callgraph": 1.0,
      "pdg": null
    },
    "mixed": {
      "callgraph": 1.0,
      "pdg": 1.0
    }
  },
  "_f1BandsNote": "FU-B-2 re-reconciled: the intra slice is now STATEMENT-granular (the persisted REACHING_DEF edge carries the FULL ordered list of def/use source-line pairs for its (block-pair, binding) group, and the projection walks the self-edge def->use chain forward to fixpoint to recover a coalesced block's interior statements — including a SAME-BINDING reassignment chain `acc@24->acc@25->acc@26`, which all share one deduped edge and need the whole pair list, not just the first pair). The annotationFingerprint MOVED deliberately (d5cb0045->f5792b91): three intra fixtures had their intra_AIS re-reconciled UP to the now-measurable statement set — intra-dataflow-chain {10}->{8,9,10}, intra-control-guard {9,11,13}->{9,11,12,13}, intra-dataflow-reassign {6,7}->{6,7,8} — each restoring the original source-derived belief the prior block-granularity reconciliation had under-counted. Per-fixture justification (NOT a metric re-fit): (a) intra-dataflow-chain (downstream) is independently corroborated — the U2 dynamic value-diff oracle had ALREADY proved behavioral_AIS={8,9,10} (and flagged 8/9 as a manual-annotation circularity miss) BEFORE the edit; (b) intra-control-guard (downstream) is justified by source semantics — line 12 `const z = y + 1` is control-dependent on the guard AND data-dependent on `y@11` in the reached post-guard body block — and likewise corroborated by the downstream U2 oracle; (c) intra-dataflow-reassign is UPSTREAM, so U2 is oracle-direction-EXCLUDED and does NOT justify it — line 8 `total = total + b` is kept purely on SOURCE SEMANTICS: it is the immediately-reaching definition of the criterion use `return total`@9 (the last unkilled def of `total` reaching line 9), a genuine upstream reaching-def. After re-reconciliation intra/pdg stays F1=1.0 (P=1.0 R=1.0 FPIS=0 across all 7 intra fixtures) and mixed/pdg is F1=1.0 (the FU-A intra-tag keeps cross-function callee lines on the inter axis; the recovered interior intra statements match exactly). U7-rework numbers (statement-anchored PDG slice). The two engines are scored at DIFFERENT granularities against DIFFERENT ground truths: PDG at intra-procedural LINE granularity vs intra_AIS (seeded on criterion.line), call-graph at inter-procedural SYMBOL granularity vs inter_AIS. NON-NULL bands: intra/pdg=1.0 (the line-seeded slice exactly reproduces intra_AIS across all 6 intra fixtures — FPIS=0, FNIS=0), mixed/pdg=1.0 (see below), inter/callgraph=1.0 (full cross-function callee recall), mixed/callgraph=1.0 (reaches every callee). mixed/pdg is back at 1.0 after FU-A: the U1 inter-procedural slice still UNIONS the cross-function (callee) reach into affectedStatements, but each statement now carries a projection-only scope:'intra'|'inter' tag (intra iff its owning function file + 1-based start line match the criterion's), and the intra-line axis is scored against the intra-tagged statements only (pdgLineCis(...,'intra')). So the callee lines are no longer counted as intra-axis FPIS (precision climbs 0.468->1.0) while recall stays 1.0 (FNIS=0 — every intra_AIS line is still in the slice). The cross-function reach is preserved on the separate inter symbol axis; the separated per-axis view is in the report's 'Unified impact axes' section. The one-sided band still catches any FURTHER drop (a real recall regression, or added intra over-approximation). NULL cells: intra/callgraph (a self-contained function calls no other symbol → CIS and inter_AIS both empty → F1 n/a) and inter/pdg (a pure-inter router has an empty intra_AIS by design; the line-seeded slice returns the router's own control-dependent statements as FPIS, precision 0, recall n/a → F1 n/a). The intra_AIS of 6 fixtures was reconciled against the live traversal during the rework (CFG block-coalescing of straight-line statements + the combined CDG+RD reverse slice picking up data deps); see each ground-truth.json rationale for the per-fixture correction. These four non-null bands are the live regression guard."
}