.agents/plugins/marketplace.json

@@ -6,7 +6,7 @@
   "plugins": [
     {
       "name": "gitnexus",
-      "version": "1.6.12",
+      "version": "1.6.11",
       "source": {
         "source": "local",
         "path": "./gitnexus-claude-plugin"

.claude-plugin/marketplace.json

@@ -11,7 +11,7 @@
   "plugins": [
     {
       "name": "gitnexus",
-      "version": "1.6.12",
+      "version": "1.6.11",
       "source": "./gitnexus-claude-plugin",
       "description": "Code intelligence powered by a knowledge graph. Provides execution flow tracing, blast radius analysis, and augmented search across your codebase."
     }

.claude/skills/gitnexus-cli/SKILL.md

@@ -34,18 +34,6 @@ Run from the project root. This parses all source files, builds the knowledge gr
 
 For Spring runtime enrichment, pass a JSON bundle, one endpoint JSON file, or a directory containing endpoint files. Route evidence is authoritative only when `runtimeConfirmed === true`; `runtimeSource` records provenance and may also accompany `handler-conflict`. Env/configprops values are never persisted.
 
-## Index storage and retention
-
-Default location is `<repo>/.gitnexus/`. Override with environment variables (also documented in README):
-
-| Env | Effect |
-| --- | ------ |
-| `GITNEXUS_STORAGE_PATH` | One complete external index directory. Wins if both storage vars are set. |
-| `GITNEXUS_STORAGE_ROOT` | Absolute root; GitNexus creates an isolated `<repo-basename>-<12-hex>/` slot per repository. |
-| `GITNEXUS_CONTENT_RETENTION` | `full` (default) keeps file text; `symbol` keeps snippets; `none` keeps the graph only. |
-
-`list_repos`, `gitnexus://repo/{name}/context`, and HTTP `GET /api/repos` / `GET /api/repo` expose `storagePath`, `contentRetention`, and `sourceAvailable`. HTTP `/api/file` and `/api/grep` return 410 unless retention is `full`. MCP `include_content` may still return symbol spans when retention is `symbol`.
-
 Use `node .gitnexus/run.cjs analyze --watch` for a long-lived local Git repository. It performs an initial analysis, queues scanner-admitted file changes, and retries intact failed batches with bounded backoff. Watch refreshes update only the graph: they skip AGENTS.md / CLAUDE.md injection and standard skill installation, so run a one-shot `analyze` when those generated files need updating. Watch rejects one-shot or context-output flags including `--force`, embedding flags, `--skills`, `--default-branch`, `--skip-agents-md`, `--skip-skills`, `--no-stats`, `--self-commit`, `--index-only`, and `--skip-git`. It never pulls remotes. Scheduled remote clone/pull is a different command: `gitnexus auto-sync`. Bare `gitnexus watch` is reserved and does not start either job. Running MCP and `serve` processes periodically check for a published replacement and reopen it without a restart. MCP checks are throttled to once every five seconds, so a tool call before the next check can briefly use the previous index.
 
 ### status — Check index freshness

.claude/skills/gitnexus-debugging/SKILL.md

@@ -42,7 +42,6 @@ diagnosis.
 ```
 
 > If "Index is stale" → run `node .gitnexus/run.cjs analyze` in terminal.
-> Hot-tool `staleness` names which index answered (`branch`/`lastCommit`) and how fresh it is (`status`). Re-analyze only for `behind` or `diverged` — `current` is identity, `unknown` is unmeasurable.
 
 ## Checklist
 

.claude/skills/gitnexus-exploring/SKILL.md

@@ -36,7 +36,6 @@ the bound repository and index freshness alongside your explanation.
 ```
 
 > If step 2 says "Index is stale" → run `node .gitnexus/run.cjs analyze` in terminal.
-> Hot-tool `staleness` names which index answered (`branch`/`lastCommit`) and how fresh it is (`status`). Re-analyze only for `behind` or `diverged` — `current` is identity, `unknown` is unmeasurable.
 
 ## Checklist
 

.claude/skills/gitnexus-guide/SKILL.md

@@ -16,7 +16,6 @@ For any task involving code understanding, debugging, impact analysis, or refact
 3. **Follow the skill's workflow and checklist**
 
 > If step 1 warns the index is stale, run `node .gitnexus/run.cjs analyze` in the terminal first.
-> On `query` / `context` / `impact` / `cypher`, read `staleness.status` and `staleness.branch`/`lastCommit` before using the answer. Re-analyze only for `behind` or `diverged`.
 
 ## Skills
 
@@ -84,51 +83,15 @@ Notes: `offset` ≥ `total` returns an empty page (with `total` still reported).
 
 ### Inline staleness signal (`query` / `context` / `impact` / `cypher`)
 
-These four hot read tools attach a non-blocking `staleness` field to every response, in the shape `{ status, branch?, lastCommit, indexedAt, measuredAgainst, commitsBehind?, hint? }`. It answers two different questions at once: **which index answered** and **how fresh it is**. The identity half is why the field is present even when nothing is wrong — an answer computed from a branch-pinned index is otherwise indistinguishable from one computed from the default branch (#3291):
+These four hot read tools attach a non-blocking `staleness` field to their response when the index is behind the checkout's current HEAD — the same `{ commitsBehind, hint }` shape `list_repos` already reports — so a direct tool call surfaces a behind-HEAD index without a separate `list_repos` call:
 
 ```jsonc
 { /* …the tool's normal result… */
-  "staleness": {
+  "staleness": { "commitsBehind": 3, "hint": "⚠️ Index is 3 commits behind HEAD. Run analyze tool to update." }
-    "status": "current",
-    "branch": "feature/checkout-v2",
-    "lastCommit": "4f2a1c9e8b7d6a5c4e3f2a1b0c9d8e7f6a5b4c3d",
-    "indexedAt": "2026-09-15T07:12:00.000Z",
-    "measuredAgainst": "HEAD"
-  }
 }
 ```
 
-`status: "current"` here means *this index is at the HEAD of the clone it was built from* — not that it is current with the default branch. `measuredAgainst` names what `commitsBehind` is counted against: the checked-out HEAD of that clone, never the remote. `branch` is the branch the index represents; it is absent for a detached HEAD, a non-git folder, or a legacy index that never recorded one, so read `lastCommit` when you need an identifier that is always present.
+The field is **absent when the index is current** (or when the freshness check can't run), so its presence is the signal. It is only ever added to object results — raw-array `cypher` output and error envelopes are returned unchanged. `@group`-targeted calls do not carry it (multi-repo staleness is ill-defined). When you see it, the graph may be behind the working tree — re-run `analyze` before trusting blast-radius or dependence answers.
-
-When the index is behind that HEAD, the count and hint ride along:
-
-```jsonc
-{ /* …the tool's normal result… */
-  "staleness": {
-    "status": "behind", "commitsBehind": 3, "branch": "main",
-    "lastCommit": "a0c945022d06b8815f93ffd8838df9ed5c08cbc0",
-    "indexedAt": "2026-09-04T20:45:47.481Z", "measuredAgainst": "HEAD",
-    "hint": "⚠️ Index is 3 commits behind HEAD. Run analyze tool to update."
-  }
-}
-```
-
-`commitsBehind` is present only when git counted the gap. When git could not count it but HEAD still resolves to a commit other than the indexed one — usually because the indexed commit is no longer in the clone's history — the index is provably not at HEAD with no countable gap, so no number is reported:
-
-```jsonc
-{ /* …the tool's normal result… */
-  "staleness": {
-    "status": "diverged", "branch": "main",
-    "lastCommit": "a0c945022d06b8815f93ffd8838df9ed5c08cbc0",
-    "indexedAt": "2026-09-04T20:45:47.481Z", "measuredAgainst": "HEAD",
-    "hint": "⚠️ Index is not at HEAD and the commit gap could not be counted — the recorded commit may no longer be in this clone's history. Run analyze tool to update."
-  }
-}
-```
-
-So: **read `status` before using `commitsBehind`**, and read `branch`/`lastCommit` before assuming which ref the answer describes. `status: "unknown"` means the freshness check could not run at all (a `--skip-git` folder has no history to measure) — the ref is still reported, because which index answered is knowable even when its freshness is not. The field is only ever added to object results — raw-array `cypher` output and error envelopes are returned unchanged. `@group`-targeted calls do not carry it (multi-repo staleness is ill-defined). Re-run `analyze` only for `behind` or `diverged` — those mean the index is not at this clone's HEAD. `unknown` is unmeasurable, not stale; analyze cannot make it `current` unless git history exists.
-
-`list_repos` and the HTTP repo routes are unchanged: they omit `staleness` entirely for a current index and report the ref through their own top-level `branch` / `lastCommit` / `indexedAt` fields.
 
 ### Taint findings (`explain`)
 

.claude/skills/gitnexus-impact-analysis/SKILL.md

@@ -53,7 +53,6 @@ Repository: <name> (<path>)   Worktree: <path>   Index: <commit>, <n> behind HEA
 ```
 
 > If "Index is stale" → run `node .gitnexus/run.cjs analyze` in terminal.
-> Hot-tool `staleness` names which index answered (`branch`/`lastCommit`) and how fresh it is (`status`). Re-analyze only for `behind` or `diverged` — `current` is identity, `unknown` is unmeasurable.
 > If `.gitnexus/run.cjs` is missing, replace `node .gitnexus/run.cjs` with `npx gitnexus` in the fallback commands.
 
 ## Checklist

.claude/skills/gitnexus-refactoring/SKILL.md

@@ -46,7 +46,6 @@ checkout and reports nothing changed, which reads as a verified refactor.
 ```
 
 > If "Index is stale" → run `node .gitnexus/run.cjs analyze` in terminal.
-> Hot-tool `staleness` names which index answered (`branch`/`lastCommit`) and how fresh it is (`status`). Re-analyze only for `behind` or `diverged` — `current` is identity, `unknown` is unmeasurable.
 
 ## Checklists
 

.factory-plugin/marketplace.json

@@ -1,19 +0,0 @@
-{
-  "name": "gitnexus-marketplace",
-  "owner": {
-    "name": "GitNexus",
-    "email": "nico@gitnexus.dev"
-  },
-  "metadata": {
-    "description": "Code intelligence powered by a knowledge graph — execution flows, blast radius, and semantic search",
-    "homepage": "https://github.com/nicosxt/gitnexus"
-  },
-  "plugins": [
-    {
-      "name": "gitnexus",
-      "version": "1.6.12",
-      "source": "./gitnexus-factory-plugin",
-      "description": "Code intelligence powered by a knowledge graph. Provides execution flow tracing, blast radius analysis, and augmented search across your codebase."
-    }
-  ]
-}

.gitattributes

@@ -15,7 +15,6 @@
 *.so binary
 *.dll binary
 *.dylib binary
-*.lbug_extension binary
 
 # TypeScript sources are always text for diff purposes. Git's binary
 # heuristic fires when EITHER blob in a pair carries a NUL, so a source

.github/actions/setup-gitnexus-web/action.yml

@@ -19,8 +19,8 @@ runs:
         # Browsers are installed explicitly by e2e. Typecheck only needs types.
         PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD: '1'
 
-    # Compile shared with the web package's TypeScript 7. Do not npm-ci
+    # Compile shared with the web package's TypeScript 5. Do not npm-ci
-    # gitnexus-shared (a second optional-platform install, ~7 minutes).
+    # gitnexus-shared (TypeScript 7 optional-platform install, ~7 minutes).
     - name: Build gitnexus-shared
       # node + lib/tsc.js — same on Windows/macOS/Linux. Do not use .bin/tsc
       # (tsc.cmd on Windows; execFileSync cannot launch .cmd without a shell).

.github/dependabot.yml

@@ -89,13 +89,6 @@ updates:
       # tree-sitter-cli follows the runtime's version cadence. Bump when
       # regenerating vendor/tree-sitter-proto/src/parser.c, not on a schedule.
       - dependency-name: tree-sitter-cli
-      # Pin @ladybugdb/core so a daily bump cannot ship a skewed FTS artifact.
-      # The extension version is a separate upstream constant, not derivable
-      # from the core version (see vendor/lbug-fts/manifest.json).
-      - dependency-name: '@ladybugdb/core'
-      - dependency-name: typescript
-        update-types:
-          - version-update:semver-major
 
   # gitnexus-web (thin frontend client).
   - package-ecosystem: npm
@@ -114,10 +107,6 @@ updates:
     labels:
       - dependencies
       - frontend
-    ignore:
-      - dependency-name: typescript
-        update-types:
-          - version-update:semver-major
 
   # Shared types package.
   - package-ecosystem: npm
@@ -135,9 +124,3 @@ updates:
       include: scope
     labels:
       - dependencies
-    ignore:
-      # Keep shared on the same TypeScript major as CLI/web. A shared-only
-      # major bump reintroduces a compiler split this repo unified.
-      - dependency-name: typescript
-        update-types:
-          - version-update:semver-major

.github/scripts/check-tree-sitter-upgrade-readiness.py

@@ -68,9 +68,7 @@ GRAMMARS: dict[str, tuple[str, str, str]] = {
     "tree-sitter-typescript": ("tree-sitter/tree-sitter-typescript", "master",  "typescript/src/parser.c"),
     # Vendored parsers — kept here so the upstream coords for drift
     # detection are co-located with every other grammar's coords.
-    "tree-sitter-objc":       ("tree-sitter-grammars/tree-sitter-objc", "master", "src/parser.c"),
     "tree-sitter-proto":      ("coder3101/tree-sitter-proto",        "main",   "src/parser.c"),
-    "tree-sitter-zig":        ("tree-sitter-grammars/tree-sitter-zig", "master", "src/parser.c"),
 }
 
 # npm-installed grammars deliberately held below npm latest (surfaced so reviewers

.github/scripts/fetch-lbug-fts-artifacts.mjs

@@ -1,144 +0,0 @@
-#!/usr/bin/env node
-/**
- * Fetch Ladybug FTS artifacts into gitnexus/vendor/lbug-fts/prebuilds/.
- *
- * Lives outside the published package (`files` includes `scripts` wholesale).
- * Reads versions, filename, and tuple→upstream-platform mapping from
- * vendor/lbug-fts/manifest.json so the gate and runtime cannot drift.
- *
- * Usage: node .github/scripts/fetch-lbug-fts-artifacts.mjs
- */
-import { createHash } from 'node:crypto';
-import { existsSync, mkdirSync, readFileSync, writeFileSync } from 'node:fs';
-import path from 'node:path';
-import { fileURLToPath, pathToFileURL } from 'node:url';
-
-const REPO_ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..', '..');
-const VENDOR = path.join(REPO_ROOT, 'gitnexus', 'vendor', 'lbug-fts');
-const PREBUILDS = path.join(VENDOR, 'prebuilds');
-const MANIFEST_PATH = path.join(VENDOR, 'manifest.json');
-
-/** Only the Ladybug official extension host — never a manifest-supplied origin. */
-const OFFICIAL_REPO = 'https://extension.ladybugdb.com/';
-const EXACT_VERSION = /^\d+\.\d+\.\d+$/;
-const SAFE_UPSTREAM = /^(linux_amd64|linux_arm64|osx_amd64|osx_arm64|win_amd64)$/;
-
-/**
- * Build the official artifact URL from allowlisted fields only.
- * `officialRepo` in the manifest must match {@link OFFICIAL_REPO}; the
- * origin itself is a constant so an edited manifest cannot redirect the fetch.
- */
-export function officialArtifactUrl(manifest, upstreamPlatform) {
-  const officialRepo = String(manifest?.officialRepo ?? '');
-  if (officialRepo !== OFFICIAL_REPO) {
-    throw new Error(`refusing unofficial FTS repo: '${officialRepo}'`);
-  }
-  const version = String(manifest?.extensionVersion ?? '');
-  if (!EXACT_VERSION.test(version)) {
-    throw new Error(`unsafe extensionVersion: '${version}'`);
-  }
-  if (!SAFE_UPSTREAM.test(String(upstreamPlatform ?? ''))) {
-    throw new Error(`unsafe upstream platform: '${upstreamPlatform}'`);
-  }
-  const filename = String(manifest?.filename ?? '');
-  if (!SAFE_FILENAME.test(filename)) {
-    throw new Error(`unsafe FTS artifact filename: '${filename}'`);
-  }
-  return `${OFFICIAL_REPO}v${version}/${upstreamPlatform}/fts/${filename}`;
-}
-
-const sha256 = (buf) => createHash('sha256').update(buf).digest('hex');
-
-const readExistingHash = (filePath) => {
-  if (!existsSync(filePath)) return null;
-  return sha256(readFileSync(filePath));
-};
-
-export const supportedTuples = (manifest) => manifest.tuples.map((entry) => entry.tuple);
-
-const SAFE_TUPLE = /^(darwin|linux|win32)-(x64|arm64)$/;
-const SAFE_FILENAME = /^[\w.-]+\.lbug_extension$/;
-
-/** Relative-path containment — not a prefix match (rejects `prebuilds-evil`). */
-const isPathInsideRoot = (root, candidate) => {
-  const relative = path.relative(root, candidate);
-  if (path.isAbsolute(relative)) return false;
-  return relative !== '' && !relative.startsWith(`..${path.sep}`) && relative !== '..';
-};
-
-export function assertSafeArtifactDest({ prebuildsDir, tuple, filename }) {
-  if (!SAFE_TUPLE.test(String(tuple ?? ''))) {
-    throw new Error(
-      `unsafe FTS artifact tuple: '${tuple}' (expected (darwin|linux|win32)-(x64|arm64))`,
-    );
-  }
-  if (!SAFE_FILENAME.test(String(filename ?? ''))) {
-    throw new Error(`unsafe FTS artifact filename: '${filename}' (expected *.lbug_extension)`);
-  }
-  const dest = path.join(prebuildsDir, tuple, filename);
-  if (!isPathInsideRoot(prebuildsDir, dest)) {
-    throw new Error(`FTS artifact dest is not inside prebuildsDir: ${dest}`);
-  }
-  return dest;
-}
-
-async function fetchBuffer(url) {
-  // codeql[js/request-forgery] — origin is OFFICIAL_REPO; path segments are allowlisted.
-  // lgtm[js/request-forgery]
-  // codeql[js/file-access-to-http] — versions/platforms are regex-pinned, not raw file bytes.
-  const res = await fetch(url, { signal: AbortSignal.timeout(120_000) });
-  if (!res.ok) {
-    throw new Error(`GET ${url} → ${res.status} ${res.statusText}`);
-  }
-  return Buffer.from(await res.arrayBuffer());
-}
-
-const writeAllowlistedArtifact = (prebuildsDir, dest, buf) => {
-  if (!isPathInsideRoot(prebuildsDir, dest)) {
-    throw new Error(`FTS artifact dest is not inside prebuildsDir: ${dest}`);
-  }
-  // codeql[js/http-to-file-access] — dest is assertSafeArtifactDest + containment-checked.
-  writeFileSync(dest, buf);
-};
-
-export async function refreshArtifacts({
-  manifest = JSON.parse(readFileSync(MANIFEST_PATH, 'utf8')),
-  prebuildsDir = PREBUILDS,
-  download = fetchBuffer,
-} = {}) {
-  mkdirSync(prebuildsDir, { recursive: true });
-  const lines = [];
-  for (const { tuple, upstreamPlatform } of manifest.tuples) {
-    const dest = assertSafeArtifactDest({
-      prebuildsDir,
-      tuple,
-      filename: manifest.filename,
-    });
-    mkdirSync(path.dirname(dest), { recursive: true });
-    const url = officialArtifactUrl(manifest, upstreamPlatform);
-    const previousHash = readExistingHash(dest);
-    const previousSize = previousHash ? readFileSync(dest).byteLength : 0;
-    const buf = await download(url);
-    const nextHash = sha256(buf);
-    writeAllowlistedArtifact(prebuildsDir, dest, buf);
-    const changed = previousHash !== nextHash;
-    console.log(
-      changed
-        ? `[fts-fetch] ${tuple}: ${previousHash ?? '(new)'} (${previousSize} B) → ${nextHash} (${buf.byteLength} B)`
-        : `[fts-fetch] ${tuple}: unchanged ${nextHash} (${buf.byteLength} B)`,
-    );
-    lines.push(`${nextHash}  ./${tuple}/${manifest.filename}`);
-  }
-  lines.sort();
-  writeFileSync(path.join(prebuildsDir, 'SHA256SUMS'), `${lines.join('\n')}\n`);
-  return lines;
-}
-
-const invokedDirectly =
-  process.argv[1] && pathToFileURL(path.resolve(process.argv[1])).href === import.meta.url;
-if (invokedDirectly) {
-  refreshArtifacts().catch((err) => {
-    console.error(`[fts-fetch] ${err instanceof Error ? err.message : err}`);
-    process.exit(1);
-  });
-}

.github/scripts/test_check_tree_sitter_upgrade_readiness.py

@@ -9,8 +9,8 @@ which is deliberately dependency-free so it runs on any vanilla runner. Run with
 (pytest also discovers ``unittest.TestCase`` classes, so a future pytest CI job
 picks these up unchanged.)
 
-These tests lock in the #858 fix: the 7 vendored grammars
+These tests lock in the #858 fix: the 5 vendored grammars
-(c/swift/kotlin/dart/objc/proto/zig) are classified from the shared manifest
+(c/swift/kotlin/dart/proto) are classified from the shared manifest
 (.github/vendored-grammars.json), their ABI is read from gitnexus/vendor/<name>,
 and the report never renders a bare ``?`` placeholder. All network is mocked.
 """
@@ -192,7 +192,7 @@ class AssertCurrent(TestCase):
     def test_assert_current_is_network_free_and_passes(self):
         report, code = self._run_assert_current()  # raises if any urlopen fires
         self.assertEqual(code, 0)
-        # All 7 vendored grammars are introspected from the repo (ABI 14), not skipped.
+        # All 5 vendored grammars are introspected from the repo (ABI 14), not skipped.
         for name in readiness.VENDORED_NAMES:
             self.assertIn(f"{name}: vendored ABI", report)
 
@@ -324,7 +324,6 @@ class ReportRendering(TestCase):
         # which is what removes the old "? (fetch failed)" for tree-sitter-proto.
         self.assertNotIn("tree-sitter-proto", _render_report.last_npm_calls)
         self.assertNotIn("tree-sitter-dart", _render_report.last_npm_calls)
-        self.assertNotIn("@tree-sitter-grammars/tree-sitter-zig", _render_report.last_npm_calls)
         self.assertNotIn("Could not check", self.report)
         self.assertNotIn("fetch failed", self.report)
 
@@ -344,11 +343,11 @@ class ReportRendering(TestCase):
         cells = [c.strip() for c in self._matrix_row("tree-sitter-swift").strip().strip("|").split("|")]
         self.assertEqual(cells[6], "n/a")  # Upstream ABI column
 
-    def test_row_diff_regex_captures_all_grammar_statuses(self):
+    def test_row_diff_regex_captures_all_fifteen_grammar_statuses(self):
         # The change-detection bot keys on this regex: group 1 = grammar name,
         # group 2 = the Status cell ONLY (not the whole tail). It must match every
         # row after the format change so status transitions keep being detected.
-        self.assertEqual(len(self.rows), len(readiness.GRAMMARS))
+        self.assertEqual(len(self.rows), 15)
         for name in readiness.VENDORED_NAMES:
             self.assertIn(name, self.rows)
         # group 2 is the Status cell — held c renders exactly "Vendored — held",
@@ -365,16 +364,15 @@ class ReportRendering(TestCase):
         # Counts are derived from _render_report()'s mock corpus (all npm peer
         # deps mocked permissive): of the 10 npm-installed grammars, 9 render
         # Ready and 1 — tree-sitter-cpp — is the intentional pin (#1242), so it is
-        # not counted ready. The 4 blockers are that same pinned tree-sitter-cpp
+        # not counted ready. The 3 blockers are that same pinned tree-sitter-cpp
-        # plus three held vendored grammars: ABI-held tree-sitter-c (#1242/#858),
+        # plus two held vendored grammars: ABI-held tree-sitter-c (#1242/#858) and
         # tree-sitter-kotlin (pinned to an unreleased fwcd main commit for `fun
         # interface` support — ABI 14 is in range, but a hold counts as a blocker
-        # until it is lifted), and tree-sitter-objc. If a grammar is added/removed
+        # until it is lifted). If a grammar is added/removed or a pin/hold changes,
-        # or a pin/hold changes,
         # update _render_report()'s mock AND these expected counts together; a
         # mismatch here means the report prose drifted, not the regex.
         self.assertEqual(ready.groups(), ("9", "10"))
-        self.assertEqual(blockers.group(1), "4")
+        self.assertEqual(blockers.group(1), "3")
 
     def _matrix_row(self, name: str) -> str:
         for line in self.report.splitlines():

.github/scripts/update-vendored-grammars.mjs

@@ -21,7 +21,7 @@
  *   node update-vendored-grammars.mjs            # detect only → JSON report on stdout
  *   node update-vendored-grammars.mjs --apply X  # re-vendor grammar X in place
  *
- * tree-sitter-c and tree-sitter-objc are MONITORED but report-only (`hold`): c is ABI-pinned at 0.21.4
+ * tree-sitter-c is MONITORED but report-only (`hold`): it is ABI-pinned at 0.21.4
  * (#1242/#858) and must not auto-bump without a tree-sitter runtime upgrade, so an
  * available c update is detected + reported but never auto-applied — even if it is
  * ABI-13/14. A maintainer re-vendors it deliberately.

.github/scripts/verify-workflow-run-pr-identity.cjs

@@ -1,274 +0,0 @@
-// Resolve the open PR for a trusted workflow_run consumer.
-//
-// Shared by commit-fork-prebuilds.yml and pr-autofix-publish.yml.
-// workflow_run.pull_requests[] is empty on fork PRs, and
-// GET /repos/{base}/commits/{sha}/pulls is also empty because the fork head
-// commit is not in the base repo's commit graph. The authoritative lookup is
-// GET /repos/{base}/pulls?head={owner}:{branch}&state=open using
-// workflow_run.head_repository + workflow_run.head_branch (server-controlled).
-// That same query works for same-repo PRs (owner is the base repo owner).
-//
-// The current PR tip may have moved past the SHA the producer built; that is
-// not an identity failure — the caller decides whether to lease-push or just
-// comment. Two open PRs from the same fork head (same owner:branch into this
-// repo) are an identity failure: artifact pr_number is untrusted and must not
-// pick among them. Set SCHEMA_PATTERN to the artifact schema allowlist
-// (defaults to the tree-sitter prebuild schema).
-'use strict';
-
-const fs = require('node:fs');
-const { spawnSync } = require('node:child_process');
-
-const SCHEMA_PATTERN = /^gitnexus\.ts-prebuild\/v[0-9]+$/;
-const IDENTITY_PATTERNS = {
-  pr_number: /^[0-9]+$/,
-  head_sha: /^[0-9a-f]{40}$/,
-  head_ref: /^[A-Za-z0-9._/-]+$/,
-  repo: /^[A-Za-z0-9._-]+\/[A-Za-z0-9._-]+$/,
-};
-
-function allowlistField(key, value, pattern) {
-  const text = value == null ? '' : String(value);
-  if (!text || !pattern.test(text)) {
-    throw new Error(`metadata.${key} failed allowlist (got: ${JSON.stringify(text)})`);
-  }
-  return text;
-}
-
-function forkHeadOwner(headRepo) {
-  const slash = headRepo.indexOf('/');
-  if (slash <= 0 || slash === headRepo.length - 1) {
-    throw new Error(`head_repo must be owner/name (got: ${JSON.stringify(headRepo)})`);
-  }
-  return headRepo.slice(0, slash);
-}
-
-function compileSchemaPattern(value) {
-  if (value instanceof RegExp) return value;
-  if (typeof value === 'string' && value.length > 0) {
-    try {
-      return new RegExp(value);
-    } catch {
-      throw new Error('SCHEMA_PATTERN is not a valid regular expression');
-    }
-  }
-  return SCHEMA_PATTERN;
-}
-
-function allowlistMetadata(raw, schemaPattern) {
-  const parsed = typeof raw === 'string' ? JSON.parse(raw) : raw;
-  if (!parsed || typeof parsed !== 'object' || Array.isArray(parsed)) {
-    throw new Error('metadata.json must be an object');
-  }
-  return {
-    schema: allowlistField('schema', parsed.schema, compileSchemaPattern(schemaPattern)),
-    pr_number: allowlistField('pr_number', parsed.pr_number, IDENTITY_PATTERNS.pr_number),
-    head_sha: allowlistField('head_sha', parsed.head_sha, IDENTITY_PATTERNS.head_sha),
-    head_ref: allowlistField('head_ref', parsed.head_ref, IDENTITY_PATTERNS.head_ref),
-    head_repo: allowlistField('head_repo', parsed.head_repo, IDENTITY_PATTERNS.repo),
-    base_repo: allowlistField('base_repo', parsed.base_repo, IDENTITY_PATTERNS.repo),
-  };
-}
-
-function allowlistAuthority(authority) {
-  return {
-    head_sha: allowlistField('head_sha', authority.head_sha, IDENTITY_PATTERNS.head_sha),
-    head_repo: allowlistField('head_repo', authority.head_repo, IDENTITY_PATTERNS.repo),
-    head_branch: allowlistField('head_ref', authority.head_branch, IDENTITY_PATTERNS.head_ref),
-    base_repo: allowlistField('base_repo', authority.base_repo, IDENTITY_PATTERNS.repo),
-  };
-}
-
-function verifyArtifactAgainstWorkflowRun(meta, authority) {
-  if (meta.head_sha !== authority.head_sha) {
-    throw new Error(
-      `Artifact head_sha (${meta.head_sha}) != workflow_run.head_sha (${authority.head_sha}) — refusing.`,
-    );
-  }
-  if (meta.head_repo !== authority.head_repo) {
-    throw new Error(
-      `Artifact head_repo (${meta.head_repo}) != workflow_run.head_repository (${authority.head_repo}) — refusing.`,
-    );
-  }
-  if (meta.base_repo !== authority.base_repo) {
-    throw new Error('Artifact base_repo does not match $GITHUB_REPOSITORY — refusing.');
-  }
-  if (meta.head_ref !== authority.head_branch) {
-    throw new Error(
-      `Artifact head_ref (${meta.head_ref}) != workflow_run.head_branch (${authority.head_branch}) — refusing.`,
-    );
-  }
-}
-
-function matchOpenPullsFromForkHead(pulls, { headRepo, headBranch, baseRepo }) {
-  if (!Array.isArray(pulls)) {
-    throw new Error('GitHub pulls?head= lookup returned a non-array');
-  }
-  return pulls.filter((pr) => {
-    return (
-      pr &&
-      pr.state === 'open' &&
-      Number.isInteger(pr.number) &&
-      pr.head &&
-      pr.head.repo &&
-      pr.head.repo.full_name === headRepo &&
-      pr.head.ref === headBranch &&
-      pr.base &&
-      pr.base.repo &&
-      pr.base.repo.full_name === baseRepo
-    );
-  });
-}
-
-function resolveVerifiedPullRequest({ meta, authority, pulls, schemaPattern }) {
-  const cleanMeta = allowlistMetadata(meta, schemaPattern);
-  const cleanAuthority = allowlistAuthority(authority);
-  verifyArtifactAgainstWorkflowRun(cleanMeta, cleanAuthority);
-
-  const matched = matchOpenPullsFromForkHead(pulls, {
-    headRepo: cleanAuthority.head_repo,
-    headBranch: cleanAuthority.head_branch,
-    baseRepo: cleanAuthority.base_repo,
-  });
-
-  if (matched.length === 0) {
-    throw new Error(
-      `No open PR from ${cleanAuthority.head_repo}:${cleanAuthority.head_branch} targeting ${cleanAuthority.base_repo} — refusing.`,
-    );
-  }
-
-  // Artifact pr_number is untrusted. Do not use it to pick among several open
-  // PRs that share this fork head (same owner:branch into this repo, different
-  // base branches). Fail closed unless GitHub-controlled fields leave exactly one.
-  if (matched.length !== 1) {
-    throw new Error(
-      `Ambiguous open PRs from ${cleanAuthority.head_repo}:${cleanAuthority.head_branch} targeting ${cleanAuthority.base_repo} (${matched
-        .map((pr) => pr.number)
-        .join(',')}) — refusing.`,
-    );
-  }
-
-  const chosen = matched[0];
-  const expected = Number(cleanMeta.pr_number);
-  if (chosen.number !== expected) {
-    throw new Error(
-      `Artifact pr_number (${cleanMeta.pr_number}) is not the open PR(s) from this fork head (${chosen.number}) — refusing.`,
-    );
-  }
-
-  const currentHeadSha = typeof chosen.head.sha === 'string' ? chosen.head.sha : '';
-  return {
-    pr_number: String(chosen.number),
-    head_ref: cleanAuthority.head_branch,
-    head_sha: cleanAuthority.head_sha,
-    head_repo: cleanAuthority.head_repo,
-    current_head_sha: currentHeadSha,
-    branch_moved: Boolean(currentHeadSha && currentHeadSha !== cleanAuthority.head_sha),
-  };
-}
-
-function flattenGhListPages(parsed) {
-  if (!Array.isArray(parsed)) {
-    throw new Error('GitHub pulls?head= lookup returned a non-array');
-  }
-  if (parsed.length === 0) return parsed;
-  if (parsed.every((page) => Array.isArray(page))) {
-    return parsed.flat();
-  }
-  return parsed;
-}
-
-function listOpenPullsByHead({ ghRepo, headOwner, headBranch, runGh }) {
-  const run = runGh || ((args) => spawnSync('gh', args, { encoding: 'utf8' }));
-  const result = run([
-    'api',
-    '--paginate',
-    '--slurp',
-    '-X',
-    'GET',
-    `repos/${ghRepo}/pulls`,
-    '-f',
-    'state=open',
-    '-f',
-    `head=${headOwner}:${headBranch}`,
-  ]);
-  if (result.status !== 0) {
-    const err = (result.stderr || result.stdout || '').trim();
-    throw new Error(`GitHub pulls?head= lookup failed: ${err || `exit ${result.status}`}`);
-  }
-  const stdout = (result.stdout || '').trim();
-  if (!stdout) {
-    throw new Error('GitHub pulls?head= lookup returned an empty body');
-  }
-  let parsed;
-  try {
-    parsed = JSON.parse(stdout);
-  } catch {
-    throw new Error('GitHub pulls?head= lookup returned non-JSON');
-  }
-  return flattenGhListPages(parsed);
-}
-
-function main() {
-  const schemaPattern = compileSchemaPattern(process.env.SCHEMA_PATTERN);
-  const raw = fs.readFileSync(process.env.META_PATH, 'utf8');
-  const meta = allowlistMetadata(raw, schemaPattern);
-  const authority = allowlistAuthority({
-    head_sha: process.env.WF_HEAD_SHA,
-    head_repo: process.env.WF_HEAD_REPO,
-    head_branch: process.env.WF_HEAD_BRANCH,
-    base_repo: process.env.GH_REPO,
-  });
-  const pulls = listOpenPullsByHead({
-    ghRepo: authority.base_repo,
-    headOwner: forkHeadOwner(authority.head_repo),
-    headBranch: authority.head_branch,
-  });
-  const verified = resolveVerifiedPullRequest({ meta, authority, pulls, schemaPattern });
-  if (verified.branch_moved) {
-    console.log(
-      `PR head moved to ${verified.current_head_sha}; delivering against built SHA ${verified.head_sha} (lease will refuse if the branch moved).`,
-    );
-  }
-  console.log(
-    `Verified identity: PR=${verified.pr_number} head_sha=${verified.head_sha} head_repo=${verified.head_repo} head_ref=${verified.head_ref}.`,
-  );
-  const out = process.env.GITHUB_OUTPUT;
-  if (!out) {
-    throw new Error('GITHUB_OUTPUT is unset');
-  }
-  fs.appendFileSync(
-    out,
-    [
-      `pr_number=${verified.pr_number}`,
-      `head_ref=${verified.head_ref}`,
-      `head_sha=${verified.head_sha}`,
-      `head_repo=${verified.head_repo}`,
-    ].join('\n') + '\n',
-  );
-}
-
-if (require.main === module) {
-  try {
-    main();
-  } catch (err) {
-    console.error(`::error::${err instanceof Error ? err.message : String(err)}`);
-    process.exit(1);
-  }
-}
-
-module.exports = {
-  SCHEMA_PATTERN,
-  IDENTITY_PATTERNS,
-  allowlistField,
-  compileSchemaPattern,
-  allowlistMetadata,
-  allowlistAuthority,
-  forkHeadOwner,
-  verifyArtifactAgainstWorkflowRun,
-  matchOpenPullsFromForkHead,
-  flattenGhListPages,
-  resolveVerifiedPullRequest,
-  listOpenPullsByHead,
-  main,
-};

.github/vendored-grammars.json

@@ -6,11 +6,6 @@
       "upstream": { "npm": "tree-sitter-c" },
       "hold": "ABI-pinned at 0.21.4 (#1242/#858) — needs a tree-sitter runtime upgrade before bumping"
     },
-    "objc": {
-      "name": "tree-sitter-objc",
-      "upstream": { "npm": "tree-sitter-objc" },
-      "hold": "Pinned at 3.0.2 for the Objective-C provider MVP; carries darwin/linux arm64+x64 prebuilds compatible with the current tree-sitter runtime (linux-arm64 built from vendored source because the upstream npm artifact is mislabeled)"
-    },
     "swift": {
       "name": "tree-sitter-swift",
       "upstream": { "npm": "tree-sitter-swift" }
@@ -27,10 +22,6 @@
     "proto": {
       "name": "tree-sitter-proto",
       "upstream": { "github": "coder3101/tree-sitter-proto" }
-    },
-    "zig": {
-      "name": "tree-sitter-zig",
-      "upstream": { "npm": "@tree-sitter-grammars/tree-sitter-zig" }
     }
   }
 }

.github/workflows/build-tree-sitter-prebuilds.yml

@@ -7,7 +7,7 @@ name: Build tree-sitter prebuilds
 #
 # Grammars covered here (the at-risk set — everything else already ships 6
 # upstream prebuilds AND stays dependency-review-tracked, so it is left alone).
-# All seven are vendored under gitnexus/vendor/; `kind` (below) only picks where
+# All five are vendored under gitnexus/vendor/; `kind` (below) only picks where
 # the build job fetches the C source to compile:
 #   - tree-sitter-c      (vendored prebuild-only; built from the published npm
 #                         package — closes upstream's 4/6 ARM gap #2116 for a
@@ -17,22 +17,15 @@ name: Build tree-sitter prebuilds
 #   - tree-sitter-kotlin (vendored source; built from gitnexus/vendor/ — pinned to
 #                         an unreleased main commit for `fun interface` support
 #                         (#169) that no npm release carries yet)
-#   - tree-sitter-objc   (vendored source; built from gitnexus/vendor/ — pinned
-#                         for the Objective-C provider MVP)
 #   - tree-sitter-swift  (vendored source; built from gitnexus/vendor/ — its
 #                         prebuilds were originally upstream-shipped, now
 #                         GitNexus-cross-built like the rest for uniformity)
-#   - tree-sitter-zig    (vendored source; built from gitnexus/vendor/ — moved
-#                         off npm optionalDependency so `npm i -g gitnexus`
-#                         no longer warns on peerOptional tree-sitter@^0.22.1.
-#                         Upstream linux-arm64 prebuild is a mispackaged
-#                         x86-64 binary; this workflow rebuilds all seven.)
 #
 # Output: gitnexus/vendor/<grammar>/prebuilds/<platform-arch>/<grammar>.node for
 # all 6 targets ({linux,darwin,win32}-{x64,arm64}). tree-sitter grammars are
 # N-API, so one ABI-stable .node per platform-arch works across all Node majors.
 #
-# COST DISCIPLINE — this is a HEAVY native matrix (up to 7 grammars x 6 runners,
+# COST DISCIPLINE — this is a HEAVY native matrix (up to 3 grammars x 6 runners,
 # incl. macOS + arm64). It is DELIBERATELY NOT wired into normal PR/push CI. It
 # runs only:
 #   1. on manual dispatch (workflow_dispatch); or
@@ -40,9 +33,8 @@ name: Build tree-sitter prebuilds
 #      OR an edit to the grammar's build-affecting source (parser.c / grammar.js /
 #      binding.gyp / scanner / bindings). The `guard` job is the real gate (it
 #      diffs BOTH the recorded version AND the source files vs the PR base); the
-#      `paths:` filter below keeps ordinary code PRs at ZERO matrix time. PR
+#      `paths:` filter below keeps ordinary code PRs at ZERO matrix time and
-#      filters see the cumulative diff, so the guard separately skips updates
+#      excludes the prebuilds the job commits back, so it never retriggers itself.
-#      containing only the prebuilds the job commits back.
 # Net effect: an ordinary code PR triggers nothing; touching one grammar's source
 # costs exactly one matrix run for that grammar. Delivery of the rebuilt binaries:
 #   - same-repo PR  -> committed straight onto the PR's own branch (in the SAME PR);
@@ -63,7 +55,7 @@ on:
   workflow_dispatch:
     inputs:
       grammars:
-        description: 'Comma-separated grammar shortnames to build (c,dart,proto,kotlin,objc,swift,zig), or "all".'
+        description: 'Comma-separated grammar shortnames to build (c,dart,proto,kotlin,swift), or "all".'
         required: false
         type: string
         default: 'all'
@@ -88,14 +80,13 @@ on:
       # Any build-affecting change under a vendored grammar triggers a rebuild —
       # not just a version bump — so editing the vendored source (parser.c,
       # grammar.js, binding.gyp, scanner, bindings) re-cuts the prebuilds too.
-      # Excludes PRs containing only prebuilds. A source PR still matches after a
+      # The prebuilds we commit back are EXCLUDED (negated last) so the bot's own
-      # bot commit because PR filters use the cumulative diff; `guard` stops the
+      # in-PR commit can never retrigger this workflow (no build->commit->build loop).
-      # build->commit->build loop using the synchronize event's before/head diff.
       - 'gitnexus/vendor/tree-sitter-*/**'
       - '!gitnexus/vendor/tree-sitter-*/prebuilds/**'
       # Self-test: re-run the guard if a future grammar pin is reintroduced in
       # the main package.json (optionalDependencies fallback). No-op otherwise —
-      # all seven grammars are now fully vendored (including kotlin, objc, and zig).
+      # all five grammars are now fully vendored (kotlin included).
       - 'gitnexus/package.json'
       # Self-test: re-run the guard (normally a no-op) when the recipe changes.
       - '.github/workflows/build-tree-sitter-prebuilds.yml'
@@ -123,7 +114,7 @@ jobs:
       matrix: ${{ steps.decide.outputs.matrix }}
       release_app: ${{ steps.relapp.outputs.configured }}
     steps:
-      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
         with:
           fetch-depth: 0 # need base history to diff recorded versions
           persist-credentials: false
@@ -132,9 +123,6 @@ jobs:
         id: decide
         env:
           EVENT: ${{ github.event_name }}
-          ACTION: ${{ github.event.action }}
-          BEFORE_SHA: ${{ github.event.before }}
-          HEAD_SHA: ${{ github.event.pull_request.head.sha }}
           # Untrusted dispatch inputs — read via env only, validated in JS.
           INPUT_GRAMMARS: ${{ inputs.grammars }}
           INPUT_REF: ${{ inputs.ref }}
@@ -143,7 +131,7 @@ jobs:
         run: |
           set -euo pipefail
           node --input-type=module - <<'NODE'
-          import { execFileSync, execSync } from 'node:child_process';
+          import { execSync } from 'node:child_process';
           import fs from 'node:fs';
           import { appendFileSync } from 'node:fs';
 
@@ -165,18 +153,10 @@ jobs:
             // unreleased main commit for `fun interface` support (#169) that no
             // npm release carries yet — so it must build from the vendored source.
             kotlin: { name: 'tree-sitter-kotlin', kind: 'vendored' },
-            // Objective-C is vendored WITH its source and its native bindings
-            // must be recut together with the pinned grammar snapshot.
-            objc:   { name: 'tree-sitter-objc',   kind: 'vendored' },
             // swift is vendored WITH its source (parser.c/scanner.c/binding.gyp),
             // so it builds from gitnexus/vendor/ like dart/proto. Its prebuilds
             // were originally upstream-shipped; rebuilding them here unifies it.
             swift:  { name: 'tree-sitter-swift',  kind: 'vendored' },
-            // zig is vendored WITH its source (parser.c/binding.gyp). Moved off
-            // the npm optionalDependency so published installs no longer warn
-            // on peerOptional tree-sitter@^0.22.1. Upstream linux-arm64
-            // prebuild is a mispackaged x86-64 binary; rebuild here.
-            zig:    { name: 'tree-sitter-zig',    kind: 'vendored' },
           };
           const PLATFORMS = [
             { platform_arch: 'linux-x64',    os: 'ubuntu-24.04' },
@@ -207,30 +187,6 @@ jobs:
           const event = process.env.EVENT;
           const force = process.env.FORCE === 'true';
 
-          // PR path filters and the source/version checks below see the entire
-          // PR, so excluding prebuilds there does NOT prevent a rebuild loop.
-          // Check the whole push (not HEAD^ or the author's identity): a push
-          // containing source edits followed by a binary commit must still build.
-          if (event === 'pull_request' && process.env.ACTION === 'synchronize') {
-            const before = process.env.BEFORE_SHA;
-            const head = process.env.HEAD_SHA;
-            for (const sha of [before, head]) {
-              if (!sha || !/^[0-9a-fA-F]{40}$/.test(sha)) {
-                throw new Error('synchronize requires valid before/head SHAs; refusing an unbounded rebuild');
-              }
-            }
-            // Fail closed if either commit is unavailable. Never fall back to
-            // the cumulative PR diff, which would re-enable the loop.
-            const changed = execFileSync('git', [
-              'diff', '--name-only', '--no-renames', '-z', before, head, '--',
-            ], { encoding: 'utf8' }).split('\0').filter(Boolean);
-            if (changed.every((p) => /^gitnexus\/vendor\/tree-sitter-[^/]+\/prebuilds\//.test(p))) {
-              appendFileSync(process.env.GITHUB_OUTPUT, 'any=false\nmatrix={"include":[]}\n');
-              console.log('::notice::Push changes only prebuild outputs (or no files) — skipping native matrix.');
-              process.exit(0);
-            }
-          }
-
           // Select which grammar shortnames are in play.
           let selected;
           if (event === 'workflow_dispatch') {
@@ -287,9 +243,9 @@ jobs:
             } else {
               // pull_request: build when the recorded version changed OR any
               // build-affecting source file under the vendored grammar changed vs
-              // the PR base. Exclude generated outputs from build inputs; the
+              // the PR base. The prebuilds/ subtree is excluded from the diff so
-              // synchronize check above prevents rebuilding the original source
+              // the bot's own in-PR commit (which adds ONLY prebuilds) never reads
-              // change after every generated-prebuild commit.
+              // as a source change — this is the other half of the no-loop guard.
               const base = recordedVersion(baseRoot, name);
               const versionChanged = !!head && head !== base;
               let sourceChanged = false;
@@ -392,7 +348,7 @@ jobs:
     # and compiling them under emulation on the arm runners is slow.
     timeout-minutes: 45
     steps:
-      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
         with:
           persist-credentials: false # this job uploads artifacts (artipacked)
 
@@ -510,16 +466,12 @@ jobs:
               dart:   "void main() { print(\"hi\"); }",
               proto:  "syntax = \"proto3\";\nmessage M { int32 id = 1; }",
               kotlin: "fun main() { println(\"hi\") }",
-              objc:   "@interface GNValidationProbe : NSObject\n@end",
               swift:  "func greet() { print(\"hi\") }",
-              zig:    "pub fn main() void {}",
             };
-            const src = snippets[process.env.GRAMMAR];
-            if (!src) throw new Error("no validate snippet for grammar: " + process.env.GRAMMAR);
             const lang = require("node-gyp-build")(process.cwd());
             const Parser = require("tree-sitter");
             const p = new Parser(); p.setLanguage(lang);
-            const tree = p.parse(src);
+            const tree = p.parse(snippets[process.env.GRAMMAR]);
             if (!tree || !tree.rootNode || tree.rootNode.hasError) {
               throw new Error("parse failed/error: " + (tree && tree.rootNode && tree.rootNode.type));
             }
@@ -565,7 +517,7 @@ jobs:
           app-id: ${{ secrets.RELEASE_APP_ID }}
           private-key: ${{ secrets.RELEASE_APP_PRIVATE_KEY }}
 
-      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
         with:
           token: ${{ steps.app-token.outputs.token }}
           # On a (non-fork) PR, check out the PR's HEAD branch — not the merge ref —

.github/workflows/ci-devcontainer.yml

@@ -36,7 +36,7 @@ jobs:
       # persist-credentials: false — this job only reads (tests and syntax
       # checks) and never pushes. The setting keeps GITHUB_TOKEN out of
       # .git/config, which zizmor flags as the "artipacked" issue.
-      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
         with:
           persist-credentials: false
       - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
@@ -57,7 +57,7 @@ jobs:
       # persist-credentials: false — this is a read-only build smoke that
       # never pushes. The setting keeps GITHUB_TOKEN out of .git/config,
       # which zizmor flags as the "artipacked" issue.
-      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
         with:
           persist-credentials: false
       - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0

.github/workflows/ci-e2e.yml

@@ -14,7 +14,7 @@ jobs:
     outputs:
       web_changed: ${{ steps.filter.outputs.web }}
     steps:
-      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
         with:
           persist-credentials: false
       - uses: dorny/paths-filter@ceb8a2b8f2d89434be7ff52d3de7ec3738c5cc9d # v3
@@ -31,7 +31,7 @@ jobs:
     runs-on: ubuntu-latest
     timeout-minutes: 20
     steps:
-      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
         with:
           persist-credentials: false
 

.github/workflows/ci-quality.yml

@@ -13,7 +13,7 @@ jobs:
     # canceled prettier at the 5-minute job cap; lint needed 7m41s the same run.
     timeout-minutes: 10
     steps:
-      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
         with:
           persist-credentials: false
       - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
@@ -28,7 +28,7 @@ jobs:
     runs-on: ubuntu-latest
     timeout-minutes: 10
     steps:
-      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
         with:
           persist-credentials: false
       - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
@@ -43,7 +43,7 @@ jobs:
     runs-on: ubuntu-latest
     timeout-minutes: 10
     steps:
-      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
         with:
           persist-credentials: false
       # tsc --noEmit reads source + gitnexus-shared/dist. Skip prepare/postinstall
@@ -53,9 +53,6 @@ jobs:
           lifecycle-scripts: 'false'
       - run: npx tsc --noEmit
         working-directory: gitnexus
-      - name: Typecheck tests
-        run: npm run typecheck:tests
-        working-directory: gitnexus
 
   typecheck-web:
     runs-on: ubuntu-latest
@@ -64,7 +61,7 @@ jobs:
     # run is cold again.
     timeout-minutes: 15
     steps:
-      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
         with:
           persist-credentials: false
       - uses: ./.github/actions/setup-gitnexus-web
@@ -87,7 +84,7 @@ jobs:
     runs-on: ubuntu-latest
     timeout-minutes: 5
     steps:
-      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
         with:
           persist-credentials: false
       - name: Validate workflow concurrency convention

.github/workflows/ci-report.yml

@@ -125,7 +125,7 @@ jobs:
 
       - name: Checkout (for vitest config)
         if: steps.meta.outputs.skip != 'true'
-        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
         with:
           sparse-checkout: gitnexus/vitest.config.ts
           sparse-checkout-cone-mode: false
@@ -287,7 +287,6 @@ jobs:
           # ── Locate test results ──
           RESULTS_FILE=$(find_first "$DIR/test-reports" "test-results.json")
           WEB_RESULTS_FILE=$(find_first "$DIR/test-reports" "web-test-results.json")
-          PYTEST_RESULTS_FILE=$(find_first "$DIR/test-reports" "pytest-results.json")
 
           sum_results() {
             local file=$1
@@ -304,21 +303,12 @@ jobs:
           # framework, not as a top-line metric).
           read -r CLI_T CLI_P CLI_F CLI_S _ CLI_D <<< "$(sum_results "$RESULTS_FILE")"
           read -r WEB_T WEB_P WEB_F WEB_S _ WEB_D <<< "$(sum_results "$WEB_RESULTS_FILE")"
-          read -r PY_T PY_P PY_F PY_S _ PY_D <<< "$(sum_results "$PYTEST_RESULTS_FILE")"
 
-          TOTAL=$((CLI_T + WEB_T + PY_T))
+          TOTAL=$((CLI_T + WEB_T))
-          PASSED=$((CLI_P + WEB_P + PY_P))
+          PASSED=$((CLI_P + WEB_P))
-          FAILED=$((CLI_F + WEB_F + PY_F))
+          FAILED=$((CLI_F + WEB_F))
-          SKIPPED=$((CLI_S + WEB_S + PY_S))
+          SKIPPED=$((CLI_S + WEB_S))
           DURATION=$((CLI_D > WEB_D ? CLI_D : WEB_D))
-          DURATION=$((DURATION > PY_D ? DURATION : PY_D))
-          EXECUTION_ERRORS=0
-          for rf in "$RESULTS_FILE" "$WEB_RESULTS_FILE" "$PYTEST_RESULTS_FILE"; do
-            if [ -n "$rf" ] && [ -f "$rf" ]; then
-              errors=$(jq -r '(.executionFailures // []) | length' "$rf")
-              EXECUTION_ERRORS=$((EXECUTION_ERRORS + errors))
-            fi
-          done
 
           # ── Status helpers ──
           status_icon() {
@@ -395,22 +385,22 @@ jobs:
             if [ "$TOTAL" -gt 0 ] 2>/dev/null; then
               echo "### Test Results"
               echo ""
-              echo "| Tests | Passed | Failed | Unverified | Duration |"
+              echo "| Tests | Passed | Failed | Skipped | Duration |"
               echo "|-------|--------|--------|---------|----------|"
               echo "| ${TOTAL} | ${PASSED} | ${FAILED} | ${SKIPPED} | ${DURATION}s |"
               echo ""
 
-              if [[ "$FAILED" == "0" && "$SKIPPED" == "0" && "$EXECUTION_ERRORS" == "0" && "$TESTS" == "success" ]]; then
+              if [ "$FAILED" = "0" ]; then
                 echo "✅ All **${PASSED}** tests passed"
               else
-                echo "❌ Test execution is incomplete or failed: **${FAILED}** failed, **${SKIPPED}** unverified, **${EXECUTION_ERRORS}** runner/suite errors."
+                echo "❌ **${FAILED}** failed / **${PASSED}** passed"
               fi
               if [ "$SKIPPED" != "0" ]; then
                 echo ""
                 echo "<details>"
-                echo "<summary>${SKIPPED} test(s) have no recorded pass — expand for details</summary>"
+                echo "<summary>${SKIPPED} test(s) skipped — expand for details</summary>"
                 echo ""
-                for rf in "$RESULTS_FILE" "$WEB_RESULTS_FILE" "$PYTEST_RESULTS_FILE"; do
+                for rf in "$RESULTS_FILE" "$WEB_RESULTS_FILE"; do
                   if [ -n "$rf" ] && [ -f "$rf" ]; then
                     jq -r '
                       .testResults[]

.github/workflows/ci-tests.yml

@@ -24,20 +24,19 @@ jobs:
         shard: ${{ fromJSON(needs.shard-plan.outputs.cov_shards) }}
     # Fail loudly (don't silently skip) if the FTS extension is unavailable, so
     # FTS-dependent lbug integration suites are guaranteed to run in CI.
-    # Same contract for Zig's vendored grammar: this runner is linux-x64,
+    # Same contract for Zig's optionalDependency grammar: this runner is
-    # which vendor/tree-sitter-zig ships a prebuild for, so an absent grammar
+    # linux-x64, which @tree-sitter-grammars/tree-sitter-zig publishes a
-    # here is a packaging regression and not an unsupported platform. Without
+    # prebuild for, so an absent grammar here is a packaging regression and not
-    # it every Zig suite skips and the job is green having never executed the
+    # an unsupported platform. Without it every Zig suite skips and the job is
-    # native Zig parser once.
+    # green having never executed the native Zig parser once.
     env:
       GITNEXUS_REQUIRE_FTS: '1'
       GITNEXUS_REQUIRE_ZIG: '1'
-      GITNEXUS_REQUIRE_VECTOR: '1'
     steps:
       # persist-credentials: false — runs tests + uploads a blob artifact; the
       # default-persisted token must not be capturable through it (zizmor
       # credential-persistence / artipacked audit). The job never pushes.
-      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
         with:
           persist-credentials: false
       - uses: ./.github/actions/setup-gitnexus
@@ -80,8 +79,8 @@ jobs:
         uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
         with:
           name: coverage-blob-${{ matrix.shard }}
-          path: gitnexus/.vitest/blob/
+          path: gitnexus/.vitest-reports/
-          # .vitest is a dotdir; upload-artifact excludes hidden files by
+          # .vitest-reports is a dotdir; upload-artifact excludes hidden files by
           # default, which would upload an empty artifact and break the merge.
           include-hidden-files: true
           retention-days: 5
@@ -100,7 +99,7 @@ jobs:
     env:
       GITNEXUS_REQUIRE_FTS: '1'
     steps:
-      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
         with:
           persist-credentials: false
       - uses: ./.github/actions/setup-gitnexus
@@ -110,19 +109,19 @@ jobs:
         uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
         with:
           pattern: coverage-blob-*
-          path: gitnexus/.vitest/blob
+          path: gitnexus/.vitest-reports
           merge-multiple: true
       - name: Merge coverage + enforce thresholds
         run: >-
-          npx vitest --mergeReports=.vitest/blob
+          npx vitest --mergeReports
           --reporter=default
-          --reporter=./scripts/execution-reporter.ts
+          --reporter=json
           --outputFile=test-results.json
           --coverage
           --coverage.reporter=json-summary
           --coverage.reporter=json
           --coverage.reporter=text
-          --coverage.thresholds.autoUpdate=false
+          --coverage.thresholdAutoUpdate=false
         working-directory: gitnexus
       # gitnexus-shared already built by setup-gitnexus above
       - name: Install gitnexus-web dependencies
@@ -132,8 +131,7 @@ jobs:
         run: >-
           npx vitest run
           --reporter=default
-          --reporter=../gitnexus/scripts/execution-reporter.ts
+          --reporter=json
-          --includeTaskLocation
           --outputFile=web-test-results.json
         working-directory: gitnexus-web
       - name: Run docker-server integration tests
@@ -142,7 +140,7 @@ jobs:
         if: always()
         uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
         with:
-          name: coverage-reports
+          name: test-reports
           path: |
             gitnexus/coverage/coverage-summary.json
             gitnexus/coverage/coverage-final.json
@@ -164,7 +162,7 @@ jobs:
     steps:
       - id: gen
         run: |
-          TOTAL=4         # cross-platform (windows/macOS) shards per OS
+          TOTAL=3         # cross-platform (windows/macOS) shards per OS
           COV_TOTAL=3     # ubuntu coverage shards (merged before thresholds)
           if [ "$TOTAL" -lt 1 ] || [ "$COV_TOTAL" -lt 1 ]; then
             echo "shard totals must be >= 1" >&2; exit 1
@@ -225,7 +223,7 @@ jobs:
     steps:
       # persist-credentials: false — runs tests only, never pushes (zizmor
       # credential-persistence / artipacked audit).
-      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
         with:
           persist-credentials: false
       - uses: ./.github/actions/setup-gitnexus
@@ -256,48 +254,8 @@ jobs:
         shell: bash
         env:
           SHARD: ${{ matrix.shard }}/${{ needs.shard-plan.outputs.total }}
-          GITNEXUS_TEST_REPORT: ${{ matrix.os }}-${{ matrix.shard }}.json
         run: npx tsx scripts/run-cross-platform.ts --shard="$SHARD"
         working-directory: gitnexus
-      - name: Upload platform execution report
-        if: always()
-        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
-        with:
-          name: execution-${{ matrix.os }}-${{ matrix.shard }}
-          path: gitnexus/${{ matrix.os }}-${{ matrix.shard }}.json
-          if-no-files-found: error
-          retention-days: 5
-
-  # Branch protection still requires these six names from the three-shard
-  # matrix. Keep them as aggregate gates as the real matrix grows: every native
-  # shard and the complete execution audit must pass before any gate succeeds.
-  # These jobs only verify dependency results; tests run in cross-platform above.
-  protected-platform-checks:
-    name: ${{ matrix.os }} (platform-sensitive) ${{ matrix.check }}/3
-    needs: [cross-platform, test-completeness]
-    if: always()
-    permissions: {}
-    strategy:
-      fail-fast: false
-      matrix:
-        os: [windows-latest, macos-latest]
-        check: [1, 2, 3]
-    runs-on: ubuntu-latest
-    timeout-minutes: 2
-    steps:
-      - name: Require all native shards and the execution audit
-        shell: bash
-        env:
-          NATIVE_RESULT: ${{ needs.cross-platform.result }}
-          EXECUTION_RESULT: ${{ needs.test-completeness.result }}
-        run: |
-          echo "Compatibility gate for the former three-shard check name."
-          echo "All native shards: $NATIVE_RESULT"
-          echo "Complete execution audit: $EXECUTION_RESULT"
-          if [[ "$NATIVE_RESULT" != "success" || "$EXECUTION_RESULT" != "success" ]]; then
-            echo "::error::Every native shard and the execution audit must succeed."
-            exit 1
-          fi
 
   # Tree-sitter ABI gate (#1922). Two halves, both blocking:
   #   1. Static, offline: assert every grammar's compiled ABI loads on the
@@ -315,7 +273,7 @@ jobs:
     runs-on: ${{ matrix.os }}
     timeout-minutes: 20
     steps:
-      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
         with:
           persist-credentials: false
       - uses: ./.github/actions/setup-gitnexus
@@ -326,12 +284,10 @@ jobs:
         shell: bash
         run: python3 .github/scripts/check-tree-sitter-upgrade-readiness.py --assert-current
 
-      # GITNEXUS_REQUIRE_ZIG=1: every OS in this matrix has a committed
+      # GITNEXUS_REQUIRE_ZIG=1: every OS in this matrix has a published
-      # vendored tree-sitter-zig prebuild (linux-arm64 is rebuilt by the
+      # tree-sitter-zig prebuild, so the smoke's "optional grammar may be
-      # prebuild workflow; ubuntu/windows/macos latest are x64/arm64 with
+      # absent" exemption is revoked here and an ABI-broken Zig binding fails
-      # shipped binaries), so the smoke's "optional grammar may be absent"
+      # the job instead of being accepted as a clean absence.
-      # exemption is revoked here and an ABI-broken Zig binding fails the
-      # job instead of being accepted as a clean absence.
       - name: Run parser-loader ABI load-smoke (dynamic)
         run: npx vitest run test/unit/parser-loader-abi.test.ts
         env:
@@ -359,7 +315,7 @@ jobs:
       # from a tarball and never pushes back; the token in .git/config would
       # be at risk of leaking through any future artifact-upload step
       # (zizmor artipacked audit). Disable upfront.
-      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
         with:
           persist-credentials: false
       # Skip prepare/postinstall/build here. `npm pack` runs prepack, which
@@ -472,7 +428,7 @@ jobs:
     steps:
       # persist-credentials: false — builds and import-links only, never pushes
       # (zizmor credential-persistence / artipacked audit).
-      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
         with:
           persist-credentials: false
       - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
@@ -524,8 +480,11 @@ jobs:
   # heap, so parallel forks both skew the timings and OOM the worker pool — they
   # must run one file at a time.
   #
-  # Every opt-in suite is discovered by run-benchmarks.ts. A failing worker
+  # go-pipeline-benchmark.test.ts is deliberately NOT included: its
-  # benchmark is a failure to investigate, never a reason to omit that suite.
+  # worker-pool (#1848) suite spins a real worker pool that exits unexpectedly
+  # under vitest's fork pool (reproduced in validation), which would make this
+  # gate flaky. Go is already guarded by its non-gated O(n^2) tripwire (runs in
+  # the main coverage job) plus its golden capture-parity test.
   benchmarks:
     name: benchmarks (GITNEXUS_BENCH)
     runs-on: ubuntu-latest
@@ -535,7 +494,7 @@ jobs:
       # and never pushes; the default-persisted token in .git/config would be at
       # risk of leaking through an artifact upload (zizmor credential-persistence
       # / artipacked audit). Mirrors the packaged-install-smoke job below.
-      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
         with:
           persist-credentials: false
       - uses: ./.github/actions/setup-gitnexus
@@ -562,15 +521,6 @@ jobs:
         run: node --import tsx bench/kotlin-star-route-constants/measure.mjs --check
         working-directory: gitnexus
 
-      - name: tRPC identifier-mount route extractor guards (#3339)
-        if: ${{ !cancelled() }}
-        # Build-free: inline-router control vs identifier-mounted subrouters
-        # and a transitive mount chain; fingerprints route paths and guards
-        # scaling + widening overhead (compose must stay linear in procedure
-        # count, and must not degrade the inline scan).
-        run: node --import tsx bench/trpc-route-extractor/measure.mjs --check
-        working-directory: gitnexus
-
       - name: Cross-language scope-capture fingerprint + scaling guards
         # Runs even after an earlier guard fails (#2895). Every step here was
         # fail-fast, so the FIRST failing --check aborted the job and every guard
@@ -635,59 +585,6 @@ jobs:
         run: node --import tsx bench/finalize-reexport/measure.mjs --check
         working-directory: gitnexus
 
-      - name: Parse dispatch-round cadence guards (#3194, #3196)
-        if: ${{ !cancelled() }}
-        # Build-free: asserts parse-cache pack membership is unchanged
-        # (fingerprint — every cache key derives from it), that a fixed corpus
-        # still batches into a fixed number of dispatch rounds, and that the
-        # round budget counts UTF-8 bytes rather than UTF-16 code units. Round
-        # boundaries are deliberately invisible to graph output, so no test can
-        # see these regress. Rationale and history: see the header of
-        # bench/parse-dispatch-rounds/measure.mjs.
-        run: node --import tsx bench/parse-dispatch-rounds/measure.mjs --check
-        working-directory: gitnexus
-
-      - name: Python workspace import-scan guards (#3254)
-        if: ${{ !cancelled() }}
-        # Build-free: same baseline approach as parse-dispatch-rounds —
-        # exact link/lookalike floors plus a fingerprint, then ratio timing
-        # only (scan scaling and from-token prefilter advantage). See
-        # bench/python-workspace-import-scan/measure.mjs.
-        run: node --import tsx bench/python-workspace-import-scan/measure.mjs --check
-        working-directory: gitnexus
-
-      - name: Rust Cargo target membership guards (#3253)
-        if: ${{ !cancelled() }}
-        # Build-free: same baseline approach as parse-dispatch-rounds —
-        # exact membership floors plus a fingerprint, then ratio timing
-        # only (loadRustCargoTargets 4n/n). Pins typical-Rust completeness
-        # (derive / println!), include!-abort, and src/target vs Cargo
-        # artifact layouts. See bench/rust-cargo-targets/measure.mjs.
-        run: node --import tsx bench/rust-cargo-targets/measure.mjs --check
-        working-directory: gitnexus
-
-      - name: Swift Package.swift import-resolve guards (#2964, #2931)
-        if: ${{ !cancelled() }}
-        # Build-free: same baseline approach as parse-dispatch-rounds —
-        # exact declared/SDK/undeclared floors plus a fingerprint, then
-        # ratio timing only (resolveSwiftImportTarget, swiftPackageStrategy,
-        # parseSwiftPackageManifest 4n/n). Pins declaration-only resolve,
-        # https:// factory survival, and #2931 segment-boundary membership.
-        # See bench/swift-package-imports/measure.mjs.
-        run: node --import tsx bench/swift-package-imports/measure.mjs --check
-        working-directory: gitnexus
-
-      - name: MCP tools/list countRepos vs listRepos guards (#3259, #3184)
-        if: ${{ !cancelled() }}
-        # Build-free: exact registry cardinality + tool-roster + schema-flag
-        # floors, then ratio timing only (countRepos/listRepos and
-        # listTools/listRepos). No millisecond ceiling — this repo has
-        # already been bitten by a fixed ms budget. Isolated GITNEXUS_HOME;
-        # fixture is N real git repos so listRepos pays rev-list. See
-        # bench/mcp-tools-list/measure.mjs.
-        run: node --import tsx bench/mcp-tools-list/measure.mjs --check
-        working-directory: gitnexus
-
       - name: C++ qualified-namespace resolution guards (#2788)
         if: ${{ !cancelled() }}
         # Build-free: asserts resolveCppQualifiedNamespaceMember resolves an
@@ -807,13 +704,6 @@ jobs:
         run: node --import tsx bench/kotlin-import-target/measure.mjs --check
         working-directory: gitnexus
 
-      - name: Ruby gem-boundary correctness + scaling guards (#3096)
-        if: ${{ !cancelled() }}
-        # Includes real manifest loading; checks scoped resolution and scaling
-        # as sibling projects or declared gem counts grow independently.
-        run: node --import tsx bench/ruby-gem-resolution/measure.mjs --check
-        working-directory: gitnexus
-
       - name: Receiver-resolution drop guards
         if: ${{ !cancelled() }}
         # NOT build-free: this one runs the real pipeline, so it needs dist/
@@ -853,35 +743,6 @@ jobs:
         run: node --import tsx bench/scope-emission/measure.mjs --check
         working-directory: gitnexus
 
-      - name: Zig cross-file static-gating guards (#3162)
-        if: ${{ !cancelled() }}
-        # Build-free: fingerprints cross-file dead-call classification and
-        # guards the workspace enrichment pass across file-count scaling.
-        run: node --import tsx bench/zig-cross-file-resolution/measure.mjs --check
-        working-directory: gitnexus
-
-      - name: Objective-C workspace resolution guards (#3179)
-        if: ${{ !cancelled() }}
-        # Build-free: fingerprints spread (typed self/super/sibling) and
-        # protocol-candidate evidence, and gates linear file-count scaling
-        # of emitPostResolutionEdges. Import lookup is the shared
-        # import-target `objc` arm; this is the C#/Zig analog for the
-        # workspace message-send pass.
-        run: node --import tsx bench/objective-c-resolution/measure.mjs --check
-        working-directory: gitnexus
-
-      - name: Callable-value reference resolution guards (#3399)
-        if: ${{ !cancelled() }}
-        # Build-free: pins the resolved-target SET of `resolveValueRefTarget`
-        # (exact site/resolved/declined counts plus an order-independent
-        # fingerprint) and asserts its per-site cost stays independent of
-        # workspace size across a 4x file-count step. The pass resolves a
-        # qualified receiver through `scopes.qualifiedNames`, a workspace-wide
-        # index: keyed it is O(1) per site, scanned it is O(files) — a
-        # regression a fixture cannot see and a 257-file binding table can.
-        run: node --import tsx bench/value-ref-resolution/measure.mjs --check
-        working-directory: gitnexus
-
       - name: CFG construction time / disk / memory guards (#2081 M1)
         if: ${{ !cancelled() }}
         # Build-free: asserts collectFunctionCfgs output is unchanged
@@ -915,18 +776,26 @@ jobs:
 
       - name: Cross-language pipeline benchmarks (GITNEXUS_BENCH, serial)
         if: ${{ !cancelled() }}
+        # cpp-adl-benchmark.test.ts and csharp-razor-view-components-benchmark.test.ts
+        # are not `*-pipeline-benchmark.test.ts` files but belong here for the
+        # same reason: they are skipIf-gated on GITNEXUS_BENCH, so the scaling
+        # guards they hold never run in the main coverage job.
         env:
-          GITNEXUS_WORKER_READY_TIMEOUT_MS: '60000'
+          GITNEXUS_BENCH: '1'
-        run: npm run test:benchmarks
+        run: >-
+          npx vitest run --no-file-parallelism
+          test/integration/cobol-pipeline-benchmark.test.ts
+          test/integration/csharp-pipeline-benchmark.test.ts
+          test/integration/csharp-razor-view-components-benchmark.test.ts
+          test/integration/cpp-adl-benchmark.test.ts
+          test/integration/data-route-table-benchmark.test.ts
+          test/integration/instance-ownership-pipeline-benchmark.test.ts
+          test/integration/spring-bean-resource-benchmark.test.ts
+          test/integration/spring-dynamic-lookup-benchmark.test.ts
+          test/integration/rust-pipeline-benchmark.test.ts
+          test/integration/php-pipeline-benchmark.test.ts
+          test/integration/ruby-pipeline-benchmark.test.ts
         working-directory: gitnexus
-      - name: Upload benchmark execution report
-        if: always()
-        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
-        with:
-          name: execution-benchmarks
-          path: gitnexus/benchmarks.json
-          if-no-files-found: error
-          retention-days: 5
 
   # Locked eval suite. setup-uv and uv itself are immutable so CI exercises
   # exactly the dependency graph developers run from eval/uv.lock.
@@ -936,97 +805,17 @@ jobs:
     timeout-minutes: 15
     steps:
       # persist-credentials: false — runs tests only, never pushes.
-      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
         with:
           persist-credentials: false
-      - uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
+      - uses: astral-sh/setup-uv@11f9893b081a58869d3b5fccaea48c9e9e46f990 # v8.3.2
         with:
           version: '0.11.23'
           python-version: '3.13'
           enable-cache: true
           cache-dependency-glob: eval/uv.lock
-      - run: uv run --locked --extra dev python -m pytest tests -q --junitxml=pytest-locked.xml
+      - run: uv run --locked --extra dev python -m pytest tests -q
         working-directory: eval
-      - name: Upload pytest execution report
-        if: always()
-        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
-        with:
-          name: execution-pytest-locked
-          path: eval/pytest-locked.xml
-          if-no-files-found: error
-          retention-days: 5
-      - uses: ./.github/actions/setup-gitnexus
-        with:
-          build: 'true'
-      - name: Exercise real workflow evidence preflight
-        if: ${{ !cancelled() }}
-        run: >-
-          npx vitest run test/unit/skill-evolution-workflow.test.ts
-          --reporter=default --reporter=./scripts/execution-reporter.ts --outputFile=preflight.json
-        working-directory: gitnexus
-      - name: Upload preflight execution report
-        if: always()
-        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
-        with:
-          name: execution-preflight
-          path: gitnexus/preflight.json
-          if-no-files-found: error
-          retention-days: 5
-
-  test-completeness:
-    name: every test executed
-    needs:
-      [
-        shard-plan,
-        coverage-merge,
-        cross-platform,
-        benchmarks,
-        eval-tests,
-        eval-containment-linux,
-        eval-containment-windows,
-      ]
-    if: ${{ !cancelled() }}
-    runs-on: ubuntu-latest
-    timeout-minutes: 10
-    steps:
-      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
-        with:
-          persist-credentials: false
-      - uses: ./.github/actions/setup-gitnexus
-      - name: Download coverage and web reports
-        uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
-        with:
-          name: coverage-reports
-      - name: Download all execution receipts
-        if: ${{ !cancelled() }}
-        uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
-        with:
-          pattern: execution-*
-          path: execution-reports
-          merge-multiple: true
-      - name: Require a real pass for every collected test
-        env:
-          PLATFORM_SHARDS: ${{ needs.shard-plan.outputs.total }}
-        run: |
-          cp gitnexus/test-results.json execution-reports/coverage.json
-          cd gitnexus
-          node --import tsx scripts/test-completeness.ts ../execution-reports test-results.json "$PLATFORM_SHARDS"
-      - name: Require a real pass for every pytest case
-        if: ${{ !cancelled() }}
-        run: python3 eval/check_test_execution.py execution-reports eval/pytest-results.json
-      - name: Upload complete test reports
-        if: always()
-        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
-        with:
-          name: test-reports
-          path: |
-            gitnexus/coverage/coverage-summary.json
-            gitnexus/coverage/coverage-final.json
-            gitnexus/test-results.json
-            gitnexus-web/web-test-results.json
-            eval/pytest-results.json
-          if-no-files-found: error
-          retention-days: 5
 
   # Native Linux ownership and Bubblewrap boundary. The environment flag makes
   # the real namespace test mandatory; a missing/blocked bwrap is a failure.
@@ -1036,14 +825,9 @@ jobs:
     timeout-minutes: 20
     env:
       GITNEXUS_REQUIRE_BWRAP_CANARY: '1'
-      # This job installs bubblewrap, the pinned runtime and a built GitNexus,
-      # so the offline sweep runs here with nothing provisioning-stubbed: real
-      # containment, real mounts, real graph. A missing piece fails the job
-      # rather than silently falling back to the stubbed path.
-      GITNEXUS_REQUIRE_FULL_SWEEP: '1'
       GITNEXUS_REQUIRE_CLAUDE_CANARY: '1'
     steps:
-      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
         with:
           persist-credentials: false
       - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
@@ -1053,7 +837,7 @@ jobs:
           cache-dependency-path: |
             gitnexus/package-lock.json
             gitnexus-shared/package-lock.json
-      - uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
+      - uses: astral-sh/setup-uv@11f9893b081a58869d3b5fccaea48c9e9e46f990 # v8.3.2
         with:
           version: '0.11.23'
           python-version: '3.13'
@@ -1063,7 +847,7 @@ jobs:
         run: |
           set -euo pipefail
           sudo apt-get update
-          sudo apt-get install --yes --no-install-recommends bubblewrap ripgrep socat
+          sudo apt-get install --yes --no-install-recommends bubblewrap socat
           apparmor_userns=/proc/sys/kernel/apparmor_restrict_unprivileged_userns
           if [[ -r "${apparmor_userns}" ]] && [[ "$(<"${apparmor_userns}")" == '1' ]]; then
             sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0
@@ -1099,21 +883,8 @@ jobs:
           tests/test_process_control.py
           tests/test_proposer_sandbox.py
           tests/test_workflow_bench_sessions.py
-          tests/test_ce_plugin_runtime.py
+          tests/test_ce_plugin_runtime.py -q
-          tests/test_offline_sweep_integration.py
-          tests/test_mock_provider.py
-          tests/test_evolve.py::test_outer_runner_pid_namespace_kills_setsid_descendant
-          tests/test_oracle_assets.py::test_hidden_vitest_config_executes_sibling_oracle_against_candidate_checkout
-          -q --junitxml=pytest-ubuntu.xml
         working-directory: eval
-      - name: Upload pytest execution report
-        if: always()
-        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
-        with:
-          name: execution-pytest-ubuntu
-          path: eval/pytest-ubuntu.xml
-          if-no-files-found: error
-          retention-days: 5
 
   # Native Windows Job Object canary. POSIX-only tests skip by platform, while
   # the grandchild delayed-write test must execute and pass on this runner.
@@ -1122,10 +893,10 @@ jobs:
     runs-on: windows-latest
     timeout-minutes: 15
     steps:
-      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
         with:
           persist-credentials: false
-      - uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
+      - uses: astral-sh/setup-uv@11f9893b081a58869d3b5fccaea48c9e9e46f990 # v8.3.2
         with:
           version: '0.11.23'
           python-version: '3.13'
@@ -1134,14 +905,5 @@ jobs:
       - name: Prove Windows process-tree ownership
         run: >-
           uv run --locked --extra dev python -m pytest
-          tests/test_process_control.py tests/test_model_gateway.py
+          tests/test_process_control.py -q
-          -k "not locked_litellm" -q --junitxml=pytest-windows.xml
         working-directory: eval
-      - name: Upload pytest execution report
-        if: always()
-        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
-        with:
-          name: execution-pytest-windows
-          path: eval/pytest-windows.xml
-          if-no-files-found: error
-          retention-days: 5

.github/workflows/claude.yml

@@ -129,7 +129,7 @@ jobs:
             core.setOutput('code_review', isCodeReview ? 'true' : 'false');
 
       - name: Checkout repository
-        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
         with:
           repository: ${{ steps.pr.outputs.is_pr == 'true' && steps.pr.outputs.repo || github.repository }}
           ref: ${{ steps.pr.outputs.is_pr == 'true' && steps.pr.outputs.sha || '' }}

.github/workflows/codeql.yml

@@ -42,13 +42,13 @@ jobs:
 
     steps:
       - name: Checkout
-        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
         with:
           # Don't leave GITHUB_TOKEN in .git/config for downstream steps to read.
           persist-credentials: false
 
       - name: Initialize CodeQL
-        uses: github/codeql-action/init@2892aa5e19bbd11bc0cff5427e3b750a04d9e3c2 # v4.38.2
+        uses: github/codeql-action/init@cdf488f595d80d6e07e03d4674febd5ab45fa938 # v4.37.9
         with:
           languages: ${{ matrix.language }}
           queries: security-and-quality
@@ -77,16 +77,8 @@ jobs:
               # GitHub PR CodeQL gate, so this file is excluded to avoid
               # re-filing js/regex-injection on every push of the same line.
               - 'gitnexus/src/server/grep-params.ts'
-              # Tests construct tmpdir fixtures and pass them into production
-              # read-only probes (openSync(..., 'r')). CodeQL models that as
-              # js/insecure-temporary-file even though nothing is created.
-              - '**/test/**'
-              # CI vendor fetch: origin is the official Ladybug repo; dest is
-              # regex-pinned and containment-checked. Inline suppressions do
-              # not clear the PR CodeQL gate (same as grep-params.ts).
-              - '.github/scripts/fetch-lbug-fts-artifacts.mjs'
 
       - name: Perform CodeQL Analysis
-        uses: github/codeql-action/analyze@2892aa5e19bbd11bc0cff5427e3b750a04d9e3c2 # v4.38.2
+        uses: github/codeql-action/analyze@cdf488f595d80d6e07e03d4674febd5ab45fa938 # v4.37.9
         with:
           category: '/language:${{ matrix.language }}'

.github/workflows/commit-fork-prebuilds.yml

@@ -135,53 +135,68 @@ jobs:
       # workflow_run event. The allowlist above only proves the fields are
       # well-formed — not that they refer to the PR/SHA that actually triggered
       # us. A fork-controlled build could mutate metadata.json to reference
-      # another PR/SHA and redirect our write-scoped push.
+      # another PR/SHA and redirect our write-scoped push. Authority sources are
-      #
+      # all server-controlled: workflow_run.head_sha, head_repository.full_name,
-      # This job's `if:` already restricts to forks, so pull_requests[] is empty
+      # and pull_requests[].number (empty on forks -> commits/{sha}/pulls).
-      # by design and GET /repos/{base}/commits/{sha}/pulls is also empty (the
-      # fork commit is not in the base graph). Authority is workflow_run.head_sha
-      # + head_repository.full_name + head_branch, resolved via
-      # pulls?head={owner}:{branch}. The script comes from THIS default-branch
-      # checkout (the same trust anchor as this workflow file).
-      - name: Checkout identity verifier
-        if: steps.meta.outputs.deliver == 'true'
-        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
-        with:
-          persist-credentials: false
-          sparse-checkout: .github/scripts/verify-workflow-run-pr-identity.cjs
-          sparse-checkout-cone-mode: false
-          path: trusted
-
       - name: Verify metadata against workflow_run authority
-        id: verify
         if: steps.meta.outputs.deliver == 'true'
         env:
           GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
           GH_REPO: ${{ github.repository }}
-          META_PATH: meta-in/metadata.json
+          META_PR_NUMBER: ${{ steps.meta.outputs.pr_number }}
-          SCHEMA_PATTERN: '^gitnexus\.ts-prebuild/v[0-9]+$'
+          META_HEAD_SHA: ${{ steps.meta.outputs.head_sha }}
+          META_HEAD_REPO: ${{ steps.meta.outputs.head_repo }}
           WF_HEAD_SHA: ${{ github.event.workflow_run.head_sha }}
           WF_HEAD_REPO: ${{ github.event.workflow_run.head_repository.full_name }}
-          WF_HEAD_BRANCH: ${{ github.event.workflow_run.head_branch }}
+          WF_PR_NUMBERS: ${{ toJSON(github.event.workflow_run.pull_requests.*.number) }}
         shell: bash
-        run: node trusted/.github/scripts/verify-workflow-run-pr-identity.cjs
+        run: |
+          set -euo pipefail
+
+          # 1) head_sha must match exactly — the commit GitHub ran the producer against.
+          if [ "${META_HEAD_SHA}" != "${WF_HEAD_SHA}" ]; then
+            echo "::error::Artifact head_sha (${META_HEAD_SHA}) != workflow_run.head_sha (${WF_HEAD_SHA}) — refusing."
+            exit 1
+          fi
+          # 2) head_repo must match exactly.
+          if [ "${META_HEAD_REPO}" != "${WF_HEAD_REPO}" ]; then
+            echo "::error::Artifact head_repo (${META_HEAD_REPO}) != workflow_run.head_repository (${WF_HEAD_REPO}) — refusing."
+            exit 1
+          fi
+          # 3) pr_number must reference an open PR with this head SHA. Forks have
+          # an empty pull_requests[] by design — fall back to commits/{sha}/pulls.
+          allowed_numbers=$(jq -c '.' <<< "${WF_PR_NUMBERS}")
+          if [ "${allowed_numbers}" = "[]" ]; then
+            echo "workflow_run.pull_requests empty (fork) — using commits/{sha}/pulls."
+            allowed_numbers=$(gh api "repos/${GH_REPO}/commits/${WF_HEAD_SHA}/pulls" \
+              --jq '[.[] | select(.state == "open") | .number]' 2>/dev/null || echo "[]")
+            if [ "${allowed_numbers}" = "[]" ]; then
+              echo "::error::No open PR for head ${WF_HEAD_SHA} — refusing."
+              exit 1
+            fi
+          fi
+          if ! jq -e --argjson n "${META_PR_NUMBER}" 'index($n) != null' <<< "${allowed_numbers}" >/dev/null; then
+            echo "::error::Artifact pr_number (${META_PR_NUMBER}) not in authoritative list (${allowed_numbers}) — refusing."
+            exit 1
+          fi
+          echo "Verified identity: PR=${META_PR_NUMBER} head_sha=${META_HEAD_SHA} head_repo=${META_HEAD_REPO}."
 
       # Pinned to v6.0.3 (same SHA used by build-tree-sitter-prebuilds.yml).
       # persist-credentials: false — push auth is provided inline at push time,
       # never written to .git/config on disk.
       - name: Checkout fork PR head
-        if: steps.meta.outputs.deliver == 'true' && steps.verify.outcome == 'success'
+        if: steps.meta.outputs.deliver == 'true'
-        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
         with:
-          repository: ${{ steps.verify.outputs.head_repo }}
+          repository: ${{ steps.meta.outputs.head_repo }}
-          ref: ${{ steps.verify.outputs.head_sha }}
+          ref: ${{ steps.meta.outputs.head_sha }}
           token: ${{ secrets.GITHUB_TOKEN }}
           persist-credentials: false
           fetch-depth: 0
           path: pr-checkout
 
       - name: Place prebuilds into the fork checkout
-        if: steps.meta.outputs.deliver == 'true' && steps.verify.outcome == 'success'
+        if: steps.meta.outputs.deliver == 'true'
         env:
           DL: prebuilds-in
           CHECKOUT: pr-checkout
@@ -223,12 +238,12 @@ jobs:
 
       - name: Commit and push to the fork branch
         id: push
-        if: steps.meta.outputs.deliver == 'true' && steps.verify.outcome == 'success'
+        if: steps.meta.outputs.deliver == 'true'
         working-directory: pr-checkout
         env:
-          HEAD_REF: ${{ steps.verify.outputs.head_ref }}
+          HEAD_REF: ${{ steps.meta.outputs.head_ref }}
-          HEAD_REPO: ${{ steps.verify.outputs.head_repo }}
+          HEAD_REPO: ${{ steps.meta.outputs.head_repo }}
-          HEAD_SHA: ${{ steps.verify.outputs.head_sha }}
+          HEAD_SHA: ${{ steps.meta.outputs.head_sha }}
           # Push auth only — supplied via env, never interpolated into the command.
           GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
         shell: bash
@@ -288,11 +303,11 @@ jobs:
           fi
 
       - name: Comment delivery outcome
-        if: always() && steps.meta.outputs.deliver == 'true' && steps.verify.outcome == 'success' && steps.push.outcome != 'skipped'
+        if: always() && steps.meta.outputs.deliver == 'true' && steps.push.outcome != 'skipped'
         env:
           GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
           GH_REPO: ${{ github.repository }}
-          PR: ${{ steps.verify.outputs.pr_number }}
+          PR: ${{ steps.meta.outputs.pr_number }}
           RESULT: ${{ steps.push.outputs.result }}
           RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
         shell: bash

.github/workflows/dependency-review.yml

@@ -28,7 +28,7 @@ jobs:
 
     steps:
       - name: Checkout
-        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
         with:
           persist-credentials: false
 

.github/workflows/docker.yml

@@ -74,10 +74,8 @@ jobs:
           - name: gitnexus-web
             dockerfile: Dockerfile.web
             slug: gitnexus-web
-          # CLI / `gitnexus serve` backend. Tree-sitter natives live in this
+          # CLI / `gitnexus serve` backend. Heavy native deps (tree-sitter,
-          # image. onnxruntime-node is opt-in (`gitnexus embeddings install` or
+          # onnxruntime-node) live only in this image.
-          # a bind-mounted prefix / GITNEXUS_EMBEDDING_URL); npm is stripped
-          # at runtime so the image cannot auto-heal the embedding stack.
           - name: gitnexus
             dockerfile: Dockerfile.cli
             slug: gitnexus
@@ -103,7 +101,7 @@ jobs:
       # When triggered by workflow_call the caller passes the RC tag as an input;
       # we check out that tag so the Dockerfile and package.json match the built image.
       # For tag-push events github.ref is already the tag ref — no override needed.
-      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
         with:
           ref: ${{ inputs.tag || github.ref }}
 
@@ -140,10 +138,10 @@ jobs:
 
       # Required for multi-platform (linux/arm64) emulation.
       - name: Set up QEMU
-        uses: docker/setup-qemu-action@99012661954931238ded8c8b007157a8430204e1 # v4.4.0
+        uses: docker/setup-qemu-action@96fe6ef7f33517b61c61be40b68a1882f3264fb8 # v4.2.0
 
       - name: Set up Docker Buildx
-        uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4.4.1
+        uses: docker/setup-buildx-action@37fe631027851001ddb9b187196cc803df7f5f0e # v4.3.0
 
       - name: Install Cosign
         uses: sigstore/cosign-installer@6f9f17788090df1f26f669e9d70d6ae9567deba6 # v4.1.2

.github/workflows/gitleaks.yml

@@ -29,7 +29,7 @@ jobs:
 
     steps:
       - name: Checkout
-        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
         with:
           # Full history needed for the on-push full-history scan; on PRs the
           # action diffs against the base ref so the cost is bounded by the PR.

.github/workflows/gitnexus-review-agent.yml

@@ -303,7 +303,7 @@ jobs:
       - name: Checkout trusted workflow control plane
         id: checkout-control
         if: steps.context.outputs.ready == 'true'
-        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
         with:
           repository: ${{ github.repository }}
           ref: ${{ steps.context.outputs.control_sha }}
@@ -315,7 +315,7 @@ jobs:
       - name: Checkout exact PR head as passive data
         id: checkout-head
         if: steps.context.outputs.ready == 'true'
-        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
         with:
           repository: ${{ steps.context.outputs.head_repo }}
           ref: ${{ steps.context.outputs.head_sha }}
@@ -1294,7 +1294,7 @@ jobs:
           steps.claude-recheck.outcome == 'success'
         # Use the low-level base action: the high-level GitHub action can restore
         # project configuration from a moving base branch before invoking Claude.
-        uses: anthropics/claude-code-action/base-action@12dd8d74c712f5f3669365b2369b558c495b1104 # v1
+        uses: anthropics/claude-code-action/base-action@3553f84341b92da26052e28acf1aa898f9511f32 # v1
         env:
           CLAUDE_CODE_SUBPROCESS_ENV_SCRUB: '1'
           CLAUDE_CODE_ADDITIONAL_DIRECTORIES_CLAUDE_MD: '0'
@@ -1447,7 +1447,7 @@ jobs:
         if: >-
           steps.precheck.outputs.repair_reason != '' &&
           steps.repair-recheck.outcome == 'success'
-        uses: anthropics/claude-code-action/base-action@12dd8d74c712f5f3669365b2369b558c495b1104 # v1
+        uses: anthropics/claude-code-action/base-action@3553f84341b92da26052e28acf1aa898f9511f32 # v1
         env:
           CLAUDE_CODE_SUBPROCESS_ENV_SCRUB: '1'
           CLAUDE_CODE_ADDITIONAL_DIRECTORIES_CLAUDE_MD: '0'

.github/workflows/gitnexus-skill-evolution.yml

@@ -4,22 +4,17 @@
 # overlay. The gate is evidence FOR a PR, never a bypass of one — nothing
 # merges without review.
 #
-# Activation and operations checklist.
+# Activation checklist (the scheduled lane is OFF by default).
-# [x] Configure at least one model secret on the `gitnexus-evolution`
+# [ ] Configure the repository secret GITNEXUS_BENCH_AUTH_TOKEN (an Anthropic
-#     Environment: GITNEXUS_BENCH_ANTHROPIC_API_KEY (Anthropic API key — not
+#     API key — benchmark sessions bill real usage; the Claude Code OAuth
-#     the Claude Code OAuth token; legacy GITNEXUS_BENCH_AUTH_TOKEN is still
+#     subscription token does not work here).
-#     accepted) and/or GITNEXUS_BENCH_OPENAI_API_KEY. Sessions bill real usage.
+# [ ] Configure the RELEASE_APP_ID and RELEASE_APP_PRIVATE_KEY secrets (the
-#     OpenAI keys are not native to Claude Code; the loop starts a loopback
-#     LiteLLM proxy and keeps the OpenAI key off the sandboxed agent. With only
-#     the OpenAI secret, or with provider=openai, dispatch-time Claude model
-#     defaults are gpt-5.6-sol with xhigh reasoning effort.
-# [x] Configure the RELEASE_APP_ID and RELEASE_APP_PRIVATE_KEY secrets (the
 #     App that opens the promotion PR). The Mint-App-Token step hard-fails
 #     without them once a promotion is detected. Verify the App installation
 #     is scoped to this repo with only Contents: RW + Pull requests: RW.
 # [x] Create the protected Environment `gitnexus-evolution` with a
 #     deployment-branch rule restricting it to `main`, and ideally scope the
-#     four secrets above to that Environment. workflow_dispatch runs this
+#     three secrets above to that Environment. workflow_dispatch runs this
 #     workflow (and eval/workflow_bench/evolve.py) from the *dispatched ref*,
 #     so this server-side rule — not a code-side guard the branch could edit
 #     away — is what stops a non-main branch from running with the secrets.
@@ -45,35 +40,13 @@
 #     most weekly. Revisit if run frequency increases or the threat model
 #     changes; stopping already bounds the exposure window to the job's own
 #     runtime on 1 day out of 7.
-# [ ] Install and verify the runner survival policy below before enabling
+# [ ] Run workflow_dispatch once and confirm: containment preflight passes,
-#     scheduled runs. A run
-#     spans ~15h and apt-daily-upgrade.timer fires daily (~06:34), so every
-#     scheduled run crosses it. On 2026-08-02 unattended-upgrades upgraded
-#     openssl at 07:54:02 and needrestart restarted the Actions runner five
-#     seconds later: the job went to Canceled, and a cancelled job skips even
-#     `if: always()`, so the evidence artifact died with it. Keep installing
-#     updates, but never let them restart services here:
-#         /etc/needrestart/conf.d/90-gitnexus-evolution.conf
-#         $nrconf{restart} = 'l';
-#     A drop-in, so a needrestart package upgrade cannot clobber it. Nothing
-#     is left unpatched in practice — the box is stopped between runs, so the
-#     new binaries take effect at the next boot.
-# [x] Run workflow_dispatch once and confirm: containment preflight passes,
 #     the benchmark completes inside the job timeout, the results artifact
-#     uploads, and a promotion (if any) opens a well-formed PR. Run
+#     uploads, and a promotion (if any) opens a well-formed PR.
-#     29907431284 (2026-07-22) went green end to end in 14h45m and reached a
+# [ ] Set the repository variable GITNEXUS_EVOLUTION_ENABLED=true.
-#     gate decision (`insufficient_evidence`, no promotion).
+# Roll back by setting that variable to false. Note: workflow_dispatch always
-# [ ] Confirm a workers=3 dispatch has zero excluded runs (review sessions in
+# runs the full benchmark loop regardless of GITNEXUS_EVOLUTION_ENABLED and
-#     33962002890 averaged ~19m serial, well under the 90m session ceiling).
+# bills real API usage on GITNEXUS_BENCH_AUTH_TOKEN.
-#     Then set GITNEXUS_EVOLUTION_WORKERS=3 and
-#     GITNEXUS_EVOLUTION_ENABLED=true for scheduled runs. Scheduled runs
-#     require both values, so leaving the var unset is an immediate rollback.
-#     Dispatch defaults to 3; pass workers=1 only to debug a contended host.
-#     Weekly generations reuse matching incumbent/CE cells from the previous
-#     artifact so the paid matrix is the new candidate, not a 54-cell replay.
-#     Wall clock is quantised by ceil(cells_per_task / workers), and a review
-#     task is 9 cells cold, so 4 costs host contention for exactly the wall
-#     clock of 3. The next step up that buys anything is 5 (3 waves -> 2).
 name: GitNexus skill evolution
 
 on:
@@ -95,51 +68,21 @@ on:
         required: false
         default: '3'
         type: string
-      workers:
-        description: 'Benchmark cells of one task to run at once — 3 fits the evolution box; drop to 1 only if siblings hit the session ceiling'
-        required: false
-        default: '3'
-        type: string
       model:
         description: 'Model for the benchmark arms (match the model your skill users run)'
         required: false
-        default: 'gpt-5.6-sol'
+        default: 'claude-sonnet-5'
         type: string
       proposer_model:
         description: 'Model for the proposer/diagnosis session — a stronger model is fine (one session per generation)'
         required: false
-        default: 'gpt-5.6-sol'
+        default: 'claude-opus-4-8'
         type: string
-      effort:
-        description: 'Reasoning effort for every proposer and benchmark session'
-        required: false
-        default: xhigh
-        type: choice
-        options:
-          - low
-          - medium
-          - high
-          - xhigh
-          - max
-      provider:
-        description: 'Model backend. auto uses Anthropic when that secret exists; openai forces the loopback OpenAI gateway even if an Anthropic key is also configured.'
-        required: false
-        default: openai
-        type: choice
-        options:
-          - auto
-          - openai
-          - anthropic
       include_expensive:
         description: 'Include tasks marked expensive: true'
         required: false
         default: false
         type: boolean
-      seed_from_previous:
-        description: "Seed the proposer with the previous run's evidence and rejected proposal. Turn off to start from a blank slate — required when the earlier evidence is not trustworthy (e.g. produced before a harness-integrity fix), since a tainted proposal would otherwise propagate into every later generation."
-        required: false
-        default: true
-        type: boolean
 
 concurrency:
   group: ${{ github.workflow }}
@@ -154,105 +97,35 @@ jobs:
       github.repository == 'abhigyanpatwari/GitNexus' &&
       (
         github.event_name == 'workflow_dispatch' ||
-        (
+        vars.GITNEXUS_EVOLUTION_ENABLED == 'true'
-          vars.GITNEXUS_EVOLUTION_ENABLED == 'true' &&
-          vars.GITNEXUS_EVOLUTION_WORKERS == '3'
-        )
       )
     runs-on: [self-hosted, linux, x64, gitnexus-evolution]
     # Gate promotion runs on a protected Environment. An admin must attach a
-    # deployment-branch rule (main only) and ideally scope the model and App
+    # deployment-branch rule (main only) and ideally scope the three secrets to
-    # secrets to it — server-side enforcement a dispatched non-main ref cannot bypass by
+    # it — server-side enforcement a dispatched non-main ref cannot bypass by
     # editing its own workflow copy. See the activation checklist above.
     environment: gitnexus-evolution
-    # Three budgets have to nest, longest first, or the evidence is lost:
+    timeout-minutes: 1440 # self-hosted ceiling is 5 days (7200min); 24h is a generous margin over a single-generation serial run
-    #   EventBridge instance uptime (24h from ~02:45)
-    #     > this job timeout (21h)
-    #       > the benchmark step timeout (19h, set on the step below)
-    # A job-level timeout CANCELS the job, so the upload step never runs and a
-    # multi-hour generation's evidence dies with it; a step-level timeout only
-    # fails that step, and `if: always()` still uploads what the sweep wrote.
-    # The instance must outlive the job for the same reason — when the box
-    # stops the runner just disappears mid-step. Scheduled runs can start well
-    # after the cron (the 2026-08-01 run was queued 65min late), so the job
-    # budget has to absorb that delay and still land inside the uptime window.
-    # A Friday workflow_dispatch on a box that already booted for Saturday's
-    # cron inherits leftover uptime, not a fresh 24h. Run 33962002890 started
-    # Friday 10:57 UTC and vanished at the Saturday 03:00 stop — 51 finished
-    # sessions never uploaded. run-evolution.sh therefore passes
-    # --max-runtime-from-instance-window, and the CLI derives its cap from
-    # /proc/uptime at startup, so the sweep fails in-process and this always()
-    # upload still runs.
-    timeout-minutes: 1260
     permissions:
       contents: read # The promotion PR uses a short-lived App token minted below.
-      actions: read # Read the previous run's evidence artifact to seed the proposer.
     env:
       GENERATIONS: ${{ inputs.generations || '1' }}
       RUNS: ${{ inputs.runs || '3' }}
-      # A manual input wins; scheduled runs use the repository rollout knob.
+      MODEL: ${{ inputs.model || 'claude-sonnet-5' }}
-      # Both fall back to serial — see workflow_bench.runner --workers for why.
+      PROPOSER_MODEL: ${{ inputs.proposer_model || 'claude-opus-4-8' }}
-      WORKERS: ${{ inputs.workers || vars.GITNEXUS_EVOLUTION_WORKERS || '1' }}
-      MODEL: ${{ inputs.model || 'gpt-5.6-sol' }}
-      PROPOSER_MODEL: ${{ inputs.proposer_model || 'gpt-5.6-sol' }}
-      EFFORT: ${{ inputs.effort || 'xhigh' }}
-      PROVIDER: ${{ inputs.provider || 'openai' }}
       INCLUDE_EXPENSIVE: ${{ inputs.include_expensive && '1' || '' }}
     steps:
       - name: Require the benchmark auth secret
         env:
-          HAS_ANTHROPIC: ${{ secrets.GITNEXUS_BENCH_ANTHROPIC_API_KEY != '' || secrets.GITNEXUS_BENCH_AUTH_TOKEN != '' }}
+          HAS_TOKEN: ${{ secrets.GITNEXUS_BENCH_AUTH_TOKEN != '' }}
-          HAS_OPENAI: ${{ secrets.GITNEXUS_BENCH_OPENAI_API_KEY != '' }}
         run: |
           set -euo pipefail
-          if [[ "${HAS_ANTHROPIC}" != 'true' && "${HAS_OPENAI}" != 'true' ]]; then
+          if [[ "${HAS_TOKEN}" != 'true' ]]; then
-            echo '::error::Configure GITNEXUS_BENCH_ANTHROPIC_API_KEY (Anthropic API key, not the Claude Code OAuth token) and/or GITNEXUS_BENCH_OPENAI_API_KEY. The evolution loop runs real benchmark sessions.'
+            echo '::error::GITNEXUS_BENCH_AUTH_TOKEN is not configured. The evolution loop runs real benchmark sessions and needs an Anthropic API key (not the Claude Code OAuth token).'
-            exit 1
-          fi
-          case "${PROVIDER}" in
-            openai)
-              if [[ "${HAS_OPENAI}" != 'true' ]]; then
-                echo '::error::provider=openai requires GITNEXUS_BENCH_OPENAI_API_KEY on the gitnexus-evolution environment.'
-                exit 1
-              fi
-              ;;
-            anthropic)
-              if [[ "${HAS_ANTHROPIC}" != 'true' ]]; then
-                echo '::error::provider=anthropic requires GITNEXUS_BENCH_ANTHROPIC_API_KEY on the gitnexus-evolution environment.'
-                exit 1
-              fi
-              ;;
-            auto)
-              ;;
-            *)
-              echo "::error::Unknown provider '${PROVIDER}' (expected auto, openai, or anthropic)."
-              exit 1
-              ;;
-          esac
-
-      - name: Verify runner survival policy
-        run: |
-          set -euo pipefail
-          needrestart_policy=/etc/needrestart/conf.d/90-gitnexus-evolution.conf
-          needrestart_line="\$nrconf{restart} = 'l';"
-          if [[ ! -r "${needrestart_policy}" ]] || ! grep -Fqx "${needrestart_line}" "${needrestart_policy}"; then
-            echo "::error::${needrestart_policy} must contain: ${needrestart_line}"
-            exit 1
-          fi
-          # The runner sets job processes to 500; the host oom-guard rewrites
-          # them to -900. Read once and the check loses that race.
-          oom_score_adjustment="$(</proc/self/oom_score_adj)"
-          deadline=$((SECONDS + 5))
-          while (( oom_score_adjustment > -900 && SECONDS < deadline )); do
-            sleep 0.05
-            oom_score_adjustment="$(</proc/self/oom_score_adj)"
-          done
-          if (( oom_score_adjustment > -900 )); then
-            echo "::error::Runner.Worker descendants require OOMScoreAdjust=-900 or stronger; effective value is ${oom_score_adjustment}."
             exit 1
           fi
 
-      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
         with:
           persist-credentials: false
           fetch-depth: 0
@@ -265,33 +138,18 @@ jobs:
             gitnexus/package-lock.json
             gitnexus-shared/package-lock.json
 
-      - uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
+      - uses: astral-sh/setup-uv@11f9893b081a58869d3b5fccaea48c9e9e46f990 # v8.3.2
         with:
           version: '0.11.23'
           python-version: '3.13'
           enable-cache: true
           cache-dependency-glob: eval/uv.lock
 
-      - name: Fetch pinned Compound Engineering review comparator
-        env:
-          CE_COMMIT: 3ad9b51bceecf0158e590c882034d0398dbb9c5c
-        run: |
-          set -euo pipefail
-          destination="${RUNNER_TEMP}/compound-engineering-plugin"
-          rm -rf "${destination}"
-          git clone --filter=blob:none --no-checkout \
-            https://github.com/EveryInc/compound-engineering-plugin.git "${destination}"
-          git -C "${destination}" checkout --detach "${CE_COMMIT}"
-          test "$(git -C "${destination}" rev-parse HEAD)" = "${CE_COMMIT}"
-
       - name: Install sandbox runtime and pinned Claude CLI
         run: |
           set -euo pipefail
-          # This box is stopped six days a week, so persistent apt timers can
+          sudo apt-get update
-          # begin their catch-up run shortly after boot. Wait for dpkg instead
+          sudo apt-get install --yes --no-install-recommends bubblewrap socat
-          # of racing the same package lock and failing the weekly lane.
-          sudo apt-get -o DPkg::Lock::Timeout=600 update
-          sudo apt-get -o DPkg::Lock::Timeout=600 install --yes --no-install-recommends bubblewrap ripgrep socat
           apparmor_userns=/proc/sys/kernel/apparmor_restrict_unprivileged_userns
           if [[ -r "${apparmor_userns}" ]] && [[ "$(<"${apparmor_userns}")" == '1' ]]; then
             sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0
@@ -315,203 +173,78 @@ jobs:
           test "$("${canary_runtime}/node_modules/@anthropic-ai/claude-code-linux-x64/claude" --version)" = \
             '2.1.214 (Claude Code)'
 
-      - name: Verify contained review execution before paid sessions
-        working-directory: eval
-        env:
-          GITNEXUS_REQUIRE_BWRAP_CANARY: '1'
-          GITNEXUS_REQUIRE_CLAUDE_CANARY: '1'
-          CLAUDE_CANARY_BIN: ${{ runner.temp }}/claude-canary/node_modules/@anthropic-ai/claude-code-linux-x64/claude
-        run: |
-          set -euo pipefail
-          uv run --locked --extra dev python -m pytest tests/test_proposer_sandbox.py -q
-
       - name: Install monorepo root dependencies
         run: |
           set -euo pipefail
           # The benchmark's task bindings sandbox-copy node_modules from the
           # monorepo root as well as gitnexus-shared and gitnexus (see the
-          # sandbox_dependencies entries in tasks.scenarios.yaml). gitnexus
+          # sandbox_copy entries in tasks.scenarios.yaml). The two steps below
-          # npm ci plus build compiles shared through parent lib/tsc.js; do
+          # install the subpackage trees; the root tree needs its own install
-          # not npm ci gitnexus-shared (second TypeScript 7 optional install).
+          # or capture_task_dependency_binding aborts at task binding on the
+          # missing root node_modules.
           npm ci
 
+      - name: Build pinned shared runtime
+        run: |
+          set -euo pipefail
+          npm ci
+          npm run build
+        working-directory: gitnexus-shared
+
       - name: Install and build pinned GitNexus runtime
         run: |
           set -euo pipefail
           npm ci
           npm run build
-          # Task YAML still mounts gitnexus-shared/node_modules. Keep an empty
-          # directory so capture_task_dependency_binding does not abort, without
-          # installing TypeScript 7 inside shared.
-          mkdir -p ../gitnexus-shared/node_modules
         working-directory: gitnexus
 
       - name: Point the benchmark task repo at the checkout
         run: |
           set -euo pipefail
-          # tasks.review.scenarios.yaml addresses the target repo as ~/GitNexus (the
+          # tasks.scenarios.yaml addresses the target repo as ~/GitNexus (the
           # developer-local convention). On the runner the repo is the checkout
           # at ${GITHUB_WORKSPACE}; link it so runner_tasks.py can resolve the
           # task `repo` path. The benchmark only clones the repo (copy-on-write)
           # and mounts dependencies read-only, so the checkout is never mutated.
-          if [[ -e "${HOME}/GitNexus" && ! -L "${HOME}/GitNexus" ]]; then
-            echo '::error::~/GitNexus exists and is not a symlink; refusing to place the checkout inside it.'
-            exit 1
-          fi
           ln -sfn "${GITHUB_WORKSPACE}" "${HOME}/GitNexus"
-          # The review corpus pins historical object ids. Fetch main so those
-          # objects are present even when actions/checkout selected another ref.
-          git -C "${GITHUB_WORKSPACE}" fetch --no-tags --quiet \
-            "${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git" \
-            '+refs/heads/main:refs/remotes/origin/main'
-          baseline_sha="$(git -C "${GITHUB_WORKSPACE}" rev-parse --verify 'refs/remotes/origin/main^{commit}')"
-          echo "Fetched review corpus history at ${baseline_sha}"
-
-      - name: Seed the proposer with the previous run's evidence
-        id: seed
-        # Scheduled runs always seed; a dispatch can opt out to start clean.
-        if: github.event_name != 'workflow_dispatch' || inputs.seed_from_previous
-        # Best-effort seeding must not consume the benchmark's budget. This
-        # step walks up to 10 prior runs and every iteration blocks on network
-        # it does not control (`gh run download` of a multi-hundred-megabyte
-        # artifact). Unbounded, a wedged download sits here until the 21h job
-        # timeout CANCELS the job — and a cancelled job skips even
-        # `if: always()`, so the sweep never starts and nothing is uploaded.
-        # Bounding the step instead fails it in minutes, which is a loud,
-        # cheap, re-runnable failure rather than a silent 21h loss. 15 minutes
-        # is an order of magnitude above the observed walk (well under a
-        # minute) and a rounding error against the 19h sweep it protects.
-        timeout-minutes: 15
-        env:
-          GH_TOKEN: ${{ github.token }}
-        run: |
-          set -euo pipefail
-          # Without this the weekly lane is memoryless: `--seed-results` is the
-          # only way a run sees what already lost (evolve stages the prior
-          # proposal when present and summarizes promotion.json when present),
-          # and with the default --generations 1 there is no earlier generation
-          # in-process to supply it. Every Saturday would otherwise propose
-          # from a blank slate and could re-propose the same rejected candidate
-          # forever. Best-effort by design: a first run, an expired artifact,
-          # or a download failure must not cost a whole generation.
-          if ! command -v gh >/dev/null; then
-            echo '::warning::gh is not installed on this runner — proposing without prior evidence. The promotion-PR step needs gh too.'
-            exit 0
-          fi
-          if ! previous_runs="$(gh run list \
-            --repo "${GITHUB_REPOSITORY}" \
-            --workflow gitnexus-skill-evolution.yml \
-            --branch main \
-            --status completed \
-            --limit 10 \
-            --json databaseId \
-            --jq "map(.databaseId) | map(select(. != ${GITHUB_RUN_ID})) | .[]")"; then
-            echo '::warning::Prior workflow runs could not be listed; proposing without prior evidence.'
-            exit 0
-          fi
-          if [[ -z "${previous_runs}" ]]; then
-            echo 'No prior completed run to seed from; the proposer starts from the learnings queue only.'
-            exit 0
-          fi
-          seed_root="${RUNNER_TEMP}/wfseed"
-          rm -rf "${seed_root}"
-          install -d -m 0700 "${seed_root}"
-          seed=''
-          # Failed sweeps deliberately upload partial evidence, so "completed"
-          # is the right population. Walk newest-first until one still-retained
-          # artifact actually contains benchmark rows; an empty latest run must
-          # not hide an older useful one.
-          for previous in ${previous_runs}; do
-            if [[ ! "${previous}" =~ ^[0-9]+$ ]]; then
-              echo "::warning::Ignoring malformed prior run id: ${previous}"
-              continue
-            fi
-            run_root="${seed_root}/${previous}"
-            install -d -m 0700 "${run_root}"
-            if ! gh run download "${previous}" --repo "${GITHUB_REPOSITORY}" --dir "${run_root}"; then
-              echo "::warning::Evidence from run ${previous} could not be downloaded (expired or absent); trying an older run."
-              continue
-            fi
-            unsafe="$(find "${run_root}" ! -type d ! -type f -print -quit)"
-            if [[ -n "${unsafe}" ]]; then
-              echo "::warning::Run ${previous} contains a non-regular artifact entry; trying an older run."
-              continue
-            fi
-            # upload-artifact normalizes directories/files to 0755/0644, while
-            # the evidence reader deliberately requires transcript paths to be
-            # owner-only. Restore that trust-boundary invariant after download.
-            if ! chmod -R go-rwx "${run_root}"; then
-              echo "::warning::Evidence permissions from run ${previous} could not be restricted; trying an older run."
-              continue
-            fi
-            # The artifact holds gen-N/bench/{results.jsonl,promotion.json,...};
-            # the highest generation is the one that actually reached the gate.
-            latest="$(find "${run_root}" -type f -path '*/gen-*/bench/results.jsonl' | sort -V | tail -1)"
-            if [[ -z "${latest}" || -L "${latest}" || ! -f "${latest}" ]]; then
-              echo "::warning::Run ${previous} uploaded no usable benchmark results; trying an older run."
-              continue
-            fi
-            # Existence is insufficient: an interrupted run may leave an empty,
-            # malformed, or session/infra-only JSONL. Reuse the same bounded
-            # selection and transcript/digest preflight the proposer will use,
-            # so an unusable newer run cannot hide an older useful one.
-            if uv run --project eval --locked --extra dev python -c \
-              'from pathlib import Path; import json, sys, tempfile; from workflow_bench.evolve import load_jsonl, select_evidence, stage_proposer_evidence_bundle, summarize_gate; result = Path(sys.argv[1]); root = result.parent; rows = select_evidence(load_jsonl(result)); rows or sys.exit(10); promotion = root / "promotion.json"; gate = summarize_gate(json.loads(promotion.read_text())) if promotion.is_file() else []; prior = root.parent / "proposal.md"; prior = prior if prior.is_file() and not prior.is_symlink() else None; dest = Path(tempfile.mkdtemp(prefix="wfseed-preflight-")) / "bundle"; stage_proposer_evidence_bundle(dest, results_dir=root, evidence=rows, learnings=[], gate_summary=gate, prior_proposal=prior)' \
-              "${latest}"; then
-              :
-            else
-              usability_status=$?
-              echo "::warning::Run ${previous} failed evidence preflight (exit ${usability_status}); trying an older run."
-              continue
-            fi
-            seed="$(dirname "${latest}")"
-            echo "Seeding the proposer from run ${previous}: ${seed}"
-            break
-          done
-          if [[ -z "${seed}" ]]; then
-            echo '::warning::No usable prior benchmark artifact found; proposing without prior evidence.'
-            exit 0
-          fi
-          echo "seed=${seed}" >> "${GITHUB_OUTPUT}"
 
       - name: Run the propose → benchmark → gate loop
         id: loop
-        # Kill the sweep with time left in the job to upload what it produced.
-        # See the budget nesting on the job above.
-        timeout-minutes: 1140
         env:
-          GITNEXUS_BENCH_ANTHROPIC_API_KEY: ${{ secrets.GITNEXUS_BENCH_ANTHROPIC_API_KEY || secrets.GITNEXUS_BENCH_AUTH_TOKEN }}
+          GITNEXUS_BENCH_AUTH_TOKEN: ${{ secrets.GITNEXUS_BENCH_AUTH_TOKEN }}
-          GITNEXUS_BENCH_OPENAI_API_KEY: ${{ secrets.GITNEXUS_BENCH_OPENAI_API_KEY }}
-          # The step's stdout is a pipe, so CPython block-buffers it and a
-          # multi-hour generation would report nothing until it exits (run
-          # 29907431284 emitted every line at the same timestamp, 14h45m in).
-          PYTHONUNBUFFERED: '1'
-          SEED_RESULTS: ${{ steps.seed.outputs.seed }}
-          EVOLUTION_PROFILE: review
-          CE_PLUGIN_DIR: ${{ runner.temp }}/compound-engineering-plugin
-          CE_PLUGIN_VERSION: 3.24.0
         run: |
           set -euo pipefail
-          ./workflow_bench/run-evolution.sh --apply
+          out_root="${RUNNER_TEMP}/wfevolve"
+          echo "out_root=${out_root}" >> "${GITHUB_OUTPUT}"
+          extra=()
+          if [[ -n "${INCLUDE_EXPENSIVE}" ]]; then
+            extra+=(--include-expensive)
+          fi
+          uv run --locked --extra dev python -m workflow_bench.evolve \
+            --tasks workflow_bench/tasks.scenarios.yaml \
+            --model "${MODEL}" \
+            --proposer-model "${PROPOSER_MODEL}" \
+            --generations "${GENERATIONS}" \
+            --runs "${RUNS}" \
+            --claude-bin "${RUNNER_TEMP}/claude-canary/node_modules/@anthropic-ai/claude-code-linux-x64/claude" \
+            --out-root "${out_root}" \
+            --apply \
+            "${extra[@]}"
         working-directory: eval
 
       - name: Upload benchmark evidence
-        # Unconditional: the sweep writes results.jsonl and transcripts as it
+        if: always() && steps.loop.outputs.out_root != ''
-        # goes, so a killed or failed generation still has evidence worth
-        # keeping — and that is exactly the run whose evidence is needed.
-        if: always()
         uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
         with:
           name: gitnexus-evolution-${{ github.run_id }}-${{ github.run_attempt }}
-          # Addressed directly rather than carried from the sweep step: that is
+          path: ${{ steps.loop.outputs.out_root }}
-          # the step whose death is the reason this upload matters, and a value
-          # threaded from it would not be there when it counts.
-          path: ${{ runner.temp }}/wfevolve
           retention-days: 14
           if-no-files-found: warn
 
       - name: Detect and bound the applied promotion
         id: promotion
+        env:
+          OUT_ROOT: ${{ steps.loop.outputs.out_root }}
         run: |
           set -euo pipefail
           changed="$(git status --porcelain)"
@@ -526,7 +259,7 @@ jobs:
           while IFS= read -r line; do
             path="${line:3}"
             case "${path}" in
-              .claude/skills/gitnexus-review/*|gitnexus/skills/gitnexus-review/*|gitnexus-claude-plugin/skills/gitnexus-review/*|gitnexus-cursor-integration/skills/gitnexus-review/*) ;;
+              .claude/skills/*|gitnexus/skills/*|gitnexus-claude-plugin/skills/*) ;;
               *)
                 echo "::error::Promotion touched a path outside the skill trees: ${path}"
                 exit 1
@@ -540,7 +273,7 @@ jobs:
           # generation's decisions could surface in the PR body. The heredoc
           # uses a per-run random delimiter so a summary value that ever
           # contains the marker cannot close the block early and inject keys.
-          promotion_file="$(find "${RUNNER_TEMP}/wfevolve" -name promotion.json | sort -V | tail -1)"
+          promotion_file="$(find "${OUT_ROOT}" -name promotion.json | sort -V | tail -1)"
           delim="PROMOTION_EOF_$(openssl rand -hex 16)"
           {
             echo "summary<<${delim}"
@@ -583,7 +316,7 @@ jobs:
           git config user.name 'gitnexus-evolution[bot]'
           git config user.email 'gitnexus-evolution[bot]@users.noreply.github.com'
           git checkout -b "${branch}"
-          git add .claude/skills gitnexus/skills gitnexus-claude-plugin/skills gitnexus-cursor-integration/skills/gitnexus-review
+          git add .claude/skills gitnexus/skills gitnexus-claude-plugin/skills
           git commit -m 'feat(skills): promoted evolution overlay (gate-passed)'
 
           # The App token reaches git through GIT_ASKPASS reading step env at

.github/workflows/grammar-update-monitor.yml

@@ -44,7 +44,7 @@ jobs:
     permissions:
       contents: read
     steps:
-      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
         with:
           persist-credentials: false
 

.github/workflows/impact-pdg-mutation-report.yml

@@ -37,7 +37,7 @@ jobs:
       # artifact and never pushes; the default-persisted token in .git/config
       # must not be capturable through that upload (zizmor credential-persistence
       # / artipacked audit). Mirrors ci-tests.yml.
-      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
         with:
           persist-credentials: false
 

.github/workflows/pr-autofix-apply.yml

@@ -336,7 +336,7 @@ jobs:
       # Push auth is provided inline at push time via the URL.
       - name: Checkout PR head
         if: steps.locate.outputs.found == 'true'
-        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v5.0.4
         with:
           repository: ${{ steps.locate.outputs.head_repo }}
           ref: ${{ steps.locate.outputs.head_sha }}

.github/workflows/pr-autofix-publish.yml

@@ -121,35 +121,64 @@ jobs:
       # metadata.json to reference another PR or SHA, redirecting our
       # write-scoped sticky/check-run onto an attacker-chosen target.
       #
-      # Authority is workflow_run.head_sha + head_repository.full_name +
+      # Authority sources are all server-controlled GitHub event fields:
-      # head_branch, resolved via pulls?head={owner}:{branch}. That query
+      #   - workflow_run.head_sha
-      # works for same-repo PRs and forks; commits/{sha}/pulls is empty
+      #   - workflow_run.head_repository.full_name
-      # for fork SHAs. The script comes from THIS default-branch checkout
+      #   - workflow_run.pull_requests[].number  (within-repo PRs only;
-      # (the same trust anchor as this workflow file).
+      #     empty array on fork PRs — fall back to commits/{sha}/pulls)
       #
-      # Always verify — including the changed_lines=0 path — so the
-      # check-run SHA cannot be an unverified artifact field.
       # Mismatch => fail loud BEFORE any sticky/check-run side effect.
-      - name: Checkout identity verifier
-        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
-        with:
-          persist-credentials: false
-          sparse-checkout: .github/scripts/verify-workflow-run-pr-identity.cjs
-          sparse-checkout-cone-mode: false
-          path: trusted
-
       - name: Verify metadata against workflow_run authority
         id: verify
+        if: steps.meta.outputs.changed_lines != '0'
         env:
           GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
           GH_REPO: ${{ github.repository }}
-          META_PATH: autofix-in/metadata.json
+          META_PR_NUMBER: ${{ steps.meta.outputs.pr_number }}
-          SCHEMA_PATTERN: '^gitnexus\.pr-autofix/v[0-9]+$'
+          META_HEAD_SHA: ${{ steps.meta.outputs.head_sha }}
+          META_HEAD_REPO: ${{ steps.meta.outputs.head_repo }}
           WF_HEAD_SHA: ${{ github.event.workflow_run.head_sha }}
           WF_HEAD_REPO: ${{ github.event.workflow_run.head_repository.full_name }}
-          WF_HEAD_BRANCH: ${{ github.event.workflow_run.head_branch }}
+          WF_PR_NUMBERS: ${{ toJSON(github.event.workflow_run.pull_requests.*.number) }}
         shell: bash
-        run: node trusted/.github/scripts/verify-workflow-run-pr-identity.cjs
+        run: |
+          set -euo pipefail
+
+          # 1) head_sha must match exactly. workflow_run.head_sha is the
+          # commit GitHub actually ran the producer against — definitive.
+          if [ "${META_HEAD_SHA}" != "${WF_HEAD_SHA}" ]; then
+            echo "::error::Artifact head_sha (${META_HEAD_SHA}) does not match workflow_run.head_sha (${WF_HEAD_SHA}) — refusing to publish."
+            exit 1
+          fi
+
+          # 2) head_repo must match exactly. Same authority anchor.
+          if [ "${META_HEAD_REPO}" != "${WF_HEAD_REPO}" ]; then
+            echo "::error::Artifact head_repo (${META_HEAD_REPO}) does not match workflow_run.head_repository (${WF_HEAD_REPO}) — refusing to publish."
+            exit 1
+          fi
+
+          # 3) pr_number must reference an open PR with this head SHA.
+          # Within-repo PRs: workflow_run.pull_requests[] is populated.
+          # Fork PRs: that array is empty by GitHub design — fall back
+          # to the REST commit-to-PRs lookup. Fail closed if the lookup
+          # finds no matching open PR (avoids attacker-forged PR ids).
+          allowed_numbers=$(jq -c '.' <<< "${WF_PR_NUMBERS}")
+          if [ "${allowed_numbers}" = "[]" ]; then
+            echo "workflow_run.pull_requests is empty (fork PR) — falling back to commits/{sha}/pulls."
+            allowed_numbers=$(gh api "repos/${GH_REPO}/commits/${WF_HEAD_SHA}/pulls" \
+              --jq '[.[] | select(.state == "open") | .number]' 2>/dev/null || echo "[]")
+            if [ "${allowed_numbers}" = "[]" ]; then
+              echo "::error::No open PR found for head ${WF_HEAD_SHA} via commits/{sha}/pulls — refusing to publish."
+              exit 1
+            fi
+          fi
+
+          if ! jq -e --argjson n "${META_PR_NUMBER}" 'index($n) != null' <<< "${allowed_numbers}" >/dev/null; then
+            echo "::error::Artifact pr_number (${META_PR_NUMBER}) is not in the authoritative PR list (${allowed_numbers}) — refusing to publish."
+            exit 1
+          fi
+
+          echo "Verified: metadata identity matches workflow_run authority (PR=${META_PR_NUMBER}, head_sha=${META_HEAD_SHA}, head_repo=${META_HEAD_REPO})."
 
       - name: Upsert sticky summary comment
         # Only post when ci-quality found something fixable (= the
@@ -158,14 +187,14 @@ jobs:
         # so we skip it.
         if: >-
           always()
-          && steps.verify.outcome == 'success'
+          && steps.meta.outputs.pr_number != ''
           && steps.meta.outputs.changed_lines != '0'
         env:
           GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
           GH_REPO: ${{ github.repository }}
-          PR: ${{ steps.verify.outputs.pr_number }}
+          PR: ${{ steps.meta.outputs.pr_number }}
           CHANGED: ${{ steps.meta.outputs.changed_lines }}
-          HEAD_SHA: ${{ steps.verify.outputs.head_sha }}
+          HEAD_SHA: ${{ steps.meta.outputs.head_sha }}
           RUN_ID: ${{ github.run_id }}
         shell: bash
         run: |
@@ -256,11 +285,11 @@ jobs:
         #   fixes-available  → conclusion: neutral
         # `neutral` does not block branch-protection required-checks but
         # is visually distinct from a green pass.
-        if: always() && steps.verify.outcome == 'success'
+        if: always() && steps.meta.outputs.head_sha != ''
         env:
           GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
           GH_REPO: ${{ github.repository }}
-          HEAD_SHA: ${{ steps.verify.outputs.head_sha }}
+          HEAD_SHA: ${{ steps.meta.outputs.head_sha }}
           CHANGED: ${{ steps.meta.outputs.changed_lines }}
         shell: bash
         run: |

.github/workflows/pr-autofix.yml

@@ -51,7 +51,7 @@ jobs:
     runs-on: ubuntu-latest
     timeout-minutes: 10
     steps:
-      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
         with:
           # PR head commit (not the synthetic merge ref) — we need the
           # exact tree the contributor pushed so suggestions line up.

.github/workflows/publish.yml

@@ -162,7 +162,7 @@ jobs:
       should_run: ${{ steps.decide.outputs.should_run }}
       head_sha: ${{ steps.decide.outputs.head_sha }}
     steps:
-      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
         with:
           fetch-depth: 0
           fetch-tags: true
@@ -332,7 +332,7 @@ jobs:
       # on the RC path.
       - name: Checkout (RC)
         if: needs.route.outputs.mode == 'rc'
-        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
         with:
           fetch-depth: 0
           fetch-tags: true
@@ -349,7 +349,7 @@ jobs:
 
       - name: Checkout (stable)
         if: needs.route.outputs.mode == 'stable'
-        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
         # No `token:` — actions/checkout uses GITHUB_TOKEN by default. Stable
         # path performs no git pushes; the default scope is sufficient.
         with:
@@ -692,20 +692,7 @@ jobs:
           git add ../gitnexus-claude-plugin/.claude-plugin/plugin.json \
             ../.claude-plugin/marketplace.json \
             ../gitnexus-claude-plugin/.codex-plugin/plugin.json \
-            ../.agents/plugins/marketplace.json \
+            ../.agents/plugins/marketplace.json
-            ../gitnexus-factory-plugin/.factory-plugin/plugin.json \
-            ../gitnexus-factory-plugin/mcp.json \
-            ../.factory-plugin/marketplace.json \
-            ../gitnexus-claude-plugin/skills/gitnexus-plan/mcp.json \
-            ../gitnexus-claude-plugin/skills/gitnexus-work/mcp.json \
-            ../gitnexus-claude-plugin/skills/gitnexus-review/mcp.json \
-            ../gitnexus-claude-plugin/skills/gitnexus-lfg/mcp.json \
-            ../gitnexus-claude-plugin/skills/gitnexus-guide/mcp.json \
-            ../gitnexus-claude-plugin/skills/gitnexus-cli/mcp.json \
-            ../gitnexus-claude-plugin/skills/gitnexus-debugging/mcp.json \
-            ../gitnexus-claude-plugin/skills/gitnexus-exploring/mcp.json \
-            ../gitnexus-claude-plugin/skills/gitnexus-impact-analysis/mcp.json \
-            ../gitnexus-claude-plugin/skills/gitnexus-refactoring/mcp.json
           git commit -m "release: ${VTAG}" --allow-empty
           RELEASE_SHA="$(git rev-parse HEAD)"
           echo "Detached release commit: $RELEASE_SHA"
@@ -844,7 +831,7 @@ jobs:
           fi
 
       - name: Create GitHub Release
-        uses: softprops/action-gh-release@efb35369e0ad2afab669f228072c1b0d510eae64 # v2
+        uses: softprops/action-gh-release@3d0d9888cb7fd7b750713d6e236d1fcb99157228 # v2
         with:
           tag_name: ${{ steps.vtag-gate.outputs.vtag }}
           name: >-

.github/workflows/scorecard.yml

@@ -33,7 +33,7 @@ jobs:
 
     steps:
       - name: Checkout
-        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
         with:
           persist-credentials: false
 
@@ -53,6 +53,6 @@ jobs:
           retention-days: 5
 
       - name: Upload to Security tab
-        uses: github/codeql-action/upload-sarif@2892aa5e19bbd11bc0cff5427e3b750a04d9e3c2 # v4.38.2
+        uses: github/codeql-action/upload-sarif@cdf488f595d80d6e07e03d4674febd5ab45fa938 # v4.37.9
         with:
           sarif_file: results.sarif

.github/workflows/skill-sync.yml

@@ -47,7 +47,7 @@ jobs:
     timeout-minutes: 15
     steps:
       # persist-credentials: false — runs a read-only test, never pushes.
-      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
         with:
           persist-credentials: false
       - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0

.github/workflows/tree-sitter-upgrade-readiness.yml

@@ -4,7 +4,7 @@ name: Tree-sitter Upgrade Readiness
 #   1. Peer-dep compatibility — can each NPM-installed grammar install cleanly
 #      with tree-sitter@0.25.0 without --legacy-peer-deps?
 #   2. Vendored grammars — each grammar in .github/vendored-grammars.json
-#      (c/swift/kotlin/dart/proto/objc) is classified by its vendored ABI, read
+#      (c/swift/kotlin/dart/proto) is classified by its vendored ABI, read
 #      straight from gitnexus/vendor/<name>/src/parser.c (NOT node_modules,
 #      which is never populated for vendored grammars — that mismatch is why
 #      the report used to render bare "?" placeholders, #858).
@@ -52,7 +52,7 @@ jobs:
       report: ${{ steps.readiness.outputs.report }}
       exit_code: ${{ steps.readiness.outputs.exit_code }}
     steps:
-      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
         with:
           persist-credentials: false
 

.github/workflows/triage-sweep.yml

@@ -59,7 +59,7 @@ jobs:
     timeout-minutes: 30
     steps:
       - name: Checkout repository
-        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
         with:
           sparse-checkout: .github/scripts/triage
           sparse-checkout-cone-mode: false

.github/workflows/trivy.yml

@@ -45,15 +45,15 @@ jobs:
 
     steps:
       - name: Checkout
-        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
         with:
           persist-credentials: false
 
       - name: Setup Buildx
-        uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4.4.1
+        uses: docker/setup-buildx-action@37fe631027851001ddb9b187196cc803df7f5f0e # v4.3.0
 
       - name: Build image (load locally for scan)
-        uses: docker/build-push-action@c3c9e263c25d99ce0380d002d59b67737d91b0dc # v7.4.0
+        uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7.3.0
         with:
           context: .
           file: ${{ matrix.image.dockerfile }}
@@ -76,7 +76,7 @@ jobs:
           exit-code: '0'
 
       - name: Upload to Security tab
-        uses: github/codeql-action/upload-sarif@2892aa5e19bbd11bc0cff5427e3b750a04d9e3c2 # v4.38.2
+        uses: github/codeql-action/upload-sarif@cdf488f595d80d6e07e03d4674febd5ab45fa938 # v4.37.9
         with:
           sarif_file: trivy-${{ matrix.image.name }}.sarif
           category: trivy-${{ matrix.image.name }}

.github/workflows/workflow-lint.yml

@@ -31,7 +31,7 @@ jobs:
       contents: read
     steps:
       - name: Checkout
-        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
         with:
           persist-credentials: false
 
@@ -53,7 +53,7 @@ jobs:
 
     steps:
       - name: Checkout
-        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
         with:
           persist-credentials: false
 
@@ -76,7 +76,7 @@ jobs:
         continue-on-error: true
 
       - name: Upload SARIF
-        uses: github/codeql-action/upload-sarif@2892aa5e19bbd11bc0cff5427e3b750a04d9e3c2 # v4.38.2
+        uses: github/codeql-action/upload-sarif@cdf488f595d80d6e07e03d4674febd5ab45fa938 # v4.37.9
         with:
           sarif_file: zizmor.sarif
           category: zizmor

.github/zizmor.yml

@@ -18,11 +18,9 @@ rules:
       # untrusted half (pr-autofix.yml) runs fork code with permissions:{}
       # and produces only a diff artifact (data, not executable code). The
       # publish job consumes the artifact, allowlist-validates every field
-      # of metadata.json, then cross-checks identity against
+      # of metadata.json before exporting to $GITHUB_OUTPUT, never checks
-      # workflow_run.head_sha / head_repository / head_branch via
+      # out fork code, and never executes anything fork-controlled. Header
-      # pulls?head=owner:branch (commits/{sha}/pulls is empty for fork SHAs).
+      # comment in the file documents the split.
-      # It never checks out fork code and never executes anything
-      # fork-controlled. Header comment in the file documents the split.
       - pr-autofix-publish.yml
 
       # workflow_run is the trusted half of the vendored-grammar prebuild
@@ -31,8 +29,7 @@ rules:
       # validates the .node prebuilds and uploads them as artifacts. This
       # consumer downloads ONLY those artifacts + metadata.json,
       # allowlist-validates every metadata field, cross-checks identity against
-      # workflow_run.head_sha / head_repository / head_branch via
+      # the workflow_run authority (head_sha / head_repo / pr_number), and
-      # pulls?head=owner:branch (commits/{sha}/pulls is empty for fork SHAs), and
       # checks out the fork head pinned to that HEAD SHA solely to ADD prebuild
       # files (never executes fork code) before pushing. Header comment in the
       # file documents the split.

.gitignore

@@ -31,7 +31,6 @@ npm-debug.log*
 
 # Testing
 coverage/
-.vitest/
 .tmp-test/
 gitnexus/.tmp-test/
 
@@ -66,7 +65,6 @@ repomix-output*
 # Playwright artifacts
 gitnexus-web/playwright-report/
 gitnexus-web/test-results/
-gitnexus-web/e2e/screenshots/
 
 # Python test artifacts
 eval/.coverage
@@ -74,8 +72,6 @@ eval/.hypothesis/
 
 # Local docs — planning output (gitnexus-plan / gitnexus-work) stays local, not tracked
 docs/*
-!docs/fork/
-!docs/fork/**
 
 gitnexus/test/fixtures/mini-repo/*.md
 gitnexus/test/fixtures/mini-repo/.claude
@@ -132,13 +128,5 @@ local_docs/
 .context/
 gitnexus/web/
 
-# Local copies of CI execution receipts.
-gitnexus/benchmarks.json
-gitnexus/preflight.json
-gitnexus/test-results.json
-gitnexus/windows-latest-*.json
-gitnexus/macos-latest-*.json
-gitnexus-web/web-test-results.json
-
 # Machine-local skill-evolution evidence (consumed by eval/workflow_bench/evolve.py)
 eval/workflow_bench/learnings.jsonl

.vercelignore

@@ -1,28 +0,0 @@
-# Keep Vercel uploads under the 100 MB file limit — SPA only needs web + shared sources.
-.git
-.gitnexus
-.gitnexus/**
-node_modules
-**/node_modules
-gitnexus/**
-!gitnexus/package.json
-eval
-eval/**
-.claude
-.cursor
-.github
-docs
-Documentation
-.devcontainer
-gitnexus-claude-plugin
-gitnexus-cursor-integration
-pr-swarm-review
-ci-personas
-*.sqlite*
-*.db
-dist
-**/dist
-coverage
-**/coverage
-playwright-report
-test-results

AGENTS.md

@@ -1,7 +1,7 @@
-<!-- version: 1.17.0 -->
+<!-- version: 1.14.0 -->
-<!-- Last updated: 2026-09-24 -->
+<!-- Last updated: 2026-07-16 -->
 
-Last reviewed: 2026-09-24
+Last reviewed: 2026-07-16
 
 **Project:** GitNexus · **Environment:** dev · **Maintainer:** repository maintainers (see GitHub)
 
@@ -39,7 +39,6 @@ Commands and gotchas live under **Repo reference** below and in **[CONTRIBUTING.
 ## Reference docs
 
 - **[ARCHITECTURE.md](ARCHITECTURE.md)**, **[CONTRIBUTING.md](CONTRIBUTING.md)**, **[GUARDRAILS.md](GUARDRAILS.md)**
-- **Objective-C provider work:** read **[docs/languages/objective-c-provider.md](docs/languages/objective-c-provider.md)** before changing Objective-C parsing or resolution.
 - **Call & inheritance resolution (RFC #909 Ring 3):** See ARCHITECTURE.md § Scope-Resolution Pipeline. All languages resolve calls and inheritance through the scope-resolution pipeline (`Registry.lookup`, `preEmitInheritanceEdges`, `emitHeritageEdges`, `buildMro` → `MethodDispatchIndex`). **Shared code in `gitnexus/src/core/ingestion/` must not name languages** — plug language behavior in via `LanguageProvider` / `ScopeResolver` hooks. A language plugs in by implementing `ScopeResolver` (`scope-resolution/contract/scope-resolver.ts`) and registering it in `SCOPE_RESOLVERS`. (The legacy call-resolution DAG + `@heritage` capture path were removed in RING4-1 #942.)
 - **Cursor:** `.cursor/index.mdc` (always-on); `.cursor/rules/*.mdc` (glob-scoped). Legacy `.cursorrules` deprecated.
 - **GitNexus:** standard skills in `.claude/skills/gitnexus-*/`; MCP rules in `gitnexus:start` block below.
@@ -91,9 +90,6 @@ mirror. `gitnexus/test/unit/shipped-skills-sync.test.ts` guards the copies. Toke
 
 | Date | Version | Change |
 |------|---------|--------|
-| 2026-09-24 | 1.17.0 | Clones with the same `origin` URL now share a store automatically; `--no-share` records a lasting opt-out (#3352). |
-| 2026-09-24 | 1.16.0 | Documented the shared worktree index store (`<GITNEXUS_HOME>/stores/`, `analyze --share-with`, `GITNEXUS_SHARED_STORE=off`) in the storage notes (#3352). |
-| 2026-09-07 | 1.15.0 | Added the Objective-C provider guide as the required reference before changing Objective-C parsing or resolution. |
 | 2026-07-20 | 1.14.0 | `gitnexus-review` gains a coordinated swarm: six `ci-personas/` lanes the CI review agent dispatches as subagents (via the `Agent` tool), with a bounded critic gate and sidechain-excluded evidence. |
 | 2026-07-16 | 1.13.0 | `gitnexus-plan` asks plan depth up front (quick/standard/deep) in interactive runs; `gitnexus-lfg` gate slimmed to proceed/stop (Deepen stays as the route-back mechanism). |
 | 2026-07-16 | 1.12.0 | Renamed `gitnexus-pr-review` to `gitnexus-review`; added PR URL/number, branch/range, and local-change targets plus install migration (setup warns on a legacy `gitnexus-pr-review` dir and leaves it in place; uninstall removes it). |
@@ -118,7 +114,6 @@ mirror. `gitnexus/test/unit/shipped-skills-sync.test.ts` guards the copies. Toke
 This project is indexed by GitNexus as **GitNexus** (248612 symbols, 565510 relationships, 918 execution flows).
 
 > Index stale? Run `node .gitnexus/run.cjs analyze --index-only` from the project root — it auto-selects an available runner. No `.gitnexus/run.cjs` yet? Bootstrap with `npx`, `bunx`, or `pnpm dlx` — e.g. `bunx gitnexus@latest analyze` (npm 11 npx crash; #1939).
-> On query/context/impact/cypher object results, read staleness.status and branch/lastCommit. Re-analyze only for behind or diverged — current is clone HEAD, not main.
 
 ## Always Do
 
@@ -197,7 +192,6 @@ npx gitnexus serve                         # HTTP API on port 4747 (from any ind
 
 ### Gotchas
 
-- `npm install` in `gitnexus/` triggers `prepare` (builds via `tsc`) and `postinstall` (`build-tree-sitter-grammars.cjs` activates committed prebuilds in place under `vendor/`, and only source-builds when none matches). A C/C++ toolchain (`python3`, `make`, `g++`) is needed only for that source-build fallback.
+- `npm install` in `gitnexus/` triggers `prepare` (builds via `tsc`) and `postinstall` (materializes the vendored grammars into `node_modules/`, then prefers a committed prebuild per platform-arch and only source-builds when none matches). A C/C++ toolchain (`python3`, `make`, `g++`) is needed only for that source-build fallback.
-- The vendored grammars `tree-sitter-{c,dart,proto,swift,kotlin,zig}` are handled uniformly: c is required; dart/proto/swift/kotlin/zig are optional and skippable via `GITNEXUS_SKIP_OPTIONAL_GRAMMARS=1`. Install warnings appear only when no prebuild matches the platform-arch and no toolchain is present, and are non-fatal — only that language's parsing is unavailable.
+- The vendored grammars `tree-sitter-{c,dart,proto,swift,kotlin}` are handled uniformly: c is required; dart/proto/swift/kotlin are optional and skippable via `GITNEXUS_SKIP_OPTIONAL_GRAMMARS=1`. Install warnings appear only when no prebuild matches the platform-arch and no toolchain is present, and are non-fatal — only that language's parsing is unavailable.
 - ESLint configured via `eslint.config.mjs` (TS, React Hooks, unused-imports). No `npm run lint` script; use `npx eslint .`. Prettier runs via lint-staged. CI checks both in `ci-quality.yml`.
-- Index storage defaults to `<repo>/.gitnexus/`. `GITNEXUS_STORAGE_PATH` selects one complete external index directory and wins over `GITNEXUS_STORAGE_ROOT`, which creates an isolated `<repo-basename>-<12-hex>/` slot per repository. Linked worktrees share one store under `<GITNEXUS_HOME>/stores/<key>/` (one immutable graph per commit, private graphs for checkouts with local changes, shared parse caches); clones with the same `origin` URL join a registered sibling's store automatically (`analyze --share-with` names one, `--no-share` opts out and is remembered), and `GITNEXUS_SHARED_STORE=off` or either storage env var disables sharing (#3352). `GITNEXUS_CONTENT_RETENTION` is `full` (default), `symbol`, or `none`. MCP `list_repos`, `gitnexus://repo/{name}/context`, and HTTP `GET /api/repos` / `GET /api/repo` expose `storagePath`, `contentRetention`, and `sourceAvailable`. HTTP `/api/file` and `/api/grep` return 410 unless retention is `full`; MCP `include_content` may still return symbol spans at `symbol`.

ARCHITECTURE.md

@@ -24,7 +24,7 @@ Monorepo: **CLI/MCP** (`gitnexus/`) + **browser UI** (`gitnexus-web/`).
    - **HTTP bridge:** `serve.ts` → Express (`api.ts`, `mcp-http.ts`) for web UI
    - **CLI direct:** `gitnexus query|context|impact|cypher` in `tool.ts`
 
-4. **Staleness** — `core/git-staleness.ts` compares indexed `lastCommit` to `HEAD` and classifies the result as `current`, `behind`, `diverged` (HEAD moved off the indexed commit, gap uncountable) or `unknown`; `core/staleness-status.ts` builds the one `staleness` payload that MCP `list_repos`, the read tools and the `serve` repo routes all emit.
+4. **Staleness** — `staleness.ts` compares indexed `lastCommit` to `HEAD`, surfaces hints.
 
 ## MCP tools
 
@@ -377,7 +377,7 @@ CI auto-discovers the set via `tsx`. No workflow edit required.
 
 ## Language-agnostic graph feeding
 
-18 languages → single unified graph. Four abstraction layers:
+16 languages → single unified graph. Four abstraction layers:
 
 ```
  Unified Graph Schema (44 node types, 21 relationship types)
@@ -405,7 +405,7 @@ Each language implements `LanguageProvider` (`language-provider.ts`). Key fields
 | `descriptionExtractor` | Optional hook returning a symbol's doc-comment text as its `description`; feeds the embedding metadata header so doc-only terms are semantically searchable (issue #2270). Most languages register `createLeadingDocDescriptionExtractor` (shared, language-neutral; per-language comment/wrapper config passed at the call site) |
 | `definitionPropertiesExtractor` | Optional language-owned hook for structured, clone-safe definition metadata. Shared ingestion persists these properties opaquely; the owning provider supplies the extraction semantics. |
 
-18 providers in `languages/index.ts` via `satisfies Record<SupportedLanguages, LanguageProvider>` — missing a language is a compile error.
+16 providers in `languages/index.ts` via `satisfies Record<SupportedLanguages, LanguageProvider>` — missing a language is a compile error.
 
 ### Unified capture tags
 
@@ -485,34 +485,14 @@ CLI (analyze.ts) → runFullAnalysis(repoPath, options, callbacks)
   ├── lbug.wal       # Write-ahead log
   ├── lbug.shadow    # Shadow sidecar (checkpoint staging)
   ├── lbug.lock      # Single-writer lock
-  ├── lbug.wal.checkpoint, lbug.checkpoint.{intent,apply}.lock  # checkpoint-in-flight artifacts; left behind only by an interrupted checkpoint, consumed by the next writable open
   ├── lbug.{wal,shadow}.dirty-recovery  # parked sidecars from a crashed run; safe to delete
   ├── gitnexus.json  # lastCommit, indexedAt, stats (primary metadata file)
   └── meta.json      # legacy mirror of gitnexus.json, kept in sync (see MIGRATION.md)
 
 ~/.gitnexus/
-  ├── registry.json  # Global repo registry (MCP discovery)
+  └── registry.json  # Global repo registry (MCP discovery)
-  └── stores/<key>/  # Shared sibling index store (see below)
-      ├── caches/                         # parse cache + durable ParsedFile store
-      ├── commits/<commit>-<featureKey>/  # one immutable graph per commit + settings
-      └── checkouts/<slot>/               # one checkout's metadata, membership, and
-                                          # private graph when it has local edits
 ```
 
-The flat `<repo>/.gitnexus/` layout applies to a standalone repository and
-whenever `GITNEXUS_STORAGE_PATH` / `GITNEXUS_STORAGE_ROOT` is set. A repository
-with linked worktrees, and clones with the same `origin` URL, share one
-`stores/<key>/` automatically (a clone opts out with `analyze --no-share`;
-`GITNEXUS_SHARED_STORE=off` turns sharing off entirely). Each sharing checkout
-keeps only a `.gitnexus/store.json` pointer to its store. Path resolution lives
-in `shared-store.ts`.
-
-Read-only opens self-heal an interrupted checkpoint: the refusal is
-classified and cleared by one writable open (probe + `CHECKPOINT`) before
-the read-only open is retried — see `sidecar-recovery.ts`
-(`isReadOnlyCheckpointInProgressError`) and the
-`lbug-interrupted-checkpoint-recovery` integration test.
-
 Managed by `repo-manager.ts`.
 
 ## LadybugDB schema
@@ -562,5 +542,4 @@ Node IDs use arity suffix (`#<paramCount>`): `Method:file:Class.method#1` vs `#2
 - [RUNBOOK.md](RUNBOOK.md) — operational commands and recovery
 - [GUARDRAILS.md](GUARDRAILS.md) — safety boundaries for humans and agents
 - [TESTING.md](TESTING.md) — how to run tests
-- [docs/languages/objective-c-provider.md](docs/languages/objective-c-provider.md) — Objective-C provider behavior and limits
 - `AGENTS.md` / `CLAUDE.md` — agent workflows and tool usage

CLAUDE.md

@@ -65,7 +65,6 @@ See the `<!-- gitnexus:start --> … <!-- gitnexus:end -->` block in **[AGENTS.m
 This project is indexed by GitNexus as **GitNexus** (248612 symbols, 565510 relationships, 918 execution flows).
 
 > Index stale? Run `node .gitnexus/run.cjs analyze --index-only` from the project root — it auto-selects an available runner. No `.gitnexus/run.cjs` yet? Bootstrap with `npx`, `bunx`, or `pnpm dlx` — e.g. `bunx gitnexus@latest analyze` (npm 11 npx crash; #1939).
-> On query/context/impact/cypher object results, read staleness.status and branch/lastCommit. Re-analyze only for behind or diverged — current is clone HEAD, not main.
 
 ## Always Do
 

CONTRIBUTING.md

@@ -16,19 +16,14 @@ This project uses the [PolyForm Noncommercial License 1.0.0](https://polyformpro
 **Prerequisites:** Node.js — `gitnexus/` requires `^22.18.0 || >=24.11.0` and `gitnexus-web/` requires `^20.19.0 || >=22.12.0` (enforced via the `engines` field in each package). Use `nvm install` to match the local version.
 
 1. Clone the repository.
-2. **CLI / MCP package:** `cd gitnexus && npm install && npm run build`
+2. **Shared package:** `cd gitnexus-shared && npm install && npm run build`
-   `prepare` / `scripts/build.js` compiles `gitnexus-shared` with
+3. **CLI / MCP package:** `cd ../gitnexus && npm install && npm run build`
-   `node …/typescript/lib/tsc.js` from this package. Do not `npm install` or
+4. **Web UI (if needed):** `cd ../gitnexus-web && npm install`
-   `npm ci` inside `gitnexus-shared/` — that is a second TypeScript 7
+5. Run tests as described in [TESTING.md](TESTING.md).
-   optional-platform install and is not what `setup-gitnexus` does.
-3. **Web UI (if needed):** `cd gitnexus-web && npm install`
-   If you skipped step 2, compile shared with the web compiler first:
-   `cd gitnexus-shared && node ../gitnexus-web/node_modules/typescript/lib/tsc.js`
-4. Run tests as described in [TESTING.md](TESTING.md).
 
-The CLI build imports `gitnexus-shared`, so `gitnexus-shared/dist` must exist
+The CLI build imports `gitnexus-shared`, so a fresh clone must install and build
-before `gitnexus` typecheck. That emit uses a parent package's TypeScript 7
+the shared package before running `npm install` in `gitnexus/`. This is the same
-`lib/tsc.js`, matching `setup-gitnexus`, `setup-gitnexus-web`, and Vercel.
+order used by the repository's `setup-gitnexus` CI action.
 
 ### Containerized development (optional)
 
@@ -75,7 +70,7 @@ Commits within a PR may use any style — only the **merged PR title** shows up
 ## Before you open a PR
 
 - [ ] Tests pass for the packages you touched (`gitnexus` and/or `gitnexus-web`).
-- [ ] Typecheck passes: `npx tsc --noEmit` in `gitnexus/` and `npx tsc -b --noEmit` in `gitnexus-web/`. Those commands use TypeScript 7. The web app tsconfig lists `lib` `DOM`/`DOM.Iterable` and `jsx: react-jsx` so React/JSX typecheck on 7; Vite/`vitest` keep `@vitejs/plugin-react` with the automatic JSX runtime. Build `gitnexus-shared/dist` first with a parent `lib/tsc.js` (a `gitnexus` install/`npm run build` does this). Repo ESLint stays syntax-only on a TypeScript 5.x peer until typescript-eslint supports 7.
+- [ ] Typecheck passes: `npx tsc --noEmit` in `gitnexus/` and `npx tsc -b --noEmit` in `gitnexus-web/`.
 - [ ] No secrets, tokens, or machine-specific paths committed.
 - [ ] Documentation updated if behavior or public CLI/MCP contract changes.
 - [ ] Every new `GITNEXUS_*` environment variable has a row in the **Environment variables** table in [README.md](README.md) — variable, default, effect, and when to tune it.

DoD.md

@@ -113,7 +113,7 @@ Run the commands relevant to the touched area. If something cannot be run in the
 
 ### 4.1 Build ordering
 
-- [ ] `gitnexus-shared/` dist is built before consuming packages are typechecked or tested (CI uses the `setup-gitnexus` action, which compiles shared with the parent TypeScript 7 `lib/tsc.js` — local runs must match).
+- [ ] `gitnexus-shared/` dist is built before consuming packages are typechecked or tested (CI uses the `setup-gitnexus` action for this — local runs must match).
 
 ### 4.2 If `gitnexus/` changed
 
@@ -123,13 +123,13 @@ Run the commands relevant to the touched area. If something cannot be run in the
 
 ### 4.3 If `gitnexus-web/` changed
 
-- [ ] `cd gitnexus-web && npx tsc -b --noEmit` (TypeScript 7 typechecks React/JSX: `jsx: react-jsx`, `lib` includes `DOM`)
+- [ ] `cd gitnexus-web && npx tsc -b --noEmit`
 - [ ] `cd gitnexus-web && npm test`
 - [ ] `cd gitnexus-web && npm run test:e2e` when browser flows or user-facing UI behavior changed
 
 ### 4.4 If `gitnexus-shared/` changed
 
-- [ ] Shared package builds cleanly from a parent TypeScript 7 shim after that parent is installed (`cd gitnexus-shared && node ../gitnexus/node_modules/typescript/lib/tsc.js`, or `node ../gitnexus-web/node_modules/typescript/lib/tsc.js` after a web install). Do not `npm install` / `npm ci` in `gitnexus-shared/` for this check.
+- [ ] Shared package builds cleanly (`npm run build` in `gitnexus-shared/`)
 - [ ] Dependent packages still typecheck and test after the shared change — verify both CLI and web consumers together
 
 ### 4.5 If CI workflows or release pipelines changed

Dockerfile.cli

@@ -6,11 +6,8 @@ ARG TARGETPLATFORM
 ARG NPM_VERSION=11.14.1
 
 # -- Builder -----------------------------------------------------------
-# Native modules (tree-sitter-*, node-gyp builds for
+# Native modules (tree-sitter-*, onnxruntime-node, node-gyp builds for
 # tree-sitter-proto / tree-sitter-swift) require python3 + a C/C++ toolchain.
-# onnxruntime-node is not installed by `npm ci`; local embeddings need
-# `gitnexus embeddings install` (or HTTP env / a bind-mounted prefix). The
-# runtime stage strips npm, so this image cannot auto-heal the stack.
 # node:22-bookworm-slim
 FROM node:22-bookworm-slim@sha256:9f6d5975c7dca860947d3915877f85607946403fc55349f39b4bc3688448bb6e AS builder
 ARG NPM_VERSION
@@ -55,9 +52,8 @@ RUN npm run postinstall --prefix gitnexus
 FROM node:22-bookworm-slim@sha256:9f6d5975c7dca860947d3915877f85607946403fc55349f39b4bc3688448bb6e AS runtime
 
 # curl for the healthcheck; git for cloning; procps for watch process identity;
-# ca-certificates for TLS verification; openssh-client so auto-sync SSH remotes
+# ca-certificates for TLS verification.
-# can clone (git invokes `ssh`; --no-install-recommends omits it from git).
+RUN apt-get update && apt-get install -y --no-install-recommends curl git procps ca-certificates && rm -rf /var/lib/apt/lists/* \
-RUN apt-get update && apt-get install -y --no-install-recommends curl git procps ca-certificates openssh-client && rm -rf /var/lib/apt/lists/* \
     && rm -rf /usr/local/lib/node_modules/npm \
     && rm -rf /usr/local/lib/node_modules/corepack \
     && rm -f /usr/local/bin/npm /usr/local/bin/npx /usr/local/bin/corepack

README.md

@@ -1,4 +1,6 @@
-# GitNexus 
+# GitNexus (Akon Labs)
+
+**⚠️ Important Notice:** GitNexus has NO official cryptocurrency, token, or coin. Any token/coin using the GitNexus name on Pump.fun or any other platform is **not affiliated with, endorsed by, or created by** this project or its maintainers. Do not purchase any cryptocurrency claiming association with GitNexus.
 
 <div align="center">
 
@@ -24,7 +26,7 @@
     </a>
   </p>
 
-  <p><strong>The context engine for Enterprise Codebases</strong></p>
+  <p><strong>The nervous system for agent context.</strong></p>
 
   <p>
     Indexes any codebase into a knowledge graph — every dependency, call chain, cluster, and execution flow —
@@ -34,6 +36,7 @@
   <p>
     💬 <a href="https://discord.gg/MgJrmsqr62">Discord</a> ·
     🌐 <a href="https://gitnexus.vercel.app">Web UI</a> ·
+    🏢 <a href="https://akonlabs.com">Enterprise (SaaS & self-hosted)</a>
   </p>
 
 </div>
@@ -71,7 +74,7 @@ That's it. `analyze` indexes the codebase, installs agent skills, registers Clau
 
 > **No C++ toolchain?** Set `GITNEXUS_SKIP_OPTIONAL_GRAMMARS=1` before `npm install -g gitnexus` to skip the vendored grammar materialize/build for `tree-sitter-dart`, `tree-sitter-proto`, `tree-sitter-swift`, and `tree-sitter-kotlin` — those four languages won't be parsed, but install completes in seconds without `python3`/`make`/`g++`. Strict `=1` only — any other value falls through to the rebuild.
 
-> **Local embeddings are opt-in.** Default `npm install` does not fetch `@huggingface/transformers` or `onnxruntime-node`. Run `gitnexus embeddings install` (or `gitnexus analyze --embeddings`, which auto-heals) to fetch the stack through your npm registry config into `~/.gitnexus/embedding-runtime`. CUDA GPU binaries still use NuGet via `--cuda` ([#2370](https://github.com/abhigyanpatwari/GitNexus/issues/2370)). The prefix needs Node with `module.registerHooks` (≥ 22.15 on 22.x, ≥ 23.5 on 23.x). A leftover 1.6.12 package-first tree is residual until a clean reinstall; `--force` only refreshes prefix overrides.
+> **Behind an HTTP proxy / regional firewall?** `onnxruntime-node`'s postinstall downloads optional CUDA binaries from `api.nuget.org` and ignores `HTTP_PROXY`/`HTTPS_PROXY` ([#2370](https://github.com/abhigyanpatwari/GitNexus/issues/2370)). The embedding stack is an optional dependency, so a failed download no longer breaks the install — and it self-heals: the first `gitnexus analyze --embeddings` (or `gitnexus embeddings install`) fetches the stack through your npm registry config (mirrors/proxies apply, no NuGet) into `~/.gitnexus/embedding-runtime` (override with `GITNEXUS_EMBEDDING_RUNTIME_DIR`). The on-demand prefix needs Node with `module.registerHooks` (≥ 22.15 on 22.x, ≥ 23.5 on 23.x); on older Node, keep the stack in the install itself with `ONNXRUNTIME_NODE_INSTALL=skip npm install -g gitnexus` (works on every supported Node).
 
 > **About `tree-sitter-kotlin`:** like Dart/Proto/Swift, Kotlin is a **vendored** grammar (under `gitnexus/vendor/tree-sitter-kotlin`). Upstream ships **source only** (no prebuilt binaries), so GitNexus cross-builds the platform prebuilds itself (via the `build-tree-sitter-prebuilds` GitHub Actions workflow) and vendors them — the same uniform pipeline used for Dart, Proto, and Swift. `node-gyp-build` selects the right `.node` at require time, so **no C/C++ toolchain is needed**. If no prebuild matches your platform-arch, only Kotlin (`.kt`/`.kts`) parsing is unavailable; the rest of `gitnexus` is unaffected.
 
@@ -99,10 +102,6 @@ The proxy strips `Origin` before forwarding, so the server's CSRF guard does not
 
 Indexing is memory-bound. If `gitnexus-server` runs out of memory on a large repo, raise its `plan`, which sets available RAM: `standard` is 2 GB, `pro` is 4 GB. Raise `sizeGB` only if the disk fills with clones and indexes.
 
-### Deploy to RepoCloud
-
-[![Deploy on RepoCloud](https://d16t0pc4846x52.cloudfront.net/deploylobe.svg)](https://repocloud.io/details/gitnexus/)
-
 ## Two Ways to Use GitNexus
 
 |             | **CLI + MCP** (recommended)                                                        | **Web UI**                                                           |
@@ -158,7 +157,7 @@ flowchart TB
 
 ## What Your AI Agent Gets
 
-### 19 MCP tools (17 per-repo + 2 group)
+### 17 MCP tools (15 per-repo + 2 group)
 
 | Tool             | What It Does                                                           |
 | ---------------- | ---------------------------------------------------------------------- |
@@ -177,12 +176,10 @@ flowchart TB
 | `api_impact`     | Pre-change impact report for an API route handler                      |
 | `explain`        | Explain persisted taint findings (source→sink flows, `--pdg` indexes)  |
 | `pdg_query`      | Query control/data dependence at statement level (`--pdg` indexes)     |
-| `read_file`      | Read a checkout file (optional 0-indexed slice; `maxLines` cap)        |
-| `grep`           | Regex search of the working tree for indexed files (1-based hits)      |
 | `group_list`     | List configured repository groups                                      |
 | `group_sync`     | Rebuild a group's Contract Registry and cross-repo links               |
 
-> Per-repo read-only tools take an optional `repo` parameter. Omit it when only one repo is indexed, an MCP default is configured, or the GitNexus process cwd is inside a registered path without crossing into an unindexed nested Git checkout; otherwise pass it explicitly. Mutating tools require `repo` when multiple repos are indexed and no MCP default exists. Per-repo tools also take an optional `branch` for indexes pinned with `gitnexus analyze --branch`, except `read_file` and `grep`, which read the checkout and do not accept `branch`. Omitting `branch` queries the workspace index, which follows your checked-out working tree — switching branches and re-running `gitnexus analyze` updates it incrementally. `explain` and `pdg_query` need an index built with `gitnexus analyze --pdg`.
+> Per-repo read-only tools take an optional `repo` parameter. Omit it when only one repo is indexed, an MCP default is configured, or the GitNexus process cwd is inside a registered path without crossing into an unindexed nested Git checkout; otherwise pass it explicitly. Mutating tools require `repo` when multiple repos are indexed and no MCP default exists. Per-repo tools also take an optional `branch` for indexes pinned with `gitnexus analyze --branch`. Omitting `branch` queries the workspace index, which follows your checked-out working tree — switching branches and re-running `gitnexus analyze` updates it incrementally. `explain` and `pdg_query` need an index built with `gitnexus analyze --pdg`.
 
 ### Resources for instant context
 
@@ -235,13 +232,12 @@ When a repo contains an `.agents/` directory, the standard and generated skills
 | **Cursor**               | Yes | Yes    | Yes (postToolUse, [manual install](gitnexus-cursor-integration/README.md#hook-install))                           | **Full**     |
 | **Antigravity** (Google) | Yes | Yes    | Yes (AfterTool, [Gemini CLI hooks schema](https://geminicli.com/docs/hooks/reference/))[¹](#fn-antigravity-hooks) | **Full**     |
 | **Codex**                | Yes | Yes    | Yes (PreToolUse + PostToolUse, [Codex hooks](https://developers.openai.com/codex/hooks))                          | **Full**     |
-| **Factory** (Droid)      | Yes | Yes    | Yes (PostToolUse, [plugin](gitnexus-factory-plugin/))                                                             | **Full**     |
 | **OpenCode**             | Yes | Yes    | —                                                                                                                 | MCP + Skills |
 | **CodeBuddy** (Tencent)  | Yes | Yes    | —                                                                                                                 | MCP + Skills |
 | **Qoder** (Alibaba)      | Yes | Yes    | —                                                                                                                 | MCP + Skills |
 | **Windsurf**             | Yes | —      | —                                                                                                                 | MCP          |
 
-> **Full** means MCP tools + agent skills + hooks that enrich searches with graph context. **Claude Code** and **Codex** go deepest: their PreToolUse hooks enrich the search before it runs, and their PostToolUse hooks also detect a stale index after commits and prompt the agent to reindex. **Cursor**, **Antigravity**, and **Factory** augment from a post-tool hook only, so they enrich the result rather than the query and do not carry the stale-index hint.
+> **Claude Code** and **Codex** get the deepest integration: MCP tools + agent skills + PreToolUse hooks that enrich searches with graph context + PostToolUse hooks that detect a stale index after commits and prompt the agent to reindex.
 
 <a id="fn-antigravity-hooks"></a>
 
@@ -285,21 +281,6 @@ codex plugin marketplace add abhigyanpatwari/GitNexus
 
 > **Codex notes:** SessionStart is intentionally not registered — Codex reads [AGENTS.md natively](https://developers.openai.com/codex/guides/agents-md), which already carries the GitNexus context block. Newly installed hooks need a one-time approval in Codex via `/hooks` before they run. Pick **one** install route (`gitnexus setup -c codex` **or** the plugin): plugin hooks load alongside `~/.codex/hooks.json`, so installing both can fire duplicate hooks per tool call.
 
-**Factory** (Droid) — MCP + skills via `gitnexus setup -c droid`, or add the server manually to `~/.factory/mcp.json` ([user scope](https://docs.factory.ai/cli/configuration/mcp), applies to all projects):
-
-```json
-{
-  "mcpServers": {
-    "gitnexus": {
-      "command": "npx",
-      "args": ["-y", "gitnexus@latest", "mcp"]
-    }
-  }
-}
-```
-
-`gitnexus setup -c droid` also installs skills to `~/.factory/skills/`. For the PostToolUse search-augment hook, install the bundled [`gitnexus-factory-plugin/`](gitnexus-factory-plugin/) — from a marketplace that includes this repo, run `droid plugin install gitnexus@<marketplace>`, or point Droid at it via `extraKnownMarketplaces` in `.factory/settings.json`. Factory reads [`AGENTS.md` natively](https://docs.factory.ai/), which already carries the GitNexus context block.
-
 **Cursor** (`~/.cursor/mcp.json` — global, works for all projects):
 
 ```json
@@ -424,8 +405,7 @@ backoff. Invalid `.gitnexusrc` or ignore-file reloads pause ordinary refreshes
 until the control file is fixed. Stop the watcher with Ctrl+C.
 
 Watch mode accepts `--debounce`, `--workers`, `--worker-timeout`,
-`--max-file-size`, `--max-processes`, `--max-process-branching`,
+`--max-file-size`, `--branch`, `--pdg`, `--name`, `--allow-duplicate-name`, and
-`--max-process-trace-depth`, `--max-entry-point-candidates`, `--branch`, `--pdg`, `--skip-fts`, `--name`, `--allow-duplicate-name`, and
 `--verbose`. Explicit one-shot options such as `--force`, `--repair-fts`,
 embedding flags, `--skills`, `--self-commit`, `--index-only`, and `--skip-git`
 are rejected. Unsupported defaults from `.gitnexusrc` are ignored with a
@@ -459,7 +439,6 @@ The token may be set in the shell, `.env.local`, or `.env` in the working direct
 gitnexus analyze --force         # Full graph + FTS rebuild (reuses unchanged parser output)
 gitnexus analyze --no-parse-cache # Full rebuild that re-parses every source file
 gitnexus analyze --repair-fts    # Fast path: rebuild/verify only FTS indexes on existing index data
-gitnexus analyze --skip-fts      # Index graph/embeddings without loading FTS or building keyword indexes
 gitnexus analyze --skills        # Generate repo-specific skill files from detected communities
 gitnexus analyze --skip-embeddings  # Skip embedding generation (faster)
 gitnexus analyze --embeddings [limit]  # Enable embedding generation (slower, better search)
@@ -471,17 +450,12 @@ gitnexus analyze --verbose       # Log skipped files when parsers are unavailabl
 gitnexus analyze --worker-timeout 60  # Increase worker idle timeout for slow parses
 gitnexus analyze --workers <n>   # Parse worker pool size (>=1; default: cores-1, capped at 16,
                                  # auto-sized to the repo). 0 is rejected — there is no sequential mode.
-gitnexus analyze --max-processes <n>  # Process-detection process cap (replaces dynamic max(20, round(symbols/10)))
-gitnexus analyze --max-entry-point-candidates <n>  # Ranked entry-point pool (default 200; raise when the warning names it)
 gitnexus analyze --spring-actuator ./actuator  # Enrich with local Spring Boot Actuator JSON snapshots
 gitnexus analyze --asyncapi-spec ./docs/asyncapi  # Resolve broker addresses from AsyncAPI 3.x documents
-gitnexus analyze --memory-budget 3000  # Main-thread V8 heap in MB (>= 200); overrides the auto-sizer and --max-old-space-size
 gitnexus analyze --wal-checkpoint-threshold 67108864  # LadybugDB WAL auto-checkpoint threshold in bytes
                                  # (default 67108864 = 64 MiB; -1 keeps Ladybug stock ~16 MiB)
 ```
 
-`--skip-fts` (or `GITNEXUS_SKIP_FTS=1`) disables FTS extension loading and keyword-index construction for this analysis. Graph queries, communities, processes, and existing embeddings remain available. Status and search report "FTS disabled for this index". Remove both the flag and environment setting and run `analyze` again to restore keyword search, even at the same commit. Only the exact environment value `1` enables the opt-out; the flag takes precedence. It cannot be combined with `--repair-fts`. Disabling an existing FTS index may require one graph-store rebuild to avoid unsafe writes through native indexes.
-
 `--spring-actuator` is explicitly opt-in and accepts either a JSON bundle keyed by `mappings`, `beans`, `conditions`, `configprops`, and/or `env`, or a directory containing endpoint-named JSON files. It confirms matching static nodes and adds conservative runtime-only routes, beans, and property keys. The configured input is excluded from source scanning; only normalized repository-relative exclusions are retained for future scans, never absolute paths. Env/configprops values, origins, condition messages, and source names are never persisted or printed. Because snapshots are external runtime state, an enabled run always rebuilds; the first later run without the option rebuilds once to remove runtime evidence. The same path can be set as `springActuator` in `.gitnexusrc`.
 
 `--asyncapi-spec` is explicitly opt-in and accepts a directory of AsyncAPI documents or a single document; the path is resolved against the repository root, so a committed `docs/asyncapi` and an absolute cache written by something else both work. Each `operations[]` entry of an **AsyncAPI 3.x** document can contribute a `Destination` node keyed by broker and address, with `action: send` emitting `PUBLISHES_TO` and `action: receive` emitting `CONSUMES_FROM`, so a document and source code that name one address on one broker land on the same node. Edges start at the document, not at a callable — a document states that the service talks to an address, not which method does — and no address a document names is ever attached to an unresolved source site.
@@ -526,25 +500,18 @@ gitnexus auto-sync reset             # Clear failure state; leaves clones and in
 ```yaml
 sync_interval_minutes: 10
 analyze_timeout: 5m
-# Extra hosts beyond github.com, gitlab.com, and gitee.com. Exact names only.
-# allowed_hosts: [gitlab.mycompany.com]
 projects:
   - local_path: /absolute/path/to/clones
     branches: [main, master]
-    # pdg: omit = preserve live index mode; true = keep PDG current;
-    # false = init default (warns, then strips PDG on the next successful rebuild).
-    # Do not paste pdg: false onto an existing watch file unless you intend to drop PDG.
-    pdg: false
     overwrite_local_changes: false
     remote_urls:
       - git@github.com:owner/repo.git
 ```
 
 - `sync_interval_minutes` must be at least `5`; `local_path` must be an absolute path. Clones are stored below it as `host/namespace/repo`.
-- Remote URLs may use SSH SCP or HTTPS. Hosts are github.com, gitlab.com, and gitee.com unless listed in top-level `allowed_hosts` (exact DNS names, no wildcards). The CLI image includes OpenSSH; mount keys yourself. Invalid `watch_config.yml` skips auto-sync immediately. Auto-sync honors `.gitnexusrc` embeddings (HTTP embeddings env still required in the image).
+- Remote URLs must use SSH SCP form and are limited to GitHub, GitLab, or Gitee.
 - `branches` are tried in order. The legacy `branch` field is supported, but do not set both.
-- Set per-project `pdg: true` to keep the full control-flow, control/data-dependence, and taint layers current. Untouched configs that omit `pdg` preserve an existing index's mode and cannot silently strip PDG data. Do not paste `pdg: false` from this example onto an existing watch file unless you intend to drop PDG; an explicit `false` opt-out logs a warning before removing existing PDG data. Auto-sync requests atomic incremental publication where supported, so readers keep using the previous graph until a successful update is ready and a failed staged analysis leaves it intact; unsupported paths retain the analyzer's existing in-place behavior.
+- Analysis runs in an isolated worker; `analyze_timeout` defaults to, and cannot exceed, half of `sync_interval_minutes`. Timeout and `auto-sync stop` request safe cancellation; a worker in native work exits after reaching a JS-visible safe point. Until then, auto-sync reports `cancelling` or `stopping` and retains ownership so another auto-sync cannot take over, for up to 5 seconds — after that the parent stops waiting and leaves the worker to exit on its own rather than killing it mid-write. This behavior is the same on macOS and Windows. `overwrite_local_changes` defaults to `false`, so a dirty local clone is skipped rather than overwritten; setting it to `true` also deletes untracked files in the clone, while keeping ignored paths.
-- Analysis runs in an isolated worker; `analyze_timeout` defaults to half of `sync_interval_minutes`, but may be longer (for example, a `30m` analysis timeout with `5` minute polling) up to Node's timer limit. If a polling tick arrives while analysis is active, it is coalesced into one immediate follow-up run using the newest commit. If the parent times out and leaves that worker running, the follow-up is deferred to the next interval so a leftover lock holder is not counted as a hard analyze failure. Timeout and `auto-sync stop` request safe cancellation; a worker in native work exits after reaching a JS-visible safe point. Until then, auto-sync reports `cancelling` or `stopping` and retains ownership so another auto-sync cannot take over, for up to 5 seconds — after that the parent stops waiting and leaves the worker to exit on its own rather than killing it mid-write. This behavior is the same on macOS and Windows. `overwrite_local_changes` defaults to `false`, so a dirty local clone is skipped rather than overwritten; setting it to `true` also deletes untracked files in the clone, while keeping ignored paths.
 - Add `group_name` only after creating that group with `gitnexus group create <name>`. Partial clone output is isolated and removed after 14 days.
 
 See the [full auto-sync configuration and runtime reference](gitnexus/README.md#gitnexus-auto-sync) for concurrency, timeouts, failure thresholds, and runtime files.
@@ -599,25 +566,21 @@ Notes:
 
 - The default branch is resolved as: `--default-branch` > `.gitnexusrc` `defaultBranch`/`branch` > auto-detected `origin/HEAD` > `main`.
 - `skipContextFiles` / `skipAiContext` are aliases for `skipAgentsMd` — they skip the `AGENTS.md` / `CLAUDE.md` block only. They do **not** imply `skipSkills`. `indexOnly` is the stronger option that skips all file injection.
-- Supported keys: `defaultBranch` (`branch`), `skipAgentsMd` (`skipContextFiles`, `skipAiContext`), `skipSkills`, `indexOnly`, `stats`/`noStats`, `embeddings`, `dropEmbeddings`, `name`, `allowDuplicateName`, `maxFileSize`, `workerTimeout`, `walCheckpointThreshold`, `workers`, `maxProcesses`, `maxProcessBranching`, `maxProcessTraceDepth`, `maxEntryPointCandidates`, `springActuator`, `embeddingThreads`, `embeddingBatchSize`, `embeddingSubBatchSize`, `embeddingDevice`.
+- Supported keys: `defaultBranch` (`branch`), `skipAgentsMd` (`skipContextFiles`, `skipAiContext`), `skipSkills`, `indexOnly`, `stats`/`noStats`, `embeddings`, `dropEmbeddings`, `name`, `allowDuplicateName`, `maxFileSize`, `workerTimeout`, `walCheckpointThreshold`, `workers`, `springActuator`, `embeddingThreads`, `embeddingBatchSize`, `embeddingSubBatchSize`, `embeddingDevice`.
-- The file is JSON only. Unknown keys and wrong JSON types fail fast with an actionable error before analysis starts. Process-detection knobs (`maxProcesses`, `maxProcessBranching`, `maxProcessTraceDepth`, `maxEntryPointCandidates`) that are not a positive integer warn and fall through to env, then the built-in default.
+- The file is JSON only. Unknown keys and invalid values fail fast with an actionable error before analysis starts.
 
 </details>
 
 <details>
 <summary><strong>Environment variables</strong></summary>
 
-Most `analyze` knobs are also CLI flags (`--workers`, `--worker-timeout`, `--max-file-size`, `--verbose`). Use the env-var form when you'd otherwise repeat the same flag every run, or when invoking GitNexus from a long-running host (MCP server, eval-server, CI shell) that already manages its own environment. CLI flags take precedence over `.gitnexusrc`, which takes precedence over env vars, which take precedence over built-in defaults.
+Most `analyze` knobs are also CLI flags (`--workers`, `--worker-timeout`, `--max-file-size`, `--verbose`). Use the env-var form when you'd otherwise repeat the same flag every run, or when invoking GitNexus from a long-running host (MCP server, eval-server, CI shell) that already manages its own environment. CLI flags take precedence over env vars; env vars take precedence over built-in defaults.
 
 | Variable                                        | Default                            | Effect                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   | Tune when…                                                                                                                                                                            |
 | ----------------------------------------------- | ---------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| `GITNEXUS_TEST_REPORT` | unset | Write a JSON execution receipt from `scripts/run-cross-platform.ts`; accepts a basename ending in `.json`. | CI platform shards use distinct filenames so the completeness check can require every report. |
 | `GITNEXUS_WORKER_POOL_SIZE`                     | `cores - 1`, capped at 16          | Parse worker pool size (must be ≥ 1). Equivalent to `--workers <n>`. The worker pool is the sole parse path — there is no sequential parser, so `0` is rejected with an actionable error (the pool self-heals via quarantine + respawn).                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 | Constrained containers (cgroup CPU limits) or CI runners with explicit quotas. To narrow down a worker crash set `1` for a single-worker pool — not `0`.                              |
 | `GITNEXUS_PARSE_CHUNK_CONCURRENCY`              | `2`                                | Number of chunks whose file contents may be read into memory in parallel while the pool dispatches the current chunk. Worker dispatch itself stays serial.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               | Repos large enough to chunk (multi-MB total source) where disk I/O is a measurable fraction of analyze wall-clock.                                                                    |
 | `GITNEXUS_VERBOSE`                              | unset                              | When `1`, enables verbose ingestion logs (skipped-file warnings, per-chunk throughput, parse-cache stats). Equivalent to `--verbose`.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    | Debugging an analyze that "completed" but seems to have missed files; tuning `--workers` / chunk concurrency against observable throughput.                                           |
-| `GITNEXUS_EMBEDDING_RETRY_TIMEOUTS`             | unset                              | When truthy (`1`/`true`/`yes`), per-attempt HTTP embedding timeouts (`TimeoutError` on fetch or body read) go through the bounded `GITNEXUS_EMBEDDING_MAX_ATTEMPTS` retry loop instead of failing the job. Any other value leaves it off, so cloud/default timeouts remain terminal.                                                                                                                                                                                                                                                                                                                                                      | Local accelerators that drop a device lock when the client disconnects and succeed on the next request (observed with FastFlowLM on Ryzen AI).                                        |
-| `GITNEXUS_EMBEDDING_SIDECAR_TIMEOUT_MS`         | `180000` (3 minutes)               | Per-request IPC timeout for local embedding sidecar embed batches. On overrun the parent SIGKILLs the sidecar child and rejects the batch. Init still uses the HF download budget (`HF_DOWNLOAD_TIMEOUT_MS` × attempts), not this knob.                                                                                                                                                                                                                                                                                                                                  | Large embed batches or slow local ONNX inference cause sidecar request timeouts during `analyze --embeddings`, `embeddings sync`, serve, or MCP.                                      |
-| `GITNEXUS_VECTOR_MAX_DISTANCE` | `0.6` for CLI/MCP `query`; `0.5` for standalone semantic search | Maximum cosine distance for vector hits, applied to both indexed search and exact-scan fallback. Hits must have `distance < cutoff`. Unset/blank uses the default; non-numeric, non-finite, zero, or negative values warn and use the default; finite values above `2` warn and clamp to `2`. | Paraphrased queries have weak semantic recall with your embedding model. See [Vector cutoff tuning](#vector-cutoff-tuning). |
 | `GITNEXUS_ANALYZER_IDENTITY_IN_PROCESS_GUARDS`  | unset                              | When truthy (`1`/`true`/`yes`), forces in-process cache-guard validation once a batch has ≥128 requests. In-process mode also auto-selects when `packageRoot`/`buildRoot` fail `W_OK` with `EACCES`/`EROFS`. Otherwise those large batches use a Node subprocess probe. Batches under 128 always stay in-process.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        | Trusted or read-only installs where two identity subprocess spawns per analyze dominate wall time; leave unset to keep the default isolation path on writable trees. |
 | `GITNEXUS_RESOLVE_DEF_GRAPH_ID_MEMO`            | on (unset)                         | Memoizes `resolveDefGraphId` per `nodeLookup` instance (WeakMap). Enabled by default. Set to `0`/`false`/`off`/`no` to disable and recompute on every call (debug / bisect memo bugs).                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   | Suspecting stale graph-id resolution after a lookup rebuild, or comparing memo vs uncached cost on a large index. |
 | `GITNEXUS_AUTH_TOKEN`                           | unset                              | Bearer token required when `eval-server` binds beyond loopback. May also be read from `.env.local` or `.env`; shell values take precedence.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              | Exposing the evaluation HTTP tools to a container, VM, or LAN.                                                                                                                        |
@@ -626,18 +589,9 @@ Most `analyze` knobs are also CLI flags (`--workers`, `--worker-timeout`, `--max
 | `GITNEXUS_PROFILE_DEFERRED_SLOW_MS`             | `3000` (verbose) / `5000`          | Per-file threshold in ms above which `processCallsFromExtracted` emits a `slow file …` log line. Parsed via `Number()`: accepts integers (`5000`), scientific notation (`2.5e3`), decimals (`.5`), and hex (`0x10`). Non-finite or non-positive values fall back to the default.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         | Hunting a few outlier files dominating the deferred call-resolution stage; lower to surface more, raise to focus only on the worst.                                                   |
 | `PROF_LBUG_LOAD`                                | unset                              | When `1`, emits one `[lbug-load prof]` summary line per `loadGraphToLbug` call breaking the graph-DB persistence wall into stages (`csv-emit` / `copy-nodes` / `copy-rels` / `fallback` / `total`) plus node & edge counts. Zero-cost when unset.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        | Attributing large-repo analyze wall time across CSV generation vs. LadybugDB `COPY` (issue #2203) — the analyze "emit" timing is the scope-resolution bucket, not this DB-write path. |
 | `GITNEXUS_MAX_FILE_SIZE`                        | `512` (KB)                         | Walker skip threshold in KB. Hard cap is `32768` (tree-sitter buffer ceiling). Equivalent to `--max-file-size <kb>`.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     | Indexing repos with intentionally-large source files (generated parsers, vendored bundles) that should still be parsed.                                                               |
-| `GITNEXUS_MAX_PROCESSES`                        | dynamic (`max(20, round(symbols/10))`) | Analyze-time process-detection process cap. Equivalent to `--max-processes <n>` / `.gitnexusrc` `maxProcesses`. Explicit values replace the dynamic formula (not a multiplier). `0` is invalid, not unlimited. Changing this re-detects flows on the next analyze without `--force`. Distinct from query-time `IMPACT_MAX_CHUNKS`. | `[processes] … whole flows are MISSING` names `--max-processes` after entry points were never traced or flows were dropped. Tracing does not start the next entry once collected traces already reach `maxProcesses * 2`; a started entry can still emit every trace that entry produces. |
-| `GITNEXUS_MAX_PROCESS_BRANCHING`                | `4`                                | Analyze-time per-node branching cap during flow tracing. Equivalent to `--max-process-branching <n>`. Shape-only: raising it shortens fewer traces; it does not restore whole missing flows. | A flow is present but `calleesDropped` is high at debug. |
-| `GITNEXUS_MAX_PROCESS_TRACE_DEPTH`              | `10`                               | Analyze-time DFS depth cap during flow tracing. Equivalent to `--max-process-trace-depth <n>`. Shape-only. | A reported flow is shorter than the code path (`tracesDepthCapped` at debug). |
-| `GITNEXUS_MAX_ENTRY_POINT_CANDIDATES`           | `200`                              | Ranked entry-point candidate pool. Equivalent to `--max-entry-point-candidates <n>`. Raising `--max-processes` alone does not clear `entryPointCandidatesDropped`. Doubling the current cap is the usual first raise; setting it to the full remaining candidate count can exhaust CPU and memory. | The `[processes]` warning reports candidate entry points that never ranked in. |
 | `GITNEXUS_WORKER_SUB_BATCH_TIMEOUT_MS`          | `30000`                            | Worker idle timeout in milliseconds before retry/fallback. Equivalent to `--worker-timeout <seconds>` × 1000.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            | Slow-parsing files (large minified JS, deeply-nested TS types) that legitimately need more than 30s.                                                                                  |
 | `GITNEXUS_WORKER_READY_TIMEOUT_MS`              | `5000`                             | Startup budget in milliseconds for a parse worker to load its grammar bindings and report `{type:'ready'}`. Slots that miss it are treated as startup crashes.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           | Slow or heavily loaded hosts where a full pool cold-starting concurrently needs more than 5s, and analyze aborts with "did not report ready within 5000ms".                           |
 | `GITNEXUS_FTS_STEMMER`                          | `porter`                           | Stemmer used when rebuilding BM25/FTS indexes. Use `none` for CJK-heavy repositories, or a language stemmer such as `german`, `french`, or `spanish` for matching repository comments. Re-run `gitnexus analyze --repair-fts` after changing it.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         | Keyword search quality is poor for non-English comments or identifiers under English stemming.                                                                                        |
-| `GITNEXUS_STORAGE_PATH`                         | unset (`<repo>/.gitnexus/`)        | Complete external index directory. This preserves the existing configuration semantics and takes precedence over `GITNEXUS_STORAGE_ROOT` when both are set.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              | You already keep one repository index outside its checkout or need one explicit index location.                                                                                       |
-| `GITNEXUS_STORAGE_ROOT`                         | unset                              | Absolute root directory for external indexes. GitNexus creates an isolated `<repo-basename>-<canonical-path-hash>/` slot beneath it for each repository, then registers the resolved slot so `status`, MCP, and `serve` can reopen it later.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             | You want to manage multiple repository indexes centrally or keep generated data outside source checkouts.                                                                             |
-| `GITNEXUS_SHARED_STORE`                         | unset (on)                         | Set to `off` (or `0`, `false`, `no`) to turn off shared index stores for both linked git worktrees and sibling clones; every checkout then indexes into its own `.gitnexus/`. Sharing is also off whenever `GITNEXUS_STORAGE_PATH` or `GITNEXUS_STORAGE_ROOT` is set.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   | Disk or memory is not a concern, or you want each worktree's index fully independent.                                                                                                 |
-| `GITNEXUS_CONTENT_RETENTION`                    | `full`                             | Source-text retention profile: `full` keeps file and symbol text, `symbol` keeps symbol snippets without full file content, and `none` keeps the structural graph without source body text.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  | You need to reduce persisted source text while preserving graph structure.                                                                                                            |
-| `GITNEXUS_SKIP_FTS` | unset | When exactly `1`, skips FTS extension loading and keyword index creation during analyze. Equivalent to `--skip-fts`; a later analyze without either option restores FTS. | Graph-only consumers with their own retrieval, or short-lived indexes that do not need keyword search. |
 | `GITNEXUS_WAL_CHECKPOINT_THRESHOLD`             | `67108864` (64 MiB)                | LadybugDB WAL auto-checkpoint threshold in bytes. Equivalent to `--wal-checkpoint-threshold <bytes>`. `-1` keeps LadybugDB's stock threshold (~16 MiB). Larger thresholds reduce checkpoint frequency but increase the WAL size at rotation time — choose a smaller value on disk-constrained environments.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              | You need a larger or smaller WAL auto-checkpoint threshold for your analyze workload.                                                                                                 |
 | `GITNEXUS_LBUG_BUFFER_POOL_SIZE`                | min(2 GiB, 80% RAM)                | LadybugDB buffer-pool ceiling in bytes for every GitNexus database (analyze, MCP server, serve, group bridges). `0` restores LadybugDB's native unbounded default of 80% of system RAM; invalid values warn and fall back to the default (#2557). During `analyze` the pool is right-sized to the graph, scaled on non-4 KiB-page hosts by the page-size granule ratio up to min(2 GiB × pageSize/4 KiB, 80% RAM) (#2631); this env var overrides all of that as an absolute value.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      | A long-lived `gitnexus mcp` or a big incremental `analyze` uses too much memory, or a huge repo's working set genuinely needs a pool larger than 2 GiB.                               |
 | `GITNEXUS_LBUG_MAX_DB_SIZE`                     | `17179869184` (16 GiB)             | Maximum size in bytes of a single LadybugDB database file — an mmap/disk-address-space ceiling, not a memory limit (it does not constrain the buffer pool). Invalid values silently fall back to the default.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            | Indexing a genuinely huge monorepo whose on-disk graph index approaches 16 GiB.                                                                                                       |
@@ -657,22 +611,6 @@ Most `analyze` knobs are also CLI flags (`--workers`, `--worker-timeout`, `--max
 | `GITNEXUS_PUBLIC_ORIGIN`                        | unset                              | The single browser origin `serve` is reached through, added to the CORS allowlist and to the write-route origin guard. A wildcard bind (`0.0.0.0`) has no host identity, so without this the server's own UI is refused. **Setting it currently refuses to start:** `serve` has no authentication, requests carrying no `Origin` header already reach `POST /api/analyze` and `DELETE /api/repo`, and this is the setting that would admit browser writes on top of that. Matching rules for when the gate lifts: the hostname must match exactly, and so must the scheme. A value with no scheme (`app.example.com`) means `https`, since a bare host comes from platform service discovery and those terminate TLS; spell out `http://app.example.com` for plain HTTP. An explicit port must match; with no port, any port on that hostname is accepted. Anything that is not one reachable host (a list, `*`, a bare port number, a `:0` port, a trailing dot) warns at startup and allows nothing.                   | `gitnexus serve` runs behind a reverse proxy or on a wildcard bind, and the UI's index/delete requests return `origin_not_allowed`.                                                   |
 | `GITNEXUS_TRUST_PROXY`                          | `loopback, linklocal, uniquelocal` | Express `trust proxy` value — which upstream hops may set `X-Forwarded-*`, and so what the per-IP rate limiter reads as the client IP. Set it to the exact number of proxies you control. Every hop past that is one more entry of the chain the caller gets to write. `false`/`no`/`off` (and a `0` hop count) trust no hop; a proxy list Express can compile (`loopback`, `10.0.0.0/8, 127.0.0.1`) names them instead. `true`/`yes`/`on` is **rejected**: it reads the client-controlled leftmost `X-Forwarded-For` entry, so a spoofed chain earns a fresh rate-limit key per request, and express-rate-limit rejects it too (`ERR_ERL_PERMISSIVE_TRUST_PROXY`). Counts above `16` are rejected as well, as a sanity ceiling rather than a safety boundary. Any invalid value warns and falls back to the default. Bind non-loopback with this unset and `serve` warns: a load balancer outside the private ranges is untrusted, so every request keys to the balancer and the per-IP limit becomes one shared limit. | `serve` sits behind a load balancer outside the private ranges (AWS ALB, Cloudflare, CGNAT), where every request otherwise collapses to the proxy hop and rate limiting goes global.  |
 
-### Vector cutoff tuning
-
-`GITNEXUS_VECTOR_MAX_DISTANCE` controls which vector candidates enter hybrid search. Cosine distance is lower for more similar vectors; the appropriate cutoff depends on the embedding model and repository. A higher cutoff can recover useful semantic matches, but can also admit less relevant hits and change the fused ranking. The default is `0.6` for CLI and MCP `query`, and `0.5` for the standalone semantic-search function.
-
-For an index that already has embeddings, compare the default with `0.8` and `1.0` on representative queries, including paraphrases, exact symbol names, and queries that should have no relevant result:
-
-```bash
-GITNEXUS_VECTOR_MAX_DISTANCE=0.8 gitnexus query "how are expired sessions removed" --limit 10
-```
-
-Check both whether the expected symbols appear and where they rank. In [#3457](https://github.com/abhigyanpatwari/GitNexus/issues/3457), the reporter's `voyage-code-4` sample found 20 of 32 paraphrased targets at `0.6`, 25 at `0.8`, and 28 at `1.0`. Those values are a tuning starting point for that model, not universal recommendations: one paraphrased target fell from first to twelfth at `1.0`. The sample covered eight repositories, recorded no raw distances, used agent-written queries after reading the code, and did not send Voyage's `input_type`.
-
-Set the variable in the process that runs the search. For MCP, add it to the server's launch environment and restart the server; for `gitnexus serve`, set it in that process's environment and restart it. Setting it only during `analyze`, or exporting it in another shell, does not configure an existing server. Changing the cutoff requires no reindex.
-
-The cutoff only filters available vector candidates. It cannot add missing embeddings or repair a mismatch between the indexing and query-time embedding configuration; see the [embeddings runbook](RUNBOOK.md#embeddings). A cutoff of `2` is very permissive, but still rejects distance exactly `2` and retains the search candidate limits.
-
 </details>
 
 <details>
@@ -713,13 +651,12 @@ GitNexus builds a complete knowledge graph of your codebase through a multi-phas
 | Kotlin     | ✓       | ✓              | ✓       | ✓        | ✓                | ✓                     | —      | ✓          | ✓            |
 | C#         | ✓       | ✓              | ✓       | ✓        | ✓                | ✓                     | ✓      | ✓          | ✓            |
 | Go         | ✓       | —              | ✓       | ✓        | ✓                | ✓                     | ✓      | ✓          | ✓            |
-| Rust       | ✓       | ✓              | ✓       | ✓        | ✓                | ✓                     | ✓      | ✓          | ✓            |
+| Rust       | ✓       | ✓              | ✓       | ✓        | ✓                | ✓                     | —      | ✓          | ✓            |
 | PHP        | ✓       | ✓              | ✓       | —        | ✓                | ✓                     | ✓      | ✓          | ✓            |
 | Ruby       | ✓       | —              | ✓       | ✓        | —                | ✓                     | —      | ✓          | ✓            |
 | Swift      | —       | —              | ✓       | ✓        | ✓                | ✓                     | ✓      | ✓          | ✓            |
 | C          | —       | —              | ✓       | —        | ✓                | ✓                     | —      | ✓          | ✓            |
 | C++        | —       | —              | ✓       | ✓        | ✓                | ✓                     | —      | ✓          | ✓            |
-| Objective-C | ✓      | —              | ✓       | ✓        | ✓                | —                     | —      | —          | —            |
 | Dart       | ✓       | —              | ✓       | ✓        | ✓                | ✓                     | —      | ✓          | ✓            |
 | Zig        | ✓       | —              | ✓       | —        | ✓                | ✓                     | ✓      | —          | ✓            |
 
@@ -731,9 +668,7 @@ GitNexus builds a complete knowledge graph of your codebase through a multi-phas
 
 GitNexus uses a **global registry** so one MCP server can serve multiple indexed repos. No per-project MCP config needed — set it up once and it works everywhere.
 
-Each `gitnexus analyze` stores the index in `.gitnexus/` inside the repo by default (portable, gitignored). `GITNEXUS_STORAGE_PATH` selects one complete external index directory and preserves the established configuration behavior. To manage multiple repositories under one external directory, set `GITNEXUS_STORAGE_ROOT`; GitNexus derives an isolated `<repo-basename>-<canonical-path-hash>/` slot beneath it for each repository. If both variables are set, `GITNEXUS_STORAGE_PATH` takes precedence. GitNexus registers the resolved slot in `~/.gitnexus/registry.json`, allowing later `status`, MCP, and `serve` commands to reopen the index without repeating the environment variable. LadybugDB connections are opened lazily on first query and evicted after 5 minutes of inactivity (max 5 concurrent). Read-only tools can omit `repo` when only one repo is indexed, an MCP default is configured, or the GitNexus process cwd is inside a registered path without crossing into an unindexed nested Git checkout. Outside those paths—and for mutating tools with multiple indexed repos and no MCP default—pass `repo` explicitly.
+Each `gitnexus analyze` stores the index in `.gitnexus/` inside the repo (portable, gitignored) and registers a pointer in `~/.gitnexus/registry.json`. When an AI agent starts, the MCP server reads the registry and can serve any indexed repo. LadybugDB connections are opened lazily on first query and evicted after 5 minutes of inactivity (max 5 concurrent). Read-only tools can omit `repo` when only one repo is indexed, an MCP default is configured, or the GitNexus process cwd is inside a registered path without crossing into an unindexed nested Git checkout. Outside those paths—and for mutating tools with multiple indexed repos and no MCP default—pass `repo` explicitly.
-
-**Worktrees share one index store.** When a repository has linked worktrees (`git worktree add`), the main checkout and every worktree index into one store at `~/.gitnexus/stores/<repo>/` instead of each keeping a full `.gitnexus/`. Checkouts at the same commit with no local changes read one shared, read-only graph: one copy on disk and one open database in MCP. A checkout with uncommitted changes gets its own graph, copied from the nearest shared graph and updated incrementally rather than rebuilt. Parse caches are shared too. Each worktree keeps a small `.gitnexus/store.json` pointer, and an index it had before sharing is left in place; `gitnexus status` reports it and `gitnexus clean --local-index --force` removes it. `gitnexus clean` in one worktree removes only that worktree's slot and any shared graph no other checkout uses; `gitnexus clean --gc` also drops slots whose worktree was deleted. Independent clones of one repository share too: when another registered clone has the same `origin` URL, `gitnexus analyze` in a clone joins that clone's store (or starts one the other clone joins on its next analyze). A lone clone keeps its own `.gitnexus/`. `gitnexus analyze --share-with <name-or-path>` joins a specific checkout's store after checking the `origin` URLs match, and `--no-share` moves a clone back to its own `.gitnexus/` and keeps it out until `--share-with`. On filesystems with copy-on-write clones (APFS, btrfs, XFS) a checkout's private graph shares its unchanged pages with the shared graph on disk; elsewhere it is a full copy, and `gitnexus status` says which. Queries cannot combine two graphs, because LadybugDB reads one database per query, so a checkout with edits always has a complete graph of its own. Set `GITNEXUS_SHARED_STORE=off` (or `0`, `false`, `no`) to turn sharing off for worktrees and clones alike.
 
 <details>
 <summary><strong>Architecture diagram</strong></summary>
@@ -950,10 +885,9 @@ The web UI uses the same indexing pipeline as the CLI but runs entirely in WebAs
 
 ```bash
 git clone https://github.com/abhigyanpatwari/gitnexus.git
-cd gitnexus/gitnexus-web && npm install
+cd gitnexus/gitnexus-shared && npm install && npm run build
-# Compile sibling gitnexus-shared with this package's TypeScript 7 (do not npm ci shared).
+cd ../gitnexus-web && npm install
-cd ../gitnexus-shared && node ../gitnexus-web/node_modules/typescript/lib/tsc.js
+npm run dev
-cd ../gitnexus-web && npm run dev
 # Then in another terminal, start the backend the frontend connects to:
 npx gitnexus@latest serve
 ```
@@ -1013,7 +947,7 @@ docker compose --env-file .env up -d
 Files:
 
 - [Dockerfile.web](Dockerfile.web) — builds `gitnexus-shared` and `gitnexus-web`, then serves the production frontend.
-- [Dockerfile.cli](Dockerfile.cli) — builds the CLI/server (with its native deps) and runs `gitnexus serve --host 0.0.0.0`. Local embeddings are **not** in the image (`onnxruntime-node` is opt-in; runtime npm is stripped). Bind-mount a prefix or set `GITNEXUS_EMBEDDING_URL`.
+- [Dockerfile.cli](Dockerfile.cli) — builds the CLI/server (with its native deps) and runs `gitnexus serve --host 0.0.0.0`.
 - [docker-compose.yaml](docker-compose.yaml) — starts both signed images side by side.
 - [.env.example](.env.example) — overrides for image names, container names, ports, and the workspace mount.
 
@@ -1153,7 +1087,7 @@ Built by the community — not officially maintained, but worth checking out.
 
 ## Security & Privacy
 
-- **CLI**: everything runs locally on your machine. No network calls. Indexes are stored in `.gitnexus/` by default (gitignored), in the complete external directory selected by `GITNEXUS_STORAGE_PATH`, or in repository-specific slots beneath `GITNEXUS_STORAGE_ROOT`. Global registry at `~/.gitnexus/` stores only paths and metadata.
+- **CLI**: everything runs locally on your machine. No network calls. Index stored in `.gitnexus/` (gitignored). Global registry at `~/.gitnexus/` stores only paths and metadata.
 - **Web**: everything runs in your browser. No code uploaded to any server. API keys stored in localStorage only.
 - Open source — audit the code yourself.
 

RUNBOOK.md

@@ -87,14 +87,6 @@ npx gitnexus analyze --force
 
 If it recurs, the cause is almost always environmental rather than a code defect: check free disk space on the volume holding `.gitnexus/`, make sure no second `analyze` is running against the same repo (both use `.gitnexus/csv` for staging), then run `npx gitnexus doctor`. The check compares in-memory relationship totals (including streamed rows) against what the DB hands back, and is deliberately skipped on incremental runs, where the two counts are not comparable.
 
-**Weak semantic recall despite existing embeddings:** If paraphrased queries look like keyword-only search, the distance cutoff may be too strict for the embedding model. CLI and MCP `query` default to `0.6`. From the indexed repository, try a broader cutoff:
-
-```bash
-GITNEXUS_VECTOR_MAX_DISTANCE=0.8 npx gitnexus query "how are expired sessions removed" --limit 10
-```
-
-Compare target inclusion and ranking against the default; a broader cutoff also admits less relevant hits. Set the variable in the MCP/serve launch environment and restart that process when tuning a server. A cutoff change needs no reindex and setting it only during `analyze` does not persist it. If the index has no vectors, generate embeddings first; if the embedding model or dimensions differ between indexing and querying, align that configuration and regenerate vectors. See [Vector cutoff tuning](README.md#vector-cutoff-tuning) for validation rules and the limited `voyage-code-4` evidence from #3457.
-
 **Large repos:** Analyze may skip or limit embedding work when node counts are very high; watch CLI output.
 
 ---
@@ -195,58 +187,6 @@ If the error text is `"Only one write transaction at a time is allowed in the sy
 
 ---
 
-## File acquisition/reclaim guard recovery
-
-The portable file-lock backend uses `analyze.lock.guard` beside `analyze.lock`.
-Every acquisition, including an empty slot, exclusively creates the guard before
-inspecting, reclaiming, creating, and verifying the main lock. It removes the
-guard before returning a workload handle or waiting on a live workload holder.
-Linux abstract-socket and Windows named-pipe locking are unchanged.
-
-A stalled or crashed guard owner blocks file acquisition even when its PID is
-dead, its metadata is incomplete, or no main lock exists. **The guard is never
-automatically stolen.** Guard contention times out after at most 30 seconds,
-capped by the remaining acquisition timeout. This separate ceiling applies even
-when `GITNEXUS_INDEX_LOCK_TIMEOUT_MS` is zero or negative (unbounded workload wait).
-A guard-cleanup failure rejects acquisition; it must not start unprotected work.
-
-Manual recovery is an outage procedure, not an age/PID-based cleanup:
-
-1. Identify the exact lock directory named in the error. This shared primitive
-   also protects group sync and registry operations, not just repo analysis.
-2. Stop **all relevant writers** and prevent restart: editor/agent hooks, watch
-   processes, scheduled jobs, services, and any containers sharing the directory.
-   Account for paused processes and every host with access. If quiescence cannot
-   be established, do not remove the guard. PID metadata is diagnostic only.
-3. While restart remains disabled, inspect and preserve the guard/main records
-   for diagnosis, then remove only that directory's orphan `analyze.lock.guard`
-   and, if present, its orphan `analyze.lock`. Do not remove databases or sidecars
-   as part of lock recovery. Do not use a recursive or wildcard cleanup.
-4. Ensure all participating writers use the guarded version and the same locking
-   backend/domain, then restart in a controlled fashion.
-
-**Upgrade requires a coordinated stop/upgrade/restart.** Concurrent older
-versions ignore the guard and can still displace live locks; mixed-version
-mutual exclusion is not guaranteed. The file protocol assumes reliable atomic
-local-filesystem `O_EXCL` creation and cooperating processes. Network/distributed
-filesystems, external file replacement, and uncoordinated manual deletion are not
-covered. A process crash while holding the short-lived guard trades automatic
-recovery for fail-closed safety. Denied file creation returns a non-owning
-`lockFree` handle only when neither workload lock nor acquisition guard exists;
-unreadable paths fail closed. No staging sweep runs without ownership. Analysis,
-registry transactions, group synchronization, and embeddings sync refuse
-`lockFree` handles, including an otherwise up-to-date analysis on a file-backend
-read-only mount.
-The socket backend can still acquire ownership on a read-only index mount.
-Heterogeneous permissions are not proof that another process cannot write.
-
-If guard cleanup fails after this attempt created its workload record, acquisition
-is refused and token-exact workload cleanup is attempted before returning the
-error. Failed or unverifiable cleanup must be diagnosed under the same quiesced
-recovery procedure above; never delete a possibly active successor's record.
-
----
-
 ## Where to dig deeper
 
 - Architecture overview: [ARCHITECTURE.md](ARCHITECTURE.md)  

TESTING.md

@@ -19,7 +19,6 @@ From `gitnexus/`:
 | Command                       | What it runs                                       | When to use                           |
 | ----------------------------- | -------------------------------------------------- | ------------------------------------- |
 | `npm test`                    | Full suite (all 3 vitest projects)                 | Before opening a PR                   |
-| `npm run typecheck:tests`     | TypeScript checks for source, tests, and helpers   | Before opening a PR                   |
 | `npm run test:unit`           | Unit tests only (`test/unit/`)                     | Tight development loop                |
 | `npm run test:integration`    | Integration tests (`test/integration/`)            | After changing pipelines, DB, workers |
 | `npm run test:coverage`       | Full suite + v8 coverage with thresholds           | Checking coverage impact              |
@@ -27,11 +26,6 @@ From `gitnexus/`:
 | `npm run test:cross-platform` | Platform-sensitive subset only                     | Debugging a Windows/macOS issue       |
 | `npm run test:watch`          | Vitest in watch mode                               | Active development                    |
 
-Vitest transpiles TypeScript without checking types. Run `npm run typecheck:tests`
-in `gitnexus/` to check test code and helpers against the production types. This
-uses `tsc --noEmit -p tsconfig.test.json`; parser input files under `test/fixtures/`
-are excluded because they are sample source code, not part of the test program.
-
 ### `gitnexus-web/` commands
 
 From `gitnexus-web/`:
@@ -45,9 +39,7 @@ From `gitnexus-web/`:
 ### Before opening a PR
 
 ```bash
-# gitnexus-shared/dist must exist first. `npm install` / `npm run build` in
+cd gitnexus && npx tsc --noEmit && npm test
-# gitnexus/ compiles it via parent `lib/tsc.js` (do not npm ci gitnexus-shared).
-cd gitnexus && npx tsc --noEmit && npm run typecheck:tests && npm test
 cd ../gitnexus-web && npx tsc -b --noEmit && npm test
 ```
 
@@ -125,45 +117,7 @@ GitHub Actions (`.github/workflows/ci.yml`) orchestrate:
 | `ci-scope-parity.yml` | discover, parity                                                  | Scope-resolution parity for all migrated languages                    |
 | `ci-e2e.yml`          | e2e (chromium)                                                    | Playwright E2E, gated on `gitnexus-web/**` changes                    |
 
-The `CI Gate` job in `ci.yml` requires the quality and test workflows to pass.
+The `CI Gate` job in `ci.yml` is the single required check for branch protection. It requires quality, tests, e2e, and scope-parity to all pass.
-The browser E2E workflow must pass or be skipped because no web files changed.
-
-Branch protection also requires six platform check names from the former
-three-shard matrix. These names remain as aggregate gates: all native shards
-and the `every test executed` audit must succeed before any of them passes.
-Failed, cancelled, skipped, or missing dependency results fail these gates.
-The actual native tests run in the current Windows/macOS shard matrix.
-
-The `typecheck` job runs both the production compiler check and
-`npm run typecheck:tests`. A type error in either check fails the job and the CI gate.
-
-### Complete execution, including platform and benchmark tests
-
-The required `every test executed` job reconciles execution receipts from Ubuntu
-coverage, every Windows/macOS shard, the serial benchmark run, and the real Python
-workflow preflight. It requires a recorded pass for every collected test. A skip
-on Linux is satisfied only by a pass of that exact test in another required job.
-Test identities include the file, suite/title and source location; ambiguous
-parameterized cases must have unique titles. Missing receipts, missing test files,
-unhandled runner errors, failed hooks, failed assertions, and tests with no pass
-all fail the gate. Web tests are checked separately with the same rules.
-
-The locked pytest suite and Linux/Windows containment jobs also upload JUnit
-receipts. The same gate checks every Python test file was collected and every
-case passed in at least one job, while preserving failures from any job. The
-Linux containment job supplies Bubblewrap, the pinned CLI, and built Vitest
-dependencies for tests that cannot run in the basic Python job. Python results
-are included in the combined PR report.
-
-The PR report shows the reconciled result as **Unverified**. Zero means every test
-has execution evidence; individual OS logs still show tests that require another
-OS as skipped. Failed executions remain failures even if another job passes.
-
-`npm run test:benchmarks` discovers all tests gated by `GITNEXUS_BENCH` and runs
-them serially. Keep timing measurements out of parallel coverage workers. The
-`eval-tests` job installs the locked Python dependencies and runs the workflow
-preflight Vitest tests as well as pytest; those tests must exercise real Python
-validation, never a stubbed success.
 
 ## Regression testing
 

docker-server.test.mjs

@@ -331,8 +331,8 @@ const respondOk = (_req, res) => {
 //
 //   upstream       request handler, replaceable mid-test via `ctx.handler`;
 //                  null points the proxy at a port nothing ever listens on
-//   listenAfterMs  bind the upstream this late after the first refused attempt
+//   listenAfterMs  bind the upstream this late, so the first attempt(s) hit
-//                  (a single-instance restart window)
+//                  ECONNREFUSED (a single-instance restart window)
 //   schemeless     drop http:// from GITNEXUS_UPSTREAM_URL, the way Render's
 //                  `fromService: { property: hostport }` yields it
 //   env            extra environment for docker-server.mjs
@@ -366,15 +366,18 @@ async function withProxy(
       })
     : null;
 
-  // Keep the upstream port bound until the proxy port is chosen. Releasing it
+  // A late (or never) bind needs its port reserved up front; otherwise let the
-  // sooner lets the OS assign both services the same port and proxy to itself.
+  // OS assign one at listen time.
-  const reservation = server ?? createServer();
+  const upstreamPort =
-  const upstreamPort = await new Promise((r) =>
+    server && listenAfterMs === 0
-    reservation.listen(0, '127.0.0.1', () => r(reservation.address().port)),
+      ? await new Promise((r) => server.listen(0, '127.0.0.1', () => r(server.address().port)))
-  );
+      : await getFreePort();
-  const port = await getFreePort();
+  const bindTimer =
-  let bindTimer = null;
+    server && listenAfterMs > 0
+      ? setTimeout(() => server.listen(upstreamPort, '127.0.0.1'), listenAfterMs)
+      : null;
 
+  const port = await getFreePort();
   const target = `127.0.0.1:${upstreamPort}`;
   const proc = spawnServerWithEnv(dir, port, {
     GITNEXUS_UPSTREAM_URL: schemeless ? target : `http://${target}`,
@@ -387,25 +390,18 @@ async function withProxy(
     ...env,
   });
   proc.stderr.setEncoding('utf8');
-  const collectStderr = (chunk) => {
+  proc.stderr.on('data', (chunk) => {
     ctx.stderr += chunk;
-    // Process startup must not consume the restart window or skip the retry.
+  });
-    if (server && listenAfterMs > 0 && !bindTimer && ctx.stderr.includes('ECONNREFUSED; retry')) {
-      bindTimer = setTimeout(() => server.listen(upstreamPort, '127.0.0.1'), listenAfterMs);
-    }
-  };
-  proc.stderr.on('data', collectStderr);
   try {
     await waitForServer(port);
-    if (!server || listenAfterMs > 0) await new Promise((r) => reservation.close(r));
     await fn(port, ctx);
   } finally {
-    proc.stderr.off('data', collectStderr);
     if (bindTimer) clearTimeout(bindTimer);
     await killAndWait(proc);
-    if (reservation.listening) {
+    if (server?.listening) {
-      reservation.closeAllConnections?.();
+      server.closeAllConnections?.();
-      await new Promise((r) => reservation.close(r));
+      await new Promise((r) => server.close(r));
     }
     await rm(dir, { recursive: true, force: true });
   }
@@ -665,8 +661,8 @@ it('returns 502 when the upstream is unreachable', async () => {
 
 // -- Connection-retry across an upstream restart window ---------------------
 //
-// `listenAfterMs: 400` binds the upstream 400ms after the first ECONNREFUSED,
+// `listenAfterMs: 400` binds the upstream late, so the first attempt hits
-// so the request must be retried — a single-instance restart. The default 3
+// ECONNREFUSED and must be retried — a single-instance restart. The default 3
 // attempts (backoff 250ms, 500ms) span ~750ms, so a retry lands after the bind.
 
 it('retries a connection-refused POST and succeeds once the upstream is up', async () => {
@@ -680,7 +676,6 @@ it('retries a connection-refused POST and succeeds once the upstream is up', asy
     assert.match(res.body, /"ok":true/);
     assert.equal(ctx.calls, 1, 'upstream must run the job exactly once (no double-execute)');
     assert.equal(ctx.body, '{"repo":"x"}', 'buffered body replayed intact');
-    assert.match(ctx.stderr, /ECONNREFUSED; retry/, 'the restart gap must exercise a retry');
   });
 });
 

docs/languages/jupyter-notebook.md

@@ -1,44 +0,0 @@
-# Jupyter notebook (.ipynb) indexing
-
-Status: implemented (Python code cells)
-
-GitNexus indexes Jupyter notebooks by extracting Python code cells and parsing them with the existing Python language provider. Notebooks are not executed.
-
-## Goal
-
-After `analyze`, functions, classes, and imports defined in Python code cells are queryable like ordinary `.py` files.
-
-## Compatibility
-
-- `.ipynb` is detected as Python (`gitnexus-shared` `EXTENSION_MAP` and `pythonProvider.extensions`).
-- Extraction lives in `gitnexus/src/core/ingestion/ipynb-extractor.ts`. Shared ingestion modules do not name nbformat AST types.
-- Notebooks are **not** Python import targets. A notebook may import `.py` modules; `import some_notebook` does not resolve to an `.ipynb`.
-- Files over the walker size cap (default 512KB, `GITNEXUS_MAX_FILE_SIZE`) are skipped like any other oversized file. Output-heavy notebooks may need a higher cap. Outputs are not stripped in this slice.
-- Group-layer FastAPI/Flask/Django scanners that require a `.py` suffix still ignore notebooks.
-
-## Kernel and magics
-
-- Skip the file when `kernelspec.language` or `language_info.name` is present and is not a Python-family name (`python`, `python2`, `python3`, `ipython`, `python 3`, `ipython3`). `python` and `python3` together still index.
-- If `kernelspec.language` is missing, `kernelspec.name` is used only when it is an obvious language id (`python3`, `ir`, `julia-1.8`). Conda env names are ignored.
-- If `language_info.name` is missing, `language_info.file_extension` (`.py` vs `.r` / `.jl`) is used the same way.
-- If those fields are absent, code cells are treated as Python unless a cell's own `language`, `metadata.language`, or `metadata.vscode.languageId` says otherwise.
-- A cell whose first non-empty line is a foreign cell magic (`%%bash`, `%%html`, `%%sql`, `%%R`) is skipped. Python-body cell magics (`%%time`, `%%timeit`, `%%capture`, `%%prun`, `%%debug`, `%%px`, `%%python`) stay, with the magic line commented.
-- `%run`, `%load`, and `%loadpy` of a local `.py` path become `import module` on that same line so the notebook links to the file. URLs, `..` paths, and `.ipynb` targets stay comments. The notebook is not executed.
-- Sage, SageMath, MicroPython, Pyodide, PyPy, and PySpark kernels are indexed as Python. SQL, R, and Julia cells are not.
-- A cell with an unclosed string or bracket is commented out so it cannot hide later cells. Those cells are concatenated in order; one syntax error no longer drops the rest of the notebook.
-- Line magics (`%`), shell (`!`), and IPython help (`train?`, `?train`) are commented in place so JSON line mapping stays affine.
-- nbformat v4 `cells` / `source` is the normal path. nbformat v3 `worksheets[].cells` and code-cell `input` are accepted. A leading UTF-8 BOM is accepted. Markdown and raw cells are not code.
-
-## Line numbers
-
-Graph `startLine` / `endLine` are 0-based coordinates in the on-disk `.ipynb` JSON. FTS/MCP symbol snippets reconstruct cell Python; they do not slice raw JSON.
-
-Concatenating cells in document order is notebook semantics. An earlier cell with a syntax error may cause later definitions to be missed; that is a documented limitation, not a per-cell fallback parse.
-
-## Tests
-
-- `gitnexus/test/unit/ipynb-extractor.test.ts`
-- `gitnexus/test/unit/ingestion-utils.test.ts` (`.ipynb` detection)
-- `gitnexus/test/integration/resolvers/ipynb-python-pipeline.test.ts`
-
-No Jupyter, nbconvert, or nbformat runtime dependency.

docs/languages/objective-c-provider.md

@@ -1,109 +0,0 @@
-# Objective-C Language Provider
-
-Status: implemented
-
-The deterministic provider is covered by focused unit and integration tests. The parser-loader ABI
-smoke runs in the published multi-OS test matrix, and the native prebuild workflow owns
-Objective-C together with all six vendored grammar targets. This status describes the implemented
-MVP; it does not promise full Objective-C runtime dispatch.
-
-## Goal
-
-Add deterministic, symbol-level Objective-C analysis to GitNexus. The first release must support high-confidence code navigation and direct static dependency analysis for `.m`, `.mm`, and Objective-C `.h` files. It must not imply that Objective-C runtime dispatch is fully resolved.
-
-The provider belongs in the existing language-provider and scope-resolution extension points. Shared ingestion code must remain language-agnostic.
-
-## Compatibility contract
-
-- Existing language detection and parsing must remain unchanged.
-- A `.h` file must be classified from its content or surrounding context; it cannot be unconditionally claimed by Objective-C because C and C++ also use that extension.
-- If Objective-C grammar loading fails, the error must clearly name the missing provider/grammar and cannot corrupt a previously valid index.
-- Provider and grammar versions must be stored in index metadata. A version change that can alter node identity or edges requires a full rebuild.
-- No LLM participates in parsing, name resolution, or edge creation. Analysis is Tree-sitter plus deterministic static resolution.
-
-## MVP model
-
-The provider must extract and connect:
-
-- Classes, superclasses, protocols, categories, class extensions, properties, ivars, C functions, imports, declarations, and implementations.
-- Instance and class methods, preserving their complete multi-part selector.
-- Inheritance, protocol conformance, import, declaration/implementation, host-class/category, and statically resolved call relationships.
-
-Stable identity must include enough ownership to distinguish same-named methods. Recommended forms are:
-
-```text
-objc:class:<ClassName>
-objc:protocol:<ProtocolName>
-objc:category:<HostClass>:<CategoryName>
-objc:method:<Owner>:-:<selector>
-objc:method:<Owner>:+:<selector>
-objc:function:<qualified-or-file-scoped-name>
-```
-
-For example, `-loadData:completion:` and `+loadData:completion:` are different symbols. A category method remains linked to both its category and host class; querying the host class must expose distributed implementations.
-
-## Resolution policy
-
-Resolution must be conservative. A missing or dynamic target is evidence of uncertainty, not proof that no target exists.
-
-| Receiver case                                                                                             | Required result                                                                                                        |
-| --------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------- |
-| Explicit class name, `self`, or `super`                                                                   | Resolve when the owner is statically known.                                                                            |
-| Local, parameter, property, or ivar with known static type                                                | Resolve to matching owner and selector.                                                                                |
-| Protocol-typed receiver                                                                                   | Link the protocol method and identify possible implementations as candidates.                                          |
-| Multiple host-class/category implementations of one selector                                              | Record all static candidates as evidence; do not emit a certain call edge because runtime image-load order is unknown. |
-| `id`, `Class`, macros, reflection, `performSelector:`, `NSInvocation`, runtime injection, or unknown type | Store selector/location with `resolution=unresolved`; do not emit a certain call edge.                                 |
-
-The provider should first collect file-local declarations, imports, and types, then resolve across the repository. It must use structured Tree-sitter captures or AST traversal, not regular expressions over source text. Multi-part selectors, block arguments, nullability annotations, generics, macros, and multiline declarations make a regex-only extractor unsafe.
-
-## Imports and incremental correctness
-
-- Resolve quoted project imports against the current directory, configured include roots, and indexed headers. Model framework imports as external-module evidence without downloading SDK source.
-- Merge `@interface`, `@implementation`, categories, and extensions across files.
-- A changed header, protocol, class declaration, or category invalidates importing and affected implementation/call-resolution state. Incremental output after such a change must match a full rebuild.
-- Index metadata must record provider version, grammar version, include/exclude configuration, and parsing options used for resolution.
-
-## Implementation sequence
-
-1. Add and package a pinned Objective-C Tree-sitter grammar; verify macOS arm64 and the production Linux runner can load it.
-2. Add language detection for `.m`, `.mm`, and content-classified `.h` files.
-3. Implement AST extraction and stable IDs for declarations and definitions.
-4. Implement repository-level merge, imports, inheritance, protocol, and category relationships.
-5. Add conservative message-send resolution and explicit unresolved evidence.
-6. Integrate invalidation, metadata comparison, MCP/CLI output, and fixtures.
-
-## Fixtures and acceptance
-
-Create a minimal Objective-C fixture containing a class, protocol, category, extension, superclass, properties, ivars, C function, imports, multi-part selector, block parameter, `self`, `super`, protocol receiver, and `id` receiver. Use `symodulebridge` as a real integration fixture after the minimal suite is stable.
-
-The acceptance bar is:
-
-- `query "SYModuleCaller"` yields class/method semantic nodes, not only file nodes.
-- `context "SYModuleCaller" --file <path>` yields declaration, implementation, imports, and known references.
-- Known statically typed message sends create call edges; dynamic sends are marked unresolved.
-- Same selector on multiple classes, a category override, and `+` versus `-` methods remain distinct.
-- A `.m`, `.h`, protocol, or category edit produces results equivalent to a clean rebuild.
-- Generated documentation, dependency directories, and build output are excluded through explicit indexing configuration.
-
-## Non-goals
-
-The MVP does not promise exact runtime type inference for `id` or `instancetype`, reflection, swizzling, arbitrary category replacement, dynamic selector construction, or complete impact analysis across every runtime dispatch path. Tool results must surface confidence and unresolved evidence rather than presenting guesses as certain graph facts.
-
-## Current implementation coverage
-
-Implemented capabilities:
-
-- Vendored `tree-sitter-objc` grammar, registered through the existing Tree-sitter loader.
-- `.m` and `.mm` language mapping plus content-based `.h` classification so plain C/C++ headers are not unconditionally claimed.
-- LanguageProvider extraction for classes, protocols, categories, extensions, methods, properties, ivars, C functions, imports, unresolved message evidence, stable Objective-C qualified names, and provider/grammar metadata.
-- Length-preserving preprocessing of bare, file-scope all-caps macro markers before Tree-sitter parsing. This recovers declarations after wrappers such as `RCT_EXTERN_C_BEGIN` / `RCT_EXTERN_C_END` without expanding macros or adding framework-specific rules.
-- ScopeResolver edges for imports, inheritance, protocol conformance, category host membership, implementation evidence, and conservative static message sends.
-- Persisted query/context support for Objective-C class and method nodes, including implementation evidence via `DECLARES`.
-- Regression tests for grammar loading, `.h` classification, stable identities, conservative calls, metadata feature mismatch, persisted query/context behavior, and incremental-vs-force parity for Objective-C fixture edits.
-
-Known limits of this MVP:
-
-- The first version does not perform full Objective-C runtime dispatch, swizzling, dynamic selector construction, macro expansion, or `id` flow inference. Bare file-scope marker macros are elided only to preserve parser recovery; their expansion semantics are not interpreted.
-- Protocol receiver handling records the protocol method and candidate implementation evidence, but candidate implementations are not emitted as certain call edges.
-- When a host class and one or more named categories define the same selector, the provider records candidate evidence rather than choosing a runtime winner or emitting multiple certain call edges.
-- Objective-C++ `.mm` files are parsed with the Objective-C grammar path for this MVP; deep C++ semantic extraction inside Objective-C++ bodies remains outside this provider.

eval/.gitignore

@@ -14,4 +14,3 @@ build/
 # Environment
 .env
 .venv/
-.venv

eval/analysis/analyze_results.py

@@ -17,6 +17,7 @@ Usage:
 
 import json
 import logging
+import os
 import subprocess
 import sys
 from pathlib import Path

eval/bridge/mcp_bridge.py

@@ -14,6 +14,7 @@ import subprocess
 import sys
 import threading
 import time
+from pathlib import Path
 from typing import Any
 
 from constants import (

eval/check_test_execution.py

@@ -1,121 +0,0 @@
-"""Require a real passing execution of every pytest case across the CI jobs."""
-
-from __future__ import annotations
-
-import json
-from pathlib import Path
-import sys
-import xml.etree.ElementTree as ET
-
-
-RECEIPTS = ("pytest-locked.xml", "pytest-ubuntu.xml", "pytest-windows.xml")
-
-
-def reconcile(reports: list[Path], expected_files: list[str]) -> dict:
-    if not reports:
-        raise ValueError("No pytest execution reports supplied")
-    tests: dict[tuple[str, str], set[str]] = {}
-    durations: list[float] = []
-    for report in reports:
-        root = ET.parse(report).getroot()
-        suites = [root] if root.tag == "testsuite" else list(root.findall("testsuite"))
-        if root.tag not in {"testsuite", "testsuites"} or not suites:
-            raise ValueError(f"Invalid pytest report: {report}")
-        seen: set[tuple[str, str]] = set()
-        duration = 0.0
-        for suite in suites:
-            cases = suite.findall("testcase")
-            if not cases:
-                raise ValueError(f"Empty pytest suite in {report}")
-            counts = {"tests": len(cases), "failures": 0, "errors": 0, "skipped": 0}
-            for case in cases:
-                key = (case.get("classname", ""), case.get("name", ""))
-                if not all(key) or key in seen:
-                    raise ValueError(f"Missing or ambiguous pytest identity in {report}: {key}")
-                seen.add(key)
-                for status in ("failures", "errors", "skipped"):
-                    tag = {"failures": "failure", "errors": "error", "skipped": "skipped"}[status]
-                    counts[status] += len(case.findall(tag))
-                status = (
-                    "failed"
-                    if case.find("failure") is not None or case.find("error") is not None
-                    else "pending"
-                    if case.find("skipped") is not None
-                    else "passed"
-                )
-                tests.setdefault(key, set()).add(status)
-            for name, count in counts.items():
-                if int(suite.get(name, "-1")) != count:
-                    raise ValueError(f"Inconsistent pytest {name} count in {report}")
-            duration += float(suite.get("time", "0"))
-        durations.append(duration)
-
-    modules = {module for module, _ in tests}
-    for file in expected_files:
-        module = file.removesuffix(".py").replace("/", ".")
-        if not any(name == module or name.startswith(module + ".") for name in modules):
-            raise ValueError(f"Pytest file was never collected: {file}")
-
-    suites_by_module: dict[str, dict] = {}
-    failed: list[str] = []
-    pending = 0
-    for (module, name), statuses in sorted(tests.items()):
-        status = "failed" if "failed" in statuses else "passed" if "passed" in statuses else "pending"
-        full_name = f"{module}::{name}"
-        if status == "failed":
-            failed.append(full_name)
-        pending += status == "pending"
-        suite = suites_by_module.setdefault(
-            module,
-            {
-                "name": module,
-                "status": "passed",
-                "assertionResults": [],
-                "endTime": round(max(durations) * 1000),
-            },
-        )
-        suite["assertionResults"].append(
-            {
-                "fullName": full_name,
-                "ancestorTitles": [module],
-                "title": name,
-                "status": status,
-            }
-        )
-        if status == "failed":
-            suite["status"] = "failed"
-    return {
-        "success": not failed and pending == 0,
-        "numTotalTests": len(tests),
-        "numPassedTests": len(tests) - len(failed) - pending,
-        "numFailedTests": len(failed),
-        "numPendingTests": pending,
-        "numTotalTestSuites": len(suites_by_module),
-        "numFailedTestSuites": sum(s["status"] == "failed" for s in suites_by_module.values()),
-        "executionFailures": failed,
-        "startTime": 0,
-        "testResults": list(suites_by_module.values()),
-    }
-
-
-def main() -> int:
-    if len(sys.argv) != 3:
-        raise ValueError("Usage: check_test_execution.py <reports-dir> <output.json>")
-    directory, output = map(Path, sys.argv[1:])
-    root = Path(__file__).resolve().parent
-    expected = [p.relative_to(root).as_posix() for p in (root / "tests").rglob("test_*.py")]
-    report = reconcile([directory / name for name in RECEIPTS], expected)
-    output.write_text(json.dumps(report), encoding="utf-8")
-    print(
-        f"Eval execution: {report['numPassedTests']}/{report['numTotalTests']} passed; "
-        f"{report['numPendingTests']} unverified; {report['numFailedTests']} failures"
-    )
-    for suite in report["testResults"]:
-        for case in suite["assertionResults"]:
-            if case["status"] != "passed":
-                print(f"{case['status']}: {case['fullName']}", file=sys.stderr)
-    return 0 if report["success"] else 1
-
-
-if __name__ == "__main__":
-    sys.exit(main())

eval/pyproject.toml

@@ -6,10 +6,7 @@ readme = "README.md"
 requires-python = ">=3.11"
 dependencies = [
     "mini-swe-agent>=2.0.0",
-    "litellm[proxy]!=1.82.7,!=1.82.8,>=1.99.0",
+    "litellm!=1.82.7,!=1.82.8,>=1.83.7",
-    "cryptography>=50.0.0",
-    "python-multipart>=0.0.30",
-    "restrictedpython>=8.3",
     "datasets>=3.0.0",
     "typer>=0.12.0",
     "rich>=13.0.0",

eval/run_eval.py

@@ -27,10 +27,12 @@ import threading
 import time
 from itertools import product
 from pathlib import Path
+from typing import Any
 
 import typer
 import yaml
 from rich.console import Console
+from rich.live import Live
 from rich.table import Table
 
 from utils.errors import is_debug_enabled, log_safe_exception

eval/tests/bench_fixtures.py

@@ -1,75 +0,0 @@
-"""Shared row shapes for the sweep tests.
-
-Building the finalization tests turned up what a real scored review row must
-carry: the report renders the whole review metric set, so an incomplete row
-fails in string formatting rather than in the logic under test. That is a
-property of the fixture, not of production - the shape lives here once so each
-test does not rediscover it.
-"""
-
-from __future__ import annotations
-
-from typing import Any
-
-
-def scored_review_row(**overrides: Any) -> dict[str, Any]:
-    """One admissible review cell, with zero-valued metrics written out."""
-
-    row: dict[str, Any] = {
-        "ok": True,
-        "error_kind": None,
-        "error_detail": None,
-        "resolved": True,
-        "review_evidence_valid": True,
-        "review_score": {"weighted_f1": 0.5},
-        "review_weighted_f1": 0.5,
-        "review_true_positives": 1,
-        "review_false_positives": 0,
-        "review_false_negatives": 0,
-        "review_precision": 0.5,
-        "review_recall": 0.5,
-        "review_f1": 0.5,
-        "review_weighted_precision": 0.5,
-        "review_weighted_recall": 0.5,
-        "review_blocker_recall": 1.0,
-        "review_severity_accuracy": 1.0,
-        "review_category_accuracy": 1.0,
-        "review_grounded_evidence": 1.0,
-        "review_verdict_correct": True,
-        "review_clean_control": True,
-        "review_clean_pass": True,
-        "transcript_missing": False,
-        "transcript_artifacts": [],
-        "num_turns": 3,
-        "duration_s": 1.0,
-        "cost_usd": 0.5,
-        "input_tokens": 1,
-        "output_tokens": 1,
-        "cache_creation_input_tokens": 0,
-        "cache_read_input_tokens": 0,
-        "diff_files": 0,
-        "diff_insertions": 0,
-        "diff_deletions": 0,
-    }
-    row.update(overrides)
-    return row
-
-
-def unusable_review_row(**overrides: Any) -> dict[str, Any]:
-    """A cell that ran but produced evidence nothing can be scored from."""
-
-    # Merged into one mapping rather than passed as explicit keywords beside
-    # **overrides: Python rejects a duplicate keyword in the call expression
-    # itself, so unusable_review_row(error_kind=...) raised TypeError before
-    # scored_review_row could apply the override this helper advertises.
-    return scored_review_row(
-        **{
-            "ok": False,
-            "resolved": False,
-            "review_evidence_valid": False,
-            "error_kind": "review-evidence-invalid",
-            "review_score": None,
-            "review_weighted_f1": None,
-            **overrides,
-        }
-    )

eval/tests/fixtures/fake_claude.py

@@ -1,175 +0,0 @@
-#!/usr/bin/env python3
-"""A stand-in for the Claude Code CLI: real HTTP, real tool execution, real stream-json.
-
-Not a mock of the harness's own code. It does what the CLI does at the two
-boundaries the harness depends on - it calls ANTHROPIC_BASE_URL for a turn, it
-EXECUTES the tool blocks that come back, and it prints the stream-json event
-sequence the parent parses. Only Write really executes - it is what produces the
-review artifact, so the artifact path has to be genuine end to end. Skill is
-MODELLED: it validates the request and returns a synthetic result, because the
-parent's evidence gate keys on the request/result pair rather than on a skill
-having loaded, and a fixture cannot load a real one. Bash is stubbed outright:
-arbitrary shell from a scripted reply buys no fidelity for the paths this
-exercises and plenty of ways to damage the host. Everything between
-those boundaries (the sandbox,
-the artifact capture, the scoring, the row) stays real, which is the whole
-point: those are the layers that shipped bugs no unit test could see.
-
-Reads the prompt from stdin, as the real CLI does under "-p --input-format text".
-"""
-
-from __future__ import annotations
-
-import json
-import os
-import pathlib
-import sys
-import urllib.request
-
-
-def _turn(base_url: str, prompt: str) -> dict:
-    request = urllib.request.Request(
-        base_url.rstrip("/") + "/v1/messages",
-        data=json.dumps({"model": os.environ.get("ANTHROPIC_MODEL", "mock"), "max_tokens": 1024,
-                         "messages": [{"role": "user", "content": prompt}]}).encode(),
-        headers={"Content-Type": "application/json",
-                 "x-api-key": os.environ.get("ANTHROPIC_API_KEY", ""),
-                 "anthropic-version": "2023-06-01"},
-    )
-    with urllib.request.urlopen(request, timeout=30) as response:
-        return json.load(response)
-
-
-def _run_tool(name: str, params: dict) -> str:
-    """Write executes for real - it is what produces the review artifact.
-
-    Skill and Bash do not: see the module docstring for which is modelled and
-    which is stubbed, and why neither can be genuine here.
-    """
-
-    if name == "Write":
-        target = pathlib.Path(params["file_path"])
-        target.parent.mkdir(parents=True, exist_ok=True)
-        # Atomic, exactly as the real Write tool does it: temp file beside the
-        # target, then rename. This is the operation the read-only workspace
-        # boundary has to permit for the artifact directory and refuse for the
-        # workspace, so a stand-in that wrote in place would prove nothing.
-        staging = target.with_name(target.name + ".tmp.fake")
-        staging.write_text(params.get("content", ""))
-        os.replace(staging, target)
-        return f"wrote {target}"
-    if name == "Skill":
-        # Modelled explicitly rather than falling through to a generic success.
-        # The parent's evidence gate keys on a Skill request with a non-error
-        # result, so leaving this unimplemented let an unexecuted skill satisfy
-        # the gate - the gate would have been measuring the fixture, not a skill.
-        skill = params.get("skill") or params.get("command") or params.get("name")
-        if not skill:
-            raise NotImplementedError("Skill request carried no skill name")
-        return f"loaded skill {skill}"
-    if name == "Bash":
-        return "(bash suppressed in the stand-in)"
-    # An unsupported tool is a FAILED tool run, not a quiet success. Returning a
-    # plain string here made the parent's evidence gate read an unexecuted Skill
-    # request as a successful invocation.
-    raise NotImplementedError(f"unsupported tool {name}")
-
-
-def main() -> int:
-    # stdin, because that is where the real CLI takes it under
-    # "-p --input-format text": the parent pipes prompt bytes in. Scanning argv
-    # for a non-flag token picks up a flag's VALUE instead ("text"), which is
-    # exactly what the prompt-fidelity test caught.
-    prompt = sys.stdin.read()
-    base_url = os.environ.get("ANTHROPIC_BASE_URL")
-    if not base_url:
-        print(json.dumps({"type": "result", "subtype": "error", "is_error": True,
-                          "session_id": "fake-session", "num_turns": 0}), flush=True)
-        return 1
-
-    emit = lambda event: print(json.dumps(event), flush=True)  # noqa: E731
-    emit({"type": "system", "subtype": "init", "session_id": "fake-session"})
-
-    try:
-        message = _turn(base_url, prompt)
-    except (OSError, ValueError) as exc:
-        # A provider failure is a failed SESSION, not a crashed process: dying
-        # here leaves no terminal result event, so the parent reports a generic
-        # stream error instead of the upstream failure it actually saw.
-        emit({"type": "result", "subtype": "error", "is_error": True,
-              "session_id": "fake-session", "num_turns": 0,
-              "error": f"provider request failed: {type(exc).__name__}: {exc}"})
-        return 1
-    blocks = message.get("content", [])
-    emit({"type": "assistant", "message": {"role": "assistant", "content": blocks}})
-
-    tool_results = []
-    for block in blocks:
-        if block.get("type") == "tool_use":
-            # A refused write is a tool ERROR the session reports and carries
-            # on from, not a crash. Letting it kill the process would lose the
-            # result event and misreport a working boundary as a broken run.
-            failed = False
-            try:
-                output = _run_tool(block["name"], block.get("input", {}))
-            except (OSError, NotImplementedError) as exc:
-                output, failed = f"error: {type(exc).__name__}: {exc}", True
-            # is_error is load-bearing: the parent treats an ABSENT is_error as
-            # success, so a refused or unsupported tool would otherwise be
-            # scored as a completed one.
-            tool_results.append({
-                "type": "tool_result", "tool_use_id": block["id"],
-                "content": output, "is_error": failed,
-            })
-    if tool_results:
-        emit({"type": "user", "message": {"role": "user", "content": tool_results}})
-
-    # Unknown is not zero. A reply carrying no usage used to become four
-    # zero-valued fields plus a fabricated cost, which the harness then treats
-    # as a real measurement - the exact confusion the accounting this fixture
-    # feeds exists to prevent.
-    usage = message.get("usage")
-    # Every field that gets forwarded is validated, not just the required two.
-    # The parent's well_formed check tests only that the four keys are PRESENT,
-    # so an unvalidated cache value rides into a success result and is recorded
-    # as a real measurement. A field good enough to report is good enough to
-    # check.
-    countable = lambda v: isinstance(v, int) and not isinstance(v, bool) and v >= 0  # noqa: E731
-    if not isinstance(usage, dict) or not all(
-        countable(usage.get(f)) for f in ("input_tokens", "output_tokens")
-    ) or not all(
-        countable(usage[f])
-        for f in ("cache_read_input_tokens", "cache_creation_input_tokens")
-        if f in usage
-    ):
-        emit({"type": "result", "subtype": "error", "is_error": True,
-              "session_id": "fake-session", "num_turns": 1,
-              "error": "provider reply carried no usable usage; refusing to report a measured run"})
-        return 1
-    emit({
-        "type": "result",
-        "subtype": "success",
-        "is_error": False,
-        "session_id": "fake-session",
-        "num_turns": 1,
-        "duration_ms": 1200,
-        # A measured zero is not the same as unmeasured; the parent rejects a
-        # collapsed cost, so report a real one.
-        "total_cost_usd": 0.42,
-        # Forward exactly the fields the provider reported. Defaulting the
-        # absent ones to 0 fabricated a complete measurement out of an
-        # incomplete reply - and worse, it made the parent's own completeness
-        # check (runner_sessions.USAGE_FIELDS / well_formed) unfirable from any
-        # offline test, because the stand-in always satisfied it.
-        "usage": {
-            field: usage[field]
-            for field in ("input_tokens", "output_tokens",
-                          "cache_read_input_tokens", "cache_creation_input_tokens")
-            if field in usage
-        },
-    })
-    return 0
-
-
-if __name__ == "__main__":
-    raise SystemExit(main())

eval/tests/test_check_test_execution.py

@@ -1,142 +0,0 @@
-"""The execution gate must reject absent, skipped, failed, or ambiguous evidence."""
-
-from pathlib import Path
-import json
-import shutil
-import subprocess
-import sys
-import xml.etree.ElementTree as ET
-
-import pytest
-
-from check_test_execution import reconcile
-
-
-def receipt(tmp_path: Path, filename: str, cases: list[tuple[str, str]], module="tests.test_native") -> Path:
-    suite = ET.Element("testsuite", tests=str(len(cases)), failures="0", errors="0", skipped="0", time="1.5")
-    for name, status in cases:
-        case = ET.SubElement(suite, "testcase", classname=module, name=name)
-        if status != "passed":
-            ET.SubElement(case, status)
-            count = {"failure": "failures", "error": "errors", "skipped": "skipped"}[status]
-            suite.set(count, str(int(suite.get(count)) + 1))
-    path = tmp_path / filename
-    ET.ElementTree(suite).write(path, encoding="utf-8")
-    return path
-
-
-def test_platform_skip_requires_a_matching_real_pass(tmp_path):
-    linux = receipt(tmp_path, "linux.xml", [("test_windows", "skipped"), ("test_posix", "passed")])
-    windows = receipt(tmp_path, "windows.xml", [("test_windows", "passed"), ("test_posix", "skipped")])
-    report = reconcile([linux, windows], ["tests/test_native.py"])
-    assert report["success"] is True
-    assert report["numPassedTests"] == report["numTotalTests"] == 2
-    assert report["numPendingTests"] == report["numFailedTests"] == 0
-
-
-def test_unresolved_skip_is_not_a_pass(tmp_path):
-    path = receipt(tmp_path, "report.xml", [("test_missing_runtime", "skipped")])
-    report = reconcile([path], [])
-    assert report["success"] is False
-    assert report["numPendingTests"] == 1
-    assert report["numPassedTests"] == 0
-    assert report["testResults"][0]["assertionResults"][0]["ancestorTitles"] == ["tests.test_native"]
-
-
-@pytest.mark.parametrize("status", ["failure", "error"])
-def test_failure_is_not_hidden_by_another_job_passing(tmp_path, status):
-    failed = receipt(tmp_path, "failed.xml", [("test_shared", status)])
-    passed = receipt(tmp_path, "passed.xml", [("test_shared", "passed")])
-    report = reconcile([failed, passed], [])
-    assert report["success"] is False
-    assert report["numFailedTests"] == 1
-    assert report["numPassedTests"] == 0
-    assert report["executionFailures"] == ["tests.test_native::test_shared"]
-
-
-def test_same_title_in_another_file_does_not_resolve_a_skip(tmp_path):
-    pending = receipt(tmp_path, "pending.xml", [("test_same", "skipped")])
-    unrelated = receipt(tmp_path, "unrelated.xml", [("test_same", "passed")], module="tests.test_other")
-    report = reconcile([pending, unrelated], [])
-    assert report["numTotalTests"] == 2
-    assert report["numPendingTests"] == 1
-
-
-def test_duplicate_identity_is_rejected(tmp_path):
-    path = receipt(tmp_path, "duplicate.xml", [("test_same", "passed"), ("test_same", "skipped")])
-    with pytest.raises(ValueError, match="ambiguous pytest identity"):
-        reconcile([path], [])
-
-
-def test_missing_receipt_is_rejected(tmp_path):
-    with pytest.raises(FileNotFoundError):
-        reconcile([tmp_path / "missing.xml"], [])
-
-
-def test_empty_receipt_is_rejected(tmp_path):
-    path = receipt(tmp_path, "empty.xml", [])
-    with pytest.raises(ValueError, match="Empty pytest suite"):
-        reconcile([path], [])
-
-
-def test_missing_file_is_rejected(tmp_path):
-    path = receipt(tmp_path, "report.xml", [("test_present", "passed")])
-    with pytest.raises(ValueError, match="never collected: tests/test_missing.py"):
-        reconcile([path], ["tests/test_missing.py"])
-
-
-def test_inconsistent_counts_are_rejected(tmp_path):
-    path = receipt(tmp_path, "report.xml", [("test_present", "passed")])
-    path.write_text(path.read_text().replace('tests="1"', 'tests="2"'))
-    with pytest.raises(ValueError, match="Inconsistent pytest tests count"):
-        reconcile([path], [])
-
-
-def test_pytest_testsuites_wrapper_and_test_classes(tmp_path):
-    path = receipt(tmp_path, "report.xml", [("test_method", "passed")], module="tests.test_native.TestNative")
-    suite = ET.parse(path).getroot()
-    root = ET.Element("testsuites")
-    root.append(suite)
-    ET.ElementTree(root).write(path, encoding="utf-8")
-    assert reconcile([path], ["tests/test_native.py"])["success"] is True
-
-
-def test_missing_report_list_is_rejected():
-    with pytest.raises(ValueError, match="No pytest execution reports"):
-        reconcile([], [])
-
-
-@pytest.mark.parametrize("scenario", ["passed", "pending", "failed", "missing-receipt", "missing-file"])
-def test_cli_requires_complete_execution_evidence(tmp_path, scenario):
-    script = tmp_path / "check_test_execution.py"
-    shutil.copyfile(Path(__file__).parents[1] / script.name, script)
-    tests = tmp_path / "tests"
-    tests.mkdir()
-    (tests / "test_native.py").write_text("# inventory fixture\n")
-    reports = tmp_path / "reports"
-    reports.mkdir()
-    receipt(reports, "pytest-locked.xml", [("test_native", "skipped" if scenario == "pending" else "passed")])
-    receipt(reports, "pytest-ubuntu.xml", [("test_native", "error" if scenario == "failed" else "skipped")])
-    if scenario != "missing-receipt":
-        receipt(reports, "pytest-windows.xml", [("test_native", "skipped")])
-    if scenario == "missing-file":
-        (tests / "test_missing.py").write_text("# must be collected\n")
-    output = tmp_path / "pytest-results.json"
-    result = subprocess.run(
-        [sys.executable, str(script), str(reports), str(output)],
-        capture_output=True,
-        text=True,
-        timeout=10,
-    )
-    assert result.returncode == (0 if scenario == "passed" else 1)
-    if scenario.startswith("missing-"):
-        assert not output.exists()
-        assert ("pytest-windows.xml" if scenario == "missing-receipt" else "never collected") in result.stderr
-    else:
-        report = json.loads(output.read_text())
-        assert report["success"] is (scenario == "passed")
-        assert report["numTotalTests"] == 1
-        assert report["numPassedTests"] == (scenario == "passed")
-        assert report["numPendingTests"] == (scenario == "pending")
-        assert report["numFailedTests"] == (scenario == "failed")
-        assert "Eval execution:" in result.stdout

eval/tests/test_comparator_reuse.py

@@ -1,467 +0,0 @@
-"""Comparator-row reuse: skip unchanged incumbent/CE cells, never candidates."""
-
-from __future__ import annotations
-
-import hashlib
-import os
-from datetime import UTC, datetime, timedelta
-from pathlib import Path
-
-import pytest
-
-from workflow_bench import comparator_reuse
-from workflow_bench.comparator_reuse import (
-    ComparatorReuseExpectation,
-    TaskReuseBinding,
-    materialize_reused_row,
-    row_is_reusable_comparator,
-    select_reusable_comparator_rows,
-)
-from workflow_bench.proposer_sandbox import SandboxError
-from workflow_bench.runner_sessions import PARENT_EVENT_STREAM_SOURCE
-
-
-requires_openat = pytest.mark.skipif(
-    os.open not in os.supports_dir_fd,
-    reason="comparator reuse resolves every artifact against a pinned directory descriptor",
-)
-
-
-def _digest(text: str = "blob") -> str:
-    return hashlib.sha256(text.encode()).hexdigest()
-
-
-def _artifact(name: str = "session-1.jsonl", payload: bytes = b'{"type":"ok"}\n') -> dict:
-    return {
-        "path": f"transcripts/{name}",
-        "sha256": hashlib.sha256(payload).hexdigest(),
-        "bytes": len(payload),
-        "source": PARENT_EVENT_STREAM_SOURCE,
-    }
-
-
-def _row(**overrides) -> dict:
-    base = {
-        "task": "review-pr-2718-defect",
-        "arm": "review",
-        "run": 0,
-        "ok": True,
-        "error_kind": None,
-        "model": "gpt-5.6-sol",
-        "benchmark_model": "gpt-5.6-sol",
-        "effort": "xhigh",
-        "sandbox_backend": "bwrap",
-        "task_base_sha": "a" * 40,
-        "task_prompt_digest": _digest("prompt"),
-        "oracle_digest": _digest("oracle"),
-        "oracle_command_digest": _digest("oracle-cmd"),
-        "oracle_manifest_digest": _digest("oracle-man"),
-        "skill_digest": _digest("skill"),
-        "candidate_overlay_digest": None,
-        "review_evidence_valid": True,
-        # Production sets this whenever the review source exists, which is the
-        # normal path for a valid review; the fixture predated the requirement.
-        "review_artifact": "review-pr-2718-defect-review-run0.review.json",
-        "review_score": {"weighted_f1": 0.4},
-        "review_weighted_f1": 0.4,
-        "transcript_missing": False,
-        "transcript_artifacts": [_artifact()],
-        "recorded_at": datetime.now(UTC).isoformat(),
-        "runtime_digest": _digest("cli"),
-        "task_asset_manifest_digest": _digest("assets"),
-        "sandbox_dependency_manifest_digest": _digest("deps"),
-    }
-    base.update(overrides)
-    return base
-
-
-def _expected(**overrides) -> ComparatorReuseExpectation:
-    now = datetime.now(UTC)
-    values = dict(
-        model="gpt-5.6-sol",
-        effort="xhigh",
-        sandbox_backend="bwrap",
-        runtime_digest=_digest("cli"),
-        now=now,
-        max_age=timedelta(days=90),
-        tasks={
-            "review-pr-2718-defect": TaskReuseBinding(
-                task_base_sha="a" * 40,
-                task_prompt_digest=_digest("prompt"),
-                oracle_digest=_digest("oracle"),
-                oracle_command_digest=_digest("oracle-cmd"),
-                oracle_manifest_digest=_digest("oracle-man"),
-                task_asset_manifest_digest=_digest("assets"),
-                sandbox_dependency_manifest_digest=_digest("deps"),
-            )
-        },
-        skill_digests={"review": _digest("skill"), "ce_review": None},
-        ce_plugin_version="3.24.0",
-        ce_plugin_manifest_digest=_digest("ce"),
-    )
-    values.update(overrides)
-    return ComparatorReuseExpectation(**values)
-
-
-def test_matching_incumbent_review_row_is_reusable() -> None:
-    assert row_is_reusable_comparator(_row(), _expected()) is True
-
-
-def test_candidate_rows_are_never_reusable() -> None:
-    assert row_is_reusable_comparator(_row(arm="candidate_review"), _expected()) is False
-
-
-def test_skill_digest_drift_rejects_reuse() -> None:
-    assert row_is_reusable_comparator(_row(), _expected(skill_digests={"review": _digest("other")})) is False
-
-
-def test_excluded_or_failed_rows_are_not_reusable() -> None:
-    expected = _expected()
-    assert row_is_reusable_comparator(_row(error_kind="session-error", ok=False), expected) is False
-    assert row_is_reusable_comparator(_row(ok=False), expected) is False
-    assert row_is_reusable_comparator(_row(review_evidence_valid=False), expected) is False
-    assert row_is_reusable_comparator(_row(recorded_at=(datetime.now(UTC) - timedelta(days=91)).isoformat()), expected) is False
-
-
-def test_runtime_digest_mismatch_rejects_when_both_sides_are_bound() -> None:
-    row = _row(runtime_digest=_digest("old-cli"))
-    assert row_is_reusable_comparator(row, _expected(runtime_digest=_digest("new-cli"))) is False
-    assert row_is_reusable_comparator(row, _expected(runtime_digest=_digest("old-cli"))) is True
-    # A row with no runtime_digest was measured by a harness that recorded none,
-    # which is the drift this lock exists to catch - not evidence of agreement.
-    assert row_is_reusable_comparator(_row(runtime_digest=None), _expected()) is False
-    # And a sweep that cannot determine its own digest must not reuse either.
-    assert row_is_reusable_comparator(_row(), _expected(runtime_digest=None)) is False
-
-
-def test_ce_review_matches_plugin_digest_not_repo_skill() -> None:
-    row = _row(
-        arm="ce_review",
-        skill_digest=None,
-        ce_plugin_version="3.24.0",
-        ce_plugin_manifest_digest=_digest("ce"),
-    )
-    assert row_is_reusable_comparator(row, _expected()) is True
-    assert (
-        row_is_reusable_comparator(row, _expected(ce_plugin_manifest_digest=_digest("other")))
-        is False
-    )
-
-
-def test_select_drops_conflicting_duplicates() -> None:
-    first = _row(review_weighted_f1=0.4)
-    second = _row(review_weighted_f1=0.9, recorded_at=datetime.now(UTC).isoformat())
-    selected = select_reusable_comparator_rows([first, second], expected=_expected())
-    assert selected == {}
-    same = select_reusable_comparator_rows([first, dict(first)], expected=_expected())
-    assert ("review-pr-2718-defect", "review", 0) in same
-
-
-@requires_openat
-def test_materialize_copies_transcript_and_review_artifacts(tmp_path: Path) -> None:
-    payload = b'{"type":"result"}\n'
-    source = tmp_path / "prior"
-    dest = tmp_path / "fresh"
-    (source / "transcripts").mkdir(parents=True)
-    dest.mkdir()
-    transcript = source / "transcripts" / "session-1.jsonl"
-    transcript.write_bytes(payload)
-    transcript.chmod(0o600)
-    review = source / "review-pr-2718-defect-review-run0.review.json"
-    review.write_text('{"verdict":"comment"}\n')
-    patch = source / "review-pr-2718-defect-review-run0.patch"
-    patch.write_text("diff\n")
-    row = _row(
-        review_artifact=review.name,
-        transcript_artifacts=[_artifact(payload=payload)],
-    )
-
-    copied = materialize_reused_row(row, source_dir=source, dest_dir=dest)
-
-    assert copied["reused"] is True
-    assert copied["reused_from_recorded_at"] == row["recorded_at"]
-    assert (dest / "transcripts" / "session-1.jsonl").read_bytes() == payload
-    assert (dest / review.name).read_text() == review.read_text()
-    assert (dest / patch.name).read_text() == "diff\n"
-    assert copied["transcript_artifacts"][0]["sha256"] == hashlib.sha256(payload).hexdigest()
-
-
-@pytest.mark.skipif(os.name == "nt", reason="symlink creation may require elevated Windows privileges")
-@requires_openat
-def test_a_reused_artifact_is_copied_from_the_inode_that_was_checked(tmp_path: Path) -> None:
-    """The reuse source is a directory another sweep wrote and may still write.
-
-    Validating a path and then re-opening it hands a concurrent writer the gap:
-    replace the checked file with a symlink and the copy follows it out of the
-    results directory. Swapping the path while the descriptor is held is that
-    same substitution, made deterministic.
-    """
-
-    (tmp_path / "transcript.jsonl").write_bytes(b"verified\n")
-    decoy = tmp_path / "decoy.jsonl"
-    decoy.write_bytes(b"substituted\n")
-
-    with comparator_reuse._open_real_directory(tmp_path, label="reuse source") as dir_fd:
-        with comparator_reuse._open_regular("transcript.jsonl", dir_fd=dir_fd, label="transcript") as descriptor:
-            (tmp_path / "transcript.jsonl").unlink()
-            (tmp_path / "transcript.jsonl").symlink_to(decoy)
-            comparator_reuse._copy_owner_only(descriptor, "copy.jsonl", dir_fd=dir_fd)
-
-        assert (tmp_path / "copy.jsonl").read_bytes() == b"verified\n"
-        with pytest.raises(SandboxError, match="regular non-symlink"):
-            with comparator_reuse._open_regular("transcript.jsonl", dir_fd=dir_fd, label="transcript"):
-                pass
-
-
-@pytest.mark.skipif(os.name == "nt", reason="symlink creation may require elevated Windows privileges")
-@requires_openat
-def test_a_symlinked_transcripts_directory_is_refused_on_both_sides(tmp_path: Path) -> None:
-    """`O_NOFOLLOW` refuses the leaf, not the directory above it.
-
-    A `transcripts` symlink on the source side makes reuse read a file outside
-    the results directory; one on the destination side writes the copy outside
-    this sweep's evidence. Neither is covered by the per-file guards that let
-    _resolved_directory tolerate a symlinked root.
-    """
-
-    payload = b'{"type":"result"}\n'
-    outside = tmp_path / "outside"
-    (outside / "transcripts").mkdir(parents=True)
-    (outside / "transcripts" / "session-1.jsonl").write_bytes(payload)
-    row = _row(transcript_artifacts=[_artifact(payload=payload)])
-
-    linked_source = tmp_path / "linked-source"
-    linked_source.mkdir()
-    (linked_source / "transcripts").symlink_to(outside / "transcripts", target_is_directory=True)
-    dest = tmp_path / "fresh"
-    dest.mkdir()
-    with pytest.raises(SandboxError, match="transcript source must be a real directory"):
-        materialize_reused_row(row, source_dir=linked_source, dest_dir=dest)
-
-    source = tmp_path / "prior"
-    (source / "transcripts").mkdir(parents=True)
-    (source / "transcripts" / "session-1.jsonl").write_bytes(payload)
-    linked_dest = tmp_path / "linked-dest"
-    linked_dest.mkdir()
-    (linked_dest / "transcripts").symlink_to(outside / "transcripts", target_is_directory=True)
-    with pytest.raises(SandboxError, match="transcript destination must be a real directory"):
-        materialize_reused_row(row, source_dir=source, dest_dir=linked_dest)
-
-
-@requires_openat
-@pytest.mark.skipif(os.name == "nt", reason="symlink creation may require elevated Windows privileges")
-def test_a_renamed_transcripts_directory_cannot_redirect_a_copy(tmp_path: Path) -> None:
-    """The directory is pinned, not re-walked from its name.
-
-    An lstat that passed and a pathname used afterwards are two different
-    directories the moment a concurrent writer renames the first one away. This
-    performs exactly that substitution — rename, then leave a symlink in its
-    place — while the descriptor is held, which is what makes the race testable
-    without timing.
-    """
-
-    payload = b'{"type":"result"}\n'
-    results = tmp_path / "results"
-    transcripts = results / "transcripts"
-    transcripts.mkdir(parents=True)
-    (transcripts / "session-1.jsonl").write_bytes(payload)
-    outside = tmp_path / "outside"
-    outside.mkdir()
-
-    with comparator_reuse._open_real_directory(results, label="reuse source") as root_fd:
-        with comparator_reuse._open_real_directory(
-            "transcripts", dir_fd=root_fd, label="transcript source"
-        ) as dir_fd:
-            transcripts.rename(results / "moved")
-            (results / "transcripts").symlink_to(outside, target_is_directory=True)
-            with comparator_reuse._open_regular(
-                "session-1.jsonl", dir_fd=dir_fd, label="transcript"
-            ) as artifact_fd:
-                comparator_reuse._copy_owner_only(artifact_fd, "copy.jsonl", dir_fd=dir_fd)
-
-    assert (results / "moved" / "copy.jsonl").read_bytes() == payload
-    assert not (outside / "copy.jsonl").exists()
-
-
-@requires_openat
-def test_a_transcript_rewritten_mid_copy_is_refused_not_recorded(tmp_path: Path, monkeypatch) -> None:
-    """The digest has to describe the bytes that were written.
-
-    A held descriptor stops the pathname being substituted; it does not stop the
-    inode being rewritten, and the prior sweep's directory is one this sweep
-    treats as concurrently writable. Hashing the source and then reading it
-    again to copy let the row keep the expected digest while the destination
-    held different bytes.
-    """
-
-    payload = b'{"type":"result"}\n'
-    source = tmp_path / "prior"
-    (source / "transcripts").mkdir(parents=True)
-    transcript = source / "transcripts" / "session-1.jsonl"
-    transcript.write_bytes(payload)
-    dest = tmp_path / "fresh"
-    dest.mkdir()
-    row = _row(transcript_artifacts=[_artifact(payload=payload)])
-
-    # Rewrite the inode in the window the copy reads through — same length, so
-    # only the digest can tell, which is the point.
-    real_read = comparator_reuse.os.read
-    rewritten = {"done": False}
-
-    def rewrite_then_read(fd: int, size: int) -> bytes:
-        if not rewritten["done"]:
-            rewritten["done"] = True
-            with open(transcript, "r+b") as handle:
-                handle.write(b'{"type":"TAMPER"}')
-        return real_read(fd, size)
-
-    monkeypatch.setattr(comparator_reuse.os, "read", rewrite_then_read)
-    with pytest.raises(SandboxError, match="drifted"):
-        materialize_reused_row(row, source_dir=source, dest_dir=dest)
-    monkeypatch.undo()
-
-    # And nothing unvouched-for is left behind for the proposer to read.
-    assert not (dest / "transcripts" / "session-1.jsonl").exists()
-
-
-@requires_openat
-def test_materialize_rejects_same_directory_and_missing_transcript(tmp_path: Path) -> None:
-    source = tmp_path / "prior"
-    source.mkdir()
-    row = _row()
-    with pytest.raises(SandboxError, match="same results directory"):
-        materialize_reused_row(row, source_dir=source, dest_dir=source)
-    dest = tmp_path / "fresh"
-    dest.mkdir()
-    with pytest.raises(SandboxError, match="missing"):
-        materialize_reused_row(row, source_dir=source, dest_dir=dest)
-
-
-@requires_openat
-def test_a_reused_row_ages_from_its_first_measurement_not_the_copy():
-    """Reuse chains must not refresh the clock.
-
-    materialize_reused_row restamps recorded_at with the copy time, so aging
-    against that field let a row be copied forward every generation and outlive
-    max_age forever. The original measurement time is the one that counts.
-    """
-
-    original = (datetime.now(UTC) - timedelta(days=91)).isoformat()
-    chained = _row(recorded_at=datetime.now(UTC).isoformat(), reused_from_recorded_at=original)
-    assert row_is_reusable_comparator(chained, _expected()) is False
-    # The same row inside the window is still reusable.
-    fresh = _row(
-        recorded_at=datetime.now(UTC).isoformat(),
-        reused_from_recorded_at=(datetime.now(UTC) - timedelta(days=1)).isoformat(),
-    )
-    assert row_is_reusable_comparator(fresh, _expected()) is True
-
-
-def test_a_future_dated_row_is_corrupt_not_fresh():
-    ahead = (datetime.now(UTC) + timedelta(days=2)).isoformat()
-    assert row_is_reusable_comparator(_row(recorded_at=ahead), _expected()) is False
-
-
-def test_a_changed_sandbox_dependency_is_not_the_same_baseline():
-    """The environment is part of the measurement.
-
-    This branch itself changes `sandbox_dependencies` in the review corpus, so a
-    prior row measured against the old set is a measurement of a different
-    machine. Reusing it would compare a fresh candidate to a baseline built
-    somewhere else and hand the promotion gate a false comparison.
-    """
-
-    assert row_is_reusable_comparator(
-        _row(sandbox_dependency_manifest_digest=_digest("other-deps")), _expected()
-    ) is False
-    assert row_is_reusable_comparator(
-        _row(task_asset_manifest_digest=_digest("other-assets")), _expected()
-    ) is False
-    # A row that predates the field is not evidence of agreement either.
-    assert row_is_reusable_comparator(_row(sandbox_dependency_manifest_digest=None), _expected()) is False
-
-
-@pytest.mark.skipif(os.name == "nt", reason="symlink creation may require elevated Windows privileges")
-def test_reuse_directories_allow_a_symlinked_parent_but_not_a_symlinked_leaf(tmp_path: Path):
-    """Pins a deliberate difference from the sandbox's mount-root check.
-
-    proposer_sandbox refuses every symlink hop because a hop changes what an
-    untrusted session is handed. A reuse directory is data, and every file
-    inside it is validated on its own, so a symlinked parent is allowed -
-    rejecting it would break a symlinked artifacts directory or macOS's /var
-    for no gain. The leaf itself must still be a real directory.
-    """
-
-    real = tmp_path / "real"
-    real.mkdir()
-    (real / "inner").mkdir()
-    linked_parent = tmp_path / "linked"
-    linked_parent.symlink_to(real, target_is_directory=True)
-
-    # Reached through a symlinked parent: allowed, and resolved to the real path.
-    # The identity returned alongside it is what pins the root against a swap
-    # between the check and the open; the symlink policy itself is unchanged.
-    resolved, identity = comparator_reuse._resolved_directory(linked_parent / "inner", label="probe")
-    assert resolved == (real / "inner").resolve()
-    inner_stat = (real / "inner").stat()
-    assert identity == (inner_stat.st_dev, inner_stat.st_ino)
-
-    # The leaf itself being a symlink is still refused.
-    with pytest.raises(SandboxError, match="must be a real directory"):
-        comparator_reuse._resolved_directory(linked_parent, label="probe")
-
-
-def test_a_review_row_without_its_artifact_is_not_reusable() -> None:
-    """A score is a claim about evidence, not the evidence itself.
-
-    materialize_reused_row copies the review artifact only when the row names
-    one, so accepting a row without it would carry a scored review forward with
-    nothing for a proposer to read.
-    """
-
-    row = _row()
-    assert row_is_reusable_comparator(row, _expected()) is True
-    without = {**row, "review_artifact": ""}
-    assert row_is_reusable_comparator(without, _expected()) is False
-    missing = {k: v for k, v in row.items() if k != "review_artifact"}
-    assert row_is_reusable_comparator(missing, _expected()) is False
-
-
-@requires_openat
-def test_a_reuse_root_replaced_after_the_check_is_refused(tmp_path: Path, monkeypatch) -> None:
-    """Check and use must name the same directory, not the same string.
-
-    _resolved_directory lstats a name and the open re-walks that same name, so
-    a prior sweep that swaps its results root in between is opened somewhere
-    else. The leaf-symlink rule does not cover it - a replacement that is
-    itself a real directory passes every check the policy makes - and the
-    failure is silent, folding another directory's rows into this sweep's
-    comparator baseline.
-    """
-
-    original = tmp_path / "results"
-    original.mkdir()
-    resolved, stale_identity = comparator_reuse._resolved_directory(original, label="probe")
-
-    # Replaced by a different REAL directory: the name still resolves and still
-    # passes the symlink policy, but it is not the inode that was checked.
-    original.rename(tmp_path / "moved")
-    original.mkdir()
-    assert comparator_reuse._resolved_directory(original, label="probe")[1] != stale_identity
-
-    monkeypatch.setattr(
-        comparator_reuse, "_resolved_directory", lambda *_a, **_k: (resolved, stale_identity)
-    )
-    with pytest.raises(SandboxError, match="replaced between the check and the open"):
-        with comparator_reuse._open_pinned_root(original, label="probe"):
-            pass
-
-
-@requires_openat
-def test_a_stable_reuse_root_opens_normally(tmp_path: Path) -> None:
-    """The guard rejects nothing that holds still - a directory matches itself."""
-
-    root = tmp_path / "results"
-    root.mkdir()
-    with comparator_reuse._open_pinned_root(root, label="probe") as fd:
-        assert os.fstat(fd).st_ino == root.stat().st_ino

eval/tests/test_mcp_bridge.py

@@ -1,7 +1,7 @@
 """Tests for MCPBridge._find_gitnexus_command() and subprocess spawn."""
 import subprocess
 import unittest
-from unittest.mock import MagicMock, patch
+from unittest.mock import MagicMock, call, patch
 
 
 class TestFindGitnexusCommand(unittest.TestCase):

eval/tests/test_measure_evolution_cost.py

@@ -1,122 +0,0 @@
-"""Cost model for the evolution wall clock: measured cells, real schedules."""
-
-from __future__ import annotations
-
-import pytest
-
-from workflow_bench.measure_evolution_cost import (
-    CANDIDATE_ARM,
-    SHA_OVERHEAD_SECONDS,
-    DURATIONS_BY_ARM,
-    PROPOSER_SECONDS,
-    REVIEW_ARMS,
-    expected_task_seconds,
-    fed_makespan,
-    fed_pool_enabled,
-    generation_seconds,
-    graph_pipeline_enabled,
-    paid_arms,
-    task_cells,
-    wave_makespan,
-)
-
-
-def test_every_arm_has_its_own_unsorted_sample():
-    assert set(DURATIONS_BY_ARM) == set(REVIEW_ARMS)
-    for arm, sample in DURATIONS_BY_ARM.items():
-        assert len(sample) >= 10, arm
-        # Sorting would hand each task a uniform block and hide the variance
-        # the whole model exists to price.
-        assert list(sample) != sorted(sample), arm
-    assert PROPOSER_SECONDS > 0
-    assert SHA_OVERHEAD_SECONDS > 0
-
-
-def test_weekly_reuse_pays_the_candidate_arm_only():
-    assert paid_arms(weekly=True, reuse_enabled=True) == (CANDIDATE_ARM,)
-    assert paid_arms(weekly=False, reuse_enabled=True) == REVIEW_ARMS
-    assert paid_arms(weekly=True, reuse_enabled=False) == REVIEW_ARMS
-
-
-def test_cells_are_submitted_run_major_arm_minor():
-    # runner.py: [(run_idx, arm) for run_idx in range(runs) for arm in arms].
-    # At workers=3 that puts one cell of each arm in every wave.
-    cells = task_cells(2, REVIEW_ARMS, 0)
-    assert len(cells) == 6
-    expected = [DURATIONS_BY_ARM[arm][run] for run in range(2) for arm in REVIEW_ARMS]
-    assert cells == expected
-
-
-def test_overhead_is_charged_per_sha_and_outside_the_pool():
-    # Two properties at once: the residual sits outside the schedule, where more
-    # workers cannot dissolve it, and it scales with SHAs rather than cells.
-    assert task_cells(1, (CANDIDATE_ARM,), 0) == [DURATIONS_BY_ARM[CANDIDATE_ARM][0]]
-    wide = generation_seconds(
-        task_count=1, runs=3, arms=REVIEW_ARMS, workers=9, fed_pool=True, unique_shas=5
-    )
-    assert wide >= PROPOSER_SECONDS + 5 * SHA_OVERHEAD_SECONDS
-
-
-def test_sweep_overhead_does_not_shrink_with_the_arm_count():
-    """The bias that made weekly look cheaper than it is.
-
-    A seeded weekly generation pays one arm instead of three but builds exactly
-    the same graphs. Charging the residual per cell billed it a third of a cost
-    the real sweep still pays; per SHA, the two attribute the same setup.
-    """
-
-    kwargs = dict(task_count=6, runs=3, workers=3, fed_pool=False, unique_shas=5)
-    weekly = generation_seconds(arms=(CANDIDATE_ARM,), **kwargs)
-    cold = generation_seconds(arms=REVIEW_ARMS, **kwargs)
-    weekly_sessions = 6 * expected_task_seconds(3, (CANDIDATE_ARM,), 3, fed_pool=False)
-    cold_sessions = 6 * expected_task_seconds(3, REVIEW_ARMS, 3, fed_pool=False)
-    # Whatever each wall is, the non-session part is identical.
-    assert round(weekly - weekly_sessions) == round(cold - cold_sessions)
-    # Cycling wraps, so a task can ask for more runs than the sample holds.
-    long_sample = task_cells(len(DURATIONS_BY_ARM[CANDIDATE_ARM]) + 2, (CANDIDATE_ARM,), 0)
-    assert len(long_sample) == len(DURATIONS_BY_ARM[CANDIDATE_ARM]) + 2
-
-
-def test_a_wave_costs_its_slowest_cell_and_a_fed_pool_does_not():
-    slow = [10.0, 1.0, 1.0, 10.0, 1.0, 1.0]
-    assert wave_makespan(slow, 3) == 20.0
-    # Fed: one worker takes the first 10; the second 10 lands on a worker that
-    # has already cleared a 1, and the remaining 1s fill the third.
-    assert fed_makespan(slow, 3) == 11.0
-    assert fed_makespan(slow, 1) == wave_makespan(slow, 1) == 24.0
-
-
-def test_expected_task_seconds_is_alignment_averaged_and_deterministic():
-    waved = expected_task_seconds(3, REVIEW_ARMS, 3, fed_pool=False)
-    assert waved == expected_task_seconds(3, REVIEW_ARMS, 3, fed_pool=False)
-    assert expected_task_seconds(0, REVIEW_ARMS, 3, fed_pool=False) == 0.0
-    assert expected_task_seconds(3, (), 3, fed_pool=False) == 0.0
-    # The barrier can only cost time, never save it.
-    assert waved >= expected_task_seconds(3, REVIEW_ARMS, 3, fed_pool=True)
-
-
-def test_a_generation_pays_one_proposer_session_on_top_of_its_tasks():
-    one = generation_seconds(
-        task_count=1, runs=3, arms=REVIEW_ARMS, workers=3, fed_pool=False, unique_shas=1
-    )
-    two = generation_seconds(
-        task_count=2, runs=3, arms=REVIEW_ARMS, workers=3, fed_pool=False, unique_shas=1
-    )
-    # Each extra task adds exactly one task's makespan. The proposer and the
-    # per-SHA sweep overhead are both paid once, not per task.
-    assert two - one == pytest.approx(
-        one - PROPOSER_SECONDS - SHA_OVERHEAD_SECONDS, abs=2.0
-    )
-
-
-def test_feature_flags_read_the_runner_not_the_wish():
-    assert graph_pipeline_enabled("def _run_sweep(): pass") == 0
-    assert graph_pipeline_enabled("graph_prefetch = GraphPrefetch(...)") == 1
-    assert fed_pool_enabled("def _run_wave(): pass") == 0
-    assert fed_pool_enabled("def _run_fed_pool(): pass") == 1
-
-
-@pytest.mark.parametrize("workers", [1, 3, 8])
-def test_more_workers_never_lengthen_a_task(workers):
-    serial = expected_task_seconds(3, REVIEW_ARMS, 1, fed_pool=True)
-    assert expected_task_seconds(3, REVIEW_ARMS, workers, fed_pool=True) <= serial

eval/tests/test_mock_provider.py

@@ -1,280 +0,0 @@
-"""The mock has to be right about the wire, or every test built on it lies."""
-
-from __future__ import annotations
-
-import json
-import os
-import shutil
-import subprocess
-import urllib.request
-from pathlib import Path
-
-import pytest
-
-from workflow_bench.mock_provider import MockProvider, Reply
-from workflow_bench.provider_usage import (
-    ANTHROPIC,
-    LITELLM_NORMALIZED,
-    OPENAI_RESPONSES,
-    normalize_usage,
-)
-
-
-def _post(url: str, payload: dict) -> tuple[int, bytes]:
-    request = urllib.request.Request(
-        url, data=json.dumps(payload).encode(), headers={"Content-Type": "application/json"}
-    )
-    with urllib.request.urlopen(request, timeout=10) as response:
-        return response.status, response.read()
-
-
-def test_anthropic_messages_returns_a_usable_message() -> None:
-    with MockProvider([Reply(text="reviewed")]) as provider:
-        _status, raw = _post(provider.base_url + "/v1/messages", {"model": "m", "messages": []})
-    body = json.loads(raw)
-    assert body["role"] == "assistant"
-    assert body["content"][0]["text"] == "reviewed"
-    assert body["stop_reason"] == "end_turn"
-
-
-def test_a_scripted_tool_call_is_carried_as_a_tool_use_block() -> None:
-    """Tool blocks are how a mocked run produces real artifacts.
-
-    The CLI executes what it is asked to run, so a Write block makes it write
-    that file for real inside the sandbox - which is how an artifact-producing
-    cell can be exercised with no model involved.
-    """
-
-    write = {"name": "Write", "input": {"file_path": "/review-output/review-output.json", "content": "{}"}}
-    with MockProvider([Reply(text="writing", tools=[write])]) as provider:
-        _status, raw = _post(provider.base_url + "/v1/messages", {"model": "m", "messages": []})
-    body = json.loads(raw)
-    block = body["content"][1]
-    assert block["type"] == "tool_use" and block["name"] == "Write"
-    assert block["input"]["file_path"] == "/review-output/review-output.json"
-    assert body["stop_reason"] == "tool_use", "a turn ending in a tool call must say so"
-
-
-def test_streaming_emits_the_event_sequence_a_consumer_expects() -> None:
-    with MockProvider([Reply(text="hi")]) as provider:
-        request = urllib.request.Request(
-            provider.base_url + "/v1/messages",
-            data=json.dumps({"model": "m", "messages": [], "stream": True}).encode(),
-            headers={"Content-Type": "application/json"},
-        )
-        with urllib.request.urlopen(request, timeout=10) as response:
-            assert response.headers["Content-Type"] == "text/event-stream"
-            body = response.read().decode()
-
-    events = [line[len("event: ") :] for line in body.splitlines() if line.startswith("event: ")]
-    assert events[0] == "message_start"
-    assert events[-1] == "message_stop"
-    assert "content_block_delta" in events
-    # message_delta carries the final usage, which is where output tokens land.
-    assert events[-2] == "message_delta"
-
-
-def test_each_protocol_reports_usage_in_its_own_arithmetic() -> None:
-    """The whole point: the two providers count the same numbers differently.
-
-    Anthropic's cache fields ADD to input_tokens; OpenAI's are SUBSETS of it.
-    Scripting one Reply and serving it both ways is what makes that asymmetry
-    testable without a paid request.
-    """
-
-    reply = Reply(input_tokens=2_000, output_tokens=300, cache_read_input_tokens=7_000, cache_creation_input_tokens=1_000)
-
-    with MockProvider([reply, reply]) as provider:
-        _s, anthropic_raw = _post(provider.base_url + "/v1/messages", {"model": "m", "messages": []})
-        _s, openai_raw = _post(provider.base_url + "/v1/responses", {"model": "m", "input": []})
-
-    anthropic = normalize_usage(ANTHROPIC, json.loads(anthropic_raw)["usage"])
-    openai = normalize_usage(OPENAI_RESPONSES, json.loads(openai_raw)["usage"])
-
-    assert anthropic.total_input_tokens == 10_000
-    assert openai.total_input_tokens == 10_000, "same billed work, stated as the whole"
-    assert anthropic.ordinary_input_tokens == 2_000
-    assert openai.ordinary_input_tokens == 2_000, "recovered by subtraction, not addition"
-    assert openai.cache_read_input_tokens == 7_000
-
-
-def test_a_scripted_failure_is_returned_as_one() -> None:
-    """Billed failures are part of what the accounting must survive."""
-
-    with MockProvider([Reply(status_code=529, error_body={"error": {"type": "overloaded_error"}})]) as provider:
-        try:
-            _post(provider.base_url + "/v1/messages", {"model": "m", "messages": []})
-            raise AssertionError("the scripted failure was not returned")
-        except urllib.error.HTTPError as exc:
-            assert exc.code == 529
-
-
-def test_requests_are_recorded_for_assertions() -> None:
-    with MockProvider() as provider:
-        _post(provider.base_url + "/v1/messages", {"model": "claude-sonnet-4-5", "messages": [{"role": "user"}]})
-    assert len(provider.requests) == 1
-    assert provider.requests[0].body["model"] == "claude-sonnet-4-5"
-    assert provider.requests[0].path.endswith("/v1/messages")
-
-
-def test_an_unscripted_turn_gets_the_default_rather_than_stalling() -> None:
-    """A real run makes more calls than a test wants to enumerate."""
-
-    with MockProvider([Reply(text="first")], default=Reply(text="fallback")) as provider:
-        _s, one = _post(provider.base_url + "/v1/messages", {"model": "m", "messages": []})
-        _s, two = _post(provider.base_url + "/v1/messages", {"model": "m", "messages": []})
-    assert json.loads(one)["content"][0]["text"] == "first"
-    assert json.loads(two)["content"][0]["text"] == "fallback"
-
-
-def test_a_request_through_the_real_gateway_records_native_usage(tmp_path, monkeypatch) -> None:
-    """The whole stack minus the model: proxy, translation, callback, log.
-
-    This is the path that shipped three separate defects invisible to unit
-    tests - the usage variable never reaching the proxy subprocess, the
-    callback failing to import when loaded by path, and failures never
-    recorded. All three live between the gateway and the provider, which is
-    exactly the span this exercises.
-    """
-
-
-    import yaml
-
-    from workflow_bench import model_gateway
-    from workflow_bench.model_gateway import OpenAIGateway
-    from workflow_bench.provider_usage import USAGE_LOG_ENV_VAR
-
-    if shutil.which("litellm") is None:
-        import pytest
-
-        pytest.skip("litellm console script absent; the proxy cannot start here")
-
-    usage_log = tmp_path / "provider_usage.jsonl"
-    monkeypatch.setenv(USAGE_LOG_ENV_VAR, str(usage_log))
-
-    reply = Reply(input_tokens=2_000, output_tokens=300, cache_read_input_tokens=7_000, cache_creation_input_tokens=1_000)
-    with MockProvider(default=reply) as provider:
-        original = model_gateway.write_openai_litellm_config
-
-        def config(path, names):
-            original(path, names)
-            document = yaml.safe_load(path.read_text())
-            for entry in document["model_list"]:
-                entry["litellm_params"]["api_base"] = f"{provider.base_url}/v1"
-            path.write_text(yaml.safe_dump(document))
-            return path
-
-        monkeypatch.setattr(model_gateway, "write_openai_litellm_config", config)
-        with OpenAIGateway(
-            openai_api_key="mock-key", model_names=["gpt-4.1"], work_dir=tmp_path / "gw", ready_timeout_s=60
-        ) as gateway:
-            request = urllib.request.Request(
-                gateway.base_url + "/v1/messages",
-                data=json.dumps({"model": "gpt-4.1", "max_tokens": 32, "messages": [{"role": "user", "content": "ping"}]}).encode(),
-                headers={"Content-Type": "application/json", "x-api-key": gateway.auth_token, "anthropic-version": "2023-06-01"},
-            )
-            with urllib.request.urlopen(request, timeout=60):
-                pass
-
-    assert usage_log.exists(), "the callback never wrote - the env did not reach the proxy"
-    events = [json.loads(line) for line in usage_log.read_text().splitlines()]
-    assert events, "the proxy started but recorded nothing"
-    event = events[-1]
-    native = event["native_usage"]
-    # LiteLLM hands a callback its OWN normalised object, not the upstream body:
-    # an OpenAI Responses reply arrives as prompt_tokens / prompt_tokens_details.
-    # Asserting the wire shape here is what proved the shipped adapter read keys
-    # that are never present.
-    assert native["prompt_tokens_details"]["cached_tokens"] == 7_000
-    assert native["prompt_tokens_details"]["cache_write_tokens"] == 1_000
-    assert event["provider"] == LITELLM_NORMALIZED
-    assert event["call_type"] == "anthropic_messages", "the observed call type, not a Responses one"
-
-    usage = normalize_usage(event["provider"], native)
-    assert usage.total_input_tokens == 10_000
-    assert usage.cache_read_input_tokens == 7_000
-    assert usage.cache_write_input_tokens == 1_000
-    assert usage.ordinary_input_tokens == 2_000
-    assert usage.complete, "a run that cannot interpret its own usage measured nothing"
-
-
-def test_probe_what_identity_the_real_cli_actually_sends(tmp_path: Path) -> None:
-    """An experiment, not an assertion: which fields could correlate a request to a cell?
-
-    Per-cell usage attribution is unbuilt because one proxy serves the whole
-    sweep, so anything read from the proxy environment is identical for every
-    request. Attribution needs something that travels WITH the request, and
-    what the Claude Code CLI actually sends is not documented anywhere I can
-    check - guessing it is how the last three accounting bugs happened.
-
-    So this drives the REAL pinned CLI against the mock and prints the
-    identity-bearing fields that arrive. It asserts only that a request was
-    made; the value is the recorded evidence, which the job log preserves.
-    """
-
-    claude = os.environ.get("CLAUDE_CANARY_BIN")
-    if not claude or not Path(claude).exists():
-        pytest.skip("no pinned Claude CLI here; the containment job supplies CLAUDE_CANARY_BIN")
-
-    with MockProvider(default=Reply(text="ok")) as provider:
-        subprocess.run(
-            [claude, "-p", "--input-format", "text", "--output-format", "stream-json", "--verbose"],
-            input=b"say ok",
-            capture_output=True,
-            timeout=120,
-            env={
-                **os.environ,
-                "ANTHROPIC_BASE_URL": provider.base_url,
-                "ANTHROPIC_API_KEY": "offline-probe",
-                "HOME": str(tmp_path),
-            },
-        )
-
-    assert provider.requests, "the real CLI never reached the mock provider"
-    request = provider.requests[0]
-    interesting = {
-        "header:" + name: value
-        for name, value in request.headers.items()
-        if any(k in name.lower() for k in ("session", "user", "trace", "request-id", "conversation", "metadata"))
-    }
-    interesting.update(
-        {f"body:{key}": request.body[key] for key in ("metadata", "user", "session_id") if key in request.body}
-    )
-    print("\nIDENTITY FIELDS THE REAL CLI SENDS:")
-    print("  body keys:", sorted(request.body))
-    print("  candidate correlators:", interesting or "NONE — per-cell attribution needs another mechanism")
-
-
-def test_scripted_tools_survive_the_responses_protocol_too() -> None:
-    """The gateway uses Responses BECAUSE it carries tool use.
-
-    Emitting only output_text there meant a scripted Write or Skill crossed the
-    gateway with the tool dropped, so a mock claiming to serve both protocols
-    was wrong about the one the gateway actually runs.
-    """
-
-    write = {"name": "Write", "input": {"file_path": "/review-output/review-output.json", "content": "{}"}}
-    with MockProvider([Reply(text="writing", tools=[write])]) as provider:
-        _status, raw = _post(provider.base_url + "/v1/responses", {"model": "m", "input": []})
-
-    output = json.loads(raw)["output"]
-    calls = [item for item in output if item["type"] == "function_call"]
-    assert len(calls) == 1, "the scripted tool must cross the Responses path"
-    assert calls[0]["name"] == "Write"
-    assert json.loads(calls[0]["arguments"])["file_path"] == "/review-output/review-output.json"
-
-
-def test_an_omitted_cache_field_stays_omitted_on_the_responses_wire_too() -> None:
-    """Absence must survive both protocols, not just the Anthropic one.
-
-    `_int_or_none` reads an absent detail key as unknown and a present 0 as a
-    measured zero, so serializing 0 for a scripted None would claim a
-    measurement the reply never made.
-    """
-
-    with MockProvider([Reply(input_tokens=2_000, cache_read_input_tokens=None)]) as provider:
-        _status, raw = _post(provider.base_url + "/v1/responses", {"model": "m", "input": []})
-
-    details = json.loads(raw)["usage"]["input_tokens_details"]
-    assert "cached_tokens" not in details, "an omitted field must not serialize as a measured zero"
-    assert details["cache_write_tokens"] == 0, "a scripted 0 is still a real measurement"

eval/tests/test_model_gateway.py

@@ -1,435 +0,0 @@
-"""Credential routing for the OpenAI loopback gateway."""
-
-from __future__ import annotations
-
-import subprocess
-import os
-import json
-import signal
-import socket
-import time
-import sys
-import threading
-import urllib.request
-from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
-from pathlib import Path
-from unittest import mock
-
-import pytest
-import yaml
-
-
-from workflow_bench.model_gateway import (
-    DEFAULT_GATEWAY_READY_TIMEOUT_S,
-    GATEWAY_READY_TIMEOUT_ENV,
-    GATEWAY_REQUEST_TIMEOUT_S,
-    OpenAIGateway,
-    gateway_ready_timeout_s,
-    anthropic_api_key_from_environ,
-    claude_gateway_model_env,
-    is_openai_model,
-    litellm_proxy_argv,
-    openai_backend_model,
-    openai_litellm_config,
-    resolve_model_access,
-    write_openai_litellm_config,
-)
-
-
-def test_supervisor_reports_proxy_failure_without_aborting_on_its_stdin_reader():
-    supervisor = Path(__file__).resolve().parents[1] / "workflow_bench" / "gateway_supervisor.py"
-    process = subprocess.Popen(
-        [sys.executable, str(supervisor), sys.executable, "-c", "raise RuntimeError('proxy failed')"],
-        stdin=subprocess.PIPE,
-        stdout=subprocess.PIPE,
-        stderr=subprocess.PIPE,
-    )
-    try:
-        # Keep the owner pipe open: the proxy exits independently of its owner.
-        process.wait(timeout=10)
-        assert process.returncode == 1
-        assert b"proxy failed" in process.stderr.read()
-    finally:
-        process.stdin.close()
-        if process.poll() is None:
-            process.kill()
-        process.wait(timeout=5)
-
-
-def test_locked_litellm_translates_messages_to_offline_responses(monkeypatch, tmp_path):
-    from workflow_bench import model_gateway
-
-    observed = []
-
-    class Upstream(BaseHTTPRequestHandler):
-        def log_message(self, *args):
-            pass
-
-        def do_POST(self):
-            body = json.loads(self.rfile.read(int(self.headers["Content-Length"])))
-            observed.append((self.path, self.headers.get("Authorization"), body))
-            response = {
-                "id": "resp_offline",
-                "object": "response",
-                "created_at": int(time.time()),
-                "status": "completed",
-                "model": "gpt-4.1",
-                "error": None,
-                "output": [
-                    {
-                        "id": "msg_offline",
-                        "type": "message",
-                        "role": "assistant",
-                        "status": "completed",
-                        "content": [{"type": "output_text", "text": "offline pong", "annotations": []}],
-                    }
-                ],
-                "usage": {
-                    "input_tokens": 1,
-                    "output_tokens": 2,
-                    "total_tokens": 3,
-                    "input_tokens_details": {"cached_tokens": 0},
-                    "output_tokens_details": {"reasoning_tokens": 0},
-                },
-            }
-            payload = json.dumps(response).encode()
-            self.send_response(200)
-            self.send_header("Content-Type", "application/json")
-            self.send_header("Content-Length", str(len(payload)))
-            self.end_headers()
-            self.wfile.write(payload)
-
-    upstream = ThreadingHTTPServer(("127.0.0.1", 0), Upstream)
-    worker = threading.Thread(target=upstream.serve_forever, daemon=True)
-    worker.start()
-    original = model_gateway.write_openai_litellm_config
-
-    def config(path, names):
-        original(path, names)
-        document = yaml.safe_load(path.read_text())
-        for entry in document["model_list"]:
-            entry["litellm_params"]["api_base"] = f"http://127.0.0.1:{upstream.server_port}/v1"
-        path.write_text(yaml.safe_dump(document))
-        return path
-
-    monkeypatch.setattr(model_gateway, "write_openai_litellm_config", config)
-    try:
-        with OpenAIGateway(
-            openai_api_key="offline-upstream-secret",
-            model_names=["gpt-4.1"],
-            work_dir=tmp_path / "gateway",
-            ready_timeout_s=60,
-        ) as gateway:
-            port = gateway.port
-            request = urllib.request.Request(
-                gateway.base_url + "/v1/messages",
-                data=json.dumps(
-                    {
-                        "model": "gpt-4.1",
-                        "max_tokens": 32,
-                        "messages": [{"role": "user", "content": "ping"}],
-                    }
-                ).encode(),
-                headers={
-                    "Content-Type": "application/json",
-                    "x-api-key": gateway.auth_token,
-                    "anthropic-version": "2023-06-01",
-                },
-            )
-            with urllib.request.urlopen(request, timeout=30) as response:
-                translated = json.load(response)
-            assert "offline pong" in json.dumps(translated)
-            assert len(observed) == 1 and observed[0][0] == "/v1/responses"
-            assert observed[0][1] == "Bearer offline-upstream-secret"
-            assert gateway.auth_token != "offline-upstream-secret"
-        with socket.socket() as client:
-            assert client.connect_ex(("127.0.0.1", port)) != 0
-        gateway.close()  # ownership close is idempotent
-    finally:
-        upstream.shutdown()
-        upstream.server_close()
-        worker.join(timeout=5)
-
-
-@pytest.mark.parametrize("termination", ["terminate", "kill"])
-@pytest.mark.parametrize("phase", ["ready", "startup"])
-def test_gateway_lifetime_ends_with_its_parent(tmp_path, termination, phase):
-    ready = tmp_path / "ready.json"
-    proxy_pid = tmp_path / "proxy-pid"
-    proxy = tmp_path / "proxy.py"
-    proxy.write_text(f"""import os,sys
-from pathlib import Path
-from http.server import BaseHTTPRequestHandler, HTTPServer
-class Handler(BaseHTTPRequestHandler):
-    def do_GET(self):
-        self.send_response({200 if phase == "ready" else 503}); self.end_headers()
-Path(sys.argv[2]).write_text(str(os.getpid()))
-HTTPServer(('127.0.0.1', int(sys.argv[1])), Handler).serve_forever()
-""")
-    parent_code = f"""
-import json,os,sys,time,subprocess
-from pathlib import Path
-from workflow_bench import model_gateway
-model_gateway.litellm_proxy_argv=lambda **kwargs: [sys.executable, {str(proxy)!r}, str(kwargs['port']), {str(proxy_pid)!r}]
-gateway=model_gateway.OpenAIGateway(openai_api_key='offline-secret', model_names=['gpt-4.1'], work_dir=Path({str(tmp_path / "gateway")!r}), ready_timeout_s=10)
-if {phase == "startup"!r}:
-    Path({str(ready)!r}).write_text(json.dumps({{'port':gateway.port}}))
-gateway.__enter__()
-if gateway._process.stdin is not None:
-    assert not os.get_inheritable(gateway._process.stdin.fileno())
-Path({str(ready)!r}).write_text(json.dumps({{'port':gateway.port}}))
-time.sleep(60)
-"""
-    parent = subprocess.Popen(
-        [sys.executable, "-c", parent_code],
-        cwd=Path(__file__).resolve().parents[1],
-        stdout=subprocess.PIPE,
-        stderr=subprocess.PIPE,
-        text=True,
-    )
-    try:
-        deadline = time.monotonic() + 12
-        while (not ready.exists() or not proxy_pid.exists()) and parent.poll() is None and time.monotonic() < deadline:
-            time.sleep(0.02)
-        if not ready.exists():
-            _, stderr = parent.communicate(timeout=1)
-            pytest.fail(stderr)
-        port = json.loads(ready.read_text())["port"]
-        getattr(parent, termination)()
-        parent.wait(timeout=5)
-        deadline = time.monotonic() + 15
-        while time.monotonic() < deadline:
-            with socket.socket() as client:
-                if client.connect_ex(("127.0.0.1", port)) != 0:
-                    break
-            time.sleep(0.02)
-        else:
-            pytest.fail("gateway port survived abrupt parent death")
-    finally:
-        if parent.poll() is None:
-            parent.kill()
-            parent.wait(timeout=5)
-        if proxy_pid.exists():
-            try:
-                if os.name == "nt":
-                    os.kill(int(proxy_pid.read_text()), signal.SIGTERM)
-                else:
-                    os.killpg(int(proxy_pid.read_text()), signal.SIGKILL)
-            except OSError:
-                # Successful ownership cleanup has already removed this process.
-                pass
-
-
-@pytest.mark.parametrize(
-    ("model", "expected"),
-    [
-        ("gpt-4.1", True),
-        ("gpt-4o-mini", True),
-        ("openai/gpt-4.1", True),
-        ("o3", True),
-        ("o4-mini", True),
-        ("claude-sonnet-5", False),
-        ("free-coder", False),
-        ("pinned-model", False),
-    ],
-)
-def test_is_openai_model(model: str, expected: bool) -> None:
-    assert is_openai_model(model) is expected
-
-
-def test_openai_litellm_config_routes_each_id_to_openai_and_env_key(tmp_path: Path) -> None:
-    config = openai_litellm_config(["gpt-4.1", "openai/gpt-4.1-mini", "gpt-4.1"])
-    assert [row["model_name"] for row in config["model_list"]] == ["gpt-4.1", "openai/gpt-4.1-mini"]
-    assert config["model_list"][0]["litellm_params"]["model"] == "openai/gpt-4.1"
-    assert config["model_list"][1]["litellm_params"]["model"] == "openai/gpt-4.1-mini"
-    assert all(row["litellm_params"]["api_key"] == "os.environ/OPENAI_API_KEY" for row in config["model_list"])
-    assert all(row["model_info"] == {"mode": "responses"} for row in config["model_list"])
-    assert all(row["litellm_params"]["timeout"] == GATEWAY_REQUEST_TIMEOUT_S for row in config["model_list"])
-    assert config["litellm_settings"]["request_timeout"] == GATEWAY_REQUEST_TIMEOUT_S
-    path = write_openai_litellm_config(tmp_path / "litellm.yaml", ["gpt-4.1"])
-    assert yaml.safe_load(path.read_text())["model_list"][0]["model_name"] == "gpt-4.1"
-    if os.name != "nt":
-        # Windows chmod exposes a read-only flag, not POSIX access bits.
-        assert path.stat().st_mode & 0o777 == 0o600
-
-
-def test_resolve_model_access_starts_proxy_only_for_openai_ids() -> None:
-    openai = resolve_model_access(
-        auth_token=None,
-        openai_api_key="sk-openai",
-        base_url=None,
-        models=["gpt-4.1", "gpt-4.1"],
-    )
-    assert openai.start_proxy is True
-    assert openai.openai_api_key == "sk-openai"
-
-    anthropic = resolve_model_access(
-        auth_token="sk-ant",
-        openai_api_key="sk-openai",
-        base_url=None,
-        models=["claude-sonnet-5"],
-    )
-    assert anthropic.start_proxy is False
-
-    existing = resolve_model_access(
-        auth_token="proxy-master",
-        openai_api_key=None,
-        base_url="http://127.0.0.1:4000",
-        models=["free-coder"],
-    )
-    assert existing.start_proxy is False
-
-
-def test_resolve_model_access_rejects_openai_ids_without_a_key_and_mixed_providers() -> None:
-    with pytest.raises(ValueError, match="GITNEXUS_BENCH_OPENAI_API_KEY"):
-        resolve_model_access(
-            auth_token="sk-ant",
-            openai_api_key=None,
-            base_url=None,
-            models=["gpt-4.1"],
-        )
-    with pytest.raises(ValueError, match="mix"):
-        resolve_model_access(
-            auth_token=None,
-            openai_api_key="sk-openai",
-            base_url=None,
-            models=["gpt-4.1", "claude-sonnet-5"],
-        )
-    with pytest.raises(ValueError, match="--base-url"):
-        resolve_model_access(
-            auth_token=None,
-            openai_api_key=None,
-            base_url="http://127.0.0.1:4000",
-            models=["free-coder"],
-        )
-
-
-def test_claude_gateway_aliases_pin_every_internal_tier_to_the_session_model() -> None:
-    env = claude_gateway_model_env("gpt-4.1")
-    assert env["ANTHROPIC_MODEL"] == "gpt-4.1"
-    assert env["ANTHROPIC_DEFAULT_HAIKU_MODEL"] == "gpt-4.1"
-    assert env["CLAUDE_CODE_SUBAGENT_MODEL"] == "gpt-4.1"
-    # High-effort reasoning outlives Claude Code's default client timeout.
-    assert env["API_TIMEOUT_MS"] == str(GATEWAY_REQUEST_TIMEOUT_S * 1000)
-
-
-def test_openai_gateway_never_leaves_proxy_output_on_an_undrained_pipe(tmp_path: Path) -> None:
-    # Nothing reads the proxy's output after startup, so a pipe would block the
-    # proxy once its request logs filled the buffer and hang every session.
-    gateway = OpenAIGateway(
-        openai_api_key="sk-openai-secret",
-        model_names=["gpt-4.1"],
-        work_dir=tmp_path / "gw",
-        ready_timeout_s=0.1,
-    )
-    captured: dict[str, object] = {}
-
-    def fake_popen(argv, **kwargs):
-        captured.update(kwargs)
-        raise OSError("no proxy in this test")
-
-    with mock.patch.object(subprocess, "Popen", fake_popen):
-        with pytest.raises(RuntimeError, match="failed to start the OpenAI LiteLLM gateway"):
-            gateway.__enter__()
-
-    assert captured["stderr"] is subprocess.STDOUT
-    assert captured["stdout"] is not subprocess.PIPE
-    assert getattr(captured["stdout"], "name", "") == str(gateway.log_path)
-    if os.name != "nt":
-        assert gateway.log_path.stat().st_mode & 0o777 == 0o600
-
-
-def test_gateway_startup_budget_outlives_a_cold_litellm_import(monkeypatch, tmp_path: Path) -> None:
-    # Importing LiteLLM takes ~17s on a cold container filesystem and the proxy
-    # binds its port only afterwards, so a sub-20s budget fails as "connection
-    # refused" on a proxy that was merely still starting.
-    monkeypatch.delenv(GATEWAY_READY_TIMEOUT_ENV, raising=False)
-    assert DEFAULT_GATEWAY_READY_TIMEOUT_S >= 60
-    assert gateway_ready_timeout_s() == DEFAULT_GATEWAY_READY_TIMEOUT_S
-    assert (
-        OpenAIGateway(
-            openai_api_key="sk-openai-secret",
-            model_names=["gpt-4.1"],
-            work_dir=tmp_path / "gw",
-        ).ready_timeout_s
-        == DEFAULT_GATEWAY_READY_TIMEOUT_S
-    )
-
-    monkeypatch.setenv(GATEWAY_READY_TIMEOUT_ENV, "42.5")
-    assert gateway_ready_timeout_s() == 42.5
-
-    for bad in ("0", "-1", "soon", "nan", "inf", "-inf", "1e999"):
-        monkeypatch.setenv(GATEWAY_READY_TIMEOUT_ENV, bad)
-        with pytest.raises(ValueError, match=GATEWAY_READY_TIMEOUT_ENV):
-            gateway_ready_timeout_s()
-
-
-@pytest.mark.parametrize("timeout", [0.0, -1.0, float("nan"), float("inf"), float("-inf")])
-def test_gateway_rejects_invalid_explicit_readiness_budgets(tmp_path: Path, timeout: float) -> None:
-    with pytest.raises(ValueError, match="finite and positive"):
-        OpenAIGateway(
-            openai_api_key="sk-offline-test",
-            model_names=["gpt-4.1"],
-            work_dir=tmp_path / "gw",
-            ready_timeout_s=timeout,
-        )
-
-
-def test_gateway_readiness_timeout_reports_the_proxy_log_and_the_override(tmp_path: Path) -> None:
-    gateway = OpenAIGateway(
-        openai_api_key="sk-openai-secret",
-        model_names=["gpt-4.1"],
-        work_dir=tmp_path / "gw",
-        ready_timeout_s=0.1,
-    )
-    gateway.work_dir.mkdir(parents=True)
-    gateway.log_path.write_text("ImportError: litellm proxy extras missing")
-
-    with pytest.raises(RuntimeError) as excinfo:
-        gateway._wait_until_ready()
-
-    message = str(excinfo.value)
-    assert "ImportError: litellm proxy extras missing" in message
-    assert GATEWAY_READY_TIMEOUT_ENV in message
-
-
-def test_openai_backend_model_preserves_openai_prefix() -> None:
-    assert openai_backend_model("gpt-4.1") == "openai/gpt-4.1"
-    assert openai_backend_model("openai/gpt-4.1") == "openai/gpt-4.1"
-
-
-def test_litellm_proxy_argv_uses_console_script_not_python_module(tmp_path: Path, monkeypatch) -> None:
-    # litellm 1.87 ships a console script and no litellm.__main__, so
-    # `python -m litellm` dies before the health check. Pin the supported argv.
-    # Under `uv run`, sys.executable is the base CPython — the script lives in
-    # VIRTUAL_ENV/bin instead.
-    python = tmp_path / "base" / "python"
-    venv_bin = tmp_path / "venv" / "bin"
-    python.parent.mkdir(parents=True)
-    venv_bin.mkdir(parents=True)
-    litellm = venv_bin / "litellm"
-    python.write_text("#!/bin/sh\n")
-    litellm.write_text("#!/bin/sh\n")
-    python.chmod(0o755)
-    litellm.chmod(0o755)
-    monkeypatch.setenv("VIRTUAL_ENV", str(tmp_path / "venv"))
-    monkeypatch.delenv("PATH", raising=False)
-    config = tmp_path / "litellm.yaml"
-    config.write_text("model_list: []\n")
-    argv = litellm_proxy_argv(
-        config=config,
-        host="127.0.0.1",
-        port=4010,
-        python_executable=str(python),
-    )
-    assert argv[0] == str(litellm.resolve())
-    assert "-m" not in argv
-    assert argv[1:] == ["--config", str(config), "--host", "127.0.0.1", "--port", "4010"]
-
-
-def test_anthropic_api_key_prefers_the_named_env_and_keeps_the_legacy_alias(monkeypatch) -> None:
-    monkeypatch.delenv("GITNEXUS_BENCH_ANTHROPIC_API_KEY", raising=False)
-    monkeypatch.setenv("GITNEXUS_BENCH_AUTH_TOKEN", "legacy-secret")
-    assert anthropic_api_key_from_environ() == "legacy-secret"
-    monkeypatch.setenv("GITNEXUS_BENCH_ANTHROPIC_API_KEY", "named-secret")
-    assert anthropic_api_key_from_environ() == "named-secret"

eval/tests/test_offline_session_integration.py

@@ -1,195 +0,0 @@
-"""A session end to end with only the model faked.
-
-The layers between the CLI and the row are where this harness has actually
-shipped bugs - the artifact that could not be written, the usage that was never
-recorded, the evidence that was scored from the wrong directory. Every one of
-them sat below the level its tests exercised. These run the real session path
-against a scripted provider, so the only thing not real is what the model says.
-"""
-
-from __future__ import annotations
-
-import sys
-from pathlib import Path
-
-import pytest
-
-from workflow_bench.mock_provider import MockProvider, Reply
-from workflow_bench.proposer_sandbox import (
-    host_workspace_write_boundary,
-    prepare_review_workspace,
-    prepare_sandbox,
-)
-from workflow_bench.review_scoring import REVIEW_OUTPUT, parse_review_output
-from workflow_bench.runner_sessions import run_claude
-
-FAKE_CLI = Path(__file__).parent / "fixtures" / "fake_claude.py"
-REVIEW_JSON = '{"schema_version": 1, "verdict": "approve", "findings": []}'
-
-
-def _session(clone: Path, provider: MockProvider, **overrides):
-    return run_claude(
-        "review the change",
-        clone,
-        claude_bin=str(FAKE_CLI),
-        timeout=60,
-        env={
-            "ANTHROPIC_BASE_URL": provider.base_url,
-            "ANTHROPIC_API_KEY": "offline",
-            "PATH": "/usr/bin:/bin",
-        },
-        **overrides,
-    )
-
-
-@pytest.fixture
-def clone(tmp_path: Path) -> Path:
-    workspace = tmp_path / "clone"
-    workspace.mkdir()
-    (workspace / "source.ts").write_text("export const answer = 42;\n")
-    return workspace
-
-
-def test_a_session_records_the_usage_the_provider_reported(clone: Path) -> None:
-    """Token counts must survive the CLI boundary, not be invented after it."""
-
-    reply = Reply(input_tokens=2_000, output_tokens=300, cache_read_input_tokens=7_000, cache_creation_input_tokens=1_000)
-    with MockProvider(default=reply) as provider:
-        record = _session(clone, provider)
-
-    assert record["ok"] is True, record.get("error_detail")
-    assert record["input_tokens"] == 2_000
-    assert record["cache_read_input_tokens"] == 7_000
-    assert record["cache_creation_input_tokens"] == 1_000
-    assert record["output_tokens"] == 300
-    # A measured zero would be indistinguishable from an unmeasured one.
-    assert record["cost_usd"] == 0.42
-    assert record["num_turns"] == 1
-
-
-def test_a_scripted_write_produces_a_review_artifact_the_scorer_accepts(clone: Path) -> None:
-    """The full artifact path: model asks, CLI writes atomically, scorer reads.
-
-    This is the operation that shipped empty for a whole run. Nothing here
-    fakes the write, the directory, or the parse - only the decision to write.
-    """
-
-    with prepare_sandbox(
-        clone=clone, claude_bin=Path(sys.executable), backend="host-unsafe", preflight=False
-    ) as sandbox:
-        artifact = prepare_review_workspace(sandbox, REVIEW_OUTPUT)
-        write = {"name": "Write", "input": {"file_path": str(artifact), "content": REVIEW_JSON}}
-        with MockProvider(default=Reply(text="reviewing", tools=[write])) as provider:
-            # Take the command configuration from the sandbox the way run_arm
-            # does, rather than calling run_claude bare. On host-unsafe the
-            # prefix is [] by construction, so this pins the WIRING, not the
-            # isolation - a bwrap run would carry a real prefix through here.
-            record = _session(
-                clone,
-                provider,
-                command_prefix=sandbox.command_prefix_for(),
-                require_pid_namespace=sandbox.require_pid_namespace,
-            )
-            assert record["ok"] is True, record.get("error_detail")
-            # Read inside the scope: prepare_sandbox removes the private root on exit.
-            verdict, findings = parse_review_output(artifact)
-
-    assert verdict == "approve"
-    assert findings == ()
-
-
-def test_the_provider_saw_the_prompt_the_harness_meant_to_send(clone: Path) -> None:
-    """A run that measures the wrong prompt measures nothing."""
-
-    with MockProvider() as provider:
-        _session(clone, provider)
-
-    assert provider.requests, "the session never reached the provider"
-    sent = provider.requests[0].body["messages"][0]["content"]
-    assert "review the change" in sent
-
-
-def test_a_provider_failure_surfaces_as_a_failed_session_not_a_silent_pass(clone: Path) -> None:
-    """An upstream 529 must not be recorded as a usable measurement."""
-
-    failing = Reply(status_code=529, error_body={"error": {"type": "overloaded_error"}})
-    with MockProvider(default=failing) as provider:
-        record = _session(clone, provider)
-
-    assert record["ok"] is False
-    assert record["error_kind"] is not None
-
-
-def test_the_write_boundary_refuses_the_workspace_and_permits_the_artifact(clone: Path, tmp_path: Path) -> None:
-    """The contract the empty-artifact run violated, on the backend available here.
-
-    A review must not change the workspace, and must still be able to write its
-    artifact ATOMICALLY - temp file beside the target, then rename - which is
-    what needs a writable parent DIRECTORY rather than a writable file. Both
-    halves are asserted through the real session, with the real boundary
-    applied, and the model scripted to attempt each one.
-
-    Scope: this is the host-unsafe boundary, which its own docstring calls
-    best-effort because a session that can chmod can undo it. The kernel-enforced
-    version is bubblewrap's --ro-bind, which needs namespaces this machine cannot
-    create; that half stays with the real-sandbox canary in CI.
-    """
-
-    artifacts = tmp_path / "artifacts"
-    artifacts.mkdir()
-    target = artifacts / REVIEW_OUTPUT
-    protected = clone / "source.ts"
-    before = protected.read_text()
-
-    write_artifact = {"name": "Write", "input": {"file_path": str(target), "content": REVIEW_JSON}}
-    tamper = {"name": "Write", "input": {"file_path": str(protected), "content": "tampered"}}
-
-    # No writable= entry: the boundary only governs paths INSIDE the workspace
-    # (it refuses one that escapes), and the artifact directory deliberately
-    # lives outside it - that relocation is the fix for the empty-artifact run.
-    with host_workspace_write_boundary(clone):
-        with MockProvider(default=Reply(text="writing", tools=[write_artifact, tamper])) as provider:
-            record = _session(clone, provider)
-
-    assert record["ok"] is True, record.get("error_detail")
-    # The artifact landed, written the way the agent's Write tool does it.
-    verdict, _findings = parse_review_output(target)
-    assert verdict == "approve"
-    assert not list(artifacts.glob("*.tmp.*")), "the rename landed rather than a copy"
-    # The workspace did not move.
-    assert protected.read_text() == before, "the read-only workspace was modified"
-
-
-def test_a_reply_missing_cache_usage_is_refused_not_zero_filled(clone: Path) -> None:
-    """An omitted cache field must not arrive as a measured zero.
-
-    The parent already demands all four USAGE_FIELDS before it calls a session
-    measured (runner_sessions.well_formed). The stand-in used to default the
-    absent ones to 0, which both fabricated a complete measurement AND made
-    that parent guard unfirable from any offline test - it was always
-    satisfied. Scripting the absence is what proves the guard still fires.
-    """
-
-    partial = Reply(input_tokens=2_000, output_tokens=300, cache_read_input_tokens=None)
-    with MockProvider(default=partial) as provider:
-        record = _session(clone, provider)
-
-    assert record["ok"] is False, "an incomplete usage report is not a usable measurement"
-    assert record["error_kind"] == "session-error"
-
-
-@pytest.mark.parametrize("bad", [-5, True, "1200"], ids=["negative", "boolean", "string"])
-def test_a_nonsense_cache_value_is_refused_rather_than_forwarded(clone: Path, bad: object) -> None:
-    """A field good enough to report is good enough to validate.
-
-    The parent's well_formed check tests only that the four keys are PRESENT,
-    so an unvalidated cache value would ride into a success result and be
-    recorded as a real measurement.
-    """
-
-    reply = Reply(input_tokens=2_000, output_tokens=300)
-    object.__setattr__(reply, "cache_read_input_tokens", bad)
-    with MockProvider(default=reply) as provider:
-        record = _session(clone, provider)
-
-    assert record["ok"] is False, f"{bad!r} must not be recorded as a measured cache value"

eval/tests/test_offline_sweep_integration.py

@@ -1,348 +0,0 @@
-"""A whole sweep, offline: real runner, real sessions, scripted model.
-
-The layers between a model turn and a promotion decision had never been
-exercised together. Unit tests covered each in isolation and the paid runs that
-would have covered the composition kept dying, so the contracts BETWEEN them
-went unverified - and that is where this harness has repeatedly shipped bugs.
-
-This drives runner.main() the way the workflow does. Everything is real: task
-selection, hidden-oracle capture, the sandbox, the CLI subprocess, artifact
-capture, review scoring against the oracle, aggregation, the health guard, and
-the promotion gate. Only the model is scripted, through MockProvider.
-
-Two provisioning steps are stubbed because this environment cannot supply them,
-and neither is harness logic: the pinned gitnexus runtime mounts (no
-node_modules in a worktree) and the sanitized graph build (needs the gitnexus
-CLI at a mounted path). Containment is host-unsafe here; bubblewrap stays with
-the real-sandbox canary in the containment job.
-"""
-
-from __future__ import annotations
-
-import json
-import re
-import subprocess
-import sys
-from pathlib import Path
-from types import SimpleNamespace
-
-import os
-import shutil
-
-import pytest
-
-from workflow_bench import oracle_assets, runner
-from workflow_bench.mock_provider import MockProvider, Reply
-
-FAKE_CLI = Path(__file__).parent / "fixtures" / "fake_claude.py"
-ARMS = ("ce_review", "review", "candidate_review")
-
-# When set, the sweep runs with NOTHING provisioning-stubbed: real bubblewrap
-# containment, the real pinned runtime mounts, and the real sanitized graph
-# build. The named CI job installs all three, so a missing one there is a
-# regression rather than an unsupported machine - it FAILS instead of quietly
-# degrading to the stubbed path, which is the whole point of the gate.
-FULL_SWEEP_ENV = "GITNEXUS_REQUIRE_FULL_SWEEP"
-FULL_SWEEP = os.environ.get(FULL_SWEEP_ENV) == "1"
-
-# The runner refuses --unsafe-no-bwrap whenever CI is set, because that mode runs
-# sessions with bypassPermissions behind a boundary its own docstring calls "not
-# a security boundary". Deleting CI to get past that refusal would run an
-# uncontained agent sweep on the runner holding the checkout and credentials, so
-# the stubbed path is skipped under CI instead. The containment job sets
-# GITNEXUS_REQUIRE_FULL_SWEEP=1 and takes the real bubblewrap path, so CI keeps
-# its coverage; only the uncontained convenience run is given up.
-pytestmark = pytest.mark.skipif(
-    not FULL_SWEEP and bool(os.environ.get("CI")),
-    reason="an uncontained sweep must not run in CI; the containment job runs it with GITNEXUS_REQUIRE_FULL_SWEEP=1",
-)
-
-# The review output and the hidden labels are DELIBERATELY different shapes -
-# the labels carry line_start/line_end and no recommendation. Only a real run
-# surfaces that; it is why these are written out rather than shared.
-FINDING = {
-    "id": "f1", "severity": "high", "category": "correctness", "path": "src/sum.js",
-    "line": 1, "end_line": 1, "blocking": True, "scenario": "review-defect",
-    "evidence": "export const total = (a, b) => a - b;", "recommendation": "use a + b",
-}
-LABEL = {"id": "f1", "severity": "high", "category": "correctness",
-         "path": "src/sum.js", "line_start": 1, "line_end": 1}
-SECOND_LABEL = {"id": "f2", "severity": "high", "category": "correctness",
-                "path": "src/scale.js", "line_start": 1, "line_end": 1}
-SECOND_FINDING = {
-    "id": "f2", "severity": "high", "category": "correctness", "path": "src/scale.js",
-    "line": 1, "end_line": 1, "blocking": True, "scenario": "review-defect",
-    "evidence": "export const twice = (n) => n + 2;", "recommendation": "use n * 2",
-}
-
-
-def _git(repo: Path, *args: str) -> str:
-    return subprocess.run(["git", "-C", str(repo), *args], check=True,
-                          capture_output=True, text=True).stdout.strip()
-
-
-@pytest.fixture
-def bench(tmp_path: Path):
-    """A self-contained corpus: one repo, one task, one hidden label."""
-
-    repo = tmp_path / "repo"
-    (repo / "src").mkdir(parents=True)
-    (repo / "src" / "sum.js").write_text("export const total = (a, b) => a - b;\n")
-    (repo / "src" / "scale.js").write_text("export const twice = (n) => n + 2;\n")
-    _git(repo, "init", "-q", ".")
-    _git(repo, "config", "user.email", "t@t")
-    _git(repo, "config", "user.name", "t")
-    _git(repo, "add", "-A")
-    _git(repo, "commit", "-q", "-m", "fixture")
-    sha = _git(repo, "rev-parse", "HEAD")
-
-    oracles = tmp_path / "oracles"
-    oracles.mkdir()
-    (oracles / "review-fixture-defect.labels.json").write_text(
-        json.dumps({"schema_version": 1, "findings": [LABEL]})
-    )
-    (oracles / "review-fixture-second.labels.json").write_text(
-        json.dumps({"schema_version": 1, "findings": [SECOND_LABEL]})
-    )
-
-    tasks = tmp_path / "tasks.yaml"
-    tasks.write_text(
-        "tasks:\n"
-        "  - id: review-fixture-defect\n"
-        "    class: review-defect\n"
-        f"    repo: {repo}\n"
-        f"    ref: {sha}\n"
-        "    prompt: Review this change and report actionable defects.\n"
-        '    verify: test -s "$GITNEXUS_BENCH_REVIEW_OUTPUT"\n'
-        "    oracle:\n"
-        '      command: test -s "$GITNEXUS_BENCH_REVIEW_OUTPUT"\n'
-        "      files: [{ source: review-fixture-defect.labels.json, target: review-labels.json }]\n"
-        # A SECOND task, because the thing a cross-task scheduler changes is
-        # invisible with one: waves are per-task, so a single task cannot show
-        # ordering, packing, or a breaker that spans a task boundary.
-        "  - id: review-fixture-second\n"
-        "    class: review-defect\n"
-        f"    repo: {repo}\n"
-        f"    ref: {sha}\n"
-        "    prompt: Review the scaling helper and report actionable defects.\n"
-        '    verify: test -s "$GITNEXUS_BENCH_REVIEW_OUTPUT"\n'
-        "    oracle:\n"
-        '      command: test -s "$GITNEXUS_BENCH_REVIEW_OUTPUT"\n'
-        "      files: [{ source: review-fixture-second.labels.json, target: review-labels.json }]\n"
-    )
-
-    plugin = tmp_path / "ce-plugin"
-    (plugin / ".claude-plugin").mkdir(parents=True)
-    (plugin / ".claude-plugin" / "plugin.json").write_text(
-        json.dumps({"name": "compound-engineering", "version": "0.0.0-fixture"})
-    )
-    for skill in ("ce-plan", "ce-work", "ce-code-review"):
-        directory = plugin / "skills" / skill
-        directory.mkdir(parents=True)
-        (directory / "SKILL.md").write_text(f"---\nname: {skill}\ndescription: fixture\n---\nFixture.\n")
-
-    overlay = tmp_path / "overlay" / ".claude" / "skills" / "gitnexus-review"
-    overlay.mkdir(parents=True)
-    (overlay / "SKILL.md").write_text("---\nname: gitnexus-review\ndescription: fixture\n---\nCandidate.\n")
-
-    return SimpleNamespace(tasks=tasks, oracles=oracles, plugin=plugin,
-                           overlay=tmp_path / "overlay", out=tmp_path / "out")
-
-
-def _stub_provisioning(monkeypatch: pytest.MonkeyPatch) -> None:
-    """Replace what this machine cannot supply - and nothing else.
-
-    Under FULL_SWEEP nothing is replaced: the runtime mounts and the graph are
-    built for real, so the sweep exercises containment and provisioning too.
-    """
-
-    if FULL_SWEEP:
-        if shutil.which("bwrap") is None:
-            pytest.fail(f"{FULL_SWEEP_ENV}=1 but bubblewrap is absent")
-        return
-
-    monkeypatch.setattr(runner, "trusted_gitnexus_runtime_mounts", lambda: ())
-
-    def materialize(worktree, *, sanitized_head=None, **_kwargs):
-        # The one-clone registry guard reads this before any session runs.
-        meta = Path(worktree) / ".gitnexus"
-        meta.mkdir(parents=True, exist_ok=True)
-        (meta / "meta.json").write_text(
-            json.dumps({"indexedAt": "2026-09-08T00:00:00Z", "lastCommit": sanitized_head or "0" * 40})
-        )
-
-    def fake_graph(**kwargs):
-        kwargs["env"].graph_snapshots[kwargs["graph_key"]] = SimpleNamespace(
-            digest="fixture-graph", manifest_digest="fixture-graph-manifest",
-            dependency_content_digest=None, dependency_manifest_digest=None,
-            materialize=materialize,
-        )
-
-    monkeypatch.setattr(runner, "ensure_task_graph", fake_graph)
-
-
-def _sweep(bench, monkeypatch: pytest.MonkeyPatch, findings: list[dict], verdict: str, *, invoke_skill: bool = True):
-    """Run the real CLI against a model scripted to return `findings`."""
-
-    _stub_provisioning(monkeypatch)
-    monkeypatch.setattr(
-        oracle_assets, "ORACLE_ROOT", bench.oracles, raising=False
-    )
-    monkeypatch.setattr(
-        runner, "capture_task_oracles",
-        lambda tasks, root=bench.oracles: oracle_assets.capture_task_oracles(tasks, root=root),
-    )
-    def review_for(body: str) -> str:
-        # Per task: the second task's defect is in another file, so replying
-        # with the first task's finding would score it wrong. A cross-task
-        # scheduler makes which task a request belongs to load-bearing.
-        chosen = findings
-        if findings and "scaling helper" in body:
-            chosen = [SECOND_FINDING if f is FINDING else f for f in findings]
-        return json.dumps({"schema_version": 1, "verdict": verdict, "findings": chosen})
-
-    class Scripted(MockProvider):
-        def next_reply(self) -> Reply:
-            body = json.dumps(self.requests[-1].body if self.requests else {})
-            target = re.search(r"(/[^\s\"']*review-output\.json)", body)
-            skill = re.search(r"\b(gitnexus-review|ce-code-review)\b", body)
-            return Reply(
-                text="reviewing",
-                tools=[
-                    # The evidence gate needs a Skill request with a non-error
-                    # result: a review that never invoked its skill measured the
-                    # model, not the skill.
-                    *([{"name": "Skill", "input": {"skill": skill.group(1) if skill else "gitnexus-review"}}]
-                      if invoke_skill else []),
-                    {"name": "Write", "input": {
-                        "file_path": target.group(1) if target else str(bench.out / "unmatched-review-output.json"),
-                        "content": review_for(body)}},
-                ],
-                input_tokens=2_000, output_tokens=300,
-                cache_read_input_tokens=7_000, cache_creation_input_tokens=1_000,
-            )
-
-    with Scripted() as provider:
-        monkeypatch.setattr(sys, "argv", [
-            "runner", "--tasks", str(bench.tasks), "--arms", *ARMS,
-            "--runs", "1", "--workers", "1", "--out", str(bench.out),
-            "--base-url", provider.base_url, "--anthropic-api-key", "offline",
-            "--claude-bin", str(FAKE_CLI),
-            *([] if FULL_SWEEP else ["--unsafe-no-bwrap"]),
-            "--model", "mock-model",
-            "--ce-plugin-dir", str(bench.plugin), "--ce-plugin-version", "0.0.0-fixture",
-            "--candidate-overlay", str(bench.overlay),
-        ])
-    
-        try:
-            code = runner.main()
-        except SystemExit as exc:
-            code = exc.code
-
-    rows = [json.loads(line) for line in (bench.out / "results.jsonl").read_text().splitlines()]
-    return code, rows, provider
-
-
-def _row(rows: list[dict], arm: str, task: str = "review-fixture-defect") -> dict:
-    return next(r for r in rows if r["arm"] == arm and r["task"] == task)
-
-
-def test_a_correct_review_scores_and_the_sweep_exits_clean(bench, monkeypatch) -> None:
-    """The whole path, green: every arm measured, scored, and accounted for."""
-
-    code, rows, provider = _sweep(bench, monkeypatch, [FINDING], "request_changes")
-
-    assert code in (None, 0), f"sweep did not succeed: {code}"
-    tasks = {"review-fixture-defect", "review-fixture-second"}
-    assert len(rows) == len(ARMS) * len(tasks)
-    assert len(provider.requests) == len(ARMS) * len(tasks), "each cell must reach the provider once"
-    assert {r["task"] for r in rows} == tasks, "both tasks must have run"
-
-    row = _row(rows, "review")
-    assert row["ok"] is True and row["resolved"] is True
-    assert row["skill_invoked"] is True
-    assert (row["review_true_positives"], row["review_false_positives"], row["review_false_negatives"]) == (1, 0, 0)
-    assert row["review_f1"] == 1.0
-    # The provider's own numbers survived the CLI, the parser and the row.
-    assert row["cache_read_input_tokens"] == 7_000
-    assert row["input_tokens"] == 2_000
-
-    # Each task scored against ITS OWN oracle. This is what a cross-task
-    # scheduler puts at risk: interleaving cells from different tasks means a
-    # mis-routed context or artifact scores one task against another's labels,
-    # and both would still look "green" per row.
-    second = _row(rows, "review", task="review-fixture-second")
-    assert second["resolved"] is True and second["review_f1"] == 1.0
-    assert second["review_artifact"] == "review-fixture-second-review-run0.review.json"
-
-    for name in ("results.jsonl", "report.md", "promotion.json"):
-        assert (bench.out / name).is_file(), f"{name} was not written"
-    assert (bench.out / "review-fixture-defect-review-run0.review.json").is_file()
-
-
-def test_one_run_cannot_promote_a_candidate(bench, monkeypatch) -> None:
-    """The gate refuses on insufficient paired runs, and says so."""
-
-    _sweep(bench, monkeypatch, [FINDING], "request_changes")
-    promotion = json.loads((bench.out / "promotion.json").read_text())
-
-    assert promotion["run_status"] == "complete"
-    decision = next(d for d in promotion["decisions"] if d["candidate_arm"] == "candidate_review")
-    assert decision["decision"] == "insufficient_evidence"
-    assert any("valid paired runs" in reason for reason in decision["reasons"])
-
-
-def test_a_finding_in_the_wrong_place_scores_zero_but_stays_valid_evidence(bench, monkeypatch) -> None:
-    """Being wrong is a quality result, not a broken measurement.
-
-    The negative control that makes the passing case mean something: same
-    harness, same well-formed artifact, only the answer changed.
-    """
-
-    wrong = {**FINDING, "path": "src/WRONG.js", "line": 99, "end_line": 99}
-    _code, rows, _provider = _sweep(bench, monkeypatch, [wrong], "request_changes")
-
-    row = _row(rows, "review")
-    assert (row["review_true_positives"], row["review_false_positives"], row["review_false_negatives"]) == (0, 1, 1)
-    assert row["review_f1"] == 0.0
-    assert row["resolved"] is False
-    assert row["error_kind"] == "oracle-failed", "a wrong answer is not a session or evidence failure"
-    assert row["review_evidence_valid"] is True, "the artifact was well formed; only the answer was wrong"
-
-
-def test_approving_defective_code_is_a_miss_with_no_false_positive(bench, monkeypatch) -> None:
-    """The other half of the control: silence scores differently from a wrong guess."""
-
-    _code, rows, _provider = _sweep(bench, monkeypatch, [], "approve")
-
-    row = _row(rows, "review")
-    assert (row["review_true_positives"], row["review_false_positives"], row["review_false_negatives"]) == (0, 0, 1)
-    assert row["review_precision"] is None, "precision is undefined with no predictions, not zero"
-    assert row["review_verdict_correct"] is False, "approving defective code is the wrong verdict"
-    assert row["review_evidence_valid"] is True
-
-
-def test_a_review_that_never_invoked_its_skill_is_not_a_measurement(bench, monkeypatch) -> None:
-    """The gate that separates measuring a SKILL from measuring a model.
-
-    Added because a mutation exposed it: forcing skill_was_invoked_events to
-    return True left every other test here passing, so nothing pinned the gate.
-    The artifact is written and correct in this run - only the skill request is
-    missing - so a pass would mean the arm scored a review it never performed.
-    """
-
-    code, rows, _provider = _sweep(bench, monkeypatch, [FINDING], "request_changes", invoke_skill=False)
-
-    row = _row(rows, "review")
-    assert row["skill_invoked"] is False
-    assert row["error_kind"] == "skill-not-invoked"
-    assert code not in (None, 0), "the sweep must not report success on unusable evidence"
-
-    # The row still carries its own score - the artifact was well formed - and
-    # aggregate() DOES count it in the arm's quality median (the KNOWN GAP noted
-    # above aggregate(); test_workflow_bench pins the resulting 0.5). Filtering
-    # it out of the median alone inverted a promotion, because valid_runs and
-    # excluded_runs kept counting it. It counts for cost either way: the session
-    # ran and was billed.
-    assert row["review_weighted_f1"] == 1.0
-    assert row["review_evidence_valid"] is True

eval/tests/test_oracle_assets.py

@@ -5,9 +5,7 @@ from __future__ import annotations
 import argparse
 import hashlib
 import os
-import shutil
 import subprocess
-import sys
 from pathlib import Path
 from types import SimpleNamespace
 
@@ -15,13 +13,7 @@ import pytest
 
 from workflow_bench import oracle_assets, runner
 from workflow_bench.evolution import evaluate_candidate
-from workflow_bench.oracle_assets import (
+from workflow_bench.oracle_assets import capture_task_oracle, staged_task_oracle
-    capture_task_oracle,
-    require_hidden_harness_absent,
-    review_case_setup_command,
-    staged_task_oracle,
-    with_hidden_harness_apply_exclude,
-)
 
 
 def oracle_task(*, command: str = "true", source: str = "oracle.test.ts") -> dict[str, object]:
@@ -60,7 +52,6 @@ def bench_args() -> argparse.Namespace:
         claude_bin="claude",
         timeout=5,
         model="pinned-model",
-        effort="xhigh",
         base_url=None,
         auth_token=None,
     )
@@ -157,73 +148,6 @@ def test_clone_sanitization_prunes_harness_checkout_and_recoverable_history(tmp_
     assert git(clone, "status", "--porcelain=v1", "--untracked-files=all").stdout == ""
 
 
-def test_require_hidden_harness_absent_fails_closed_on_leftover_tree(tmp_path: Path) -> None:
-    clone = tmp_path / "clone"
-    hidden = clone / "eval" / "workflow_bench"
-    hidden.mkdir(parents=True)
-    (hidden / "review_cases").mkdir()
-
-    with pytest.raises(ValueError, match="hidden harness visible"):
-        require_hidden_harness_absent(clone)
-
-    shutil.rmtree(hidden)
-    require_hidden_harness_absent(clone)
-
-
-def test_hidden_harness_apply_exclude_is_idempotent() -> None:
-    raw = "git apply eval/workflow_bench/review_cases/pr.patch && rm -rf eval/workflow_bench"
-    once = with_hidden_harness_apply_exclude(raw)
-    assert once == review_case_setup_command("pr.patch")
-    assert with_hidden_harness_apply_exclude(once) == once
-    assert with_hidden_harness_apply_exclude("true") == "true"
-
-
-def test_review_setup_skips_sanitized_harness_hunks(tmp_path: Path) -> None:
-    repo = tmp_path / "repo"
-    repo.mkdir()
-
-    def git(*args: str, check: bool = True) -> subprocess.CompletedProcess[str]:
-        result = subprocess.run(
-            ["git", "-C", str(repo), *args],
-            check=False,
-            capture_output=True,
-            text=True,
-        )
-        if check and result.returncode != 0:
-            pytest.fail(f"git {' '.join(args)} failed: {result.stderr}")
-        return result
-
-    git("init", "--quiet", "--initial-branch=main")
-    git("config", "user.name", "Review Setup")
-    git("config", "user.email", "review-setup.invalid")
-    (repo / "visible.py").write_text("old\n")
-    hidden = repo / "eval" / "workflow_bench"
-    hidden.mkdir(parents=True)
-    (hidden / "learnings.jsonl").write_text("{}\n")
-    git("add", "--all")
-    git("commit", "--quiet", "-m", "base with harness file")
-    (repo / "visible.py").write_text("new\n")
-    (hidden / "learnings.jsonl").write_text("{}\nextra\n")
-    patch = git("diff").stdout
-    git("checkout", "--", ".")
-    shutil.rmtree(hidden)
-    patch_path = hidden / "review_cases" / "case.patch"
-    patch_path.parent.mkdir(parents=True)
-    patch_path.write_text(patch)
-
-    rejected = git("apply", "--check", str(patch_path.relative_to(repo)), check=False)
-    assert rejected.returncode != 0
-    assert "learnings.jsonl" in rejected.stderr
-
-    setup = with_hidden_harness_apply_exclude(
-        "git apply eval/workflow_bench/review_cases/case.patch && rm -rf eval/workflow_bench"
-    )
-    applied = subprocess.run(["/bin/sh", "-lc", setup], cwd=repo, check=False, capture_output=True, text=True)
-    assert applied.returncode == 0, applied.stderr
-    assert (repo / "visible.py").read_text() == "new\n"
-    assert not hidden.exists()
-
-
 def test_clone_sanitization_prunes_remote_history_when_head_never_had_harness(tmp_path: Path) -> None:
     source = tmp_path / "source"
     source.mkdir()
@@ -375,26 +299,6 @@ def test_vacuous_authored_test_cannot_self_certify_resolution(
     assert record["error_kind"] == "oracle-failed"
 
 
-def test_host_unsafe_oracle_executes_beside_candidate_and_is_removed(tmp_path: Path) -> None:
-    from workflow_bench.proposer_sandbox import prepare_sandbox
-
-    source = tmp_path / "oracles"
-    write_oracle(
-        source,
-        b"from pathlib import Path\nassert (Path(__file__).resolve().parents[2] / 'candidate.txt').read_text() == 'candidate'\n",
-    )
-    snapshot = capture_task_oracle(
-        oracle_task(command='python3 "$GITNEXUS_BENCH_ORACLE_ROOT/nested/oracle.test.ts"'), root=source
-    )
-    clone = tmp_path / "clone"
-    clone.mkdir()
-    (clone / "candidate.txt").write_text("candidate")
-    with prepare_sandbox(clone=clone, claude_bin=sys.executable, backend="host-unsafe") as session:
-        passed, detail = runner._run_hidden_oracle(snapshot, clone, bench_args(), session)
-        assert passed, detail
-        assert not list(clone.glob(".wfbench-oracle-*"))
-
-
 def test_oracle_path_and_bytes_appear_only_after_the_model_session(
     monkeypatch: pytest.MonkeyPatch,
     tmp_path: Path,

eval/tests/test_process_control.py

@@ -7,12 +7,7 @@ import os
 import signal
 import sys
 import time
-import threading
-import subprocess
-import json
-from concurrent.futures import ThreadPoolExecutor
 from pathlib import Path
-from types import SimpleNamespace
 
 import pytest
 
@@ -28,101 +23,6 @@ from workflow_bench.process_control import (
 PYTHON = sys.executable
 
 
-def test_cancel_before_spawn_never_starts_the_child(tmp_path):
-    event = threading.Event()
-    event.set()
-    sentinel = tmp_path / "spawned"
-    result = run_managed([PYTHON, "-c", f"open({str(sentinel)!r}, 'w').close()"], timeout=60, cancel_event=event)
-    assert result.state == "cancelled"
-    assert not sentinel.exists()
-
-
-@pytest.mark.skipif(os.name == "nt", reason="POSIX signal escalation")
-def test_cancel_after_spawn_kills_a_term_ignoring_child():
-    event = threading.Event()
-    started = time.monotonic()
-    result = run_managed(
-        [
-            PYTHON,
-            "-c",
-            "import signal,time; signal.signal(signal.SIGTERM, signal.SIG_IGN); print('ready',flush=True); time.sleep(60)",
-        ],
-        timeout=60,
-        terminate_grace=0.1,
-        cancel_event=event,
-        stdout_observer=lambda chunk: event.set() if b"ready" in chunk else None,
-    )
-    assert result.state == "cancelled" and result.forced_kill
-    assert not result.timed_out
-    assert time.monotonic() - started < 5
-
-
-@pytest.mark.skipif(os.name == "nt", reason="POSIX parent signals")
-@pytest.mark.parametrize("signum", [signal.SIGINT, signal.SIGTERM])
-def test_signal_cancels_active_wave_before_assets_are_released(tmp_path, signum):
-    ready = tmp_path / "child-pid"
-    assets = tmp_path / "assets"
-    assets.write_text("shared")
-    command = (
-        f"import os,time; from pathlib import Path; Path({str(ready)!r}).write_text(str(os.getpid())); time.sleep(60)"
-    )
-    script = f"""
-import json,sys
-from pathlib import Path
-from workflow_bench.process_control import cancellation_scope, run_managed
-from workflow_bench.runner import sweep_task_cells
-rows=[]
-def run(index, arm):
-    if index == 0:
-        return {{'resolved': True, 'error_kind': None}}
-    result=run_managed([sys.executable, '-c', {command!r}], timeout=60)
-    assert Path({str(assets)!r}).exists(), 'assets removed while a worker was active'
-    return {{'resolved': False, 'error_kind': result.state}}
-with cancellation_scope(handle_signals=True) as event:
-    streak, tripped=sweep_task_cells([(i, 'review') for i in range(10)], workers=2, run=run,
-        on_start=lambda *args: None, on_record=lambda i,a,r: rows.append([i,r]),
-        outage_streak=0, outage_limit=5, cancel_event=event)
-Path({str(assets)!r}).unlink()
-print(json.dumps({{'rows': rows, 'stopped': event.is_set(), 'tripped': tripped}}))
-"""
-    process = subprocess.Popen(
-        [PYTHON, "-c", script],
-        cwd=Path(__file__).resolve().parents[1],
-        stdout=subprocess.PIPE,
-        stderr=subprocess.PIPE,
-        text=True,
-    )
-    try:
-        deadline = time.monotonic() + 10
-        while not ready.exists() and process.poll() is None and time.monotonic() < deadline:
-            time.sleep(0.01)
-        assert ready.exists()
-        child_pid = int(ready.read_text())
-        started = time.monotonic()
-        process.send_signal(signum)
-        stdout, stderr = process.communicate(timeout=15)
-        assert process.returncode == 0, stderr
-        assert time.monotonic() - started < 15
-        report = json.loads(stdout)
-        assert report["stopped"] and not report["tripped"], "cancelled, not an outage"
-        assert [row[0] for row in report["rows"]] == [0, 1]
-        assert report["rows"][0][1]["resolved"] is True
-        assert report["rows"][1][1]["error_kind"] == "cancelled"
-        with pytest.raises(ProcessLookupError):
-            os.kill(child_pid, 0)
-        assert not assets.exists()
-    finally:
-        if process.poll() is None:
-            process.kill()
-            process.wait(timeout=5)
-        if ready.exists():
-            try:
-                os.killpg(int(ready.read_text()), signal.SIGKILL)
-            except ProcessLookupError:
-                # Successful cancellation has already reaped this process group.
-                pass
-
-
 def test_managed_process_captures_normal_exit() -> None:
     result = run_managed(
         [PYTHON, "-c", "import sys; print('out'); print('err', file=sys.stderr)"],
@@ -224,64 +124,6 @@ def test_parent_stdout_capture_reports_overflow_without_stopping_drain() -> None
     assert result.stdout_tail.endswith("END")
 
 
-def test_echo_stdout_streams_child_progress_and_stays_off_by_default(capfd) -> None:
-    command = [PYTHON, "-c", "import os; os.write(1, b'[task][arm][run 0] starting\\n')"]
-
-    quiet = run_managed(command, timeout=5)
-    assert quiet.ok
-    assert "starting" not in capfd.readouterr().err
-
-    echoed = run_managed(command, timeout=5, echo_stdout=True)
-    assert echoed.ok
-    captured = capfd.readouterr()
-    assert "[task][arm][run 0] starting" in captured.err
-    # Echoing is a passthrough, not a redirect: the tail stays intact for the
-    # caller that reports it after the process ends.
-    assert "starting" in echoed.stdout_tail
-
-
-def test_echo_reaches_the_log_while_the_child_is_still_running(tmp_path: Path, monkeypatch) -> None:
-    """Streaming has to be prompt, not merely eventual.
-
-    A sweep emits one line every ~45 minutes. Draining with `read(8192)` still
-    delivers every byte, so the tail and the capture look correct — but nothing
-    surfaces until the pipe closes, which turns a 15-hour job into a silent one
-    and is the whole reason this passthrough exists.
-
-    The child here refuses to exit until the echoed line has been observed, so
-    an implementation that only flushes at EOF deadlocks and fails on the
-    timeout rather than passing on a technicality.
-    """
-    released = tmp_path / "echo-observed"
-
-    class Sink:
-        def write(self, data: bytes) -> int:
-            if b"first-line" in data:
-                released.write_text("go")
-            return len(data)
-
-        def flush(self) -> None:
-            pass
-
-    monkeypatch.setattr(process_control.sys, "stderr", SimpleNamespace(buffer=Sink()))
-    script = """
-import pathlib, sys, time
-sys.stdout.write('first-line\\n')
-sys.stdout.flush()
-target = pathlib.Path(%r)
-for _ in range(400):
-    if target.exists():
-        break
-    time.sleep(0.05)
-""" % str(released)
-
-    result = run_managed([PYTHON, "-c", script], timeout=15, echo_stdout=True)
-
-    assert released.exists(), "the line never reached the echo sink while the child ran"
-    assert result.ok
-    assert "first-line" in result.stdout_tail
-
-
 def test_incomplete_stdin_delivery_cannot_report_success() -> None:
     result = run_managed(
         [PYTHON, "-c", "import os,time; os.close(0); time.sleep(0.05)"],
@@ -611,57 +453,3 @@ def test_windows_normal_parent_with_grandchild_is_not_successful_evidence(tmp_pa
     assert result.forced_kill
     assert not result.ok
     assert not sentinel.exists()
-
-
-@pytest.mark.skipif(os.name == "nt", reason="POSIX process-group ownership canary")
-def test_concurrent_cells_reap_only_their_own_process_tree(tmp_path: Path) -> None:
-    """One cell timing out must not touch a sibling cell running beside it.
-
-    `run_managed` reaps by process group. Cells only ever ran one at a time
-    before, so nothing exercised what happens when a `killpg` fires while other
-    owned trees are alive — a leaked or shared pgid would take the siblings
-    down with it, and the sweep would read that as two more excluded runs.
-    """
-    survivor_sentinel = tmp_path / "survivor-finished"
-    victim_sentinel = tmp_path / "victim-escaped"
-
-    # Each cell spawns a descendant, like a sandboxed session does.
-    survivor = """
-import pathlib, subprocess, sys, time
-child = subprocess.Popen([sys.executable, '-c', "import time; time.sleep(2)"])
-time.sleep(1.0)
-pathlib.Path(%r).write_text('finished')
-child.wait()
-print('survivor-done', flush=True)
-""" % str(survivor_sentinel)
-    victim = """
-import pathlib, signal, subprocess, sys, time
-signal.signal(signal.SIGTERM, signal.SIG_IGN)
-subprocess.Popen([
-    sys.executable, '-c',
-    "import signal,time,pathlib; signal.signal(signal.SIGTERM, signal.SIG_IGN); time.sleep(1.5); pathlib.Path(%r).write_text('escaped')"
-])
-while True:
-    time.sleep(0.01)
-""" % str(victim_sentinel)
-
-    def cell(source: str, timeout: float):
-        return run_managed([PYTHON, "-c", source], timeout=timeout, terminate_grace=0.1)
-
-    with ThreadPoolExecutor(max_workers=3) as pool:
-        futures = [
-            pool.submit(cell, survivor, 10.0),
-            pool.submit(cell, victim, 0.2),
-            pool.submit(cell, survivor, 10.0),
-        ]
-        first, doomed, second = (future.result() for future in futures)
-
-    time.sleep(1.8)
-
-    assert doomed.state == "forced-kill"
-    assert not victim_sentinel.exists(), "the timed-out cell leaked a descendant"
-    # The siblings were mid-flight when the killpg fired.
-    assert first.ok and second.ok
-    assert "survivor-done" in first.stdout_tail
-    assert "survivor-done" in second.stdout_tail
-    assert survivor_sentinel.exists()

eval/tests/test_promotion_apply.py

@@ -3,15 +3,11 @@
 import json
 import os
 import stat
-import shlex
-import subprocess
 from pathlib import Path, PurePosixPath
 
 import pytest
-import yaml
 
 from workflow_bench import evolve, promotion_apply
-from workflow_bench import evolution, runner
 from workflow_bench.evolution import (
     CANDIDATE_SKILLS,
     MAX_CANDIDATE_OVERLAY_BYTES,
@@ -40,120 +36,6 @@ def _git(repo: Path, *arguments: str) -> str:
     )
 
 
-@pytest.mark.parametrize(
-    "arms", [["candidate_review"], ["candidate_workflow"], ["candidate_review", "candidate_workflow"]]
-)
-@pytest.mark.parametrize(
-    "tamper",
-    [
-        None,
-        "aborted",
-        "metric",
-        "verdict",
-        "nan",
-        "huge-int",
-        "runs",
-        "task",
-        "duplicate-task",
-        "exclusions",
-        "old-schema",
-    ],
-)
-def test_produced_promotion_round_trips_to_transactional_apply(tmp_path, arms, tamper):
-    from tests.test_evolve import bound_task_fixture, promotion_fixture
-
-    overlay = tmp_path / "overlay"
-    repo = tmp_path / "repo"
-    for arm in arms:
-        skill = "gitnexus-review" if arm == "candidate_review" else "gitnexus-plan"
-        relative = PurePosixPath(f".claude/skills/{skill}/SKILL.md")
-        (overlay / relative).parent.mkdir(parents=True, exist_ok=True)
-        (overlay / relative).write_text("candidate")
-        for target in mirror_targets(relative):
-            (repo / target).parent.mkdir(parents=True, exist_ok=True)
-            (repo / target).write_text("incumbent")
-    frozen = tmp_path / "frozen"
-    digest = freeze_overlay(overlay, frozen)
-    bases = destination_base_digests(frozen, repo_root=repo)
-    paired = {}
-    for arm in arms:
-        for name, candidate in ((evolution.CANDIDATE_ARMS[arm], False), (arm, True)):
-            paired[name] = runner.aggregate(
-                [
-                    {
-                        "resolved": True,
-                        "cost_usd": 0.8 if candidate else 1.0,
-                        "review_weighted_f1": 1.0 if candidate else 0.5,
-                        "review_blocker_recall": 1.0,
-                        "review_false_positives": 0,
-                        "review_verdict_correct": True,
-                        "review_clean_control": False,
-                        "review_clean_pass": False,
-                    }
-                    for _ in range(3)
-                ]
-            )
-    policy = evolution.promotion_policy(arms)
-    promotion = {
-        **promotion_fixture(),
-        **evolution.promotion_evidence(
-            {"task-a": paired},
-            policy=policy,
-            model="bench-model",
-            complete=True,
-        ),
-        "required_candidate_arms": arms,
-        "candidate_overlay_digest": digest,
-        "target_base_digests": bases,
-        "selected_tasks": [bound_task_fixture()],
-    }
-    promotion = json.loads(json.dumps(promotion))
-    decision = promotion["decisions"][0]
-    row = decision["tasks"][0]
-    if tamper == "aborted":
-        promotion["run_status"] = "aborted"
-    elif tamper == "metric":
-        decision["metric"] = "fabricated"
-    elif tamper == "verdict":
-        decision["decision"] = "keep_incumbent"
-    elif tamper == "nan":
-        row["candidate"][policy[arms[0]]["metric"]] = float("nan")
-    elif tamper == "huge-int":
-        row["candidate"][policy[arms[0]]["metric"]] = 10**1000
-    elif tamper == "runs":
-        row["candidate"]["valid_runs"] = 1
-    elif tamper == "task":
-        row["task"] = "unselected"
-    elif tamper == "duplicate-task":
-        decision["tasks"].append(dict(row))
-    elif tamper == "exclusions":
-        row["candidate"]["excluded_runs"] = 1
-    elif tamper == "old-schema":
-        promotion["schema_version"] = 5
-
-    def validate():
-        return evolve.validate_promotion_for_apply(
-            promotion,
-            overlay_digest=digest,
-            benchmark_model="bench-model",
-            proposer_model="proposer-model",
-            effort="xhigh",
-            selected_tasks=[bound_task_fixture()],
-            target_base_digests=bases,
-            required_candidate_arms=arms,
-            policy=policy,
-        )
-
-    if tamper:
-        with pytest.raises(ValueError):
-            validate()
-        assert all((repo / path).read_text() == "incumbent" for path in bases)
-    else:
-        assert all(decision["decision"] == "promote" for decision in validate())
-        written = apply_promoted_overlay(frozen, repo_root=repo, expected_target_bases=bases)
-        assert all((repo / path).read_text() == "candidate" for path in written)
-
-
 def test_evolve_reexports_public_promotion_helpers():
     assert evolve.mirror_targets is mirror_targets
     assert evolve.freeze_overlay is freeze_overlay
@@ -170,44 +52,6 @@ def test_mirror_targets_cover_canonical_and_shipped_copies():
     ]
 
 
-def test_review_mirror_targets_include_cursor_distribution():
-    targets = mirror_targets(PurePosixPath(".claude/skills/gitnexus-review/SKILL.md"))
-    assert targets == [
-        PurePosixPath(".claude/skills/gitnexus-review/SKILL.md"),
-        PurePosixPath("gitnexus/skills/gitnexus-review/SKILL.md"),
-        PurePosixPath("gitnexus-claude-plugin/skills/gitnexus-review/SKILL.md"),
-        PurePosixPath("gitnexus-cursor-integration/skills/gitnexus-review/SKILL.md"),
-    ]
-
-
-def test_workflow_stages_every_review_mirror_and_rejects_unrelated_changes(tmp_path):
-    overlay = tmp_path / "overlay"
-    relative = PurePosixPath(".claude/skills/gitnexus-review/SKILL.md")
-    (overlay / relative).parent.mkdir(parents=True)
-    (overlay / relative).write_text("candidate")
-    repo = tmp_path / "repo"
-    targets = mirror_targets(relative)
-    for path in targets:
-        (repo / path).parent.mkdir(parents=True, exist_ok=True)
-        (repo / path).write_text("incumbent")
-    (repo / "unrelated.txt").write_text("before")
-    _git(repo, "init", "-q")
-    _git(repo, "add", ".")
-    _git(repo, "-c", "user.name=Fixture", "-c", "user.email=fixture@example.test", "commit", "-qm", "base")
-    apply_promoted_overlay(overlay, repo_root=repo)
-    workflow_path = Path(__file__).resolve().parents[2] / ".github/workflows/gitnexus-skill-evolution.yml"
-    steps = yaml.safe_load(workflow_path.read_text())["jobs"]["evolve"]["steps"]
-    publish = next(step["run"] for step in steps if step.get("name") == "Open the promotion PR")
-    staging = next(line for line in publish.splitlines() if line.startswith("git add "))
-    _git(repo, *shlex.split(staging)[1:])
-    assert _git(repo, "diff", "--cached", "--name-only").splitlines() == sorted(map(str, targets))
-    assert {(repo / path).read_text() for path in targets} == {"candidate"}
-    (repo / "unrelated.txt").write_text("after")
-    guard = next(step["run"] for step in steps if step.get("name") == "Detect and bound the applied promotion")
-    guarded = subprocess.run(["bash", "-c", guard], cwd=repo, capture_output=True, text=True)
-    assert guarded.returncode == 1 and "outside the skill trees: unrelated.txt" in guarded.stdout
-
-
 def test_apply_promoted_overlay_writes_all_mirrors(tmp_path):
     overlay = tmp_path / "overlay"
     skill_md = overlay / ".claude" / "skills" / "gitnexus-plan" / "SKILL.md"
@@ -648,7 +492,14 @@ def test_committed_destination_bases_ignore_and_reject_live_target_edits(tmp_pat
     assert dirty.read_text() == "user edit"
 
 
-def test_mirror_roots_cover_every_candidate_skill_including_cursor_review():
+def test_mirror_roots_cover_every_candidate_skill_and_omit_none_that_ships_to_cursor():
+    # promotion_apply.mirror_targets writes canonical + MIRROR_SKILL_ROOTS, which
+    # today omits the Cursor tree. That is only safe because no candidate skill is
+    # cursor-shipped. If a future edit adds a cursor-shipped skill (e.g.
+    # gitnexus-review) to CANDIDATE_SKILLS, apply_promoted_overlay would rewrite
+    # the other trees and silently skip Cursor — the PR #2488 asymmetric-sync bug
+    # class. Pin the invariant to the filesystem, the source of truth the TS drift
+    # guard already enforces.
     repo_root = Path(__file__).resolve().parents[2]
     cursor_root = repo_root / "gitnexus-cursor-integration" / "skills"
     for skill in sorted(CANDIDATE_SKILLS):
@@ -656,9 +507,10 @@ def test_mirror_roots_cover_every_candidate_skill_including_cursor_review():
         assert canonical.is_dir(), f"candidate skill {skill} has no canonical .claude/skills dir"
         for target in mirror_targets(PurePosixPath(".claude", "skills", skill, "SKILL.md")):
             assert (repo_root / target).is_file(), f"candidate skill mirror missing on disk: {target}"
-        if (cursor_root / skill).exists():
+        assert not (cursor_root / skill).exists(), (
-            expected = PurePosixPath("gitnexus-cursor-integration/skills", skill, "SKILL.md")
+            f"candidate skill {skill} ships to Cursor, but MIRROR_SKILL_ROOTS does not cover "
-            assert expected in mirror_targets(PurePosixPath(".claude", "skills", skill, "SKILL.md"))
+            "gitnexus-cursor-integration/skills — promotion would sync it asymmetrically"
+        )
 
 
 def test_committed_destination_bases_reject_overlay_adding_uncommitted_target(tmp_path):

eval/tests/test_proposer_sandbox.py

@@ -9,130 +9,37 @@ import stat
 import subprocess
 import sys
 import threading
-from dataclasses import replace
 from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
 from pathlib import Path
-from types import SimpleNamespace
 
 import pytest
 
-from workflow_bench import runner, runner_artifacts
+from workflow_bench import runner
-from workflow_bench import proposer_sandbox
 
 from workflow_bench.process_control import ManagedProcessResult, run_managed
-
-
 from workflow_bench.proposer_sandbox import (
     MAX_BUNDLE_BYTES,
     MAX_EVIDENCE_FILE_BYTES,
-    SANDBOX_CLAUDE,
     SANDBOX_NODE,
     SANDBOX_NODE_PREFIX,
-    SANDBOX_EVIDENCE,
-    SANDBOX_GITNEXUS_CLI,
-    SANDBOX_GIT_EXCLUDES,
     VITE_TEMP_DIR,
     SANDBOX_PATH,
-    SANDBOX_REVIEW_OUTPUT,
     SANDBOX_PYTHON3,
     SANDBOX_SHELL_PREFIX,
     SANDBOX_USER_SKILLS,
-    SANDBOX_WORKSPACE,
     ReadOnlyMount,
     SandboxError,
     _runtime_mount_args,
     build_claude_settings,
     build_sandbox_environment,
-    _force_rmtree,
-    host_workspace_write_boundary,
-    prepare_review_workspace,
     prepare_sandbox,
     preflight_bubblewrap,
-    sandbox_workspace_write_boundary,
     stage_evidence_bundle,
     stage_task_assets,
 )
-from workflow_bench.review_scoring import REVIEW_OUTPUT, parse_review_output
 from workflow_bench.task_assets import TaskAssetCache, stage_task_assets as stage_immutable_task_assets
 
 
-@pytest.mark.parametrize("entry", ["directory", "relative-link", "absolute-link"])
-def test_review_preparation_rejects_a_reused_artifact_directory(tmp_path, entry):
-    clone = tmp_path / "clone"
-    clone.mkdir()
-    sentinel = tmp_path / "sentinel"
-    sentinel.write_text("must survive")
-    with prepare_sandbox(clone=clone, claude_bin=sys.executable, backend="host-unsafe") as sandbox:
-        stale = proposer_sandbox.review_output_path(sandbox, "review-output.json").parent
-        if entry == "directory":
-            stale.mkdir()
-            (stale / "review-output.json").write_text("a previous cell's verdict")
-        else:
-            # relpath, not a hand-written "../sentinel": stale is
-            # <private_root>/review-output, which is nowhere near tmp_path, so the
-            # literal produced a dangling link and the assertion below proved nothing.
-            stale.symlink_to(
-                sentinel if entry == "absolute-link" else Path(os.path.relpath(sentinel, stale.parent))
-            )
-        with pytest.raises(SandboxError, match="already exists"):
-            proposer_sandbox.prepare_review_workspace(sandbox, "review-output.json")
-    assert sentinel.read_text() == "must survive"
-
-
-def test_review_preparation_leaves_a_clone_entry_of_the_same_name_alone(tmp_path):
-    # The artifact no longer lives in the workspace, so a file that happens to
-    # share its name is just one of the repository's own files.
-    clone = tmp_path / "clone"
-    clone.mkdir()
-    (clone / "review-output.json").write_text("repository content")
-    with prepare_sandbox(clone=clone, claude_bin=sys.executable, backend="host-unsafe") as sandbox:
-        output = proposer_sandbox.prepare_review_workspace(sandbox, "review-output.json")
-    assert (clone / "review-output.json").read_text() == "repository content"
-    assert clone not in output.parents
-
-
-def test_review_preparation_creates_a_private_directory_and_not_the_file(tmp_path):
-    clone = tmp_path / "clone"
-    clone.mkdir()
-    with prepare_sandbox(clone=clone, claude_bin=sys.executable, backend="host-unsafe") as sandbox:
-        output = proposer_sandbox.prepare_review_workspace(sandbox, "review-output.json")
-        assert output == proposer_sandbox.review_output_path(sandbox, "review-output.json")
-        # The DIRECTORY is what has to exist and be writable: the agent writes
-        # a temp file beside the target and renames it.
-        assert output.parent.is_dir()
-        assert stat.S_IMODE(output.parent.stat().st_mode) == 0o700
-        # The file is deliberately absent — absence is how "never written" is
-        # told apart from "written badly".
-        assert not output.exists()
-
-
-def test_review_preparation_preserves_existing_runtime_files_and_tracks_only_created_paths(tmp_path):
-    clone = tmp_path / "clone"
-    clone.mkdir()
-    (clone / "bunfig.toml").write_text("existing configuration\n")
-    with prepare_sandbox(clone=clone, claude_bin=sys.executable, backend="host-unsafe") as sandbox:
-        proposer_sandbox.prepare_review_workspace(replace(sandbox, backend="bwrap"), "review-output.json")
-        created = json.loads((sandbox.private_root / "review-created-paths.json").read_text())
-        assert "bunfig.toml" not in created
-        assert ".npmrc" in created
-        assert ".mcp.json" in created
-        assert json.loads((clone / ".mcp.json").read_text()) == {}
-        assert (clone / "bunfig.toml").read_text() == "existing configuration\n"
-        assert (clone / ".git/commondir").read_text() == ".\n"
-
-
-def test_review_preparation_rejects_a_runtime_symlink_parent(tmp_path):
-    clone = tmp_path / "clone"
-    clone.mkdir()
-    outside = tmp_path / "outside"
-    outside.mkdir()
-    (clone / ".claude").symlink_to(outside, target_is_directory=True)
-    with prepare_sandbox(clone=clone, claude_bin=sys.executable, backend="host-unsafe") as sandbox:
-        with pytest.raises(SandboxError):
-            proposer_sandbox.prepare_review_workspace(replace(sandbox, backend="bwrap"), "review-output.json")
-    assert list(outside.iterdir()) == []
-
-
 def test_environment_is_allowlisted_and_shell_children_are_credential_free(monkeypatch) -> None:
     monkeypatch.setenv("AWS_SECRET_ACCESS_KEY", "cloud-secret")
     monkeypatch.setenv("GITHUB_TOKEN", "github-secret")
@@ -158,11 +65,6 @@ def test_environment_is_allowlisted_and_shell_children_are_credential_free(monke
     assert settings["sandbox"]["failIfUnavailable"] is True
     assert settings["sandbox"]["allowUnsandboxedCommands"] is False
     assert settings["sandbox"]["network"]["deniedDomains"] == ["*"]
-    assert SANDBOX_EVIDENCE in settings["sandbox"]["filesystem"]["allowRead"]
-    # Headless `claude -p` (2.1.247) never dispatches PreToolUse from any
-    # settings source, so a hook here would be confinement theater: it would
-    # read as a control in review while enforcing nothing at runtime.
-    assert "hooks" not in settings
     # ENV_SCRUB forces "default" mode; the proposer's tools (Bash writes the
     # overlay) run headless only because they are explicitly pre-approved.
     # Requesting a non-default defaultMode would merely warn, so it must be gone.
@@ -170,169 +72,6 @@ def test_environment_is_allowlisted_and_shell_children_are_credential_free(monke
     assert "defaultMode" not in settings["permissions"]
 
 
-def test_unsafe_host_session_translates_virtual_paths_and_disables_containment(tmp_path) -> None:
-    clone = tmp_path / "clone"
-    evidence = tmp_path / "evidence"
-    for directory in (clone, evidence):
-        directory.mkdir()
-
-    with prepare_sandbox(
-        clone=clone,
-        backend="host-unsafe",
-        claude_bin=sys.executable,
-        read_only_mounts=(ReadOnlyMount(evidence, "/evidence"),),
-    ) as sandbox:
-        assert sandbox.command_prefix == []
-        assert sandbox.require_pid_namespace is False
-        assert sandbox.host_path("/workspace/review-output.json") == str(clone / "review-output.json")
-        assert sandbox.host_path("/evidence/selected-rows.json") == str(evidence / "selected-rows.json")
-        # The review artifact left the workspace, so the host-unsafe backend has
-        # to translate its new home too. Untranslated, the review prompt names a
-        # path that exists on neither backend and the cell writes nothing.
-        assert sandbox.host_path(
-            f"{proposer_sandbox.SANDBOX_REVIEW_OUTPUT}/review-output.json"
-        ) == str(proposer_sandbox.review_output_path(sandbox, "review-output.json"))
-        assert sandbox.host_text(
-            f"write {proposer_sandbox.SANDBOX_REVIEW_OUTPUT}/review-output.json"
-        ) == f"write {proposer_sandbox.review_output_path(sandbox, 'review-output.json')}"
-        assert sandbox.host_text("read /evidence and write /workspace/out") == (
-            f"read {evidence} and write {clone}/out"
-        )
-        # Sessions spawn the binary directly, so it must be the host executable
-        # rather than the sandbox-only mount target.
-        assert sandbox.claude_bin != SANDBOX_CLAUDE
-        assert Path(sandbox.claude_bin).exists()
-        assert sandbox.environment()["HOME"] == str(sandbox.home)
-        assert "CLAUDE_CODE_SUBPROCESS_ENV_SCRUB" not in sandbox.environment()
-        assert all(
-            Path(entry).is_dir() for entry in sandbox.environment()["PATH"].split(":")
-        )
-        unsafe_settings = json.loads(sandbox.settings_json)
-        assert unsafe_settings["sandbox"]["enabled"] is False
-        assert unsafe_settings["sandbox"]["failIfUnavailable"] is False
-        assert "disableBypassPermissionsMode" not in unsafe_settings["permissions"]
-
-
-def test_host_workspace_write_boundary_keeps_only_the_review_artifact_writable(tmp_path) -> None:
-    clone = tmp_path / "clone"
-    nested = clone / "src"
-    nested.mkdir(parents=True)
-    source = nested / "source.ts"
-    source.write_text("trusted\n")
-    output = clone / "review-output.json"
-    output.write_text("")
-    original_source_mode = stat.S_IMODE(source.stat().st_mode)
-    original_output_mode = stat.S_IMODE(output.stat().st_mode)
-
-    with host_workspace_write_boundary(clone, writable=(output,)):
-        with pytest.raises(OSError):
-            source.write_text("tampered\n")
-        with pytest.raises(OSError):
-            (clone / "extra.py").write_text("nope\n")
-        output.write_text('{"schema_version":1}\n')
-
-    assert source.read_text() == "trusted\n"
-    assert output.read_text() == '{"schema_version":1}\n'
-    assert not (clone / "extra.py").exists()
-    assert stat.S_IMODE(source.stat().st_mode) == original_source_mode
-    assert stat.S_IMODE(output.stat().st_mode) == original_output_mode
-
-
-def test_host_workspace_write_boundary_keeps_files_under_an_allowed_directory(tmp_path) -> None:
-    clone = tmp_path / "clone"
-    artifacts = clone / "artifacts"
-    artifacts.mkdir(parents=True)
-    existing = artifacts / "review-output.json"
-    existing.write_text("{}\n")
-    (clone / "src").mkdir()
-    locked = clone / "src" / "source.ts"
-    locked.write_text("trusted\n")
-
-    with host_workspace_write_boundary(clone, writable=(artifacts,)):
-        existing.write_text('{"schema_version":1}\n')
-        with pytest.raises(OSError):
-            locked.write_text("tampered\n")
-
-    assert existing.read_text() == '{"schema_version":1}\n'
-    assert locked.read_text() == "trusted\n"
-
-
-def test_host_workspace_write_boundary_rejects_a_symlinked_writable_artifact(tmp_path) -> None:
-    clone = tmp_path / "clone"
-    clone.mkdir()
-    target = tmp_path / "outside.json"
-    target.write_text("{}\n")
-    output = clone / "review-output.json"
-    output.symlink_to(target)
-    with pytest.raises(SandboxError, match="non-symlink"):
-        with host_workspace_write_boundary(clone, writable=(output,)):
-            pass
-
-
-def test_sandbox_workspace_write_boundary_is_noop_unless_host_unsafe(tmp_path) -> None:
-    clone = tmp_path / "clone"
-    clone.mkdir()
-    source = clone / "source.ts"
-    source.write_text("trusted\n")
-    bwrap_sandbox = SimpleNamespace(backend="bwrap", clone=clone)
-    with sandbox_workspace_write_boundary(
-        bwrap_sandbox,
-        read_only_workspace=True,
-        writable=(),
-    ):
-        source.write_text("still writable under bwrap no-op\n")
-    assert source.read_text() == "still writable under bwrap no-op\n"
-
-    output = clone / "review-output.json"
-    output.write_text("")
-    source.write_text("trusted\n")
-    unsafe = SimpleNamespace(backend="host-unsafe", clone=clone)
-    with sandbox_workspace_write_boundary(
-        unsafe,
-        read_only_workspace=True,
-        writable=(output,),
-    ):
-        with pytest.raises(OSError):
-            source.write_text("tampered\n")
-        output.write_text("ok\n")
-    assert source.read_text() == "trusted\n"
-    assert output.read_text() == "ok\n"
-
-
-def test_force_rmtree_deletes_nonempty_directories_copied_from_a_locked_workspace(tmp_path) -> None:
-    locked = tmp_path / "locked"
-    nested = locked / "gitnexus-shared" / "src"
-    nested.mkdir(parents=True)
-    (nested / "index.ts").write_text("export {}\n")
-    os.chmod(nested, 0o500)
-    os.chmod(locked / "gitnexus-shared", 0o500)
-    os.chmod(locked, 0o500)
-
-    copied = tmp_path / "sandbox-tmp" / "tmp.XXXX" / "gitnexus-shared"
-    copied.parent.mkdir(parents=True)
-    shutil.copytree(locked / "gitnexus-shared", copied)
-    assert stat.S_IMODE(copied.stat().st_mode) & 0o222 == 0
-
-    _force_rmtree(copied.parent)
-    assert not copied.parent.exists()
-
-
-def test_host_unsafe_sandbox_cleanup_survives_readonly_tmpdir_copies(tmp_path) -> None:
-    clone = tmp_path / "clone"
-    clone.mkdir()
-    leftover = None
-    with prepare_sandbox(clone=clone, backend="host-unsafe", claude_bin=sys.executable) as sandbox:
-        leftover = sandbox.private_root
-        copied = sandbox.temp / "tmp.XXXX" / "gitnexus-shared" / "src"
-        copied.mkdir(parents=True)
-        (copied / "index.ts").write_text("export {}\n")
-        os.chmod(copied, 0o500)
-        os.chmod(copied.parent, 0o500)
-        os.chmod(copied.parent.parent, 0o500)
-    assert leftover is not None
-    assert not leftover.exists()
-
-
 @pytest.mark.parametrize(
     "bad_url",
     ["https://user:secret@example.test", "https://example.test/path?token=x", "file:///tmp/model"],
@@ -431,16 +170,6 @@ def test_sandbox_command_has_minimal_mounts_and_no_host_root_bind(tmp_path: Path
         assert probe.returncode == 0, probe.stderr
         assert probe.stdout == f"/home/agent|{SANDBOX_PATH}"
 
-        gitnexus_index = argv.index(SANDBOX_GITNEXUS_CLI)
-        gitnexus_wrapper = Path(argv[gitnexus_index - 1])
-        assert stat.S_IMODE(gitnexus_wrapper.stat().st_mode) == 0o500
-        assert "/opt/gitnexus/dist/cli/index.js" in gitnexus_wrapper.read_text()
-
-        excludes_index = argv.index(SANDBOX_GIT_EXCLUDES)
-        excludes = Path(argv[excludes_index - 1])
-        assert stat.S_IMODE(excludes.stat().st_mode) == 0o400
-        assert "/.bash_profile" in excludes.read_text().splitlines()
-
         # The evidence-provenance.mjs plan-writer's PATH-scan trusts a Python 3
         # candidate only if it (and its directory) is owned by root or by the
         # current process — real /usr/bin/python3 is root-owned on the host,
@@ -888,63 +617,6 @@ finally:
     assert not (clone / "oracle-leak.txt").exists()
 
 
-@pytest.mark.skipif(
-    os.environ.get("GITNEXUS_REQUIRE_BWRAP_CANARY") != "1",
-    reason="real Bubblewrap canary is mandatory in the named Ubuntu CI job",
-)
-def test_read_only_review_workspace_exposes_only_one_writable_artifact(tmp_path: Path) -> None:
-    clone = tmp_path / "clone"
-    clone.mkdir()
-    source = clone / "source.ts"
-    source.write_text("trusted\n")
-    script = """
-import os
-from pathlib import Path
-try:
-    Path('/workspace/source.ts').write_text('tampered')
-except OSError:
-    pass
-else:
-    raise SystemExit('review source remained writable')
-# Write the way the agent's Write tool does: a temp file beside the target,
-# then rename. Writing in place would pass against the mount shape that
-# shipped every artifact empty, which is the regression this canary exists for.
-target = Path('/review-output/review-output.json')
-staging = target.with_name(target.name + '.tmp.1.abc')
-staging.write_text('{"schema_version":1}')
-os.replace(staging, target)
-"""
-    with prepare_sandbox(clone=clone, claude_bin=Path(sys.executable), preflight=True) as sandbox:
-        output = proposer_sandbox.prepare_review_workspace(sandbox, "review-output.json")
-        result = run_managed(
-            [
-                *sandbox.command_prefix_for(
-                    read_only_workspace=True,
-                    extra_writable_mounts=(
-                        ReadOnlyMount(
-                            source=output.parent, target=proposer_sandbox.SANDBOX_REVIEW_OUTPUT
-                        ),
-                    ),
-                ),
-                "/usr/bin/python3",
-                "-c",
-                script,
-            ],
-            timeout=10,
-            env=sandbox.environment(),
-            require_pid_namespace=True,
-        )
-
-        # Inside the sandbox scope: the artifact now lives under the session's
-        # private root, which prepare_sandbox removes on exit. run_arm reads it
-        # here too, while the session is still alive.
-        assert result.ok, result.stderr_tail
-        assert source.read_text() == "trusted\n"
-        assert output.read_text() == '{"schema_version":1}'
-        # The staging file is gone: the rename landed rather than a copy.
-        assert list(output.parent.iterdir()) == [output]
-
-
 @pytest.mark.skipif(os.name == "nt", reason="symlink creation may require elevated Windows privileges")
 @pytest.mark.parametrize("operation", ["stage", "sandbox"])
 def test_clone_root_symlink_is_rejected_before_host_access(tmp_path: Path, operation: str) -> None:
@@ -1167,15 +839,13 @@ def test_clone_controlled_mcp_replacement_is_never_executed_or_credentialed(tmp_
     os.environ.get("GITNEXUS_REQUIRE_CLAUDE_CANARY") != "1",
     reason="real Claude/Bash/MCP canary is mandatory in the named Ubuntu CI job",
 )
-@pytest.mark.parametrize("review_layout", [False, True])
+def test_real_claude_bare_auth_inner_sandbox_and_mcp_permissions(tmp_path: Path) -> None:
-def test_real_claude_auth_inner_sandbox_and_mcp_permissions(tmp_path: Path, review_layout: bool) -> None:
     """Exercise the exact CLI boundary without contacting a paid model."""
 
     claude = Path(os.environ["CLAUDE_CANARY_BIN"]).resolve()
     assert claude.is_file()
     clone = tmp_path / "clone"
     clone.mkdir()
-    (clone / "canary.txt").write_text("hook-readable")
     fake_mcp = clone / "fake_mcp.py"
     fake_mcp.write_text(
         """import json
@@ -1202,7 +872,7 @@ for line in sys.stdin:
             }]
         }
     elif method == "tools/call":
-        Path("/tmp/mcp-called").write_text("ok")
+        Path("/workspace/mcp-called").write_text("ok")
         result = {"content": [{"type": "text", "text": "repository list ready"}]}
     else:
         result = {}
@@ -1211,35 +881,6 @@ for line in sys.stdin:
     )
     fake_mcp.chmod(0o500)
 
-    review_command = """test -z "${ANTHROPIC_API_KEY:-}" && python3 - <<'PY'
-import json
-import os
-import subprocess
-from pathlib import Path
-source = Path('/workspace/canary.txt')
-assert 'hook-readable' in source.read_text()
-assert 'changed for review' in subprocess.check_output(['git', 'diff', '--', 'canary.txt'], text=True)
-for operation in (lambda: source.write_text('forbidden'), lambda: source.rename(source.with_name('renamed')), source.unlink):
-    try:
-        operation()
-    except OSError:
-        pass
-    else:
-        raise AssertionError('source mutation was allowed')
-target = Path('/review-output/review-output.json')
-staging = target.with_name(target.name + '.tmp.1.abc')
-staging.write_text(json.dumps({'schema_version': 1, 'verdict': 'approve', 'findings': []}))
-os.replace(staging, target)
-PY"""
-    if review_layout:
-        for command in (
-            ["git", "init", "-q"],
-            ["git", "add", "canary.txt", "fake_mcp.py"],
-            ["git", "-c", "user.name=Canary", "-c", "user.email=canary@example.test", "commit", "-qm", "fixture"],
-        ):
-            subprocess.run(command, cwd=clone, check=True, capture_output=True)
-        (clone / "canary.txt").write_text("hook-readable\nchanged for review\n")
-
     observed_tool_results: dict[str, dict] = {}
 
     class ModelHandler(BaseHTTPRequestHandler):
@@ -1269,19 +910,7 @@ PY"""
                     and isinstance(block.get("tool_use_id"), str)
                 }
             )
-            if "toolu_read_canary" not in tool_result_ids:
+            if "toolu_mcp_canary" not in tool_result_ids:
-                blocks = [
-                    {
-                        "type": "tool_use",
-                        "id": "toolu_read_canary",
-                        "name": "Read",
-                        "input": {
-                            "file_path": "/workspace/canary.txt",
-                        },
-                    }
-                ]
-                stop_reason = "tool_use"
-            elif "toolu_mcp_canary" not in tool_result_ids:
                 blocks = [
                     {
                         "type": "tool_use",
@@ -1298,9 +927,7 @@ PY"""
                         "id": "toolu_bash_canary",
                         "name": "Bash",
                         "input": {
-                            "command": review_command
+                            "command": ('test -z "${ANTHROPIC_API_KEY:-}" && printf canary > /workspace/bash-called')
-                            if review_layout
-                            else ('test -z "${ANTHROPIC_API_KEY:-}" && printf canary > /workspace/bash-called')
                         },
                     }
                 ]
@@ -1397,22 +1024,6 @@ PY"""
             }
         )
         with prepare_sandbox(clone=clone, claude_bin=claude, preflight=True) as sandbox:
-            output = None
-            before = {}
-            if review_layout:
-                output = proposer_sandbox.prepare_review_workspace(sandbox, "review-output.json")
-                before = runner_artifacts.workspace_snapshot(clone)
-                sandbox = replace(
-                    sandbox,
-                    command_prefix=sandbox.command_prefix_for(
-                        read_only_workspace=True,
-                        extra_writable_mounts=(
-                            ReadOnlyMount(
-                                output.parent, proposer_sandbox.SANDBOX_REVIEW_OUTPUT
-                            ),
-                        ),
-                    ),
-                )
             result = sandbox.run(
                 [
                     sandbox.claude_bin,
@@ -1421,6 +1032,7 @@ PY"""
                     "text",
                     "--output-format",
                     "json",
+                    "--bare",
                     "--settings",
                     sandbox.settings_json,
                     "--strict-mcp-config",
@@ -1432,12 +1044,7 @@ PY"""
                     # authoritative empirical gate for that behavior.
                     "--model",
                     "claude-canary-20260718",
-                    "--tools",
-                    "Read",
-                    "Bash",
-                    "mcp__gitnexus__list_repos",
                     "--allowedTools",
-                    "Read",
                     "Bash",
                     "mcp__gitnexus__list_repos",
                 ],
@@ -1446,121 +1053,17 @@ PY"""
                     auth_token="offline-canary-key",
                     base_url=f"http://127.0.0.1:{server.server_port}",
                 ),
-                stdin_data=b"Use all three available tools, then finish.",
+                stdin_data=b"Use both available tools, then finish.",
             )
-            assert result.ok, result.stderr_tail + result.stdout_tail
-            report = json.loads(result.stdout_tail)
-            assert report["subtype"] == "success" and report["is_error"] is False, report
-            read_result = observed_tool_results["toolu_read_canary"]
-            assert read_result.get("is_error") is not True, read_result
-            assert "hook-readable" in json.dumps(read_result)
-            bash_result = observed_tool_results["toolu_bash_canary"]
-            assert bash_result.get("is_error") is not True, bash_result
-            assert (sandbox.temp / "mcp-called").read_text() == "ok"
-            if review_layout:
-                assert output is not None
-                runner_artifacts.enforce_phase_workspace(clone, before, allowed_artifact=None)
-                assert json.loads(output.read_text())["verdict"] == "approve"
-                assert (clone / "canary.txt").read_text() == "hook-readable\nchanged for review\n"
     finally:
         server.shutdown()
         server.server_close()
         thread.join(timeout=5)
 
-    if not review_layout:
+    assert result.ok, result.stderr_tail + result.stdout_tail
-        assert (clone / "bash-called").read_text() == "canary"
+    report = json.loads(result.stdout_tail)
-
+    assert report["subtype"] == "success" and report["is_error"] is False, report
-
+    bash_result = observed_tool_results["toolu_bash_canary"]
-def test_review_artifact_binds_a_writable_directory_outside_the_workspace(tmp_path):
+    assert bash_result.get("is_error") is not True, bash_result
-    """The bwrap argv, since the mount shape is the whole bug.
+    assert (clone / "bash-called").read_text() == "canary"
-
+    assert (clone / "mcp-called").read_text() == "ok"
-    bwrap cannot create a mount point inside an already-read-only bind, so a
-    writable path has to live outside /workspace — and it has to be the
-    directory, or the agent has nowhere to put the temp file it renames into
-    place.
-    """
-
-    clone = tmp_path / "clone"
-    clone.mkdir()
-    with prepare_sandbox(clone=clone, claude_bin=sys.executable, backend="host-unsafe") as session:
-        sandbox = replace(session, backend="bwrap")
-        output = proposer_sandbox.review_output_path(sandbox, "review-output.json")
-        output.parent.mkdir(mode=0o700)
-        argv = sandbox.command_prefix_for(
-            read_only_workspace=True,
-            extra_writable_mounts=(
-                proposer_sandbox.ReadOnlyMount(
-                    source=output.parent,
-                    target=proposer_sandbox.SANDBOX_REVIEW_OUTPUT,
-                ),
-            ),
-        )
-
-        target = proposer_sandbox.SANDBOX_REVIEW_OUTPUT
-        assert not target.startswith(proposer_sandbox.SANDBOX_WORKSPACE + "/")
-        # The workspace itself is bound read-only...
-        workspace_at = argv.index(proposer_sandbox.SANDBOX_WORKSPACE)
-        assert argv[workspace_at - 2] == "--ro-bind"
-        # ...and the artifact directory is bound writable, as a directory.
-        artifact_at = argv.index(target)
-        assert argv[artifact_at - 2] == "--bind"
-        assert Path(argv[artifact_at - 1]) == output.parent
-        assert Path(argv[artifact_at - 1]).is_dir()
-        assert f"{proposer_sandbox.SANDBOX_WORKSPACE}/review-output.json" not in argv
-
-
-@pytest.mark.skipif(
-    os.environ.get("GITNEXUS_REQUIRE_BWRAP_CANARY") != "1",
-    reason="real Bubblewrap canary is mandatory in the named Ubuntu CI job",
-)
-def test_real_bubblewrap_lets_a_review_artifact_be_written_atomically(tmp_path: Path) -> None:
-    """The filesystem contract the EROFS defect broke, under a real sandbox.
-
-    Argv assertions cannot establish this. The artifact came back empty because
-    an atomic write - temp file beside the target, then rename - needs a
-    WRITABLE PARENT DIRECTORY, and only a real bwrap invocation shows whether
-    the mount grants one. A deterministic writer stands in for the agent: no
-    model session, no credentials.
-
-    Scope: this proves the filesystem and process contract of the production
-    mount configuration. It does not establish that a particular agent CLI's
-    own file-access policy permits the same operation - that is a second,
-    independent gate.
-    """
-
-    clone = tmp_path / "clone"
-    clone.mkdir()
-    (clone / "tracked.txt").write_text("original\n")
-
-    with prepare_sandbox(clone=clone, claude_bin=Path(sys.executable), preflight=True) as sandbox:
-        review_output = prepare_review_workspace(sandbox, REVIEW_OUTPUT)
-        # The production configuration, not a hand-built mount tuple: the same
-        # command_prefix_for call run_arm makes for a review cell.
-        prefix = sandbox.command_prefix_for(
-            read_only_workspace=True,
-            extra_writable_mounts=(
-                ReadOnlyMount(source=review_output.parent, target=SANDBOX_REVIEW_OUTPUT),
-            ),
-        )
-        target = f"{SANDBOX_REVIEW_OUTPUT}/{REVIEW_OUTPUT}"
-        script = (
-            # 1. temp file beside the destination, then atomic rename over it.
-            f'printf %s \'{{"schema_version": 1, "verdict": "approve", "findings": []}}\' > {target}.tmp && '
-            f"mv {target}.tmp {target} && "
-            # 2. the workspace must refuse the write that the mount forbids.
-            f"(printf x >> {SANDBOX_WORKSPACE}/tracked.txt 2>/dev/null && echo WORKSPACE-WRITABLE || echo workspace-readonly)"
-        )
-        result = subprocess.run(
-            [*prefix, "/bin/sh", "-c", script],
-            capture_output=True, text=True, timeout=60, check=False,
-        )
-
-        assert result.returncode == 0, f"atomic write failed inside the sandbox: {result.stderr[-400:]}"
-        assert "workspace-readonly" in result.stdout, "the workspace must stay read-only"
-        assert (clone / "tracked.txt").read_text() == "original\n", "the clone was modified"
-
-        # Read while the session is alive: the artifact lives under the private
-        # root that prepare_sandbox removes on exit, which is also why run_arm
-        # consumes it before leaving the scope.
-        _verdict, findings = parse_review_output(review_output)
-        assert findings == ()

eval/tests/test_provider_usage.py

@@ -1,132 +0,0 @@
-"""The two providers' accounting equations, encoded literally.
-
-Adding OpenAI's cache fields to its input_tokens double-counts, because they are
-subsets of it. Subtracting Anthropic's under-counts, because they are additional
-categories. A single generic struct cannot be right for both, so these tests
-pin each equation rather than the field names.
-"""
-
-from __future__ import annotations
-
-import pytest
-
-from workflow_bench.provider_usage import (
-    ANTHROPIC,
-    OPENAI_RESPONSES,
-    UsageSemanticsError,
-    normalize_usage,
-)
-
-
-def _openai(input_tokens: int, cached: int | None = None, cache_write: int | None = None) -> dict:
-    details: dict[str, int] = {}
-    if cached is not None:
-        details["cached_tokens"] = cached
-    if cache_write is not None:
-        details["cache_write_tokens"] = cache_write
-    return {
-        "input_tokens": input_tokens,
-        "input_tokens_details": details,
-        "output_tokens": 300,
-        "output_tokens_details": {"reasoning_tokens": 250},
-    }
-
-
-def test_openai_uncached_request_is_all_ordinary_input() -> None:
-    usage = normalize_usage(OPENAI_RESPONSES, _openai(1000, cached=0, cache_write=0))
-    assert usage.ordinary_input_tokens == 1000
-    assert usage.total_input_tokens == 1000
-    assert (usage.cache_read_input_tokens, usage.cache_write_input_tokens) == (0, 0)
-
-
-def test_openai_cache_creation_keeps_the_parts_summing_to_input_tokens() -> None:
-    """The subsets must reconstruct the whole, never exceed it."""
-
-    usage = normalize_usage(OPENAI_RESPONSES, _openai(1000, cached=0, cache_write=400))
-    assert usage.ordinary_input_tokens == 600
-    assert (
-        usage.ordinary_input_tokens
-        + usage.cache_read_input_tokens
-        + usage.cache_write_input_tokens
-        == usage.total_input_tokens
-    )
-
-
-def test_openai_cache_hit_plus_new_write_uses_the_documented_subtraction() -> None:
-    usage = normalize_usage(OPENAI_RESPONSES, _openai(10_000, cached=7_000, cache_write=1_000))
-    assert usage.ordinary_input_tokens == 2_000
-    assert usage.total_input_tokens == 10_000, "input_tokens is the whole, not a component"
-
-
-def test_openai_reasoning_tokens_decompose_output_rather_than_adding_to_it() -> None:
-    usage = normalize_usage(OPENAI_RESPONSES, _openai(100, cached=0, cache_write=0))
-    assert usage.output_tokens == 300
-    assert usage.reasoning_output_tokens == 250
-    assert usage.reasoning_output_tokens <= usage.output_tokens
-
-
-def test_anthropic_uncached_total_is_just_input_tokens() -> None:
-    usage = normalize_usage(
-        ANTHROPIC,
-        {"input_tokens": 1000, "cache_creation_input_tokens": 0,
-         "cache_read_input_tokens": 0, "output_tokens": 200},
-    )
-    assert usage.total_input_tokens == 1000
-    assert usage.ordinary_input_tokens == 1000
-
-
-def test_anthropic_cached_total_adds_the_cache_categories() -> None:
-    """The opposite equation to OpenAI's, on deliberately identical numbers."""
-
-    usage = normalize_usage(
-        ANTHROPIC,
-        {"input_tokens": 2_000, "cache_creation_input_tokens": 1_000,
-         "cache_read_input_tokens": 7_000, "output_tokens": 200},
-    )
-    assert usage.total_input_tokens == 10_000
-    assert usage.ordinary_input_tokens == 2_000
-
-
-def test_the_same_numbers_mean_different_totals_on_the_two_providers() -> None:
-    """The whole reason a shared struct is unsafe, in one assertion."""
-
-    openai = normalize_usage(OPENAI_RESPONSES, _openai(10_000, cached=7_000, cache_write=1_000))
-    anthropic = normalize_usage(
-        ANTHROPIC,
-        {"input_tokens": 10_000, "cache_creation_input_tokens": 1_000,
-         "cache_read_input_tokens": 7_000, "output_tokens": 300},
-    )
-    assert openai.total_input_tokens == 10_000
-    assert anthropic.total_input_tokens == 18_000
-    assert openai.ordinary_input_tokens == 2_000
-    assert anthropic.ordinary_input_tokens == 10_000
-
-
-def test_missing_native_cache_fields_are_unknown_and_never_zero() -> None:
-    """A zero we invented is indistinguishable from a zero the provider reported."""
-
-    usage = normalize_usage(OPENAI_RESPONSES, {"input_tokens": 1000, "output_tokens": 10})
-    assert usage.cache_read_input_tokens is None
-    assert usage.cache_write_input_tokens is None
-    assert usage.ordinary_input_tokens is None, "cannot subtract what was never reported"
-    assert usage.total_input_tokens == 1000
-    assert not usage.complete
-    assert "cache_read_input_tokens" in usage.unknown_fields
-
-
-def test_an_absent_usage_object_is_entirely_unknown() -> None:
-    usage = normalize_usage(ANTHROPIC, None)
-    assert not usage.complete
-    assert usage.total_input_tokens is None
-
-
-def test_an_unknown_provider_is_refused_rather_than_guessed() -> None:
-    with pytest.raises(UsageSemanticsError, match="refusing to guess"):
-        normalize_usage("some-new-provider", {"input_tokens": 1})
-
-
-def test_cache_subsets_larger_than_the_whole_are_rejected() -> None:
-    """Nonsense arithmetic must surface, not silently produce a negative."""
-
-    with pytest.raises(UsageSemanticsError, match="exceed input_tokens"):
-        normalize_usage(OPENAI_RESPONSES, _openai(100, cached=90, cache_write=50))

eval/tests/test_provider_usage_capture.py

@@ -1,327 +0,0 @@
-"""What the proxy writes must outlive the translation that follows it.
-
-Claude Code receives an Anthropic-shaped response, which has nowhere to put
-OpenAI's cached_tokens, cache_write_tokens or reasoning_tokens. If those are not
-captured before the translation, the only remaining record of them is a bill.
-"""
-
-from __future__ import annotations
-
-import contextlib
-import json
-from pathlib import Path
-from types import SimpleNamespace
-
-import pytest
-
-from workflow_bench import litellm_usage_callback, provider_usage
-from workflow_bench.litellm_usage_callback import USAGE_LOG_ENV_VAR, ProviderUsageLogger
-from workflow_bench.model_gateway import (
-    OpenAIGateway,
-    USAGE_CALLBACK_MODULE,
-    openai_litellm_config,
-    write_openai_litellm_config,
-)
-from workflow_bench.provider_usage import (
-    ANTHROPIC,
-    LITELLM_NORMALIZED,
-    USAGE_ENV_VARS,
-    normalize_usage,
-)
-
-
-class _Usage:
-    """Stands in for the provider usage model LiteLLM hands the callback."""
-
-    def __init__(self, payload: dict) -> None:
-        self._payload = payload
-
-    def model_dump(self) -> dict:
-        return self._payload
-
-
-def _openai_response(usage: dict) -> SimpleNamespace:
-    return SimpleNamespace(
-        id="resp_68f2c1",
-        # The model that actually answered, which is not the role the caller asked for.
-        model="gpt-5.6-sol-2026-08-01",
-        usage=_Usage(usage),
-    )
-
-
-# The shape a callback actually receives: LiteLLM normalises usage into its own
-# Chat-Completions-style object before any logger sees it, so an OpenAI reply
-# arrives as prompt_tokens / prompt_tokens_details. Confirmed against a real
-# proxy in tests/test_mock_provider.py; a fixture in the wire shape would test
-# an object this code path never gets.
-NATIVE = {
-    "prompt_tokens": 48_000,
-    "prompt_tokens_details": {"cached_tokens": 44_000, "cache_write_tokens": 1_000},
-    "completion_tokens": 900,
-    "completion_tokens_details": {"reasoning_tokens": 640},
-}
-
-
-@pytest.fixture
-def logged(tmp_path: Path, monkeypatch: pytest.MonkeyPatch):
-    log = tmp_path / "provider_usage.jsonl"
-    monkeypatch.setenv(USAGE_LOG_ENV_VAR, str(log))
-
-    def emit(usage: dict) -> dict:
-        ProviderUsageLogger()._append(
-            "success",
-            {"model": "claude-sonnet-4-5", "custom_llm_provider": "openai", "call_type": "responses"},
-            _openai_response(usage),
-            0.0,
-            1.0,
-        )
-        return json.loads(log.read_text().splitlines()[-1])
-
-    return emit
-
-
-def test_native_openai_usage_survives_the_anthropic_translation(logged) -> None:
-    event = logged(NATIVE)
-    native = event["native_usage"]
-    # Verbatim: the fields an Anthropic-shaped response cannot carry.
-    assert native["prompt_tokens_details"]["cached_tokens"] == 44_000
-    assert native["prompt_tokens_details"]["cache_write_tokens"] == 1_000
-    assert native["completion_tokens_details"]["reasoning_tokens"] == 640
-    assert event["response_id"] == "resp_68f2c1"
-
-
-def test_the_actual_model_is_recorded_separately_from_the_requested_role(logged) -> None:
-    """Pricing must follow what answered, not what the caller named."""
-
-    event = logged(NATIVE)
-    assert event["requested_model"] == "claude-sonnet-4-5"
-    assert event["actual_model"] == "gpt-5.6-sol-2026-08-01"
-    assert "cell_id" not in event, "a proxy-wide variable cannot identify a cell"
-
-
-def test_the_captured_event_normalizes_with_openai_arithmetic(logged) -> None:
-    """Capture and normalization must agree end to end, not just in isolation."""
-
-    event = logged(NATIVE)
-    # The provider the LOG recorded, not one the test supplies - passing
-    # OPENAI_RESPONSES by hand here is what hid the adapter-key mismatch.
-    # LITELLM_NORMALIZED, not OPENAI_RESPONSES: a proxy callback never sees the
-    # upstream body. Measured against a real gateway - the Responses adapter
-    # found none of its keys there and reported every field unknown.
-    assert event["provider"] == LITELLM_NORMALIZED
-    assert event["provider_label"] == "openai"
-    usage = normalize_usage(event["provider"], event["native_usage"])
-    assert usage.total_input_tokens == 48_000
-    assert usage.ordinary_input_tokens == 3_000
-    assert usage.cache_read_input_tokens == 44_000
-    assert usage.complete
-
-
-def test_usage_without_details_normalizes_to_unknown_rather_than_zero(logged) -> None:
-    """The mutation the accounting must not survive: dropped details, silent zeros."""
-
-    stripped = {k: v for k, v in NATIVE.items() if k != "prompt_tokens_details"}
-    event = logged(stripped)
-    usage = normalize_usage(event["provider"], event["native_usage"])
-    assert usage.cache_read_input_tokens is None
-    assert usage.ordinary_input_tokens is None
-    assert not usage.complete
-
-
-def test_a_failed_request_is_still_accounted_for(logged, tmp_path: Path) -> None:
-    """The money was spent whether or not the cell produced an artifact."""
-
-    import asyncio
-
-    logger = ProviderUsageLogger()
-    args = ({"model": "claude-sonnet-4-5"}, _openai_response(NATIVE), 0.0, 1.0)
-    # Every hook LiteLLM can call, not the private helper underneath them: the
-    # sync failure hook was missing entirely and _append could never show that.
-    logger.log_failure_event(*args)
-    asyncio.run(logger.async_log_failure_event(*args))
-    events = [json.loads(line) for line in (tmp_path / "provider_usage.jsonl").read_text().splitlines()]
-    assert len(events) == 2, "both failure hooks must record"
-    assert all(e["status"] == "failure" for e in events)
-
-
-def test_every_public_outcome_hook_records(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
-    """Overriding a subset silently drops whichever path LiteLLM actually uses."""
-
-    import asyncio
-
-    monkeypatch.setenv(USAGE_LOG_ENV_VAR, str(tmp_path / "usage.jsonl"))
-    logger = ProviderUsageLogger()
-    args = ({"model": "m"}, _openai_response(NATIVE), 0.0, 1.0)
-    logger.log_success_event(*args)
-    logger.log_failure_event(*args)
-    asyncio.run(logger.async_log_success_event(*args))
-    asyncio.run(logger.async_log_failure_event(*args))
-
-    events = [json.loads(line) for line in (tmp_path / "usage.jsonl").read_text().splitlines()]
-    assert [e["status"] for e in events] == ["success", "failure", "success", "failure"]
-
-
-def test_the_logger_never_raises_into_the_proxy(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
-    """Accounting is evidence, not control flow."""
-
-    monkeypatch.setenv(USAGE_LOG_ENV_VAR, str(tmp_path / "missing-dir" / "usage.jsonl"))
-    ProviderUsageLogger()._append("success", {}, object(), 0.0, 1.0)
-
-
-def test_no_log_is_written_when_the_destination_is_unset(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
-    monkeypatch.delenv(USAGE_LOG_ENV_VAR, raising=False)
-    ProviderUsageLogger()._append("success", {}, _openai_response(NATIVE), 0.0, 1.0)
-    assert not list(tmp_path.iterdir())
-
-
-def test_the_generated_config_loads_the_callback_from_beside_itself(tmp_path: Path) -> None:
-    """LiteLLM resolves the dotted path relative to the config directory."""
-
-    config = write_openai_litellm_config(tmp_path / "litellm.yaml", ["gpt-5.6-sol"])
-    assert openai_litellm_config(["gpt-5.6-sol"])["litellm_settings"]["callbacks"] == [
-        f"{USAGE_CALLBACK_MODULE}.handler"
-    ]
-    installed = config.parent / f"{USAGE_CALLBACK_MODULE}.py"
-    assert installed.is_file(), "the proxy cannot import a callback that was never placed"
-    # Importing it, not grepping it: a text search passes even when the module
-    # cannot load, which is exactly how a package-relative import survived
-    # review here. This is the deployment configuration, so load it the way the
-    # proxy does - by path, as a top-level module.
-    import importlib.util
-
-    spec = importlib.util.spec_from_file_location(USAGE_CALLBACK_MODULE, installed)
-    assert spec is not None and spec.loader is not None
-    module = importlib.util.module_from_spec(spec)
-    spec.loader.exec_module(module)
-    assert isinstance(module.handler, module.ProviderUsageLogger)
-
-
-def test_the_gateway_forwards_the_usage_environment_into_the_proxy(
-    tmp_path: Path, monkeypatch: pytest.MonkeyPatch
-) -> None:
-    """The proxy is a separate process with a constructed environment.
-
-    Popen(env=...) replaces the parent environment rather than extending it, so
-    a variable the callback reads is simply absent unless the gateway forwards
-    it by name. Without this the accounting looks configured and silently
-    records nothing on every request - the in-process tests above cannot see
-    that, because they never cross the subprocess boundary.
-    """
-
-    for name in USAGE_ENV_VARS:
-        monkeypatch.setenv(name, f"value-for-{name}")
-    captured: dict[str, dict[str, str]] = {}
-
-    class _Popen:
-        def __init__(self, *_a, **kwargs):
-            captured["env"] = kwargs["env"]
-            raise RuntimeError("stop before launching a real proxy")
-
-    # The console-script resolver runs before Popen and is absent in this
-    # environment (the same reason two gateway tests fail here); the argv it
-    # builds is not what this test is about.
-    monkeypatch.setattr(
-        "workflow_bench.model_gateway.litellm_proxy_argv",
-        lambda **_k: ["/bin/true"],
-    )
-    monkeypatch.setattr("workflow_bench.model_gateway.subprocess.Popen", _Popen)
-    gateway = OpenAIGateway(
-        openai_api_key="sk-test",
-        model_names=["gpt-5.6-sol"],
-        work_dir=tmp_path,
-    )
-    with contextlib.suppress(Exception):
-        gateway.__enter__()
-
-    env = captured.get("env")
-    assert env is not None, "the proxy was never constructed"
-    for name in USAGE_ENV_VARS:
-        assert env.get(name) == f"value-for-{name}", f"{name} never reached the proxy"
-    # The credential allowlist is still an allowlist, not the parent environment.
-    assert "PATH" in env and len(env) < 40
-
-
-def test_an_unresolvable_provider_is_refused_rather_than_guessed() -> None:
-    """LiteLLM says "openai" for Chat Completions too, and it counts differently."""
-
-    from workflow_bench.provider_usage import canonical_provider
-
-    # Every openai call reaching this callback has already been normalised by
-    # LiteLLM, whatever endpoint it used - the observed call_type for a Claude
-    # Code request through the gateway is "anthropic_messages". The adapter has
-    # to match the object in hand, not the protocol on the wire.
-    assert canonical_provider("openai", "responses") == LITELLM_NORMALIZED
-    assert canonical_provider("openai", "anthropic_messages") == LITELLM_NORMALIZED
-    assert canonical_provider("anthropic", "completion") == ANTHROPIC
-    # An unrecognised provider is still refused rather than guessed.
-    assert canonical_provider("some-new-provider", "responses") is None
-    assert canonical_provider(None, None) is None
-
-
-def test_request_identity_cannot_come_from_the_proxy_environment() -> None:
-    """One proxy serves the whole sweep, so its environment identifies the sweep.
-
-    attach_openai_gateway wraps all of _run_sweep, and cells run concurrently
-    under --workers, interleaving requests through that single process. Any
-    variable forwarded at launch is therefore constant for every event it ever
-    records. Pinned so a future change does not reintroduce a per-cell
-    environment variable that would silently stamp one value on every request.
-    """
-
-    assert USAGE_ENV_VARS == (
-        "GITNEXUS_BENCH_PROVIDER_USAGE",
-        "GITNEXUS_BENCH_SWEEP_ID",
-    ), "a per-cell variable here would be constant across concurrent cells"
-
-
-def test_a_request_records_its_session_so_attribution_stays_possible(logged) -> None:
-    """The per-request half of identity, recorded even when the provider omits it."""
-
-    event = logged(NATIVE)
-    assert "session_id" in event, "absent attribution is still a fact about the run"
-
-
-def test_the_callback_imports_the_way_litellm_actually_loads_it(tmp_path: Path) -> None:
-    """By path, as a top-level module, with no parent package and no sys.path entry.
-
-    LiteLLM resolves a dotted callback through spec_from_file_location against
-    the config directory, so the copied file is not part of workflow_bench when
-    it runs. A relative or sibling import therefore raises ImportError and the
-    proxy exits before becoming ready - which the in-package tests cannot see,
-    because they import it as workflow_bench.litellm_usage_callback.
-    """
-
-    import importlib.util
-    import shutil
-
-    source = Path(litellm_usage_callback.__file__)
-    installed = tmp_path / f"{USAGE_CALLBACK_MODULE}.py"
-    shutil.copy(source, installed)
-
-    spec = importlib.util.spec_from_file_location(USAGE_CALLBACK_MODULE, installed)
-    assert spec is not None and spec.loader is not None
-    module = importlib.util.module_from_spec(spec)
-    spec.loader.exec_module(module)  # ImportError here is the proxy refusing to start
-    assert hasattr(module, "handler")
-
-
-def test_the_callbacks_copied_constants_match_the_canonical_ones() -> None:
-    """The copies are deliberate; drifting apart silently is not.
-
-    The callback cannot import from the package (see the test above), so it
-    carries its own literals. These assertions are what keep the duplication
-    honest.
-    """
-
-    assert litellm_usage_callback.USAGE_LOG_ENV_VAR == provider_usage.USAGE_LOG_ENV_VAR
-    assert litellm_usage_callback.SWEEP_ID_ENV_VAR == provider_usage.SWEEP_ID_ENV_VAR
-    for label, call_type in (
-        ("openai", "responses"),
-        ("openai", "completion"),
-        ("openai", None),
-        ("anthropic", "completion"),
-        ("mystery", "responses"),
-    ):
-        assert litellm_usage_callback.canonical_provider(label, call_type) == provider_usage.canonical_provider(
-            label, call_type
-        ), f"resolver drifted for {label!r}/{call_type!r}"

eval/tests/test_reuse_round_trip.py

@@ -1,268 +0,0 @@
-"""A row the runner actually emits must satisfy the reuse reader.
-
-Every existing comparator-reuse test builds its rows by hand. That proves the
-predicate's logic and nothing about the producer: a fixture can satisfy
-eligibility while a real emitted row never does, and the audit that counts key
-names cannot tell the difference. These tests carry one record through the
-production path instead:
-
-    real run_cell -> production JSONL writer -> load_result_rows
-        -> row_is_reusable_comparator
-
-Only the expensive dependencies are replaced - the model session, sandbox
-launch, repository acquisition, graph preparation. The digest fields the reuse
-binding compares are assembled by run_cell itself from its TaskCellContext, so
-they stay real: they are the subject of the test, not scaffolding around it.
-"""
-
-from __future__ import annotations
-
-import json
-from datetime import UTC, datetime, timedelta
-from pathlib import Path
-from types import SimpleNamespace
-from typing import Any
-
-import pytest
-
-from workflow_bench import runner
-from workflow_bench.proposer_sandbox import redact_text
-from workflow_bench.model_gateway import credential_secrets
-from workflow_bench.runner_sessions import PARENT_EVENT_STREAM_SOURCE
-from workflow_bench.comparator_reuse import (
-    ComparatorReuseExpectation,
-    TaskReuseBinding,
-    load_result_rows,
-    row_is_reusable_comparator,
-)
-
-TASK_ID = "review-pr-2718-defect"
-SHA = "a" * 40
-
-
-def _snapshot(prefix: str) -> SimpleNamespace:
-    return SimpleNamespace(
-        digest=f"{prefix}-content",
-        manifest_digest=f"{prefix}-manifest",
-        dependency_content_digest=f"{prefix}-dep-content",
-        dependency_manifest_digest=f"{prefix}-dep-manifest",
-        command_digest=f"{prefix}-command",
-        materialize=lambda *a, **k: None,
-    )
-
-
-def _write_like_the_sweep(tmp_path: Path, row: dict[str, Any]) -> Path:
-    """Serialize exactly as ``keep`` does in _run_sweep, redaction included.
-
-    json.dumps + write_text would skip the redaction the real writer applies,
-    so a change there could break reusable rows without failing this test - and
-    redaction is not cosmetic here, since it rewrites the row's own bytes.
-    """
-
-    results = tmp_path / "results.jsonl"
-    secrets = credential_secrets(
-        SimpleNamespace(auth_token="sk-ant-should-never-appear", base_url=None)
-    )
-    with results.open("a") as handle:
-        handle.write(redact_text(json.dumps(row), secrets) + "\n")
-    return results
-
-
-@pytest.fixture
-def emitted_row(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> dict[str, Any]:
-    """One record from the real run_cell, with only expensive work replaced."""
-
-    worktree = tmp_path / "clone"
-    worktree.mkdir()
-
-    # The session is what costs money; everything it returns is scripted. The
-    # record's binding fields are NOT set here - run_cell derives them.
-    def fake_run_arm(*_a: Any, **_k: Any) -> dict[str, Any]:
-        return {
-            "ok": True,
-            "error_kind": None,
-            "error_detail": None,
-            "resolved": True,
-            "review_evidence_valid": True,
-            "review_score": {"weighted_f1": 0.5},
-            "review_weighted_f1": 0.5,
-            "skill_invoked": True,
-            "skill_digest": "skill-digest",
-            "transcript_missing": False,
-            "transcript_artifacts": [
-                {
-                    "path": "transcripts/session-1.jsonl",
-                    "sha256": __import__("hashlib").sha256(b'{"type":"ok"}\n').hexdigest(),
-                    "bytes": 14,
-                    "source": PARENT_EVENT_STREAM_SOURCE,
-                }
-            ],
-            "session_ids": ["s1"],
-            "num_turns": 3,
-            "duration_s": 1.0,
-            "cost_usd": 0.5,
-            "input_tokens": 1,
-            "output_tokens": 1,
-            "cache_creation_input_tokens": 0,
-            "cache_read_input_tokens": 0,
-        }
-
-    for name, value in {
-        "run_arm": fake_run_arm,
-        "copy_isolated_tree": lambda *a, **k: worktree,
-        "make_worktree": lambda *a, **k: worktree,
-        "sanitize_clone_for_hidden_oracles": lambda *a, **k: SHA,
-        "stage_task_assets": lambda *a, **k: (),
-        "isolated_gitnexus_registry_mount": lambda *a, **k: None,
-        "seed_evaluated_skills": lambda *a, **k: None,
-        "apply_candidate_overlay": lambda *a, **k: None,
-        "require_hidden_harness_absent": lambda *a, **k: None,
-        "require_skill_fingerprint": lambda *a, **k: None,
-        "enforce_work_evidence": lambda *a, **k: None,
-        "skill_fingerprint": lambda *a, **k: "skill-digest",
-        "capture_patch": lambda *a, **k: b"",
-        "implementation_diff_digest": lambda *a, **k: "",
-        "diff_churn": lambda *a, **k: {},
-        "_prepare_untracked_for_diff": lambda *a, **k: None,
-        "remove_clone": lambda *a, **k: None,
-        "ce_plugin_dir_for_arm": lambda *a, **k: None,
-        "ce_plugin_mounts_for_arm": lambda *a, **k: (),
-        "current_runtime_digest": lambda: "runtime-digest",
-        "build_sandbox_environment": lambda *a, **k: {},
-        "credential_secrets": lambda *a, **k: (),
-        # run_cell requires an immutable base commit before it will record a
-        # cell; the git plumbing is expensive setup, the SHA it returns is not
-        # part of the reuse binding under test.
-        "_sandbox_git": lambda *a, **k: SHA,
-        # The artifact copy is real; only the read of the agent-written file is
-        # replaced, since no agent ran to write one.
-        "_bounded_regular_bytes": lambda *a, **k: b'{"schema_version":1}',
-    }.items():
-        monkeypatch.setattr(runner, name, value)
-
-    class _Sandbox:
-        clone = worktree
-        private_root = tmp_path / "private"
-        backend = "test-double"
-        settings_json = "{}"
-        require_pid_namespace = False
-
-        def __enter__(self) -> _Sandbox:
-            return self
-
-        def __exit__(self, *_exc: Any) -> bool:
-            return False
-
-        def command_prefix_for(self, **_k: Any) -> list[str]:
-            return []
-
-        def run(self, *_a: Any, **_k: Any) -> SimpleNamespace:
-            return SimpleNamespace(ok=True, returncode=0, stdout_tail="", stderr_tail="")
-
-        def environment(self, **_k: Any) -> dict[str, str]:
-            return {}
-
-        def host_text(self, value: str) -> str:
-            return value
-
-    monkeypatch.setattr(runner, "prepare_sandbox", lambda **_k: _Sandbox())
-
-    ctx = runner.TaskCellContext(
-        task={"id": TASK_ID, "prompt": "review it", "verify": "true"},
-        oracle_snapshot=_snapshot("oracle"),
-        repo=tmp_path / "repo",
-        task_sha=SHA,
-        graph_snapshot=_snapshot("graph"),
-        graph_snapshot_error=None,
-        asset_snapshot=_snapshot("asset"),
-        asset_snapshot_error=None,
-        args=SimpleNamespace(
-            model="gpt-5.6-sol", effort="xhigh", timeout=60, claude_bin="claude",
-            base_url=None, auth_token=None, permission_mode=None, arms=["review"],
-            proposer_model=None, outage_streak=5, runs=1, workers=1,
-        ),
-        out_dir=tmp_path / "out",
-        ce_plugin_snapshot=None,
-        trees_dir=tmp_path / "trees",
-        bwrap_bin=Path("/bin/true"),
-        runtime_mounts=(),
-        candidate_overlay=None,
-        overlay_digest=None,
-        sandbox_backend="test-double",
-        clone_template=None,
-        sanitized_head=SHA,
-    )
-    (tmp_path / "out").mkdir(exist_ok=True)
-    (tmp_path / "trees").mkdir(exist_ok=True)
-    (tmp_path / "private").mkdir(exist_ok=True)
-    # run_cell records review_artifact only when the review source exists, and
-    # reuse now requires it - a scored review with no artifact is a claim about
-    # evidence rather than the evidence. Production writes this file; the
-    # fixture has to as well, or the emitted row is one production never emits.
-    review_dir = tmp_path / "private" / "review-output"
-    review_dir.mkdir(exist_ok=True)
-    (review_dir / "review-output.json").write_text('{"schema_version": 1, "verdict": "approve", "findings": []}')
-    return runner.run_cell(ctx, 0, "review")
-
-
-def _expectation(**overrides: Any) -> ComparatorReuseExpectation:
-    """Bindings from the sweep's own configuration, not copied out of the row.
-
-    Copying the emitted values back in would make producer and consumer agree
-    because the test arranged it, which is the blind spot being closed.
-    """
-
-    binding = TaskReuseBinding(
-        task_base_sha=SHA,
-        task_prompt_digest=runner.hashlib.sha256(b"review it").hexdigest(),
-        oracle_digest="oracle-content",
-        oracle_command_digest="oracle-command",
-        oracle_manifest_digest="oracle-manifest",
-        task_asset_manifest_digest="asset-manifest",
-        sandbox_dependency_manifest_digest="asset-dep-manifest",
-    )
-    values: dict[str, Any] = dict(
-        model="gpt-5.6-sol",
-        effort="xhigh",
-        sandbox_backend="test-double",
-        runtime_digest="runtime-digest",
-        now=datetime.now(UTC),
-        max_age=timedelta(days=90),
-        tasks={TASK_ID: binding},
-        skill_digests={"review": "skill-digest"},
-        ce_plugin_version=None,
-        ce_plugin_manifest_digest=None,
-    )
-    values.update(overrides)
-    return ComparatorReuseExpectation(**values)
-
-
-def test_a_row_the_runner_emitted_survives_serialization_and_qualifies(
-    emitted_row: dict[str, Any], tmp_path: Path
-) -> None:
-    """The producer/consumer contract, end to end through the real writer."""
-
-    results = _write_like_the_sweep(tmp_path, emitted_row)
-    rows = load_result_rows(results)
-    assert len(rows) == 1, "the production row must survive the reader"
-
-    assert row_is_reusable_comparator(rows[0], _expectation()) is True
-
-
-def test_a_changed_binding_rejects_the_same_emitted_row(
-    emitted_row: dict[str, Any], tmp_path: Path
-) -> None:
-    """Fails closed on drift, so the positive case is not vacuous."""
-
-    row = load_result_rows(_write_like_the_sweep(tmp_path, emitted_row))[0]
-
-    binding = TaskReuseBinding(
-        task_base_sha=SHA,
-        task_prompt_digest=runner.hashlib.sha256(b"review it").hexdigest(),
-        oracle_digest="oracle-content",
-        oracle_command_digest="oracle-command",
-        oracle_manifest_digest="oracle-manifest",
-        task_asset_manifest_digest="asset-manifest",
-        sandbox_dependency_manifest_digest="DIFFERENT-dependencies",
-    )
-    assert row_is_reusable_comparator(row, _expectation(tasks={TASK_ID: binding})) is False

eval/tests/test_review_corpus.py

@@ -1,59 +0,0 @@
-import hashlib
-import json
-from pathlib import Path
-
-import yaml
-
-from workflow_bench.oracle_assets import review_case_setup_command
-
-
-BENCH_ROOT = Path(__file__).parents[1] / "workflow_bench"
-
-
-def test_review_corpus_is_immutable_and_task_bound():
-    manifest = json.loads((BENCH_ROOT / "review_cases" / "manifest.json").read_text())
-    tasks = yaml.safe_load((BENCH_ROOT / "tasks.review.scenarios.yaml").read_text())["tasks"]
-    by_id = {task["id"]: task for task in tasks}
-
-    assert len(manifest["cases"]) >= 6
-    assert sum(case["id"].endswith("-defect") for case in manifest["cases"]) >= 4
-    assert sum(case["id"].endswith("-clean") for case in manifest["cases"]) >= 2
-    assert set(by_id) == {case["id"] for case in manifest["cases"]}
-
-    for case in manifest["cases"]:
-        assert len(case["base_sha"]) == len(case["head_sha"]) == 40
-        assert len(case["human_verification_commit"]) == 40
-        patch = BENCH_ROOT / "review_cases" / case["patch"]
-        assert case["patch"]
-        assert "defect" not in case["patch"]
-        assert "clean" not in case["patch"]
-        assert hashlib.sha256(patch.read_bytes()).hexdigest() == case["patch_sha256"]
-        task = by_id[case["id"]]
-        assert task["ref"] == case["base_sha"]
-        assert task["sandbox_copy"] == [f"eval/workflow_bench/review_cases/{patch.name}"]
-        assert task["setup"] == review_case_setup_command(patch.name)
-        assert any(
-            dep.get("source") == "gitnexus-shared/dist" and dep.get("target") == "gitnexus-shared/dist"
-            for dep in task["sandbox_dependencies"]
-        )
-
-
-def test_hidden_labels_are_not_recoverable_from_visible_task_input():
-    tasks_path = BENCH_ROOT / "tasks.review.scenarios.yaml"
-    tasks = yaml.safe_load(tasks_path.read_text())["tasks"]
-
-    for task in tasks:
-        visible = json.dumps(
-            {
-                "prompt": task["prompt"],
-                "setup": task["setup"],
-                "sandbox_copy": task["sandbox_copy"],
-            },
-            sort_keys=True,
-        )
-        assert "review-labels.json" not in visible
-        assert "-defect" not in visible
-        assert "-clean" not in visible
-        for oracle_file in task["oracle"]["files"]:
-            assert oracle_file["source"] not in visible
-            assert oracle_file["target"] == "review-labels.json"

eval/tests/test_review_scoring.py

@@ -1,357 +0,0 @@
-import json
-from dataclasses import replace
-from pathlib import Path
-
-import pytest
-
-from workflow_bench.oracle_assets import OracleFileSnapshot, TaskOracleSnapshot
-from workflow_bench.review_scoring import (
-    ExpectedFinding,
-    ReviewFinding,
-    expected_findings,
-    parse_review_output,
-    score_review,
-)
-
-
-@pytest.mark.parametrize("noise", [False, True])
-def test_complete_misses_are_measured_zero(noise):
-    actual = (ReviewFinding("noise", "low", "other.py", 1, 1, "style", "s", "e", "r", False),) if noise else ()
-    score = score_review("comment" if noise else "approve", actual, (expected(),))
-    assert score["f1"] == score["weighted_f1"] == 0
-
-
-def test_downgraded_blocker_loses_weight_and_blocker_credit():
-    actual = ReviewFinding("a", "low", "src/api.ts", 20, 20, "correctness", "s", "e", "r", False)
-    score = score_review("comment", (actual,), (expected(),))
-    assert score["weighted_recall"] == 0.2
-    assert score["blocker_recall"] == 0
-    assert score["verdict_correct"] is False
-
-
-@pytest.mark.parametrize("size", [2, 17, 100])
-def test_maximum_matching_at_every_supported_size(size):
-    a = ReviewFinding("a", "high", "src/api.ts", 1, 1, "a", "s", "e", "r", True)
-    actual = [a, replace(a, finding_id="b", line=10, end_line=10, category="b")]
-    labels = [
-        expected(finding_id="broad", line_start=1, line_end=10, category="a"),
-        expected(finding_id="tight", line_start=1, line_end=1, category="b"),
-    ]
-    for i in range(2, size):
-        actual.append(replace(a, finding_id=str(i), path=f"{i}.py"))
-        labels.append(expected(finding_id=str(i), path=f"{i}.py", line_start=1, line_end=1))
-    for findings in (actual, list(reversed(actual))):
-        for expected_labels in (labels, list(reversed(labels))):
-            assert score_review("request_changes", findings, expected_labels)["true_positives"] == size
-
-
-def test_dense_matching_handles_the_full_finding_limit():
-    a = ReviewFinding("a", "high", "src/api.ts", 20, 20, "correctness", "s", "e", "r", True)
-    actual = [replace(a, finding_id=str(i)) for i in range(100)]
-    labels = [expected(finding_id=str(i)) for i in range(100)]
-    assert score_review("request_changes", actual, labels)["true_positives"] == 100
-
-
-@pytest.mark.parametrize("large_side", ["actual", "expected"])
-def test_maximum_matching_with_asymmetric_large_inputs(large_side):
-    a = ReviewFinding("a", "high", "src/api.ts", 1, 1, "a", "s", "e", "r", True)
-    actual = [a, replace(a, finding_id="b", line=10, end_line=10, category="b")]
-    labels = [
-        expected(finding_id="broad", line_start=1, line_end=10, category="a"),
-        expected(finding_id="tight", line_start=1, line_end=1, category="b"),
-    ]
-    for i in range(15):
-        if large_side == "actual":
-            actual.append(replace(a, finding_id=str(i), path=f"extra-{i}.py"))
-        else:
-            labels.append(expected(finding_id=str(i), path=f"extra-{i}.py"))
-    assert score_review("request_changes", actual, labels)["true_positives"] == 2
-
-
-def finding(**overrides):
-    values = {
-        "id": "actual-1",
-        "severity": "high",
-        "path": "src/api.ts",
-        "line": 20,
-        "end_line": 24,
-        "category": "correctness",
-        "scenario": "A missing guard lets an invalid request reach the sink.",
-        "evidence": "The changed call at line 20 bypasses validate().",
-        "recommendation": "Restore validation before the call.",
-        "blocking": True,
-    }
-    values.update(overrides)
-    return values
-
-
-def expected(**overrides):
-    values = {
-        "finding_id": "expected-1",
-        "severity": "high",
-        "path": "src/api.ts",
-        "line_start": 18,
-        "line_end": 22,
-        "category": "correctness",
-    }
-    values.update(overrides)
-    return ExpectedFinding(**values)
-
-
-def test_parse_review_output_requires_the_strict_schema(tmp_path: Path):
-    output = tmp_path / "review-output.json"
-    output.write_text(
-        json.dumps(
-            {
-                "schema_version": 1,
-                "verdict": "request_changes",
-                "findings": [finding()],
-            }
-        )
-    )
-
-    verdict, findings = parse_review_output(output)
-
-    assert verdict == "request_changes"
-    assert findings[0].path == "src/api.ts"
-    assert findings[0].blocking is True
-
-
-@pytest.mark.parametrize(
-    "document, message",
-    [
-        ({"schema_version": 1, "verdict": "approve", "findings": [finding()]}, "approve"),
-        (
-            {
-                "schema_version": 1,
-                "verdict": "request_changes",
-                "findings": [finding(blocking=False)],
-            },
-            "blocking",
-        ),
-        (
-            {
-                "schema_version": 1,
-                "verdict": "comment",
-                "findings": [finding(path="../escape.ts")],
-            },
-            "repository-relative",
-        ),
-    ],
-)
-def test_parse_review_output_rejects_incoherent_or_unsafe_documents(tmp_path: Path, document, message):
-    output = tmp_path / "review-output.json"
-    output.write_text(json.dumps(document))
-    with pytest.raises(ValueError, match=message):
-        parse_review_output(output)
-
-
-def test_expected_findings_are_loaded_from_hidden_snapshot_only():
-    payload = json.dumps(
-        {
-            "schema_version": 1,
-            "findings": [
-                {
-                    "id": "hidden-1",
-                    "severity": "critical",
-                    "path": "src/auth.ts",
-                    "line_start": 40,
-                    "line_end": 44,
-                    "category": "security",
-                }
-            ],
-        }
-    ).encode()
-    snapshot = TaskOracleSnapshot(
-        command="true",
-        command_digest="command",
-        manifest_digest="manifest",
-        digest="all",
-        files=(
-            OracleFileSnapshot(
-                target="review-labels.json",
-                payload=payload,
-                sha256="payload",
-            ),
-        ),
-    )
-
-    assert expected_findings(snapshot)[0].finding_id == "hidden-1"
-
-
-def test_score_review_matches_by_path_and_overlapping_range():
-    actual = (
-        ReviewFinding(
-            finding_id="actual-1",
-            severity="high",
-            path="src/api.ts",
-            line=20,
-            end_line=24,
-            category="correctness",
-            scenario="scenario",
-            evidence="evidence",
-            recommendation="fix",
-            blocking=True,
-        ),
-        ReviewFinding(
-            finding_id="noise",
-            severity="low",
-            path="src/other.ts",
-            line=1,
-            end_line=1,
-            category="style",
-            scenario="noise",
-            evidence="noise",
-            recommendation="noise",
-            blocking=False,
-        ),
-    )
-
-    score = score_review("request_changes", actual, (expected(),))
-
-    assert score["true_positives"] == 1
-    assert score["false_positives"] == 1
-    assert score["false_negatives"] == 0
-    assert score["recall"] == 1
-    assert score["precision"] == 0.5
-    assert score["blocker_recall"] == 1
-    assert score["verdict_correct"] is True
-
-
-def test_score_review_is_independent_of_finding_list_order():
-    expected_labels = (
-        expected(finding_id="broad", line_start=1, line_end=10, category="a"),
-        expected(finding_id="tight", line_start=5, line_end=5, category="b"),
-    )
-    first = ReviewFinding(
-        finding_id="a",
-        severity="high",
-        path="src/api.ts",
-        line=5,
-        end_line=5,
-        category="a",
-        scenario="s",
-        evidence="e",
-        recommendation="r",
-        blocking=True,
-    )
-    second = ReviewFinding(
-        finding_id="b",
-        severity="high",
-        path="src/api.ts",
-        line=1,
-        end_line=1,
-        category="b",
-        scenario="s",
-        evidence="e",
-        recommendation="r",
-        blocking=True,
-    )
-    forward = score_review("request_changes", (first, second), expected_labels)
-    reverse = score_review("request_changes", (second, first), expected_labels)
-    assert forward["true_positives"] == reverse["true_positives"]
-    assert forward["false_positives"] == reverse["false_positives"]
-    assert forward["false_negatives"] == reverse["false_negatives"]
-    assert forward["weighted_f1"] == reverse["weighted_f1"]
-
-
-def test_score_review_prefers_maximum_cardinality_over_greedy_category_match():
-    expected_labels = (
-        expected(finding_id="broad", line_start=1, line_end=10, category="a"),
-        expected(finding_id="tight", line_start=1, line_end=1, category="b"),
-    )
-    actual = (
-        ReviewFinding(
-            finding_id="actual-1",
-            severity="high",
-            path="src/api.ts",
-            line=1,
-            end_line=1,
-            category="a",
-            scenario="s",
-            evidence="e",
-            recommendation="r",
-            blocking=True,
-        ),
-        ReviewFinding(
-            finding_id="actual-2",
-            severity="high",
-            path="src/api.ts",
-            line=10,
-            end_line=10,
-            category="b",
-            scenario="s",
-            evidence="e",
-            recommendation="r",
-            blocking=True,
-        ),
-    )
-
-    score = score_review("request_changes", actual, expected_labels)
-
-    assert score["true_positives"] == 2
-    assert score["false_positives"] == 0
-    assert score["false_negatives"] == 0
-
-
-def test_clean_control_rewards_an_empty_approval_and_penalizes_noise():
-    clean = score_review("approve", (), ())
-    noisy = score_review(
-        "comment",
-        (
-            ReviewFinding(
-                finding_id="noise",
-                severity="medium",
-                path="src/ok.ts",
-                line=1,
-                end_line=1,
-                category="correctness",
-                scenario="noise",
-                evidence="noise",
-                recommendation="noise",
-                blocking=False,
-            ),
-        ),
-        (),
-    )
-
-    assert clean["weighted_f1"] is None
-    assert clean["precision"] is None
-    assert clean["recall"] is None
-    assert clean["clean_pass"] is True
-    assert clean["verdict_correct"] is True
-    assert noisy["false_positives"] == 1
-    assert noisy["weighted_precision"] == 0
-    assert noisy["recall"] is None
-    assert noisy["clean_pass"] is False
-    assert noisy["verdict_correct"] is False
-
-
-def test_parse_review_output_names_the_actual_failure(tmp_path: Path):
-    """One message per cause.
-
-    Folding empty, malformed and encoding failures together makes a sandbox that
-    left the artifact at 0 bytes indistinguishable from an encoding fault: every
-    such cell reports "not valid UTF-8 JSON". A file the agent never created
-    escaped that fold — lstat sat outside the try, so it raised
-    FileNotFoundError — but only as a bare OSError, naming no cause at all.
-    """
-
-    missing = tmp_path / "never-written.json"
-    with pytest.raises(ValueError, match="was never written"):
-        parse_review_output(missing)
-
-    empty = tmp_path / "empty.json"
-    empty.touch()
-    with pytest.raises(ValueError, match="is empty"):
-        parse_review_output(empty)
-
-    not_utf8 = tmp_path / "latin1.json"
-    not_utf8.write_bytes(b'{"verdict": "\xff\xfe"}')
-    with pytest.raises(ValueError, match="not valid UTF-8"):
-        parse_review_output(not_utf8)
-
-    prose = tmp_path / "prose.json"
-    prose.write_text("Here is my review of the changes.", encoding="utf-8")
-    with pytest.raises(ValueError, match="not valid JSON"):
-        parse_review_output(prose)

eval/tests/test_runner_hardening.py

@@ -2,19 +2,12 @@
 
 import hashlib
 import json
-import shutil
-import subprocess
-from contextlib import nullcontext
-from pathlib import Path
-from types import SimpleNamespace
 
 import pytest
 
-from workflow_bench import proposer_sandbox, runner, runner_artifacts, runner_sessions
+from workflow_bench import runner, runner_artifacts, runner_sessions
 from workflow_bench.evolution import skill_fingerprint
-from workflow_bench.oracle_assets import review_case_setup_command
 from workflow_bench.process_control import ManagedProcessError, ManagedProcessResult
-from workflow_bench.proposer_sandbox import SandboxError
 
 
 def _report(**overrides) -> str:
@@ -282,16 +275,6 @@ def test_phase_workspace_ignores_claude_sandbox_bootstrap_noise(tmp_path):
     (tmp_path / ".claude" / "commands").mkdir(parents=True)
     (tmp_path / ".claude" / ".cc-writes").write_text("{}")
     (tmp_path / ".env").write_text("")
-    (tmp_path / ".bash_profile").write_text("")
-    (tmp_path / ".bashrc").write_text("")
-    (tmp_path / ".gitconfig").write_text("")
-    (tmp_path / ".idea").mkdir()
-    (tmp_path / ".profile").write_text("")
-    (tmp_path / ".ripgreprc").write_text("")
-    (tmp_path / ".vscode").mkdir()
-    (tmp_path / ".zprofile").write_text("")
-    (tmp_path / ".zshrc").write_text("")
-    (tmp_path / "scripts").write_text("")
     (tmp_path / ".env.development.local").write_text("")
     (tmp_path / ".npmrc").write_text("")
     (tmp_path / "package.json").write_text("{}")
@@ -303,30 +286,6 @@ def test_phase_workspace_ignores_claude_sandbox_bootstrap_noise(tmp_path):
     runner_artifacts.enforce_phase_workspace(tmp_path, before, allowed_artifact=artifact)
 
 
-def test_phase_workspace_records_a_root_scripts_symlink(tmp_path):
-    target = tmp_path / "helper.py"
-    target.write_text("planted\n")
-    before = runner_artifacts.workspace_snapshot(tmp_path)
-    (tmp_path / "scripts").symlink_to(target)
-    artifact = tmp_path / "review-output.md"
-    artifact.write_text("new review")
-
-    with pytest.raises(ValueError, match="unauthorized workspace path"):
-        runner_artifacts.enforce_phase_workspace(tmp_path, before, allowed_artifact=artifact)
-
-
-def test_phase_workspace_still_rejects_writes_under_scripts(tmp_path):
-    scripts = tmp_path / "scripts"
-    scripts.mkdir()
-    before = runner_artifacts.workspace_snapshot(tmp_path)
-    (scripts / "helper.py").write_text("planted\n")
-    artifact = tmp_path / "review-output.md"
-    artifact.write_text("new review")
-
-    with pytest.raises(ValueError, match="unauthorized workspace path"):
-        runner_artifacts.enforce_phase_workspace(tmp_path, before, allowed_artifact=artifact)
-
-
 def test_phase_workspace_still_rejects_a_genuinely_unauthorized_change(tmp_path):
     # The bootstrap-noise exclusion must stay narrow: an actual source-file
     # edit outside the allowed artifact still has to be caught.
@@ -422,682 +381,3 @@ def test_phase_workspace_still_sees_writes_under_a_pre_existing_nested_claude_di
 
     with pytest.raises(ValueError, match="unauthorized workspace path"):
         runner_artifacts.enforce_phase_workspace(tmp_path, before, allowed_artifact=artifact)
-
-
-def _cell_context(tmp_path, **overrides):
-    """A TaskCellContext whose per-task inputs are all present and valid."""
-    snapshot = SimpleNamespace(
-        digest="asset-digest",
-        manifest_digest="asset-manifest",
-        dependency_content_digest="dep-content",
-        dependency_manifest_digest="dep-manifest",
-    )
-    graph = SimpleNamespace(
-        digest="graph-digest",
-        manifest_digest="graph-manifest",
-        materialize=lambda *_a, **_k: None,
-    )
-    oracle = SimpleNamespace(
-        digest="oracle-digest",
-        command_digest="oracle-command",
-        manifest_digest="oracle-manifest",
-    )
-    fields = {
-        "task": {"id": "task-a", "class": "demo", "prompt": "do the thing"},
-        "oracle_snapshot": oracle,
-        "repo": tmp_path / "repo",
-        "task_sha": "a" * 40,
-        "graph_snapshot": graph,
-        "graph_snapshot_error": None,
-        "asset_snapshot": snapshot,
-        "asset_snapshot_error": None,
-        "args": SimpleNamespace(
-            claude_bin="claude",
-            model="pinned-model",
-            proposer_model=None,
-            effort="xhigh",
-            auth_token=None,
-        ),
-        "out_dir": tmp_path / "out",
-        "ce_plugin_snapshot": None,
-        "trees_dir": tmp_path / "trees",
-        "bwrap_bin": tmp_path / "bwrap",
-        "runtime_mounts": (),
-        "candidate_overlay": None,
-        "overlay_digest": None,
-    }
-    fields.update(overrides)
-    fields["out_dir"].mkdir(parents=True, exist_ok=True)
-    return runner.TaskCellContext(**fields)
-
-
-def _stub_cell_dependencies(monkeypatch, tmp_path):
-    """Replace everything a cell shells out to, so only its own logic runs.
-
-    Returns the clone it will hand out and the list its teardown appends to.
-    """
-    removed: list[Path] = []
-    worktree = tmp_path / "clone"
-    worktree.mkdir()
-    monkeypatch.setattr(runner, "make_worktree", lambda *_a, **_k: worktree)
-    monkeypatch.setattr(runner, "sanitize_clone_for_hidden_oracles", lambda *_a, **_k: "b" * 40)
-    monkeypatch.setattr(runner, "stage_task_assets", lambda *_a, **_k: [])
-    monkeypatch.setattr(runner, "isolated_gitnexus_registry_mount", lambda *_a, **_k: None)
-    monkeypatch.setattr(runner, "ce_plugin_mounts_for_arm", lambda *_a, **_k: [])
-    monkeypatch.setattr(runner, "ce_plugin_dir_for_arm", lambda *_a, **_k: None)
-    monkeypatch.setattr(runner, "prepare_sandbox", lambda **_k: nullcontext(SimpleNamespace(run=None)))
-    monkeypatch.setattr(runner, "skill_fingerprint", lambda *_a, **_k: "skill-digest")
-    monkeypatch.setattr(runner, "require_skill_fingerprint", lambda *_a, **_k: None)
-    monkeypatch.setattr(runner, "_sandbox_git", lambda *_a, **_k: "c" * 40)
-    monkeypatch.setattr(runner, "implementation_diff_digest", lambda *_a, **_k: "")
-    monkeypatch.setattr(runner, "_prepare_untracked_for_diff", lambda *_a, **_k: None)
-    monkeypatch.setattr(runner, "diff_churn", lambda *_a, **_k: {})
-    monkeypatch.setattr(runner, "enforce_work_evidence", lambda *_a, **_k: None)
-    monkeypatch.setattr(runner, "capture_patch", lambda *_a, **_k: b"diff")
-    monkeypatch.setattr(runner, "run_arm", lambda *_a, **_k: {"resolved": True, "ok": True, "error_kind": None})
-    monkeypatch.setattr(runner, "remove_clone", lambda path: removed.append(path))
-    return worktree, removed
-
-
-def test_run_cell_returns_a_row_bound_to_its_task_and_snapshots(monkeypatch, tmp_path):
-    _, removed = _stub_cell_dependencies(monkeypatch, tmp_path)
-
-    record = runner.run_cell(_cell_context(tmp_path), 2, "workflow")
-
-    assert record["resolved"] is True
-    assert record["error_kind"] is None
-    # The row has to carry its own coordinates: once cells stop running in a
-    # predictable order, position in results.jsonl identifies nothing.
-    assert record["task"] == "task-a"
-    assert record["arm"] == "workflow"
-    assert record["run"] == 2
-    assert record["task_asset_snapshot_digest"] == "asset-digest"
-    assert record["sanitized_graph_snapshot_digest"] == "graph-digest"
-    assert record["oracle_digest"] == "oracle-digest"
-    assert removed == [tmp_path / "clone"]
-
-
-@pytest.mark.parametrize(
-    "failure",
-    [
-        ManagedProcessError(
-            ["setup"],
-            ManagedProcessResult(
-                state="timeout",
-                returncode=-15,
-                stdout_tail="",
-                stderr_tail="",
-                duration_s=1.0,
-            ),
-        ),
-        SandboxError("sandbox refused"),
-        OSError("disk went away"),
-        RuntimeError("overlay drifted"),
-        ValueError("bad binding"),
-    ],
-    ids=["managed-process", "sandbox", "os", "runtime", "value"],
-)
-def test_run_cell_records_an_expected_failure_and_still_removes_its_clone(monkeypatch, tmp_path, failure):
-    _, removed = _stub_cell_dependencies(monkeypatch, tmp_path)
-
-    def explode(*_args, **_kwargs):
-        raise failure
-
-    monkeypatch.setattr(runner, "run_arm", explode)
-    record = runner.run_cell(_cell_context(tmp_path), 0, "workflow")
-
-    assert record["error_kind"] == "infra-error"
-    assert record["resolved"] is False
-    # A cell owns its clone for its whole lifetime; the sweep has no other
-    # chance to reclaim it, so the finally must survive every expected failure.
-    assert removed == [tmp_path / "clone"]
-
-
-def test_run_cell_redacts_the_auth_token_from_the_failure_it_prints(monkeypatch, tmp_path, capsys):
-    _stub_cell_dependencies(monkeypatch, tmp_path)
-    secret = "sk-ant-not-a-real-key"
-
-    def explode(*_args, **_kwargs):
-        raise ManagedProcessError(
-            ["claude"],
-            ManagedProcessResult(
-                state="exited",
-                returncode=1,
-                stdout_tail="",
-                stderr_tail=f"ANTHROPIC_API_KEY={secret}",
-                duration_s=1.0,
-            ),
-        )
-
-    monkeypatch.setattr(runner, "run_arm", explode)
-    context = _cell_context(tmp_path)
-    context.args.auth_token = secret
-    # ManagedProcessError stringifies up to 1000 raw bytes of stderr_tail, and
-    # this line streams live into the CI log now that the sweep's stdout is
-    # echoed. results.jsonl already redacts the same field.
-    runner.run_cell(context, 0, "workflow")
-
-    assert secret not in capsys.readouterr().out
-
-
-def test_run_cell_lets_an_unexpected_failure_escape_rather_than_scoring_it(monkeypatch, tmp_path):
-    _, removed = _stub_cell_dependencies(monkeypatch, tmp_path)
-
-    def explode(*_args, **_kwargs):
-        raise KeyError("harness bug")
-
-    monkeypatch.setattr(runner, "run_arm", explode)
-    # A harness bug recorded as an ordinary infra-error would be averaged into
-    # the evidence and counted toward the outage breaker. It must crash instead.
-    with pytest.raises(KeyError):
-        runner.run_cell(_cell_context(tmp_path), 0, "workflow")
-    assert removed == [tmp_path / "clone"]
-
-
-def test_run_cell_reports_a_cleanup_failure_over_its_primary_outcome(monkeypatch, tmp_path):
-    _stub_cell_dependencies(monkeypatch, tmp_path)
-
-    def refuse(_path):
-        raise OSError("clone is busy")
-
-    monkeypatch.setattr(runner, "remove_clone", refuse)
-    record = runner.run_cell(_cell_context(tmp_path), 1, "workflow")
-
-    assert record["error_kind"] == "cleanup-failure"
-    assert record["resolved"] is False
-    assert "primary=None" in record["error_detail"]
-    assert "clone is busy" in record["error_detail"]
-
-
-def _git(repo, *args):
-    return subprocess.run(["git", "-C", str(repo), *args], check=True, capture_output=True, text=True)
-
-
-def test_run_cell_runs_the_arm_against_a_copy_of_the_clone_template(monkeypatch, tmp_path):
-    """run_cell must copy the template, never re-clone.
-
-    run_cell takes the clone-template branch on essentially every multi-cell
-    sweep: it copies a pre-sanitized template rather than paying `git clone
-    --no-local` plus repack/prune/fsck per cell. Asserting on a copy the test
-    makes itself proves nothing about that branch — the clone the arm receives
-    is what has to come from the template, carrying the template's sanitized
-    HEAD rather than a recomputed one.
-    """
-
-    repo = tmp_path / "repo"
-    repo.mkdir()
-    _git(repo, "init", "--quiet")
-    _git(repo, "checkout", "--quiet", "-b", "main")
-    (repo / "from-template.txt").write_text("sanitized\n")
-    _git(repo, "add", "-A")
-    _git(repo, "-c", "user.name=test", "-c", "user.email=test@invalid", "commit", "--quiet", "-m", "base")
-    sha = _git(repo, "rev-parse", "HEAD").stdout.strip()
-    trees = tmp_path / "trees"
-    trees.mkdir()
-    template = runner.make_worktree(repo, sha, trees)
-    template_head = _git(template, "rev-parse", "HEAD").stdout.strip()
-
-    _stub_cell_dependencies(monkeypatch, tmp_path)
-
-    def fail_if_recloned(*_args, **_kwargs):
-        raise AssertionError("clone template present: run_cell must not re-clone")
-
-    monkeypatch.setattr(runner, "make_worktree", fail_if_recloned)
-    monkeypatch.setattr(runner, "sanitize_clone_for_hidden_oracles", fail_if_recloned)
-
-    seen: dict[str, object] = {}
-
-    def record_arm(_arm, _task, worktree, _args, **_kwargs):
-        seen["worktree"] = worktree
-        seen["head"] = _git(worktree, "rev-parse", "HEAD").stdout.strip()
-        seen["content"] = (worktree / "from-template.txt").read_text()
-        # The copy is a private checkout: what the cell writes must not reach
-        # the template the other cells of this task still copy from.
-        (worktree / "from-template.txt").write_text("cell-local\n")
-        return {"resolved": True, "ok": True, "error_kind": None}
-
-    monkeypatch.setattr(runner, "run_arm", record_arm)
-
-    runner.run_cell(
-        _cell_context(tmp_path, clone_template=template, sanitized_head=template_head),
-        0,
-        "workflow",
-    )
-
-    assert seen["content"] == "sanitized\n"
-    assert seen["head"] == template_head
-    assert seen["worktree"] != template
-    assert (template / "from-template.txt").read_text() == "sanitized\n"
-
-
-def test_run_cell_does_not_mask_the_staged_review_patch_before_setup(monkeypatch, tmp_path):
-    """Review setup applies a patch staged under eval/workflow_bench.
-
-    Overlaying the empty oracle mask on that path is the CI abort:
-    `git apply` dies with `can't open patch`. The staged copy must stay
-    visible to sandboxed setup, then be gone before the model starts.
-    """
-
-    worktree, _ = _stub_cell_dependencies(monkeypatch, tmp_path)
-    patch = worktree / "eval" / "workflow_bench" / "review_cases" / "pr-2718.patch"
-    patch.parent.mkdir(parents=True)
-    patch.write_text("diff --git a/visible.py b/visible.py\n")
-    captured: dict[str, object] = {}
-
-    def fake_prepare(**kwargs):
-        captured["mounts"] = kwargs.get("read_only_mounts", [])
-
-        def run(_command, **_kwargs):
-            leftover = worktree / "eval" / "workflow_bench"
-            if leftover.exists():
-                shutil.rmtree(leftover)
-            return SimpleNamespace(ok=True)
-
-        return nullcontext(SimpleNamespace(run=run))
-
-    monkeypatch.setattr(runner, "prepare_sandbox", fake_prepare)
-    context = _cell_context(
-        tmp_path,
-        task={
-            "id": "review-pr-2718-defect",
-            "class": "review-defect",
-            "prompt": "review the local diff",
-            "setup": review_case_setup_command("pr-2718.patch"),
-        },
-    )
-    record = runner.run_cell(context, 0, "workflow")
-
-    assert record.get("error_kind") is None
-    targets = [getattr(mount, "target", None) for mount in captured["mounts"] if mount is not None]
-    assert not any(target and "eval/workflow_bench" in str(target) for target in targets)
-    assert not (worktree / "eval" / "workflow_bench").exists()
-
-
-def test_run_cell_fails_closed_when_setup_leaves_the_hidden_harness(monkeypatch, tmp_path):
-    worktree, _ = _stub_cell_dependencies(monkeypatch, tmp_path)
-    leftover = worktree / "eval" / "workflow_bench" / "review_cases"
-    leftover.mkdir(parents=True)
-    (leftover / "pr-2718.patch").write_text("diff\n")
-
-    def fake_prepare(**kwargs):
-        return nullcontext(SimpleNamespace(run=lambda *_a, **_k: SimpleNamespace(ok=True)))
-
-    monkeypatch.setattr(runner, "prepare_sandbox", fake_prepare)
-    context = _cell_context(
-        tmp_path,
-        task={
-            "id": "review-pr-2718-defect",
-            "class": "review-defect",
-            "prompt": "review the local diff",
-            "setup": review_case_setup_command("pr-2718.patch"),
-        },
-    )
-    record = runner.run_cell(context, 0, "workflow")
-
-    assert record["error_kind"] == "infra-error"
-    assert "hidden harness visible" in str(record["error_detail"])
-
-
-def test_run_cell_fails_closed_when_a_per_task_snapshot_never_materialized(tmp_path):
-    # The snapshots are prepared once per task, before any cell. If that failed,
-    # every cell of the task has to record it rather than run against nothing.
-    context = _cell_context(tmp_path, asset_snapshot=None, asset_snapshot_error=OSError("no assets"))
-    record = runner.run_cell(context, 0, "workflow")
-
-    assert record["error_kind"] == "infra-error"
-    assert "no assets" in str(record["error_detail"])
-
-
-def _progress():
-    """Collector for what a sweep started and kept, readable after it raises."""
-    return SimpleNamespace(started=[], kept=[], streak=0, tripped=False)
-
-
-def _sweep(cells, *, workers, run, outage_limit=5, streak=0, into=None):
-    """Drive sweep_task_cells, recording what it started and kept.
-
-    Pass ``into`` a ``_progress()`` when the sweep is expected to raise: the
-    collector survives the exception, the return value does not.
-    """
-    result = _progress() if into is None else into
-    result.streak, result.tripped = runner.sweep_task_cells(
-        cells,
-        workers=workers,
-        run=run,
-        on_start=lambda run_idx, arm: result.started.append((run_idx, arm)),
-        on_record=lambda run_idx, arm, _record: result.kept.append((run_idx, arm)),
-        outage_streak=streak,
-        outage_limit=outage_limit,
-    )
-    return result
-
-
-def _row(error_kind=None):
-    return {"resolved": error_kind is None, "error_kind": error_kind}
-
-
-CELLS = [(run_idx, arm) for run_idx in range(3) for arm in ("workflow", "candidate_workflow")]
-
-
-@pytest.mark.parametrize("workers", [1, 3, 8])
-@pytest.mark.parametrize("primary", ["review-evidence-invalid", "skill-not-invoked", "session-error"])
-def test_unusable_review_evidence_stops_a_54_cell_sweep(workers, primary):
-    cells = [(i, "review") for i in range(54)]
-    result = _sweep(
-        cells,
-        workers=workers,
-        run=lambda *_: {
-            "resolved": False,
-            "error_kind": primary,
-            "review_evidence_valid": False,
-        },
-    )
-    assert result.tripped
-    assert 5 <= len(result.started) <= 5 + workers - 1
-    assert result.kept == result.started
-
-
-def test_measured_review_miss_resets_the_outage_streak():
-    result = _sweep(
-        CELLS,
-        workers=1,
-        streak=4,
-        run=lambda *_: {
-            "resolved": False,
-            "error_kind": "oracle-failed",
-            "review_evidence_valid": True,
-            "review_weighted_f1": 0.0,
-        },
-    )
-    assert result.streak == 0 and not result.tripped
-
-
-def test_invalid_review_with_primary_skill_error_is_excluded_from_aggregation():
-    result = runner.aggregate(
-        [
-            {
-                "resolved": False,
-                "error_kind": "skill-not-invoked",
-                "review_evidence_valid": False,
-            }
-        ]
-    )
-    assert result["excluded_runs"] == 1 and result["valid_runs"] == 0
-
-
-def test_sweep_keeps_rows_in_submission_order_whatever_order_they_finish():
-    # Cells finish in whatever order the machine allows, but a wave is folded
-    # in submission order — the outage streak counts consecutive failures, and
-    # "consecutive" in completion order would make the trip point flaky.
-    import threading
-
-    first_wave = CELLS[:3]
-    rendezvous = threading.Barrier(3, timeout=10)
-    release_first = threading.Event()
-    fast_finished = threading.Event()
-    finished: list[tuple[int, str]] = []
-    result: list[SimpleNamespace] = []
-
-    def run(run_idx, arm):
-        cell = (run_idx, arm)
-        if cell in first_wave:
-            rendezvous.wait()
-            if cell == first_wave[0]:
-                release_first.wait(timeout=10)
-            else:
-                finished.append(cell)
-                if len(finished) == 2:
-                    fast_finished.set()
-        return _row()
-
-    sweep = threading.Thread(target=lambda: result.append(_sweep(CELLS, workers=3, run=run)))
-    sweep.start()
-    try:
-        assert fast_finished.wait(timeout=10)
-        assert first_wave[0] not in finished
-        assert set(finished) == set(first_wave[1:])
-    finally:
-        release_first.set()
-        sweep.join(timeout=10)
-
-    assert not sweep.is_alive()
-    assert result[0].kept == CELLS
-    assert result[0].started == CELLS
-    assert result[0].tripped is False
-
-
-@pytest.mark.parametrize("workers", [1, 2, 3])
-def test_sweep_trips_the_breaker_within_one_wave_of_the_serial_point(workers):
-    # Serial stops after the 5th consecutive systemic failure. Cells already in
-    # flight when the breaker trips cannot be recalled or erased from the
-    # evidence, so the overrun is bounded by the wave and every completed row
-    # is kept. Ten cells make the bound visible rather than hidden by the end.
-    long_task = [(run_idx, arm) for run_idx in range(5) for arm in ("workflow", "candidate_workflow")]
-
-    result = _sweep(long_task, workers=workers, run=lambda *_: _row("session-error"))
-
-    assert result.tripped is True
-    assert 5 <= len(result.started) <= 5 + workers - 1
-    assert result.kept == result.started
-    assert len(result.started) < len(long_task)
-
-
-def test_sweep_reads_a_real_failure_as_signal_rather_than_an_outage():
-    # resolved=False with no systemic error_kind is the benchmark working, not
-    # the harness failing; it must reset the streak instead of tripping.
-    result = _sweep(CELLS, workers=3, run=lambda *_: {"resolved": False, "error_kind": None})
-
-    assert result.tripped is False
-    assert result.streak == 0
-    assert result.kept == CELLS
-
-
-def test_sweep_surfaces_an_unexpected_worker_failure_instead_of_dropping_the_cell():
-    def run(run_idx, arm):
-        if (run_idx, arm) == (0, "candidate_workflow"):
-            raise KeyError("harness bug")
-        return _row()
-
-    # A Future holds its exception until read. Unread, this cell would vanish
-    # from the evidence with no crash and no row — fewer runs in an arm's
-    # aggregate, silently.
-    with pytest.raises(KeyError):
-        _sweep(CELLS, workers=3, run=run)
-
-
-def test_sweep_runs_cells_of_a_wave_at_the_same_time():
-    import threading
-
-    barrier = threading.Barrier(3, timeout=10)
-
-    def run(run_idx, arm):
-        # Deadlocks unless all three cells of the wave are genuinely in flight
-        # together — a pool that serialised them would time out here.
-        barrier.wait()
-        return _row()
-
-    result = _sweep(CELLS, workers=3, run=run)
-
-    assert result.kept == CELLS
-
-
-def test_sweep_of_one_worker_never_leaves_the_calling_thread():
-    import threading
-
-    caller = threading.current_thread()
-    seen: list[threading.Thread] = []
-
-    def run(run_idx, arm):
-        seen.append(threading.current_thread())
-        return _row()
-
-    # Ctrl-C reaches only the main thread, so the serial default has to stay on
-    # it: a cell on a worker thread is outside the reach of the cleanup that
-    # kills its sandboxed process tree.
-    _sweep(CELLS, workers=1, run=run)
-
-    assert seen == [caller] * len(CELLS)
-
-
-@pytest.mark.parametrize("failure", [KeyError("harness bug"), SystemExit(97)])
-def test_sweep_keeps_the_rows_of_cells_that_finished_beside_a_failing_one(failure):
-    def run(run_idx, arm):
-        if (run_idx, arm) == (0, "candidate_workflow"):
-            raise failure
-        return _row()
-
-    progress = _progress()
-    # The failing cell's two siblings completed and spent their budget before
-    # the harness bug surfaced. Reading the futures in order and raising on the
-    # first failure would drop their rows: money spent, no evidence written.
-    with pytest.raises(type(failure)):
-        _sweep(CELLS, workers=3, run=run, into=progress)
-
-    assert progress.kept == [(0, "workflow"), (1, "workflow")]
-
-
-def test_sweep_hands_a_ctrl_c_back_without_waiting_for_the_running_cells(monkeypatch):
-    import threading
-
-    in_flight = threading.Barrier(3, timeout=10)
-    release = threading.Event()
-    finished: list[tuple[int, str]] = []
-
-    def run(run_idx, arm):
-        in_flight.wait()
-        release.wait(timeout=10)
-        finished.append((run_idx, arm))
-        return _row()
-
-    def interrupt_once_the_wave_is_running(_futures, *_args, **_kwargs):
-        # Stands in for the Ctrl-C an operator types mid-wave: an async
-        # KeyboardInterrupt is delivered to the main thread, which is the one
-        # blocked here waiting on the wave.
-        in_flight.wait()
-        raise KeyboardInterrupt
-
-    monkeypatch.setattr(runner, "wait", interrupt_once_the_wave_is_running)
-    try:
-        with runner.cancellation_scope(release), pytest.raises(KeyboardInterrupt):
-            _sweep(CELLS, workers=2, run=run)
-
-        # Cancellation releases active work before joining; no worker can
-        # outlive the assets the interrupted sweep is about to clean up.
-        assert set(finished) == set(CELLS[:2])
-    finally:
-        release.set()
-
-
-def test_workers_is_bounded_at_both_ends_before_the_sweep_starts():
-    base = ["--tasks", "tasks.yaml", "--model", "pinned-model"]
-
-    assert runner.build_parser().parse_args(base).workers == 1
-    at_max = runner.build_parser().parse_args([*base, "--workers", str(runner.MAX_WORKERS)])
-    assert at_max.workers == runner.MAX_WORKERS
-
-    # A mistyped worker count has to fail at the command line: hours later it
-    # only shows up as timed-out sessions, which the promotion gate throws away.
-    for rejected in ("0", "-1", str(runner.MAX_WORKERS + 1)):
-        with pytest.raises(SystemExit):
-            runner.build_parser().parse_args([*base, "--workers", rejected])
-
-
-def test_partial_wave_submission_preserves_rows_and_original_interruption(monkeypatch):
-    original = runner.ThreadPoolExecutor.submit
-    submissions = 0
-
-    def submit(pool, *args, **kwargs):
-        nonlocal submissions
-        submissions += 1
-        if submissions == 2:
-            raise KeyboardInterrupt("submission interrupted")
-        return original(pool, *args, **kwargs)
-
-    monkeypatch.setattr(runner.ThreadPoolExecutor, "submit", submit)
-    records = []
-    with pytest.raises(KeyboardInterrupt, match="submission interrupted"):
-        runner.sweep_task_cells(
-            CELLS,
-            workers=2,
-            run=lambda *_: _row(),
-            on_start=lambda *_: None,
-            on_record=lambda *record: records.append(record),
-            outage_streak=0,
-            outage_limit=5,
-        )
-    assert len(records) == 2
-    assert records[1][2]["error_kind"] == "cancelled"
-
-
-@pytest.mark.parametrize("error_kind", ["infra-error", "cleanup-failure"])
-def test_progress_line_reports_an_unmeasured_failure_as_unmeasured_not_as_free(error_kind):
-    dead = runner.infra_error_record(RuntimeError("bwrap died"))
-    dead["error_kind"] = error_kind
-
-    line = runner.cell_progress_line("task", "workflow", 0, dead)
-
-    # The 0.0s are placeholders for numbers no session ever produced; printed
-    # as numbers they read as a cell that ran instantly for free.
-    assert "cost=n/a" in line
-    assert "took=n/a" in line
-    assert f"error_kind={error_kind}" in line
-    # results.jsonl is promotion evidence — only the display changes.
-    assert dead["cost_usd"] == 0.0
-    assert dead["duration_s"] == 0.0
-
-
-def test_progress_line_reports_the_numbers_a_real_run_measured():
-    line = runner.cell_progress_line(
-        "task",
-        "workflow",
-        1,
-        {
-            "resolved": True,
-            "input_tokens": 10,
-            "output_tokens": 2,
-            "cost_usd": 0.5,
-            "duration_s": 12.0,
-            "error_kind": None,
-        },
-    )
-
-    assert "cost=$0.5" in line
-    assert "took=12.0s" in line
-    assert "error_kind=none" in line
-
-
-def test_claude_settings_allow_the_review_artifact_directory():
-    """The second gate on the artifact path.
-
-    The bwrap bind is not the only thing that decides whether the agent can
-    write: the CLI applies this filesystem policy to its own tools, so a path
-    missing from allowWrite is unwritable however the mount is shaped. The
-    artifact lived under /workspace when this list was written, which is why
-    moving it out needed this entry and nothing caught the omission.
-    """
-
-    settings = json.loads(proposer_sandbox.build_claude_settings(sandbox_enabled=True))
-    filesystem = settings["sandbox"]["filesystem"]
-    assert proposer_sandbox.SANDBOX_REVIEW_OUTPUT in filesystem["allowWrite"]
-    assert proposer_sandbox.SANDBOX_REVIEW_OUTPUT in filesystem["allowRead"]
-    assert filesystem["denyRead"] == ["/"]
-
-
-def test_review_contract_tells_the_agent_the_writable_path():
-    prompt = runner.REVIEW_PROMPT.format(task="task text")
-    assert f"{runner.SANDBOX_REVIEW_OUTPUT}/{runner.REVIEW_OUTPUT}" in prompt
-    assert f"{runner.SANDBOX_WORKSPACE}/{runner.REVIEW_OUTPUT}" not in prompt
-    # The JSON shape survives .format() with its braces intact.
-    assert '{"schema_version":1' in prompt
-    artifact = f"{runner.SANDBOX_REVIEW_OUTPUT}/{runner.REVIEW_OUTPUT}"
-    assert runner.CE_REVIEW_PROMPT.format(task="task text").count(artifact) == 1
-
-
-def test_enforce_phase_workspace_can_require_an_untouched_workspace(tmp_path):
-    (tmp_path / "tracked.py").write_text("original\n")
-    before = runner_artifacts.workspace_snapshot(tmp_path)
-    runner_artifacts.enforce_phase_workspace(tmp_path, before, allowed_artifact=None)
-
-    (tmp_path / "tracked.py").write_text("the review edited the code it was reviewing\n")
-    with pytest.raises(ValueError, match="changed the read-only workspace"):
-        runner_artifacts.enforce_phase_workspace(tmp_path, before, allowed_artifact=None)

eval/tests/test_sanitized_graph.py

@@ -27,32 +27,6 @@ def test_prebuilt_graph_and_harness_assets_are_rejected(task):
         sanitized_graph.validate_no_prebuilt_graph_assets(task)
 
 
-def test_review_case_patches_are_allowed_sandbox_copy():
-    sanitized_graph.validate_no_prebuilt_graph_assets(
-        {"sandbox_copy": ["eval/workflow_bench/review_cases/pr-2718.patch"]}
-    )
-
-
-@pytest.mark.parametrize(
-    "task",
-    [
-        {"sandbox_copy": ["eval/workflow_bench"]},
-        {"sandbox_copy": ["eval/workflow_bench/evolve.py"]},
-        {
-            "sandbox_dependencies": [
-                {
-                    "source": "eval/workflow_bench/review_cases/pr-2718.patch",
-                    "target": "patch",
-                }
-            ]
-        },
-    ],
-)
-def test_non_corpus_harness_paths_stay_rejected(task):
-    with pytest.raises(SandboxError, match="prebuilt graph or harness"):
-        sanitized_graph.validate_no_prebuilt_graph_assets(task)
-
-
 def test_graph_environment_is_offline_deterministic_and_ignores_target_gitignore():
     env = sanitized_graph._graph_environment()
 
@@ -60,10 +34,6 @@ def test_graph_environment_is_offline_deterministic_and_ignores_target_gitignore
     assert env["GITNEXUS_NO_GITIGNORE"] == "1"
     assert env["GITNEXUS_WORKER_POOL_SIZE"] == "1"
     assert env["GITNEXUS_PARSE_CHUNK_CONCURRENCY"] == "1"
-    assert env["GITNEXUS_WORKER_READY_TIMEOUT_MS"] == str(
-        sanitized_graph.GRAPH_WORKER_READY_TIMEOUT_MS
-    )
-    assert int(env["GITNEXUS_WORKER_READY_TIMEOUT_MS"]) >= 60_000
     assert "ANTHROPIC_API_KEY" not in env
 
 
@@ -212,22 +182,6 @@ def test_prepare_sanitized_graph_builds_once_from_parentless_tree_and_caches_onl
     assert removed == [seed]
 
 
-def test_prepare_sanitized_graph_requires_head_when_given_a_template(tmp_path: Path):
-    with pytest.raises(SandboxError, match="sanitized HEAD"):
-        sanitized_graph.prepare_sanitized_graph(
-            {},
-            repo=tmp_path,
-            resolved_sha="b" * 40,
-            parent=tmp_path,
-            cache=SimpleNamespace(),  # type: ignore[arg-type]
-            claude_bin="claude",
-            bwrap_bin="bwrap",
-            runtime_mounts=(),
-            clone_template=tmp_path,
-            sanitized_head=None,
-        )
-
-
 def test_graph_snapshot_rejects_arm_sanitization_identity_drift(tmp_path: Path):
     assets = SimpleNamespace(
         digest="digest",

eval/tests/test_session_progress.py

@@ -1,372 +0,0 @@
-"""Live progress reporting for long headless sessions.
-
-Progress includes event metadata and bounded, redacted tool argument/result
-previews. Model prose and raw event streams are never echoed. These tests pin
-that boundary along with the signals that distinguish work from a wedged run.
-"""
-
-from __future__ import annotations
-
-import io
-import json
-import time
-
-from workflow_bench.runner_sessions import SessionProgress, neutralize_ci_log_text
-
-
-def _drain_lines(stream: io.StringIO) -> list[str]:
-    return [line for line in stream.getvalue().splitlines() if line.strip()]
-
-
-def _observe(progress: SessionProgress, chunk: bytes) -> None:
-    progress.observe(chunk)
-    progress._emit_pending()
-
-
-def test_progress_bounds_unanswered_tools_and_undrained_messages() -> None:
-    progress = SessionProgress("bounded", stream=io.StringIO())
-    for index in range(2000):
-        progress.observe(
-            (
-                json.dumps(
-                    {
-                        "type": "assistant",
-                        "message": {
-                            "content": [
-                                {"type": "tool_use", "id": str(index), "name": "Bash", "input": {"command": "true"}}
-                            ]
-                        },
-                    }
-                )
-                + "\n"
-            ).encode()
-        )
-    assert len(progress._pending_tools) <= 256
-    assert len(progress._pending_messages) <= 256
-    assert "1999" in progress._pending_tools
-    assert "0" not in progress._pending_tools
-    progress.observe(
-        (
-            json.dumps(
-                {
-                    "type": "user",
-                    "message": {
-                        "content": [{"type": "tool_result", "tool_use_id": "1999", "content": "recent result"}]
-                    },
-                }
-            )
-            + "\n"
-        ).encode()
-    )
-    progress._emit_pending()
-    assert "recent result" in progress._stream.getvalue()
-    assert "1999" not in progress._pending_tools
-
-
-def test_progress_reports_bounded_redacted_tool_io_but_never_model_prose() -> None:
-    stream = io.StringIO()
-    progress = SessionProgress(
-        "gen 0 proposer",
-        stream=stream,
-        heartbeat_s=3600,
-        secrets=("SECRET-TOKEN-abc123",),
-    )
-    events = [
-        {"type": "system", "subtype": "init"},
-        {
-            "type": "assistant",
-            "message": {
-                "content": [
-                    {"type": "text", "text": "SECRET-REASONING-abc123"},
-                    {
-                        "type": "tool_use",
-                        "id": "t1",
-                        "name": "Grep",
-                        "input": {
-                            "pattern": "TODO",
-                            "path": "/workspace",
-                            "token": "SECRET-TOKEN-abc123",
-                        },
-                    },
-                ]
-            },
-        },
-        {
-            "type": "user",
-            "message": {
-                "content": [
-                    {
-                        "type": "tool_result",
-                        "tool_use_id": "t1",
-                        "is_error": False,
-                        "content": "src/a.py:1: TODO " + "x" * 1000,
-                    }
-                ]
-            },
-        },
-        {"type": "result", "num_turns": 1, "is_error": False, "total_cost_usd": 1.5},
-    ]
-    for event in events:
-        _observe(progress, (json.dumps(event) + "\n").encode())
-
-    output = stream.getvalue()
-    assert "SECRET-REASONING-abc123" not in output
-    assert "SECRET-TOKEN-abc123" not in output
-    assert "[REDACTED]" in output
-    assert "session initialized" in output
-    assert "turn 1 · Grep" in output
-    assert 'tool Grep input={"pattern":"TODO","path":"/workspace","token":"[REDACTED]"}' in output
-    assert "tool Grep result=ok output=" in output
-    assert "truncated" in output
-    assert "finished · 1 turns · ok · $1.50" in output
-
-
-def test_progress_reports_errors_and_mcp_io_but_skips_other_tool_payloads() -> None:
-    stream = io.StringIO()
-    progress = SessionProgress("flow", stream=stream, heartbeat_s=3600)
-    events = [
-        {
-            "type": "assistant",
-            "message": {
-                "content": [
-                    {
-                        "type": "tool_use",
-                        "id": "m1",
-                        "name": "mcp__gitnexus__query",
-                        "input": {"search_query": "call resolution"},
-                    },
-                    {
-                        "type": "tool_use",
-                        "id": "e1",
-                        "name": "Edit",
-                        "input": {"file_path": "secret.py", "new_string": "do not log"},
-                    },
-                ]
-            },
-        },
-        {
-            "type": "user",
-            "message": {
-                "content": [
-                    {
-                        "type": "tool_result",
-                        "tool_use_id": "m1",
-                        "is_error": True,
-                        "content": "repository is not indexed",
-                    },
-                    {
-                        "type": "tool_result",
-                        "tool_use_id": "e1",
-                        "content": "edited secret.py",
-                    },
-                ]
-            },
-        },
-    ]
-    for event in events:
-        _observe(progress, (json.dumps(event) + "\n").encode())
-
-    output = stream.getvalue()
-    assert 'tool mcp__gitnexus__query input={"search_query":"call resolution"}' in output
-    assert 'tool mcp__gitnexus__query result=error output="repository is not indexed"' in output
-    assert "do not log" not in output
-    assert "edited secret.py" not in output
-
-
-def test_progress_distinguishes_mcp_semantic_errors_from_transport_success() -> None:
-    stream = io.StringIO()
-    progress = SessionProgress("flow", stream=stream, heartbeat_s=3600)
-    events = [
-        {
-            "type": "assistant",
-            "message": {
-                "content": [
-                    {
-                        "type": "tool_use",
-                        "id": "m1",
-                        "name": "mcp__gitnexus__impact",
-                        "input": {"target": "missing", "direction": "upstream"},
-                    }
-                ]
-            },
-        },
-        {
-            "type": "user",
-            "message": {
-                "content": [
-                    {
-                        "type": "tool_result",
-                        "tool_use_id": "m1",
-                        "is_error": False,
-                        "content": [
-                            {
-                                "type": "text",
-                                "text": '{"error":"Target missing not found"}\n\n---\n**Next:** retry',
-                            }
-                        ],
-                    }
-                ]
-            },
-        },
-    ]
-    for event in events:
-        _observe(progress, (json.dumps(event) + "\n").encode())
-
-    assert "tool mcp__gitnexus__impact result=semantic-error" in stream.getvalue()
-
-
-def test_progress_calls_out_api_retries_because_that_is_the_stuck_signature() -> None:
-    stream = io.StringIO()
-    progress = SessionProgress("proposer", stream=stream, heartbeat_s=3600)
-    event = {
-        "type": "system",
-        "subtype": "api_retry",
-        "attempt": 7,
-        "max_retries": 10,
-        "retry_delay_ms": 34199.87,
-        "error": "unknown",
-    }
-    _observe(progress, (json.dumps(event) + "\n").encode())
-
-    line = _drain_lines(stream)[-1]
-    assert "API retry 7/10 in 34s" in line
-    assert "no response from the model endpoint" in line
-
-
-def test_progress_speaks_up_while_a_session_is_silent() -> None:
-    stream = io.StringIO()
-    with SessionProgress("proposer", stream=stream, heartbeat_s=0.05):
-        time.sleep(0.35)
-
-    heartbeats = [line for line in _drain_lines(stream) if "still running" in line]
-    assert heartbeats, "a silent session must still report that it is alive"
-    assert "0 turns" in heartbeats[0]
-
-
-def test_progress_survives_partial_chunks_garbage_and_unbounded_lines() -> None:
-    stream = io.StringIO()
-    progress = SessionProgress("proposer", stream=stream, heartbeat_s=3600)
-    payload = json.dumps(
-        {"type": "assistant", "message": {"content": [{"type": "tool_use", "id": "t1", "name": "Bash"}]}}
-    ).encode()
-    # An event split across reads, non-JSON noise, and a huge newline-free run.
-    _observe(progress, payload[:10])
-    _observe(progress, payload[10:] + b"\nnot json at all\n")
-    _observe(progress, b"x" * (4 * 1024 * 1024))
-    _observe(progress, b'\n{"type":"result","num_turns":2,"is_error":true}\n')
-
-    output = stream.getvalue()
-    assert "turn 1 · Bash" in output
-    assert "finished · 2 turns · error" in output
-
-
-def test_progress_sanitizes_a_hostile_tool_name() -> None:
-    stream = io.StringIO()
-    progress = SessionProgress("proposer", stream=stream, heartbeat_s=3600)
-    event = {
-        "type": "assistant",
-        "message": {"content": [{"type": "tool_use", "id": "t1", "name": "Bash\nFAKE-LOG-LINE injected"}]},
-    }
-    _observe(progress, (json.dumps(event) + "\n").encode())
-
-    assert "FAKE-LOG-LINE" not in stream.getvalue()
-    assert len(_drain_lines(stream)) == 1
-
-
-def test_progress_redacts_non_ascii_secrets_before_json_escaping() -> None:
-    stream = io.StringIO()
-    secret = "tokén-密码"
-    progress = SessionProgress("flow", stream=stream, heartbeat_s=3600, secrets=(secret,))
-    event = {
-        "type": "assistant",
-        "message": {
-            "content": [
-                {"type": "tool_use", "id": "t1", "name": "Grep", "input": {"token": secret}},
-            ]
-        },
-    }
-    _observe(progress, (json.dumps(event, ensure_ascii=False) + "\n").encode())
-
-    output = stream.getvalue()
-    assert secret not in output
-    assert json.dumps(secret)[1:-1] not in output
-    assert "[REDACTED]" in output
-
-
-def test_cell_failure_detail_line_explains_why_a_cell_failed() -> None:
-    from workflow_bench.runner import cell_failure_detail_line
-
-    assert cell_failure_detail_line("t", "workflow", 0, {"error_kind": None}) is None
-    assert cell_failure_detail_line("t", "workflow", 0, {"error_kind": "x"}) is None
-
-    line = cell_failure_detail_line(
-        "trivial-status-json-alias",
-        "candidate_workflow",
-        1,
-        {
-            "error_kind": "plan-evidence-invalid",
-            "error_detail": "unauthorized workspace path\ntoken=sk-secret-value",
-        },
-        ("sk-secret-value",),
-    )
-    assert line is not None
-    assert line.startswith("[trivial-status-json-alias][candidate_workflow][run 1] detail: ")
-    assert "unauthorized workspace path" in line
-    assert "sk-secret-value" not in line
-    assert "\n" not in line
-
-
-def test_cell_failure_detail_line_bounds_a_huge_detail() -> None:
-    from workflow_bench.runner import MAX_CELL_DETAIL_CHARS, cell_failure_detail_line
-
-    line = cell_failure_detail_line(
-        "t", "workflow", 0, {"error_kind": "session-error", "error_detail": {"stdout_tail": "y" * 50_000}}
-    )
-    assert line is not None
-    assert "truncated" in line
-    assert len(line) < MAX_CELL_DETAIL_CHARS + 200
-
-
-def test_progress_neutralizes_github_actions_annotation_forms() -> None:
-    rewritten = neutralize_ci_log_text(
-        "gitnexus/src/cli/optional-grammars.ts(18,36): error TS2307: Cannot find module "
-        "'gitnexus-shared'\n::error::Composite projects may not disable incremental compilation.\n"
-        "##[error]tsc failed"
-    )
-    assert "): error TS2307" not in rewritten
-    assert "): compiler-error TS2307" in rewritten
-    assert "::error::" not in rewritten
-    assert "[:]error::" in rewritten
-    assert "##[error]" not in rewritten
-    assert "# [error]tsc failed" in rewritten
-
-    stream = io.StringIO()
-    progress = SessionProgress("review-pr-2718-defect-ce_review-run0", stream=stream, heartbeat_s=3600)
-    events = [
-        {
-            "type": "assistant",
-            "message": {
-                "content": [{"type": "tool_use", "id": "b1", "name": "Bash", "input": {"command": "npx tsc --noEmit"}}]
-            },
-        },
-        {
-            "type": "user",
-            "message": {
-                "content": [
-                    {
-                        "type": "tool_result",
-                        "tool_use_id": "b1",
-                        "is_error": True,
-                        "content": "gitnexus/src/cli/optional-grammars.ts(18,36): error TS2307: Cannot find module 'gitnexus-shared'",
-                    }
-                ]
-            },
-        },
-    ]
-    for event in events:
-        _observe(progress, (json.dumps(event) + "\n").encode())
-
-    output = stream.getvalue()
-    assert "): error TS2307" not in output
-    assert "): compiler-error TS2307" in output
-    assert "result=error" in output

eval/tests/test_sweep_finalization.py

@@ -1,279 +0,0 @@
-"""The real sweep must reach the right finalization decision.
-
-`enforce_measurement_health` is unit-tested and the call site is pinned
-structurally, but neither shows the guard running inside a sweep. These drive
-the real `_run_sweep` with cell execution scripted and everything downstream of
-it left alone: folding, aggregation, the artifact writers, the health guard and
-the exit selection.
-
-The below-breaker case is the decisive one. A fixture of many unusable cells
-aborts through the pre-existing outage breaker instead - `review-evidence-invalid`
-is systemic with a limit of 5 - and would pass whether or not the finalization
-guard exists. One fresh unusable cell stays under that threshold, so only the
-guard can catch it.
-"""
-
-from __future__ import annotations
-
-import json
-import threading
-from collections.abc import Callable
-from pathlib import Path
-from types import SimpleNamespace
-from typing import Any
-
-import pytest
-
-from tests.bench_fixtures import scored_review_row, unusable_review_row
-from workflow_bench import runner
-
-TASK = {
-    "id": "review-pr-2718-defect",
-    "repo": "~/GitNexus",
-    "ref": "a" * 40,
-    "prompt": "review it",
-    "verify": "true",
-    "class": "review-defect",
-}
-
-
-def _args(out: Path, **overrides: Any) -> SimpleNamespace:
-    values: dict[str, Any] = dict(
-        arms=["review"], claude_bin="claude", effort="xhigh", model="gpt-5.6-sol",
-        out=out, outage_streak=runner.DEFAULT_OUTAGE_STREAK, promotion_max_task_regression=10.0,
-        promotion_metric="review_weighted_f1", promotion_min_improvement=1.0,
-        promotion_min_runs=1, proposer_model=None, reuse_results=None, runs=1, workers=1,
-        timeout=60, base_url=None, auth_token=None, permission_mode=None,
-    )
-    values.update(overrides)
-    return SimpleNamespace(**values)
-
-
-def _snapshot(prefix: str) -> SimpleNamespace:
-    return SimpleNamespace(
-        digest=f"{prefix}-content", manifest_digest=f"{prefix}-manifest",
-        dependency_content_digest=f"{prefix}-dep", dependency_manifest_digest=f"{prefix}-depman",
-        command_digest=f"{prefix}-command", materialize=lambda *a, **k: None,
-    )
-
-
-def _sweep(
-    tmp_path: Path,
-    monkeypatch: pytest.MonkeyPatch,
-    record: dict[str, Any] | Callable[[int], dict[str, Any]],
-    *,
-    runs: int = 1,
-    cancel_event: threading.Event | None = None,
-    candidate_arms: list[str] | None = None,
-    arms: list[str] | None = None,
-    after_cell: Callable[[int, str], None] | None = None,
-):
-    """Drive the real _run_sweep; only cell execution and setup are scripted.
-
-    ``after_cell`` runs once a cell's record exists, which is how a test sets
-    cancellation deterministically at a known point instead of racing a sleep.
-    """
-
-    out = tmp_path / "out"
-
-    def scripted_cell(_ctx: Any, run_idx: int, arm: str) -> dict[str, Any]:
-        row = dict(record(run_idx) if callable(record) else record)
-        row.update({"task": TASK["id"], "arm": arm, "run": run_idx, "class": TASK["class"]})
-        if after_cell is not None:
-            after_cell(run_idx, arm)
-        return row
-
-    monkeypatch.setattr(runner, "run_cell", scripted_cell)
-    monkeypatch.setattr(runner, "ensure_task_graph", lambda **k: k["env"].graph_snapshots.__setitem__(
-        k["graph_key"], _snapshot("graph")))
-    monkeypatch.setattr(runner.TaskAssetCache, "prepare", lambda self, *a, **k: _snapshot("asset"))
-    # Binding resolution clones the repo and verifies the ref; that is expensive
-    # setup, and the bindings it would return are supplied directly instead.
-    monkeypatch.setattr(
-        runner, "resolve_task_bindings",
-        lambda tasks, expected, **k: list(expected),
-    )
-
-    return runner._run_sweep(
-        _args(out, runs=runs, arms=arms or ["review"]),
-        parser=SimpleNamespace(error=lambda m: (_ for _ in ()).throw(SystemExit(2))),
-        tasks=[TASK],
-        skipped_expensive=[],
-        oracle_snapshots=[_snapshot("oracle")],
-        expected_task_bindings=[{"repo_identity": str(tmp_path / "repo"), "resolved_sha": "a" * 40}],
-        ce_plugin_config=None,
-        bwrap_bin=Path("/bin/true"),
-        sandbox_backend="test-double",
-        runtime_mounts=(),
-        candidate_arms=candidate_arms or [],
-        candidate_overlay=None,
-        overlay_digest=None,
-        promotion_target_bases={},
-        cancel_event=cancel_event,
-    ), out
-
-
-def test_one_unusable_cell_below_the_breaker_reaches_the_finalization_guard(
-    tmp_path: Path, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
-) -> None:
-    """The decisive case: too few failures to trip the breaker, so only the guard can catch it."""
-
-    streak = runner.systemic_outage_streak("review-evidence-invalid", 0)
-    assert streak < runner.DEFAULT_OUTAGE_STREAK, "fixture must stay under the breaker"
-
-    unusable = unusable_review_row()
-    with pytest.raises(SystemExit) as exc:
-        _sweep(tmp_path, monkeypatch, unusable)
-    assert exc.value.code == 1
-    out = capsys.readouterr().out
-    assert "review: UNUSABLE" in out, "the guard must name the arm and its status"
-    assert "systemic-outage" not in out, "the breaker must not have tripped"
-
-
-def test_a_zero_score_stays_a_valid_negative_measurement(
-    tmp_path: Path, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
-) -> None:
-    """0.0 is a present measurement, not missing evidence.
-
-    A truthiness check on the score would misread it as absent and turn a
-    quality result into an execution-health failure.
-    """
-
-    zeroed = scored_review_row(
-        resolved=False, error_kind="oracle-failed",
-        review_score={"weighted_f1": 0.0}, review_weighted_f1=0.0,
-    )
-    _sweep(tmp_path, monkeypatch, zeroed)
-    out = capsys.readouterr().out
-    assert "review: OBSERVED_OK" in out
-    assert "UNUSABLE" not in out
-
-
-def test_finalization_persists_results_and_report(
-    tmp_path: Path, monkeypatch: pytest.MonkeyPatch
-) -> None:
-    """Evidence must survive the sweep, and say the same thing the exit does."""
-
-    scored = scored_review_row(resolved=False, error_kind="oracle-failed", review_weighted_f1=0.2)
-    _result, out = _sweep(tmp_path, monkeypatch, scored)
-    rows = [json.loads(line) for line in (out / "results.jsonl").read_text().splitlines()]
-    assert len(rows) == 1 and rows[0]["review_weighted_f1"] == 0.2
-    assert (out / "report.md").is_file()
-
-
-def test_cancellation_without_an_outage_exits_130(
-    tmp_path: Path, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
-) -> None:
-    """An interrupted sweep is interrupted, not aborted.
-
-    One admissible cell lands first so the measurement-health guard classifies
-    the arm DEGRADED rather than UNUSABLE - otherwise the guard would supply
-    exit 1 and this test would pass without ever exercising exit selection.
-    """
-
-    cancel_event = threading.Event()
-    with pytest.raises(SystemExit) as exc:
-        _sweep(
-            tmp_path, monkeypatch, lambda _run: scored_review_row(),
-            runs=3, cancel_event=cancel_event,
-            after_cell=lambda run_idx, _arm: cancel_event.set() if run_idx == 0 else None,
-        )
-    stdout = capsys.readouterr().out
-    report = (tmp_path / "out" / "report.md").read_text()
-    assert "Sweep cancelled" in report, "an interruption must be reported as one"
-    assert "systemic-outage" not in stdout, "no breaker trip in this scenario"
-    assert exc.value.code == 130
-
-
-def test_an_outage_keeps_exit_1_even_though_the_breaker_cancels(
-    tmp_path: Path, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
-) -> None:
-    """Precedence: the breaker sets cancel_event, so order decides the exit.
-
-    Testing cancellation first would relabel every outage a Ctrl-C. The first
-    cell is admissible for the same reason as above, and the failures after it
-    are consecutive and systemic, which is what the breaker actually counts.
-    """
-
-    def cell(run_idx: int) -> dict[str, Any]:
-        return scored_review_row() if run_idx == 0 else unusable_review_row()
-
-    cancel_event = threading.Event()
-    with pytest.raises(SystemExit) as exc:
-        _sweep(tmp_path, monkeypatch, cell,
-               runs=1 + runner.DEFAULT_OUTAGE_STREAK, cancel_event=cancel_event)
-    stdout = capsys.readouterr().out
-    report = (tmp_path / "out" / "report.md").read_text()
-    assert "systemic-outage" in stdout, "the real breaker must have tripped"
-    assert cancel_event.is_set(), "the breaker cancels in-flight work"
-    assert "Sweep aborted" in report
-    assert exc.value.code == 1, "an outage must not become the 130 of a Ctrl-C"
-
-
-def test_an_interrupted_sweep_keeps_the_evidence_it_already_paid_for(
-    tmp_path: Path, monkeypatch: pytest.MonkeyPatch
-) -> None:
-    """Cancellation must not discard rows that already cost money.
-
-    The completed-run persistence test cannot show this: it never interrupts, so
-    it would pass even if the writer only ran on the clean path.
-    """
-
-    cancel_event = threading.Event()
-    with pytest.raises(SystemExit):
-        _sweep(
-            tmp_path, monkeypatch, lambda _run: scored_review_row(review_weighted_f1=0.42),
-            runs=3, cancel_event=cancel_event,
-            after_cell=lambda run_idx, _arm: cancel_event.set() if run_idx == 0 else None,
-        )
-    rows = [
-        json.loads(line)
-        for line in (tmp_path / "out" / "results.jsonl").read_text().splitlines()
-    ]
-    assert len(rows) == 1, "the cell that completed before cancellation must survive"
-    assert rows[0]["review_weighted_f1"] == 0.42, "its measurement must survive intact"
-    assert (tmp_path / "out" / "report.md").is_file()
-
-
-def test_an_interrupted_sweep_emits_nothing_that_authorizes_promotion(
-    tmp_path: Path, monkeypatch: pytest.MonkeyPatch
-) -> None:
-    """The semantic condition, not the absence of a file.
-
-    promotion.json is still written for an aborted run - it is the record of why
-    nothing was promoted. What must hold is that nothing in it authorizes a
-    promotion from partial evidence.
-    """
-
-    cancel_event = threading.Event()
-    with pytest.raises(SystemExit):
-        _sweep(
-            tmp_path, monkeypatch, lambda _run: scored_review_row(),
-            runs=3, cancel_event=cancel_event,
-            # The candidate arm has to RUN, not merely appear in promotion
-            # metadata: _run_sweep builds cells only from args.arms, so naming it
-            # in candidate_arms alone left the candidate with no results at all -
-            # and then "insufficient_evidence" would hold because nothing ran,
-            # not because partial evidence is barred from promoting.
-            arms=["review", "candidate_review"],
-            candidate_arms=["candidate_review"],
-            after_cell=(
-                lambda run_idx, arm: cancel_event.set()
-                if run_idx == 0 and arm == "candidate_review"
-                else None
-            ),
-        )
-    rows = [
-        json.loads(line)
-        for line in (tmp_path / "out" / "results.jsonl").read_text().splitlines()
-    ]
-    assert any(r["arm"] == "candidate_review" for r in rows), (
-        "the candidate must have produced evidence, or insufficient_evidence "
-        "would hold merely because nothing ran"
-    )
-    promotion = json.loads((tmp_path / "out" / "promotion.json").read_text())
-    assert promotion["run_status"] == "aborted"
-    assert promotion["decisions"], "an aborted run still has to say what it decided"
-    for decision in promotion["decisions"]:
-        assert decision["decision"] == "insufficient_evidence"
-        assert any("partial evidence" in reason for reason in decision["reasons"])

eval/tests/test_task_assets.py

@@ -5,15 +5,15 @@ from __future__ import annotations
 import os
 import stat
 import subprocess
-from pathlib import Path, PurePosixPath
+from pathlib import Path
 
 import pytest
 
 from workflow_bench.proposer_sandbox import VITE_TEMP_DIR, SandboxError
 from workflow_bench.oracle_assets import TaskOracleSnapshot
 from workflow_bench.runner_tasks import resolve_task_bindings
-from workflow_bench.task_assets import TaskAssetCache, _is_harness_sandbox_copy, stage_task_assets
+from workflow_bench.task_assets import TaskAssetCache, stage_task_assets
-from workflow_bench import runtime_mounts, task_assets
+from workflow_bench import task_assets
 
 
 SHA = "a" * 40
@@ -443,53 +443,3 @@ def test_non_node_modules_dependency_snapshot_has_no_vite_temp(tmp_path: Path) -
         snapshot = cache.prepare(task, repo=repo, resolved_sha=SHA)
         captured = {entry.path.as_posix() for entry in snapshot.dependencies[0].entries}
         assert not any(path.endswith(VITE_TEMP_DIR) for path in captured)
-
-
-def test_review_case_sandbox_copy_is_read_from_the_harness_not_the_task_repo(
-    monkeypatch, tmp_path: Path
-) -> None:
-    repo = tmp_path / "task-repo"
-    repo.mkdir()
-    (repo / "eval" / "workflow_bench").mkdir(parents=True)
-    harness = tmp_path / "harness"
-    patch = harness / "eval" / "workflow_bench" / "review_cases" / "pr-2718.patch"
-    patch.parent.mkdir(parents=True)
-    patch.write_bytes(b"diff --git a/a b/a\n")
-    monkeypatch.setattr(runtime_mounts, "HARNESS_ROOT", harness)
-
-    task = {
-        "sandbox_copy": ["eval/workflow_bench/review_cases/pr-2718.patch"],
-        "sandbox_dependencies": [],
-    }
-    with TaskAssetCache(tmp_path / "cache") as cache:
-        snapshot = cache.prepare(task, repo=repo, resolved_sha=SHA)
-        copied = snapshot.root / "sandbox-copy" / "eval" / "workflow_bench" / "review_cases" / "pr-2718.patch"
-        assert copied.read_bytes() == b"diff --git a/a b/a\n"
-
-
-def test_review_case_sandbox_copy_does_not_fall_back_to_the_task_repo(
-    monkeypatch, tmp_path: Path
-) -> None:
-    repo = tmp_path / "task-repo"
-    planted = repo / "eval" / "workflow_bench" / "review_cases" / "pr-2718.patch"
-    planted.parent.mkdir(parents=True)
-    planted.write_bytes(b"from-task-repo")
-    harness = tmp_path / "harness"
-    harness.mkdir()
-    monkeypatch.setattr(runtime_mounts, "HARNESS_ROOT", harness)
-
-    task = {
-        "sandbox_copy": ["eval/workflow_bench/review_cases/pr-2718.patch"],
-        "sandbox_dependencies": [],
-    }
-    with TaskAssetCache(tmp_path / "cache") as cache:
-        with pytest.raises(SandboxError, match="unavailable"):
-            cache.prepare(task, repo=repo, resolved_sha=SHA)
-
-
-def test_harness_sandbox_copy_does_not_treat_parent_escapes_as_corpus() -> None:
-    assert _is_harness_sandbox_copy(PurePosixPath("eval/workflow_bench/review_cases/pr.patch"))
-    assert not _is_harness_sandbox_copy(
-        PurePosixPath("eval/workflow_bench/review_cases/../oracles/hidden.json")
-    )
-    assert not _is_harness_sandbox_copy(PurePosixPath("eval/workflow_bench/oracles"))

eval/tests/test_workflow_bench.py

@@ -1,39 +1,24 @@
 """Unit tests for workflow benchmark aggregation, reporting, task, and CI contracts."""
 
 import json
-import os
 import re
-import shlex
 import subprocess
-import threading
 from pathlib import Path
 
 import pytest
 import yaml
 
-from typing import Any
-
-from workflow_bench import runner
-from workflow_bench.evolution import CANDIDATE_ARMS
-from workflow_bench.process_control import _CANCELLATION, cancellation_scope
 from workflow_bench.runner import (
     aggregate,
-    GraphBuildEnv,
-    arm_health,
     broken_incumbent_arms,
-    unhealthy_arms,
-    unmeasured_arms,
     build_parser,
     infra_error_record,
-    next_graph_prefetch_target,
     normalized_model_identifier,
     parse_shortstat,
-    prefetch_next_graph,
     render_report,
     savings,
     select_tasks,
     systemic_outage_streak,
-    task_has_planned_paid_cells,
 )
 
 
@@ -76,15 +61,6 @@ def test_aggregate_takes_medians_and_counts_resolved():
         "diff_deletions": 5,
         "class": "demo",
         "resolved": 2,
-        # None of these are reused, so every resolution was measured this sweep.
-        "resolved_fresh": 2,
-        # Health is counted separately from resolution: all three executed and
-        # produced usable evidence, including the one that resolved nothing.
-        "fresh_attempts": 3,
-        "admissible": 3,
-        "execution_failures": 0,
-        "evidence_failures": 0,
-        "health_reasons": [],
         "runs": 3,
         "valid_runs": 3,
         "excluded_runs": 0,
@@ -113,7 +89,7 @@ def task_row(task_id: str, **overrides):
             "command": "true",
             "files": [
                 {
-                    "source": "trivial-status-json-alias.oracle.test.ts",
+                    "source": "trivial-version-alias.oracle.test.ts",
                     "target": "oracle.test.ts",
                 }
             ],
@@ -185,7 +161,7 @@ def test_eval_ci_uses_locked_uv_and_blocking_native_containment_jobs():
         step for step in containment["steps"] if str(step.get("uses", "")).startswith("actions/setup-node@")
     )
     claude_lock = json.loads((repo_root / ".github" / "claude-canary-runtime" / "package-lock.json").read_text())
-    setup_uv = "astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7"
+    setup_uv = "astral-sh/setup-uv@11f9893b081a58869d3b5fccaea48c9e9e46f990"
     assert workflow.count(setup_uv) >= 3
     assert workflow.count("version: '0.11.23'") >= 3
     assert workflow.count("uv run --locked --extra dev python -m pytest") >= 3
@@ -195,11 +171,6 @@ def test_eval_ci_uses_locked_uv_and_blocking_native_containment_jobs():
     assert containment["env"] == {
         "GITNEXUS_REQUIRE_BWRAP_CANARY": "1",
         "GITNEXUS_REQUIRE_CLAUDE_CANARY": "1",
-        # This job is the only place with bubblewrap, the pinned runtime and a
-        # built GitNexus together, so it is where the offline sweep runs with
-        # nothing provisioning-stubbed. Pinned here so the gate cannot be
-        # dropped and leave the sweep silently running the stubbed path.
-        "GITNEXUS_REQUIRE_FULL_SWEEP": "1",
     }
     assert containment["timeout-minutes"] == 20
     assert containment_node_setup["with"] == {
@@ -244,17 +215,7 @@ def test_eval_ci_uses_locked_uv_and_blocking_native_containment_jobs():
         "tests/test_proposer_sandbox.py",
         "tests/test_workflow_bench_sessions.py",
         "tests/test_ce_plugin_runtime.py",
-        # The offline sweep, run here with nothing stubbed: this job is the only
-        # one carrying bubblewrap, the pinned runtime and a built GitNexus.
-        "tests/test_offline_sweep_integration.py",
-        # Carries the real-CLI identity probe, which needs CLAUDE_CANARY_BIN -
-        # set only on this job. Omitted from this list it skipped everywhere.
-        "tests/test_mock_provider.py",
-        # These also need runtime dependencies absent from the locked pytest job.
-        "tests/test_evolve.py::test_outer_runner_pid_namespace_kills_setsid_descendant",
-        "tests/test_oracle_assets.py::test_hidden_vitest_config_executes_sibling_oracle_against_candidate_checkout",
         "-q",
-        "--junitxml=pytest-ubuntu.xml",
     ]
     bwrap_canary_marker = re.compile(
         r'@pytest\.mark\.skipif\(\s*os\.environ\.get\("GITNEXUS_REQUIRE_BWRAP_CANARY"\)',
@@ -270,87 +231,18 @@ def test_eval_ci_uses_locked_uv_and_blocking_native_containment_jobs():
     assert "eval-containment-windows:" in workflow
 
 
-@pytest.fixture(scope="module")
-def ci_test_jobs():
-    repo_root = Path(__file__).resolve().parents[2]
-    workflow = yaml.safe_load((repo_root / ".github" / "workflows" / "ci-tests.yml").read_text())
-    return workflow["jobs"]
-
-
-def test_platform_ci_preserves_required_check_names_with_complete_execution_gates(ci_test_jobs):
-    jobs = ci_test_jobs
-    gate = jobs["protected-platform-checks"]
-    matrix = gate["strategy"]["matrix"]
-    names = {
-        "tests / " + gate["name"].replace("${{ matrix.os }}", platform).replace("${{ matrix.check }}", str(check))
-        for platform in matrix["os"]
-        for check in matrix["check"]
-    }
-    assert names == {
-        f"tests / {platform} (platform-sensitive) {check}/3"
-        for platform in ("windows-latest", "macos-latest")
-        for check in (1, 2, 3)
-    }
-    plan = jobs["shard-plan"]["steps"][0]["run"]
-    total = int(re.search(r"^TOTAL=(\d+)\b", plan, re.MULTILINE)[1])
-    native_names = {
-        "tests / "
-        + jobs["cross-platform"]["name"]
-        .replace("${{ matrix.os }}", platform)
-        .replace("${{ matrix.shard }}", str(shard))
-        .replace("${{ needs.shard-plan.outputs.total }}", str(total))
-        for platform in jobs["cross-platform"]["strategy"]["matrix"]["os"]
-        for shard in range(1, total + 1)
-    }
-    assert names.isdisjoint(native_names)
-    assert set(gate["needs"]) == {"cross-platform", "test-completeness"}
-    assert gate["if"] == "always()"
-    assert not gate.get("continue-on-error", False)
-    assert not jobs["cross-platform"].get("continue-on-error", False)
-    assert not jobs["test-completeness"].get("continue-on-error", False)
-    assert gate["permissions"] == {}
-    assert len(gate["steps"]) == 1
-    step = gate["steps"][0]
-    assert not step.get("continue-on-error", False)
-    assert "if" not in step
-    assert step["shell"] == "bash"
-    assert step["env"] == {
-        "NATIVE_RESULT": "${{ needs.cross-platform.result }}",
-        "EXECUTION_RESULT": "${{ needs.test-completeness.result }}",
-    }
-
-
-@pytest.mark.parametrize("native_result", ["success", "failure", "cancelled", "skipped", ""])
-@pytest.mark.parametrize("execution_result", ["success", "failure", "cancelled", "skipped", ""])
-def test_required_platform_check_fails_unless_every_dependency_passed(ci_test_jobs, native_result, execution_result):
-    step = ci_test_jobs["protected-platform-checks"]["steps"][0]
-    result = subprocess.run(
-        ["bash", "--noprofile", "--norc", "-e", "-o", "pipefail", "-c", step["run"]],
-        env={**os.environ, "NATIVE_RESULT": native_result, "EXECUTION_RESULT": execution_result},
-        capture_output=True,
-        text=True,
-        timeout=10,
-        check=False,
-    )
-    expected = 0 if native_result == execution_result == "success" else 1
-    assert result.returncode == expected, result.stdout + result.stderr
-
-
 def test_shipped_scenarios_opt_out_the_cross_module_cell_and_rebuild_graph_assets():
     task_file = Path(__file__).resolve().parents[1] / "workflow_bench" / "tasks.scenarios.yaml"
     tasks = yaml.safe_load(task_file.read_text())["tasks"]
     selected, skipped = select_tasks(tasks, include_expensive=False)
     assert [task["id"] for task in selected] == [
-        "trivial-status-json-alias",
+        "trivial-version-alias",
-        "inv-bug-c-system-include",
+        "inv-bug-pdg-note",
         "inv-feature-list-repos-filter",
     ]
     assert skipped == ["cross-module-parse-retry"]
     assert all(not task.get("sandbox_copy") for task in tasks)
     assert all(task["sandbox_dependencies"] for task in tasks)
-    assert all(
-        any(dep.get("source") == "gitnexus-shared/dist" for dep in task["sandbox_dependencies"]) for task in tasks
-    )
     assert all(task["oracle"]["command"] and task["oracle"]["files"] for task in tasks)
     assert all("./node_modules/.bin/vitest run" in task["oracle"]["command"] for task in tasks)
     assert all("npx vitest" not in task["oracle"]["command"] for task in tasks)
@@ -428,36 +320,6 @@ def test_aggregate_excludes_unverified_transcript_evidence():
     assert agg["excluded_runs"] == 1
 
 
-def test_aggregate_excludes_invalid_review_artifacts_from_quality_metrics():
-    scored = record(
-        cost_usd=1.0,
-        review_weighted_f1=0.8,
-        review_true_positives=2,
-        review_false_positives=0,
-        review_false_negatives=1,
-        review_precision=1.0,
-        review_recall=0.67,
-        review_f1=0.8,
-        review_weighted_precision=0.8,
-        review_weighted_recall=0.8,
-        review_blocker_recall=1.0,
-        review_severity_accuracy=1.0,
-        review_category_accuracy=1.0,
-        review_grounded_evidence=1.0,
-        review_clean_control=False,
-    )
-    agg = aggregate(
-        [
-            scored,
-            record(cost_usd=2.0, resolved=False, error_kind="review-evidence-invalid"),
-        ]
-    )
-    assert agg["valid_runs"] == 1
-    assert agg["excluded_runs"] == 1
-    assert agg["review_weighted_f1"] == 0.8
-    assert agg["review_true_positives"] == 2
-
-
 def test_render_report_surfaces_excluded_and_unverified_runs():
     results = {
         "t": {
@@ -565,600 +427,3 @@ def test_outage_streak_flag_defaults_and_disables():
     base = ["--tasks", "tasks.yaml", "--model", "claude-sonnet-4-20250514"]
     assert build_parser().parse_args(base).outage_streak == 5
     assert build_parser().parse_args([*base, "--outage-streak", "0"]).outage_streak == 0
-
-
-def test_run_evolution_script_is_the_shared_ci_and_local_entrypoint():
-    eval_dir = Path(__file__).resolve().parents[1]
-    script = eval_dir / "workflow_bench" / "run-evolution.sh"
-    workflow = eval_dir.parent / ".github" / "workflows" / "gitnexus-skill-evolution.yml"
-    assert script.is_file()
-    assert script.stat().st_mode & 0o111
-    workflow_text = workflow.read_text()
-    assert "./workflow_bench/run-evolution.sh --apply" in workflow_text
-    assert "python -m workflow_bench.evolve" not in workflow_text
-
-    env = {
-        "PATH": os.environ.get("PATH", "/usr/bin"),
-        "MODEL": "claude-sonnet-5",
-        "PROPOSER_MODEL": "claude-opus-4-8",
-        "EFFORT": "xhigh",
-        "GENERATIONS": "1",
-        "RUNS": "3",
-        "WORKERS": "2",
-        "PROVIDER": "openai",
-        "INCLUDE_EXPENSIVE": "1",
-        "SEED_RESULTS": "/tmp/seed-bench",
-        "CLAUDE_BIN": "/opt/claude",
-        "OUT_ROOT": "/tmp/wfevolve",
-        "CE_PLUGIN_DIR": "/tmp/ce-plugin",
-        "CE_PLUGIN_VERSION": "3.24.0",
-        "HOME": os.environ.get("HOME", "/tmp"),
-    }
-    printed = subprocess.run(
-        [str(script), "--dry-run", "--apply"],
-        check=True,
-        capture_output=True,
-        text=True,
-        env=env,
-    )
-    argv = shlex.split(printed.stdout)
-    assert argv[:7] == ["uv", "run", "--locked", "--extra", "dev", "python", "-m"]
-    assert argv[7] == "workflow_bench.evolve"
-    assert argv[argv.index("--tasks") + 1] == "workflow_bench/tasks.review.scenarios.yaml"
-    assert argv[argv.index("--arms") + 1] == "review"
-    assert argv[argv.index("--ce-plugin-version") + 1] == "3.24.0"
-    assert argv[argv.index("--model") + 1] == "gpt-5.6-sol"
-    assert argv[argv.index("--proposer-model") + 1] == "gpt-5.6-sol"
-    assert argv[argv.index("--effort") + 1] == "xhigh"
-    assert argv[argv.index("--workers") + 1] == "2"
-    assert argv[argv.index("--claude-bin") + 1] == "/opt/claude"
-    assert argv[argv.index("--out-root") + 1] == "/tmp/wfevolve"
-    assert argv[argv.index("--seed-results") + 1] == "/tmp/seed-bench"
-    assert "--apply" in argv
-    assert "--include-expensive" in argv
-    assert "claude-sonnet-5" not in argv
-    assert printed.stderr  # rewrite notice goes to stderr
-
-
-def test_planned_paid_cells_treat_missing_reuse_as_paid():
-    task = {"id": "review-pr-2718-defect"}
-    assert task_has_planned_paid_cells(
-        task,
-        arms=["ce_review", "review", "candidate_review"],
-        runs=3,
-        reusable_rows={},
-        reuse_source=None,
-    )
-    reuse_source = Path("/tmp/seed")
-    rows = {
-        (task["id"], arm, run_idx): {}
-        for run_idx in range(3)
-        for arm in ("ce_review", "review", "candidate_review")
-    }
-    assert not task_has_planned_paid_cells(
-        task,
-        arms=["ce_review", "review", "candidate_review"],
-        runs=3,
-        reusable_rows=rows,
-        reuse_source=reuse_source,
-    )
-    del rows[(task["id"], "candidate_review", 0)]
-    assert task_has_planned_paid_cells(
-        task,
-        arms=["ce_review", "review", "candidate_review"],
-        runs=3,
-        reusable_rows=rows,
-        reuse_source=reuse_source,
-    )
-
-
-def test_next_graph_prefetch_skips_ready_shas_and_fully_reused_tasks(tmp_path: Path):
-    first = {"id": "review-a"}
-    second = {"id": "review-b"}
-    third = {"id": "review-c"}
-    reuse_source = tmp_path / "seed"
-    reused_second = {
-        (second["id"], arm, 0): {} for arm in ("ce_review", "review", "candidate_review")
-    }
-    target = next_graph_prefetch_target(
-        [
-            (first, {"repo_identity": "/repo", "resolved_sha": "aaa"}),
-            (second, {"repo_identity": "/repo", "resolved_sha": "bbb"}),
-            (third, {"repo_identity": "/repo", "resolved_sha": "ccc"}),
-        ],
-        arms=["ce_review", "review", "candidate_review"],
-        runs=1,
-        reusable_rows=reused_second,
-        reuse_source=reuse_source,
-        ready_keys={("/repo", "aaa")},
-    )
-    assert target is not None
-    task, binding, key = target
-    assert task["id"] == "review-c"
-    assert key == ("/repo", "ccc")
-    assert binding["resolved_sha"] == "ccc"
-
-
-def test_prefetch_next_graph_runs_ensure_on_a_background_thread(monkeypatch):
-    started = threading.Event()
-    seen: list[tuple[str, str]] = []
-
-    def fake_ensure(**kwargs):
-        seen.append(kwargs["graph_key"])
-        started.set()
-
-    monkeypatch.setattr("workflow_bench.runner.ensure_task_graph", fake_ensure)
-    cancel = threading.Event()
-    job = prefetch_next_graph(
-        task={"id": "review-b"},
-        binding={"repo_identity": "/repo", "resolved_sha": "bbb"},
-        graph_key=("/repo", "bbb"),
-        env=GraphBuildEnv(
-            trees=Path("/tmp"),
-            task_asset_cache=None,
-            claude_bin="claude",
-            bwrap_bin="bwrap",
-            sandbox_backend="bwrap",
-            runtime_mounts=(),
-            clone_templates={},
-            clone_template_errors={},
-            graph_snapshots={},
-            graph_snapshot_errors={},
-        ),
-        cancel_event=cancel,
-    )
-    assert job.key == ("/repo", "bbb")
-    assert started.wait(timeout=2)
-    job.join()
-    assert seen == [("/repo", "bbb")]
-
-
-def test_a_reused_resolution_does_not_count_as_this_sweeps_health():
-    """resolved counts evidence; resolved_fresh counts evidence measured today.
-
-    broken_incumbent_arms reads resolved_fresh because a reused row proves last
-    generation's environment worked. Counting it would make an arm whose cells
-    were all reused look healthy in exactly the run where a broken environment
-    should have been caught.
-    """
-
-    reused = [record(resolved=True, reused=True), record(resolved=True, reused=True)]
-    agg = aggregate(reused)
-    assert agg["resolved"] == 2
-    assert agg["resolved_fresh"] == 0
-    assert broken_incumbent_arms({"t": {"review": agg}}, {"review"}) == ["review"]
-
-    mixed = aggregate([record(resolved=True, reused=True), record(resolved=True)])
-    assert mixed["resolved_fresh"] == 1
-    assert broken_incumbent_arms({"t": {"review": mixed}}, {"review"}) == []
-
-
-def test_graph_build_env_ready_keys_covers_successes_and_failures():
-    """A key that failed is attempted, not pending.
-
-    next_graph_prefetch_target skips keys already in ready_keys. If a failed
-    build were omitted, the sweep would prefetch it again every iteration and
-    pay a full clone and offline index each time for a build that cannot
-    succeed.
-    """
-
-    env = GraphBuildEnv(
-        trees=Path("/tmp"),
-        task_asset_cache=None,
-        claude_bin="claude",
-        bwrap_bin="bwrap",
-        sandbox_backend="bwrap",
-        runtime_mounts=(),
-        clone_templates={("/repo", "aaa"): (Path("/tmp/a"), "aaa")},
-        clone_template_errors={("/repo", "bbb"): OSError("clone failed")},
-        graph_snapshots={("/repo", "ccc"): object()},
-        graph_snapshot_errors={("/repo", "ddd"): OSError("index failed")},
-    )
-    assert env.ready_keys() == {
-        ("/repo", "aaa"),
-        ("/repo", "bbb"),
-        ("/repo", "ccc"),
-        ("/repo", "ddd"),
-    }
-
-
-def _cell(**overrides) -> dict[str, Any]:
-    """One results.jsonl row, healthy unless told otherwise."""
-
-    base = record(resolved=True)
-    base.update({"error_kind": None, "review_evidence_valid": True, "transcript_missing": False})
-    base.update(overrides)
-    return base
-
-
-def _arms(**by_arm) -> dict[str, dict[str, dict[str, Any]]]:
-    return {"task0": {arm: aggregate(rows) for arm, rows in by_arm.items()}}
-
-
-def test_a_reviewer_that_scores_badly_is_not_an_unhealthy_harness():
-    """Reconstructed from Actions run 33962002890's logged observations.
-
-    Every completed cell was resolved=False with error_kind=oracle-failed, at a
-    median score of 0.212 — the reviews ran, wrote artifacts and were scored.
-    That is a valid negative for the quality gate to judge. Diagnosing it as a
-    broken environment is the confusion this classification exists to end.
-    """
-
-    scored_but_wrong = [_cell(resolved=False, error_kind="oracle-failed") for _ in range(3)]
-    results = _arms(review=scored_but_wrong, ce_review=list(scored_but_wrong))
-    assert unhealthy_arms(results, {"review", "ce_review"}) == []
-    health = arm_health(results, {"review"})["review"]
-    assert health.admissible == 3 and health.fresh_attempts == 3
-    assert (health.execution_failures, health.evidence_failures) == (0, 0)
-
-
-def test_an_all_zero_score_is_still_a_valid_negative():
-    zeroed = [_cell(resolved=False, error_kind="oracle-failed", review_weighted_f1=0.0) for _ in range(3)]
-    assert unhealthy_arms(_arms(review=zeroed), {"review"}) == []
-
-
-def test_artifacts_that_were_never_written_are_an_unhealthy_harness():
-    """Reconstructed from Actions run 33912693948.
-
-    All 41 artifacts came back 0 bytes because the mount made an atomic write
-    impossible. The reviews could not produce evidence at all — the opposite of
-    the case above, and the one a health check must catch. The old caller
-    excluded review arms entirely, so it could not have.
-    """
-
-    unwritable = [_cell(resolved=False, ok=False, error_kind="review-evidence-invalid") for _ in range(3)]
-    flagged = unhealthy_arms(_arms(review=unwritable), {"review"})
-    assert [h.arm for h in flagged] == ["review"]
-    assert flagged[0].evidence_failures == 3
-    assert "review-evidence-invalid" in flagged[0].reasons
-
-
-def test_one_admissible_cell_leaves_an_arm_degraded_not_healthy():
-    """Mixed outcomes are DEGRADED. One usable measurement does not erase two failures.
-
-    Not fatal - the sweep still produced evidence - but calling it healthy is
-    how a partly-broken environment passes review.
-    """
-
-    mixed = [
-        _cell(resolved=False, error_kind="oracle-failed"),
-        _cell(resolved=False, ok=False, error_kind="session-error"),
-        _cell(resolved=False, ok=False, error_kind="infra-error"),
-    ]
-    results = _arms(review=mixed)
-    health = arm_health(results, {"review"})["review"]
-    assert health.status == "DEGRADED"
-    assert unhealthy_arms(results, {"review"}) == [], "degraded is diagnostic, not fatal"
-    assert health.execution_failures == 2, "failures must stay visible, not be erased"
-    assert health.admissible == 1
-
-
-def test_a_row_that_fails_both_ways_is_only_subtracted_once():
-    """run_arm can produce a row that is an execution AND an evidence failure.
-
-    It keeps the first error_kind — a session-error survives — and still sets
-    review_evidence_valid=False when the artifact will not parse. Counting that
-    row against admissible twice zeroed an arm that held a real measurement,
-    which arm_health reports as UNUSABLE and the measurement gate then fails on.
-    """
-
-    both = _cell(resolved=False, ok=False, error_kind="session-error", review_evidence_valid=False)
-    results = _arms(review=[both, _cell(resolved=True, error_kind="oracle-failed")])
-    health = arm_health(results, {"review"})["review"]
-    assert (health.execution_failures, health.evidence_failures) == (1, 1)
-    assert health.fresh_attempts == 2
-    assert health.admissible == 1
-    assert health.status == "DEGRADED"
-    assert unhealthy_arms(results, {"review"}) == []
-
-
-def test_reused_rows_alone_leave_current_health_unknown():
-    """Historical success cannot certify this sweep's environment."""
-
-    reused = [_cell(reused=True) for _ in range(3)]
-    results = _arms(review=reused)
-    assert unmeasured_arms(results, {"review"}) == ["review"]
-    assert unhealthy_arms(results, {"review"}) == []
-    assert arm_health(results, {"review"})["review"].measured is False
-
-
-def test_the_paid_canary_survives_a_prior_run_with_more_run_indices():
-    """The canary counts planned cells, not every key reuse selection returned.
-
-    Reuse selection accepts any non-negative prior `run`, so a results directory
-    produced with --runs 5 leaves keys this sweep never plans. Comparing against
-    those made the "arm is fully reused" test false exactly when it was true,
-    and the incumbent went a whole sweep without one measured cell.
-    """
-
-    tasks = [{"id": "task0"}, {"id": "task1"}]
-    reusable = {(task["id"], "review", run): {} for task in tasks for run in range(5)}
-
-    dropped = runner.drop_canary_reuse_key(reusable, arm="review", tasks=tasks, runs=3)
-
-    assert dropped == ("task0", "review", 0)
-    assert dropped not in reusable
-    # A second call is a no-op: the arm now has its paid cell.
-    assert runner.drop_canary_reuse_key(reusable, arm="review", tasks=tasks, runs=3) is None
-
-
-def test_an_arm_with_a_planned_paid_cell_keeps_every_reusable_row():
-    tasks = [{"id": "task0"}]
-    reusable = {("task0", "review", 0): {}}
-
-    assert runner.drop_canary_reuse_key(reusable, arm="review", tasks=tasks, runs=2) is None
-    assert len(reusable) == 1
-
-
-def test_reused_successes_do_not_mask_fresh_execution_failures():
-    rows = [_cell(reused=True), _cell(reused=True), _cell(ok=False, error_kind="session-error")]
-    flagged = unhealthy_arms(_arms(review=rows), {"review"})
-    assert [h.arm for h in flagged] == ["review"]
-    assert flagged[0].fresh_attempts == 1 and flagged[0].execution_failures == 1
-
-
-def test_a_parseable_artifact_does_not_excuse_a_failed_session():
-    """Artifact parseability must not override an execution failure."""
-
-    rows = [_cell(ok=False, error_kind="session-error", review_evidence_valid=True) for _ in range(2)]
-    flagged = unhealthy_arms(_arms(review=rows), {"review"})
-    assert [h.arm for h in flagged] == ["review"]
-    assert flagged[0].execution_failures == 2
-
-
-def test_a_single_unusable_review_is_caught_below_the_breaker_threshold():
-    """The decisive regression for the finalization guard.
-
-    A fixture of 41 empty artifacts would abort through the outage breaker -
-    review-evidence-invalid is systemic and the limit is 5 - so it proves
-    nothing about this path. One fresh unusable cell is under that threshold,
-    which leaves the finalization check as the only thing that can catch it.
-    """
-
-    streak = 0
-    for _ in range(1):
-        streak = runner.systemic_outage_streak("review-evidence-invalid", streak)
-    assert streak < runner.DEFAULT_OUTAGE_STREAK, "fixture must not reach the breaker"
-
-    results = _arms(review=[_cell(resolved=False, ok=False, error_kind="review-evidence-invalid")])
-    with pytest.raises(SystemExit) as exc:
-        runner.enforce_measurement_health(results, {"review"})
-    assert exc.value.code == 1
-
-
-def test_finalization_reports_every_arm_and_names_no_cause(capsys):
-    """Status for each arm; an empty artifact does not become an EROFS diagnosis."""
-
-    results = _arms(
-        review=[_cell(resolved=False, ok=False, error_kind="review-evidence-invalid")],
-        ce_review=[_cell(resolved=False, error_kind="oracle-failed")],
-    )
-    with pytest.raises(SystemExit):
-        runner.enforce_measurement_health(results, {"review", "ce_review"})
-    out = capsys.readouterr().out
-    assert "review: UNUSABLE" in out
-    assert "ce_review: OBSERVED_OK" in out
-    assert "cause=undetermined" in out
-    assert "EROFS" not in out and "mount" not in out
-
-
-def test_valid_negatives_do_not_abort_finalization(capsys):
-    """The 16h run's shape must survive the real guard, not just the classifier."""
-
-    scored_but_wrong = [_cell(resolved=False, error_kind="oracle-failed") for _ in range(3)]
-    health = runner.enforce_measurement_health(
-        _arms(review=scored_but_wrong, ce_review=list(scored_but_wrong)), {"review", "ce_review"}
-    )
-    assert {h.status for h in health.values()} == {"OBSERVED_OK"}
-    assert "UNUSABLE" not in capsys.readouterr().out
-
-
-def test_reused_only_arm_is_reported_unknown_by_finalization(capsys):
-    runner.enforce_measurement_health(_arms(review=[_cell(reused=True)]), {"review"})
-    assert "review: UNKNOWN" in capsys.readouterr().out
-
-
-def test_run_sweep_calls_the_health_guard_and_not_the_legacy_helper():
-    """Pins the wiring the caller correction exposed.
-
-    Reads the compiled code object's global references rather than the source
-    text: deleting the call removes the name and fails this test, which is the
-    mutation check. It does NOT prove the guard runs end to end - _run_sweep
-    needs bwrap and a sandbox, so no test here drives it.
-    """
-
-    referenced = runner._run_sweep.__code__.co_names
-    assert "enforce_measurement_health" in referenced
-    assert "broken_incumbent_arms" not in referenced
-
-
-def test_ce_review_is_classified_even_though_it_is_not_a_candidate_arm():
-    """ce_review is a comparator, absent from CANDIDATE_ARMS.
-
-    Dropping the `- {"review"}` exclusion alone would have left it unchecked.
-    """
-
-    assert "ce_review" not in set(CANDIDATE_ARMS.values())
-    health = arm_health(_arms(ce_review=[_cell()]), {"review", "ce_review"})
-    assert "ce_review" in health
-
-
-def _packed_cells(tasks: int, runs: int, arms: tuple[str, ...]) -> list[tuple[str, int, str]]:
-    return [(f"t{t}", r, a) for t in range(tasks) for r in range(runs) for a in arms]
-
-
-def test_packed_sweep_runs_every_cell_and_folds_in_submission_order():
-    """Fold order is the contract the breaker rests on.
-
-    Cells finish in whatever order the pool returns them, but the breaker counts
-    CONSECUTIVE systemic failures, which only means something in a fixed order.
-    """
-
-    cells = _packed_cells(3, 2, ("review", "candidate_review"))
-    folded: list[tuple[str, int, str]] = []
-    streak, tripped = runner.sweep_packed_cells(
-        cells,
-        workers=4,
-        run=lambda task, run_idx, arm: {"error_kind": None, "review_evidence_valid": True},
-        on_start=lambda *_: None,
-        on_record=lambda task, run_idx, arm, _rec: folded.append((task, run_idx, arm)),
-        outage_streak=0,
-        outage_limit=0,
-    )
-    assert folded == cells
-    assert (streak, tripped) == (0, False)
-
-
-def test_packed_sweep_trips_the_breaker_on_the_same_cell_waves_would():
-    """Packing must not change WHEN a doomed run aborts, only how it is fed."""
-
-    cells = _packed_cells(3, 3, ("review",))
-    fail_from = 2
-    folded: list[int] = []
-
-    def run(task: str, run_idx: int, arm: str) -> dict[str, Any]:
-        index = cells.index((task, run_idx, arm))
-        systemic = index >= fail_from
-        return {
-            "error_kind": "session-error" if systemic else None,
-            "review_evidence_valid": not systemic,
-        }
-
-    streak, tripped = runner.sweep_packed_cells(
-        cells,
-        workers=2,
-        run=run,
-        on_start=lambda *_: None,
-        on_record=lambda t, r, a, _rec: folded.append(cells.index((t, r, a))),
-        outage_streak=0,
-        outage_limit=runner.DEFAULT_OUTAGE_STREAK,
-    )
-    assert tripped is True
-    assert streak == runner.DEFAULT_OUTAGE_STREAK
-    # Five consecutive systemic failures starting at index 2 -> trips on index 6.
-    assert folded[-1] == fail_from + runner.DEFAULT_OUTAGE_STREAK - 1
-    assert folded == sorted(folded), "records must fold in submission order"
-
-
-def test_packed_sweep_skips_a_task_whose_assets_never_arrive():
-    """A task that cannot be prepared is skipped, not run against nothing."""
-
-    cells = _packed_cells(3, 2, ("review",))
-    ran: list[str] = []
-    runner.sweep_packed_cells(
-        cells,
-        workers=3,
-        run=lambda task, run_idx, arm: ran.append(task)
-        or {"error_kind": None, "review_evidence_valid": True},
-        on_start=lambda *_: None,
-        on_record=lambda *_: None,
-        outage_streak=0,
-        outage_limit=0,
-        await_ready=lambda task: task != "t1",
-    )
-    assert set(ran) == {"t0", "t2"}
-    assert "t1" not in ran
-
-
-def test_packed_sweep_workers_inherit_the_runs_cancellation_event():
-    """A worker that cannot see the event runs on after the sweep is cancelled.
-
-    The cells are submitted from a producer THREAD, and a new thread starts with
-    an empty context - so copying the context at submission copies the wrong one
-    unless the caller's is captured first. run_managed falls back to
-    _CANCELLATION when no event is passed, which is how a cell's subprocesses
-    learn the run was cancelled at all.
-    """
-
-    seen: list[threading.Event | None] = []
-    event = threading.Event()
-    with cancellation_scope(event):
-        runner.sweep_packed_cells(
-            _packed_cells(2, 1, ("review",)),
-            workers=2,
-            run=lambda *_: seen.append(_CANCELLATION.get()) or {"error_kind": None},
-            on_start=lambda *_: None,
-            on_record=lambda *_: None,
-            outage_streak=0,
-            outage_limit=0,
-        )
-    assert seen and all(observed is event for observed in seen)
-
-
-def test_packed_sweep_window_must_keep_the_pool_fed():
-    with pytest.raises(ValueError, match="window must be at least workers"):
-        runner.sweep_packed_cells(
-            _packed_cells(1, 1, ("review",)),
-            workers=4,
-            run=lambda *_: {"error_kind": None},
-            on_start=lambda *_: None,
-            on_record=lambda *_: None,
-            outage_streak=0,
-            outage_limit=0,
-            window=2,
-        )
-
-
-def test_a_raising_packed_cell_still_persists_its_settled_siblings():
-    """A crash in one cell must not erase the evidence of cells that finished.
-
-    run_cell deliberately lets unexpected harness exceptions propagate, and the
-    wave scheduler answers that by folding every non-failing sibling before it
-    re-raises. The packed scheduler has to hold the same contract: the later
-    cells already ran and already cost money, so losing their rows would mean
-    paying for evidence the sweep then throws away.
-    """
-
-    folded: list[tuple[int, str]] = []
-    started = threading.Event()
-
-    def run(task_id: str, run_idx: int, arm: str) -> dict[str, Any]:
-        if run_idx == 0:
-            # Let the later cell finish first, so there is settled evidence to
-            # lose at the moment this one raises.
-            started.wait(timeout=5)
-            raise RuntimeError("harness bug in cell 0")
-        started.set()
-        return {"error_kind": None}
-
-    with pytest.raises(RuntimeError, match="harness bug in cell 0"):
-        runner.sweep_packed_cells(
-            _packed_cells(1, 2, ("review",)),
-            workers=2,
-            run=run,
-            on_start=lambda *_: None,
-            on_record=lambda task_id, run_idx, arm, _rec: folded.append((run_idx, arm)),
-            outage_streak=0,
-            outage_limit=0,
-        )
-
-    assert (1, "review") in folded, "the sibling that completed was never recorded"
-
-
-def test_an_uninvoked_skill_still_counts_toward_the_arm_median():
-    """Pins a KNOWN GAP, not a desired behaviour.
-
-    A cell whose skill never ran still moves the arm's quality median, even
-    though an arm exists to measure a SKILL. The narrow fix - filtering those
-    rows out of the quality metrics - is worse than the gap: valid_runs and
-    excluded_runs keep counting them, so the promotion gate sees N clean runs
-    while the median came from fewer. Since the dropped rows are systematically
-    an arm's worst, that biases toward promoting, and it was measured flipping
-    keep_incumbent to promote.
-
-    Closing it honestly needs a scored-run count and a paired-equality check in
-    the promotion gate. Pinned here so the half-fix cannot be reapplied without
-    someone reading why it was reverted.
-    """
-
-    good = record(review_weighted_f1=1.0, cost_usd=2.0)
-    uninvoked = record(
-        review_weighted_f1=0.0, cost_usd=4.0, error_kind="skill-not-invoked", skill_invoked=False
-    )
-    agg = aggregate([good, uninvoked])
-
-    assert agg["review_weighted_f1"] == 0.5, "the uninvoked row is counted - the known gap"
-    assert agg["cost_usd"] == 3.0
-    # The invariant that makes the half-fix unsafe: the median and the run count
-    # the gate reads must cover the same rows.
-    assert agg["valid_runs"] == 2

eval/tests/test_workflow_bench_evolution.py

@@ -8,18 +8,14 @@ from types import SimpleNamespace
 import pytest
 
 from workflow_bench.evolution import (
-    CANDIDATE_SKILLS,
     MAX_CANDIDATE_ENTRIES,
     apply_candidate_overlay,
     candidate_overlay_digest,
     evaluate_candidate,
-    evaluate_review_candidate,
     required_candidate_arms,
-    seed_evaluated_skills,
     skill_fingerprint,
     unexercised_overlay_skills,
 )
-from workflow_bench.promotion_apply import mirror_targets
 from workflow_bench.process_control import ManagedProcessResult
 from workflow_bench.runner import aggregate, build_parser
 
@@ -189,7 +185,8 @@ def test_candidate_overlay_is_skill_only_and_content_addressed(tmp_path):
     review_skill = review_overlay / ".claude" / "skills" / "gitnexus-review" / "SKILL.md"
     review_skill.parent.mkdir(parents=True)
     review_skill.write_text("review candidate\n")
-    assert candidate_overlay_digest(review_overlay)
+    with pytest.raises(ValueError, match="plan,work"):
+        candidate_overlay_digest(review_overlay)
 
     invalid = tmp_path / "invalid"
     source = invalid / "gitnexus" / "src" / "cli" / "index.ts"
@@ -241,116 +238,6 @@ def test_required_candidate_arms_are_minimal_for_touched_skills(tmp_path):
         "candidate_workflow_direct",
     ]
 
-    review = tmp_path / "review"
-    write_overlay_skill(review, "gitnexus-review")
-    assert required_candidate_arms(review) == ["candidate_review"]
-
-
-def test_review_gate_is_quality_first_and_requires_repeated_evidence():
-    def arm(score, blocker=1.0, false_positives=0, runs=3):
-        return {
-            "runs": runs,
-            "valid_runs": runs,
-            "excluded_runs": 0,
-            "class": "review-defect",
-            "review_weighted_f1": score,
-            "review_blocker_recall": blocker,
-            "review_false_positives": false_positives,
-            "review_clean_control": False,
-            "review_verdict_correct": True,
-        }
-
-    decision = evaluate_review_candidate(
-        {
-            "case-a": {
-                "review": arm(0.6),
-                "candidate_review": arm(0.8),
-            }
-        },
-        incumbent_arm="review",
-        candidate_arm="candidate_review",
-        model="pinned-model",
-    )
-    assert decision["decision"] == "promote"
-
-    regression = evaluate_review_candidate(
-        {
-            "case-a": {
-                "review": arm(0.6, blocker=1.0),
-                "candidate_review": arm(0.8, blocker=0.0),
-            }
-        },
-        incumbent_arm="review",
-        candidate_arm="candidate_review",
-        model="pinned-model",
-    )
-    assert regression["decision"] == "keep_incumbent"
-    assert any("blocker recall" in reason for reason in regression["reasons"])
-
-
-def test_review_gate_rejects_added_false_positives_on_clean_controls():
-    base = {
-        "runs": 3,
-        "valid_runs": 3,
-        "excluded_runs": 0,
-        "class": "review-clean",
-        "review_weighted_f1": 1.0,
-        "review_blocker_recall": 1.0,
-        "review_clean_control": True,
-        "review_clean_pass": True,
-        "review_verdict_correct": True,
-    }
-    decision = evaluate_review_candidate(
-        {
-            "clean": {
-                "review": {**base, "review_false_positives": 0},
-                "candidate_review": {**base, "review_false_positives": 1, "review_clean_pass": False},
-            }
-        },
-        incumbent_arm="review",
-        candidate_arm="candidate_review",
-        model="pinned-model",
-    )
-    assert decision["decision"] == "keep_incumbent"
-    assert any("clean control" in reason for reason in decision["reasons"])
-
-
-@pytest.mark.parametrize("wrong_verdict,bad_blocker", [(True, False), (False, True), (False, False)])
-def test_review_gate_preserves_every_repeat_safeguard(wrong_verdict, bad_blocker):
-    from workflow_bench.runner import aggregate
-
-    def row(score, verdict=True, blocker=1.0):
-        return {
-            "resolved": True,
-            "review_weighted_f1": score,
-            "review_blocker_recall": blocker,
-            "review_false_positives": 0,
-            "review_verdict_correct": verdict,
-            "review_clean_control": False,
-            "review_clean_pass": False,
-        }
-
-    incumbent = aggregate([row(0.5) for _ in range(3)])
-    candidate = aggregate([row(0.8), row(0.8), row(0.8, not wrong_verdict, 0.0 if bad_blocker else 1.0)])
-    decision = evaluate_review_candidate(
-        {"case": {"review": incumbent, "candidate_review": candidate}},
-        incumbent_arm="review",
-        candidate_arm="candidate_review",
-        model="pinned-model",
-    )
-    assert decision["decision"] == ("keep_incumbent" if wrong_verdict or bad_blocker else "promote")
-
-
-def test_review_gate_treats_an_empty_corpus_as_insufficient_evidence():
-    decision = evaluate_review_candidate(
-        {},
-        incumbent_arm="review",
-        candidate_arm="candidate_review",
-        model="pinned-model",
-    )
-    assert decision["decision"] == "insufficient_evidence"
-    assert any("no paired review task results" in reason for reason in decision["reasons"])
-
 
 @pytest.mark.skipif(os.name == "nt", reason="candidate overlays require the Linux outer sandbox")
 def test_apply_candidate_overlay_creates_a_clean_ephemeral_commit(tmp_path):
@@ -427,12 +314,6 @@ def test_apply_candidate_overlay_creates_a_clean_ephemeral_commit(tmp_path):
     ) == candidate_overlay_digest(overlay)
     assert incumbent.read_text() == "candidate\n"
     git_commands = [command for command in sandbox.commands if command[0] == "/usr/bin/git"]
-    assert git_commands[0][-4:] == [
-        "add",
-        "-f",
-        "--",
-        ".claude/skills/gitnexus-work/SKILL.md",
-    ]
     assert [command[-1] for command in git_commands[:2]] == [
         ".claude/skills/gitnexus-work/SKILL.md",
         "--",
@@ -468,181 +349,6 @@ def test_candidate_overlay_rejects_linked_destination_parents(tmp_path):
         apply_candidate_overlay(overlay, repo, sandbox=sandbox)
 
 
-@pytest.mark.skipif(os.name == "nt", reason="candidate overlays require the Linux outer sandbox")
-def test_apply_candidate_overlay_force_adds_historically_ignored_skill(tmp_path):
-    repo = tmp_path / "repo"
-    repo.mkdir()
-    subprocess.run(["git", "init", "--quiet", str(repo)], check=True)
-    (repo / ".gitignore").write_text(".claude/skills/*\n")
-    (repo / "README").write_text("subject\n")
-    subprocess.run(["git", "-C", str(repo), "add", "."], check=True)
-    subprocess.run(
-        [
-            "git",
-            "-C",
-            str(repo),
-            "-c",
-            "user.name=test",
-            "-c",
-            "user.email=test@invalid",
-            "commit",
-            "--quiet",
-            "-m",
-            "historical checkout that ignores skills",
-        ],
-        check=True,
-    )
-
-    overlay = tmp_path / "candidate"
-    write_overlay_skill(overlay, "gitnexus-review")
-
-    class LocalSandbox:
-        def __init__(self):
-            self.clone = repo
-
-        def run(self, command, **kwargs):
-            if command[0] == "/bin/mkdir":
-                return ManagedProcessResult(
-                    state="exited",
-                    returncode=0,
-                    stdout_tail="",
-                    stderr_tail="",
-                    duration_s=0.0,
-                )
-            translated = [str(repo) if item == "/workspace" else item for item in command]
-            completed = subprocess.run(
-                translated,
-                cwd=repo,
-                env=dict(kwargs["env"]),
-                capture_output=True,
-                text=True,
-                check=False,
-            )
-            return ManagedProcessResult(
-                state="exited",
-                returncode=completed.returncode,
-                stdout_tail=completed.stdout,
-                stderr_tail=completed.stderr,
-                duration_s=0.0,
-            )
-
-    apply_candidate_overlay(overlay, repo, sandbox=LocalSandbox())
-    assert (repo / ".claude" / "skills" / "gitnexus-review" / "SKILL.md").read_text() == (
-        "gitnexus-review candidate\n"
-    )
-    status = subprocess.run(
-        ["git", "-C", str(repo), "status", "--porcelain"],
-        check=True,
-        capture_output=True,
-        text=True,
-    )
-    assert status.stdout == ""
-
-
-@pytest.mark.skipif(os.name == "nt", reason="skill seeds require the Linux outer sandbox")
-def test_seed_evaluated_skills_installs_missing_review_skill_and_is_idempotent(tmp_path):
-    repo = tmp_path / "clone"
-    repo.mkdir()
-    subprocess.run(["git", "init", "--quiet", str(repo)], check=True)
-    # Historical review SHAs ignore the whole skill tree and lack today's
-    # `!.claude/skills/gitnexus-review/` allowlist. Seeding must still commit.
-    (repo / ".gitignore").write_text(".claude/skills/*\n")
-    (repo / "README").write_text("subject\n")
-    subprocess.run(["git", "-C", str(repo), "add", "."], check=True)
-    subprocess.run(
-        [
-            "git",
-            "-C",
-            str(repo),
-            "-c",
-            "user.name=test",
-            "-c",
-            "user.email=test@invalid",
-            "commit",
-            "--quiet",
-            "-m",
-            "historical checkout without review skill",
-        ],
-        check=True,
-    )
-    before = subprocess.run(
-        ["git", "-C", str(repo), "rev-parse", "HEAD"],
-        check=True,
-        capture_output=True,
-        text=True,
-    ).stdout.strip()
-
-    source = tmp_path / "harness"
-    skill = source / ".claude" / "skills" / "gitnexus-review" / "SKILL.md"
-    persona = source / ".claude" / "skills" / "gitnexus-review" / "ci-personas" / "lens.md"
-    persona.parent.mkdir(parents=True)
-    skill.write_text("current incumbent review skill\n")
-    persona.write_text("persona\n")
-
-    class LocalSandbox:
-        def __init__(self):
-            self.clone = repo
-
-        def run(self, command, **kwargs):
-            if command[0] == "/bin/mkdir":
-                return ManagedProcessResult(
-                    state="exited",
-                    returncode=0,
-                    stdout_tail="",
-                    stderr_tail="",
-                    duration_s=0.0,
-                )
-            translated = [str(repo) if item == "/workspace" else item for item in command]
-            completed = subprocess.run(
-                translated,
-                cwd=repo,
-                env=dict(kwargs["env"]),
-                capture_output=True,
-                text=True,
-                check=False,
-            )
-            return ManagedProcessResult(
-                state="exited",
-                returncode=completed.returncode,
-                stdout_tail=completed.stdout,
-                stderr_tail=completed.stderr,
-                duration_s=0.0,
-            )
-
-    sandbox = LocalSandbox()
-    seed_evaluated_skills(source, repo, sandbox=sandbox, arm="review")
-    assert skill_fingerprint(repo, "review") is not None
-    assert (repo / ".claude" / "skills" / "gitnexus-review" / "SKILL.md").read_text() == (
-        "current incumbent review skill\n"
-    )
-    assert (
-        repo / ".claude" / "skills" / "gitnexus-review" / "ci-personas" / "lens.md"
-    ).read_text() == "persona\n"
-    status = subprocess.run(
-        ["git", "-C", str(repo), "status", "--porcelain"],
-        check=True,
-        capture_output=True,
-        text=True,
-    )
-    assert status.stdout == ""
-    after = subprocess.run(
-        ["git", "-C", str(repo), "rev-parse", "HEAD"],
-        check=True,
-        capture_output=True,
-        text=True,
-    ).stdout.strip()
-    assert after != before
-
-    seed_evaluated_skills(source, repo, sandbox=sandbox, arm="review")
-    again = subprocess.run(
-        ["git", "-C", str(repo), "rev-parse", "HEAD"],
-        check=True,
-        capture_output=True,
-        text=True,
-    ).stdout.strip()
-    assert again == after
-
-
 @pytest.mark.skipif(os.name == "nt", reason="skill links are rejected by the Linux sandbox harness")
 def test_skill_fingerprint_rejects_linked_skill_roots(tmp_path):
     outside = tmp_path / "outside"
@@ -718,7 +424,9 @@ def test_candidate_gate_refuses_promotion_on_unmeasured_cost():
     results = {
         "task-a": {
             "workflow_direct": aggregate([record(cost_usd=1.0) for _ in range(3)]),
-            "candidate_workflow_direct": aggregate([record(cost_usd=0.1), record(cost_usd=None), record(cost_usd=0.1)]),
+            "candidate_workflow_direct": aggregate(
+                [record(cost_usd=0.1), record(cost_usd=None), record(cost_usd=0.1)]
+            ),
         }
     }
     decision = evaluate_candidate(
@@ -802,23 +510,8 @@ def test_candidate_gate_rejects_a_partial_candidate_even_with_a_resolution_edge(
     assert any("oracle-backed quality floor" in reason for reason in decision["reasons"])
 
 
-@pytest.mark.parametrize(
+@pytest.mark.parametrize("resolved", [0, 2])
-    ("resolved", "expected_decision", "expected_reason"),
+def test_candidate_gate_never_promotes_zero_or_partial_success_for_efficiency(resolved):
-    [
-        # Nothing resolved anywhere: the task is ungated, which leaves the
-        # generation with no quality signal at all — refuse outright rather
-        # than rank a 100x cost win across runs that all failed the oracle.
-        (0, "insufficient_evidence", "no task supplied quality signal"),
-        # Partial success on a task the incumbent also partly resolves stays a
-        # quality-floor rejection: the candidate has to be reliable, not lucky.
-        (2, "keep_incumbent", "oracle-backed quality floor"),
-    ],
-)
-def test_candidate_gate_never_promotes_zero_or_partial_success_for_efficiency(
-    resolved,
-    expected_decision,
-    expected_reason,
-):
     incumbent_records = [record(cost_usd=1.0, resolved=index < resolved) for index in range(3)]
     candidate_records = [record(cost_usd=0.01, resolved=index < resolved) for index in range(3)]
     decision = evaluate_candidate(
@@ -833,184 +526,9 @@ def test_candidate_gate_never_promotes_zero_or_partial_success_for_efficiency(
         model="pinned-model",
     )
 
-    assert decision["decision"] == expected_decision
+    assert decision["decision"] == "keep_incumbent"
     assert decision["tasks"][0]["candidate_quality_floor_met"] is False
-    assert any(expected_reason in reason for reason in decision["reasons"])
+    assert any("oracle-backed quality floor" in reason for reason in decision["reasons"])
-
-
-def test_a_task_no_arm_can_resolve_is_reported_but_does_not_veto_promotion():
-    # inv-feature-list-repos-filter fails its hidden oracle on every run of
-    # both arms. Gating on it made promotion unreachable for as long as it
-    # stayed in the set, while saying nothing about the candidate.
-    solvable = {
-        "workflow": aggregate([record(cost_usd=1.0, resolved=index > 1) for index in range(3)]),
-        "candidate_workflow": aggregate([record(cost_usd=1.0) for _ in range(3)]),
-    }
-    unsolvable = {
-        "workflow": aggregate([record(cost_usd=1.0, resolved=False) for _ in range(3)]),
-        "candidate_workflow": aggregate([record(cost_usd=1.3, resolved=False) for _ in range(3)]),
-    }
-    decision = evaluate_candidate(
-        {"task-a": solvable, "task-impossible": unsolvable},
-        incumbent_arm="workflow",
-        candidate_arm="candidate_workflow",
-        model="pinned-model",
-    )
-
-    assert decision["decision"] == "promote"
-    assert decision["ungated_tasks"] == ["task-impossible"]
-    assert decision["gated_tasks"] == ["task-a"]
-    assert [row["gated"] for row in decision["tasks"]] == [True, False]
-    # The ungated task's 30% cost regression stays under the failed-task cap
-    # but must not reach the median or the (tighter) gated per-task cap.
-    assert decision["median_improvement_pct"] == 0.0
-    assert not any("above the" in reason for reason in decision["reasons"])
-    # One aggregate line, so a growing set of unsolvable tasks cannot crowd the
-    # real verdict out of the three reasons the proposer is shown — and it
-    # discloses how much of the set the verdict actually rests on.
-    assert [reason for reason in decision["reasons"] if "not gated on" in reason] == [
-        "not gated on 1 task(s) neither arm resolved: task-impossible (evidence base: 1/2 paired tasks gated)"
-    ]
-
-
-def test_an_ungated_task_still_ranks_against_the_failed_task_cost_cap():
-    # Leaving the quality gate is not leaving the spend gate: burning 9x the
-    # incumbent's cost to fail the same oracle is a regression the gate has to
-    # see, or a candidate can hide unbounded waste inside "task health".
-    solvable = {
-        "workflow": aggregate([record(cost_usd=1.0, resolved=index > 1) for index in range(3)]),
-        "candidate_workflow": aggregate([record(cost_usd=1.0) for _ in range(3)]),
-    }
-    unsolvable = {
-        "workflow": aggregate([record(cost_usd=1.0, resolved=False) for _ in range(3)]),
-        "candidate_workflow": aggregate([record(cost_usd=9.0, resolved=False) for _ in range(3)]),
-    }
-
-    decision = evaluate_candidate(
-        {"task-a": solvable, "task-impossible": unsolvable},
-        incumbent_arm="workflow",
-        candidate_arm="candidate_workflow",
-        model="pinned-model",
-    )
-
-    assert decision["decision"] == "keep_incumbent"
-    assert decision["ungated_tasks"] == ["task-impossible"]
-    assert any("failed-task cap" in reason for reason in decision["reasons"])
-
-
-def test_a_mutually_failed_task_stays_gated_when_the_skill_never_loaded():
-    # skill-not-invoked is prompt evidence, not task health: the skill under
-    # test never ran, so the task cannot be written off as beyond both arms.
-    results = {
-        "task-a": {
-            "workflow": aggregate([record(cost_usd=1.0, resolved=False) for _ in range(3)]),
-            "candidate_workflow": aggregate(
-                [record(cost_usd=0.01, resolved=False, error_kind="skill-not-invoked") for _ in range(3)]
-            ),
-        }
-    }
-
-    decision = evaluate_candidate(
-        results,
-        incumbent_arm="workflow",
-        candidate_arm="candidate_workflow",
-        model="pinned-model",
-    )
-
-    assert decision["ungated_tasks"] == []
-    assert decision["tasks"][0]["gated"] is True
-    assert decision["tasks"][0]["skill_attributable_failure"] is True
-    # Gated with teeth: the 99% cost "win" must not carry a candidate whose
-    # skill never loaded.
-    assert decision["decision"] == "keep_incumbent"
-    assert any("never invoked the skill under test" in reason for reason in decision["reasons"])
-
-
-def test_a_mutually_failed_task_stays_gated_when_its_metric_was_never_measured():
-    # Ungating is a claim about spend as well as quality. With no measured
-    # cost there is nothing to claim, so the task stays in the gate and the
-    # missing measurement is named instead of silently skipped.
-    results = {
-        "task-a": {
-            "workflow": aggregate([record(cost_usd=1.0, resolved=False) for _ in range(3)]),
-            "candidate_workflow": aggregate(
-                [record(cost_usd=None, resolved=False), *(record(cost_usd=0.1, resolved=False) for _ in range(2))]
-            ),
-        }
-    }
-
-    decision = evaluate_candidate(
-        results,
-        incumbent_arm="workflow",
-        candidate_arm="candidate_workflow",
-        model="pinned-model",
-    )
-
-    assert decision["ungated_tasks"] == []
-    assert decision["decision"] == "insufficient_evidence"
-    assert any("was not measured on every run" in reason for reason in decision["reasons"])
-
-
-def test_partial_progress_on_a_task_the_incumbent_never_resolves_is_not_punished():
-    # Resolving 1 of 3 runs where the incumbent resolves none is strictly
-    # better than resolving none — which the gate ungates and forgives. Holding
-    # the partial run to the quality floor made improvement score worse than
-    # inaction.
-    def outcome(candidate_resolved: int) -> dict[str, object]:
-        return evaluate_candidate(
-            {
-                "task-a": {
-                    "workflow": aggregate([record(cost_usd=1.0) for _ in range(3)]),
-                    "candidate_workflow": aggregate([record(cost_usd=0.5) for _ in range(3)]),
-                },
-                "task-hard": {
-                    "workflow": aggregate([record(cost_usd=1.0, resolved=False) for _ in range(3)]),
-                    "candidate_workflow": aggregate(
-                        [record(cost_usd=1.0, resolved=index < candidate_resolved) for index in range(3)]
-                    ),
-                },
-            },
-            incumbent_arm="workflow",
-            candidate_arm="candidate_workflow",
-            model="pinned-model",
-        )
-
-    no_progress = outcome(0)
-    some_progress = outcome(1)
-
-    assert no_progress["decision"] == "promote"
-    assert no_progress["ungated_tasks"] == ["task-hard"]
-    # The partial run gives the task quality signal, so it is gated — but as
-    # improvement, not as a floor failure the zero-progress candidate escapes.
-    assert some_progress["decision"] == "promote"
-    assert some_progress["ungated_tasks"] == []
-    assert some_progress["tasks"][1]["quality_floor_enforced"] is False
-    assert not any("quality floor" in reason for reason in some_progress["reasons"])
-
-
-def test_promotion_requires_a_gated_majority_of_the_paired_tasks():
-    # Two of three tasks written off as task health leaves one task deciding
-    # the whole promotion. Ungating keeps promotion reachable; it must not
-    # hollow out the evidence base that makes a promotion mean anything.
-    solvable = {
-        "workflow": aggregate([record(cost_usd=1.0, resolved=index > 1) for index in range(3)]),
-        "candidate_workflow": aggregate([record(cost_usd=0.1) for _ in range(3)]),
-    }
-    unsolvable = {
-        "workflow": aggregate([record(cost_usd=1.0, resolved=False) for _ in range(3)]),
-        "candidate_workflow": aggregate([record(cost_usd=1.0, resolved=False) for _ in range(3)]),
-    }
-
-    decision = evaluate_candidate(
-        {"task-a": solvable, "task-impossible": unsolvable, "task-impossible-2": dict(unsolvable)},
-        incumbent_arm="workflow",
-        candidate_arm="candidate_workflow",
-        model="pinned-model",
-    )
-
-    assert decision["decision"] == "insufficient_evidence"
-    assert decision["gated_tasks"] == ["task-a"]
-    assert any("evidence base is too thin" in reason for reason in decision["reasons"])
 
 
 def test_candidate_gate_promotes_on_a_two_run_resolution_margin():
@@ -1035,34 +553,3 @@ def test_overlay_skills_must_be_exercised_by_selected_candidate_arms(tmp_path):
     write_overlay_skill(plan_overlay, "gitnexus-plan")
     assert unexercised_overlay_skills(plan_overlay, ["candidate_workflow_direct"]) == ["gitnexus-plan"]
     assert unexercised_overlay_skills(plan_overlay, ["candidate_workflow"]) == []
-
-
-@pytest.mark.parametrize("skill", sorted(CANDIDATE_SKILLS))
-def test_a_promoted_skill_is_visible_to_git_status_in_every_shipped_tree(skill):
-    """A promotion the repository cannot see is a promotion that never happens.
-
-    The workflow detects an applied promotion with `git status --porcelain`,
-    which is blind to ignored paths, and `.claude/skills/*` is ignored with a
-    hand-maintained per-skill allowlist. A candidate skill missing from that
-    allowlist would leave the run reporting "No promotion this run" after the
-    gate had already said promote — silently, and only after a full generation
-    of benchmark spend.
-    """
-    repo_root = Path(__file__).resolve().parents[2]
-    from pathlib import PurePosixPath
-
-    targets = [
-        str(path.parent)
-        for path in mirror_targets(PurePosixPath(".claude/skills") / skill / "SKILL.md")
-    ]
-    ignored = [
-        target
-        for target in targets
-        if subprocess.run(
-            ["git", "check-ignore", "-q", f"{target}/SKILL.md"],
-            cwd=repo_root,
-            check=False,
-        ).returncode
-        == 0
-    ]
-    assert ignored == []

eval/tests/test_workflow_bench_sessions.py

@@ -4,7 +4,6 @@ import argparse
 import hashlib
 import json
 import os
-import shutil
 import subprocess
 import sys
 from pathlib import Path
@@ -12,10 +11,9 @@ from types import SimpleNamespace
 
 import pytest
 
-from workflow_bench import evolve, runner, runner_artifacts, runner_sessions, runtime_mounts
+from workflow_bench import evolve, runner, runner_sessions, runtime_mounts
 from workflow_bench.evolution import skill_fingerprint
-from workflow_bench.process_control import ManagedProcessError, ManagedProcessResult
+from workflow_bench.process_control import ManagedProcessResult
-from workflow_bench.proposer_sandbox import SandboxError
 from workflow_bench.runner import snapshot_plan_docs
 
 
@@ -73,7 +71,6 @@ def bench_args(**overrides):
         "claude_bin": "claude",
         "timeout": 5,
         "model": None,
-        "effort": "xhigh",
         "base_url": None,
         "auth_token": None,
         "permission_mode": None,
@@ -120,16 +117,10 @@ def skill_events(skill_input: dict, *, tool_id: str = "skill-1", is_error: bool
 
 
 def fake_sandbox(root: Path) -> SimpleNamespace:
-    # private_root is NOT the clone. Conflating them puts the review artifact
-    # directory inside the workspace, which the real sandbox never does and
-    # which hides whether the workspace was left untouched.
-    private_root = root.parent / f"{root.name}-sandbox-private"
-    private_root.mkdir(exist_ok=True)
     return SimpleNamespace(
-        backend="test-double",
         claude_bin="claude",
         clone=root,
-        private_root=private_root,
+        private_root=root,
         command_prefix=[],
         command_prefix_for=lambda **_kwargs: [],
         settings_json="{}",
@@ -176,25 +167,6 @@ def test_run_claude_forwards_the_named_model_to_every_session(monkeypatch, tmp_p
     assert captured[captured.index("--model") + 1] == "claude-sonnet-4-20250514"
 
 
-def test_run_claude_forwards_xhigh_effort_to_every_session(monkeypatch, tmp_path):
-    captured: list[str] = []
-
-    def fake_run(command, **kwargs):
-        captured.extend(command)
-        return fake_cli_result(VALID_REPORT)
-
-    monkeypatch.setattr(runner_sessions, "run_managed", fake_run)
-    runner.run_claude(
-        "task",
-        tmp_path,
-        claude_bin="claude",
-        timeout=5,
-        model="gpt-5.6-sol",
-        effort="xhigh",
-    )
-    assert captured[captured.index("--effort") + 1] == "xhigh"
-
-
 def test_run_claude_restricts_tools_via_tools_flag_outside_bare(monkeypatch, tmp_path):
     # Outside --bare, the built-in toolset defaults to everything (subagents,
     # WebFetch, Task, ...) and --allowedTools only pre-approves within that —
@@ -323,18 +295,10 @@ def test_run_arm_keeps_session_error_kind_over_verify(monkeypatch, tmp_path):
 
 def test_agent_tool_grants_are_exact_and_nomcp_has_no_graph_tools(monkeypatch, tmp_path):
     read_only = runner.allowed_agent_tools(implementation=False)
-    review_tools = runner.allowed_agent_tools(implementation=False, allow_edit=False)
     implementation = runner.allowed_agent_tools(implementation=True)
     no_mcp = runner.allowed_agent_tools(implementation=True, include_mcp=False)
 
     assert read_only == [*runner.BUILTIN_AGENT_TOOLS, *runner.GITNEXUS_READ_ONLY_TOOLS]
-    assert review_tools == [
-        tool
-        for tool in [*runner.BUILTIN_AGENT_TOOLS, *runner.GITNEXUS_READ_ONLY_TOOLS]
-        if tool != "Edit"
-    ]
-    assert "Write" in review_tools
-    assert "Edit" not in review_tools
     assert implementation == [
         *runner.BUILTIN_AGENT_TOOLS,
         *runner.GITNEXUS_READ_ONLY_TOOLS,
@@ -362,7 +326,7 @@ def test_agent_tool_grants_are_exact_and_nomcp_has_no_graph_tools(monkeypatch, t
         )
 
     assert captured[0]["allowed_tools"] == read_only  # planning
-    assert captured[1]["allowed_tools"] == review_tools  # review
+    assert captured[1]["allowed_tools"] == read_only  # review
     assert captured[2]["allowed_tools"] == implementation
     assert captured[3]["allowed_tools"] == list(runner.BUILTIN_AGENT_TOOLS)
     assert captured[3]["mcp_config_json"] == '{"mcpServers":{}}'
@@ -391,7 +355,7 @@ def test_mcp_config_uses_only_the_minimal_pinned_harness_runtime(monkeypatch, tm
         directory.mkdir(parents=True)
     (runtime / "dist" / "cli" / "index.js").write_text("")
     (runtime / "hooks" / "claude" / "resolve-analyze-cmd.cjs").write_text("")
-    (runtime / "package.json").write_text(json.dumps({"version": "9.9.9-test"}))
+    (runtime / "package.json").write_text(json.dumps({"version": runner.PINNED_GITNEXUS_VERSION}))
     (runtime / "node_modules" / "gitnexus-shared").symlink_to(shared, target_is_directory=True)
     (shared / "package.json").write_text(json.dumps({"name": "gitnexus-shared"}))
     monkeypatch.setattr(runtime_mounts, "HARNESS_ROOT", tmp_path)
@@ -415,6 +379,9 @@ def test_mcp_config_uses_only_the_minimal_pinned_harness_runtime(monkeypatch, tm
         (shared / "package.json", f"{runner.SANDBOX_GITNEXUS_SHARED}/package.json"),
         (runtime / "hooks" / "claude", f"{runner.SANDBOX_GITNEXUS}/hooks/claude"),
     ]
+    package = json.loads((runtime / "package.json").read_text())
+    assert package["version"] == runner.PINNED_GITNEXUS_VERSION
+
     mounted_sources = {mount.source for mount in mounts}
     mounted_targets = {mount.target for mount in mounts}
     assert runtime not in mounted_sources
@@ -432,118 +399,6 @@ def test_mcp_config_uses_only_the_minimal_pinned_harness_runtime(monkeypatch, tm
     assert f"{runner.SANDBOX_GITNEXUS}/hooks" not in mounted_targets
 
 
-def _install_pinned_runtime(root: Path) -> None:
-    runtime = root / "gitnexus"
-    shared = root / "gitnexus-shared"
-    for directory in (
-        runtime / "dist" / "cli",
-        runtime / "node_modules",
-        runtime / "vendor",
-        runtime / "hooks" / "claude",
-        shared / "dist",
-    ):
-        directory.mkdir(parents=True)
-    (runtime / "dist" / "cli" / "index.js").write_text("")
-    (runtime / "hooks" / "claude" / "resolve-analyze-cmd.cjs").write_text("")
-    (runtime / "package.json").write_text(json.dumps({"version": "9.9.9-test"}))
-    (runtime / "node_modules" / "gitnexus-shared").symlink_to(shared, target_is_directory=True)
-    (shared / "package.json").write_text(json.dumps({"name": "gitnexus-shared"}))
-
-
-def test_runtime_mounts_reuse_primary_checkout_node_modules_from_a_worktree(
-    monkeypatch, tmp_path
-) -> None:
-    primary = tmp_path / "primary"
-    worktree = tmp_path / "worktree"
-    _install_pinned_runtime(primary)
-    (primary / ".git" / "worktrees" / "wt").mkdir(parents=True)
-    _install_pinned_runtime(worktree)
-    shutil.rmtree(worktree / "gitnexus" / "node_modules")
-    (worktree / "gitnexus" / "node_modules").symlink_to(
-        primary / "gitnexus" / "node_modules",
-        target_is_directory=True,
-    )
-    (worktree / ".git").write_text(f"gitdir: {primary / '.git' / 'worktrees' / 'wt'}\n")
-    monkeypatch.setattr(runtime_mounts, "HARNESS_ROOT", worktree)
-
-    mounts = runner.trusted_gitnexus_runtime_mounts()
-    by_target = {mount.target: mount.source for mount in mounts}
-
-    assert by_target[f"{runner.SANDBOX_GITNEXUS}/node_modules"] == (
-        primary / "gitnexus" / "node_modules"
-    )
-    assert by_target[f"{runner.SANDBOX_GITNEXUS_SHARED}/package.json"] == (
-        primary / "gitnexus-shared" / "package.json"
-    )
-    assert by_target[f"{runner.SANDBOX_GITNEXUS}/dist"] == worktree / "gitnexus" / "dist"
-
-
-def test_runtime_mounts_reuse_primary_shared_when_only_the_inner_link_points_there(
-    monkeypatch, tmp_path
-) -> None:
-    primary = tmp_path / "primary"
-    worktree = tmp_path / "worktree"
-    _install_pinned_runtime(primary)
-    (primary / ".git" / "worktrees" / "wt").mkdir(parents=True)
-    _install_pinned_runtime(worktree)
-    linked = worktree / "gitnexus" / "node_modules" / "gitnexus-shared"
-    linked.unlink()
-    linked.symlink_to(primary / "gitnexus-shared", target_is_directory=True)
-    (worktree / ".git").write_text(f"gitdir: {primary / '.git' / 'worktrees' / 'wt'}\n")
-    monkeypatch.setattr(runtime_mounts, "HARNESS_ROOT", worktree)
-
-    mounts = runner.trusted_gitnexus_runtime_mounts()
-    by_target = {mount.target: mount.source for mount in mounts}
-
-    assert by_target[f"{runner.SANDBOX_GITNEXUS}/node_modules"] == (
-        worktree / "gitnexus" / "node_modules"
-    )
-    assert by_target[f"{runner.SANDBOX_GITNEXUS_SHARED}/package.json"] == (
-        primary / "gitnexus-shared" / "package.json"
-    )
-
-
-def test_runtime_mounts_reject_a_node_modules_symlink_outside_the_primary_checkout(
-    monkeypatch, tmp_path
-) -> None:
-    primary = tmp_path / "primary"
-    worktree = tmp_path / "worktree"
-    outsider = tmp_path / "outsider"
-    _install_pinned_runtime(primary)
-    _install_pinned_runtime(outsider)
-    (primary / ".git" / "worktrees" / "wt").mkdir(parents=True)
-    _install_pinned_runtime(worktree)
-    shutil.rmtree(worktree / "gitnexus" / "node_modules")
-    (worktree / "gitnexus" / "node_modules").symlink_to(
-        outsider / "gitnexus" / "node_modules",
-        target_is_directory=True,
-    )
-    (worktree / ".git").write_text(f"gitdir: {primary / '.git' / 'worktrees' / 'wt'}\n")
-    monkeypatch.setattr(runtime_mounts, "HARNESS_ROOT", worktree)
-
-    with pytest.raises(SandboxError, match="primary checkout"):
-        runner.trusted_gitnexus_runtime_mounts()
-
-
-def test_runtime_mounts_reject_a_node_modules_symlink_in_a_regular_checkout(
-    monkeypatch, tmp_path
-) -> None:
-    checkout = tmp_path / "checkout"
-    other = tmp_path / "other"
-    _install_pinned_runtime(checkout)
-    _install_pinned_runtime(other)
-    (checkout / ".git").mkdir()
-    shutil.rmtree(checkout / "gitnexus" / "node_modules")
-    (checkout / "gitnexus" / "node_modules").symlink_to(
-        other / "gitnexus" / "node_modules",
-        target_is_directory=True,
-    )
-    monkeypatch.setattr(runtime_mounts, "HARNESS_ROOT", checkout)
-
-    with pytest.raises(SandboxError, match="must be a real directory"):
-        runner.trusted_gitnexus_runtime_mounts()
-
-
 @pytest.mark.skipif(
     os.environ.get("GITNEXUS_REQUIRE_BWRAP_CANARY") != "1",
     reason="real Bubblewrap canary is mandatory in the named Ubuntu CI job",
@@ -603,11 +458,7 @@ def test_real_bubblewrap_runtime_mount_imports_cli_without_exposing_checkout(tmp
     assert visibility.ok, visibility.stderr_tail
     assert imported.ok, imported.stderr_tail
     assert analyze_imported.ok, analyze_imported.stderr_tail
-    # The runtime the sandbox sees must be the one this checkout built —
+    assert imported.stdout_tail.strip() == runner.PINNED_GITNEXUS_VERSION
-    # compared against the checkout itself rather than a constant, so a release
-    # bump cannot fail a benchmark that is running exactly what it should.
-    built = json.loads((runtime_mounts.HARNESS_ROOT / "gitnexus" / "package.json").read_text())
-    assert imported.stdout_tail.strip() == built["version"]
 
 
 def test_isolated_mcp_registry_contains_only_the_sandbox_clone(tmp_path):
@@ -790,23 +641,6 @@ def test_skill_invocation_parses_supported_exact_identifier_fields(skill_input):
     )
 
 
-def test_skill_invocation_accepts_plugin_qualified_identifier():
-    assert (
-        runner_sessions.skill_was_invoked_events(
-            skill_events({"skill": "compound-engineering:ce-code-review"}),
-            "ce-code-review",
-        )
-        is True
-    )
-    assert (
-        runner_sessions.skill_was_invoked_events(
-            skill_events({"skill": "compound-engineering:ce-plan"}),
-            "ce-code-review",
-        )
-        is False
-    )
-
-
 @pytest.mark.parametrize(
     "skill_input",
     [
@@ -1133,35 +967,6 @@ def test_final_result_event_must_be_last(monkeypatch, tmp_path):
     assert "not the last event" in rec["error_detail"]["event_stream_error"]
 
 
-def test_background_task_teardown_after_the_result_stays_valid_evidence(monkeypatch, tmp_path):
-    # Claude Code drains background-task bookkeeping after the final result
-    # event. Those `system` events carry no tool or usage payload, so they must
-    # not invalidate an otherwise complete session (run 29907431284 lost three
-    # runs this way, and the gate demands zero excluded runs).
-    teardown = [
-        {"type": "system", "subtype": "background_tasks_changed", "tasks": []},
-        {"type": "system", "subtype": "task_updated", "task_id": "bdw43oy7j", "patch": {"status": "killed"}},
-        {"type": "system", "subtype": "task_notification", "task_id": "bdw43oy7j", "status": "stopped"},
-    ]
-    stream = event_stream(*skill_events({"skill": "gitnexus-work"})) + "".join(
-        json.dumps(event) + "\n" for event in teardown
-    )
-    monkeypatch.setattr(runner_sessions, "run_managed", lambda *a, **k: fake_cli_result(stream))
-    rec = runner.run_claude(
-        "task",
-        tmp_path,
-        claude_bin="claude",
-        timeout=5,
-        expected_skill="gitnexus-work",
-    )
-
-    assert rec["ok"] is True
-    assert rec["error_kind"] is None
-    assert rec["skill_invoked"] is True
-    assert rec["transcript_missing"] is False
-    assert "evidence_diagnostics" not in rec
-
-
 def test_snapshot_plan_docs_detects_one_modified_plan_and_rejects_ambiguous_output(tmp_path):
     plans = tmp_path / "docs" / "plans"
     plans.mkdir(parents=True)
@@ -1254,7 +1059,7 @@ def test_planning_cannot_change_source_tests_or_downstream_skill(monkeypatch, tm
 @pytest.mark.parametrize(
     ("attack", "expected_detail"),
     [
-        ("workspace", "changed the read-only workspace"),
+        ("workspace", "unauthorized workspace path"),
         ("skill", "changed the evaluated skill fingerprint"),
     ],
 )
@@ -1270,11 +1075,7 @@ def test_review_phase_rejects_workspace_or_skill_mutation(
     expected_skill_digest = "expected-skill-fingerprint"
 
     def adversarial_review(prompt, *args, **kwargs):
-        # Write where the contract now says: the artifact directory outside the
+        (tmp_path / "review-output.md").write_text("review findings")
-        # workspace, which is the only place the agent can write atomically.
-        artifact = runner.review_output_path(sandbox, runner.REVIEW_OUTPUT)
-        artifact.parent.mkdir(parents=True, exist_ok=True)
-        artifact.write_text('{"schema_version":1,"verdict":"approve","findings":[]}')
         if attack == "workspace":
             source.write_text("review silently changed source")
         return session_record()
@@ -1305,97 +1106,6 @@ def test_review_phase_rejects_workspace_or_skill_mutation(
     assert expected_detail in rec["error_detail"]
 
 
-def test_a_cancelled_clone_copy_does_not_fall_back_to_an_uncancellable_copytree(monkeypatch, tmp_path):
-    """The reflink fallback is for a filesystem, not for a teardown.
-
-    run_managed reports cancellation as a non-OK result rather than raising, so
-    the fallback treated it like an unsupported reflink and started a copytree
-    that cannot be cancelled — waiting out exactly the full copy the outage
-    breaker set the cancellation event to avoid.
-    """
-
-    source = tmp_path / "template"
-    (source / ".git").mkdir(parents=True)
-    parent = tmp_path / "clones"
-    parent.mkdir()
-    copied: list[object] = []
-
-    monkeypatch.setattr(
-        runner_artifacts,
-        "run_managed",
-        lambda *_a, **_k: ManagedProcessResult(
-            state="cancelled",
-            returncode=None,
-            stdout_tail="",
-            stderr_tail="",
-            duration_s=0.1,
-        ),
-    )
-    monkeypatch.setattr(runner_artifacts.shutil, "copytree", lambda *a, **k: copied.append(a))
-
-    with pytest.raises(ManagedProcessError):
-        runner.copy_isolated_tree(source, parent)
-    assert copied == []
-    assert list(parent.iterdir()) == [], "the partial target must be cleaned up"
-
-
-@pytest.mark.parametrize("arm", ["review", "ce_review"])
-def test_run_arm_mounts_the_review_artifact_directory_outside_the_workspace(monkeypatch, tmp_path, arm):
-    """A writable FILE inside a read-only directory is not a writable path.
-
-    The Write tool creates `<target>.tmp.<n>.<hex>` beside the target and
-    renames it, so a read-only parent fails the temp create with EROFS and the
-    artifact stays 0 bytes. The mount target must be the directory, and it must
-    sit outside the read-only workspace.
-
-    Driven through run_arm rather than rebuilt here: an expected tuple assembled
-    in the test passes whatever run_arm actually mounts, which is the one thing
-    this needs to prove.
-    """
-
-    assert not runner.SANDBOX_REVIEW_OUTPUT.startswith(runner.SANDBOX_WORKSPACE + "/")
-    assert runner.SANDBOX_REVIEW_OUTPUT != runner.SANDBOX_WORKSPACE
-
-    verify_calls: list[dict] = []
-    sandbox = fake_sandbox(tmp_path)
-    sandbox.command_prefix_for = lambda **kwargs: verify_calls.append(kwargs) or []
-
-    def review_session(prompt, *args, **kwargs):
-        artifact = runner.review_output_path(sandbox, runner.REVIEW_OUTPUT)
-        artifact.parent.mkdir(parents=True, exist_ok=True)
-        artifact.write_text('{"schema_version":1,"verdict":"approve","findings":[]}')
-        return session_record()
-
-    monkeypatch.setattr(runner, "run_claude", review_session)
-    monkeypatch.setattr(runner, "skill_fingerprint", lambda *_a, **_k: "skill-digest")
-    monkeypatch.setattr(runner, "run_verify", lambda *a, **k: (True, "ok"))
-
-    runner.run_arm(
-        arm,
-        {"prompt": "p", "verify": "true"},
-        tmp_path,
-        bench_args(),
-        sandbox=sandbox,
-        expected_skill_digest="skill-digest",
-    )
-
-    review_output = runner.review_output_path(sandbox, runner.REVIEW_OUTPUT)
-    expected = (runner.ReadOnlyMount(source=review_output.parent, target=runner.SANDBOX_REVIEW_OUTPUT),)
-    # The EROFS bug is about the AGENT's write, so the mount that has to be the
-    # directory is the writable one on the review session — not the read-only
-    # exposure the verify command gets afterwards. Assert both: they are
-    # separate arguments to separate command prefixes.
-    writable = [call["extra_writable_mounts"] for call in verify_calls if "extra_writable_mounts" in call]
-    assert writable, "the review session must be given a writable artifact mount"
-    assert writable[-1] == expected, "mount the directory, not the file"
-    read_only = [call["extra_read_only_mounts"] for call in verify_calls if "extra_read_only_mounts" in call]
-    assert read_only, "the verify invocation must be given the artifact mount"
-    assert read_only[-1] == expected, "mount the directory, not the file"
-    assert not expected[0].target.startswith(f"{runner.SANDBOX_WORKSPACE}/")
-    # The artifact the harness later reads is the one inside that mount.
-    assert review_output.parent in review_output.parents
-
-
 def _git(repo, *args, check=True):
     return subprocess.run(["git", "-C", str(repo), *args], check=check, capture_output=True, text=True)
 
@@ -1446,34 +1156,3 @@ def test_make_worktree_clone_has_no_tags_but_keeps_all_branches(tmp_path):
 
     current = _git(target, "rev-parse", "HEAD").stdout.strip()
     assert current == other_sha
-
-
-def test_copy_isolated_tree_does_not_share_git_objects_or_refs(tmp_path):
-    repo = tmp_path / "repo"
-    repo.mkdir()
-    _git(repo, "init", "--quiet")
-    _git(repo, "checkout", "--quiet", "-b", "main")
-    sha = _git_commit(repo, "base")
-    clones = tmp_path / "clones"
-    clones.mkdir()
-    template = runner.make_worktree(repo, sha, clones)
-    (template / "marker.txt").write_text("template\n")
-
-    copy = runner.copy_isolated_tree(template, clones)
-    assert copy != template
-    assert (copy / "marker.txt").read_text() == "template\n"
-    (copy / "marker.txt").write_text("copy\n")
-    assert (template / "marker.txt").read_text() == "template\n"
-    copy_head = _git(copy, "rev-parse", "HEAD").stdout.strip()
-    template_head = _git(template, "rev-parse", "HEAD").stdout.strip()
-    assert copy_head == template_head == sha
-    # An equal initial HEAD is also what a shared ref namespace looks like, so
-    # write a ref and prove the template cannot see it. A linked worktree would
-    # pass every assertion above, including the alternates check — its `.git` is
-    # a file, so the directory inspected below simply does not exist.
-    _git(copy, "branch", "copy-only")
-    assert _git(copy, "show-ref", "--verify", "refs/heads/copy-only").returncode == 0
-    assert _git(template, "show-ref", "--verify", "refs/heads/copy-only", check=False).returncode != 0
-    assert (copy / ".git").is_dir()
-    alternates = copy / ".git" / "objects" / "info" / "alternates"
-    assert not alternates.exists()

eval/workflow_bench/README.md

@@ -1,4 +1,4 @@
-# Skill benchmark — evolve review quality, measure workflow cost
+# Workflow benchmark — observe the token savings
 
 Measures whether the `gitnexus-plan` → `gitnexus-work` engineering workflow
 actually saves tokens versus a baseline agent on the same tasks, using real
@@ -8,19 +8,18 @@ report.
 
 ## What it compares
 
-| Arm                         | Sessions                                                               | Notes                                                                                               |
+| Arm | Sessions | Notes |
-| --------------------------- | ---------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------- |
+| --- | --- | --- |
-| `workflow`                  | `gitnexus-plan` on the task, then `gitnexus-work` on the produced plan | The skills must be installed (`gitnexus setup`, or repo-local `.claude/skills/`)                    |
+| `workflow` | `gitnexus-plan` on the task, then `gitnexus-work` on the produced plan | The skills must be installed (`gitnexus setup`, or repo-local `.claude/skills/`) |
-| `candidate_workflow`        | same sessions as `workflow`, with a candidate skill overlay            | Paired with `workflow` on the same task/ref/model                                                   |
+| `candidate_workflow` | same sessions as `workflow`, with a candidate skill overlay | Paired with `workflow` on the same task/ref/model |
-| `workflow_direct`           | one `gitnexus-work` direct-mode session                                | The middle option — execution discipline without a planning pass                                    |
+| `workflow_direct` | one `gitnexus-work` direct-mode session | The middle option — execution discipline without a planning pass |
-| `candidate_workflow_direct` | same session as `workflow_direct`, with a candidate skill overlay      | Paired with `workflow_direct` on the same task/ref/model                                            |
+| `candidate_workflow_direct` | same session as `workflow_direct`, with a candidate skill overlay | Paired with `workflow_direct` on the same task/ref/model |
-| `ce_workflow`               | `ce-plan` on the task, then `ce-work` on the produced plan             | External comparator: the explicitly supplied, pinned compound-engineering plugin's plan→work family |
+| `ce_workflow` | `ce-plan` on the task, then `ce-work` on the produced plan | External comparator: the explicitly supplied, pinned compound-engineering plugin's plan→work family |
-| `ce_workflow_direct`        | one `ce-work` direct-mode session                                      | External comparator paired with `workflow_direct`                                                   |
+| `ce_workflow_direct` | one `ce-work` direct-mode session | External comparator paired with `workflow_direct` |
-| `review`                    | one `gitnexus-review` session over an immutable historical PR snapshot | Emits strict `review-output.json`; hidden human labels score quality after the session              |
+| `review` | one `gitnexus-review` session over local uncommitted changes | The task's `setup` applies the diff under review; the review is written to `review-output.md` so `verify` can gate on it |
-| `candidate_review`          | the same review with a `gitnexus-review` candidate overlay             | Paired with `review` on the same case/ref/model/runtime                                             |
+| `ce_review` | one `ce-code-review` session over the same changes | External comparator paired with `review` |
-| `ce_review`                 | one pinned `ce-code-review` session over the same changes              | External comparator paired with both review arms                                                    |
+| `baseline` | one session with the identical task text | `--disallowedTools Skill` so it cannot borrow the workflow; same repo, same MCP tools |
-| `baseline`                  | one session with the identical task text                               | `--disallowedTools Skill` so it cannot borrow the workflow; same repo, same MCP tools               |
+| `baseline_nomcp` | like baseline, graph tools also disallowed | Separates the workflow-discipline question from the GitNexus-tools question (off by default) |
-| `baseline_nomcp`            | like baseline, graph tools also disallowed                             | Separates the workflow-discipline question from the GitNexus-tools question (off by default)        |
 
 Every arm runs in a fresh detached git worktree of the task's `ref`, once per
 `--runs`. The model-visible `verify` command is recorded as
@@ -37,7 +36,7 @@ lfg's gate and work's direct-mode triage should encode.
 
 ```bash
 cd eval
-export GITNEXUS_BENCH_ANTHROPIC_API_KEY="$ANTHROPIC_API_KEY"
+export GITNEXUS_BENCH_AUTH_TOKEN="$ANTHROPIC_API_KEY"
 uv run --locked --extra dev python -m workflow_bench.runner \
   --tasks workflow_bench/tasks.scenarios.yaml --runs 3 \
   --model claude-sonnet-4-20250514
@@ -121,15 +120,10 @@ digest. Files written beneath the agent's `$HOME` are never trusted as
 evidence.
 
 Bare mode is deliberately non-interactive: it does not consult a stored
-Claude login/keychain or `ANTHROPIC_AUTH_TOKEN`. Supply an Anthropic API key
+Claude login/keychain or `ANTHROPIC_AUTH_TOKEN`. Supply one explicit API or
-through `GITNEXUS_BENCH_ANTHROPIC_API_KEY` (preferred) or `--anthropic-api-key`;
+proxy key through `GITNEXUS_BENCH_AUTH_TOKEN` (preferred) or `--auth-token`;
 the harness maps it to `ANTHROPIC_API_KEY` only for the trusted Claude parent
-and scrubs it from agent-launched tools. `GITNEXUS_BENCH_AUTH_TOKEN` and
+and scrubs it from agent-launched tools.
-`--auth-token` remain as aliases. OpenAI keys are not a drop-in
-replacement: pass `--openai-api-key` / `GITNEXUS_BENCH_OPENAI_API_KEY` with
-`gpt-*` / `o*` / `openai/*` model ids and the harness starts a loopback
-LiteLLM proxy. The OpenAI key stays on that host process; Claude still sees
-only a minted `ANTHROPIC_API_KEY` plus `ANTHROPIC_BASE_URL`.
 
 The trusted Claude CLI still needs outbound access to the explicitly supplied
 model endpoint. This is not a network broker, so the CLI itself retains that
@@ -139,30 +133,6 @@ Native benchmark execution is therefore Linux/WSL2-only. Evidence assembly
 and hand-authored overlay preparation can happen elsewhere, but
 `--initial-overlay` does not bypass containment.
 
-For a local diagnostic inside a container that blocks user namespaces, an
-operator may explicitly choose the non-containment host backend:
-
-```bash
-UNSAFE_NO_BWRAP=1 RUNS=1 ./workflow_bench/run-evolution.sh
-```
-
-This mode runs review sessions directly in disposable host worktrees and is
-**not** a security boundary: it does not isolate the network or create a PID
-namespace, and a session that can `chmod` can undo the workspace lock. The
-harness drops write bits on the whole clone, with no carve-out, so accidental
-`npm install` / analyze writes cannot invalidate review evidence. The review
-artifact is not in the clone at all: it lives in a writable directory bound at
-`/review-output`, outside the workspace.
-Sandbox cleanup restores owner write bits before deleting the private TMPDIR,
-because a session that `copytree`s the locked clone would otherwise leave
-non-empty 0555 directories that `rmtree` cannot remove. Historical review
-SHAs that gitignore `.claude/skills/*` are force-added when the harness seeds
-or overlays the evaluated `gitnexus-review` skill.
-Treat model and verifier processes as able to access host files and
-credentials available to the invoking user. It is restricted to the review
-benchmark, forbidden with `--apply` and whenever `CI` is set;
-promotion-capable and CI runs must use Bubblewrap.
-
 ## Prompt and skill evolution loop
 
 Prompts age as models and tool harnesses change. Treat the current skills and
@@ -170,36 +140,6 @@ router thresholds as an incumbent policy, not permanent truth. Candidate
 changes run offline in the same throwaway clones as the incumbent; production
 skills never rewrite themselves from a live task.
 
-On the self-hosted evolution box, `run-evolution.sh` passes
-`--max-runtime-from-instance-window` and the CLI derives its own cap from
-`/proc/uptime` at startup (24h EventBridge window minus a 90-minute upload
-reserve), in the same breath as it starts the clock that cap is measured
-against — a budget computed anywhere earlier is spent by the seconds between. A `workflow_dispatch` that lands on an
-already-running instance therefore exits in-process instead of vanishing when
-the box stops — a cancelled GitHub job skips even `if: always()`, which is
-how run 33962002890 lost 51 finished sessions. Local runs are uncapped.
-
-A review generation is 6 tasks × 3 arms × 3 runs. Serial workers=1 at ~19
-minutes per session is a 16-hour job (run 33962002890). Two harness changes
-cut that without shrinking the gate:
-
-- **Comparator reuse.** `evolve.py` forwards the seed / prior generation as
-  `--reuse-results`. Incumbent `review` and `ce_review` rows are copied into
-  the new `results.jsonl` when model, effort, task SHA, prompt digest, oracle
-  bytes, incumbent skill digest, CE plugin digest, and sandbox backend still
-  match. Candidate arms always run. A weekly generation with an unchanged
-  incumbent therefore pays 18 sessions, not 54. A promotion, model change,
-  task-corpus change, or harness `RUNTIME_DIGEST` change invalidates the
-  lock and re-runs the comparators.
-- **Sanitized clone templates.** Each unique task SHA is cloned and
-  sanitized once. Cells copy that parentless snapshot (reflink when the
-  filesystem allows) instead of `git clone --no-local` plus repack/prune/fsck
-  54 times. Isolation is a private `.git`, not a second copy of full history.
-
-Dispatch defaults to `--workers 3` so those 18 paid cells can overlap. Size
-workers to the host: a cell that loses CPU and hits the session ceiling is
-an excluded run the gate refuses.
-
 Build an overlay that mirrors only the canonical repo-local skill paths:
 
 ```text
@@ -221,7 +161,7 @@ paid work. For a work overlay:
 cd eval
 uv run --locked --extra dev python -m workflow_bench.runner \
   --tasks workflow_bench/tasks.scenarios.yaml \
-  --runs 3 --workers 1 --model claude-sonnet-4-20250514 \
+  --runs 3 --model claude-sonnet-4-20250514 \
   --arms workflow candidate_workflow \
          workflow_direct candidate_workflow_direct \
   --candidate-overlay /tmp/gn-skill-candidate
@@ -237,7 +177,7 @@ artifacts. Those artifacts are the trajectory evidence: cluster failures and
 expensive detours, propose one bounded prompt change, and feed it back as the
 next overlay.
 
-When candidate arms are present the runner also writes schema-6
+When candidate arms are present the runner also writes schema-3
 `promotion.json`. It
 binds the immutable overlay digest, benchmark model, truthful candidate origin
 (a named proposer model or `manual-initial-overlay`), selected
@@ -246,36 +186,9 @@ immutable dependency bytes, committed base digest of every apply
 destination, exact required arms, thresholds, and evidence expiry. Its default
 deterministic gate is deliberately conservative:
 
-Schema 6 binds a separate policy to each required candidate arm and records
-whether the sweep completed. Apply validates the paired metrics and recomputes
-each decision. Historical schema 5 reports remain readable; regenerate their
-benchmark evidence before applying an overlay. Editing a schema number does
-not supply the missing evidence.
-
-Review candidates optimize weighted F1 with a minimum improvement of 0.01,
-complete paired evidence on every selected task, and no per-task quality
-regression. Complete misses score zero. Matching uses maximum cardinality
-throughout the 100-finding limit. Downgraded findings receive at most their
-reported severity's weight; only blocking-severity matches count toward blocker
-recall. Every valid candidate repeat must have the correct verdict, and the
-minimum blocker recall across repeats must not regress. Clean controls retain
-their false-positive and verdict safeguards. Implementation candidates retain
-the efficiency policy below:
-
 - at least 3 paired VALID runs per task, zero excluded runs in either arm
   (session/infra-error rows therefore block promotion), and a named model;
-- a fully measured task that neither arm ever resolves remains reported but is
+- the candidate must pass the hidden oracle on every valid run for every task;
-  ungated from the quality comparison — only if its metric was measured in both
-  arms and no run hit `skill-not-invoked` (a skill that never loaded is prompt
-  evidence, not task health). An ungated task still ranks against a looser 100%
-  failed-task regression cap on the promotion metric;
-- at least half the paired tasks must stay gated, and `promotion.json` discloses
-  the gated/ungated split per decision; a set with no gated task at all is
-  `insufficient_evidence`;
-- the candidate must pass the hidden oracle on every valid run of every gated
-  task the incumbent resolves at least once — on a task the incumbent never
-  resolves, partial candidate progress counts as improvement instead of failing
-  the floor, so making some progress is never scored worse than making none;
 - no per-task resolution-rate regression (quality is lexicographically first);
 - promotion by resolution needs a margin of at least 2 resolved runs —
   a 1-run difference is noise at this run count and falls through to the
@@ -306,85 +219,28 @@ without weakening today's deterministic promotion boundary.
 
 ### Closing the loop automatically (`evolve.py`)
 
-The evolution workflow runs an offline containment preflight with the pinned
-Claude Code 2.1.214 binary before starting a paid proposer or benchmark. The
-review canary seals the workspace read-only and writes nothing into it: the
-artifact directory is bound at `/review-output` outside the workspace, and the
-file itself is deliberately absent until the session creates it, so its absence
-distinguishes "never written" from "written badly". Runtime mount placeholders
-are prepared in the disposable clone before sealing it; existing config bytes
-are preserved.
-Any pre-existing result entry, including a symlink, is rejected. Required
-canaries fail when their runtime or Bubblewrap is unavailable.
-
-The default outage limit is five consecutive unusable results, across task
-boundaries. Invalid review JSON advances this limit even when a skill or session
-error was recorded first. A valid zero-quality review resets it. Concurrent
-waves can exceed the limit by at most `workers - 1` completed cells; no further
-wave starts after a trip. Completed rows and redacted diagnostics remain in the
-partial report, the runner exits nonzero, and the evolution driver stops without
-applying or starting another generation.
-
-SIGINT and SIGTERM propagate one cancellation event through managed commands,
-including clone, setup, Claude, and verification. Executor submissions copy
-the run context so indirect subprocess helpers receive the same event. Active
-process groups or Windows Job Objects are terminated and workers joined before
-shared assets or the gateway are released. Controlled cancellation tests require
-cleanup within 15 seconds. Cancellation remains distinct from timeout and
-quality failure in recorded evidence.
-
-The gateway runs under a private supervisor watching a pipe owned only by the
-harness. Parent exit, including SIGKILL, closes that pipe and stops the proxy
-group; Windows also retains kill-on-close Job Object ownership. Keep completed
-JSONL rows, transcripts, the partial report, and gateway diagnostics when
-investigating an interrupted run. A subsequent paid comparison needs fresh
-evidence from all arms under the same dependency lock. LiteLLM pricing comes
-from that locked release's local cost map; compare no old/new-lock costs as
-quality evidence.
-
 `workflow_bench.evolve` automates the three manual arrows — propose,
 benchmark, apply — without moving the trust boundary:
 
 ```bash
 cd eval
-./workflow_bench/run-evolution.sh                  # local; no working-tree apply
+uv run --locked --extra dev python -m workflow_bench.evolve \
-./workflow_bench/run-evolution.sh --apply          # CI; same argv the workflow uses
+  --tasks workflow_bench/tasks.scenarios.yaml \
-./workflow_bench/run-evolution.sh --dry-run        # print the evolve command
+  --model claude-sonnet-4-20250514 --generations 2 \
+  --seed-results results/wfbench-<prior-run>   # optional gen-0 evidence
 ```
 
-The GitHub skill-evolution job calls this script. Do not invoke
+Each generation: a confined **proposer** session reads the incumbent plan/work
-`python -m workflow_bench.evolve` directly for a full loop. Environment knobs
+skills, the prior generation's `results.jsonl`
-match the workflow: `MODEL`, `PROPOSER_MODEL`, `GENERATIONS`, `RUNS`,
+loser rows, their session transcripts and patches, and the learning queue,
-`WORKERS`, `PROVIDER`, `EFFORT`, `SEED_RESULTS`, `INCLUDE_EXPENSIVE`. The
-checked-in production defaults are `PROVIDER=openai`, `MODEL=gpt-5.6-sol`,
-`PROPOSER_MODEL=gpt-5.6-sol`, and `EFFORT=xhigh`.
-
-The scheduled/default profile is read-only review evolution. Set
-`EVOLUTION_PROFILE=implementation` explicitly to run the legacy plan/work
-benchmark. Review mode requires `CE_PLUGIN_DIR` and `CE_PLUGIN_VERSION`.
-
-Each review generation: a confined **proposer** session reads only the incumbent
-`gitnexus-review` skill, normalized CE/incumbent/candidate result rows, bounded
-review artifacts and session transcripts, and the rejected
-`proposal.md` when available (including a workflow seed from a prior run), and
-the learning queue,
 then writes ONE bounded candidate overlay plus a reviewer-facing
-`proposal.md`. The proposer's clone is sanitized exactly like an arm's before
+`proposal.md`. The overlay is re-validated by `candidate_overlay_files`
-its session starts: it authors the artifact the arms are scored with, so
+(same boundary: Markdown under the plan/work trees, nothing else), frozen,
-letting it read `eval/workflow_bench` would hand it the task prompts and the
-hidden oracles it is about to be graded against, and a proposal could win the
-gate by encoding the expected behavior into a skill rather than by being a
-better skill. The overlay is re-validated by `candidate_overlay_files`
-(same boundary: Markdown under `gitnexus-review`, including exercised
-`ci-personas/`, nothing else), frozen,
 and exercised only by its exact required pairs. Task refs are resolved once
 before generation zero and the immutable task bindings are forwarded to every
 generated runner invocation, so a moving branch cannot change later evidence.
-The deterministic quality-first gate rejects blocker-recall regressions,
+The deterministic gate then decides. Promotion application rejects older
-new false positives on clean controls, and any weighted-score regression.
+pre-oracle evidence schemas. `promote` stops the loop; with `--apply`
-Repeated evidence (`RUNS>=3`) is required for promotion; `RUNS=1` is
-diagnostic-only. CE is the external comparator. Cost and latency are
-tiebreakers and never compensate for quality loss. `promote` stops the loop; with `--apply`
 the authorized frozen bytes
 are transactionally applied to the canonical
 `.claude/skills/` trees and their shipped mirrors as an ordinary
@@ -394,19 +250,17 @@ generation's trajectories to the next proposer. `--initial-overlay` skips
 the generation-0 proposer to benchmark a hand-written candidate;
 `--proposer-model` upgrades only the diagnosis session.
 
-**Learning queue.** Live skill runs never self-edit (see each
+**Learning queue.** Live plan/work skill runs never self-edit (see each
 skill's "Skill feedback" section) — instead they may append one-line JSON notes to
 `workflow_bench/learnings.jsonl` (gitignored, machine-local like the
 transcripts they complement). The proposer reads the queue as hints, not
 ground truth: a learning only reaches a shipped skill by surviving the same
-paired benchmark as any other candidate.
+paired benchmark as any other candidate. Legacy review/LFG rows are ignored;
+those skills do not yet have honest candidate lanes or promotion gates.
 
-For ad-hoc use, run the driver on the existing re-evaluation triggers
+Run the driver on the existing re-evaluation triggers (model/harness change,
-(model/harness change or 90-day staleness). The repository workflow runs a
+90-day staleness), not on a tight schedule — every generation costs ≥3 paired
-deliberate weekly drift check: dispatch defaults to three concurrent cells
+runs per task, and `--generations` is the only loop bound.
-of one task; scheduled concurrency still requires
-`GITNEXUS_EVOLUTION_WORKERS=3` after a clean proof. `--workers` is bounded
-to 1–8 before paid work starts. `--generations` remains the only loop bound.
 
 ## Free-model setup (no paid tokens)
 
@@ -425,30 +279,12 @@ uv run --locked --with 'litellm[proxy]' litellm --config workflow_bench/free-mod
 # 2. Point the benchmark at it
 uv run --locked --extra dev python -m workflow_bench.runner \
   --tasks workflow_bench/tasks.scenarios.yaml --runs 3 \
-  --base-url http://localhost:4000 --anthropic-api-key "$LITELLM_MASTER_KEY" --model free-coder
+  --base-url http://localhost:4000 --auth-token "$LITELLM_MASTER_KEY" --model free-coder
 ```
 
-## OpenAI API keys
-
-Claude Code still speaks Anthropic `/v1/messages`. For a paid OpenAI backend,
-do not point `--anthropic-api-key` at an `sk-...` OpenAI key. Export the OpenAI key
-and use OpenAI model ids; the driver starts the proxy itself:
-
-```bash
-export GITNEXUS_BENCH_OPENAI_API_KEY="$OPENAI_API_KEY"
-PROVIDER=openai ./workflow_bench/run-evolution.sh
-```
-
-The GitHub skill-evolution workflow accepts `GITNEXUS_BENCH_OPENAI_API_KEY` on
-the `gitnexus-evolution` environment. Dispatch with `provider=openai` to force
-that backend even when an Anthropic token is also configured (otherwise `auto`
-keeps using Anthropic whenever that secret exists). Claude default model
-inputs are then rewritten to `gpt-5.6-sol`; every proposer and benchmark
-session receives `--effort xhigh`.
-
 Caveats, honestly:
 
-- Both arms run on the same model, so the _comparison_ stays fair at any
+- Both arms run on the same model, so the *comparison* stays fair at any
   quality level — but small free models follow skills less reliably, so
   expect lower resolve rates and noisier savings than on frontier models.
   Treat free-model runs as directional; confirm headline numbers with a
@@ -475,16 +311,16 @@ Three task classes × three arms, single-repo (GitNexus itself). **Every arm
 resolved every task** — at this difficulty, pass/fail quality is saturated
 and the comparison is pure cost:
 
-| task (class)                  | arm             | resolved | cost $ | wall | turns | vs baseline cost        |
+| task (class) | arm | resolved | cost $ | wall | turns | vs baseline cost |
-| ----------------------------- | --------------- | -------- | ------ | ---- | ----- | ----------------------- |
+| --- | --- | --- | --- | --- | --- | --- |
-| trivial-version-alias         | workflow        | 1/1      | 9.16   | 16m  | 63    | −333%                   |
+| trivial-version-alias | workflow | 1/1 | 9.16 | 16m | 63 | −333% |
-| trivial-version-alias         | baseline        | 1/1      | 2.11   | 2.8m | 16    | —                       |
+| trivial-version-alias | baseline | 1/1 | 2.11 | 2.8m | 16 | — |
-| inv-bug-pdg-note              | workflow        | 1/1      | 14.56  | 21m  | 83    | −331%                   |
+| inv-bug-pdg-note | workflow | 1/1 | 14.56 | 21m | 83 | −331% |
-| inv-bug-pdg-note              | workflow_direct | 1/1      | 5.23   | 7.5m | 32    | −55%                    |
+| inv-bug-pdg-note | workflow_direct | 1/1 | 5.23 | 7.5m | 32 | −55% |
-| inv-bug-pdg-note              | baseline        | 1/1      | 3.38   | 4.7m | 22    | —                       |
+| inv-bug-pdg-note | baseline | 1/1 | 3.38 | 4.7m | 22 | — |
-| inv-feature-list-repos-filter | workflow        | 1/1      | 13.22  | 19m  | 84    | −211%                   |
+| inv-feature-list-repos-filter | workflow | 1/1 | 13.22 | 19m | 84 | −211% |
-| inv-feature-list-repos-filter | workflow_direct | 1/1      | 4.87   | 4.8m | 38    | −15% (wall +14% faster) |
+| inv-feature-list-repos-filter | workflow_direct | 1/1 | 4.87 | 4.8m | 38 | −15% (wall +14% faster) |
-| inv-feature-list-repos-filter | baseline        | 1/1      | 4.25   | 5.5m | 32    | —                       |
+| inv-feature-list-repos-filter | baseline | 1/1 | 4.25 | 5.5m | 32 | — |
 
 What the ground base says, honestly:
 
@@ -499,7 +335,7 @@ What the ground base says, honestly:
   detect_changes-before-commit) is cheap. It produced noticeably more test
   coverage than baseline for near-equal cost on the feature task.
 - **Quality didn't differentiate because nothing failed.** The regime where
-  the workflow should win on _resolve rate_ — cross-module tasks where
+  the workflow should win on *resolve rate* — cross-module tasks where
   baselines flail — is the unmeasured cell (`cross-module-parse-retry`), and
   the next thing to measure, ideally with `--runs 3+` on a free backend.
 - Caveats: n=1 per cell, one repo, one model; churn numbers from this run
@@ -517,11 +353,11 @@ If a future run shows the workflow flattering itself here, distrust the run.
 The hardest class — retry-with-backoff across the worker-pool/pipeline
 seams, transient-vs-deterministic classification:
 
-| arm                 | resolved | cost $   | wall    | turns  | churn       |
+| arm | resolved | cost $ | wall | turns | churn |
-| ------------------- | -------- | -------- | ------- | ------ | ----------- |
+| --- | --- | --- | --- | --- | --- |
-| workflow            | 1/1      | 18.32    | 37m     | 107    | 4/+373/−17  |
+| workflow | 1/1 | 18.32 | 37m | 107 | 4/+373/−17 |
-| **workflow_direct** | 1/1      | **9.53** | **15m** | **52** | 11/+244/−66 |
+| **workflow_direct** | 1/1 | **9.53** | **15m** | **52** | 11/+244/−66 |
-| baseline            | 1/1      | 18.03    | 34m     | 98     | 6/+345/−69  |
+| baseline | 1/1 | 18.03 | 34m | 98 | 6/+345/−69 |
 
 (The workflow_direct row is the clean re-run under clone isolation — the
 original was contaminated, see the integrity note below.)
@@ -555,14 +391,14 @@ category-priced freshness (`accept` for compact classes), per-category turn
 budgets, and the work-phase HEAD==pin fast path, the same
 `inv-bug-pdg-note` workflow cell re-measured (n=1):
 
-|               | ground base | optimized | delta    |
+| | ground base | optimized | delta |
-| ------------- | ----------- | --------- | -------- |
+| --- | --- | --- | --- |
-| resolved      | ✅          | ✅        | —        |
+| resolved | ✅ | ✅ | — |
-| cost $        | 14.56       | 11.70     | **−20%** |
+| cost $ | 14.56 | 11.70 | **−20%** |
-| turns         | 83          | 72        | −13%     |
+| turns | 83 | 72 | −13% |
-| output tokens | 59,789      | 53,345    | −11%     |
+| output tokens | 59,789 | 53,345 | −11% |
-| cache_read    | 6.64M       | 5.07M     | −24%     |
+| cache_read | 6.64M | 5.07M | −24% |
-| wall          | 21m         | 25m       | +15%     |
+| wall | 21m | 25m | +15% |
 
 Verified in-transcript: the compact form fired (115-line plan vs 209 for a
 simpler task pre-optimization), the plan session dropped 72→49 turns, and
@@ -575,8 +411,8 @@ this task class, so the routing rule above stands unchanged.
 ## Writing good tasks
 
 See `tasks.scenarios.yaml`. Small enough to finish headless, real enough to
-require investigation — the workflow's savings come from _not re-reading and
+require investigation — the workflow's savings come from *not re-reading and
-not re-investigating_, which trivial tasks never exercise. Keep `verify` as a
+not re-investigating*, which trivial tasks never exercise. Keep `verify` as a
 model-visible authored-test quality signal, and add an independent `oracle`
 whose source files live under `workflow_bench/oracles/`. Oracle commands must
 run only files staged beneath `$GITNEXUS_BENCH_ORACLE_ROOT`; for Vitest, include
@@ -586,7 +422,7 @@ carry build pre-hooks).
 
 ## Relation to the SWE-bench harness
 
-The rest of `eval/` benchmarks GitNexus _tools_ inside a litellm agent loop
+The rest of `eval/` benchmarks GitNexus *tools* inside a litellm agent loop
-(baseline vs graph-enhanced). This module benchmarks the _skill workflow_
+(baseline vs graph-enhanced). This module benchmarks the *skill workflow*
 inside the real CLI harness those skills ship for. Different question, same
 spirit: measure, don't assume.

eval/workflow_bench/comparator_reuse.py

@@ -1,604 +0,0 @@
-"""Reuse frozen comparator cells when the current sweep is still the same experiment.
-
-Weekly skill evolution re-runs incumbent ``review`` / ``ce_review`` (and the
-implementation incumbents) even when the model, effort, tasks, oracles,
-incumbent skill bytes, and CE plugin have not changed. Those arms are the
-baseline the gate compares a *new* candidate against — they are not the
-thing being evolved. Replaying them burns two-thirds of a generation.
-
-This module selects prior ``results.jsonl`` rows that are safe to carry
-forward. Candidate arms are never reused. A mismatch on any bound field
-falls through to a paid cell. Missing artifacts also fall through: a reused
-row that the proposer cannot read is worse than spending the tokens again.
-"""
-
-from __future__ import annotations
-
-import hashlib
-import json
-import os
-import re
-import stat
-from collections.abc import Iterator, Mapping, Sequence
-from contextlib import contextmanager
-from dataclasses import dataclass
-from datetime import UTC, datetime, timedelta
-from pathlib import Path, PurePosixPath
-from typing import Any
-
-from .evolution import CANDIDATE_ARMS, EVIDENCE_MAX_AGE_DAYS
-from .proposer_sandbox import SandboxError
-from .runner_sessions import MAX_TRANSCRIPT_BYTES, PARENT_EVENT_STREAM_SOURCE
-from .runtime_mounts import CE_ARMS
-from .task_assets import COPY_CHUNK_BYTES, _write_all
-
-REUSABLE_COMPARATOR_ARMS = frozenset(
-    {
-        "review",
-        "ce_review",
-        "workflow",
-        "workflow_direct",
-        "ce_workflow",
-        "ce_workflow_direct",
-        "baseline",
-        "baseline_nomcp",
-    }
-)
-# Must stay aligned with runner.EXCLUDED_ERROR_KINDS plus review-invalid.
-# A reused row becomes promotion evidence; excluded kinds cannot enter that set.
-REUSE_EXCLUDED_ERROR_KINDS = frozenset(
-    {
-        "session-error",
-        "infra-error",
-        "evidence-unverified",
-        "cleanup-failure",
-        "review-evidence-invalid",
-        "cancelled",
-    }
-)
-_TRANSCRIPT_NAME = re.compile(r"[A-Za-z0-9._-]{1,200}")
-CellKey = tuple[str, str, int]
-
-
-@dataclass(frozen=True)
-class TaskReuseBinding:
-    """Per-task identity the prior row must still match."""
-
-    task_base_sha: str
-    task_prompt_digest: str
-    oracle_digest: str
-    oracle_command_digest: str
-    oracle_manifest_digest: str
-    # The cell's environment is part of its identity: a comparator measured
-    # against different task assets or different sandbox dependencies is a
-    # measurement of a different machine, not a baseline for this sweep.
-    task_asset_manifest_digest: str | None = None
-    sandbox_dependency_manifest_digest: str | None = None
-
-
-@dataclass(frozen=True)
-class ComparatorReuseExpectation:
-    """Sweep-wide lock for comparator reuse. Any drift pays for a fresh cell."""
-
-    model: str
-    effort: str
-    sandbox_backend: str
-    runtime_digest: str | None
-    now: datetime
-    max_age: timedelta
-    tasks: Mapping[str, TaskReuseBinding]
-    skill_digests: Mapping[str, str | None]
-    ce_plugin_version: str | None
-    ce_plugin_manifest_digest: str | None
-
-
-def load_result_rows(path: Path) -> list[dict[str, Any]]:
-    """Load ``results.jsonl``; skip malformed lines the same way evolve does."""
-
-    rows: list[dict[str, Any]] = []
-    for line in path.read_text().splitlines():
-        if not line.strip():
-            continue
-        try:
-            row = json.loads(line)
-        except json.JSONDecodeError:
-            continue
-        if isinstance(row, dict):
-            rows.append(row)
-    return rows
-
-
-def current_runtime_digest() -> str | None:
-    """Harness lockfile digest exported by ``run-evolution.sh``, if present."""
-
-    value = os.environ.get("RUNTIME_DIGEST", "").strip()
-    return value or None
-
-
-def row_is_reusable_comparator(row: Mapping[str, Any], expected: ComparatorReuseExpectation) -> bool:
-    """True when ``row`` is a complete, still-valid comparator measurement."""
-
-    arm = row.get("arm")
-    if not isinstance(arm, str) or arm in CANDIDATE_ARMS or arm not in REUSABLE_COMPARATOR_ARMS:
-        return False
-    if row.get("error_kind") in REUSE_EXCLUDED_ERROR_KINDS:
-        return False
-    if row.get("error_kind") not in (None, ""):
-        return False
-    if row.get("ok") is not True:
-        return False
-    if row.get("transcript_missing") is True:
-        return False
-    if row.get("candidate_overlay_digest") not in (None, ""):
-        return False
-    # Age against the ORIGINAL measurement, not the copy time: materialize_reused_row
-    # restamps recorded_at, so a chained row would otherwise refresh its own clock
-    # and never expire. Bound both directions - a future stamp is corrupt, not fresh.
-    recorded = _parse_recorded_at(row.get("reused_from_recorded_at") or row.get("recorded_at"))
-    if recorded is None:
-        return False
-    age = expected.now - recorded
-    if age > expected.max_age or age < timedelta(0):
-        return False
-    if row.get("model") != expected.model and row.get("benchmark_model") != expected.model:
-        return False
-    if row.get("effort") != expected.effort:
-        return False
-    if row.get("sandbox_backend") != expected.sandbox_backend:
-        return False
-    # Fail closed. A row with no runtime_digest was measured by a harness that
-    # did not record one, which is exactly the drift this lock exists to catch;
-    # treating the absence as agreement made every legacy row reusable forever.
-    prior_runtime = row.get("runtime_digest")
-    if not isinstance(prior_runtime, str) or not prior_runtime:
-        return False
-    if not expected.runtime_digest or prior_runtime != expected.runtime_digest:
-        return False
-
-    task_id = row.get("task")
-    binding = expected.tasks.get(task_id) if isinstance(task_id, str) else None
-    if binding is None:
-        return False
-    if row.get("task_base_sha") != binding.task_base_sha:
-        return False
-    if row.get("task_prompt_digest") != binding.task_prompt_digest:
-        return False
-    if row.get("oracle_digest") != binding.oracle_digest:
-        return False
-    if row.get("oracle_command_digest") != binding.oracle_command_digest:
-        return False
-    if row.get("oracle_manifest_digest") != binding.oracle_manifest_digest:
-        return False
-    # Fail closed on both sides, as the runtime digest does: an unbound
-    # expectation means this sweep could not determine its own environment, and
-    # a row without the field was measured before it was recorded.
-    for field, bound in (
-        ("task_asset_manifest_digest", binding.task_asset_manifest_digest),
-        ("sandbox_dependency_manifest_digest", binding.sandbox_dependency_manifest_digest),
-    ):
-        prior = row.get(field)
-        if not isinstance(prior, str) or not prior or not bound or prior != bound:
-            return False
-
-    if arm in CE_ARMS:
-        if row.get("ce_plugin_version") != expected.ce_plugin_version:
-            return False
-        if row.get("ce_plugin_manifest_digest") != expected.ce_plugin_manifest_digest:
-            return False
-    else:
-        expected_skill = expected.skill_digests.get(arm)
-        if not expected_skill or row.get("skill_digest") != expected_skill:
-            return False
-
-    if arm in {"review", "ce_review"}:
-        if row.get("review_evidence_valid") is not True:
-            return False
-        # The artifact, not just the score derived from it. materialize_reused_row
-        # copies it only when the name is present, so without this a row whose
-        # artifact copy never happened could be carried forward as a scored
-        # review that a proposer then cannot read - evidence by assertion.
-        review_artifact = row.get("review_artifact")
-        if not isinstance(review_artifact, str) or not review_artifact:
-            return False
-        if not isinstance(row.get("review_score"), dict):
-            return False
-        if row.get("review_weighted_f1") is None:
-            return False
-
-    artifacts = row.get("transcript_artifacts")
-    if not isinstance(artifacts, list) or not artifacts:
-        return False
-    try:
-        for artifact in artifacts:
-            _transcript_metadata(artifact)
-    except SandboxError:
-        return False
-    return True
-
-
-def select_reusable_comparator_rows(
-    rows: Sequence[Mapping[str, Any]],
-    *,
-    expected: ComparatorReuseExpectation,
-) -> dict[CellKey, dict[str, Any]]:
-    """Index reusable rows by ``(task, arm, run)``. Conflicting duplicates drop the key."""
-
-    chosen: dict[CellKey, dict[str, Any]] = {}
-    blocked: set[CellKey] = set()
-    for row in rows:
-        if not row_is_reusable_comparator(row, expected):
-            continue
-        task_id = row["task"]
-        arm = row["arm"]
-        run = row.get("run")
-        if not isinstance(run, int) or isinstance(run, bool) or run < 0:
-            continue
-        key = (str(task_id), str(arm), run)
-        if key in blocked:
-            continue
-        previous = chosen.get(key)
-        if previous is None:
-            chosen[key] = dict(row)
-            continue
-        if _row_identity(previous) != _row_identity(row):
-            blocked.add(key)
-            chosen.pop(key, None)
-    return chosen
-
-
-def materialize_reused_row(
-    row: Mapping[str, Any],
-    *,
-    source_dir: Path,
-    dest_dir: Path,
-) -> dict[str, Any]:
-    """Copy digest-bound artifacts into this sweep's evidence dir and stamp reuse."""
-
-    source, _ = _resolved_directory(source_dir, label="reuse source")
-    dest, _ = _resolved_directory(dest_dir, label="reuse destination")
-    if source == dest:
-        raise SandboxError("comparator reuse cannot read and write the same results directory")
-
-    materialized = dict(row)
-    materialized["reused"] = True
-    # Keep the FIRST measurement time across a chain. Overwriting it with the
-    # previous copy's stamp let a row refresh its own clock every generation and
-    # outlive the max_age bound entirely.
-    materialized["reused_from_recorded_at"] = row.get("reused_from_recorded_at") or row.get("recorded_at")
-    materialized["recorded_at"] = datetime.now(UTC).isoformat()
-
-    artifacts = row.get("transcript_artifacts")
-    if not isinstance(artifacts, list) or not artifacts:
-        raise SandboxError("reused row is missing transcript_artifacts")
-
-    # Every path below is resolved against a held descriptor, never re-walked
-    # from a name. Both roots are already symlink-free (_resolved_directory
-    # resolved them), and pinning them here means the components under them
-    # cannot be swapped out from under a check that already passed.
-    with (
-        _open_pinned_root(source_dir, label="reuse source") as source_fd,
-        _open_pinned_root(dest_dir, label="reuse destination") as dest_fd,
-    ):
-        copied_artifacts: list[dict[str, Any]] = []
-        for artifact in artifacts:
-            copied_artifacts.append(_copy_transcript_artifact(source_fd, dest_fd, artifact))
-        materialized["transcript_artifacts"] = copied_artifacts
-
-        review_name = row.get("review_artifact")
-        if isinstance(review_name, str) and review_name:
-            _copy_named_artifact(source_fd, dest_fd, review_name, label="review artifact")
-
-        task = row.get("task")
-        arm = row.get("arm")
-        run = row.get("run")
-        if isinstance(task, str) and isinstance(arm, str) and isinstance(run, int) and not isinstance(run, bool):
-            patch_name = f"{task}-{arm}-run{run}.patch"
-            if _is_regular_at(patch_name, dir_fd=source_fd):
-                _copy_named_artifact(source_fd, dest_fd, patch_name, label="patch artifact")
-    return materialized
-
-
-def default_reuse_max_age() -> timedelta:
-    return timedelta(days=EVIDENCE_MAX_AGE_DAYS)
-
-
-def _row_identity(row: Mapping[str, Any]) -> tuple[Any, ...]:
-    return (
-        row.get("skill_digest"),
-        row.get("oracle_digest"),
-        row.get("review_weighted_f1"),
-        row.get("ce_plugin_manifest_digest"),
-        row.get("recorded_at"),
-    )
-
-
-def _parse_recorded_at(value: Any) -> datetime | None:
-    if not isinstance(value, str) or not value:
-        return None
-    try:
-        parsed = datetime.fromisoformat(value.replace("Z", "+00:00"))
-    except ValueError:
-        return None
-    if parsed.tzinfo is None:
-        parsed = parsed.replace(tzinfo=UTC)
-    return parsed.astimezone(UTC)
-
-
-def _transcript_metadata(metadata: Any) -> tuple[str, str, int]:
-    if not isinstance(metadata, dict) or set(metadata) != {"path", "sha256", "bytes", "source"}:
-        raise SandboxError("transcript artifact metadata must contain only path, sha256, bytes, and source")
-    relative = metadata["path"]
-    digest = metadata["sha256"]
-    size = metadata["bytes"]
-    if metadata["source"] != PARENT_EVENT_STREAM_SOURCE:
-        raise SandboxError("transcript artifact source is not the parent event stream")
-    if not isinstance(relative, str) or not isinstance(digest, str) or not re.fullmatch(r"[0-9a-f]{64}", digest):
-        raise SandboxError("transcript artifact metadata is malformed")
-    if not isinstance(size, int) or isinstance(size, bool) or size < 0 or size > MAX_TRANSCRIPT_BYTES:
-        raise SandboxError("transcript artifact byte count is out of range")
-    relative_path = PurePosixPath(relative)
-    if (
-        relative_path.is_absolute()
-        or len(relative_path.parts) != 2
-        or relative_path.parts[0] != "transcripts"
-        or any(part in {"", ".", ".."} for part in relative_path.parts)
-        or _TRANSCRIPT_NAME.fullmatch(relative_path.parts[1]) is None
-    ):
-        raise SandboxError(f"unsafe transcript artifact path: {relative!r}")
-    return relative, digest, size
-
-
-def _resolved_directory(path: Path, *, label: str) -> tuple[Path, tuple[int, int]]:
-    """An existing, non-symlink directory, resolved through its parents.
-
-    Deliberately weaker than proposer_sandbox's same-shaped helper, which
-    refuses every symlink hop in the path. That one guards a MOUNT ROOT, where
-    a hop changes what an untrusted session is handed. This one guards a DATA
-    directory whose contents are validated individually anyway - every file
-    read goes through ``_regular_file`` (lstat, symlinks rejected) and every
-    write through ``O_NOFOLLOW`` - so a symlinked parent grants nothing those
-    guards do not already cover, while refusing one would reject ordinary
-    setups such as a symlinked artifacts directory or macOS's /var.
-
-    Separately named because they make different promises. Do not merge them
-    without first deciding which promise the reuse path should make.
-    """
-
-    resolved = path.expanduser()
-    try:
-        metadata = resolved.lstat()
-    except OSError as exc:
-        raise SandboxError(f"{label} is unavailable: {resolved}: {exc}") from exc
-    if stat.S_ISLNK(metadata.st_mode) or not stat.S_ISDIR(metadata.st_mode):
-        raise SandboxError(f"{label} must be a real directory: {resolved}")
-    return resolved.resolve(), (metadata.st_dev, metadata.st_ino)
-
-
-@contextmanager
-def _open_pinned_root(path: Path, *, label: str) -> Iterator[int]:
-    """Open a checked root and prove it is still the directory that was checked.
-
-    The symlink POLICY above is deliberate and unchanged: parent hops stay
-    allowed, so a symlinked artifacts directory or macOS's /var still works.
-    What is closed here is separate from that policy - the gap between checking
-    a name and using it. lstat names one directory and resolve() re-walks the
-    same name afterwards, so a prior sweep that renames its results root and
-    drops a symlink in its place is resolved to somewhere else entirely, and
-    O_NOFOLLOW on the open cannot see a link that resolve() already followed.
-
-    Comparing the opened descriptor's identity to the checked one costs an
-    fstat and rejects nothing that holds still: a stable directory always
-    matches itself. It matters for reuse specifically because the failure is
-    silent - rows would be copied out of the wrong directory and folded into a
-    comparator baseline as though they were this sweep's own evidence.
-    """
-
-    resolved, expected = _resolved_directory(path, label=label)
-    with _open_real_directory(resolved, label=label) as fd:
-        opened = os.fstat(fd)
-        if (opened.st_dev, opened.st_ino) != expected:
-            raise SandboxError(f"{label} was replaced between the check and the open: {resolved}")
-        yield fd
-
-
-def _copy_transcript_artifact(source_fd: int, dest_fd: int, metadata: Mapping[str, Any]) -> dict[str, Any]:
-    relative, expected_digest, expected_size = _transcript_metadata(metadata)
-    name = PurePosixPath(relative).name
-    # Both `transcripts` components are opened as descriptors, not checked as
-    # names. An lstat that passes and a pathname that is used afterwards are two
-    # different directories whenever a concurrent writer renames the first one
-    # away — which the reuse directory, written by a prior sweep, invites.
-    with (
-        _open_real_directory("transcripts", dir_fd=dest_fd, label="transcript destination", create=True) as dest_dir_fd,
-        _open_real_directory("transcripts", dir_fd=source_fd, label="transcript source") as source_dir_fd,
-    ):
-        os.fchmod(dest_dir_fd, 0o700)
-        # One descriptor for the whole transfer, and ONE read of it. Hashing the
-        # source and then reading it again to copy leaves the recorded digest
-        # describing bytes that are not the bytes written: the descriptor stops
-        # the pathname being substituted, not the inode being rewritten, and
-        # this directory belongs to a sweep that may still be writing. Digest
-        # what is copied, then judge it.
-        with _open_regular(name, dir_fd=source_dir_fd, label="transcript") as artifact_fd:
-            digest, copied_bytes = _copy_owner_only(
-                artifact_fd, name, dir_fd=dest_dir_fd, max_bytes=expected_size
-            )
-            if copied_bytes != expected_size or digest != expected_digest:
-                # The destination now holds bytes no expectation vouches for.
-                os.unlink(name, dir_fd=dest_dir_fd)
-                drift = "size" if copied_bytes != expected_size else "digest"
-                raise SandboxError(f"reused transcript {drift} drifted: {relative}")
-    return {"path": relative, "sha256": digest, "bytes": expected_size, "source": PARENT_EVENT_STREAM_SOURCE}
-
-
-def _copy_named_artifact(source_fd: int, dest_fd: int, name: str, *, label: str) -> None:
-    relative = PurePosixPath(name)
-    if relative.is_absolute() or len(relative.parts) != 1 or relative.parts[0] in {"", ".", ".."}:
-        raise SandboxError(f"unsafe {label} path: {name!r}")
-    with _open_regular(name, dir_fd=source_fd, label=label) as artifact_fd:
-        # No expectation is recorded for these, so the digest is discarded - but
-        # "no recorded size" is not "no limit". The source is a prior sweep
-        # directory that can change between sweeps, so a replaced artifact could
-        # be arbitrarily large; MAX_TRANSCRIPT_BYTES is the ceiling the capture
-        # path already enforces on evidence of this kind.
-        _, copied = _copy_owner_only(artifact_fd, name, dir_fd=dest_fd, max_bytes=MAX_TRANSCRIPT_BYTES)
-        if copied > MAX_TRANSCRIPT_BYTES:
-            os.unlink(name, dir_fd=dest_fd)
-            raise SandboxError(f"reused {label} exceeds {MAX_TRANSCRIPT_BYTES} bytes: {name}")
-
-
-def _require_openat() -> None:
-    """openat is what makes a checked directory and a used directory the same one.
-
-    Without it the only alternative is to re-walk the name after the check,
-    which is exactly the race this module is guarding. Refusing is safe: the
-    caller in runner treats a SandboxError from reuse as "run a paid cell", so
-    a platform without openat pays for the cells rather than copying through a
-    directory nobody verified. The sweep itself is Linux-only anyway (bwrap,
-    /proc/uptime); this is about the unit tests and about failing loudly.
-    """
-
-    if os.open not in os.supports_dir_fd or os.lstat not in os.supports_dir_fd:
-        raise SandboxError("comparator reuse requires POSIX openat support (os.supports_dir_fd)")
-
-
-def _is_regular_at(name: str, *, dir_fd: int) -> bool:
-    """True when `name` under the pinned directory is a regular non-symlink file."""
-
-    try:
-        metadata = os.lstat(name, dir_fd=dir_fd)
-    except OSError:
-        return False
-    return stat.S_ISREG(metadata.st_mode)
-
-
-@contextmanager
-def _open_real_directory(
-    path: Path | str,
-    *,
-    dir_fd: int | None = None,
-    label: str,
-    create: bool = False,
-) -> Iterator[int]:
-    """Open one directory that is not a symlink, and hold it for every use below.
-
-    ``O_DIRECTORY | O_NOFOLLOW`` makes the check and the open a single syscall,
-    so unlike an ``lstat`` followed by a path, there is no window in which the
-    directory can be replaced. ``_resolved_directory`` still tolerates a
-    symlinked reuse ROOT — it hands this function the already-resolved path —
-    but every component below it is pinned.
-    """
-
-    _require_openat()
-    if create:
-        try:
-            os.mkdir(path, 0o700, dir_fd=dir_fd)
-        except FileExistsError:
-            # Already there is the ordinary case — a second artifact from the
-            # same row. What it already IS still has to be proven, and the
-            # O_DIRECTORY|O_NOFOLLOW open below is what proves it, so there is
-            # nothing to do here.
-            pass
-        except OSError as exc:
-            raise SandboxError(f"{label} cannot be created: {path}: {exc}") from exc
-    try:
-        descriptor = os.open(
-            path,
-            os.O_RDONLY | getattr(os, "O_DIRECTORY", 0) | getattr(os, "O_NOFOLLOW", 0),
-            dir_fd=dir_fd,
-        )
-    except FileNotFoundError as exc:
-        # Absent is a different fact from present-but-not-a-real-directory, and
-        # the caller falls through to a paid cell on either.
-        raise SandboxError(f"{label} is missing: {path}") from exc
-    except OSError as exc:
-        raise SandboxError(f"{label} must be a real directory: {path}: {exc}") from exc
-    try:
-        # O_DIRECTORY is the check on Linux; the fstat covers a platform whose
-        # os module does not define it, where the flag degrades to 0.
-        if not stat.S_ISDIR(os.fstat(descriptor).st_mode):
-            raise SandboxError(f"{label} must be a real directory: {path}")
-        yield descriptor
-    finally:
-        os.close(descriptor)
-
-
-@contextmanager
-def _open_regular(name: str, *, dir_fd: int, label: str) -> Iterator[int]:
-    """Open a regular non-symlink file under a pinned directory, and hold it.
-
-    Checking a name and then re-opening it is a race the reuse directory is
-    exposed to: it is written by a previous sweep and read by this one, so a
-    concurrent writer can replace a validated file with a symlink in between.
-    Resolving against ``dir_fd`` removes the directory half, ``O_NOFOLLOW``
-    refuses the leaf link, and the fstat comparison proves the open descriptor
-    is the inode that was checked — the same guarantee
-    evolution._bounded_regular_bytes makes for evidence files.
-    """
-
-    _require_openat()
-    try:
-        before = os.lstat(name, dir_fd=dir_fd)
-    except OSError as exc:
-        raise SandboxError(f"{label} is missing: {name}: {exc}") from exc
-    if stat.S_ISLNK(before.st_mode) or not stat.S_ISREG(before.st_mode):
-        raise SandboxError(f"{label} must be a regular non-symlink file: {name}")
-    try:
-        descriptor = os.open(name, os.O_RDONLY | getattr(os, "O_NOFOLLOW", 0), dir_fd=dir_fd)
-    except OSError as exc:
-        raise SandboxError(f"{label} is unreadable: {name}: {exc}") from exc
-    try:
-        opened = os.fstat(descriptor)
-        if not stat.S_ISREG(opened.st_mode) or (opened.st_dev, opened.st_ino) != (before.st_dev, before.st_ino):
-            raise SandboxError(f"{label} changed while opening: {name}")
-        yield descriptor
-    finally:
-        os.close(descriptor)
-
-
-def _copy_owner_only(source: int, name: str, *, dir_fd: int, max_bytes: int | None = None) -> tuple[str, int]:
-    """Copy one open file into the pinned directory; return what was written.
-
-    The digest is taken from the same buffers that are written, so it describes
-    the copy rather than a state the source was in at some earlier read.
-
-    ``max_bytes`` bounds the copy itself. The source is a prior sweep directory
-    this module already treats as concurrently writable, so a transcript
-    appended to after its metadata was recorded would otherwise be streamed to
-    EOF and only then compared against its declared size - filling the
-    destination, or never reaching EOF at all, long before the drift check could
-    reject it. Stopping one byte past the ceiling keeps that comparison
-    meaningful while bounding the work.
-    """
-
-    # O_CREAT|O_EXCL is the existence check, and unlike a stat beforehand it is
-    # atomic: a file appearing between check and open cannot slip through.
-    try:
-        descriptor = os.open(
-            name,
-            os.O_WRONLY | os.O_CREAT | os.O_EXCL | getattr(os, "O_NOFOLLOW", 0),
-            0o600,
-            dir_fd=dir_fd,
-        )
-    except FileExistsError as exc:
-        raise SandboxError(f"reuse destination already exists: {name}") from exc
-    try:
-        os.fchmod(descriptor, 0o600)
-        os.lseek(source, 0, os.SEEK_SET)
-        digest = hashlib.sha256()
-        written = 0
-        limit = None if max_bytes is None else max_bytes + 1
-        while True:
-            want = COPY_CHUNK_BYTES if limit is None else min(COPY_CHUNK_BYTES, limit - written)
-            if want <= 0:
-                break
-            chunk = os.read(source, want)
-            if not chunk:
-                break
-            digest.update(chunk)
-            written += len(chunk)
-            _write_all(descriptor, chunk)
-        os.fsync(descriptor)
-        return digest.hexdigest(), written
-    finally:
-        os.close(descriptor)

eval/workflow_bench/evolution.py

@@ -7,7 +7,6 @@ import os
 import secrets
 import stat
 import statistics
-from collections.abc import Sequence
 from pathlib import Path, PurePosixPath
 from typing import Any
 
@@ -22,94 +21,9 @@ from .proposer_sandbox import (
 CANDIDATE_ARMS = {
     "candidate_workflow": "workflow",
     "candidate_workflow_direct": "workflow_direct",
-    "candidate_review": "review",
 }
-PROMOTION_SCHEMA_VERSION = 6
-
-
-def promotion_policy(
-    candidate_arms: Sequence[str],
-    *,
-    metric: str = "cost_usd",
-    min_runs: int = 3,
-    min_improvement_pct: float = 5.0,
-    max_task_regression_pct: float = 20.0,
-) -> dict[str, dict[str, Any]]:
-    """The exact per-arm policy shared by evidence production and application."""
-    if not candidate_arms or len(set(candidate_arms)) != len(candidate_arms):
-        raise ValueError("promotion policy requires unique candidate arms")
-    policies = {}
-    for arm in candidate_arms:
-        if arm not in CANDIDATE_ARMS:
-            raise ValueError(f"unsupported candidate arm: {arm}")
-        policies[arm] = (
-            {
-                "metric": "review_weighted_f1",
-                "min_runs": min_runs,
-                "min_improvement": 0.01,
-                "quality_rule": "correct verdict on every repeat; minimum blocker recall; no clean-control regression",
-            }
-            if arm == "candidate_review"
-            else {
-                "metric": metric,
-                "min_runs": min_runs,
-                "min_improvement_pct": min_improvement_pct,
-                "max_task_regression_pct": max_task_regression_pct,
-                "max_failed_task_regression_pct": MAX_FAILED_TASK_REGRESSION_PCT,
-                "min_gated_task_ratio": MIN_GATED_TASK_RATIO,
-                "quality_rule": "no per-task resolution-rate regression",
-            }
-        )
-    return policies
-
-
-def promotion_evidence(
-    results: dict[str, dict[str, dict[str, Any]]],
-    *,
-    policy: dict[str, dict[str, Any]],
-    model: str | None,
-    complete: bool,
-) -> dict[str, Any]:
-    """Produce discriminated, recomputable decisions, including partial reports."""
-    decisions = []
-    for candidate, rules in policy.items():
-        common = {
-            "incumbent_arm": CANDIDATE_ARMS[candidate],
-            "candidate_arm": candidate,
-            "model": model,
-            "min_runs": rules["min_runs"],
-        }
-        if candidate == "candidate_review":
-            decision = evaluate_review_candidate(results, **common, min_improvement=rules["min_improvement"])
-        else:
-            decision = evaluate_candidate(
-                results,
-                **common,
-                **{
-                    key: rules[key]
-                    for key in (
-                        "metric",
-                        "min_improvement_pct",
-                        "max_task_regression_pct",
-                        "max_failed_task_regression_pct",
-                    )
-                },
-            )
-        if not complete:
-            decision["decision"] = "insufficient_evidence"
-            decision["reasons"].append("sweep aborted; partial evidence cannot promote")
-        decisions.append(decision)
-    return {
-        "schema_version": PROMOTION_SCHEMA_VERSION,
-        "run_status": "complete" if complete else "aborted",
-        "policy": policy,
-        "decisions": decisions,
-    }
-
-
 CANDIDATE_SKILLS = {
     "gitnexus-plan",
-    "gitnexus-review",
     "gitnexus-work",
 }
 # Skills each incumbent arm actually loads in its sessions. An overlay that
@@ -118,7 +32,6 @@ CANDIDATE_SKILLS = {
 ARM_SKILLS = {
     "workflow": ("gitnexus-plan", "gitnexus-work"),
     "workflow_direct": ("gitnexus-work",),
-    "review": ("gitnexus-review",),
 }
 # Repo-local prompts whose bytes are evidence for each executed arm. Keep this
 # distinct from ``ARM_SKILLS``: that mapping defines which skills a promotable
@@ -141,19 +54,6 @@ MAIN_LOOP_ONLY_WARNING = (
     "each run output, deduplicating events "
     "that share one message.id."
 )
-# Failure kinds the prompts under test cause, not the task: the skill never
-# ran at all. Both arms failing a task this way is evidence about the skills,
-# so such a task stays inside the gate however unresolvable it looks.
-SKILL_ATTRIBUTABLE_ERROR_KINDS = frozenset({"skill-not-invoked"})
-# Leaving the quality gate is not leaving the spend gate. A candidate may fail
-# the same oracle the incumbent fails, but not at a multiple of its cost — an
-# ungated task is still real money and still ranks on the metric.
-MAX_FAILED_TASK_REGRESSION_PCT = 100.0
-# Promotion must rest on a real evidence base. Half the paired tasks is the
-# loosest rule the three-task production set can carry: it tolerates the one
-# scenario neither arm resolves and refuses a generation that has quietly
-# decayed to a single gated task deciding everything.
-MIN_GATED_TASK_RATIO = 0.5
 EVIDENCE_MAX_AGE_DAYS = 90
 MAX_CANDIDATE_OVERLAY_BYTES = 4 * 1024 * 1024
 MAX_SKILL_FINGERPRINT_BYTES = 4 * 1024 * 1024
@@ -425,7 +325,7 @@ def candidate_overlay_files(overlay: Path) -> list[Path]:
         ):
             raise ValueError(
                 "candidate overlays may only contain Markdown files under "
-                ".claude/skills/gitnexus-{plan,review,work}: "
+                ".claude/skills/gitnexus-{plan,work}: "
                 f"{relative}"
             )
     return entries
@@ -445,8 +345,6 @@ def required_candidate_arms(overlay: Path) -> list[str]:
         required.append("candidate_workflow")
     if "gitnexus-work" in touched:
         required.append("candidate_workflow_direct")
-    if "gitnexus-review" in touched:
-        required.append("candidate_review")
     return required
 
 
@@ -467,69 +365,6 @@ def candidate_overlay_digest(overlay: Path) -> str:
     return digest
 
 
-def _commit_sandbox_paths(
-    sandbox: SandboxSession,
-    relative_paths: Sequence[str],
-    *,
-    message: str,
-    require_change: bool,
-) -> bool:
-    """Stage and commit paths inside the outer sandbox.
-
-    Returns True when a commit was created. ``require_change`` keeps the
-    candidate-overlay contract: a no-op overlay is an error, while an
-    incumbent skill seed may already match the historical tree.
-    """
-
-    mkdir_command = ["/bin/mkdir", "-p", f"{SANDBOX_TMP}/wfbench-empty-hooks"]
-    mkdir_result = sandbox.run(
-        mkdir_command,
-        timeout=60,
-        env=build_sandbox_environment(),
-    )
-    if not mkdir_result.ok:
-        raise ManagedProcessError(mkdir_command, mkdir_result)
-    if not relative_paths:
-        if require_change:
-            raise ValueError("candidate overlay is byte-identical to the incumbent skills")
-        return False
-
-    # Historical review SHAs gitignore `.claude/skills/*` and lack the current
-    # per-skill allowlist. Force-add so a seed or overlay of harness-owned
-    # skill bytes is not rejected as an ignored path.
-    command, added = _sandbox_overlay_git(sandbox, ["add", "-f", "--", *relative_paths])
-    if not added.ok:
-        raise ManagedProcessError(command, added)
-    command, changed = _sandbox_overlay_git(
-        sandbox,
-        ["diff", "--cached", "--quiet", "--no-ext-diff", "--no-textconv", "--"],
-    )
-    if changed.returncode == 0:
-        if require_change:
-            raise ValueError("candidate overlay is byte-identical to the incumbent skills")
-        return False
-    if changed.returncode != 1:
-        raise ManagedProcessError(command, changed)
-
-    command, committed = _sandbox_overlay_git(
-        sandbox,
-        [
-            "commit",
-            "--quiet",
-            "--no-verify",
-            "-m",
-            message,
-        ],
-        extra_config=(
-            "user.name=workflow-bench",
-            "user.email=workflow-bench@invalid",
-        ),
-    )
-    if not committed.ok:
-        raise ManagedProcessError(command, committed)
-    return True
-
-
 def apply_candidate_overlay(
     overlay: Path,
     worktree: Path,
@@ -548,96 +383,47 @@ def apply_candidate_overlay(
     for relative, content in payload:
         _replace_regular_file(worktree, relative, content)
         relative_paths.append(relative.as_posix())
-    _commit_sandbox_paths(
+
-        sandbox,
+    mkdir_command = ["/bin/mkdir", "-p", f"{SANDBOX_TMP}/wfbench-empty-hooks"]
-        relative_paths,
+    mkdir_result = sandbox.run(
-        message="benchmark candidate skill overlay",
+        mkdir_command,
-        require_change=True,
+        timeout=60,
+        env=build_sandbox_environment(),
     )
+    if not mkdir_result.ok:
+        raise ManagedProcessError(mkdir_command, mkdir_result)
+
+    command, added = _sandbox_overlay_git(sandbox, ["add", "--", *relative_paths])
+    if not added.ok:
+        raise ManagedProcessError(command, added)
+    command, changed = _sandbox_overlay_git(
+        sandbox,
+        ["diff", "--cached", "--quiet", "--no-ext-diff", "--no-textconv", "--"],
+    )
+    if changed.returncode == 0:
+        raise ValueError("candidate overlay is byte-identical to the incumbent skills")
+    if changed.returncode != 1:
+        raise ManagedProcessError(command, changed)
+
+    command, committed = _sandbox_overlay_git(
+        sandbox,
+        [
+            "commit",
+            "--quiet",
+            "--no-verify",
+            "-m",
+            "benchmark candidate skill overlay",
+        ],
+        extra_config=(
+            "user.name=workflow-bench",
+            "user.email=workflow-bench@invalid",
+        ),
+    )
+    if not committed.ok:
+        raise ManagedProcessError(command, committed)
     return digest
 
 
-def seed_evaluated_skills(
-    source_repo: Path,
-    worktree: Path,
-    *,
-    sandbox: SandboxSession,
-    arm: str,
-) -> None:
-    """Install the current evaluated skill tree into a historical clone.
-
-    Review evolution scores the current (or overlay) ``gitnexus-review`` skill
-    against a historical PR checkout. Older SHAs predate that skill, and
-    using whatever prose happened to exist at the reviewed commit would make
-    the incumbent arm a moving target. Copy the harness checkout's skill
-    bytes and commit them before setup so ``git status`` still shows only
-    the task patch.
-    """
-
-    skill_names = EVALUATED_ARM_SKILLS.get(arm)
-    if not skill_names:
-        return
-
-    source_repo = source_repo.expanduser().absolute()
-    expected_clone = Path(os.path.abspath(worktree.expanduser()))
-    sandbox_clone = Path(os.path.abspath(sandbox.clone.expanduser()))
-    if sandbox_clone != expected_clone:
-        raise ValueError("skill seed sandbox does not bind the requested clone")
-    _require_real_directory(source_repo, label="incumbent skill repository")
-    if source_repo.resolve(strict=True) != source_repo:
-        raise ValueError(f"incumbent skill repository cannot traverse symlinks: {source_repo}")
-
-    relative_paths: list[str] = []
-    total = 0
-    for skill_name in skill_names:
-        _require_directory_chain(
-            source_repo,
-            Path(".claude") / "skills" / skill_name,
-            label="incumbent skill root",
-        )
-        skill_root = source_repo / ".claude" / "skills" / skill_name
-        pending = [skill_root]
-        while pending:
-            directory = pending.pop()
-            try:
-                children = list(os.scandir(directory))
-            except OSError as exc:
-                raise ValueError(f"incumbent skill directory is unreadable: {directory}: {exc}") from exc
-            for item in children:
-                path = Path(item.path)
-                if item.is_symlink():
-                    raise ValueError(
-                        "incumbent skill seed cannot contain symlinks: "
-                        f"{path.relative_to(source_repo)}"
-                    )
-                if item.is_dir(follow_symlinks=False):
-                    pending.append(path)
-                    continue
-                if not item.is_file(follow_symlinks=False):
-                    raise ValueError(
-                        "incumbent skill seed entries must be regular files: "
-                        f"{path.relative_to(source_repo)}"
-                    )
-                total += item.stat(follow_symlinks=False).st_size
-                if total > MAX_SKILL_FINGERPRINT_BYTES:
-                    raise ValueError("incumbent skill seed exceeds the bounded evidence limit")
-                relative = Path(".claude") / "skills" / skill_name / path.relative_to(skill_root)
-                content = _bounded_regular_bytes(
-                    path,
-                    limit=MAX_SKILL_FINGERPRINT_BYTES,
-                    label="incumbent skill file",
-                )
-                _replace_regular_file(worktree, relative, content)
-                relative_paths.append(PurePosixPath(relative.as_posix()).as_posix())
-
-    _commit_sandbox_paths(
-        sandbox,
-        relative_paths,
-        message="benchmark incumbent review skill",
-        require_change=False,
-    )
-
-
 def unexercised_overlay_skills(overlay: Path, candidate_arms: list[str]) -> list[str]:
     """Overlay skills that no selected candidate arm would ever load.
 
@@ -689,133 +475,6 @@ def skill_fingerprint(worktree: Path, arm: str) -> str | None:
     return fingerprint_files(worktree, files)
 
 
-def evaluate_review_candidate(
-    results: dict[str, dict[str, dict[str, Any]]],
-    *,
-    incumbent_arm: str,
-    candidate_arm: str,
-    model: str | None,
-    min_runs: int = 3,
-    min_improvement: float = 0.01,
-) -> dict[str, Any]:
-    """Quality-first promotion gate for paired read-only review arms."""
-
-    reasons: list[str] = []
-    task_rows: list[dict[str, Any]] = []
-    insufficient = not model
-    regression = False
-    improvement = False
-    if not model:
-        reasons.append("a named --model is required so review evidence cannot drift")
-
-    for task_id, arms in sorted(results.items()):
-        if incumbent_arm not in arms or candidate_arm not in arms:
-            insufficient = True
-            reasons.append(f"{task_id}: both {incumbent_arm} and {candidate_arm} are required")
-            continue
-        incumbent = arms[incumbent_arm]
-        candidate = arms[candidate_arm]
-        incumbent_runs = int(incumbent.get("valid_runs", 0))
-        candidate_runs = int(candidate.get("valid_runs", 0))
-        incumbent_score = incumbent.get("review_weighted_f1")
-        candidate_score = candidate.get("review_weighted_f1")
-        incumbent_blockers = incumbent.get("review_blocker_recall")
-        candidate_blockers = candidate.get("review_blocker_recall")
-        incumbent_fp = incumbent.get("review_false_positives")
-        candidate_fp = candidate.get("review_false_positives")
-        clean = bool(incumbent.get("review_clean_control", candidate.get("review_clean_control", False)))
-        incumbent_clean_pass = incumbent.get("review_clean_pass")
-        candidate_clean_pass = candidate.get("review_clean_pass")
-        task_rows.append(
-            {
-                "task": task_id,
-                "class": incumbent.get("class", ""),
-                "incumbent_weighted_f1": incumbent_score,
-                "incumbent": dict(incumbent),
-                "candidate": dict(candidate),
-                "gated": True,
-                "candidate_weighted_f1": candidate_score,
-                "incumbent_blocker_recall": incumbent_blockers,
-                "candidate_blocker_recall": candidate_blockers,
-                "incumbent_false_positives": incumbent_fp,
-                "candidate_false_positives": candidate_fp,
-                "clean_control": clean,
-                "incumbent_clean_pass": incumbent_clean_pass,
-                "candidate_clean_pass": candidate_clean_pass,
-            }
-        )
-        if (
-            incumbent_runs < min_runs
-            or candidate_runs < min_runs
-            or incumbent_runs != candidate_runs
-            or incumbent.get("excluded_runs")
-            or candidate.get("excluded_runs")
-        ):
-            insufficient = True
-            reasons.append(
-                f"{task_id}: needs {min_runs} valid paired runs with zero exclusions "
-                f"(got {incumbent_runs}/{candidate_runs})"
-            )
-        required_values = (
-            (incumbent_fp, candidate_fp, incumbent_clean_pass, candidate_clean_pass)
-            if clean
-            else (incumbent_score, candidate_score, incumbent_fp, candidate_fp)
-        )
-        if any(value is None for value in required_values):
-            insufficient = True
-            reasons.append(f"{task_id}: structured review quality metrics are incomplete")
-            continue
-        if candidate.get("review_verdict_correct") is None:
-            insufficient = True
-            reasons.append(f"{task_id}: candidate verdict evidence is incomplete")
-        elif candidate["review_verdict_correct"] is not True:
-            regression = True
-            reasons.append(f"{task_id}: candidate verdict was incorrect on a valid repeat")
-        if (incumbent_blockers is None) != (candidate_blockers is None):
-            insufficient = True
-            reasons.append(f"{task_id}: blocker recall evidence is incomplete")
-        if incumbent_blockers is not None and candidate_blockers is not None and float(candidate_blockers) < float(
-            incumbent_blockers
-        ):
-            regression = True
-            reasons.append(f"{task_id}: blocker recall regressed")
-        if clean and float(candidate_fp) > float(incumbent_fp):
-            regression = True
-            reasons.append(f"{task_id}: false positives increased on a clean control")
-        if clean and bool(incumbent_clean_pass) and not bool(candidate_clean_pass):
-            regression = True
-            reasons.append(f"{task_id}: clean-control verdict regressed")
-        if not clean and float(candidate_score) + 1e-9 < float(incumbent_score):
-            regression = True
-            reasons.append(f"{task_id}: weighted review score regressed")
-        if not clean and float(candidate_score) >= float(incumbent_score) + min_improvement:
-            improvement = True
-
-    if not task_rows:
-        insufficient = True
-        reasons.append("no paired review task results were found")
-    if insufficient:
-        decision = "insufficient_evidence"
-    elif regression:
-        decision = "keep_incumbent"
-    elif not improvement:
-        decision = "keep_incumbent"
-        reasons.append("candidate did not improve weighted review quality on any corpus case")
-    else:
-        decision = "promote"
-        reasons.append("candidate improved weighted review quality without blocker or clean-control regression")
-    return {
-        "candidate_arm": candidate_arm,
-        "incumbent_arm": incumbent_arm,
-        "decision": decision,
-        "metric": "review_weighted_f1",
-        "model": model,
-        "tasks": task_rows,
-        "ungated_tasks": [],
-        "reasons": reasons,
-    }
-
-
 def evaluate_candidate(
     results: dict[str, dict[str, dict[str, Any]]],
     *,
@@ -826,18 +485,12 @@ def evaluate_candidate(
     min_runs: int = 3,
     min_improvement_pct: float = 5.0,
     max_task_regression_pct: float = 20.0,
-    max_failed_task_regression_pct: float = MAX_FAILED_TASK_REGRESSION_PCT,
 ) -> dict[str, Any]:
     """Deterministically decide whether a prompt candidate is promotable.
 
     Resolution is lexicographically primary: a cheaper candidate that fails
     more tasks never wins. With equal quality, the candidate must clear the
     configured median efficiency gain without a large per-task regression.
-
-    A task neither arm can resolve leaves the quality gate, but only on
-    evidence: a comparable metric, no skill-not-invoked run, and enough tasks
-    left inside the gate to decide anything. It still ranks against the
-    failed-task spend cap.
     """
     if metric not in PROMOTION_METRICS:
         raise ValueError(f"unsupported promotion metric: {metric}")
@@ -883,66 +536,20 @@ def evaluate_candidate(
             if (not metric_unavailable and incumbent_metric)
             else None
         )
-        # A task that both arms measured cleanly and neither ever resolved sits
-        # outside both arms' current capability. It carries no quality signal
-        # about the candidate, and its metric compares who spent more while
-        # failing the same oracle — so gating on it measures the task, not the
-        # candidate, and one such task vetoes every future promotion for as
-        # long as it stays in the set. Keep it in the evidence, out of the gate,
-        # and name it as task health instead.
-        #
-        # Ungating is itself a claim, so it needs evidence: the failures must
-        # be the task's (not a skill that never loaded) and the metric must be
-        # comparable, otherwise the task stays gated and the checks below name
-        # what is missing.
-        fully_measured = (
-            incumbent_runs >= min_runs
-            and candidate_runs >= min_runs
-            and incumbent_runs == candidate_runs
-            and not incumbent_excluded
-            and not candidate_excluded
-        )
-        skill_attributable = bool(
-            (set(incumbent.get("error_kinds", {})) | set(candidate.get("error_kinds", {})))
-            & SKILL_ATTRIBUTABLE_ERROR_KINDS
-        )
-        mutually_unresolved = fully_measured and not incumbent["resolved"] and not candidate["resolved"]
-        gated = not (mutually_unresolved and not skill_attributable and improvement is not None)
-        # The floor asks the candidate to be reliable where the incumbent is.
-        # On a task the incumbent never resolves there is no reliability to
-        # match, and holding partial candidate progress to it punished a
-        # candidate for resolving 1 of 3 runs while excusing it for resolving
-        # none — the strictly worse result. A skill that never loaded is the
-        # exception: those failures belong to the prompts, so the floor applies
-        # even with nothing on the incumbent's side to match.
-        quality_floor_enforced = bool(incumbent["resolved"]) or skill_attributable
         task_rows.append(
             {
                 "task": task_id,
                 "class": incumbent.get("class", ""),
                 "incumbent_resolved": f"{incumbent['resolved']}/{incumbent_runs}",
-                "incumbent": dict(incumbent),
-                "candidate": dict(candidate),
                 "candidate_resolved": f"{candidate['resolved']}/{candidate_runs}",
                 "incumbent_excluded_runs": incumbent_excluded,
                 "candidate_excluded_runs": candidate_excluded,
                 "candidate_quality_floor_met": candidate_runs > 0 and candidate["resolved"] == candidate_runs,
-                "quality_floor_enforced": quality_floor_enforced,
                 "incumbent_metric": incumbent_metric,
                 "candidate_metric": candidate_metric,
                 "improvement_pct": improvement,
-                "gated": gated,
-                "skill_attributable_failure": skill_attributable,
             }
         )
-        if not gated:
-            if improvement < -max_failed_task_regression_pct:
-                efficiency_regression = True
-                reasons.append(
-                    f"{task_id}: {metric} regressed {-improvement:.1f}% on a task neither arm resolved, "
-                    f"above the {max_failed_task_regression_pct:.1f}% failed-task cap"
-                )
-            continue
 
         if incumbent_runs < min_runs or candidate_runs < min_runs:
             insufficient = True
@@ -964,16 +571,11 @@ def evaluate_candidate(
         if candidate_rate < incumbent_rate:
             quality_regression = True
             reasons.append(f"{task_id}: resolution regressed from {incumbent_rate:.0%} to {candidate_rate:.0%}")
-        if quality_floor_enforced and candidate_runs > 0 and candidate["resolved"] != candidate_runs:
+        if candidate_runs > 0 and candidate["resolved"] != candidate_runs:
             quality_floor_failed = True
-            floor_trigger = (
-                "a run never invoked the skill under test"
-                if skill_attributable
-                else f"the incumbent resolves {incumbent['resolved']}/{incumbent_runs}"
-            )
             reasons.append(
                 f"{task_id}: candidate must resolve every valid run for the oracle-backed quality floor "
-                f"({floor_trigger}; got {candidate['resolved']}/{candidate_runs})"
+                f"(got {candidate['resolved']}/{candidate_runs})"
             )
         if metric_unavailable:
             insufficient = True
@@ -990,36 +592,11 @@ def evaluate_candidate(
                 f"{task_id}: {metric} regressed {-improvement:.1f}%, above the {max_task_regression_pct:.1f}% task cap"
             )
 
-    ungated_tasks = [row["task"] for row in task_rows if not row["gated"]]
-    gated_tasks = [row["task"] for row in task_rows if row["gated"]]
-    if ungated_tasks:
-        # One line, not one per task: `reasons` is truncated to three entries
-        # when it is fed back to the proposer (evolve.summarize_gate), and a
-        # growing set of unsolvable tasks must not crowd out the reason the
-        # candidate actually won or lost. The full list ships structurally.
-        reasons.append(
-            f"not gated on {len(ungated_tasks)} task(s) neither arm resolved: {', '.join(ungated_tasks)} "
-            f"(evidence base: {len(gated_tasks)}/{len(task_rows)} paired tasks gated)"
-        )
     if not task_rows:
         insufficient = True
         reasons.append("no paired task results were found")
-    elif not gated_tasks:
-        # Every paired task was ungated, so nothing in this generation says
-        # anything about candidate quality. Refuse rather than fall through to
-        # an efficiency-only verdict on runs that all failed their oracle.
-        insufficient = True
-        reasons.append("no task supplied quality signal: neither arm resolved a run anywhere in the set")
-    elif len(gated_tasks) < MIN_GATED_TASK_RATIO * len(task_rows):
-        # Ungating one unsolvable task keeps promotion reachable; ungating most
-        # of the set turns "promote" into a verdict from whatever is left.
-        insufficient = True
-        reasons.append(
-            f"promotion evidence base is too thin: {len(gated_tasks)}/{len(task_rows)} paired tasks are gated "
-            f"(at least {MIN_GATED_TASK_RATIO:.0%} required)"
-        )
 
-    improvements = [row["improvement_pct"] for row in task_rows if row["gated"] and row["improvement_pct"] is not None]
+    improvements = [row["improvement_pct"] for row in task_rows if row["improvement_pct"] is not None]
     median_improvement = round(statistics.median(improvements), 1) if improvements else None
     incumbent_resolved = sum(
         arms[incumbent_arm]["resolved"] for arms in results.values() if incumbent_arm in arms and candidate_arm in arms
@@ -1063,8 +640,6 @@ def evaluate_candidate(
         "metric": metric,
         "metric_warning": (MAIN_LOOP_ONLY_WARNING if metric in MAIN_LOOP_ONLY_METRICS else None),
         "median_improvement_pct": median_improvement,
-        "ungated_tasks": ungated_tasks,
-        "gated_tasks": gated_tasks,
         "reasons": reasons,
         "tasks": task_rows,
     }

eval/workflow_bench/free-model.litellm.yaml

@@ -9,7 +9,7 @@
 #
 #   uv run python -m workflow_bench.runner \
 #     --tasks workflow_bench/tasks.scenarios.yaml \
-#     --base-url http://localhost:4000 --anthropic-api-key "$LITELLM_MASTER_KEY" \
+#     --base-url http://localhost:4000 --auth-token "$LITELLM_MASTER_KEY" \
 #     --model free-coder
 #
 # Keep the proxy on loopback (litellm's default host). Anyone who can reach
@@ -39,5 +39,5 @@ model_list:
 
 general_settings:
   # No static default — export LITELLM_MASTER_KEY before starting the proxy
-  # and pass the same value as --anthropic-api-key (see header).
+  # and pass the same value as --auth-token (see header).
   master_key: os.environ/LITELLM_MASTER_KEY

eval/workflow_bench/gateway_supervisor.py

@@ -1,45 +0,0 @@
-"""Private gateway owner. EOF on stdin means the harness no longer exists.
-
-Executed by absolute script path so the isolated gateway environment needs no
-PYTHONPATH. The gateway receives DEVNULL, never the owner-liveness descriptor.
-"""
-
-from __future__ import annotations
-
-import os
-import signal
-import sys
-import threading
-
-from process_control import run_managed
-
-
-def main() -> int:
-    cancelled = threading.Event()
-
-    def watch_owner() -> None:
-        try:
-            # A buffered stdin lock held by a daemon aborts CPython shutdown
-            # when the proxy exits while the owner is still alive.
-            os.read(sys.stdin.fileno(), 1)
-        finally:
-            cancelled.set()
-
-    threading.Thread(target=watch_owner, daemon=True).start()
-    for signum in (signal.SIGTERM, signal.SIGINT):
-        signal.signal(signum, lambda *_: cancelled.set())
-    result = run_managed(
-        sys.argv[1:],
-        timeout=7 * 24 * 60 * 60,
-        cancel_event=cancelled,
-        echo_stdout=True,
-    )
-    if result.stderr_tail:
-        print(result.stderr_tail, file=sys.stderr, flush=True)
-    if result.detail:
-        print(result.detail, file=sys.stderr, flush=True)
-    return 0 if result.ok or result.state == "cancelled" else 1
-
-
-if __name__ == "__main__":
-    raise SystemExit(main())