GitNexus/eval/workflow_bench/oracles
Gergo Magyar d541105340 fix(workflow-bench): repair the harness defects the verbose proposer logs exposed
The first fully-logged skill-evolution run failed for five deterministic
reasons that had nothing to do with the candidate under test. Each is fixed
at the layer that actually owns the contract:

- Strict provider adapters materialize omitted optional string arguments as
  "". The MCP alias normalizer now treats a blank optional alias as absent
  (a blank REQUIRED target is still rejected), and a trusted PreToolUse hook
  strips blank strings before Read/GitNexus tool calls.
- MCP semantic errors rode home in a successful envelope and logged as
  result=ok. SessionProgress now inspects the payload and reports them as
  semantic-error.
- Claude Code's nested sandbox overlays absent root dotfiles with device
  nodes, which the provenance snapshot read as unauthorized workspace
  changes. Those names are excluded at the workspace root and hidden from
  git via an immutable excludes file.
- The proposer could not read /evidence from Bash (missing allowRead entry)
  and had no offline gitnexus runner, so it fell back to npx and hit the
  network. Both are now mounted; ripgrep is installed in CI.
- selected-rows.json advertised host artifact names that do not exist in the
  mount. Rows now name their staged patch_file/transcript_files, the prompt
  describes the real layout, and oversized bundles compact artifacts before
  dropping evidence rows so no row is silently lost.

Also replaces two benchmark scenarios that main already satisfies
(trivial-version-alias, inv-bug-pdg-note) with non-vacuous ones, verified to
fail against a pristine checkout.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-03 19:01:45 +00:00
..
cross-module-parse-retry.oracle.test.ts feat(skills): GitNexus Engineering Tool Kits (#2566) 2026-07-19 15:07:24 +01:00
inv-bug-c-system-include.oracle.test.ts fix(workflow-bench): repair the harness defects the verbose proposer logs exposed 2026-09-03 19:01:45 +00:00
inv-feature-list-repos-filter.oracle.test.ts feat(skills): GitNexus Engineering Tool Kits (#2566) 2026-07-19 15:07:24 +01:00
trivial-status-json-alias.oracle.test.ts fix(workflow-bench): repair the harness defects the verbose proposer logs exposed 2026-09-03 19:01:45 +00:00
vitest.config.mts feat(skills): GitNexus Engineering Tool Kits (#2566) 2026-07-19 15:07:24 +01:00