Commit graph

25 commits

Author SHA1 Message Date
Gergo Magyar
1054e3e038 fix(eval): make evolution evidence valid and bounded 2026-09-05 10:08:35 +00:00
Gergo Magyar
6925fb344d feat(eval): evolve review skills against historical PRs
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-04 05:32:31 +00:00
Gergo Magyar
167642ec9d feat(ci): let a dispatched evolution run start from a blank slate
Seeding is unconditional today, so the next run would inherit the rejected
proposal from a generation whose proposer could still read the hidden
oracles. That taint propagates: each generation stages the previous
proposal, so one contaminated proposal survives until the artifact expires.
Scheduled runs still always seed — memoryless weekly runs would re-propose
the same rejected candidate forever.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-03 20:14:05 +00:00
Gergo Magyar
d541105340 fix(workflow-bench): repair the harness defects the verbose proposer logs exposed
The first fully-logged skill-evolution run failed for five deterministic
reasons that had nothing to do with the candidate under test. Each is fixed
at the layer that actually owns the contract:

- Strict provider adapters materialize omitted optional string arguments as
  "". The MCP alias normalizer now treats a blank optional alias as absent
  (a blank REQUIRED target is still rejected), and a trusted PreToolUse hook
  strips blank strings before Read/GitNexus tool calls.
- MCP semantic errors rode home in a successful envelope and logged as
  result=ok. SessionProgress now inspects the payload and reports them as
  semantic-error.
- Claude Code's nested sandbox overlays absent root dotfiles with device
  nodes, which the provenance snapshot read as unauthorized workspace
  changes. Those names are excluded at the workspace root and hidden from
  git via an immutable excludes file.
- The proposer could not read /evidence from Bash (missing allowRead entry)
  and had no offline gitnexus runner, so it fell back to npx and hit the
  network. Both are now mounted; ripgrep is installed in CI.
- selected-rows.json advertised host artifact names that do not exist in the
  mount. Rows now name their staged patch_file/transcript_files, the prompt
  describes the real layout, and oversized bundles compact artifacts before
  dropping evidence rows so no row is silently lost.

Also replaces two benchmark scenarios that main already satisfies
(trivial-version-alias, inv-bug-pdg-note) with non-vacuous ones, verified to
fail against a pristine checkout.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-03 19:01:45 +00:00
Gergo Magyar
3c2648b3c9 fix(eval): trim oversized proposer evidence instead of aborting
Selected rows can exceed the 2MiB sandbox bundle even when each file is capped; shrink the seed and stage path so a fat prior artifact no longer kills the evolution job.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-03 11:19:34 +00:00
Gergo Magyar
b580c96501 fix(ci): wait for host OOM guard before evolution preflight
The runner stamps job processes at 500 faster than the host rewrite; a single read failed a live OpenAI dispatch before any model work.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-03 11:02:01 +00:00
Gergo Magyar
37415cb1ca feat(eval): route skill evolution through OpenAI
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-03 10:52:37 +00:00
Gergo Magyar
45f97045e0 refactor(eval): simplify sweep internals and needrestart check (#2785)
Reuse the incumbent skill digest instead of walking the tree twice, and
keep the needrestart grep a literal match that actionlint accepts.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-03 08:19:28 +00:00
Gergo Magyar
b7e621df4c fix(ci): gate evolution runs on runner readiness (#2785)
Prevent paid scheduled work until host survival protections and the proven three-worker rollout are explicitly in place.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-03 07:46:26 +00:00
Gergo Magyar
999b7bbede fix(eval): close skill evolution review gaps (#2785)
Keep promotion decisions monotonic and evidence-bound while preserving paid sweep results, redacting live failures, and hardening prior-run seeding.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-03 07:38:01 +00:00
Abhinav Pandey
c3eb5991c1
fix(eval): preserve complete evolution evidence 2026-09-03 05:18:46 +05:30
Abhinav Pandey
aeb853b9cb
fix(ci): harden evolution evidence reuse 2026-09-03 04:02:07 +05:30
Gergo Magyar
a5251d6b08 docs(ci): record why the runner box must not restart services mid-job
A run spans ~15h and apt-daily-upgrade.timer fires daily around 06:34, so
every scheduled run crosses it. On 2026-08-02 unattended-upgrades upgraded
openssl at 07:54:02 and needrestart restarted the Actions runner five
seconds later. The job went to Canceled 14s after that, and a cancelled
job skips even `if: always()` — so the evidence artifact died with it,
which is the one outcome the rest of this workflow's budget nesting exists
to prevent.

This is a plausible contributor to the unexplained mid-run failures in the
July dispatch cluster, none of which left an artifact behind either.

The box config itself is applied out-of-band like the rest of the instance
setup; the checklist now carries the requirement so a rebuilt box does not
silently reintroduce it.
2026-08-02 08:25:27 +00:00
Gergo Magyar
2fbd5ee515 fix(ci): make the task repo resolve the ref its tasks name
Every task in tasks.scenarios.yaml names `ref: main`, and resolving it is
the first thing task binding does. actions/checkout only creates a local
ref for the ref it checked out, so `main^{commit}` resolves on a main run
and dies with "unknown revision" on any other — which is what a
workflow_dispatch from a branch hits, before a single session starts.

The step that points the benchmark at the checkout now also makes that
checkout able to answer for the refs the tasks name. On a main run the
fetch is a no-op.
2026-08-02 07:07:15 +00:00
Gergo Magyar
8076b98fcc refactor(ci): address the evidence path directly instead of threading it
The upload step needs a path that does not depend on the sweep step
surviving. It did not need shared state to get one: `runner.temp` is
available in a step, only not in a job-level `env:`, so each of the three
consumers can name `${RUNNER_TEMP}/wfevolve` itself. That deletes the
env var and the step that published it — the previous fix swapped one
threading channel for a sturdier one where no channel was required.

Also from the same review pass:

- `announce`/`keep` drop their default-argument capture of `task["id"]`
  and `per_arm`. Late binding only bites a closure invoked after the loop
  moves on; these are called synchronously inside `sweep_task_cells`,
  which blocks until every wave completes. The trick was guarding against
  a race that cannot happen here, while implying to the next reader that
  it can.
- `_stub_cell_dependencies` returns the list its teardown appends to
  rather than taking it as an out-parameter, dropping the boilerplate
  from every call site.
- The workflow's `WORKERS` comment points at the `--workers` help text
  instead of restating it, so the rationale has one home.
2026-08-01 19:34:57 +00:00
Gergo Magyar
d18dbd4143 fix(ci): publish the evidence path from a step, not a job-level env
`${{ runner.temp }}` does not exist in a job-level `env:` block — the
runner context is only available to steps — so OUT_ROOT would have
resolved to a bare `/wfevolve` at the filesystem root. The sweep would
have failed writing there, and the upload would have pointed at nothing.
actionlint caught it; this repo lints workflows for exactly this reason.

The property that mattered is kept: the path is fixed before anything can
fail, rather than read from the sweep step's outputs — that being the
step whose death is the reason the upload matters. The first step now
publishes it to GITHUB_ENV, which every later step sees, including the
`if: always()` upload after a killed sweep.

The contract test pins the step's position and its exact line, so the
context cannot creep back into the job block.
2026-08-01 18:53:51 +00:00
Gergo Magyar
edb24da1e8 feat(ci): expose benchmark cell concurrency to the evolution lane
`--workers` reaches the sweep from evolve.py and from a workflow_dispatch
input. Both default to 1, so nothing about the scheduled lane changes:
the runner is sized for one cell at a time, and a cell starved of CPU
drifts toward its session timeout, which the gate counts as an excluded
run and refuses to decide on.

generation_timeout_seconds is left alone deliberately — it is a
worst-case sum-of-every-timeout bound (843h at current settings), already
far looser than any real run, and concurrency only makes it looser.

Raising the input is gated on the runner resize; the contract test pins
the default so the lane cannot start running 3-way on a 2-vCPU box by
accident.
2026-08-01 18:48:19 +00:00
Gergo Magyar
01be282667 fix(ci): make the evolution lane survive its own deadlines and remember prior runs
An end-to-end pass over the lane — instance start, job, artifacts,
promotion — found three ways it loses work that has already been paid for.

**Evidence died with the job.** Three budgets have to nest: EventBridge
keeps the box up 24h from ~02:45, the job timeout was also 1440min, and
the sweep had no budget of its own. A job-level timeout CANCELS the job,
so the upload step never runs; and since the box stops 24h after it
starts while a scheduled run can begin well after the cron (the
2026-08-01 run was queued 65min late), the box always won that race —
the runner would simply vanish mid-step. The job now gets 21h, the sweep
step 19h, so a wedged generation fails the step, keeps the job alive, and
still uploads. The nesting is asserted in the contract test.

**The upload could be skipped.** Its path came from an output the sweep
step wrote — the same step whose death is the reason the upload matters.
OUT_ROOT is now a job-level env constant known before anything runs, and
the upload is unconditional: results.jsonl and transcripts are appended
as the sweep goes, so a killed generation still holds the evidence that
explains why it died.

**The lane was memoryless.** `--seed-results` is how a run sees what
already lost (summarize_gate feeds the prior promotion.json to the
proposer), and with the default --generations 1 there is no earlier
generation in-process to supply it — the workflow never passed it, so
every Saturday proposed from a blank slate and could re-propose the same
rejected candidate forever. The lane now seeds from the last successful
run's artifact, best-effort: a first run, an expired artifact, a missing
gh, or a failed download proceeds without it rather than costing a
generation.

Also guards the silent-promotion path: `.claude/skills/*` is gitignored
with a hand-maintained per-skill allowlist, and `git status --porcelain`
— how the workflow detects an applied promotion — is blind to ignored
paths. A candidate skill missing from that allowlist would report "No
promotion this run" after the gate said promote. A test now asserts every
CANDIDATE_SKILLS entry is visible in all three shipped trees.
2026-08-01 17:42:48 +00:00
Gergo Magyar
5a017f2722 feat(eval): report evolution progress while the generation is still running
Run 29907431284 printed its whole 14h45m of output at one timestamp
(00:02:59.32) as the process exited: stdout is a pipe, so CPython
block-buffered it, and there was no way to tell a live run from a wedged
one. Three changes make the lane observable in the Actions log:

- PYTHONUNBUFFERED for the driver (workflow step) and for the benchmark
  subprocess (its env is an explicit minimal dict and inherits nothing),
  so lines reach the log when they are written.
- run_managed grows `echo_stdout`, a passthrough that streams a child's
  stdout to stderr as it arrives while leaving the bounded tail intact.
  evolve.py enables it for the benchmark sweep — the multi-hour phase,
  whose per-run lines previously surfaced only as a tail, and only on
  failure. It stays off everywhere else: a Claude session's stdout is the
  evidence stream and is written out only after redaction.
- The sweep now announces each cell as it starts (`3/18, 47m elapsed`)
  and reports `took=` and `error_kind=` when it finishes, so an excluded
  run — the thing that actually blocks promotion — is visible live
  instead of only in results.jsonl. evolve.py also reports the proposer's
  duration, turns, and cost once the proposal lands.
2026-08-01 17:01:03 +00:00
Gergo Magyar
bb09ce28e0 fix(eval): stop discarding completed benchmark sessions as unverifiable
The evolution loop has not been able to promote anything since it went
online. Run 29907431284 (the last green run) reached the gate and threw
away 5 of its 18 runs, and the gate requires zero excluded runs in both
paired arms — so the generation could never produce a verdict on merit.

Two causes, both in the session layer:

1. Claude Code drains background-task bookkeeping after the final result
   event (`background_tasks_changed`, `task_updated`, `task_notification`,
   all `type: "system"`). The parent-stream check required the result to
   be the literal last event, so three sessions that had exited 0 with a
   complete result and usage payload were recorded as session errors.
   Trailing `system` events carry no tool_use/tool_result/usage payload
   and cannot forge skill or cost evidence; anything else after the
   result still fails closed.

2. The 3600s per-session ceiling killed two `workflow` incumbent runs on
   inv-bug-pdg-note mid-verification. Successful `workflow` rows in the
   same run finished in ~1600-2600s across both sessions, so the ceiling
   moves to 5400s and now lives in one shared constant instead of two
   argparse defaults that could drift apart.

Also marks the activation checklist against reality: the secrets, the
Environment, the runner, and the validation dispatch are all in place;
the repository variable GITNEXUS_EVOLUTION_ENABLED is the one remaining
gap, and until it is set the Saturday cron skips the job in seconds while
the EventBridge schedule still starts the runner for the day.
2026-08-01 16:45:48 +00:00
dependabot[bot]
47f3932c8c
chore(deps): bump actions/setup-node from 6.4.0 to 7.0.0
Bumps [actions/setup-node](https://github.com/actions/setup-node) from 6.4.0 to 7.0.0.
- [Release notes](https://github.com/actions/setup-node/releases)
- [Commits](48b55a011b...8207627860)

---
updated-dependencies:
- dependency-name: actions/setup-node
  dependency-version: 7.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-07-22 20:18:29 +00:00
Gergő Magyar
5549403082
fix(eval): self-hosted skill-evolution runner + sandbox Python 3 trust fix (#2600)
* fix(eval): move skill-evolution to a self-hosted runner and fix the sandbox's Python 3 trust gap

GitHub-hosted runners hard-cap job execution at 6 hours, which is too
short once a benchmark session actually invokes Skill/MCP tools for
real (the --bare fix in #2584 means sessions no longer no-op). Move the
job onto a self-hosted runner (5-day cap instead) and document the
activation step in the workflow's own checklist.

Validating the self-hosted run surfaced a real bug: gitnexus-plan
sessions inside the bwrap sandbox failed with "planning must create or
modify exactly one plan artifact; observed 0". Root cause:
evidence-provenance.mjs's atomic plan-writer only trusts a Python 3
binary owned by root or by the current process. Inside this
--unshare-user sandbox only the calling uid is mapped (root isn't), so
the real, root-owned /usr/bin/python3 surfaces as the kernel's overflow
uid and gets correctly refused as untrusted. Fix: provision a small,
self-owned wrapper script (same pattern already used for
shell-prefix) that execs the real interpreter, so the sandbox has a
Python 3 candidate the existing trust check can actually accept --
without touching that security-sensitive validation logic at all.

Also add visibility so this class of failure isn't quiet next time:
report.md now shows why each row failed (error_kinds), not just
resolved 0/1, and the benchmark now exits non-zero when an incumbent
arm -- the currently-shipped skill -- resolves zero across every task,
since that reads as a broken harness rather than a normal candidate
miss.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(eval): close the broken_incumbent_arms zero-valid-runs gap; document runner exposure tradeoff

Addresses the two MEDIUM findings from the gitnexus-review-agent on this PR
(https://github.com/abhigyanpatwari/GitNexus/pull/2600#issuecomment-5033363096).

broken_incumbent_arms required valid_runs > 0 before flagging an incumbent,
so an incumbent that fails every run with an excluded-but-non-systemic
error_kind (e.g. evidence-unverified, which the outage-streak breaker
explicitly resets on rather than accumulates) never accumulated a single
valid run and sailed through silently -- the exact quiet no-promotion
outcome this guard exists to catch, and arguably worse than the
some-runs-resolved-zero case since here nothing completed at all.
aggregate() never marks an excluded/unverifiable row resolved=True, so
dropping the valid_runs requirement and checking resolved == 0 alone
correctly covers both cases. Added a test for exactly this all-excluded
scenario, which none of the existing three did.

Updated the workflow's own activation checklist to reflect what's actually
true now (the gitnexus-evolution environment's branch policy and the
self-hosted runner are both live, codified in infra/gitnexus-evolution/ in
a companion PR) and documented the exposure-window tradeoff the review
flagged: the runner is stopped between runs but not destroyed/recreated per
run, so it isn't fully ephemeral. Stopping already bounds the exposure
window to the job's own runtime on one day out of seven; full per-job
ephemeral provisioning is a deliberate non-goal for a job that runs at
most weekly, revisit if that changes.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* docs(eval): remove public infra/ pointers from the activation checklist

PR #2603 (the Terraform codification this checklist pointed to) got closed
-- publishing the exact IAM roles, security group rules, and self-hosted
runner topology for a real, live AWS account isn't safe to do in a public
repo, even with no literal secrets or resource IDs in the diff. The
underlying AWS/GitHub setup is unaffected and still documented privately;
this just removes the now-dangling references to a directory that won't
exist in this repo.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* ci(actionlint): register the gitnexus-evolution self-hosted runner label

actionlint rejected `runs-on: [self-hosted, linux, x64, gitnexus-evolution]`
in gitnexus-skill-evolution.yml because it can't discover custom runner
labels. Register it in .github/actionlint.yaml so the Workflow Lint check
passes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-21 14:40:20 +01:00
Claude
eea9ac92dc ci: move Node pins to the 22.18 floor
With the supported minimum raised to Node 22.18, retarget every lane and
pinned runtime that sat at a lower version so nothing builds or runs the
package on an unsupported (EBADENGINE-warning) Node:

- ci-tests.yml: node-floor-compat 22.14 -> 22.18.0 (name, comment, pin,
  version assertion) so the floor gate guards the new minimum; its #2372
  registerHooks failure mode cannot recur above 22.15. Containment-canary
  pin 22.16.0 -> 22.18.0.
- gitnexus-review-agent.yml + the pinned review/canary runtime: the
  reproducible runtime is version-locked in lockstep across
  .github/{gitnexus-review-runtime,claude-canary-runtime}/package.json and
  their lockfiles (engines), the workflow's node-version, its two
  'node --version = v22.18.0' assertions, the lockfile-engines guard, and
  NODE_VERSION. Moved all of them 22.16.0 -> 22.18.0.
- gitnexus-skill-evolution.yml: pinned runtime 22.16.0 -> 22.18.0.
- CONTRIBUTING.md prerequisite floor updated.
- review-agent-workflow.test.ts, which enforces the runtime lock, updated
  to expect 22.18.0.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 10:09:34 +00:00
Abhigyan Patwari
2ea00a2b22
fix(ci): install root and shared node_modules for the evolution benchmark (#2575)
* fix(ci): install root and shared node_modules for the evolution benchmark

The first real workflow_dispatch of the skill-evolution loop failed at task
binding: capture_task_dependency_binding aborted with

  SandboxError: sandbox_copy path is unavailable: node_modules: No such
  file or directory

The benchmark tasks sandbox-copy node_modules from three locations
(tasks.scenarios.yaml) — the monorepo root, gitnexus-shared, and gitnexus —
mirroring a full dev checkout. The install step only ran `npm ci` in
gitnexus/, so the root and gitnexus-shared node_modules never existed and
the loop died before any agent ran. Install all three (root, then build
gitnexus-shared, then build gitnexus), matching the per-package install in
ci-tests.yml plus the root deps the tasks require.

A new contract test pins all three installs so this fails in CI rather than
on the next real run — the same guard the workflow's other two P1 fixes got.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Va5uu9Ar3e45QZ5xFsG4AZ

* fix(ci): only add the missing root install; the subpackage steps already exist

The initial fix redundantly rebuilt gitnexus-shared and gitnexus inside the
gitnexus step — but the workflow already builds both in their own dedicated
steps. Only the monorepo root node_modules was missing. Add a single
"Install monorepo root dependencies" step and leave the two subpackage
build steps untouched, so the benchmark's root sandbox_copy resolves without
double-building.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Va5uu9Ar3e45QZ5xFsG4AZ

---------

Co-authored-by: Gergo Magyar <gergomagyar@icloud.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-20 09:56:54 +01:00
Gergő Magyar
becac9a5d3
feat(eval): run the skill-evolution loop online (#2571)
* feat(eval): run the skill-evolution loop online

Add a scheduled + dispatch-gated workflow that runs the offline
propose -> benchmark -> gate loop (workflow_bench.evolve) in CI with the
pinned Claude canary runtime and bubblewrap containment, uploads the
benchmark evidence as an artifact, and on a gate-passed promotion opens
a human-reviewed PR via the release App token. The applied overlay is
bounded to the canonical skill tree and its shipped mirrors; any escape
fails the run instead of reaching a PR.

The scheduled lane ships disabled behind GITNEXUS_EVOLUTION_ENABLED and
requires the new GITNEXUS_BENCH_AUTH_TOKEN secret (benchmark sessions
bill real API usage), mirroring the review agent's staged rollout.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Va5uu9Ar3e45QZ5xFsG4AZ

* fix(ci): restructure promotion-PR script so no lint suppression is needed

Replace the inline single-quoted credential helper with a GIT_ASKPASS
file written via a quoted heredoc (the App token still reaches git only
through step env at push time), and assemble the PR body from quoted
heredocs plus double-quoted printf instead of a backtick-laden
single-quoted template. Every run script in the workflow now passes
shellcheck with zero findings and zero disables; the body and askpass
rendering are smoke-tested.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Va5uu9Ar3e45QZ5xFsG4AZ

* fix(ci): apply gate-passing overlays in the evolution loop

The loop invoked workflow_bench.evolve without --apply, so
apply_promoted_overlay (its only working-tree writer, gated by
`if args.apply:`) never ran. git status stayed clean, promoted=false was
emitted every run, and the App-token/PR-open steps were unreachable dead
code — a gate-passing run went green as "No promotion this run".

validate_promotion_for_apply already runs before the apply gate, so
adding --apply lets a passing candidate reach the tree without weakening
the deterministic gate; the boundary check then confirms it stayed in the
skill trees.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Va5uu9Ar3e45QZ5xFsG4AZ

* fix(ci): provision ~/GitNexus so the benchmark repo resolves on CI

Every scenario in tasks.scenarios.yaml addresses the target repo as
~/GitNexus; runner_tasks.py resolves it with expanduser().resolve() then
`git -C <repo> rev-parse`, which raises when the path is missing. On a
hosted runner the checkout lands in $GITHUB_WORKSPACE and nothing created
~/GitNexus, so the first real run failed at task-binding.

Symlink ~/GitNexus -> $GITHUB_WORKSPACE before the loop. The checkout uses
fetch-depth: 0 (full history for the parentless clone), and the benchmark
only clones the repo copy-on-write and mounts deps read-only, so the
checkout is never mutated. GITNEXUS_BENCH_ORACLE_ROOT stays unset — it
defaults to the in-repo oracles dir and is staged by the harness.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Va5uu9Ar3e45QZ5xFsG4AZ

* fix(ci): harden promotion summary output and PR branch recovery

Three fixes to the promotion-detection and PR-open steps:

- GITHUB_OUTPUT summary used a fixed `PROMOTION_EOF` heredoc delimiter; a
  value containing that marker on its own line could close the block early
  and inject output keys. Use a per-run random delimiter, matching the
  pattern already in tree-sitter-upgrade-readiness.yml.
- The summary concatenated every generation's promotion.json (including
  rejected ones), so the PR body could show a losing generation's
  decisions. The loop returns on the first promotion, so emit only the
  highest-numbered gen-N/bench/promotion.json — the decision that fired.
- The promotion branch name omitted the run attempt. GITHUB_RUN_ID is
  stable across re-runs, so a re-run after push-succeeds/PR-create-fails
  could never push. Include ${GITHUB_RUN_ATTEMPT} (the artifact name
  already does).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Va5uu9Ar3e45QZ5xFsG4AZ

* fix(ci): least-privilege the promotion App token and gate on an Environment

The Mint-App-Token step passed only app-id + private-key, so the minted
token inherited every permission the Release App installation holds
(including Workflows: write) — far more than "push a branch, open a PR".
Switch to `client-id` (as publish.yml does) and request only
permission-contents: write + permission-pull-requests: write.

Bind the job to a protected Environment (gitnexus-evolution) so promotion
runs can be gated server-side. workflow_dispatch runs the workflow and
in-tree evolve.py from the *dispatched ref*, so a code-side ref guard is
removable by the dispatched branch itself; an Environment deployment-branch
rule (main only) is the boundary that holds. The admin steps to create it
and scope the secrets are documented in the activation checklist.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Va5uu9Ar3e45QZ5xFsG4AZ

* fix(ci): correct upload-artifact pin comment and add shell strict-mode

- The upload-artifact SHA 043fb46d… is v7.0.1 (labeled so in the sibling
  workflows that pin it); the comment mislabeled it # v6.0.0. Correct the
  comment; the pin is unchanged.
- Add `set -euo pipefail` to the two build steps that lacked it, matching
  every other run block in the file (GitHub's default shell already sets
  -eo pipefail; this adds -u and consistency).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Va5uu9Ar3e45QZ5xFsG4AZ

* docs(ci): complete the skill-evolution activation checklist

- Add RELEASE_APP_ID / RELEASE_APP_PRIVATE_KEY to the required-secrets
  checklist (the Mint step hard-fails without them on a promotion) and the
  App-install-scope verification.
- Document the protected Environment admin step and why it is the real
  boundary for the workflow_dispatch ref-secret exposure.
- Note that workflow_dispatch runs the billing loop regardless of
  GITNEXUS_EVOLUTION_ENABLED.
- Justify the weekly cron against the README's ~90-day guidance and note the
  355-minute timeout ceiling.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Va5uu9Ar3e45QZ5xFsG4AZ

* fix(eval): redact API tokens from diagnostic fields before artifact upload

results.jsonl (runner.py) and proposer-session.json (evolve.py) serialize
session records whose error_detail can carry a stderr_tail that echoed the
API key. Transcripts are redacted before persistence, but these two sinks
were not, and both land in the 14-day evolution artifact.

Run each record's serialized JSON through the existing redact_text with the
run's auth token before writing. Scoped to these diagnostic sinks only: the
promoted overlay and proposal.md are left untouched (the overlay is the
applied artifact and must stay byte-identical for apply and the
shipped-skills-sync guard).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Va5uu9Ar3e45QZ5xFsG4AZ

* test(ci): add a contract test for the skill-evolution workflow

No test exercised this workflow's path, which is why both P1 blockers
(missing --apply, unresolvable ~/GitNexus task repo) reached production.
Parse the workflow YAML and assert the structural contract: --apply is
passed, the task repo is provisioned, the promotion branch carries the run
attempt, the App token is permission-scoped and the job is Environment-
gated, the output summary uses a random delimiter and a single generation,
the artifact pin is labelled correctly, and every multi-line shell step
sets strict mode. Follows the review-agent-workflow.test.ts precedent.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Va5uu9Ar3e45QZ5xFsG4AZ

* feat(ci): run the proposer on its own (stronger) model

One `model` input drove both the benchmark arms and the proposer/diagnosis
session. Split them: `model` stays the benchmark arms (match the model your
skill users run, so a promotion is valid for them and the tasks aren't
ceiling-saturated), and a new `proposer_model` input runs the proposer —
the harder meta-reasoning task that writes the candidate skill, and only one
session per generation, so a stronger model is cheap here. evolve.py already
supports --proposer-model; the workflow just didn't expose it.

Defaults: arms = claude-sonnet-5, proposer = claude-opus-4-8 (both
overridable via workflow_dispatch). The weekly cadence bounds the added
spend. Contract test asserts the split stays wired.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Va5uu9Ar3e45QZ5xFsG4AZ

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-20 05:37:07 +01:00