Seeding is unconditional today, so the next run would inherit the rejected
proposal from a generation whose proposer could still read the hidden
oracles. That taint propagates: each generation stages the previous
proposal, so one contaminated proposal survives until the artifact expires.
Scheduled runs still always seed — memoryless weekly runs would re-propose
the same rejected candidate forever.
Co-authored-by: Cursor <cursoragent@cursor.com>
The first fully-logged skill-evolution run failed for five deterministic
reasons that had nothing to do with the candidate under test. Each is fixed
at the layer that actually owns the contract:
- Strict provider adapters materialize omitted optional string arguments as
"". The MCP alias normalizer now treats a blank optional alias as absent
(a blank REQUIRED target is still rejected), and a trusted PreToolUse hook
strips blank strings before Read/GitNexus tool calls.
- MCP semantic errors rode home in a successful envelope and logged as
result=ok. SessionProgress now inspects the payload and reports them as
semantic-error.
- Claude Code's nested sandbox overlays absent root dotfiles with device
nodes, which the provenance snapshot read as unauthorized workspace
changes. Those names are excluded at the workspace root and hidden from
git via an immutable excludes file.
- The proposer could not read /evidence from Bash (missing allowRead entry)
and had no offline gitnexus runner, so it fell back to npx and hit the
network. Both are now mounted; ripgrep is installed in CI.
- selected-rows.json advertised host artifact names that do not exist in the
mount. Rows now name their staged patch_file/transcript_files, the prompt
describes the real layout, and oversized bundles compact artifacts before
dropping evidence rows so no row is silently lost.
Also replaces two benchmark scenarios that main already satisfies
(trivial-version-alias, inv-bug-pdg-note) with non-vacuous ones, verified to
fail against a pristine checkout.
Co-authored-by: Cursor <cursoragent@cursor.com>
Selected rows can exceed the 2MiB sandbox bundle even when each file is capped; shrink the seed and stage path so a fat prior artifact no longer kills the evolution job.
Co-authored-by: Cursor <cursoragent@cursor.com>
The runner stamps job processes at 500 faster than the host rewrite; a single read failed a live OpenAI dispatch before any model work.
Co-authored-by: Cursor <cursoragent@cursor.com>
Prevent paid scheduled work until host survival protections and the proven three-worker rollout are explicitly in place.
Co-authored-by: Cursor <cursoragent@cursor.com>
`ManagedProcessError.__str__` embeds up to 1000 raw bytes of stderr_tail
(process_control.py:72-73), and run_cell printed it verbatim. That line
was inert until this branch: nothing ever printed the sweep subprocess's
stdout, on success or failure. `echo_stdout` streams it live into the job
log, so the print became a sink — and the only one of its kind here that
skipped `redact_text`, which results.jsonl and every transcript already
apply to exactly this field, for exactly this reason.
GitHub masks the registered secret, but masking only catches that literal
value; it is not the guarantee the other sinks have.
The test drives a ManagedProcessError carrying the token in stderr_tail
and asserts it never reaches stdout — verified to fail without the fix.
Also from the same pass: bind `result_indexes[0]` once in run_claude, and
give the workflow contract test a `findStep` helper instead of five
copies of the same `steps.find` predicate.
The upload step needs a path that does not depend on the sweep step
surviving. It did not need shared state to get one: `runner.temp` is
available in a step, only not in a job-level `env:`, so each of the three
consumers can name `${RUNNER_TEMP}/wfevolve` itself. That deletes the
env var and the step that published it — the previous fix swapped one
threading channel for a sturdier one where no channel was required.
Also from the same review pass:
- `announce`/`keep` drop their default-argument capture of `task["id"]`
and `per_arm`. Late binding only bites a closure invoked after the loop
moves on; these are called synchronously inside `sweep_task_cells`,
which blocks until every wave completes. The trick was guarding against
a race that cannot happen here, while implying to the next reader that
it can.
- `_stub_cell_dependencies` returns the list its teardown appends to
rather than taking it as an out-parameter, dropping the boilerplate
from every call site.
- The workflow's `WORKERS` comment points at the `--workers` help text
instead of restating it, so the rationale has one home.
`${{ runner.temp }}` does not exist in a job-level `env:` block — the
runner context is only available to steps — so OUT_ROOT would have
resolved to a bare `/wfevolve` at the filesystem root. The sweep would
have failed writing there, and the upload would have pointed at nothing.
actionlint caught it; this repo lints workflows for exactly this reason.
The property that mattered is kept: the path is fixed before anything can
fail, rather than read from the sweep step's outputs — that being the
step whose death is the reason the upload matters. The first step now
publishes it to GITHUB_ENV, which every later step sees, including the
`if: always()` upload after a killed sweep.
The contract test pins the step's position and its exact line, so the
context cannot creep back into the job block.
`--workers` reaches the sweep from evolve.py and from a workflow_dispatch
input. Both default to 1, so nothing about the scheduled lane changes:
the runner is sized for one cell at a time, and a cell starved of CPU
drifts toward its session timeout, which the gate counts as an excluded
run and refuses to decide on.
generation_timeout_seconds is left alone deliberately — it is a
worst-case sum-of-every-timeout bound (843h at current settings), already
far looser than any real run, and concurrency only makes it looser.
Raising the input is gated on the runner resize; the contract test pins
the default so the lane cannot start running 3-way on a 2-vCPU box by
accident.
An end-to-end pass over the lane — instance start, job, artifacts,
promotion — found three ways it loses work that has already been paid for.
**Evidence died with the job.** Three budgets have to nest: EventBridge
keeps the box up 24h from ~02:45, the job timeout was also 1440min, and
the sweep had no budget of its own. A job-level timeout CANCELS the job,
so the upload step never runs; and since the box stops 24h after it
starts while a scheduled run can begin well after the cron (the
2026-08-01 run was queued 65min late), the box always won that race —
the runner would simply vanish mid-step. The job now gets 21h, the sweep
step 19h, so a wedged generation fails the step, keeps the job alive, and
still uploads. The nesting is asserted in the contract test.
**The upload could be skipped.** Its path came from an output the sweep
step wrote — the same step whose death is the reason the upload matters.
OUT_ROOT is now a job-level env constant known before anything runs, and
the upload is unconditional: results.jsonl and transcripts are appended
as the sweep goes, so a killed generation still holds the evidence that
explains why it died.
**The lane was memoryless.** `--seed-results` is how a run sees what
already lost (summarize_gate feeds the prior promotion.json to the
proposer), and with the default --generations 1 there is no earlier
generation in-process to supply it — the workflow never passed it, so
every Saturday proposed from a blank slate and could re-propose the same
rejected candidate forever. The lane now seeds from the last successful
run's artifact, best-effort: a first run, an expired artifact, a missing
gh, or a failed download proceeds without it rather than costing a
generation.
Also guards the silent-promotion path: `.claude/skills/*` is gitignored
with a hand-maintained per-skill allowlist, and `git status --porcelain`
— how the workflow detects an applied promotion — is blind to ignored
paths. A candidate skill missing from that allowlist would report "No
promotion this run" after the gate said promote. A test now asserts every
CANDIDATE_SKILLS entry is visible in all three shipped trees.
* fix(ci): install root and shared node_modules for the evolution benchmark
The first real workflow_dispatch of the skill-evolution loop failed at task
binding: capture_task_dependency_binding aborted with
SandboxError: sandbox_copy path is unavailable: node_modules: No such
file or directory
The benchmark tasks sandbox-copy node_modules from three locations
(tasks.scenarios.yaml) — the monorepo root, gitnexus-shared, and gitnexus —
mirroring a full dev checkout. The install step only ran `npm ci` in
gitnexus/, so the root and gitnexus-shared node_modules never existed and
the loop died before any agent ran. Install all three (root, then build
gitnexus-shared, then build gitnexus), matching the per-package install in
ci-tests.yml plus the root deps the tasks require.
A new contract test pins all three installs so this fails in CI rather than
on the next real run — the same guard the workflow's other two P1 fixes got.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Va5uu9Ar3e45QZ5xFsG4AZ
* fix(ci): only add the missing root install; the subpackage steps already exist
The initial fix redundantly rebuilt gitnexus-shared and gitnexus inside the
gitnexus step — but the workflow already builds both in their own dedicated
steps. Only the monorepo root node_modules was missing. Add a single
"Install monorepo root dependencies" step and leave the two subpackage
build steps untouched, so the benchmark's root sandbox_copy resolves without
double-building.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Va5uu9Ar3e45QZ5xFsG4AZ
---------
Co-authored-by: Gergo Magyar <gergomagyar@icloud.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* feat(eval): run the skill-evolution loop online
Add a scheduled + dispatch-gated workflow that runs the offline
propose -> benchmark -> gate loop (workflow_bench.evolve) in CI with the
pinned Claude canary runtime and bubblewrap containment, uploads the
benchmark evidence as an artifact, and on a gate-passed promotion opens
a human-reviewed PR via the release App token. The applied overlay is
bounded to the canonical skill tree and its shipped mirrors; any escape
fails the run instead of reaching a PR.
The scheduled lane ships disabled behind GITNEXUS_EVOLUTION_ENABLED and
requires the new GITNEXUS_BENCH_AUTH_TOKEN secret (benchmark sessions
bill real API usage), mirroring the review agent's staged rollout.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Va5uu9Ar3e45QZ5xFsG4AZ
* fix(ci): restructure promotion-PR script so no lint suppression is needed
Replace the inline single-quoted credential helper with a GIT_ASKPASS
file written via a quoted heredoc (the App token still reaches git only
through step env at push time), and assemble the PR body from quoted
heredocs plus double-quoted printf instead of a backtick-laden
single-quoted template. Every run script in the workflow now passes
shellcheck with zero findings and zero disables; the body and askpass
rendering are smoke-tested.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Va5uu9Ar3e45QZ5xFsG4AZ
* fix(ci): apply gate-passing overlays in the evolution loop
The loop invoked workflow_bench.evolve without --apply, so
apply_promoted_overlay (its only working-tree writer, gated by
`if args.apply:`) never ran. git status stayed clean, promoted=false was
emitted every run, and the App-token/PR-open steps were unreachable dead
code — a gate-passing run went green as "No promotion this run".
validate_promotion_for_apply already runs before the apply gate, so
adding --apply lets a passing candidate reach the tree without weakening
the deterministic gate; the boundary check then confirms it stayed in the
skill trees.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Va5uu9Ar3e45QZ5xFsG4AZ
* fix(ci): provision ~/GitNexus so the benchmark repo resolves on CI
Every scenario in tasks.scenarios.yaml addresses the target repo as
~/GitNexus; runner_tasks.py resolves it with expanduser().resolve() then
`git -C <repo> rev-parse`, which raises when the path is missing. On a
hosted runner the checkout lands in $GITHUB_WORKSPACE and nothing created
~/GitNexus, so the first real run failed at task-binding.
Symlink ~/GitNexus -> $GITHUB_WORKSPACE before the loop. The checkout uses
fetch-depth: 0 (full history for the parentless clone), and the benchmark
only clones the repo copy-on-write and mounts deps read-only, so the
checkout is never mutated. GITNEXUS_BENCH_ORACLE_ROOT stays unset — it
defaults to the in-repo oracles dir and is staged by the harness.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Va5uu9Ar3e45QZ5xFsG4AZ
* fix(ci): harden promotion summary output and PR branch recovery
Three fixes to the promotion-detection and PR-open steps:
- GITHUB_OUTPUT summary used a fixed `PROMOTION_EOF` heredoc delimiter; a
value containing that marker on its own line could close the block early
and inject output keys. Use a per-run random delimiter, matching the
pattern already in tree-sitter-upgrade-readiness.yml.
- The summary concatenated every generation's promotion.json (including
rejected ones), so the PR body could show a losing generation's
decisions. The loop returns on the first promotion, so emit only the
highest-numbered gen-N/bench/promotion.json — the decision that fired.
- The promotion branch name omitted the run attempt. GITHUB_RUN_ID is
stable across re-runs, so a re-run after push-succeeds/PR-create-fails
could never push. Include ${GITHUB_RUN_ATTEMPT} (the artifact name
already does).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Va5uu9Ar3e45QZ5xFsG4AZ
* fix(ci): least-privilege the promotion App token and gate on an Environment
The Mint-App-Token step passed only app-id + private-key, so the minted
token inherited every permission the Release App installation holds
(including Workflows: write) — far more than "push a branch, open a PR".
Switch to `client-id` (as publish.yml does) and request only
permission-contents: write + permission-pull-requests: write.
Bind the job to a protected Environment (gitnexus-evolution) so promotion
runs can be gated server-side. workflow_dispatch runs the workflow and
in-tree evolve.py from the *dispatched ref*, so a code-side ref guard is
removable by the dispatched branch itself; an Environment deployment-branch
rule (main only) is the boundary that holds. The admin steps to create it
and scope the secrets are documented in the activation checklist.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Va5uu9Ar3e45QZ5xFsG4AZ
* fix(ci): correct upload-artifact pin comment and add shell strict-mode
- The upload-artifact SHA 043fb46d… is v7.0.1 (labeled so in the sibling
workflows that pin it); the comment mislabeled it # v6.0.0. Correct the
comment; the pin is unchanged.
- Add `set -euo pipefail` to the two build steps that lacked it, matching
every other run block in the file (GitHub's default shell already sets
-eo pipefail; this adds -u and consistency).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Va5uu9Ar3e45QZ5xFsG4AZ
* docs(ci): complete the skill-evolution activation checklist
- Add RELEASE_APP_ID / RELEASE_APP_PRIVATE_KEY to the required-secrets
checklist (the Mint step hard-fails without them on a promotion) and the
App-install-scope verification.
- Document the protected Environment admin step and why it is the real
boundary for the workflow_dispatch ref-secret exposure.
- Note that workflow_dispatch runs the billing loop regardless of
GITNEXUS_EVOLUTION_ENABLED.
- Justify the weekly cron against the README's ~90-day guidance and note the
355-minute timeout ceiling.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Va5uu9Ar3e45QZ5xFsG4AZ
* fix(eval): redact API tokens from diagnostic fields before artifact upload
results.jsonl (runner.py) and proposer-session.json (evolve.py) serialize
session records whose error_detail can carry a stderr_tail that echoed the
API key. Transcripts are redacted before persistence, but these two sinks
were not, and both land in the 14-day evolution artifact.
Run each record's serialized JSON through the existing redact_text with the
run's auth token before writing. Scoped to these diagnostic sinks only: the
promoted overlay and proposal.md are left untouched (the overlay is the
applied artifact and must stay byte-identical for apply and the
shipped-skills-sync guard).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Va5uu9Ar3e45QZ5xFsG4AZ
* test(ci): add a contract test for the skill-evolution workflow
No test exercised this workflow's path, which is why both P1 blockers
(missing --apply, unresolvable ~/GitNexus task repo) reached production.
Parse the workflow YAML and assert the structural contract: --apply is
passed, the task repo is provisioned, the promotion branch carries the run
attempt, the App token is permission-scoped and the job is Environment-
gated, the output summary uses a random delimiter and a single generation,
the artifact pin is labelled correctly, and every multi-line shell step
sets strict mode. Follows the review-agent-workflow.test.ts precedent.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Va5uu9Ar3e45QZ5xFsG4AZ
* feat(ci): run the proposer on its own (stronger) model
One `model` input drove both the benchmark arms and the proposer/diagnosis
session. Split them: `model` stays the benchmark arms (match the model your
skill users run, so a promotion is valid for them and the tasks aren't
ceiling-saturated), and a new `proposer_model` input runs the proposer —
the harder meta-reasoning task that writes the candidate skill, and only one
session per generation, so a stronger model is cheap here. evolve.py already
supports --proposer-model; the workflow just didn't expose it.
Defaults: arms = claude-sonnet-5, proposer = claude-opus-4-8 (both
overridable via workflow_dispatch). The weekly cadence bounds the added
spend. Contract test asserts the split stays wired.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Va5uu9Ar3e45QZ5xFsG4AZ
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
