Two corrections to the health-classification patch.
The regression I wrote could not have proved what it claimed. A fixture of 41
empty artifacts aborts through the outage breaker long before finalization:
review-evidence-invalid is in SYSTEMIC_ERROR_KINDS and the limit is 5, so it
trips at cell 5 through the pre-existing path. It demonstrated failure
detection, not the new guard. The decisive test now uses ONE fresh unusable
cell, asserts the streak stays under the breaker threshold, and only then
requires finalization to abort - leaving the new check as the only thing that
can catch it. Removing the call makes that test fail; restoring it passes.
The accurate defect statement is narrower than the last message claimed. Review
arms were excluded from the final incumbent-health check while the consecutive-
failure breaker gave them separate, partial coverage. They were not unguarded.
Second: "one admissible cell plus two execution failures" was asserted as
healthy. That converts "not wholly unusable" into "ran reliably", which is how
a partly-broken sweep passes review. Arms now report UNKNOWN, OBSERVED_OK,
DEGRADED or UNUSABLE. Only UNUSABLE is fatal, so eligibility and promotion are
untouched - this changes what is reported, not what is allowed.
The guard is extracted as enforce_measurement_health so it can be driven
directly, and it now reports a status line per arm. It names no cause: an empty
artifact establishes that evidence is unusable, not that a mount rejected the
write, so it prints cause=undetermined rather than guessing EROFS. It still
runs after report.md and promotion.json are written, so a failing sweep leaves
its evidence behind.
ce_review is named explicitly at the call site. It is a comparator rather than
a candidate, so it is absent from CANDIDATE_ARMS.values(), and dropping the
review exclusion alone would have left it unclassified.
The wiring test reads _run_sweep's compiled code object for the referenced
global rather than matching source text. It is honest about its limit: it
proves the call exists and would catch its removal, but no test here drives
_run_sweep end to end, which needs bwrap and a sandbox.
broken_incumbent_arms is marked LEGACY and NON-AUTHORITATIVE with removal
tracked. It has no production caller.
608 eval tests pass, ruff clean. The two test_model_gateway.py failures are
test_locked_litellm_translates_messages_to_offline_responses and
test_openai_gateway_never_leaves_proxy_output_on_an_undrained_pipe; both fail
identically on origin/main in this environment, checked directly rather than
carried forward as an inherited label.
broken_incumbent_arms infers "the environment is broken" from an arm resolving
zero tasks. That inference does not hold: a reviewer can be wrong about every
task in a hard corpus while every process, mount and capture worked perfectly.
Actions run 33962002890 is exactly that shape - 51 cells, all resolved=False
with error_kind=oracle-failed, median score 0.212, and a healthy harness.
Someone already knew this, and patched it by excluding review arms at the call
site. That leaves the unsound inference in place for workflow and
workflow_direct, and leaves review arms with no health check at all - so the
run that genuinely was broken, 33912693948, where the mount made an atomic
write impossible and all 41 artifacts came back empty, could not have been
caught here either.
So this replaces the inference rather than adding another exemption. aggregate
now classifies fresh rows into execution failures (the process or its tooling
did not complete), evidence failures (it completed but produced nothing
trustworthy or scoreable), and admissible measurements. An arm is unhealthy
only when it has fresh attempts, zero admissible measurements, and at least one
execution or evidence failure. Resolution count is no longer consulted. Arms
with only reused rows report current health as UNKNOWN rather than good.
With the inference corrected, review arms are checked again, which is what lets
the empty-artifact case be caught at all.
Deliberately unchanged: comparator reuse eligibility, quality denominators,
promotion thresholds, model settings, skill prompts and scheduler behaviour.
Failures that stop being called infrastructure failures still surface in the
counts and reasons - an agent-originated failure must not vanish from reporting
because it was reclassified. broken_incumbent_arms and its tests are left in
place; deleting behaviour belongs in its own change.
Seven regression tests, built from both runs' shapes and labelled as
reconstructed from logged observations, since 33962002890's results.jsonl did
not survive the instance shutdown. They pin: a badly-scoring reviewer is
healthy; an all-zero score is still a valid negative; empty artifacts are
unhealthy; one admissible cell keeps an arm healthy while its failures stay
visible; reused rows alone leave health unknown; reused successes do not mask
fresh failures; and a parseable artifact does not excuse a failed session.
602 eval tests pass, ruff clean.
sweep_task_cells finishes one task before starting the next and drains a wave
before refilling it, so a task with fewer cells than workers leaves workers
idle and one slow cell stalls its whole wave. sweep_packed_cells feeds every
task's cells through a single pool instead. Measured against the review corpus
it is worth about 40% of a cold sweep, and it is the only change that moves a
seeded weekly run at all - there a task is three cells and a wave is never full.
The breaker keeps its exact meaning. Cells carry a total submission order
continued across task boundaries, a folder walks results in that order, and
consecutive systemic failures are counted there, so a doomed run aborts on the
same cell it would have under waves. Verified at three failure positions.
This commit also corrects a finding from the Phase 2 prototype. I claimed the
overrun bound was set by the worker count rather than the submission window,
and that packing therefore cost nothing in breaker fidelity. That was derived
from a window sweep that only ever injected failures at one position. Driving
the real function at other positions shows the halt flag does not bound overrun
at all: the folder walks in order, so a slow early cell lets workers race ahead
and the trip is detected after those cells have already paid. An unbounded
queue overran by 11 cells where waves overrun by 2.
So the window is load-bearing and the trade is real, measured at workers=3 with
failures injected at four positions:
window 3 -> -8% wall, overrun 2 (the wave scheduler's own bound)
window 6 -> -27% wall, overrun 4
window 12 -> -42% wall, overrun 9
window 54 -> -44% wall, overrun 11
Overrun is wasted paid sessions at roughly $70 each. The default multiplier is
2, keeping the worst case within twice the wave bound while taking most of the
gain; the curve is in the constant's comment so raising it is an informed
decision rather than a guess.
Not wired in yet: _run_sweep still calls sweep_task_cells per task. Moving the
per-task graph, trees tempdir and reuse binding out of that loop behind
await_ready is the larger and riskier half, and it belongs in its own change.
595 eval tests pass, ruff clean.
Every Phase 2 number so far came from sleeping threads, which contend for
nothing, against a duration sample measured at workers=1, which contains no
contention either. That was the standing caveat on the whole result, so this
measures it.
A cell now waits for its API share and then burns a fixed number of sha256
rounds in a subprocess. Work-bounded rather than wall-clock bounded, so it takes
longer when cores are busy - that is the effect under test. A subprocess because
Python threads burning Python would measure the GIL rather than the machine.
Calibrated at 519k rounds/s, stable within 2% across three probes.
The first run of this was worthless and is recorded as such: on a 24-core host
with 3 to 6 workers nothing ever contends, since cpu_fraction 0.5 at 6 workers
is about 3 cores of demand out of 24. It measured an absence. Re-run pinned with
taskset to 4 and 2 cores.
The packing advantage survives. It holds between -40% and -47% across every host
size and CPU fraction tested, including a genuinely oversubscribed 2-core box at
cpu_fraction 0.5 with 6 workers.
But contention erodes packing more than it erodes waves, for a structural
reason: packing is what creates the concurrency. Moving from 24 cores to 2 at
cpu 0.5 and 6 workers, the faithful scheduler slows 13% while the wave slows
3.7%, and the gain narrows from 45.0% to 39.8%. Packing and a higher worker
count are therefore not independent wins - packing spends the contention
headroom first, so raising workers has to be re-argued after it lands rather
than added to it.
Three things this still does not measure, and they bound the result. The real
CPU fraction of a benchmark cell is a guess informed by roughly 180 tool calls
per session; nobody has profiled one. The evolution runner's core count decides
which column applies and is unknown here. And the burn is sha256, pure CPU,
while real cells run vitest and analyze, which are memory and IO heavy - so this
is a floor on contention, not a ceiling.
591 eval tests pass, ruff clean.
The first packed prototype was a bare ThreadPoolExecutor. It reported -43% and
none of the invariants the shipped scheduler holds, so it priced an idea nobody
could ship. This one carries them: a global submission order continued across
task boundaries, in-order folding, the real outage breaker, and per-task graph
readiness gating behind a serial builder.
The fidelity check first reported the two schedulers tripping on different
cells, 17 against 16. That was my instrumentation, not a divergence -
sweep_task_cells folds an entire wave before it evaluates the breaker, so the
last cell folded is not the cell that tripped. With the harness mirroring the
breaker's own evaluation the two agree exactly, across failures starting at
cell 0, 4 and 12, with overrun inside the workers-1 bound the wave docstring
promises.
Two results worth the exercise.
Head-of-line blocking, not the barrier, is what a naive in-order design pays.
Holding submission to `workers` cells beyond the fold pointer leaves the
faithful scheduler at -8.1% cold and -2.7% weekly: one slow cell stalls the
pointer, the window cannot slide, and it reproduces the wave almost exactly.
That is the number to quote if anyone proposes the obvious implementation.
But the overrun bound turns out to be set by the worker count, not the window.
Only `workers` cells can be running when the breaker trips; everything queued
behind them short-circuits on the halt flag. Overrun is 3 at an unbounded
window exactly as at 6, and the trip cell never moves off 16. So H2 does not
have to trade breaker fidelity for speed - a wide window takes -42% with the
semantics intact. The tension I assumed was there is not, and window=12 already
captures 97% of it.
Still an upper bound: sleeping threads do not contend, and the sample was
measured at workers=1. What this establishes is that the invariants are
affordable, which was the thing blocking H2. Not built here: the trees tempdir
lifecycle, reuse-row binding, and the cancel_event path.
591 eval tests pass, ruff clean.
measure_evolution_cost predicts wall clock from a model of what
sweep_task_cells does. This runs the real thing - real threads, the real wave
barrier, the real outage breaker - with only the paid agent session replaced by
a sleep, and times it.
Durations are the measured per-arm samples divided by 5000, so a 1416s cell
takes ~0.28s. The shape is kept on purpose: the median cell is 826s against a
5400s ceiling, and that spread is the entire reason a barrier costs anything.
Uniform random sleeps would erase the effect under test. All schedulers consume
one identical seeded plan, so a comparison cannot be an artifact of one of them
drawing luckier cells.
The model survives contact: it tracks real execution within about 10%, and
workers=1 - which runs without a pool at all - sits at 0.95, so the residual
above 1.0 at higher worker counts is per-wave thread overhead rather than a
modelling error. Two structural claims that were arithmetic are now observed.
Weekly is flat from workers=3: 3.59, 3.59, 3.59, 3.60, 3.59, 3.59 across w=3..8.
workers=4 buys nothing over workers=3 on cold, 7.68 against 7.78.
Two prototype schedulers are measured beside it, deliberately before any
production code exists. A continuously fed pool per task is worth more than the
model claimed on cold, -27.3% against a predicted -17.9%, and exactly nothing on
weekly, +0.0%, because a weekly task is one wave with nothing to feed. One pool
across all tasks beats both: -40.7% weekly and -42.9% cold at workers=3, rising
to -65.7% and -63.9% at workers=8. It also subsumes the fed pool, since packing
across tasks is a fed pool.
That reorders the backlog. Cross-task packing moves from second to first: it
dominates on both profiles, and it is the only thing that moves weekly at all.
Raising the worker count is worth nothing until it lands - under the barrier
weekly does not improve from w=3 to w=8, and speedup against serial is 1.58x for
three workers and only 2.40x for eight.
The bound on all of it: sleeping threads do not contend. Real sandboxed sessions
compete for CPU, page cache and disk, and the duration sample was itself
measured at workers=1, so it carries no contention either. These speedups are
upper bounds. The ordering is trustworthy because the schedulers were compared
under identical conditions; the magnitudes are not. The packed prototype is also
a bare ThreadPoolExecutor with no breaker folding, no per-task graph lifecycle
and no reuse binding - which is the actual cost of building it, and is not
measured here.
The simplification pass left one finding open: comparator_reuse and
proposer_sandbox both defined `_real_directory`, same name and same shape, with
different guarantees. The sandbox one rejects every symlink hop in the path; the
reuse one checks only the leaf and resolves through parents. Sharing the name
invites a consolidation that would silently tighten one of them.
They should not be merged, so the name stops claiming they could be.
proposer_sandbox guards a mount root, where a symlink hop changes what an
untrusted session is handed. comparator_reuse guards a data directory whose
contents are already validated one file at a time - reads go through
_regular_file, which lstats and rejects symlinks, and writes through O_NOFOLLOW.
A symlinked parent therefore grants nothing those guards do not already cover,
while refusing one would reject a symlinked artifacts directory or macOS's /var
for no gain.
Renamed to _resolved_directory, with the reasoning recorded at the definition,
and a test that pins both halves: a symlinked parent is accepted and resolved, a
symlinked leaf is still refused. Behavior is unchanged.
591 eval tests pass, ruff clean. The two test_model_gateway.py failures are
pre-existing and fail on main.
Simplification pass over the branch. Behavior-preserving throughout; three
reviewers, nine findings applied, two skipped.
The review-artifact block in _run_hidden_oracle was unreachable. That function
runs only in run_arm's non-review branch, while the directory it probes for is
created only in the review branch, and each sandbox serves exactly one arm - so
`review_artifact.parent.is_dir()` could never be true. It was added an hour
earlier to make the hidden oracle resolve the moved artifact; the oracle never
runs for review tasks, so the guard was dead on arrival. Deleting it also
removes the duplication it had with the verify-command wiring.
EXCLUDED_ERROR_KINDS is now one definition. runner.py and comparator_reuse.py
each carried the same six-member frozenset, kept in sync by a comment. Only one
direction is possible: runner already imports from comparator_reuse, so the
reverse import fails at module-init with a circular-import error. That is now
stated where the alias lives, so nobody tries it the other way.
ensure_task_graph and prefetch_next_graph shared ten keyword parameters, passed
through two call sites and forwarded whole between them. They now take a
GraphBuildEnv, mirroring TaskCellContext, which already bundles per-cell state
in this file. Its ready_keys() replaces an inline four-set union at the call
site.
Smaller consolidations: _sha256_file's hand-rolled chunk loop becomes
hashlib.file_digest (3.11+, already used in runner_artifacts); _copy_owner_only
reuses task_assets._write_all and COPY_CHUNK_BYTES instead of repeating the
short-write retry; its stat-then-open existence check becomes the O_EXCL failure
it was already relying on, which is atomic rather than merely narrow; and
runner_environment reads the digest through comparator_reuse.current_runtime_digest
instead of re-parsing the environment variable.
Three test docstrings summarised the branch's own history ("the branch's core
speedup", "the regression that produced fifteen runs") rather than the invariant
under test. Rewritten to state the constraint, which is what survives the merge.
Repaired the indentation left behind by the outage-streak edit and flattened the
prefetch dispatch from three nested conditionals to one.
Skipped: consolidating comparator_reuse._real_directory onto proposer_sandbox's
same-named helper - they differ, the sandbox one rejects any symlink in the
resolved path while this one checks only the leaf, so sharing it would tighten
behavior rather than preserve it. That needs a decision about which policy the
reuse path wants, not a simplification.
589 eval tests pass, ruff clean, 29 workflow contract tests pass. Unrelated and
pre-existing: two test_model_gateway.py failures, and
test_process_control.py::test_timeout_kills_term_ignoring_descendants_before_they_write,
which is a TERM-to-KILL timing flake (passes 2 of 3 in isolation) in a file this
branch does not touch.
Applies the five findings the previous review round left open.
Comparator reuse ignored the environment a row was measured in. TaskReuseBinding
carried the task and oracle identity but not the task-asset or sandbox-dependency
digests, and this branch itself changes sandbox_dependencies in the review
corpus - so a reused comparator could be measured against one dependency set and
compared against a candidate built on another, handing the gate a false
comparison. Closing it needed the digests to exist before the reuse decision, so
asset snapshots are now prepared for every task up front instead of lazily
inside the per-task loop. That also removes the concurrent TaskAssetCache.prepare
the prefetch thread could otherwise race, which the file's own "plain dict,
read-then-write race" comment warned about. Both digests fail closed on either
side, matching the runtime digest.
The broken-incumbent canary could not fire when reuse was working. It read
`resolved`, which counts reused rows, so an arm whose cells were all reused
always looked healthy - in precisely the run where a broken environment would go
unnoticed. aggregate now also reports `resolved_fresh` and the canary reads it.
That count would be vacuous if an arm were reused end to end, so the sweep keeps
one paid cell per incumbent arm and says which one it kept.
Reused rows did not participate in the outage streak, so a run of failures could
carry across them and trip on stale history. A reused success now resets the
streak the way a paid success does.
The cost model charged sweep overhead per cell, which credited a weekly
generation for shrinking work it still performs: it pays one arm instead of
three but builds exactly the same graphs. Overhead is charged per SHA now.
Weekly is 5.20h rather than the 4.80h the per-cell rate reported; cold is
10.86h. The residual still cannot be split between per-SHA and per-cell work
from one artifact, so session_durations.json records that assumption and the
direction it errs in, rather than leaving a number nobody can trace.
Clone-template reuse - the branch's core speedup, taken on essentially every
multi-cell sweep - now has a test that builds a real sanitized template, asserts
the cell runs against the copy with the template's HEAD, and fails if run_cell
re-clones. A second test asserting only on a namespace built inside the test was
written and deleted: it exercised nothing, which is the failure this review
round penalised elsewhere.
589 eval tests pass, ruff clean, 29 workflow contract tests pass. The two
test_model_gateway.py failures are pre-existing and fail on main.
Code review of this branch found the relocated review artifact was fixed in the
bwrap mount and nowhere else. Four independent layers decide whether the agent
can write it, and three still named the old location.
Claude Code applies its own filesystem policy to its own tools, and
build_claude_settings listed only /workspace, /tmp and /home/agent under
allowWrite with denyRead ["/"]. The artifact used to live under /workspace, so
this list was correct until it moved. SANDBOX_REVIEW_OUTPUT is now in allowWrite
and allowRead; without it the bwrap bind grants a write the CLI then refuses.
The task corpus still ran `test -s review-output.json` from the workspace, in a
separate sandbox invocation that never sees the artifact mount. Every review
cell would have been stamped verify-failed with resolved=False no matter how
good the review was, which also made those rows permanently unreusable and so
silently disabled this branch's own comparator reuse for review arms. The verify
and hidden-oracle commands now read the location from
GITNEXUS_BENCH_REVIEW_OUTPUT and get the directory bound read-only, mirroring
the mount-plus-env-var shape _run_hidden_oracle already used.
host_text and host_path did not translate the new path, so the host-unsafe
backend told the agent to write somewhere that exists on neither backend.
Adding the mapping exposed a second defect: host_text substituted every
occurrence of a target, and "/review-output" appears twice in
"/review-output/review-output.json" - once as the directory and once inside the
filename. Matching is now anchored to a path boundary.
Comparator reuse had three ways to accept evidence it should have rejected. A
row with no runtime_digest passed the drift lock because the guard only compared
when both sides were bound, and the branch's own test asserted that as correct;
absence is now a mismatch and the test states the rule. materialize_reused_row
overwrote recorded_at with the copy time while the age check read that field, so
a row copied forward each generation refreshed its own clock and never aged out;
the first measurement time is now preserved and aged against. A future-dated
stamp passed a one-sided bound and is now rejected as corrupt.
RUNTIME_DIGEST never reached the runner at all: runner_environment builds a
fixed dict and process_control replaces the child environment wholesale, so the
digest the workflow exports was dropped and the lock it feeds was inert. The
instance-window deadline was also checked only after run_proposer returned,
buying a proposal the generation had no room to benchmark.
The graph prefetch thread was started without copy_context, so it never saw the
cancellation ContextVar the rest of the sweep shares, and the outage breaker
returned without setting cancel_event - together, a tripped breaker would block
on joining a prefetch that was never told to stop. Both fixed, with outage
checked before cancellation at the two exits so an outage keeps exit 1 instead
of becoming a Ctrl-C's 130.
Both bwrap canaries that actually execute a write still bound the pre-fix shape
against a file this branch no longer creates, so they would have errored rather
than caught anything. They now bind the directory and write atomically - temp
file beside the target, then rename - which is the exact operation that failed
with EROFS. A source-text assertion over inspect.getsource(run_arm) was replaced
with one that inspects the real mount, and a wall-clock assertion was pinned to
a fixed monotonic clock.
Not applied, and why: binding task-asset and dependency digests into comparator
reuse needs asset snapshots prepared before the reuse decision rather than
inside the per-task loop, and shipping the comparison without that would add a
guard that silently never fires. Forcing a paid canary cell per incumbent arm
and folding reused rows into the outage streak are behaviour decisions, not
fixes. Clone-template reuse still has no test. The cost model's per-cell
residual still shrinks with arm count, overstating weekly savings by at most the
2541s residual; the docstring now says so rather than inventing a split.
585 eval tests pass, ruff clean, 29 workflow contract tests pass. The two
test_model_gateway.py failures are pre-existing and fail on main.
Every review cell in the last recorded generation returned unusable evidence.
Not some — all 41, across all three arms and all six tasks, at $3653 for the
run. The transcripts say why, 127 times across 35 of 35 sessions:
EROFS: read-only file system,
open '/workspace/review-output.json.tmp.2.90a76e583b0c'
The review arm mounted the artifact as a writable FILE at
/workspace/review-output.json while binding /workspace read-only. The Write
tool writes atomically: it creates `<target>.tmp.<n>.<hex>` beside the target
and renames it. The parent was read-only, so the temp create failed and the
artifact was never written. A writable file inside a read-only directory is
not writable to anything that writes atomically. Agents tried
/proc/self/root/workspace/... and /proc/1/root/workspace/... to get around it;
all 41 artifacts came back 0 bytes.
The artifact now lives in its own writable directory bound at /review-output,
outside the workspace. That is what a rename needs, and it lets the workspace
get stricter rather than looser: the review phase may now change nothing there
at all (enforce_phase_workspace gained allowed_artifact=None), where before it
was entitled to one path inside it. The file is no longer pre-created — the
agent writes it, and absence is now meaningful evidence.
parse_review_output reported every one of these as "review output is not valid
UTF-8 JSON". The file was empty, and its except folded OSError, UnicodeError
and JSONDecodeError into that one string, so a sandbox that made writing
impossible was indistinguishable from an encoding fault. That is why this read
as an agent-quality problem for fifteen consecutive non-green runs. Each cause
now names itself: never written, empty, not valid UTF-8, not valid JSON with
the decoder's position. run_arm also keeps the FIRST error_detail, as it
already did for error_kind, so a phase-boundary violation is no longer buried
under the parse failure it causes.
The test double conflated sandbox.private_root with the clone, which put the
artifact directory inside the workspace and would have hidden the stricter
check. Regression tests pin the mount shape in the generated bwrap argv, the
contract path in the prompt, the four parse diagnostics, and the
untouched-workspace contract.
Verified by unit tests only: this container has unprivileged user namespaces
disabled, so bwrap cannot run here and the mount was not exercised end to end.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three defects, found by auditing the model against the artifact again.
The overhead was charged inside the schedule. session_durations.json claimed
the residual was charged "per cell and serially - the pessimistic reading",
but task_cells folded it into each cell's duration, where the pool then
divided it by the worker count. The residual mixes per-cell work the pool
really does divide with per-SHA graph setup it cannot, and the artifact cannot
separate them, so it now sits outside the schedule: cold 11.09h, not 10.40h.
Alignment averaging weighted the shortest sample twice. The arm samples are 13,
14 and 14 long and the average ran over max()=14 offsets, so candidate_review's
first cell was counted twice and its last never. Averaging over lcm()=182
offsets weights every arm's sample evenly.
The wall assumed all 54 cells run. Replaying the sample's own error_kind
sequence through today's systemic_outage_streak trips the outage breaker at
cell 5 of 41. The source run executed all 41, so its runner did not break on
that sequence, but the current one would: these numbers price a HEALTHY sweep,
and a sweep with the sample's failure profile never reaches them. Stated on
generation_seconds and recorded next to the sample it qualifies.
Measurement only; no runtime behaviour changes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two errors in the model, both found by auditing it against the artifact it
claims to describe.
The arms are not interchangeable. `candidate_review` runs 1416s at the mean
against `review`'s 1204s and `ce_review`'s 1176s, and the weekly lane pays the
candidate arm and nothing else — reuse skips both incumbents. Pricing weekly
from a pooled sample charged it for arms it never runs: weekly is 4.59h, not
the 3.65h a pooled sample reported. Cells are also submitted run-major and
arm-minor, so at workers=3 every wave holds one cell of each arm and the
slowest arm sets the wave; the model now builds cells in that order.
The setup constants were invented. GRAPH_ANALYZE_SECONDS=600 and
TEMPLATE_SANITIZE_SECONDS=180 charged 3900s of per-SHA setup for a cold run —
more than the entire non-session time of the source run, which was 2541s for
41 cells and 5 SHAs. `duration_s` is the sum of a cell's Claude sessions
(runner_sessions.py), so that 2541s residual is every clone, graph build,
sandbox and teardown the sweep paid. The model now charges the measured
residual per cell, 62.0s, and no longer credits clone templates or graph
prefetch: both landed after that run and there is no measurement of them yet.
The residual bounds what they can be worth.
Cold 37452s (10.40h), weekly 16541s (4.59h), against a fed pool at 31683s and
16541s. Measurement only; no runtime behaviour changes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The cost model assumed every cell runs the 1140s mean. Cells are not uniform:
the 41 rows in Actions run 33912693948's artifact are 826s at the median,
1262s at the mean, 2976s at p90, with two pinned at the 5400s session ceiling.
A wave waits for its slowest cell, so a mean understates every concurrent
schedule — the previous model called workers=3 cold 5.99h when the same
schedule against real durations is 10.33h.
session_durations.json carries the sample in submission order with its
provenance and its caveat: every cell in that run returned unusable evidence,
so the durations are real but a clean run may sit lower. It is the only live
artifact; the 2026-07-22 green run's has expired.
The model now simulates the schedule cell by cell rather than multiplying a
mean by a wave count, averaged over all 41 rotations of the sample so no
single alignment between sample order and cell index decides the answer. It
prices today's barrier (wave_makespan) against a continuously fed pool
(fed_makespan) and reports both, and it charges the proposer session — one
per generation, measured at 344.7s — which it had been omitting entirely.
Measurement only; no runtime behaviour changes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The evolution sweep paid `sanitize` + `analyze --pdg --index-only` for every
unique task SHA on the critical path, one at a time, with nothing overlapping.
`_run_sweep` now starts the next unpaid SHA's clone template and graph snapshot
on a prefetch thread as soon as the current task's cells are dispatched, so
every SHA but the first hides behind a paid session wave. The thread is joined
before that SHA is used and before the trees tempdir is torn down, and a
prefetch failure is recorded against the SHA exactly as an inline failure is.
Tasks whose cells are all reusable comparator rows are not prefetched: they
never build a graph, so priming one would be pure cost.
Adds `measure_evolution_cost.py`, the cost model behind these numbers. It reads
the review corpus, the evolve defaults, and the workflow's workers default —
it does not start a session. Its first version charged `copy_isolated_tree`
once per paid cell, serially. `run_cell` clones inside its own pool worker, so
the clones in a wave overlap and only one is on the critical path per wave;
the model now charges `ceil(cells / workers)` waves.
Estimated review generation at workers=3: cold 21570s, weekly 7710s.
Wall clock is quantised by `ceil(cells_per_task / workers)`. A cold review task
is 9 cells, so workers=4 buys the wall clock of workers=3 and pays host
contention for it. Documented in the workflow's rollout checklist.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Reuse matching incumbent/CE cells, sanitize each SHA once, and default
dispatch workers to 3 so weekly review generations finish inside the
EventBridge window. Cap the sweep from leftover instance uptime so a
Friday dispatch still uploads evidence.
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* docs(parse): record why dispatchGroups is a required interface member
Review finding #10 argued dispatchGroups should be optional to match
`getQuarantinedPaths?` / `getStats?`. Those are compatibility accommodation
for WorkerPool shapes that predate them, not a convention for new members;
optional here would force a `?.` plus an unreachable fallback at the single
production call site. Documenting the decision so the next reader does not
re-litigate it from the neighbouring optional markers.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit addaab647377f3c4553f752fa3ca1388bcb9ca81)
* refactor(parse): simplify round accounting and dispatch setup
Simplification pass over the dispatch-rounds change. Behavior preserved:
identical graph on a full analyze (51,286 nodes / 163,092 edges).
- Drop `roundMissBytes`. `roundBufferedBytes` counts the same bytes plus the
cache hits, so it is always the greater of the two and the first disjunct of
the close condition could never fire on its own. One counter, one reset, one
check.
- Measure round bytes with `Buffer.byteLength(content, 'utf8')` instead of
`String.length`. UTF-16 code units undercount non-ASCII source by up to 3x,
so the cap meant to bound main-thread retention was letting a CJK-heavy repo
hold well past its nominal budget. Matches `estimateItemBytes` in the pool.
- Reset the durable ParsedFile directories for a round's chunks concurrently.
Each targets its own chunk-hash directory, and running them serially put N
round trips of fs work on the critical path the round exists to shorten.
The try/catch stays inside the mapped callback, so one failure still
degrades that chunk alone.
- Skip the quarantine filter entirely when nothing is quarantined, which is
every run without a worker death. It was an identity copy of every group.
- `dispatchChunkParseRound` takes `DispatchGroup<...>` rather than re-declaring
that shape inline; the type was already imported and used in its body.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 527d5b6e0ca8ae7bbc6a414c5ac7e27fd85e9995)
* refactor(parse): count round misses with the same idiom startRound uses
`drainRound` hand-rolled a reduce to count 'miss' entries while `startRound`,
one function above, filters the same predicate over the same union. Same
integer, one idiom.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 7eaa193b0cb5fa515844f36ae1401d6fb2fed7b8)
* fix(parse): honor GITNEXUS_WORKER_POOL_SIZE above the auto sizing cap
The auto pool size is bounded by source bytes so a tiny repo does not spawn a
full idle pool. That bound was also clamping the operator's env override,
because the env value is read inside `resolveAutoPoolSize()` and the result
went through `Math.min(..., workProportionalCap)`.
`DEFAULT_POOL_SIZE_CAP`'s own comment offers `GITNEXUS_WORKER_POOL_SIZE` and
`--workers <N>` as equivalent escape hatches for operators on bigger machines.
They were not. Measured on a 30MB corpus, where the byte-derived cap is 16:
--workers 24 -> pool: 24/24 active
GITNEXUS_WORKER_POOL_SIZE=24 -> pool: 16/16 active (silently ignored)
Both are deliberate operator input, so both now bypass the work-proportional
cap, which goes back to bounding only the auto default. After the fix, on the
same corpus, with identical graph output (51,286 nodes / 163,092 edges):
GITNEXUS_WORKER_POOL_SIZE=24 -> pool: 24/24 active
GITNEXUS_WORKER_POOL_SIZE=4 -> pool: 4/4 active
unset -> pool: 16/16 active
Verified by hand against the pool's own throughput log; not covered by an
automated regression test, since the pool size is only observable through
that log line and not through the progress stream a test can read.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 17ed08608c878079b2927da25cfd39c1608a02a2)
* fix(parse): bound the durable-reset fan-out and pin the pool-size override
Review follow-ups on #3200.
The round's durable ParsedFile directory resets went out as one unbounded
`Promise.all` — one recursive rm + mkdir per miss chunk, all at once. A round
can hold hundreds of small packs, and those resets compete for descriptors with
the chunk prefetch this loop already has in flight. `readFileContents` degrades
a losing read SILENTLY by documented contract, so a dropped file would vanish
from the chunk, from the graph, and from the chunk hash — shipping a narrowed
index with exit 0. Now routed through `mapConcurrent` at the same width the file
reads use, which keeps the pipelining win and caps in-flight descriptors.
An operator's pool size is now also bounded by the number of files there are to
parse, so `GITNEXUS_WORKER_POOL_SIZE=100000` on a five-file repo cannot become
the literal thread count. This applies to `--workers` and the env var alike, so
the parity the previous commit established is intact. It does NOT shrink an
incremental re-analyze: `totalParseable` counts every parseable file in the
scan, not the changed ones.
Adds the regression test a reviewer asked for. The existing coverage
(`worker-pool-resilience` calling `resolveAutoPoolSize` directly,
`analyze-worker-pool-size` mocking `runFullAnalysis`) never reaches
`runChunkedParseAndResolve`'s `effectivePoolSize`, so both stayed green through
a revert of the fix. The new test drives the real parse phase with a worker
double that writes a per-`threadId` marker, and counts them: verified it fails
on the reverted line with `expected [ 'worker-1' ] to have a length of 3 but
got 1`, and passes on HEAD.
Also corrects the `GITNEXUS_PARSE_ROUND_BYTES` docstring, which still described
the cache-miss counter deleted two commits ago.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(parse): skip caching a chunk with a stale durable generation; warn on over-subscription
Closes the two findings left open by the review of #3200.
When `prepareDurableParsedFileChunk` fails, the previous generation's shards
are still on disk, so a later warm hit would union them with the new ones. The
chunk is now recorded and its parse-cache write skipped -- the same posture
`finalizeWorkerChunk` already takes for a quarantined chunk, and for the same
reason: do not cache what we cannot vouch for. The next run re-dispatches into
a directory it can actually clear. Bounding the reset fan-out removed the
correlated trigger; this closes the individual case.
Pool size over-subscription now warns rather than caps. Silently capping is
precisely what the override exists to prevent, so an operator's number is still
honored -- but an exported GITNEXUS_WORKER_POOL_SIZE applies to every analyze
in a long-lived caller (watch auto-sync, the MCP server), including small
incremental ones, and that is easy to set once and forget. The warning names
the host's usable core count, so it is a hardware fact rather than an invented
threshold. `resolveHostParallelism` is extracted from `resolveAutoPoolSize`
rather than re-deriving the cgroup-aware fallback at the new call site.
Tests: the stale-generation skip is pinned by a new case asserting nothing is
written under any key; verified it fails without the guard with
`expected 1 to be +0`. 60 unit and 49 integration tests pass across the
affected suites.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* test(parse): guard dispatch-round cadence with a bench, not a wall-clock budget
Round boundaries are deliberately invisible to graph output — batching that
changed output would be a bug — so nothing in the repo could see the #3196 win
regress. It would have come back as a silent ~1.5x on every cold analyze. Two
earlier attempts to pin it as a unit test failed for that exact reason: one
scraped a logger line the progress stream does not carry, the other asserted
graph content that is identical either way.
Extracts the round-close fold into `createRoundBudget`, so the decision is a
shared unit the bench measures rather than a copy that drifts. The parse loop
is streaming and cannot know chunk sizes up front, so an accumulator is the
honest shape — not a planner.
Four deterministic arms, one ratio, no millisecond gate:
- layout_fingerprint — pack membership. Every cache key derives from it, so
drift needs a SCHEMA_BUMP, never a lone re-baseline.
- packs / single_file_packs — the FLOOR. `rounds` only asserts something while
the corpus over-splits (774 packs where the byte budget needs 5). This is
bench/import-target's lesson, where four heap arms read 0 B and passed every
ceiling: a ceiling says "not too big", nothing said "still measuring".
- rounds — the regression signal, both directions.
- cjk_rounds vs ascii_rounds — pins UTF-8 byte accounting. The two corpora
share a UTF-16 length and differ only in encoded size, so String.length
collapses them to equal. This is the arm no unit test could be.
- pack_scaling_ratio — (t_4n/t_n)/4, min-of-15. A ratio because wall-clock is
runner-speed-dependent and this repo has the scar: callable-value-flow's ms
gate failed twice at 2.07 and 1.975 against 1.9 with correct code, on a
sub-11ms measurement.
Every arm verified to fail before being recorded: close-every-chunk reads 774
rounds, disabling the close reads 1, reverting roundFileBytes to String.length
takes cjk_rounds 8 -> 3, and shrinking the corpus trips the shape floor.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* docs(bench): record the analyze phase breakdown and the rejected optimizations
Where analyze time actually goes, measured while landing #3194/#3196/#3200,
plus the two optimizations that looked compelling and were measured away.
The headline is that the parse work is done: a one-file-edit re-analyze is
36.5s, of which parse is 2.8s (8%). scopeResolution is 40% and the unlogged
graph emit + FTS rebuild is 49% — neither is incremental, and the ~18s sits
outside the phase runner so every phase log is blind to it.
Also records the trap that invalidated an earlier measurement: a non-git
corpus never records a schema fingerprint, so every run is a forced rebuild
and any "warm" number taken that way is fiction.
Rejected, with numbers: more workers (16/20/24 land inside run-to-run spread)
and bundling the worker entry (~250ms on a normal filesystem; the 8.6s that
motivated it was a 9p-mount artifact).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* perf(parse): batch cache packs into one dispatch round
`WorkerPool.dispatch` is a barrier, so dispatching one parse-cache pack at a
time leaves most slots idle for every round-trip. Packs are keyed by
`(language, hash(path) % 128)`, so the byte budget rarely binds: this repo
produces 1285 packs where the budget alone needs 16, and 549 of those hold a
single file. In a real analyze, 76 of 221 dispatched chunks carried one file
and cost 15.3s — 20% of the parse phase for 3.4% of the files.
Chunks now accumulate into a round bounded by `GITNEXUS_PARSE_ROUND_BYTES` of
cache-missing source (default: the chunk byte budget) and go out through a new
`WorkerPool.dispatchGroups`. Jobs are still cut at pack boundaries, so each job
carries exactly one `chunkHash` and every result stays attributable to the pack
whose cache key owns it. Cache hits ride along as round entries, and rounds
drain in `chunkIdx` order, so deferred aggregation stays deterministic.
Cold `analyze --index-only` on this repo (2234 parseable files, 16 workers):
110.3s -> 70.5s total, parse phase 74.0s -> 40.5s, 221 dispatches -> 15.
Graph output is unchanged: 51,286 nodes / 163,092 edges / 2106 clusters /
759 flows in both arms. Peak main-thread RSS 3372MB -> 3487MB (+3.4%).
`dispatchGroups` also claims the pool synchronously and rejects a concurrent
call. Two overlapping dispatches hand the same slots out twice and both stall;
the first version of this change did exactly that, and the only symptom was
every worker idle-timing out ~10s later with no indication of the cause.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(parse): bound what an open round holds, not just what it dispatches
Follow-up to the review of #3196. Three reviewers independently found the
same defect: `roundMissBytes` was the only in-loop close condition, but cache
HITS were queued into the same round without contributing to it. A warm
re-analyze misses nothing, so no round ever closed and every chunk's source
plus its cached worker output stayed resident until the tail drain — the
#2649 heap failure shape on a large repo.
- Hit entries now carry a file COUNT, not the file array, so a replayed chunk
never pins its source text. `applyChunkResults` only ever read `.length`.
- Track `roundBufferedBytes` across hits and misses and close on either cap.
Verified on a warm run: with the cap, draining starts as soon as 2MB is
buffered; without it all 221 merges land in the final 10% of the phase.
- Warm progress no longer freezes at the phase floor. `filesParsedSoFar` only
advances at drain, so a new `queuedFilesSoFar` feeds the progress events
while `filesParsedSoFar` stays the merge-accurate throughput number.
- A throw from `drainRound` used to unwind straight to `terminate()` while the
next round's workers were still busy — the #2432 mid-N-API abort hazard.
Settle the in-flight round first, then propagate.
- `dispatchGroups` returns one array per group; assert that length instead of
`?? []`, which turned a contract break into a silently empty chunk.
- Collapse `PendingWorkerChunk` into the `miss` RoundEntry it duplicated.
- Repair two stale doc comments: `dispatch`'s JSDoc had been orphaned onto
`dispatchGroups`, and `dispatchChunkParse` still described chunk overlap
that now lives in parse-impl's round machinery.
- New test: a round mixing a cache hit and a cache miss. `drainRound` walks
entries in chunkIdx order but pulls results on a separate cursor, and no
existing test put both kinds in one round with content assertions.
Cold analyze unchanged: 71.3s, 15 rounds, 51,286 nodes / 163,092 edges.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Stable cache packs introduced by 9718e1247 often fit one worker job,
leaving most workers idle. Size jobs against live pool capacity and
reuse extraction queries per native grammar instead of recompiling
on every pack. Preserve recovery readiness across dispatches and
bound failed-thread termination acknowledgment.
Bisect confirms the scheduling regression. Controlled parsing of 951
TypeScript files falls from 56.39s to 26.66s with identical graph output;
peak RSS increases roughly 10%. Add scheduling and query reuse coverage
and repair recovery fixtures that assumed single-job dispatch.
Validation: build, typecheck, formatting and 229 focused tests pass.
Full suite was interrupted during lengthy native DB testing; full
release CI remains outstanding.
Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
The oracle-mask overlay covered the same path review setup reads, so every historical cell died with can't-open-patch. Leave the staged copy visible for apply, then fail closed if it is still there when the model starts.
Co-authored-by: Cursor <cursoragent@cursor.com>
* chore: release v1.6.11
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(test): read /api/info version from package.json
server-info unit tests hardcoded 1.6.10 while buildServerInfo reads
package.json, so the 1.6.11 bump failed ubuntu coverage 3/3.
Co-authored-by: Cursor <cursoragent@cursor.com>
* refactor: read the published version from one helper
Release bumps kept breaking tests that each re-required package.json.
packageVersion() is now the single read for CLI, MCP, serve, and those tests.
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Seed the current gitnexus-review skill into older PR checkouts, force-add
historically gitignored skill paths, accept plugin-qualified Skill ids,
and lock host-unsafe workspaces to review-output.json so a generation can
finish and score. Sandbox cleanup restores owner write bits before delete
because a session that copytrees the locked clone otherwise leaves 0555
trees that rmtree cannot remove.
Co-authored-by: Cursor <cursoragent@cursor.com>
* feat(core): add cache-first npm update-check service
Shared fail-open checker: validated 24h cache under GITNEXUS_HOME,
acquireFileLock-guarded refresh, monotonic publication, hardened
registry fetch (no credentials, private-address redirects refused,
body-capped), strict x.y.z comparator, install-eligibility
classification, and an unref'd refresh scheduler for long-lived
processes. Extracts getGlobalDir into storage/global-dir.ts with a
repo-manager re-export (no caller changes).
Co-authored-by: Cursor <cursoragent@cursor.com>
* feat(cli): notify on available updates via stderr and doctor
One i18n'd stderr line on interactive invocations when the validated
cache holds a newer version (TTY-gated, CI/opt-out/eligibility-gated,
hook and help/version command identities excluded). Stale cache spawns
a detached hidden __update-check refresh child so command exit latency
is unchanged. doctor prints the cached latest version when known.
Dockerfile.cli sets GITNEXUS_NO_UPDATE_NOTIFIER=1.
Co-authored-by: Cursor <cursoragent@cursor.com>
* feat(mcp): emit one stderr update notice per process per version
Process-scoped adapter in mcpCommand (stdio and --http), dynamically
imported after the stdout sentinel, started only after connect, fully
catch-isolated. Arms the shared refresh scheduler with cleanup on
process exit. Protocol payloads stay free of update state (R15).
Co-authored-by: Cursor <cursoragent@cursor.com>
* feat(serve): expose update state on /api/info
Serve-scoped controller owns an in-memory update snapshot: one
staleness evaluation after listen, then the shared unref'd scheduler,
stopped on close/shutdown. /api/info reads only the snapshot and gains
optional latestVersion/updateAvailable fields for eligible installs;
the existing three fields are byte-compatible.
Co-authored-by: Cursor <cursoragent@cursor.com>
* feat(web): dismissible update-available banner from /api/info
Fetches server info after backend connect and on reconnect, renders a
fixed banner in the exploring view only when updateAvailable is true
and the version is undismissed, hides while the reconnect banner is
active, and fails open on fetch errors. role=status + aria-live with a
keyboard-focusable dismiss; dismissal persists per version in
localStorage. Copy in en/zh-CN common.json with version interpolation.
Co-authored-by: Cursor <cursoragent@cursor.com>
* docs(cli): document update notifications and opt-outs
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(review): apply review findings and simplify pass
Review: gate the detached refresh spawn on a live lock-owner probe so
parallel CLI invocations coalesce to one refresh child (validated P2,
three-reviewer agreement); poll /api/info on a slow cadence while
exploring so post-load server-side discoveries surface (validated P1);
add a monotonic sequence guard so overlapping server-info fetches
commit in order.
Simplify (behavior-preserving): shared truthy-env/opt-out/freshness
helpers in update-cache.ts, shared cachedUpdateNoticeLine for CLI and
doctor, extracted install-eligibility core with per-process memo,
memoized registry parsing, single evaluation per scheduler cycle,
cache-only startup evaluate in serve, flattened MCP exit handler,
shared bottom-banner shell, storage keys in ui-constants.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(update-notifier): address residual review tickets on this PR
Stop the lock-busy 1ms scheduler spin, replace clock-skewed cache
entries, move the outbound URL guard into core, and extract the serve
update controller. Pin the startup/guard/single-flight/MCP/CLI contracts
those tickets called out.
Fixes#3167#3168#3169#3170#3171#3172#3173#3174
Co-authored-by: Cursor <cursoragent@cursor.com>
* Address PR review feedback (#3175)
Fetch the npm /latest document instead of the full packument so the 64KiB cap can succeed, and treat reused lock PIDs as stale so refresh is not suppressed.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Address PR review feedback (#3175)
Register the CLI spawn suite on the OS matrix, pin MCP opt-out env, and compare versions without IEEE-754 rounding.
Co-authored-by: Cursor <cursoragent@cursor.com>
* feat(cli): add gitnexus update install and versioned command banners
Give an explicit Claude/Codex-style upgrade (`npm i -g gitnexus@version`) and print `GitNexus <Name> (version)` on every command so the running build is obvious without silent self-update.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Address PR review feedback (#3175)
- Document the pinned install as npm i -g gitnexus@<x.y.z>, not a copyable @version tag
- Wait for wall-clock-future cache repair to publish before asserting
- Restore the stdout spy if the TTY notice assertions fail
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(update-notifier): keep last known latestVersion on a failed refresh
A later offline check was wiping the pin and hiding a known update for 24h.
gitnexus update still treats a failed live fetch as checkFailed.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Address PR review feedback (#3175)
- Word update.current so a newer-than-latest install is not called the latest stable version.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(ci): hide the detached update-check spawn on Windows
The refresh child was spawned without windowsHide, so Windows CI could stall
before writing the cache and then fail cleanup with EBUSY.
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(build): build the web UI from prepack, not from every npm ci
gitnexus-web is a separate ~650-package tree (React, Vite, LangChain,
Mermaid). Because `prepare` built it, every `npm ci` in gitnexus/ also
installed and Vite-built a second product. On CI that install ran
uncached inside an execSync timeout, so a healthy-but-slow install was
SIGTERM'd mid-flight and surfaced as `spawnSync /bin/sh ETIMEDOUT` --
repeatedly killing node floor compat, a job that only import-links the
CLI dist and never needs the UI.
The UI is only needed inside the published tarball, so build it from
prepack instead. `npm run build` and `prepare` are now CLI-only; pass
--web (or npm run build:web) to include it. Jobs that pack or publish
install gitnexus-web in their own visible step, and the in-script
fallback install is untimed so a slow install can no longer be killed
halfway and reported as a build failure. The tsc/vite timeout default
goes 300s -> 600s so the remaining bounded steps have headroom.
Default build on this machine: 30s, no gitnexus-web work.
* fix(build): enforce web package artifact integrity
Co-authored-by: Cursor <cursoragent@cursor.com>
* refactor(build): clarify web packaging helpers without changing behavior
Keep the same opt-in, fail-closed, and pack/publish preserve rules while
trimming comments, sharing the test harness, and reading index.html
directly instead of probing it first.
Co-authored-by: Cursor <cursoragent@cursor.com>
* ci: skip prepare on typecheck so a cold shared install cannot cancel the job
quality/typecheck's 10-minute budget was spent on an uncached gitnexus-shared
npm install plus a full prepare tsc that tsc --noEmit does not need.
Co-authored-by: Cursor <cursoragent@cursor.com>
* ci: stop typecheck-web from canceling before the npm cache can save
Hashing gitnexus-shared into the web cache key forced a cold 650-package
install; the 10-minute job then canceled and never wrote a warm cache.
Co-authored-by: Cursor <cursoragent@cursor.com>
* ci: give format the same 10-minute budget as lint
A cold root npm ci already took 4m19s and canceled prettier at the 5-minute
cap. Lint does the same install and needed 7m41s on that run.
Co-authored-by: Cursor <cursoragent@cursor.com>
* ci: stop installing TypeScript 7 just to compile gitnexus-shared
A dedicated npm ci in gitnexus-shared took 7 minutes to add two packages
(TypeScript 7's optional per-platform binaries) and cancelled typecheck,
Windows pack, and coverage shard 1. Compile shared with gitnexus's tsc.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Address PR review feedback (#3166)
- Run tsc via execFileSync so the compiler path is never interpolated into a shell.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Address PR review feedback (#3166)
- Run tsc as node typescript/bin/tsc so Windows never has to execFile a .cmd shim.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Launch tsc via node and lib/tsc.js on every OS.
The npm .bin/tsc shim is tsc.cmd on Windows, which execFileSync cannot spawn.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(ci): lock eval containment against a dedicated shared npm ci
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Seeding is unconditional today, so the next run would inherit the rejected
proposal from a generation whose proposer could still read the hidden
oracles. That taint propagates: each generation stages the previous
proposal, so one contaminated proposal survives until the artifact expires.
Scheduled runs still always seed — memoryless weekly runs would re-propose
the same rejected candidate forever.
Co-authored-by: Cursor <cursoragent@cursor.com>
The proposer authors the overlay the benchmark arms are scored with, but its
clone was never sanitized: it could read eval/workflow_bench, i.e. the task
prompts and the hidden oracles it was about to be graded against. The last
diagnostic run did exactly that, reading
inv-feature-list-repos-filter.oracle.test.ts directly, so a proposal could
win the gate by encoding expected behavior into a skill instead of being a
better skill. Sanitize the proposer clone exactly as run_cell already does.
Also remove the PreToolUse tool-input normalizer. It never ran: headless
`claude -p` (2.1.247) dispatches no hooks from inline --settings, a settings
file, project/user/local --setting-sources, or a trusted ~/.claude.json
project entry. Keeping it would read as a control in review while enforcing
nothing, and it was the sole reason the proposer stopped using --bare —
which stays off on its own merits, since bare ignores --tools and would cost
the proposer Grep and Glob.
Blank optional arguments from the OpenAI adapter remain handled where the
code is ours: MCP aliases in local-backend normalizeToolParams. Built-in
Read still rejects pages:"" and the model self-corrects on the next turn.
Co-authored-by: Cursor <cursoragent@cursor.com>
The first fully-logged skill-evolution run failed for five deterministic
reasons that had nothing to do with the candidate under test. Each is fixed
at the layer that actually owns the contract:
- Strict provider adapters materialize omitted optional string arguments as
"". The MCP alias normalizer now treats a blank optional alias as absent
(a blank REQUIRED target is still rejected), and a trusted PreToolUse hook
strips blank strings before Read/GitNexus tool calls.
- MCP semantic errors rode home in a successful envelope and logged as
result=ok. SessionProgress now inspects the payload and reports them as
semantic-error.
- Claude Code's nested sandbox overlays absent root dotfiles with device
nodes, which the provenance snapshot read as unauthorized workspace
changes. Those names are excluded at the workspace root and hidden from
git via an immutable excludes file.
- The proposer could not read /evidence from Bash (missing allowRead entry)
and had no offline gitnexus runner, so it fell back to npx and hit the
network. Both are now mounted; ripgrep is installed in CI.
- selected-rows.json advertised host artifact names that do not exist in the
mount. Rows now name their staged patch_file/transcript_files, the prompt
describes the real layout, and oversized bundles compact artifacts before
dropping evidence rows so no row is silently lost.
Also replaces two benchmark scenarios that main already satisfies
(trivial-version-alias, inv-bug-pdg-note) with non-vacuous ones, verified to
fail against a pristine checkout.
Co-authored-by: Cursor <cursoragent@cursor.com>
* feat(zig): static-gating analysis module + fixture (ported from feat/zig-static-gated-edges-v2)
Squashes c6fe922c, 2f3c8e9e, fab088f4, 9b58af74, aef2ae83, 86b892ef, d5657861,
f3780b3a: file-local comptime bool constants, cross-file flag resolution via the
@import alias map, re-aliased const chains, == / != against known bools,
else / else-if branch awareness. The module is self-contained; the hooks that
call it land in the next commit.
* feat(zig): stamp static-gated call sites through the scope pipeline
Wires the ported gating module into the scope-resolution pipeline that
now emits every Zig CALLS edge (PR #1432), replacing the parse-worker /
call-processor hooks of the original branch, which targeted the legacy
DAG path the merged provider no longer uses.
Data flow, one new fact carried end to end:
emitZigScopeCaptures stamps `@reference.static-gated` on a call
capture whose anchor lies in a statically dead
range (body of `if (CONST_FALSE)`, else of
`if (CONST_TRUE)`), via the module's new
`collectZigStaticGatedRanges` (line/col ranges,
because a Capture keeps no node)
scope-extractor marker -> `ReferenceSite.staticGated`
buildReference -> `Reference.staticGated`
references-to-edges, -> `GraphRelationship.staticGated` on the
free-call-fallback, emitted CALLS edge (both emit paths, plus
edges.ts (tryEmitEdge*) the generic bridge)
local-backend impact -> `staticGated` on impact frontier edges
Same marker idiom as Go's `@reference.callee-position` / `embedded-pointer`:
zero-range, present or absent, so every ungated site's capture set is
byte-identical and no other language changes.
SCHEMA_BUMP 92 -> 93: parse-time captures changed.
Cross-file constants (`if (cfg.FOO)` with `cfg = @import("cfg.zig")`) are
NOT stamped yet: the module resolves them through `lookupBoolsForPath`, but
the capture emitter runs per file in the parse worker with only
`{ path, content }`, so it cannot see the sibling source. The two positive
cross-file cases in zig-static-gating.test.ts are `it.skip` with that
reason; the negative ones pass unchanged.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015ciQ3MTjpQXkCoNntZR9zG
* feat(graph): add staticGated edge property
Adds an optional `staticGated?: boolean` field to `GraphRelationship`
that flags edges originating in code branches known at index time to
be unreachable in production — e.g. `if (CONST_FALSE)` blocks where
the condition reduces to a comptime-known `false`.
Schema + persistence wiring:
- `gitnexus-shared/src/graph/types.ts` — additive optional field on
`GraphRelationship`; absent edges read identically to live ones.
- `gitnexus/src/core/lbug/schema.ts` — `staticGated BOOLEAN` column
on the `CodeRelation` REL table.
- `gitnexus/src/core/lbug/csv-generator.ts` — appends a
`staticGated` column (0/1) to the `relations.csv` written for
bulk COPY ingest.
- `gitnexus/src/core/lbug/lbug-adapter.ts` — fallback per-row
`MATCH ... CREATE` insert reads the optional column and threads
it into the relationship properties.
No language has populated this field yet — the Zig hookup lands in
the next commit. Existing DBs need a re-index for the new column to
appear; existing readers are unchanged because the field is
optional and absent on every other language's edges.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
(cherry picked from commit c8cd5efe27a77a5d1b3f05e3f4e89069b5c6b19e)
* fix(zig): gate bare literal branches; AND the flag over deduplicated free-call sites
PR #3161 review, two findings:
1. `stampZigStaticGating` returned early when the file declared no boolean
constants, but `collectZigStaticGatedRanges` also folds bare literals, so
`if (false) { foo(); }` in a constant-free file went unstamped. The early
return is gone; the range walk runs for every file.
2. `emitFreeCallFallback` deduplicates CALLS edges per (caller, callee) and
wrote `staticGated` from whichever site it met first, so a callee reached
from one live site and one dead site was gated or not by traversal order.
Emission is now deferred to the end of each file's sites and the flag is
the AND over every site that collapsed into the edge: one live site keeps
the edge live. The other emit path keys its dedup on the site range and
was not affected; `collapseByCallerTarget` in the generic bridge would
have the same shape but no language that sets the marker opts into it.
Fixture + tests: `gated_bare_literal`, `live_and_gated_same_callee` (live
site first) and `gated_then_live_same_callee` (dead site first) in
zig-static-gating.test.ts. All 70 resolver suites (3,603 tests) pass with
the shared emitter change.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015ciQ3MTjpQXkCoNntZR9zG
* test(zig): move the SCHEMA_BUMP pin to 93; rebaseline emit fingerprints for the staticGated column
Three CI failures on f4954963, all consequences of this PR:
- test/unit/incremental-parse-cache.test.ts pins SCHEMA_BUMP so concurrent
bumps cannot collide; 92 -> 93 for #3161 (parse-time call captures gain
`@reference.static-gated`), 92 added to the taken list.
- bench/emit-persistence `measure.mjs --check` and `measure-streaming.mjs
--check`: byte-identity fingerprints drift because every relationship row
now ends in a `staticGated` cell. Regenerated with the inverse-operation
evidence recorded under `_rebaselined_3161_static_gated_column` in both
baseline files: stripping ONLY the new column from the emitted CSVs
reproduces the prior fingerprints exactly (36 files, 3 rel_* files differ,
33 byte-identical; 36,000 PDG rows each +2 bytes), so no row moved between
pair files or reordered. Timing and retention gates passed throughout.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015ciQ3MTjpQXkCoNntZR9zG
* docs(graph): state the CALLS contract on staticGated; say "provably unreachable at compile time"
Review on #3161 (magyargergo): the flag must not redefine what a CALLS
edge means. The field's doc now says so explicitly: CALLS still means
"there is a resolved call site from A to B", never "B is reachable from
A"; `staticGated` is additional, statically provable path-feasibility
metadata, an opt-in analysis layer that no core pass acts on. The edge is
emitted, persisted, traversed and counted exactly as before.
Wording: "unreachable in production" -> "provably unreachable from the
indexed source at compile time" on GraphRelationship, ReferenceSite and
Reference. The index has no production build configuration and should not
claim one.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015ciQ3MTjpQXkCoNntZR9zG
* feat(zig): surface staticGated on impact byDepth items; gate if-expressions and negated/parenthesized conditions
Addresses the tri-review on #3161.
- impact: the depth traversal already selected r.staticGated but dropped it
when building the byDepth item. Forward it (present only when true) and
document the field on the impact tool's byDepth contract. Traversal and
ranking still do not act on it; that stays opt-in for consumers.
- zig-static-gating: walk `if_expression` (`const x = if (c) a() else b();`)
in addition to `if_statement`. The expression form has no field names and
no else_clause wrapper, so the arms are located positionally
(`ifExpressionArms`). Labeled-block arms are covered.
- evalCond: `parenthesized_expression` is transparent, so `!(A and B)` and
`((FLAG))` fold. Prefix `!` has no unary node in tree-sitter-zig; the
header now says exactly which shapes fold instead of "simple negation".
- fixture + tests: nine new cases (negation x2, parentheses x2,
if-expression then/else/labeled-block x5). Cross-file `@import` constants
remain skipped and now cite the tracking issue #3162.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015ciQ3MTjpQXkCoNntZR9zG
* refactor(zig): drop the unreachable cross-file gating builders; document the real wiring
gitnexus-check on b77cb4ed: `buildZigImportAliasMap` / `buildZigRawImportAliasMap`
and the per-call ancestor walk (`isCallStaticGated`, `ifBranchDirection`,
`nodesEqual`) had no caller anywhere. They were ported from the legacy
call-processor design; the scope-resolution provider stamps ranges via
`collectZigStaticGatedRanges` instead, and nothing populates the cross-file
seam yet (#3162). Remove them so the module exports only what runs.
The evaluator keeps `importAliases` + `lookupBoolsForPath` (the seam #3162
will fill); the header now says so explicitly and points at the actual
wire-up (`stampZigStaticGating` in languages/zig/captures.ts) instead of the
retired `configs/zig.ts` hook.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015ciQ3MTjpQXkCoNntZR9zG
* refactor(zig): reuse descendantsOfType and tighten static-gated capture matching
Walk if-nodes through tree-sitter instead of a hand-rolled stack, return the original capture array when nothing is gated, and register the marker as a known sub-tag so it cannot be mistaken for an anchor.
Co-authored-by: Cursor <cursoragent@cursor.com>
* style(zig): wrap a long else-clause assignment to satisfy prettier
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
The sweep printed error_kind=plan-evidence-invalid and nothing else, so
the reason a cell failed stayed in results.jsonl — an artifact uploaded
after the run, not something a watcher can read while it is still going.
Print the cell's error_detail next to its result line, redacted through
the same credential list as the artifact and bounded, since a
session-error detail carries stdout/stderr tails.
Co-authored-by: Cursor <cursoragent@cursor.com>
