`ModelResolutionOptions` was a field-for-field duplicate of the existing
public `ModelResolutionTransform`, down to a verbatim copy of its `new()`.
`pipeline::transform` then unpacked one to rebuild the other, cloning the
catalog Arc and the eligible-provider set on the way.
- Delete `ModelResolutionOptions`. `TransformOptions.model_resolution` now
holds an `Option<ModelResolutionTransform>` directly, so the TRANSFORM
step is `resolution.apply(graph)?` with no rebuild and no clones. This
is consistent with `custom_transforms`, which already holds transforms.
- Add `ModelResolutionTransform::catalog()` so the VALIDATE step can reach
the same catalog for its lint rules. That is the only new code needed.
- Drop `CatalogScope` from `operations::validate`, which was a third copy
of the same fields. The three entry points now hand a partially built
transform to `validate_resolving_models`, which completes it with the
workflow's default provider once the workflow is resolved.
- Extract `validate_child_workflow` in `manager_loop`, collapsing two
near-identical validate-and-unwrap blocks.
- Point the transform tests at their own `transform_options()` helper via
struct-update syntax instead of respelling all seven fields, and drop a
HashSet -> Vec -> HashSet round trip from the create test helper.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Follow-up cleanup on the catalog-free validation split. Same behavior,
fewer parallel code paths.
- Make the catalog an explicit `Option<&Catalog>` on `pipeline::validate`
instead of a `validate` / `validate_with_catalog` pair, so each call
site states whether catalog rules run.
- Collapse `preprocess_and_validate`, `preprocess_and_validate_structural`,
and `preprocess` into one function that takes `TransformOptions`. Its
`model_resolution` field is now the single source of truth for catalog
awareness, which drops a 12-argument signature and the
`too_many_arguments` allow.
- Replace the duplicated resolve-and-preprocess block in
`operations::validate` with one `validate_in_scope` helper, and drop the
HashSet -> Vec -> HashSet round trip on the catalog path.
- Extract `configured_default_provider`, previously duplicated between
`operations::create` and `operations::validate`.
- Delete `validate_manifest_with_environment_defaults`, which had no
callers outside its own module.
- Share the `server-model.fabro` fixture between the two CLI tests instead
of inlining it twice. The validate test now asserts the rendered output
through the usual snapshot helper, which also removes a hand-rolled
`std::fs::write` and its clippy allow.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The model indicator on a stage page hovered to provider, model, and
reasoning effort only. Seeing what a stage actually spent meant leaving
for the Billing tab, which reports per node rather than per visit.
The stage list had no token data to show, so add a per-visit `billing`
block to `GET /runs/{id}/stages`. The Billing tab's pricing rule (a
provider-reported cost wins, otherwise the server catalog prices the
tokens) was private to `billing_rollup`; move it to
`StageProjection::billed_usage` and drive both call sites from it so the
two views cannot drift.
The popover's buckets use the Billing tab's labels verbatim. It stays
scoped to one visit, so a looped node's row on the Billing tab is the sum
of what each of its visits shows here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The DOT parser created a node for every edge endpoint, and nothing
recorded whether a node came from a declaration or was synthesized from
an edge. The edge_target_exists rule only checked whether the node id
was present in the graph, which was always true by then, so a misspelled
endpoint became an attribute-free node that defaulted to shape=box — an
LLM stage. Validation emitted a prompt_on_llm_nodes warning and exited 0.
Node now carries `implicit`, set only when the parser synthesizes the
node from an edge endpoint. A declaration anywhere in the workflow
clears it, so order does not matter and subgraph declarations count.
Node::new leaves it false, so programmatic construction and graphs
deserialized from older checkpoints read as declared.
edge_target_exists treats an endpoint as valid only when it exists and
is declared, reporting each undeclared node once. The near-identical
missing-source and missing-target branches collapse into one path. The
import transform copies the flag onto spliced nodes so an edge-only node
inside an imported fragment is caught too.
parse_and_validate_human_gate had two edge-only nodes and now declares
them; it was an instance of the bug rather than a casualty of the fix.
No shipped workflow, docs example, or CLI fixture relied on the old
behavior.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The check tested for `never reconstruct it from memory`, a phrase the
Kimi edit description no longer contains after it was reworded to match
Kimi Code. A one-sided `!contains` against a literal cannot tell "the
phrase is absent because nothing leaked" from "the phrase is absent
everywhere", so it silently stopped protecting anything.
Assert the marker is present in Kimi's own description and absent from
the stock one. Removing the marker from the description now fails the
test instead of quietly disarming it, verified by doing exactly that.
Also correct the grep docs: all three sandbox implementations probe for
`rg` and fall back to POSIX `grep`, so the page should not imply a
single engine. Pre-existing, adjacent to the lines this branch touched.
Reported by Copilot review on #646.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Neither harness puts read-before-edit mechanics in the system prompt.
Kimi Code's `system.md` has no such section; the rules live in
`edit.md`, `write.md`, and `read.md`. Codex's prompts say nothing about
reading before an edit at all, and its editing guidance is attached to
`apply_patch`.
Drop the `# Reading Before Writing` section from the Kimi prompt and
carry its content in the Edit, Write, and Read descriptions, worded as
Kimi Code words it. Nothing is lost: every bullet in the removed
section was already covered by a tool description.
Two behaviors change to match upstream. Edit now says not to issue
consecutive edits against the same file, since the first invalidates
the second's `old_string` -- Kimi Code's stated reason. Read now says
not to re-read solely to confirm a write landed, which both harnesses
call out as waste; the previous prompt asked for exactly that re-read.
The gpt56 profile already followed the Codex split and is unchanged.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`active_time_ms` was only ever computed from terminal stage events, so a
stage still running contributed zero to the run rollup. A run parked in one
long agent stage reported 2m 8s of active time against 16m 53s of wall
clock — the two finished stages — while the running stage had been doing
continuous inference and tool work for over 14 minutes.
`live_run_timing` summed `filter_map(|stage| stage.timing)`, and
`stage.timing` is only written at finalization. Wall time ticked live off
`start_time`; active time did not tick at all.
Stage projections now accumulate brackets from the event log:
- Closing an inference bracket folds its span into `live_inference_ms`
instead of discarding it, including across retries, matching the
in-process stopwatch.
- Tool calls open a batch on the first outstanding call and close it when
the last one drains, so tools running concurrently within a turn count
once — the same span `execute_tool_calls` is bracketed by. Summing
per-call durations would over-count parallel tool use. Subagent tool
events are excluded; they run inside the root call's span already.
- `StageProjection::live_timing(now)` composes accumulators with any open
bracket, per handler: agent stages use the brackets, prompt and command
stages count elapsed time as inference and tool respectively, and
handlers that wait on a human, timer, condition, or child branches
report zero.
Active is clamped to wall per stage. A worker killed mid-turn leaves its
bracket open forever, and without the clamp it would tick up unbounded.
The clamp does not need to detect the dead worker: a stage cannot have been
active longer than it has existed. `watchdog.timeout` remains the authority
on whether a run is stuck. The clamp is deliberately not applied at run
level, where concurrent branches can legitimately sum past run wall time.
Timing is derived from events rather than emitted by the worker, so this
needs no event-schema change and applies to runs already stored.
`StageProjection.timing` keeps its terminal-only meaning, and the
authoritative breakdown still replaces the live estimate at terminal
events.
The billing endpoint had the same hole behind its `wall_only` fallback:
running stages reported zero inference/tool/active. Not visible in the
product, which renders only `wall_time_ms`, but wrong for any other
consumer of `GET /runs/{id}/billing`.
Parallel branch stages lose their breakdown permanently, even after
completion, because `parallel.branch.completed` carries only `duration_ms`.
That is a separate data-loss bug, tracked in #644.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`ReadBeforeWriteSandbox` blocked writes to any existing file the agent
had not read, tracked by a session read set populated only by
`read_file`, `grep`, `read_many_files`, and the Kimi `Read`.
The gpt56 profile has none of those. It mirrors Codex's tool contract --
`shell_command`, `apply_patch`/`edit_file`, `update_plan`, `web_search`
-- and reads through the shell, so its read set stayed permanently
empty and every edit to an existing file failed. In run
01KYD4360GN6SED4BYEVGYP4XT all 28 `edit_file` calls failed, 25 of them
on the guard. The agent read `package.json` with `sed` and `cat`,
hex-dumped it trying to diagnose the rejections, then routed around the
guard with `sed -i`, which the guard never covered. It prevented no
blind write; it converted content-anchored edits into an unreviewed
in-place shell rewrite.
Neither Codex nor Kimi Code enforces read-before-write at runtime.
Codex's `apply_patch` `Add File` overwrites an existing path silently;
Kimi Code's `Write` has no check at all. Both rely on the exact-match
requirement in their edit tools, which is stronger proof of inspection
than a read set, plus per-write approval.
Tool descriptions and the Kimi prompt keep telling the model to read
before editing -- that guidance matches Kimi Code's own `edit.md` and
still prevents `old_string not found` -- but no longer claim the
workspace refuses unread writes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Codex drives the GPT-5.6 models with a much narrower tool set than the
other OpenAI models: a shell, `apply_patch`, and `update_plan`. It has no
file-read, file-write, grep, glob, or fetch tool at all -- reading and
searching go through the shell, and every write goes through
`apply_patch`. Offering 5.6 fabro's extra tools advertises affordances its
instructions never mention, so this adds a profile that registers only
what Codex does.
The profile is selected per model via `agent_profile = "gpt56"` on the six
5.6 rows (three each on `openai` and `openrouter`), following the existing
Kimi-over-a-gateway pattern. Every other model on those providers keeps
its provider default, with no code branch and no version sniffing.
- `ToolVocabulary::Codex` renames `shell` to `shell_command`; a strum
alias keeps `from_any_name` resolving it to `NativeTool::Shell`, so
permissions, categories, and telemetry still key on the canonical name.
- `shell_command` gains `workdir`, passed to the `cwd` argument
`execute_shell_command` already accepted, with Codex's "always set
`workdir`, do not `cd`" guidance.
- `prompts/gpt56.md.j2` is adapted from Codex's 5.6 `base_instructions`,
which are byte-identical across Sol, Terra, and Luna. A header comment
records provenance and the departures fabro's harness forces.
This is an alignment-only pass: it matches Codex's tool contract while
keeping direct tool calls. Codex actually drives 5.6 in code mode, with a
single `exec` tool taking JavaScript and every other tool reached through
a `tools` object inside a V8 isolate. That is deliberately out of scope.
Luna's `multi_agent_version: v1` (vs v2 on Sol and Terra) is also out of
scope. It only changes the sub-agent tool set, which fabro registers from
the caller rather than the profile, and fabro's current set matches
neither version exactly.
Two server cancel-timing tests are adjusted. `gpt-5.6-sol` is the
`openai` provider's default model, so runs that name no model now build a
3-tool profile instead of an 8-tool one and reach their first stage
sooner. `full_http_lifecycle_cancel` asserted `status.kind == "blocked"`
at the instant of cancel, which the worker is free to change the moment it
is signaled; it now accepts either live state, matching the tolerance its
own comment already documents for `pending_control`.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The stage-summary preamble rendered per-stage token usage for every
completed LLM stage: "Model: kimi-k3, 92.6k tokens in / 41.1k out" at
compact fidelity and "Tokens: N in / N out" at summary:high. Agents read
that as their own remaining budget.
In run 01KYCM3EG4KMCVRDYNV93PZWBV an implementation stage stopped after 2
of 9 units, reasoning "We have around 100k tokens, but time constraints
are an issue" and recording the rest as halted "within the available
execution window". The 92.6k it saw was the preceding plan stage's
billing telemetry, the only token quantity anywhere in its context. It
had used 11% of a 1,050,000-token window and 0.8% of a 24h stage timeout,
and no harness limit was near.
These counts have no task value to the agent: they describe a different
model's usage on an earlier stage, they are stale by one stage, and
nothing in the preamble distinguishes them from a budget. Keep the model
id and files touched, which carry provenance the agent can act on.
Both tests that asserted the counts now assert their absence, so the
regression is caught rather than re-snapshotted.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Extend the Daytona Bash probe to cover the streaming toolbox-session transport in addition to the direct process exec. The two build different requests, so passing one is not evidence for the other: the `exec` regression fixed in #636 left every streaming command stalling until its timeout while the lifecycle probe reported a healthy sandbox. The session probe reuses the streaming path's own command construction and completion wait, so a transport that suppresses Daytona's exit-code bookkeeping fails at the lifecycle boundary with a remediation that names the wrapper-shell contract.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Run the streaming Bash wrapper as a child of Daytona's session shell so the provider can resume its bookkeeping and persist the command exit code. Add a regression test that exercises the sourced-command contract and preserves a nonzero exit status.
Interpolate the visible summary allowance after applying the model max_output cap, so low-output models are not asked to produce more text than the request permits.
`StreamStart` was supposed to mean "the provider is responding", but
each decoder decided for itself when to emit it, so it meant something
different per dialect:
anthropic on the `message_start` frame
bedrock on the `messageStart` frame
openai_responses latched on the first SSE event
gemini latched on the first chunk
openai_compatible never — Chat Completions has no opening frame
A consumer could not rely on it, which is why the inference-bracket
work keyed its first-output edge on content kind instead.
Ownership moves to the two loops that drive decoders — the shared SSE
loop in `transport.rs` and the AWS event-stream loop in the bedrock
provider — each emitting exactly one `StreamStart` immediately before
handing over the first framed event. The invariant is now structural:
it cannot depend on a dialect having a particular opening frame,
because no decoder is involved in producing it. `StreamDecoder`
documents that decoders must not emit it, and the four that did no
longer do.
Only `openai_compatible` changes observably, gaining the event it never
had; the other three dialects' snapshots are byte-identical, because
their opening frame was already the first framed event. The six
updated `openai_compatible` snapshots each differ by exactly one
leading `stream_start` and nothing else.
Each dialect also gets an explicit `stream_opens_with_stream_start`
assertion. The snapshots already cover this, but a snapshot can be
re-accepted silently, and this is the one event a liveness consumer
needs to hold for every provider.
No behavior change to the agent's inference bracket:
`first_output_kind()` maps `StreamStart` to `None`, so the bracket
still opens on observed content and keeps reporting which kind
arrived. The point of this change is that a content-agnostic edge now
exists at all — the one-shot coverage follow-up needs it, and it is
strictly earlier than first content.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Model default reasoning explicitly at the provider-route level so always-reasoning endpoints without effort controls receive summary headroom. Cap all summary requests at model output limits and bound retained visible summaries to the original allowance. Reuse builtin catalog fixtures and named budget constants in tests, and document the new model setting.
During a long LLM turn the durable event stream was silent: between
`agent.tool.completed` and the next `agent.message` nothing was emitted,
so "the model is generating" and "the worker is wedged" were
indistinguishable from the run store, SSE, or the UI.
The signal already existed. `AssistantTextStart` fired at exactly the
right point — after `build_request()`, after compaction, immediately
before the stream opens — then was classified as streaming noise and
thrown away. This promotes it rather than inventing a new one.
Two events, each asserting only what is provable when it is emitted:
- `agent.llm.started` carries the *requested* provider/model. No usage,
no cost, no context window: none of it exists yet, and failover can
re-target, so `agent.message` stays authoritative for what answered.
- `agent.llm.first_output` is edge-triggered on the first output of an
attempt and names what arrived. `ToolCall` is required, not optional:
a turn that opens with a tool call produces no text or reasoning
delta, so a latch keyed on those two would stay silent for exactly
the tool-heavy rounds where liveness matters most.
`agent.llm.retry` now also fires on the one previously invisible
mid-turn path — a stream that ends without a finish event, which
replays the turn and discards its output with nothing to show for it.
Its `attempt` field was already fed by two independent counters, so an
optional `phase` (open | consume) names which loop it counts.
`StageProjection.inference` projects the open bracket. `Some` means
"the event log contains an unclosed inference bracket", not "the model
is computing now" — a SIGKILLed worker leaves it open, which is the
truthful statement of what we know, and `watchdog.timeout` remains the
authority on actually-stuck.
The close is the subtle part. Terminal cancel and wall-clock timeout
tear the session down through `discard_session` without emitting a
message, error, or interrupt, so a session-lifecycle backstop is
required. It has to be `agent.session.ended`, not
`agent.session.deactivated`: deactivation is emitted by `lease.release()`
*before* the forwarder drains queued agent events, so a queued
`agent.llm.started` can arrive after it and re-open the bracket. But
`agent.session.ended` carries no stage identity, so the close takes
ordering from the event and identity from the projection, scanning for
brackets the ending session opened. A normal stage lookup there finds
no target and silently no-ops.
Presentation states what the log proves and nothing more: no progress
bar or ETA (no completion estimate exists), "reasoning" only when the
provider sent reasoning output, elapsed counted since the request
opened, and no live animation once the run is terminal.
Scope is session-backed agent stages. One-shot completions call
`client.complete` directly and never build a session; covering them
means moving the emit point into `fabro-llm`, filed as a follow-up.
`agent.output.start` was never persisted — it existed in a name map,
an `unreachable!` arm, and docs — so the rename carries no migration
risk. Corrects `events.md`, which documented it as a real emitted
event, and the v2 proposal, which mapped it to `message.part.started`
despite it firing before the request opens.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
