1. Succeeded runs now rejected — a completed run keeps checkpoint.json
around, so resume would happily restart and overwrite start.json and
conclusion.json. Now checks status.json and bails on Succeeded.
2. PID liveness check moved before checkpoint validation. The engine
writes checkpoint.json with a plain fs::write, so a concurrent
resume could see a half-written file and report "corrupt" for a
run that is simply still alive. Order is now: PID → status →
checkpoint parse → cleanup → spawn.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Two issues in the resume cleanup logic:
1. progress.jsonl was not in the stale artifact list, so attach and
logs would replay the previous attempt's events before the new run.
Added it to the cleanup list.
2. Cleanup ran before validating the checkpoint was parseable. A
crash during the original run can leave a truncated checkpoint.json
that passes exists() but fails to parse. We now load and parse the
checkpoint first; if it's corrupt we bail with the old conclusion
and failure evidence intact.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Two bugs from the refactoring:
1. (High) On resume, run_command_impl would create a fresh worktree
with skip_branch_creation=false, force-resetting the run branch
and losing file changes from the original run. Fix: force
workdir_strategy to LocalDirectory when resume=true.
2. (Low) Sleep inhibitor guard was created inside a #[cfg] block
scope, so it was dropped before resume_command ran. Fix: use
`let _guard = { ... }` pattern to keep it alive for the arm.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Resume now follows the same subprocess pattern as run: look up run
directory by ID prefix, validate checkpoint exists, clean stale
artifacts, reset status to Submitted, spawn _run_engine --resume, and
attach. This eliminates ~1600 lines of duplicated env/sandbox setup
from resume.rs.
Key changes:
- operations::start() and operations::resume() take run_dir instead
of Persisted, loading state from disk internally
- run_engine() builds RunOptions from RunRecord on disk, so callers
no longer extract record fields manually
- StartOptions flattened (no more nested InitOptions)
- FabroError::Precondition variant for start/resume guard checks
- _run_engine accepts --resume flag to dispatch to resume path
- operations::restore removed (no longer needed)
- Resume CLI stripped to just <RUN_ID> + --detach
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Follow-up to fafc0a3c. Renames local variables, function parameters,
and struct fields that hold renamed types (ExecutorOptions, RunCreateOptions,
RunOptions) from config/settings to options/run_options for consistency.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Standardize naming so all "bag of options" structs use the Options
suffix: ExecutorSettings→ExecutorOptions, RunSettings→RunOptions,
GitCheckpointSettings→GitCheckpointOptions, LifecycleConfig→LifecycleOptions,
RunCreateSettings→RunCreateOptions, StartRetroConfig→StartRetroOptions,
StartFinalizeConfig→StartFinalizeOptions, AutoMergeConfig→AutoMergeOptions.
Also renames the run_settings module to run_options.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
operations::create now handles the full pipeline: var expansion, parse,
goal override, transform, validate, config normalization, and persist.
This eliminates duplicated RunRecord construction and pipeline::persist
calls across CLI and API callers.
Key changes:
- Rename operations::create → validate, CreateOptions → ValidateOptions
- New operations::create returns Persisted, with RunCreateSettings
- Add ValidationFailed error variant with diagnostics
- Move normalize_config, default_run_dir into operations
- Delete prepare_workflow, PreparedWorkflow, CliFlags from CLI
- Make pipeline::persist and types module pub(crate)
- API catches both Parse and ValidationFailed as 400
- CLI prints diagnostics directly from error (no re-validation)
- ExecutionOverrides struct replaces 9-param function
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
These are purely sandbox concerns — they serialize/deserialize sandbox
connection info and reconstruct sandbox instances. Moving them to
fabro-sandbox improves cohesion and removes workflow-layer coupling
from sandbox lifecycle logic.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The enum had zero internal usage in fabro-workflows and naturally belongs
in fabro-sandbox alongside the sandbox implementations. Removed cfg
gates from the Exe variant (it's just a tag) and added non-exedev
fallback arms in fabro-cli to handle feature unification from fabro-api.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Colocate compute_stage_cost and format_cost with StageUsage, eliminating a
thin module that only imported from outcome.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Consolidate all record types under the records module. Files are renamed
to drop the _record suffix (run_record→run, start_record→start,
sandbox_record→sandbox) since the module path provides that context.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Move each transformer into its own file under transforms/, move
stylesheet.rs into the directory, and fold vars.rs into
variable_expansion.rs. Backward-compat re-exports in lib.rs keep all
external paths working.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Relocate conclusion.rs to records/conclusion.rs behind a new records
module, and move preamble.rs into handler/llm/preamble.rs where it is
actually used. Update all imports across fabro-cli and fabro-workflows.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-locate LLM backend implementations (AgentApiBackend, AgentCliBackend,
BackendRouter) under handler/ since they implement the CodergenBackend
trait defined in handler/agent.rs.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Gate pr_config on dry_run_mode to prevent PR creation during dry runs
- Restore em dash (—) separator in retro output
- Print "Retro unavailable" when retro is enabled but returns None
- Fix pre-existing clippy warnings (derivable_impls, needless_borrow)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Rename `settings: mut config` binding to `mut settings` in resume.rs
and update all 8 downstream references
- Deduplicate normalize_config call in run.rs by reusing the result
computed for RunRecord
- Replace expect() with graceful error handling when loading RunRecord
in the API server's execute_run
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Phase 5: Rename debug artifacts from run.toml/graph.fabro to
workflow.toml/workflow.fabro. Change write_run_config_snapshot to
byte-for-byte copy of the original TOML instead of re-serialization.
Phase 6: Add run_from_record() that builds execution state directly
from a RunRecord, bypassing prepare_workflow(). Refactor run_command
into run_command + run_command_impl to share execution logic. Simplify
run_engine_entrypoint to call run_from_record() instead of
reconstructing RunArgs and re-parsing the workflow.
Step 7k: Write RunRecord in the API server's execute_run() for
observability, enabling fabro ps/inspect for API-initiated runs.
Fix stale manifest.json reference in docs/agents/outputs.mdx.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Collapse init_run/init_run_with_records/init_run_inner into a single
init_run(run_id, files) that takes all files as a flat slice. Resume
from metadata branch now uses RunRecord's embedded graph directly
when available, falling back to graph.fabro DOT parsing for old runs.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Delete run_spec.rs and manifest.rs modules. Remove write_manifest()
from the engine, update DiskLifecycle and GitLifecycle to only write
StartRecord. Remove read_manifest() from MetadataStore. Update
run_fork to only handle run.json/start.json. Convert resume.rs to
use RunRecord/StartRecord from the metadata branch. Update all tests
and integration tests.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Introduce two new persistence types aligned to the CREATE/START lifecycle:
- RunRecord (run.json): written at CREATE with merged FabroConfig, fully
transformed Graph, and run metadata
- StartRecord (start.json): written at START with start_time, run_branch,
and base_sha
All readers (run_lookup, inspect, diff, pr, attach, detached_support,
start, run_fork, pull_request, run_rewind, resume) now read from the
new types first. Legacy manifest.json + spec.json are still written
for backward compatibility (removal in follow-up).
Also adds dry_run, auto_approve, no_retro fields to FabroConfig, derives
Default on LlmConfig and Graph, and updates docs + tests.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The dry_run_writes_jsonl_and_live_json test was timing out at 4s because
the arc() helper didn't pass --no-upgrade-check, causing every test run
to await a background GitHub API call before process exit.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Introduce `fabro_workflows::pipeline` module defining typed phases:
PARSE → TRANSFORM → VALIDATE → INITIALIZE → EXECUTE → RETRO → FINALIZE.
Each phase is a standalone function with `#[non_exhaustive]` input/output
types so the compiler enforces ordering. `Validated` uses private fields
with read-only accessors to guarantee immutability post-validation.
Split `engine.run_with_lifecycle()` into `prepare_sandbox()` +
`execute_graph()` (backward-compatible wrapper preserved). Rewrite
`WorkflowBuilder::prepare_inner()` and CLI `prepare_workflow()` to use
pipeline functions. `PreparedWorkflow` now carries a `Validated` with
accessor methods instead of raw `graph`/`source` fields.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Make fabro-core's Outcome generic over a usage/metadata type parameter
(OutcomeMeta trait), allowing fabro-workflows to use core's types
directly via a type alias instead of maintaining duplicate Outcome,
StageStatus, and FailureDetail types with bidirectional conversions.
Key changes:
- Add FailureCategory enum to fabro-core (moved from fabro-workflows'
FailureClass), with Display/FromStr/is_signature_tracked
- Add OutcomeMeta supertrait + blanket impl for the generic parameter
- Make Outcome<M>, NodeResult<M>, RunState<M>, NodeDecision<M> generic
with default type parameter M=()
- Add Graph::Meta associated type
- Update FailureDetail with serde renames (category→"failure_class",
signature→"failure_signature") for checkpoint backward compat
- Replace fabro-workflows' Outcome with type alias to
fabro_core::Outcome<Option<StageUsage>>
- Add OutcomeExt extension trait for wf-specific factory methods
(fail_classify, fail_deterministic, retry_classify, simulated, etc.)
- Delete core_adapter/outcome.rs (~170 lines of conversion functions)
- Replace FailureClass with FailureCategory throughout fabro-workflows
- Fix timeout handler to use TransientInfra category, panic handler to
use Deterministic category
Net: -144 lines, zero-cost type unification with no runtime conversions.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add verbose_enabled(), prevent_idle_sleep_enabled(), and
upgrade_check_enabled() helpers to FabroConfig to encapsulate default
values. Update all call sites in fabro-cli to use the new helpers.
Also eliminate an unnecessary clone in SubAgentManager::run_to_completion
and use extend() instead of append()+clone() in config merging.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Update references to WorkflowRunConfig, ServerConfig, apply_defaults,
and deny_unknown_fields in comments and docs to reflect the FabroConfig
unification.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace five config types with a single FabroConfig superset type. All
loading functions (load_cli_config, load_server_config, load_run_config,
parse_project_config) now return FabroConfig. This eliminates the
run_defaults indirection, into_run_defaults() conversion, and
apply_defaults() bridging method in favor of a single merge_overlay()
that works across all config layers (CLI → project → workflow).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The field was populated in constructors but never read by any code.
Selected keys are already carried by AnswerValue::MultiSelected(Vec<String>),
making this field redundant. Also removes the unused options parameter from
Answer::multi_selected().
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Remove ErrorPolicy enum (continue/fail_fast/ignore) and the k_of_n/quorum
join policies from the parallel handler, leaving only wait_all and
first_success. This deletes ~180 lines of conditional logic including
FailFast early termination, the ParallelEarlyTermination event, and all
related tests and documentation.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Introduce a reusable Warning { kind, message, details } variant in
AgentEvent so non-fatal warnings (context window usage, deprecation,
etc.) share a single event shape. The context_window warning preserves
all original fields inside the JSON details object.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Delete the single-implementor LanguageModel trait and merge its methods
into inherent impl on a renamed Model struct. Change provider field from
String to Provider enum, eliminating constant string↔enum conversions
across the codebase. Fix Provider serde attributes so OpenAi serializes
as "openai" (not "open_ai") to match catalog.json.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Introduce OOP API for the model catalog: LanguageModel trait with blanket
impl on ModelInfo, Catalog struct with typed methods (get, list,
default_for_provider, closest, build_fallback_chain, etc.), ModelRef enum
replacing ModelId, and Provider::OpenAiCompatible variant. Migrate all
callers across the workspace to use Catalog::builtin() and remove the old
free-function API.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
reqwest::Client::new() queries macOS SCDynamicStore for system proxy
settings, which calls CFBundleGetMainBundle() → readdir() on
target/debug/deps/. With 576K stale build artifacts accumulated in
that directory, each readdir() took 1.3s, causing 15s+ delays when
multiple test processes ran concurrently.
- Disable jsonschema default features to remove unnecessary reqwest@0.13
and rustls-platform-verifier dependencies
- Make reqwest::Client lazy in web_search tool (OnceLock) to avoid
constructing it during profile tests
- Mark validate_api_key_rejects_invalid_key as #[ignore] since it hits
the live Anthropic API (3.2s per invocation)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
## Summary
- **Unify foreground and detach code paths**: Both `fabro run` modes now
go through the same `create_run() + start_run()` pipeline, with
foreground adding `attach_run()`. Only `--preflight` remains as a
special case.
- **Fix three bugs in create→start→attach path**: (1) `_run_engine`
crashed for `.fabro` workflows by hardcoding `run.toml` — now falls back
to `graph.fabro`; (2) `attach_run` couldn't detect crashed engines due
to zombie processes — `start_run` now returns the `Child` handle; (3)
`create_run` ignored `--run-id`.
- **Configure nextest slow-timeout profiles**: Tighten unit test timeout
to 2s slow / 4s kill, add `e2e` profile with 10s/30s. Switch CI and docs
to `cargo nextest run`.
## Test plan
- [ ] `cargo nextest run --workspace` passes with new timeout profiles
- [ ] `fabro run <workflow>` works in foreground mode (create + start +
attach)
- [ ] `fabro run --detach <workflow>` prints run ID and exits
- [ ] `fabro attach <run>` works standalone (without child handle)
- [ ] `fabro resume <run>` works for both `.toml` and `.fabro` workflows
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Fabro <noreply@fabro.sh>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Fix false positive in backward-compat fallback: use path.exists() instead
of error chain inspection to distinguish missing run.toml from one with
a broken internal reference (e.g. missing Dockerfile)
- Skip write_run_config_snapshot in _run_engine path to prevent double
apply_defaults corrupting the snapshot on each restart
- Resolve ${env.VARNAME} refs in run_defaults.sandbox.env when falling
back for bare .fabro workflows
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This PR fixes a bug where workflow TOML configuration (including
`[pull_request]` settings) was silently dropped when running in detached
mode (`fabro run -d`). The root cause was a three-part failure:
`create.rs` checked the raw CLI argument string for a `.toml` extension
instead of the resolved path, so `run.toml` was never written to the run
directory; `RunEngine` always fell back to `graph.fabro` (a DOT file),
causing `prepare_workflow` to return `run_cfg = None` and lose all
TOML-level configuration; and `pull_request`/`asset_globs` fields in
`RunConfig` had no fallback to `run_defaults` when `run_cfg` was absent.
The fix replaces the naive file-copy approach with a proper
serialization pipeline. Rather than copying the raw TOML (which would
contain a `graph` field pointing to a nonexistent file in the run
directory), `create.rs` now calls `write_run_config_snapshot`, which
serializes the already-merged `WorkflowRunConfig` and rewrites the
`graph` field to `"graph.fabro"` — the canonical cached name. This makes
the run directory fully self-contained with all defaults merged,
environment variables resolved, and the graph path correct. `RunEngine`
in `main.rs` now unconditionally points at `run.toml`; a new
`resolve_workflow_source` helper handles the `.toml` path by loading the
config and resolving the graph path, with a backward-compatible fallback
to `graph.fabro` for older detached runs created before this change.
As defense-in-depth, fallbacks to `run_defaults` are added throughout
`run.rs` for `pull_request`, `asset_globs`, `devcontainer`, and
`sandbox.env` — ensuring bare `.fabro` files passed directly still pick
up project-level defaults. Two new unit tests verify the serialization
round-trip (confirming `graph` is rewritten and `pull_request` config is
preserved) and the missing-`run.toml` fallback behavior.
### Fabro Details
<details>
<summary>Ran 9 stages in 26m 29s for $9.17</summary>
| Stage | Duration | Cost | Retries |
|---|---|---|---|
| start | 0s | – | 0 |
| toolchain | 0s | – | 0 |
| preflight_compile | 1m 14s | – | 0 |
| preflight_lint | 13s | – | 0 |
| implement | 4m 54s | $0.71 | 0 |
| simplify_opus | 8m 41s | $1.77 | 0 |
| simplify_gpt | 10m 41s | $6.69 | 0 |
| verify | 18s | – | 0 |
| fmt | 1s | – | 0 |
| **Total** | **26m 29s** | **$9.17** | **0** |
</details>
<details>
<summary>Ran <code>ImplementPlan.fabro</code> (12 nodes and 15
edges)</summary>
```dot
digraph ImplementPlan {
graph [
goal="Implement and simplify",
model_stylesheet="
* { model: claude-opus-4-6; }
"
]
rankdir=LR
start [shape=Mdiamond, label="Start"]
exit [shape=Msquare, label="Exit"]
toolchain [label="Toolchain", shape=parallelogram, script="command -v cargo >/dev/null || { curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y && sudo ln -sf $HOME/.cargo/bin/* /usr/local/bin/; }; cargo --version 2>&1", max_retries=0]
preflight_compile [label="Preflight Compile", shape=parallelogram, script="cargo check -q --workspace 2>&1", max_retries=0]
preflight_lint [label="Preflight Lint", shape=parallelogram, script="cargo clippy -q --workspace -- -D warnings 2>&1", max_retries=0]
fix_lints [label="Fix Lints", prompt="The preflight lint step failed. Read the build output from context and fix all clippy lint warnings.", max_visits=3]
implement [label="Implement", prompt="Read the plan file referenced in the goal and implement every step. Make all the code changes described in the plan. Use red/green TDD."]
simplify_opus [label="Simplify (Opus)", prompt="@prompts/simplify.md"]
simplify_gpt [label="Simplify (GPT-54)", prompt="@prompts/simplify.md", model="gpt-54"]
verify [label="Verify", shape=parallelogram, script="cargo clippy -q --workspace -- -D warnings 2>&1 && cargo nextest run --cargo-quiet --workspace --status-level fail 2>&1", goal_gate=true, retry_target="fixup"]
fixup [label="Fixup", prompt="The verify step failed. Read the build output from context and fix all clippy lint warnings and test failures.", max_visits=3]
fmt [label="Format", shape=parallelogram, script="cargo fmt --all 2>&1", max_retries=0]
start -> toolchain
toolchain -> preflight_compile [condition="outcome=success"]
toolchain -> exit
preflight_compile -> preflight_lint [condition="outcome=success"]
preflight_compile -> exit
preflight_lint -> implement [condition="outcome=success"]
preflight_lint -> fix_lints
fix_lints -> preflight_lint
implement -> simplify_opus -> simplify_gpt -> verify
verify -> fmt [condition="outcome=success"]
verify -> fixup
fixup -> verify
fmt -> exit
}
```
</details>
⚒️ Generated with [Fabro](https://fabro.sh)
---------
Co-authored-by: Fabro <noreply@fabro.sh>
Co-authored-by: Bryan Helmkamp <bryan@brynary.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
