A failed checkpoint cancels the run, so Petri recorded it as cancelled
and the worker overrode its own outcome in memory. Runs that checkpoint
now declare required finalization, and finalize_run rejects with
checkpoint_failed before publishing. The projection and the engine
outcome report a cancelled finish carrying that failure as a workflow
failure with the checkpoint's message, and the in-memory override is
gone. A run whose checkpoint failed is never published.
Retry a host failure that storage rejects with a short backoff, so a
brief storage fault does not leave the run active and holding its
scheduler slot. A failure that never commits still leaves the run's
status alone.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Track Petri main now that required run finalization has merged there,
read workflow execution from the engine state, resume the failed-publication
regression with its admitted workflow, and keep the worker lifecycle helper
usable for rejected appends.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Checkpoints were only enabled for runs with a GitHub source, so local
folder runs, empty Local runs and dry runs stopped committing. `fabro
diff` then failed for them, and their checkpoint, run branch and diff
records disappeared from the event stream.
A run whose workspace is on the host now commits checkpoints there
again, without pushing, as on main. Docker and Daytona runs with no
GitHub source still record execution checkpoints without Git commits,
so a sandbox image without `git` cannot fail the run.
The scenario tests for crash recovery go back to asserting commits. A
workspace deleted while the run is down now fails the resumed run,
since the server keeps no copy to restore it from.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The worker minted one read token at launch and a fresh push token for
every checkpoint. The read token expired an hour into a run, so a
workspace acquired later fetched with a dead credential. Minting per
push put every push in GitHub's token-replication window, where a token
minted moments earlier is rejected with 404 "Repository not found".
The worker now keeps two InstallationTokenSource caches for the run, a
read-only one for fetches and a contents: write one for pushes. Each
fetch and push resolves through its source, which reuses one token until
it nears expiry and then mints the next. Petri's RunSource asks a
SourceCredentials provider on every fetch instead of holding a fixed
credential.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Share one credential header helper between the in-sandbox fetch and
the run branch push
- Keep only the target branch, goal, and model on the publisher instead
of a full run spec copy
- Load the worker's LLM catalog once, and mint the read token only when
the run checks something out
- Pass the source explicitly to checkpoint fetch helpers, dropping
unreachable branches, and reuse has_object in has_commit
- Move the run patch into the publication instead of cloning it, and
build it only when a publisher exists
- Add test fixture helpers for file sources and recording publishers
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Pushing the run branch and opening the pull request now happen in the
run's worker, in Fabro's run_finished hook, after the last stage and
before the run's terminal record, as the legacy publish step did. A
failed push or pull request fails the run with publish_failed instead of
leaving a warning on a run that already succeeded.
fabro-petri gains a RunPublisher the hooks call for a successful run with
its run branch, final commit, snapshot repository and patch; the worker's
GitHub publisher pushes from the snapshot repository with a push token it
mints at that moment, opens the pull request its settings ask for, and
records it. The worker resolves the server's GitHub credentials itself
for both the read-only checkout token and the push token, so the server
no longer hands it a clone credential or publishes after the run.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Since the Petri cutover, a run with a GitHub target started in an empty
workspace, nothing pushed its run branch to GitHub, and nothing asked for
the automatic pull request when it succeeded.
Fabro's hooks now check a fresh run's GitHub target out inside the
sandbox when Petri hands them the scope, before the first stage: the
workspace fetches the selected commit, tag or branch at the run's clone
depth, with a read-only token the server resolves at each worker launch.
The worker scrubs the token from its environment at startup and presents
it only to the fetch, so it never lands in the repository or its remote.
The files belong to the sandbox user, so git accepts them.
The same checkout seeds the workspace's snapshot repository with the
starting commit. Checkpoint bundles from a shallow clone then import, a
stage's own commits never make a bundle carry the source's history, a
restore into a fresh sandbox fetches the base again and applies the run's
commits, and a fork carries the base with its checkpoints.
When a successful GitHub-target run ends, the server pushes its final
commit from the snapshot repository to fabro/run/<id> with its own write
credentials, then, when the run changed files and asks for one, records
the pull request request for the existing creation supervisor. A failed
push or request is a warning notice on the run. The manual pull request
endpoint shares the request step.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- The store checks that captured bytes match their digest in every build,
hashing on a blocking thread, and the upload handler relies on that
check instead of hashing a second time.
- Capture bytes travel as `Bytes` from the hooks through the client and
the store, so uploads and retries share one buffer.
- The artifact writer takes the run ID from the hooks, so objects are
stored under the run their records name.
- Concurrent captures of the same file and content wait on one another,
so the file is uploaded and recorded once.
- When a record append fails, the hooks re-read the run's captures and
treat a record that did land as done, so a lost response does not
record the capture twice.
- An upload that finishes after its run was deleted removes itself,
instead of leaving an object nothing references.
- Listing a run's stage artifacts skips everything under `captures/`, so
an unexpected object there cannot fail the listing or the ZIP.
- The capture record derives its `digest` key from its source instead of
storing it twice, still writing and checking it on the wire.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- The artifact writer takes the digest the hooks already computed, so
captured bytes are hashed once on each side, and the dead integrity
error goes away.
- The writer is a required part of HooksSpec, not an optional field on
RunRequest, so a run with capture globs always has a writer and the
no-writer error goes away.
- ArtifactStore routes put/get and the capture methods through shared
put_at/get_at helpers.
- The upload handler parses the digest with parse_blob_hash_path before
any store reads, and builds its size-limit message from the constant.
- The default local artifact root comes from one helper used by both
config resolution and the storage-dir override.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Load the Daytona key for fork and prune through one AppState method
instead of two copied vault reads, and pass the sandbox configuration
into runtime_spec rather than building it and overwriting it. The
worker reuses the CLI's process_env_var lookup.
Share one Docker availability check and the backend-requirement
variable through fabro-test, drop the built-in plugin path and pin
constants nothing reads any more, and let enabled_plugins() exclude the
bundled kinds itself. Refresh the comments and the spawn_env test that
still described built-in plugins.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Register lazy Host, Docker, and Daytona factories for Petri execution,
fork, and prune. Share server provider configuration, preserve lease
fingerprints, and source Daytona credentials from the vault.
Remove built-in plugin setup and skip gates; add a release-mode worker
and prune regression to catch the failure that blocked nightly builds.
Co-Authored-By: Codex <noreply@openai.com>
RecoveryRequest::for_run and HooksSpec::for_run derived the author, the
identity source, the checkpoint settings and host_workspaces from the
run namespace with the same expressions. RunGitSettings, in checkpoint.rs
beside RunWorkspaces, is that derivation once; both specs carry it.
recovery::plan nested the "recorded, else found and reconciled" lookup
two loops deep and tracked a found flag over a tuple list. A Planner
holds the records, the workspace lookup and the recorded checkpoints,
and its methods read in order: targets, snapshot_of, reconcile_record,
newest. Candidate names the (key, sha) pair, and the Target struct that
duplicated its key's execution is gone; last_finish yields the
CheckpointKey itself, whose Display the failure reason now uses.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Petri's retention decides whether a released workspace is kept or removed.
Fabro's lifecycle settings decide whether a sandbox keeps running after
the run and whether a delete may remove it; none asks for removal at the
run's end, and the sandbox tab, `fabro cp`, the run's delete and the
sandbox scenarios read the container after the run. So the mapping is
`Retention::Always` for every setting, named once as `engine::RETENTION`
with the reasoning, instead of a per-setting function that released a
finished sandbox.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The commit that creates a workspace's run branch records `run.branch`
(the base commit, or the first checkpoint in a workspace with no history)
and `git.identity`. Every checkpoint record after the first carries the
stage's diff from its parent commit, with the patch as a text blob. After
the checkpoint record, the transition hook lists the stage's workspace
through the scope's environment, on the host and in a sandbox alike, and
collects every file under `[run.artifacts] include` into the blob table as
an `artifact.collected` record, skipping a file already collected under
the same path and digest. At the run's end the hooks diff the branch's
last checkpoint against its base in the snapshot repository and record
`run.diff`.
`engine::retention` maps the environment's lifecycle settings onto
Petri's workspace retention instead of always keeping every workspace:
`preserve`, `stop_on_terminal = false` and the local provider keep them,
anything else keeps a failed scope's only. The hooks docs no longer name a
redundant link target, so rustdoc passes with warnings denied.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Petri judges a workflow at admission, so Fabro's lint rules go:
`fabro-validate` (its 35 rules and the `LintRule` trait) is deleted, and
with it `fabro-acp` (only a rule and two legacy executor tests used it),
the model-resolution transform, the legacy `create`, `compile_create_run`
and `materialize_create_run` stages, and `fabro-graphviz`'s `condition`
and `fidelity` modules. `Diagnostic`, `RelatedDiagnostic` and `Severity`
move to `fabro_types::diagnostic`, the one shape every diagnostic takes.
Validation is now the same question the create handler asks. A new
server module, `petri_check`, builds Petri's check request from a
workflow bundle and the run's settings (every workflow of the bundle at
its bundle-relative path, the inputs, the run variables, the launch),
runs the check, and maps the diagnostics; Fabro adds one rule of its
own, `fabro.model.no_ready_provider`, refusing a model node when no
provider is ready. Admission, the validate and preflight endpoints and
the offline `fabro validate` all go through it:
- `validate_prepared_manifest` runs Fabro's structural pass (parse and
transform, whose diagnostics stay) and then Petri's check, on the
blocking pool from the handlers;
- the offline `validate_manifest` checks with no model client and with
an unbound input as a warning (`CheckRequest.unbound_is_warning`), so
a workflow validates before its inputs exist; a collected workflow
before upload checks with unbound inputs as errors, as before;
- preflight resolves each LLM node's selector against the ready
providers and the catalog for its probe, as the deleted transform did,
and no longer probes a model Petri refused;
- the graph render endpoint needs only the structural pass;
- a run manifest now carries its `[run.goal] file`, which Petri reads
from the bundle as it does for a version.
The transforms keep the authored model selector (`sonnet` stays
`sonnet`): Petri pins the catalog model in the admitted graph, not in
the graph Fabro displays or in the settings snapshot. Tests assert that,
and the CLI's validate, preflight and graph snapshots carry Petri's
diagnostics (`attractor.no_start`, `attractor.undeclared_node`,
`attractor.bad_on_failure`, ...) in place of the lint rules' text.
Known gaps, Petri's side: a `workflow.toml` whose `[run.environment]`
names an environment the server catalog defines but the file does not
is refused (`unsupported.workflow_toml.run.environment`), as admission
already refused it; an unbound input inside an included template
partial is a render error rather than the unbound-input warning.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Delete `Engine`, `RunEngine`, `[workflow] engine`, `[server.execution]
engine`, `FABRO_SERVER_ENGINE` and `fabro server start --engine`. The run
spec records what Petri admitted as `admission: PetriAdmission`; the
create handler always admits through `Runtime::check`; `execute_run`
always launches the Petri worker (or executes in process under the test
override); the CLI runner takes only the Petri worker path, and its
legacy control arm, artifact uploader, signal pause handlers and
credential helpers go with it. The CLI's `attach` and `events` read the
run stream only.
Two gaps this surfaced are closed here: the check adapter binds the
server's run variables as Petri compile variables (`{{ vars.* }}` in a
prompt no longer fails admission), and deleting a run removes its Petri
records, lease, platform records, projection and stream.
Tests: the config engine tests are replaced (an engine key is unknown),
the API round-trip test covers `PetriAdmission`, the server and CLI
Petri scenarios drop their engine settings, and the API tests that read
legacy event names now read the run stream or the session events. The
remaining red tests are fixtures and scenarios of the legacy executor
and the legacy event store (`fabro-store` `slate` and `run_state`,
`fabro-types` legacy `run.created` JSON, the server's handler-registry
scenarios, the CLI dry-run snapshots), which the next steps of the F4.3
series delete or port.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A Petri run on Docker or Daytona keeps its workspace inside the scope's
sandbox. Fabro's hooks now take the environment Petri hands them at
`scope_acquired`, run `git` inside the scope through it (the path Petri's
own sandbox-placed hooks take), and commit each stage on the run branch
with the same message and trailers as the host path. The commit leaves
the sandbox as a Git bundle, created against the newest ancestor the
snapshot repository already holds, split into 8 MiB parts (the plugin
transport reads one file up to 16 MiB), read out through the
environment's file transfer, fetched into the bare snapshot repository
on the host and named there under the checkpoint's ref. The repository
holds every checkpoint whatever the provider, and the platform records
name the same commits. The host path is unchanged; both sites share one
runner and the same commands.
Recovery is split: `recovery::plan` decides, over the records and the
snapshot repository alone, what every live workspace must sit on and
reconciles a lost record; `recover` applies it to host workspaces on the
server, as before, and reports a sandbox workspace's target as deferred.
The worker's hooks read the same plan at the scope's first acquisition
after a resume and bring the sandbox workspace to it before any attempt
runs there: verified or reset in a retained sandbox that still holds the
commit, else restored from a bundle of the checkpoint written into the
sandbox. Petri replaces a lease's lost sandbox on Fabro's request
(`LostSandbox::Replace`), so a removed container comes back fresh and
restored.
The checkpoint records are written for every provider now. A Docker
variant of the in-process hooks test moves a 20 MiB file through the
split transfer; the same test runs on Daytona when live credentials are
present. Three CLI scenarios run on a Docker environment: every stage's
checkpoint published from the container, a retained container whose
workspace drifted reset on restart, and a removed container replaced and
restored from the snapshot.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A Petri run answered only cancel and answers; pause, unpause and steer
were ignored with a warning. `fabro_petri::controls::RunControls` now
wraps Petri's `ControlService` per run: `engine::run` installs its pause
gate over the run's hooks, observes the run through it and wires it to
the coordinator, on a start and a resume alike, so a run paused when its
worker died resumes paused.
The worker's control channel takes a `WorkerControls` enum: the legacy
hub and pause flag, or the Petri run's controls. On Petri, `run.pause`
holds admission, `run.unpause` releases it once the record is durable,
and `run.steer` goes to the one live agent stage (Fabro's steer names no
stage); with none or several it is refused with a `run.notice` record.
The paused state is mirrored to Fabro's lifecycle as `run.paused` and
`run.unpaused` events, so the server's live status and the projection
follow Petri's own records.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
In-process tests over the memory store: every finish is committed on the
run branch with its identity trailers and recorded with its commit, the
run-end hooks reach Petri's local service through Fabro's wrapper, a
stage that fails on its own terms is committed and its failure route runs
on the committed files, a failed checkpoint records `checkpoint_failed`
with no route taken and a restart reports the run failed, and a
`[[run.hooks]]` hook blocks an agent's tool call through the forwarded
service, with the model told why.
Real-binary scenarios crash the server and its worker with SIGKILL: after
a durable finish the stage's commit is not repeated and the interrupted
stage reruns on its snapshot; a crash held before the commit reruns the
stage once; a crash held after the commit but before its record
reconciles the record from the snapshot repository; a deleted workspace
is restored; a failure route sees the same committed files after a
crash; a failed checkpoint fails the run and a restart leaves it failed.
Recovery selects the executions `inspect_run` reports incomplete, and a
run whose coordinator log is still empty is left to the worker's resume.
The worker's platform record endpoints get an API test and the generated
TypeScript client.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Fabro's hooks on a Petri run wrap the hooks the runtime installed for
`[[run.hooks]]` and forward every point. In `prepare_result`, before the
finish is recorded, they commit the stage's files on the run branch of
its host workspace with Fabro's author identity and the run, execution,
firing and attempt as trailers, and publish the commit to a snapshot
repository beside the run's workspaces under a ref per checkpoint. A
stage that failed on its own terms is committed like a successful one; a
commit that fails is fatal: the outcome becomes a `checkpoint_failed`
failure, the run is cancelled through the coordinator handle, and the
transition refuses the firing's routes. In `transition` they write the
platform checkpoint record, keyed on the Petri position and the
checkpoint's operation identity, and a failed write is a recorded
problem.
On restart the server runs the recovery protocol before it relaunches a
worker: a run with a failed checkpoint is reported failed; otherwise
every live execution's last durable finish names the snapshot its
workspace is verified against, reset to, or restored from, with a lost
record reconciled from the snapshot repository, and a finish with no
snapshot fails the run rather than resume it on stale files.
The worker reaches the platform records over two new worker-scoped
endpoints; the server reaches the table directly. A test gate directory
lets the CLI scenarios hold a checkpoint at a named point.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>