fabro/lib/crates
fabro-sh-0530[bot] 7cec7825d9
Cancel in-flight agent stages with CancellationToken (#211)
## Summary
Run cancellation now reaches in-flight agent work instead of waiting for
an agent stage to finish or recording cancellation as a failed stage.
The workflow cancellation primitive is now
`tokio_util::sync::CancellationToken`, with child tokens passed through
setup, handlers, manager-loop child runs, sandbox streaming commands,
CLI agent invocations, and API agent sessions.

### Plan Summary
- Promote run cancellation to `CancellationToken` while keeping stall
timeout separate.
- Route CLI agents through cancellable sandbox streaming with optional
timeouts.
- Bridge run cancellation into API sessions and preserve
`Error::Cancelled` propagation.
- Add typed events/projections for CLI cancellation and timeout.

## Cancellation flow
```mermaid
flowchart TB
  RunToken[Run CancellationToken]
  Executor[Core executor]
  Services[RunServices]
  Manager[Manager-loop child run]
  CLI[Agent CLI backend]
  API[Agent API backend]
  Sandbox[Sandbox streaming exec]
  Session[fabro-agent Session]

  RunToken --> Executor
  RunToken --> Services
  Services -- child_token --> Manager
  Services -- child_token --> CLI
  CLI -- child_token --> Sandbox
  Services --> API
  API -- bridge guard --> Session
```

## What changed and why
- `RunOptions`, `RunServices`, core `ExecutorOptions`, CLI/server run
state, and detached-run guards now use `CancellationToken` instead of
`Arc<AtomicBool>`. Dropping services or tokens still does not mean
cancellation; only explicit `.cancel()` does.
- Manager-loop child workflows are given child tokens so parent
cancellation propagates down, while stop/max-cycle cancellation remains
scoped to the child workflow.
- Stall timeout remains intentionally separate as a stall token and
still returns `Error::StallTimeout { node_id }`, not `Error::Cancelled`.
- Agent, prompt, human, fan-in, and parallel handler paths now pass
cancellation tokens through and avoid converting `Error::Cancelled` into
normal failed outcomes.

## Agent backend behavior
CLI-mode agents no longer launch detached `setsid` jobs with temp
stdout/stderr/exit-code polling. They run through
`Sandbox::exec_command_streaming` with a child token; a missing node
timeout passes `None` to preserve the existing unbounded agent runtime,
while explicit node timeouts still apply. Cancelled CLI runs emit
`agent.cli.cancelled`, clean temp files, and return `Error::Cancelled`;
timed-out CLI runs emit `agent.cli.timed_out` and return a handler
timeout error; `agent.cli.completed` remains natural-exit only.

API-mode agents install a per-invocation `SessionCancelBridgeGuard`
after acquiring a fresh or cached session. The guard maps the run token
into the session interrupt reason and session cancel token, and aborts
stale bridge tasks before session replacement or cache reinsertion so
reused sessions are not tied to old run tokens. `Session::initialize`
now returns `Result`, and project-doc, skill, MCP, and environment
discovery paths check cancellation and pass child tokens to sandbox
commands.

## Sandbox and event model
`Sandbox::exec_command_streaming` now accepts `Option<u64>` for timeout.
Production streaming implementations use a pending future for `None`
instead of a giant sleep, while the trait fallback maps `None` to
`u64::MAX` only when delegating to non-streaming `exec_command`.

The run event model now includes typed `agent.cli.cancelled` and
`agent.cli.timed_out` payloads with stdout, stderr, and duration, plus
conversion and projection support. OpenAPI/client regeneration was
unnecessary because the API schema already models run events with a free
event string and arbitrary properties; only Rust event types changed.

## Reviewer notes
Expect signature churn around `Session::initialize`,
`CodergenBackend::run`, `RunOptions.cancel_token`,
`StartServices.cancel_token`, and `Sandbox::exec_command_streaming`. The
main behavioral checks are that user cancellation reaches in-flight
CLI/API work and that timeout/stall paths remain distinct from user
cancellation.

### Fabro Details

<details>
<summary>Ran 9 stages in 117m 40s for $150.32</summary>

| Stage | Duration | Cost | Retries |
|---|---|---|---|
| start | 0s | – | 0 |
| toolchain | 1s | – | 0 |
| preflight_compile | 2m 8s | – | 0 |
| preflight_lint | 2m 13s | – | 0 |
| implement | 77m 12s | $56.78 | 0 |
| simplify_opus | 18m 5s | $5.83 | 0 |
| simplify_gpt | 15m 33s | $87.71 | 0 |
| verify | 1m 48s | – | 0 |
| fmt | 2s | – | 0 |
| **Total** | **117m 40s** | **$150.32** | **0** |

</details>

<details>
<summary>Ran <code>ImplementPlan.fabro</code> (12 nodes and 15
edges)</summary>

```dot
digraph ImplementPlan {
    graph [
        goal="Implement and simplify",
        model_stylesheet="
            * { model: claude-opus-4-7; }
        "
    ]
    rankdir=LR

    start [shape=Mdiamond, label="Start"]
    exit  [shape=Msquare, label="Exit"]

    toolchain         [label="Toolchain", shape=parallelogram, script="command -v cargo >/dev/null || { curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y && sudo ln -sf $HOME/.cargo/bin/* /usr/local/bin/; }; cargo --version 2>&1", max_retries=0]
    preflight_compile [label="Preflight Compile", shape=parallelogram, script="cargo check -q --workspace 2>&1", max_retries=0]
    preflight_lint    [label="Preflight Lint", shape=parallelogram, script="cargo +nightly-2026-04-14 clippy -q --workspace --all-targets -- -D warnings 2>&1", max_retries=0]
    fix_lints         [label="Fix Lints", prompt="The preflight lint step failed. Read the build output from context and fix all clippy lint warnings.", max_visits=3]
    implement         [label="Implement", prompt="Read the plan file referenced in the goal and implement every step. Make all the code changes described in the plan. Use red/green TDD."]
    simplify_opus     [label="Simplify (Opus)", prompt="@prompts/simplify.md"]
    simplify_gpt      [label="Simplify (GPT-55)", prompt="@prompts/simplify.md", model="gpt-55"]
    verify            [label="Verify", shape=parallelogram, script="cargo +nightly-2026-04-14 clippy -q --workspace --all-targets -- -D warnings 2>&1 && cargo nextest run --cargo-quiet --workspace --status-level fail 2>&1 && cargo dev docs refresh 2>&1 && cargo dev docs check 2>&1", goal_gate=true, retry_target="fixup"]
    fixup             [label="Fixup", prompt="The verify step failed. Read the build output from context and fix all clippy lint warnings, test failures, and generated docs errors.", max_visits=3]
    fmt               [label="Format", shape=parallelogram, script="cargo +nightly-2026-04-14 fmt --all 2>&1", max_retries=0]

    start -> toolchain
    toolchain -> preflight_compile [condition="outcome=succeeded"]
    toolchain -> exit
    preflight_compile -> preflight_lint [condition="outcome=succeeded"]
    preflight_compile -> exit
    preflight_lint -> implement [condition="outcome=succeeded"]
    preflight_lint -> fix_lints
    fix_lints -> preflight_lint
    implement -> simplify_opus -> simplify_gpt -> verify
    verify -> fmt   [condition="outcome=succeeded"]
    verify -> fixup
    fixup -> verify
    fmt -> exit
}

```

</details>

⚒️ Generated with [Fabro](https://fabro.sh)

---------

Co-authored-by: Fabro <noreply@fabro.sh>
Co-authored-by: Bryan Helmkamp <bryan@brynary.com>
2026-05-05 09:54:22 -04:00
..
fabro-agent Cancel in-flight agent stages with CancellationToken (#211) 2026-05-05 09:54:22 -04:00
fabro-api Read billing and stages from RunProjection with live runtimes (#213) 2026-05-05 09:32:33 -04:00
fabro-auth refactor(api): unify secret metadata types 2026-04-29 20:26:13 -04:00
fabro-checkpoint Make git metadata sandbox-native 2026-04-27 21:43:15 -07:00
fabro-cli Cancel in-flight agent stages with CancellationToken (#211) 2026-05-05 09:54:22 -04:00
fabro-client fix(cli): bound server readiness probes 2026-05-02 20:13:18 -04:00
fabro-config fix(error): preserve source chains across workspace 2026-05-01 17:30:20 -04:00
fabro-core Cancel in-flight agent stages with CancellationToken (#211) 2026-05-05 09:54:22 -04:00
fabro-dev fix(docs): use MDX comment syntax for generated fences 2026-04-29 01:27:43 -04:00
fabro-devcontainer refactor(static): centralize env var names 2026-04-24 12:29:51 -04:00
fabro-dump Encode stage visits in run stage URLs (#206) 2026-05-05 08:27:11 -04:00
fabro-github refactor(error): drop String error shims and DisplayContains test traits 2026-05-01 19:35:22 -04:00
fabro-graphviz refactor: simplify Rust review cleanup 2026-05-02 15:41:17 -04:00
fabro-hooks feat(command): distinguish cancelled commands from timeouts 2026-04-30 22:45:37 -04:00
fabro-http refactor(static): centralize env var names 2026-04-24 12:29:51 -04:00
fabro-install feat(install): add sandbox provider step to web install wizard 2026-04-27 14:59:17 -07:00
fabro-interview refactor: simplify auth and actor handling 2026-05-02 11:44:17 -04:00
fabro-llm refactor(server): simplify model availability probes 2026-05-04 11:47:37 -04:00
fabro-macros refactor(dev): simplify generated docs tooling 2026-04-24 18:41:00 -04:00
fabro-mcp fix(error): preserve source chains across workspace 2026-05-01 17:30:20 -04:00
fabro-model refactor(server): simplify model availability probes 2026-05-04 11:47:37 -04:00
fabro-oauth fix(error): preserve remaining error context 2026-05-02 10:51:15 -04:00
fabro-options-metadata refactor(dev): simplify generated docs tooling 2026-04-24 18:41:00 -04:00
fabro-proc refactor(static): centralize env var names 2026-04-24 12:29:51 -04:00
fabro-redact fix(redact): skip name fields to preserve sandbox identifiers 2026-05-03 19:57:55 -04:00
fabro-retro Cancel in-flight agent stages with CancellationToken (#211) 2026-05-05 09:54:22 -04:00
fabro-sandbox Cancel in-flight agent stages with CancellationToken (#211) 2026-05-05 09:54:22 -04:00
fabro-server Cancel in-flight agent stages with CancellationToken (#211) 2026-05-05 09:54:22 -04:00
fabro-slack refactor: simplify auth and actor handling 2026-05-02 11:44:17 -04:00
fabro-spa feat(dev): gitignore embedded spa assets 2026-04-26 21:31:11 -04:00
fabro-static feat(server): validate Daytona API key scopes 2026-05-03 17:09:01 -04:00
fabro-store Cancel in-flight agent stages with CancellationToken (#211) 2026-05-05 09:54:22 -04:00
fabro-telemetry refactor(static): centralize env var names 2026-04-24 12:29:51 -04:00
fabro-template refactor(async): lint std::process::Command across all targets 2026-04-12 13:35:57 -04:00
fabro-test fix(cli): bound server readiness probes 2026-05-02 20:13:18 -04:00
fabro-tracker fix(error): preserve remaining error context 2026-05-02 10:51:15 -04:00
fabro-types Cancel in-flight agent stages with CancellationToken (#211) 2026-05-05 09:54:22 -04:00
fabro-util refactor(error): drop String error shims and DisplayContains test traits 2026-05-01 19:35:22 -04:00
fabro-validate refactor: simplify Rust review cleanup 2026-05-02 15:41:17 -04:00
fabro-vault refactor(api): unify secret metadata types 2026-04-29 20:26:13 -04:00
fabro-workflow Cancel in-flight agent stages with CancellationToken (#211) 2026-05-05 09:54:22 -04:00