## Summary
Fabro-created Daytona sandboxes now carry the same managed-resource
labels Docker containers already use: `sh.fabro.managed=true` and
`sh.fabro.run_id=<run-id>` when a run id is available.
This moves the Docker label constants into a shared sandbox helper,
keeps Docker behavior unchanged, and applies the helper when Daytona
create params are built. User-provided Daytona labels are preserved, but
Fabro's reserved keys are authoritative on collisions. Daytona snapshot
behavior is unchanged because the snapshot API does not expose labels.
## Testing
- `cargo test -p fabro-sandbox managed_labels --no-default-features
--features docker,daytona`
- `cargo test -p fabro-sandbox
docker::tests::real_run_container_gets_name_and_labels
--no-default-features --features docker`
- `cargo test -p fabro-sandbox daytona::tests::base_params
--no-default-features --features daytona`
- `cargo test -p fabro-sandbox daytona_managed_labels_live_smoke
--no-default-features --features daytona`
- `cargo test -p fabro-sandbox --no-default-features --features
docker,daytona`
- `cargo +nightly-2026-04-14 fmt --check --all`
- `cargo +nightly-2026-04-14 clippy --workspace --all-targets -- -D
warnings`
- `cargo +nightly-2026-04-14 clippy -p fabro-sandbox --all-targets
--no-default-features --features docker,daytona -- -D warnings`
The live Daytona smoke test remains ignored; it compiles under the
Daytona feature but was not run against live credentials.
## Post-Deploy Monitoring & Validation
- Log queries/search terms: `Failed to create Daytona sandbox`,
`Daytona`, `labels`, `sh.fabro.managed`, `sh.fabro.run_id`, and sandbox
initialization errors for `provider=daytona`.
- Metrics or dashboards: Daytona sandbox creation success/error rate,
Fabro run initialization failures for Daytona runs, and Daytona resource
inventory filtered by `sh.fabro.managed=true`.
- Expected healthy signals: new Fabro-created Daytona sandboxes include
`sh.fabro.managed=true`, run-owned sandboxes include the matching
`sh.fabro.run_id`, user labels remain visible, and Daytona sandbox
creation failure rates stay at baseline.
- Failure signals and rollback trigger: any sustained increase in
Daytona sandbox creation failures, API validation errors around labels,
or missing managed labels on newly created sandboxes. Roll back this PR
or hotfix the label merge to omit Daytona labels if Daytona rejects the
keys in production.
- Validation window and owner: release owner watches the first 24 hours
after deploy, with an immediate manual Daytona dashboard/API spot-check
after the first managed Daytona run.
---
[](https://github.com/EveryInc/compound-engineering-plugin)
🤖 Generated with GPT-5 (context unknown, reasoning enabled) via
[Codex](https://openai.com/codex)
## Summary
Daytona's default snapshot runs as the `daytona` user (uid 1001), which
lacks write permission on `/`. With `run.clone.enabled = true`, sandbox
init failed at `fs.create_folder("/repos", ...)` with HTTP 400, before
the first workflow stage could run:
```
sandbox.git.failed error="Failed to create Daytona repos root" causes=["HTTP 400"]
run.failed
```
Root cause: the Daytona provider was using Docker's root-level `/repos`
layout. Docker works because its containers run as root; Daytona's
default sandbox user does not.
**Fix:** move `REPOS_ROOT` for Daytona to `/home/daytona/repos`,
alongside the existing `/home/daytona/workspace`. The path is writable
by the default sandbox user, the symlink layout is unchanged
(`/home/daytona/workspace/<repo>` →
`/home/daytona/repos/<owner>/<repo>`),
and Docker keeps its existing `/repos` path.
**Bonus — better error diagnostics.** A new `wrap_fs_error(operation,
path, error)` helper in the Daytona provider:
- includes the attempted path in the message (was just "Failed to create
Daytona repos root" with no indication of which path);
- classifies HTTP 400 as a likely permission issue and points at
snapshot configuration;
- classifies HTTP 401/403 as an API key permissions issue;
- preserves the underlying `DaytonaError` in the source chain
(per `docs/internal/error-handling-strategy.md` — verified by walking
`Error::source()` in the regression test).
So if this class of failure recurs (custom snapshot, future path
changes, ...) the user gets:
> Failed to create Daytona repos root '/home/daytona/repos' failed
> (HTTP 400). This usually means the sandbox user lacks write permission
> on the parent directory. If you're using a custom Daytona snapshot,
> ensure the sandbox user can write to '/home/daytona/repos', or use a
> path under the user's home directory (e.g. /home/daytona/...).
instead of:
> Failed to create Daytona repos root
> HTTP 400
## Test plan
- [x] `cargo build --workspace`
- [x] `cargo nextest run -p fabro-sandbox --features daytona` — 142/142
pass
- [x] `cargo nextest run -p fabro-types -p fabro-workflow` — 1365/1365
pass
- [x] New unit test `wrap_fs_error_classifies_http_400_and_403` —
asserts
top-level message contains path + hint AND walks the source chain
to prove `DaytonaError::Api { status_code: 400, .. }` is preserved
- [x] `cargo +nightly-2026-04-14 fmt --check --all`
- [x] `cargo +nightly-2026-04-14 clippy --workspace --all-targets -- -D
warnings`
- [x] **Live regression**: `daytona_clone_layout_live_smoke` against the
default `daytona-medium` snapshot — failed with `Failed to create
Daytona repos root / HTTP 400` before the change; passes
end-to-end after (provisions sandbox → clones repo → verifies
symlink + HEAD match in 2.5s)
## Related
- Closes#284 (thanks @jessmartin for the report, diagnosis, and
proposed fix)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Jess Martin <27258+jessmartin@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
## Summary
Run cancellation now reaches in-flight agent work instead of waiting for
an agent stage to finish or recording cancellation as a failed stage.
The workflow cancellation primitive is now
`tokio_util::sync::CancellationToken`, with child tokens passed through
setup, handlers, manager-loop child runs, sandbox streaming commands,
CLI agent invocations, and API agent sessions.
### Plan Summary
- Promote run cancellation to `CancellationToken` while keeping stall
timeout separate.
- Route CLI agents through cancellable sandbox streaming with optional
timeouts.
- Bridge run cancellation into API sessions and preserve
`Error::Cancelled` propagation.
- Add typed events/projections for CLI cancellation and timeout.
## Cancellation flow
```mermaid
flowchart TB
RunToken[Run CancellationToken]
Executor[Core executor]
Services[RunServices]
Manager[Manager-loop child run]
CLI[Agent CLI backend]
API[Agent API backend]
Sandbox[Sandbox streaming exec]
Session[fabro-agent Session]
RunToken --> Executor
RunToken --> Services
Services -- child_token --> Manager
Services -- child_token --> CLI
CLI -- child_token --> Sandbox
Services --> API
API -- bridge guard --> Session
```
## What changed and why
- `RunOptions`, `RunServices`, core `ExecutorOptions`, CLI/server run
state, and detached-run guards now use `CancellationToken` instead of
`Arc<AtomicBool>`. Dropping services or tokens still does not mean
cancellation; only explicit `.cancel()` does.
- Manager-loop child workflows are given child tokens so parent
cancellation propagates down, while stop/max-cycle cancellation remains
scoped to the child workflow.
- Stall timeout remains intentionally separate as a stall token and
still returns `Error::StallTimeout { node_id }`, not `Error::Cancelled`.
- Agent, prompt, human, fan-in, and parallel handler paths now pass
cancellation tokens through and avoid converting `Error::Cancelled` into
normal failed outcomes.
## Agent backend behavior
CLI-mode agents no longer launch detached `setsid` jobs with temp
stdout/stderr/exit-code polling. They run through
`Sandbox::exec_command_streaming` with a child token; a missing node
timeout passes `None` to preserve the existing unbounded agent runtime,
while explicit node timeouts still apply. Cancelled CLI runs emit
`agent.cli.cancelled`, clean temp files, and return `Error::Cancelled`;
timed-out CLI runs emit `agent.cli.timed_out` and return a handler
timeout error; `agent.cli.completed` remains natural-exit only.
API-mode agents install a per-invocation `SessionCancelBridgeGuard`
after acquiring a fresh or cached session. The guard maps the run token
into the session interrupt reason and session cancel token, and aborts
stale bridge tasks before session replacement or cache reinsertion so
reused sessions are not tied to old run tokens. `Session::initialize`
now returns `Result`, and project-doc, skill, MCP, and environment
discovery paths check cancellation and pass child tokens to sandbox
commands.
## Sandbox and event model
`Sandbox::exec_command_streaming` now accepts `Option<u64>` for timeout.
Production streaming implementations use a pending future for `None`
instead of a giant sleep, while the trait fallback maps `None` to
`u64::MAX` only when delegating to non-streaming `exec_command`.
The run event model now includes typed `agent.cli.cancelled` and
`agent.cli.timed_out` payloads with stdout, stderr, and duration, plus
conversion and projection support. OpenAPI/client regeneration was
unnecessary because the API schema already models run events with a free
event string and arbitrary properties; only Rust event types changed.
## Reviewer notes
Expect signature churn around `Session::initialize`,
`CodergenBackend::run`, `RunOptions.cancel_token`,
`StartServices.cancel_token`, and `Sandbox::exec_command_streaming`. The
main behavioral checks are that user cancellation reaches in-flight
CLI/API work and that timeout/stall paths remain distinct from user
cancellation.
### Fabro Details
<details>
<summary>Ran 9 stages in 117m 40s for $150.32</summary>
| Stage | Duration | Cost | Retries |
|---|---|---|---|
| start | 0s | – | 0 |
| toolchain | 1s | – | 0 |
| preflight_compile | 2m 8s | – | 0 |
| preflight_lint | 2m 13s | – | 0 |
| implement | 77m 12s | $56.78 | 0 |
| simplify_opus | 18m 5s | $5.83 | 0 |
| simplify_gpt | 15m 33s | $87.71 | 0 |
| verify | 1m 48s | – | 0 |
| fmt | 2s | – | 0 |
| **Total** | **117m 40s** | **$150.32** | **0** |
</details>
<details>
<summary>Ran <code>ImplementPlan.fabro</code> (12 nodes and 15
edges)</summary>
```dot
digraph ImplementPlan {
graph [
goal="Implement and simplify",
model_stylesheet="
* { model: claude-opus-4-7; }
"
]
rankdir=LR
start [shape=Mdiamond, label="Start"]
exit [shape=Msquare, label="Exit"]
toolchain [label="Toolchain", shape=parallelogram, script="command -v cargo >/dev/null || { curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y && sudo ln -sf $HOME/.cargo/bin/* /usr/local/bin/; }; cargo --version 2>&1", max_retries=0]
preflight_compile [label="Preflight Compile", shape=parallelogram, script="cargo check -q --workspace 2>&1", max_retries=0]
preflight_lint [label="Preflight Lint", shape=parallelogram, script="cargo +nightly-2026-04-14 clippy -q --workspace --all-targets -- -D warnings 2>&1", max_retries=0]
fix_lints [label="Fix Lints", prompt="The preflight lint step failed. Read the build output from context and fix all clippy lint warnings.", max_visits=3]
implement [label="Implement", prompt="Read the plan file referenced in the goal and implement every step. Make all the code changes described in the plan. Use red/green TDD."]
simplify_opus [label="Simplify (Opus)", prompt="@prompts/simplify.md"]
simplify_gpt [label="Simplify (GPT-55)", prompt="@prompts/simplify.md", model="gpt-55"]
verify [label="Verify", shape=parallelogram, script="cargo +nightly-2026-04-14 clippy -q --workspace --all-targets -- -D warnings 2>&1 && cargo nextest run --cargo-quiet --workspace --status-level fail 2>&1 && cargo dev docs refresh 2>&1 && cargo dev docs check 2>&1", goal_gate=true, retry_target="fixup"]
fixup [label="Fixup", prompt="The verify step failed. Read the build output from context and fix all clippy lint warnings, test failures, and generated docs errors.", max_visits=3]
fmt [label="Format", shape=parallelogram, script="cargo +nightly-2026-04-14 fmt --all 2>&1", max_retries=0]
start -> toolchain
toolchain -> preflight_compile [condition="outcome=succeeded"]
toolchain -> exit
preflight_compile -> preflight_lint [condition="outcome=succeeded"]
preflight_compile -> exit
preflight_lint -> implement [condition="outcome=succeeded"]
preflight_lint -> fix_lints
fix_lints -> preflight_lint
implement -> simplify_opus -> simplify_gpt -> verify
verify -> fmt [condition="outcome=succeeded"]
verify -> fixup
fixup -> verify
fmt -> exit
}
```
</details>
⚒️ Generated with [Fabro](https://fabro.sh)
---------
Co-authored-by: Fabro <noreply@fabro.sh>
Co-authored-by: Bryan Helmkamp <bryan@brynary.com>
Update test sites to call .to_string() before .contains() since the
sandbox Error enum no longer dereferences to String, add use statements
to satisfy clippy::absolute_paths, and inline the redundant
sandbox_error helpers in fabro-agent to clear needless_pass_by_value.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
